I was researching how to monitor systemd services using Grafana/Prometheus when I came across this article. I learned systemd monitoring comes pre-loaded with node_exporter and just needs to be enabled. Then I discovered the Ansible role I use already enables it for me.

Neat! No work to do.

Or so I thought. Punching node_systemd_unit_state{instance=""} into Prometheus did not show user services, units that would be shown by systemctl --user.

Turns out node_exporter only collects for root.

systemd monitoring for USER services

I found the community collector systemd_exporter which has this capability, although it isn’t documented anywhere.

It’s a completely different binary than node_exporter, and thus has to be installed and enabled. Follow the install instructions or use the cloudalchemy/ansible-systemd-exporter Ansible role. The role starts another root systemd exporter, which is not what we’re interested in.

After installing the binary, set up yet-one-more systemd service in ~/.config/systemd/user/systemd_exporter.service which runs the same collector at a user level with some changes:

  1. Add --collector.user argument.
  2. Change the metrics port (I use 9559).
  3. Remove the User= and Group= lines from the [Service].

It should look something like this:


# Ansible managed

Description=Prometheus SystemD Exporter

ExecStart=/usr/local/bin/systemd_exporter \
    --collector.user \



For completeness, this is my install-systemd_exporter.yml ansible playbook:

- hosts: all
  become: true
    - cloudalchemy.systemd_exporter
- hosts: all
    - name: Copy user systemd_exporter.service
        src: ../../templates/systemd/systemd_exporter.service
        dest: ~/.config/systemd/user/systemd_exporter.service
    - name: Enable systemd_exporter.service
        daemon-reload: true
        enabled: true
        state: started
        name: systemd_exporter
        scope: user

After running the playbook, or otherwise --user enable --now-ing systemd_exporter, I now have metrics on describing user services.