I have nextcloud notify push running as a user systemd user service. And after a few hours, it will just legitimately stop and I don’t know why.
Apr 2Apr 29 22:53:03 notify_push[2531704]: [2022-04-29 20:53:03.806642 +00:00] WARN [notify_push::connection] src/connection.rs:54: Invalid authentication message Apr 29 22:55:04 notify_push[2531704]: [2022-04-29 20:55:04.948325 +00:00] WARN [notify_push::connection] src/connection.rs:54: Invalid authentication message Apr 29 22:57:05 notify_push[2531704]: [2022-04-29 20:57:05.792296 +00:00] WARN [notify_push::connection] src/connection.rs:54: Invalid authentication message Apr 29 22:59:06 notify_push[2531704]: [2022-04-29 20:59:06.784266 +00:00] WARN [notify_push::connection] src/connection.rs:54: Invalid authentication message Apr 30 01:10:55 systemd[2530485]: Stopping Push daemon for Nextcloud clients... Apr 30 01:10:56 systemd[2530485]: notify_push.service: Succeeded. Apr 30 01:10:56 systemd[2530485]: Stopped Push daemon for Nextcloud clients.
So until I have time to investigate further into the “why” part, I setup a work around crontab command to start notyfi_push.service every hour. But running
systemctl start –user notify_push.service inside as a cron command won’t work, And even though I tried to pipe that command output to a file for debugging, I forget to redirect the stderr message, rsesulted in many frustrating hours.
finally I stumbled upon this unix.stackexchange thread https://unix.stackexchange.com/questions/474439/using-cron-to-restart-a-systemd-user-service that even come with a snippet of bash script to allow systemctl –user to run inside cron command. So, a simple .sh file later everything seem to be in order
#!/bin/bash
XDG_RUNTIME_DIR=/run/user/$(id -u)
DBUS_SESSION_BUS_ADDRESS=unix:path=${XDG_RUNTIME_DIR}/bus
export DBUS_SESSION_BUS_ADDRESS XDG_RUNTIME_DIR
systemctl start --user notify_push && systemctl status --user notify_push >> ~/notify_push_cron_log 2>&1
Conclusion
Finally, it turns out that systemd automatically stop all my user service after I exit sshd sessions. Enable lingering using loginctl as according to archlinux wiki systemd/user page: https://wiki.archlinux.org/title/systemd/User#Automatic_start-up_of_systemd_user_instances seem to fix the problem