Failed to start rootless container with systemd on Rocky Linux 8.10

[Meikel]

Hi folks,

I’m trying to start a rootless container on Rocky Linux 8.10 with systemd:

systemctl --user start dokuwiki-backup-readonly-systemd.service

and I see on the console:

Job for dokuwiki-backup-readonly-systemd.service failed because the control process exited with error code.
See “systemctl --user status dokuwiki-backup-readonly-systemd.service” and “journalctl --user -xe” for details.

The command

journalctl --user -xe

is empty while

systemctl --user status dokuwiki-backup-readonly-systemd.service -l --no-pager

shows

● dokuwiki-backup-readonly-systemd.service - Dokuwiki Backup Readonly
Loaded: loaded (/home/sysadmin/.config/containers/systemd/dokuwiki-backup-readonly-systemd.container; generated)
Active: failed (Result: exit-code) since Sun 2025-02-16 07:50:30 CET; 4min 27s ago
Process: 2076 ExecStopPost=/usr/bin/podman rm -v -f -i --cidfile=/run/user/1000/dokuwiki-backup-readonly-systemd.cid (code=exited, status=0/SUCCESS)
Process: 2010 ExecStart=/usr/bin/podman run --name=dokuwiki-backup-readonly-systemd --cidfile=/run/user/1000/dokuwiki-backup-readonly-systemd.cid --replace --rm --cgroups=split --sdnotify=conmon -d -v /home/sysadmin/containers/storage/dokuwiki-backup-readonly:/storage --publish 8080:8080 --env ALLOW_EMPTY_PASSWORD=yes --label app=dokuwiki-backup-readonly docker.io/dokuwiki/dokuwiki:stable (code=exited, status=127)
Main PID: 2010 (code=exited, status=127)

Feb 16 07:50:29 ts269pro.f11.ivi3.de systemd[1519]: Starting Dokuwiki Backup Readonly…
Feb 16 07:50:30 ts269pro.f11.ivi3.de conmon[2041]: conmon a864e6064ae2d358f8b9 : runtime stderr: runc create failed: unable to start container process: error during container init: error setting cgroup config for procHooks process: openat2 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/dokuwiki-backup-readonly-systemd.service/libpod-payload-a864e6064ae2d358f8b9e9cb3bbbcc455bf7c273fc744a12c74f5e8fd0344637/pids.max: no such file or directory
Feb 16 07:50:30 ts269pro.f11.ivi3.de conmon[2041]: conmon a864e6064ae2d358f8b9 : Failed to create container: exit status 1
Feb 16 07:50:30 ts269pro.f11.ivi3.de dokuwiki-backup-readonly-systemd[2010]: Error: runc: runc create failed: unable to start container process: error during container init: error setting cgroup config for procHooks process: openat2 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/dokuwiki-backup-readonly-systemd.service/libpod-payload-a864e6064ae2d358f8b9e9cb3bbbcc455bf7c273fc744a12c74f5e8fd0344637/pids.max: no such file or directory: OCI runtime attempted to invoke a command that was not found
Feb 16 07:50:30 ts269pro.f11.ivi3.de systemd[1519]: dokuwiki-backup-readonly-systemd.service: Main process exited, code=exited, status=127/n/a
Feb 16 07:50:30 ts269pro.f11.ivi3.de systemd[1519]: dokuwiki-backup-readonly-systemd.service: Killing process 2041 (conmon) with signal SIGKILL.
Feb 16 07:50:30 ts269pro.f11.ivi3.de systemd[1519]: dokuwiki-backup-readonly-systemd.service: Killing process 2060 (podman) with signal SIGKILL.
Feb 16 07:50:30 ts269pro.f11.ivi3.de systemd[1519]: dokuwiki-backup-readonly-systemd.service: Failed with result ‘exit-code’.
Feb 16 07:50:30 ts269pro.f11.ivi3.de systemd[1519]: Failed to start Dokuwiki Backup Readonly.

Following are the steps I took to configure the service.

I configured cgroups v2, and for me it seems to work:

podman info --format {{.Host.CgroupsVersion}}
v2

I created a quadlet for the service:

cat ~/.config/containers/systemd/dokuwiki-backup-readonly-systemd.container
[Unit]
Description=Dokuwiki Backup Readonly

[Container]
Label=app=dokuwiki-backup-readonly
ContainerName=dokuwiki-backup-readonly-systemd
Image=docker.io/dokuwiki/dokuwiki:stable
Volume=%h/containers/storage/dokuwiki-backup-readonly:/storage
Environment=ALLOW_EMPTY_PASSWORD=yes
PublishPort=8080:8080

[Install]
WantedBy=multi-user.target default.target

Then I ran

systemctl --user daemon-reload

and that’s all. Until now I did NOT (!!!) already set

loginctl enable-linger sysadmin

and hope that this is not responsible for the problem. Once my configuration works I will create another user with the same configuration and then “enable-linger” for that user.

Is such a configuration generally possible under Rocky Linux 8.10 or do I have to use Rocky Linux 9? The problem is that my hardware does not support Rocky Linux 9.

What steps need I to take to get the container running via “systemctl --user start …” or how can I further investigate the problem?

Regards,

Meikel

[Meikel]

Hello,

I did some further investigations. I ran

podman info | grep -i -A1 runtime

which shows

ociRuntime:
    name: runc

and in the internet I found considerations to prefer crun over runc. I installed crun by running

sudo dnf install -y crun

and restarted the systemd user session

systemctl --user daemon-reexec
systemctl --user daemon-reload

but it did not help to change the OCI runtime:

podman info | grep -i -A1 runtime

still shows

  ociRuntime:
    name: runc

Meanwhile I had this configuration which did not help:

cat ~/.config/containers/containers.conf

[engine]
default_runtime="crun"
disable_pids_limit = true

I removed the configuration file with

rm -f ~/.config/containers/containers.conf

To force usage of crun I started the container from command line with

podman --runtime=crun run --rm -p 8080:8080 -e ALLOW_EMPTY_PASSWORD=yes -v ~/containers/storage/dokuwiki-backup-readonly:/storage docker.io/dokuwiki/dokuwiki:stable

which gave me

Error: OCI runtime error: crun: the requested cgroup controller pids is not available

To disable “pids” I added --cgroups=disabled as following:

podman --runtime=crun run --cgroups=disabled --rm -p 8080:8080 -e ALLOW_EMPTY_PASSWORD=yes -v ~/containers/storage/dokuwiki-backup-readonly:/storage docker.io/dokuwiki/dokuwiki:stable

It seems to work. Now, from another shell on the same system I can access the Dokuwiki for example with

curl -X GET http://localhost:8080

This works fine. As I’m an end-user of podman and I don’t know how cgroups work, I’m wondering if it causes any security issues or other problems if I use “crun” as OCI Runtime and if the option “–cgroups=disabled” might cause any problems. If this is not the case, how can I add these two options

–runtime=crun
–cgroups=disabled

into my service definition ~/.config/containers/systemd/dokuwiki-backup-readonly-systemd.container ?

Thanks in advance for any help and hints.

Regards,

Meikel