Podman's Pet Manager

Introduction

Today's mantra is that managing servers should be done like cattle, not pets.

This analogy is valid for most bigger companies, where machines should be managed in a way that a relative small team can handle a large amount of workloads. You have your big cluster software, like kubernetes, or plenty of virtual machines with vmware, and use a lot of standardisation so you can manage, monitor and backup them in a structured way.

However, for most small setups, for simple self-hosters or small companies, most of this is overkill. Kubernetes is not simple, and comes with a lot of overhead, sometimes needing more then 5 servers just to run a hello world application. This does not make sense for smaller setups. Also, for some smaller setups, it makes a lot of sense to trade some performance for manageability.

Podman's Pet Manager is designed so I can run my own programs on my own servers in such a way that they are standardized so they are easy to manage, backup etc, while remaining cheap in hardware and overhead.

I am an experienced linux administrator, and that's also the target audience. We want a tool that takes away the day to day management of our software so we spend less time managing it, but not to be hiding away the actual complexity that is needed so we can still understand everything and fix any issue when needed.

In order to remain as simple as possible, a few ground rules have been defined:

• ppm should at all times be transparently clearly documented so a decent Linux admin knows exactly what is going on, making it easy to debug

• ppm projects should run as a regular user, without any root access. All steps executed as root should be fully documented and manually executed by the admin so they know what is going on

• backups are central: every app should be backed up and restored automatically. A full server reinstall should be fully automated and not just reinstall all software, but also restore all data so recovering from for example broken hardware should be extremely easy

• ppm should be distribution agnostic. Any Linux flavor should work, as making all admins agree on the same Linux distribution is impossible...

• simplicity is key. We do not care about 24/7 uptime, it is fine to stop, backup and start again every night, reliability is more important. We also target small setups, so having an embedded sqlite is usually better then a full blown database with the associated costs in hardware.

What is PPM

PPM will manage your programs.

The management of an application is 100% done under a regular user. PPM does not use root rights at all - a seperate ansible script can help with that, so the administrator still fully understands what is going on

PPM will do the following steps:

• It will read a simple config.yml at startup inside the users home directory.

• It will git clone the application definition if this has not been done yet. The git repository location is stored inside the config.yml.

• Backups are handled by a backup plugin loaded from the application definition's appinfo.yml. Every appinfo MUST load exactly one backup plugin (backups_none, backups_programinitiated or backups_scheduled); ppm refuses to run otherwise, so the application author cannot forget about backups. Unless backups_none is loaded, ppm will verify that a data/ directory exists in the home directory and restore it from backup if it does not. The backups_scheduled plugin additionally schedules a systemd user timer to take a daily backup of the data directory.

• It will render template files from the application definition, including systemd unit files so the application can be started. ppm also writes a systemd target named after the application's mainunit; the application's unit files declare WantedBy=<mainunit>.target, and appinfo.yml lists them under enableunits so they are pulled in when the target is activated. ppm start, ppm stop, ppm restart and ppm status all operate on this target.

• A state file will be written in the state directory so other applications on the same server can pick up information about this application. This can be used for example to have a single webserver that proxies to several applications on the same server - as only one application can handle port 80/443.

• ppm upgrade will pull the latest appinfo from git. If something changed it prints the next steps to take (ppm setup and ppm restart). Pass --apply to run those steps automatically. The upgrade_scheduled plugin schedules a systemd user timer that runs ppm upgrade --apply daily so each application keeps itself up to date.

What PPM will not do:

• Install ppm, and create the state directory

• Create the users.

• Enable systemd user lingering. Enable linger for an user will make systemd start the systemd user daemon on boot and keep it running when the user is not logged in. This allows us to start applications at boot time.

• put the configuration file in place.

• Create/initialize the backup repository (currently only restic is supported)

Altough all the steps mentioned here can be done using an ansible playbook that is provided here(TODO).