Systemd is incredibly versatile and most people, including myself, are unaware of its full potential. Despite its usefulness, it is often overlooked due to controversy and the current state of things when it comes to software development. Begin today your journey thought Systemd’s capabilities and discover how to efficiently manage systems with fewer processes than traditionally required.
I still find it a solution desperately looking for a problem to solve. Yes, you can control loads of dependencies, but you could already do that with the init scripts.TheInsane42 @ lemmy.world
Systemd does a lot more than that – it solves tons of painful issues and provides a cohesive ecosystem of tools to manage Linux systems. Here are a few examples:
- Start services in parallel (while init was one after the other) for a faster boot;
- Soft reboots;
- Defer service startup until they are actually needed;
- Replace a bunch of poorly integrated tools such as
- Has facilities to deal with IPv6 advanced use cases like IPv6-PD – no more need to install four packages in order to have IPv6;
- Easy policy-based routing tied to your network configuration without scripts;
- Centralizes logging and provides
journalctlas a unified interface so you can quickly browse them by specific, filters, timestamps and events.
crontabwith something auditable that actually makes sense and is easy to use. Can also be configured to run “x seconds after each boot”, to avoid duplicate processes and much more;
- Monitors the status of your services and restarts them if they crash according to your rules. Can also be used to limit execution time and trigger notifications when stuff fails;
- Monitor files and directories for changes;
- Run programs and scripts when your hardware changes;
- Mount partitions / filesystems;
- Allows you to isolate your processes with sandboxing (like jails) techniques by restricting what resources (CPU time, RAM, networking, filesystems) they can access in a single unit file;
- Setup basic firewalls for your services;
- Setup port forwards / proxies for local and remote sockets. Can also be used to “convert” between TCP and UNIX sockets.
- Listen for connections in sockets and launch a program to handle incoming data when needed. Abuse it with PHP-FPM;
- Wake your system from sleep;
- It can be your boot loader;
- Create full fledged containers both privileged and non-privileged (here and here) while providing tools such as
machinectlto manage them.
Yes, you read it right, Systemd can be used to create full fledged containers very much like LXD/LXC!
Managing Your Network
I’m struggling with networkd hysteresis (…) toggling the interface down and then back up does not (…) clear up the configuration, and setting the interface up does not reconfigure the interface. I have to run reconfigure for that. (…) what would the equivalent of ifdown and ifup be? This kind of feature would be useful for me to test config changes, debug networking issues, disconnect part of the network while I’m making some changes, etc.dr_robot @ lemmy.world
That’s a valid issue and while
systemd-networkd was designed for persistent configurations there are a few options:
systemctl reload systemd-networkdand
networkctl reload. If you change units on the filesystem a
systemctl daemon-reloadis required before the previous commands;
networkctlcommands such as
renew. Read more about them here;
- Temporary / volatile runtime units: manually drop a config under
/run/systemd/network/it will apply until you reboot the machine;
- Transient scope units: those are kind of supersede temporary units as they are managed through a D-Bus interface so 3rd party applications can manage systemd. They don’t seem to work right now for network, but this allows you to change unit options dynamically.
In most cases you can have multiple network setups in
/etc/systemd/network but only bring them online when required.
I hate systemd with a passion, as the refuses to wait for networking when you haven some service specified to be started After networking.TheInsane42 @ lemmy.world
If you apply what is written at “Cut the crap! How do I make sure that my service starts after the network is really online?” it will work. This will ensure that all configured network devices are up and have an IP address assigned before the service is started.
Then I don’t understand why maintainers all keep using network.target.TheInsane42 @ lemmy.world
Most services are able to dynamically accommodate networking changes and act accordingly. It’s rare to have services where we really need addresses and a proper link on startup. For those special cases, as described, you need to enable
systemd-networkd-wait-online.service and include
Wants=network-online.target in your service.
Programs should be designed to detect and react to networking changes, both Apache and Nginx are good examples of software that does that. There some cases such as “stuff” that needs to bind to non-existent IPs at boot (before networking) that can be dealt with
net.ipv6.ip_nonlocal_bind as described here.
It is important to also understand what “network up” really means a few other details: https://systemd.io/NETWORK_ONLINE/
Network Time Made Easy
NTP is a must have and while most systems come with
chrony systemd also has you covered for NTP:
apt purge chrony
systemctl enable systemd-timesyncd.service
/etc/systemd/timesyncd.conf and add your favorite time servers:
NTP=18.104.22.168 22.214.171.124 126.96.36.199 188.8.131.52 184.108.40.206 220.127.116.11
FallbackNTP=0.debian.pool.ntp.org 1.debian.pool.ntp.org 2.debian.pool.ntp.org 3.debian.pool.ntp.org
Now a few useful commands to work with NTP time:
timedatectl set-ntp true # update the time
hwclock --show # show the current time in your hardware clock / UEFI
hwclock --systohc # update the hardware clock with the system time we got from NTP
It was easy wasn’t it?
Fast and Secure DNS Resolution
Traditionally people are used to a bunch of nameserver entries at
/etc/resolv.conf and while it works it doesn’t provide any security (encrypted DNS, query validation etc) nor does it provide efficient caching. Systemd can do it better with
systemd-resolved is a systemd service that provides network name resolution to local applications via a D-Bus interface, the resolve NSS service, and a local DNS stub listener on
127.0.0.53. In order to use it we can:
apt install systemd-resolved
ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf
Add our resolvers to
For a more secure configuration we can, for instance, use DNS over TLS:
DNS=18.104.22.168#1dot1dot1dot1.cloudflare-dns.com 22.214.171.124#1dot1dot1dot1.cloudflare-dns.com 2606:4700:4700::1111#1dot1dot1dot1.cloudflare-dns.com 2606:4700:4700::1001#1dot1dot1dot1.cloudflare-dns.com
Start it with:
systemctl enable systemd-resolved.service
systemctl start systemd-resolved.service
In some cases you might also need to edit your network configuration to make sure your local router isn’t pushing a DNS and NTP servers into your system. Edit
/etc/systemd/network/10-eth0.network and add the following section:
One more thing… Bonus feature!
For those running containers, virtual machines or are simply using a Systemd system as a router / gateway for a local network we can also leverage it as a local DNS server. Simply append
DNSStubListenerExtra=10.0.0.1 to the
[Resolve] section and you’re set. More info here.
Restrict a Service to a Network Interface
I just learned that VPN killswitches are a thing. I would like to [restrict Transmission] in case I forget to launch my VPN client before opening Transmission.CowsLookLikeMaps@sh.itjust.works
In some cases it might be useful to restrict a daemon to a certain network interface (eg. Transmission) and for those you can override the demon’s unit (
systemctl edit transmission-daemon.service) with the following:
RestrictNetworkInterfaces=wg0 # --> your VPN interface
Another option would be to restrict the daemon to only be able to use a specific IP address:
IPAddressAllow=10.0.0.1 # --> your VPN IP here
Save the file and run
systemctl daemon-reload followed by
systemctl restart transmission-daemon.service and it should be applied.
This technique is safer than just telling a program to bind to a specific IP address because even if the program has a vulnerability or a bug systemd will still restrict it.
Non-Blocking PHP Job Queue
A common theme in web applications is to get a command from a client, eg. API POST request and then execute a long and expensive task in the background when possible – essentially a job queue. By leveraging Systemd’s ability to listen for connections we can build a very simple yet reliable job queue for PHP applications. Start with a
socket and a
Description=AppQueue Simple Socket
ListenStream = 127.0.0.1:50987
Accept = yes
WantedBy = sockets.target
Description=AppQueue Simple Service
cli.php and implemente a few lines of code to read all the standard input (via
file_get_contents("php://stdin") and run tasks accordingly.
Then, in your main application, let’s assume
index.php, you can “dispatch a background job” by connecting to the socket and sending data:
$task = "SEND-NEWSLETTER:1011"
$socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
socket_connect($socket, "127.0.0.1", 50987);
socket_write($socket, $task, strlen($task));
Systemd will listen for connections at
127.0.0.1:50987 and once data is received it will launch a new PHP process and pass the data to its standard input. This way your client won’t have to wait for
SEND-NEWSLETTER to finish as it will be running on a different process. And the best part? You can use
journalctl to audit your jobs. 🙂
Systemd was crafted with a novel approach – inspired by Apple’s
launchd – that contradicts traditional Unix philosophy, yet it has emerged as an impressively designed and unified ecosystem of tools and solutions to tackle common tasks.
It is included in most Linux distributions and can perform various functions such as networking configuration, containerization, process management, boot the system, listen for events and sockets, you name it. Now that you know how powerful, remarkably stable and mature it is… just use it!
Once you get into systemd you’ll have a few moments of pure amazement with it. One day you’ll be creating containers to suddenly realize that
journalctl can be used to inspect and affect a container in the same way you’re used to in your host machine. Another equally interesting moment in your journey is when you’ve an ARM system with 256 MB of RAM and you figure out that it just saved you 50 MB of precious RAM for other things.
Perhaps the kernel as systemd-kerneld?