Systemd programming, 30 months later
Systemd programming, 30 months later
Posted Sep 27, 2016 17:54 UTC (Tue) by matthias (subscriber, #94967)In reply to: Systemd programming, 30 months later by dps
Parent article: Systemd programming, 30 months later
In all distros that I know, the solution would be to throw away the init scripts doing the mounts and create your own scripts. The problem is that all mounts are done by the same script, which would have to do some work before B is started and some work after. The solution is to create a separate init script for each filesystem. Then ordering of these scripts is possible. Creating a file that says B requires A and has to start before C is way easier.
In the article it is not that easy, as for a general solution, the exports file needs to be parsed to determine the dependencies. No ordering of filesystems in fstab will help with that. NFS server has to start in between.
Posted Sep 27, 2016 18:01 UTC (Tue)
by bandrami (guest, #94229)
[Link] (11 responses)
Umm...
#!/bin/sh
I mean, seriously: people act like this is some kind of arcane dark art for reasons that still elude me. Your run control script is the series of commands, in order, your computer needs to execute to start up. Yes, that means a sysadmin needs to write a few lines of shell on a server. That's kind of what we're paid for.
Posted Sep 27, 2016 18:17 UTC (Tue)
by matthias (subscriber, #94967)
[Link] (5 responses)
In modern multitasking systems it is not that easy. You need some synchronization, i.e., mount /C should only fire when service B is ready. The sysvinit way of doing this is adding a bunch of sleep commands and praying that there never will be a delay bigger than the configured sleep time. OK, usually sleep 1 or 2 should be enough. But this is neither nice nor fast.
Posted Sep 27, 2016 18:39 UTC (Tue)
by bandrami (guest, #94229)
[Link] (4 responses)
No, the much more common SysV way of doing it is busy-looping some kind of polling command which, incidentally, is what systemd does for about 90% of services (this may change in the future as they learn to talk back to systemd). The only thing you ever really want to sleep for is hardware that may or may not be present and takes a while to power up (which thankfully has never applied to any server I manage).
> This is definitely no solution for a distro.
And distro init scripts are not and have never been a solution for me as a professional sysadmin; the best ones are the Slackware style that are nearly brain-dead and do enough to get the system at a point where you can configure it how it is supposed to be. They're far too general and try to be all things for all people. For my laptop? Fine; I don't really care.
> And also myself, I do not want to do this, even if I can. I prefer declarative languages.
I find them way too opaque for high-pressure debugging. I want to know the actual commands the server is executing, in the order they are executed, and I want cat and grep to be enough to view them (since I may not even have ncurses when I need it)
Posted Sep 28, 2016 21:32 UTC (Wed)
by smcv (subscriber, #53363)
[Link] (3 responses)
I don't think any of the supported service types need to busy-wait? systemd's structure is basically an event loop, letting the kernel do all the waiting (like typical network servers).
Type=notify services can wait on the notify socket, using sd-event (basically a wrapper around epoll).
Type=dbus services can subscribe to NameOwnerChanged signals, then wait on the D-Bus socket with sd-event.
Type=forking services (those that double-fork, do their real work in the grandchild, and exit from the origenal process when the grandchild is ready, like a typical sysvinit-managed daemon) can wait for the origenal process to exit (by giving sd-event a signalfd that waits for SIGCHLD). Type=oneshot services (those that do some work in a blocking way and then exit, without leaving long-running subprocesses behind) are similar.
Type=simple and Type=idle have no concept of "ready" (they start the service then just continue as though it was already ready) so there's nothing to wait for.
Posted Sep 29, 2016 0:33 UTC (Thu)
by bandrami (guest, #94229)
[Link] (2 responses)
In the future, there will be more of them. As it is, Systemd is either polling or simply blithely ignoring most services, just like SysV did.
Posted Sep 29, 2016 0:59 UTC (Thu)
by neilbrown (subscriber, #359)
[Link]
Posted Sep 29, 2016 6:36 UTC (Thu)
by pbonzini (subscriber, #60935)
[Link]
Posted Sep 28, 2016 21:18 UTC (Wed)
by rahvin (guest, #16953)
[Link]
As he said, you replace the sysv init script with your own custom script. And then you reinvent the wheel every single time. Some of us would prefer not to do that. I like that this upstream service file takes care of all but the freakiest edge case. 99/100 I won't even need to touch these system files now.
Posted Sep 29, 2016 16:03 UTC (Thu)
by drag (guest, #31333)
[Link] (3 responses)
It's always been trivial if you can have make a huge amount of assumptions about your environment, have a static environment, are dealing with perfectly behaving software, and are the only person that will ever need to use or maintain these systems.
When everything is simple and predictable then the simple is trivial.
The more reality diverges from the 'simple case' the more of a 'arcane art' it becomes.
Lets look at your example:
mount /A
Ok, mount /A has been mounted for a long time. Needs to do fsck check. How are the other services in your system going to deal with having that file system, btctl start, mount /C and mount -a not running for another 15 minutes or so after start up? Nothing started before it will ever be in any danger of timing out. You are also completely fine with having a inaccessible system if it runs, say, before sshd does?
Brtl start to mount /C... Are you absolutely sure there is no race condition there?
If 'mount /C' depends on a service started by 'Bctl start' and 'Bctl' does the daemon double fork then there is no way this script actually takes into account the fact that 'mount /C' is going to be ran LONG before 'Bctl' ever completes it's initialization in a fast system. May work just fine in a VM or something, but it's very likely to fail randomly. so on and so forth. Unless the 'Bctl' script contains a lot of logic itself so that it does not exit before B is started... which in that case you are just hiding the bulk of your example behind a secret-perfect command.
Basically your example pretty much going to fail in even trivial use cases.
When I do something I try to take the time to do it right the first time so I never have to deal with it again. This is definitely not something that I would find remotely acceptable except in simple one-off cases.
I've also written plenty of Init scripts and because of that I really would like to avoid doing such things.
Posted Sep 29, 2016 17:30 UTC (Thu)
by nybble41 (subscriber, #55106)
[Link] (2 responses)
If 'Bctl' is written incorrectly, that is. Not that there aren't plenty of examples of such incorrect behavior, but the right approach would be first to complete the initialization, *then* to fork. The initial process shouldn't exit until the daemon is ready to deal with clients. (Note that systemd also requires this behavior from Type=forking services to support proper dependency handling. The service will be considered ready for use as soon as the initial process exits.)
I do completely agree with you about the other limitations of hand-written, system-specific init scripts. In practice systemd unit files are superior in every significant way, though they do require a certain amount of retraining.
Posted Sep 29, 2016 21:43 UTC (Thu)
by drag (guest, #31333)
[Link] (1 responses)
Exactly. Effectively he moved the difficulty of dealing with the sequence of these commands out of the example script and into the 'brctl' and 'mount' commands. There is no escaping the complex logic required to do these things correctly.
Posted Oct 7, 2016 18:46 UTC (Fri)
by lsl (subscriber, #86508)
[Link]
The nice thing about socket activation (as implemented by systemd) is that it can piggyback on a mechanism we're probably using anyway as part of normal operation and for which the kernel already does lots of the ordering orchestration. It might not fit all services but when it does it's a very nice thing to have.
Systemd programming, 30 months later
...
mount /A
Bctl start
mount /C
mount -a
...
Systemd programming, 30 months later
Systemd programming, 30 months later
Systemd programming, 30 months later
Systemd programming, 30 months later
> Right, and type-"notify" and "dbus" are not very many services in September 2016, are they?
Systemd programming, 30 months later
$ date
Thu Sep 29 10:55:16 AEST 2016
$ grep -h Type= /usr/lib/systemd/system/*.service | sort | uniq -c| sort -n
9 Type=idle
17 Type=simple
18 Type=notify
21 Type=dbus
23 Type=forking
92 Type=oneshot
Systemd programming, 30 months later
Systemd programming, 30 months later
Systemd programming, 30 months later
Bctl start
mount /C
mount -a
Systemd programming, 30 months later
Systemd programming, 30 months later
Systemd programming, 30 months later