Re: Could s6-svscan ignore non-servicedir folders?

From: Olivier Brunel <jjk_at_jjacky.com>
Date: Wed, 21 Jan 2015 21:47:28 +0100

On 01/21/15 20:13, Laurent Bercot wrote:
> On 21/01/2015 18:24, Olivier Brunel wrote:
>> I'll have to setup some scripts for different init stages, using
>> s6-svscan as stage 2, as you've described elsewhere. But I also want to
>> have a system to start (and stop) services in order. I see this whole
>> idea of order/dependency is something that is being talked about, but
>> currently not supported.
>
> Yes. A dependency manager is something fundamentally different from
> a process supervisor, and I definitely do not want to mix the two.
> They can be tightly coupled, but I'm not going to make the process
> supervisor more complex to accommodate the dependency manager: all the
> necessary hooks are already there.
>
> nosh gets it right. I don't agree with all of nosh's design principles
> - especially running the system manager as process 1, which needlessly
> makes it more complex and increases the height of the supervision tree -
> but I absolutely agree with its concept of system manager changing the
> global state by, among others, sending commands to the supervision tree
> (which nosh calls "service manager"). This is the right way to perform
> dependency management with a supervision infrastructure.
>
> And it's also a real project, not something I can add to s6 in a week.
> It needs a lot of work, and it's in my long-term plans, but not in my
> short-term ones.
>
>
>> Furthermore, I want this system of mine to include other kinds of
>> services, that is one-time process/scripts that needs to be run once (on
>> boot), and that's it. And to make things simpler, I want to have it all
>> work together, mixing longrun services (s6 supervised) and oneshot
>> services when it comes to dependency/order definition.
>>
>> So I'll have servicedir of sorts, for oneshot services. And I'm planning
>> of having one folder, that I tend to call runtime repository, but that
>> would also be the scandir for s6-svscan.
>>
>> Obviously though, those aren't servicedirs in the s6 meaning, they
>> shoudln't be supervised, so I'd like for s6-svscan to check if a folder
>> does in fact have a file run, and if not to simply skip/ignore it.
>
> Sorry, but it's going to be a frank no.
> The scandir is the place where s6-svscan does its stuff. It belongs to
> s6-svscan. Private property. Don't use it for other purposes, and don't
> mess with it, or you're going to get into trouble.

Alright, got it.

The thing is, that you've referred to services & oneshots, whereas I
would refer to longrun services & oneshot services resp., i.e. I see
those as two types of services.

In fact, a reason I want/need one folder with everything (servicedirs
for both oneshot & longrun) is to avoid issues of two services with the
same name or such complications when resolving dependencies/ordering.

To maybe explain/detail a bit more how what I'm planning would work: my
"service manager" is asked to start some services, it pulls dependencies
and ordering from subfolders (of servicedirs) needs/wants/after/before.
It then starts everything in order, waiting on some services to be
started before others can be. Starting a oneshot service would mean
running the start script, and waiting for it; Starting a longrun service
would mean sending the command to its supervise process, and waiting for
the event on its fifodir.

So while the service manager does start services, when it comes to
longrun ones it always delegate to s6, as it should.

(To be more precise, all longrun servicedirs will include a down file
when created (into /run), to make sure they're started as/when needed.
So starting a longrun service actually also includes removing the down
file once the event has been received (or when sending the start
command, I'm not sure yet which is best) to reflect the change of state.)

I understand what you're saying regarding the scandir though. Now I'm
thinking I would have a folder with all servicedirs, oneshots &
longruns, and use a folder .scandir that would only contain symlinks for
all the longrun servicedirs.
That way, the scandir only has the servicedirs that needs to be
supervised, and the service manager still has a single place for all
services.

(I might also say, that I'm planning of having this folder of
servicedirs be created on boot into /run, for all enabled services. So
it is all runtime information.)

Does that sound right to you?


> That's intentional, for several reasons.
>
> * s6-svscan is a very, very vital and very, very low-level piece of
> infrastructure. Which means:
> - The simpler it is, the better. Every line of extra code in it is
> potentially a bug in your process 1. It's not the place to handle corner
> cases or convenience issues. s6-svscan and s6-supervise are the two
> programs where I'm *the most* paranoid about feature creep; everything
> I can take out of them and put somewhere else instead, I will.
> - You should not mix interfaces. Clear separation between what is
> lower-level and what is higher-level is the key to good architecture.
> s6-svscan should not have to know or care about what a one-shot is.
> The facts that one-shots and supervisable services should both be
> handled by the same "system manager" entity is of no concern to the
> supervision tree.
>
> * Don't mix config-time and run-time.
> Your one-shot scripts, or directories, belong to a collection of scripts
> that will be called whenever the global state changes, for instance at
> boot time or shutdown time. They will only be used when the state changes,
> not in normal system operation. As such, they belong in your service
> repository, along with the entirety of your service directories, even
> those that are not active. That's read-only configuration information.
> The scandir, on the other hand, is a snapshot of what is currently live;
> it's run-time information. That's not the same thing, so it doesn't
> belong in the same place. (That is also why I advise making a live copy
> of the active service directories: to separate runtime data from
> offline data.)
>
> * Clarity.
> - In a scandir, it's easy to identify the servicedirs: those are
> the non-hidden subdirectories of the scandir. Period. If you change
> that, it will be more complex to detect what is a servicedir and what
> is a one-shot. "ls" won't tell you at a glance. Remember how much of
> a pain it was, figuring out what exactly happened when entering a SysV
> runlevel, which scripts were one-shots and which ones spawned a daemon ?
> I don't want to make the same mistake. Especially with a supervision
> infrastructure, that has the concept of live data that SysV didn't have.
>
>
>> So, what do you think of this? Would you be willing to have s6-svscan
>> ignore folders not containing a run file?
>
> No. Keep out of the scandir. I swear that your system, and also the
> software you're developing, and its users, will thank you.
>
> Directories are not a scarce resource: you can always make a deeper
> hierarchy to properly categorize what you're doing. In your case, I
> would make at least 4 subdirectories of my main work directory:
> * .../service, the live scandir, containing only symlinks
> * .../services, which contains the service directories. Those could
> be live copies of the service directories, or your main service
> repository if you don't mind having supervise/ and event/ subdirectories
> in them.
> * .../oneshots, which contains your one-shot scripts.
> * .../config (or something), which contains your system manager's own
> data.
> Add more subdirectories to your convenience, but /service is a special
> snowflake and should remain that way.
>
Received on Wed Jan 21 2015 - 20:47:28 UTC