Working on shell for perp/s6/etc., need advice re logging

From: Ido Perlmuter <ido_at_ido50.net>
Date: Tue, 28 Jul 2015 10:07:02 +0300

Hello there.

A few months ago I've transitioned some of my production servers from
supervisord to perp. In order to make the transition smoother for other
system admins on my servers I've decided to create a shell resembling
supervisorctl, where you can check the status of services,
stop/start/restart them, send them signals, etc. with simple commands (such
as "restart nginx").

I've also decided to write the shell such that it would support several
supervision suites, starting with perp and s6. Of course, the shell merely
calls the relevant utilities of the suite to actually do its work. For
example, echoing "restart nginx" on the shell prompt simply calls "perpctl
q nginx" internally.

Anyway, while I've mostly implemented everything I need thus far, there is
one point that is causing me grief. supervisorctl allowed me to view the
stdout/stderr of running processes at any given time. It provided two ways
to do this, one with the "fg" command (e.g. "fg nginx"), which would bring
the process to the foreground (probably "virtually") and start displaying
its stdout/stderr; the other with the "tail" command (e.g. "tail nginx
stderr"), which allows viewing the latest messages in a specific channel
(but not both).

The thing is that supervisord basically enforces its own logging mechanism.
It stores stdout and stderr to seperate files, and it can read them
whenever it wants. With perp/s6/etc, however, the user is free to decide
what logging mechanism to use.

Currently, I've implemented the "fg" command in a pretty dumb way: I read
the service's rc.log file for perp or log/run file for s6, and take the
last argument to tinylog or s6-log to figure out where log files are
stored. Of course, this is a very bad way to do this, since I'm assuming
the services are using tinylog/s6-log, and I'm using a regular expression
that could easily miss.

I'm looking for some advice how reading the service's stdout/stderr streams
could be done in a more fool proof, general way. The only way I know to tap
into a process' output streams is via strace, but that means the user will
have to install it, and run it as root, so that's not good.

Any suggestions are welcome. If you're curious, the shell's repository is on
github <https://github.com/ido50/Svsh>, it's currently written in Perl (I
wanted get it up and running as fast as possible), I may reimplement it
later with another language.

Thanks,
Ido Perlmuter.
Received on Tue Jul 28 2015 - 07:07:02 UTC

This archive was generated by hypermail 2.3.0 : Sun May 09 2021 - 19:44:19 UTC