memmon Overview¶
memmon is a supervisor “event listener” which may be subscribed to
a concrete TICK_x
event. When memmon receives a TICK_x
event (TICK_60
is recommended, indicating activity every 60 seconds),
memmon checks that a configurable list of programs (or all
programs running under supervisor) are not exceeding a configurable amount of
memory (resident segment size, or RSS). If one or more of these processes is
consuming more than the amount of memory that memmon believes it
should, memmon will restart the process. memmon can be
configured to send an email notification when it restarts a process.
memmon is known to work on Linux and Mac OS X, but has not been tested on other operating systems (it relies on ps output and command-line switches).
memmon is incapable of monitoring the process status of processes which are not supervisord child processes. Without the –cumulative option, only the RSS of immediate children of the supervisord process will be considered.
memmon is a “console script” installed when you install
superlance
. Although memmon is an executable program, it
isn’t useful as a general-purpose script: it must be run as a
supervisor event listener to do anything useful.
memmon uses Supervisor’s XML-RPC interface. Your supervisord.conf
file must have a valid [unix_http_server]
or [inet_http_server]
section, and must have an [rpcinterface:supervisor]
section. If you are able to control your supervisord
instance with
supervisorctl
, you have already met these requirements.
Command-Line Syntax¶
$ memmon [-c] [-p processname=byte_size] [-g groupname=byte_size] \
[-a byte_size] [-s sendmail] [-m email_address] \
[-u email_uptime_limit] [-n memmon_name]
-
-h
,
--help
¶
Show program help.
-
-c
,
--cumulative
¶
Check against cumulative RSS. When calculating a process’ RSS, also consider its child processes. With this option memmon will sum up the RSS of the process to be monitored and all its children.
-
-p
<name/size pair>
,
--program
=<name/size pair>
¶ A name/size pair, e.g. “foo=1MB”. The name represents the supervisor program name that you would like memmon to monitor; the size represents the number of bytes (suffix-multiplied using “KB”, “MB” or “GB”) that should be considered “too much”.
This option can be provided more than once to have memmon monitor more than one program.
Programs can be specified using a “namespec”, to disambiguate same-named programs in different groups, e.g.
foo:bar
represents the programbar
in thefoo
group.
-
-g
<name/size pair>
,
--groupname
=<name/size pair>
¶ A groupname/size pair, e.g. “group=1MB”. The name represents the supervisor group name that you would like memmon to monitor; the size represents the number of bytes (suffix-multiplied using “KB”, “MB” or “GB”) that should be considered “too much”.
Multiple
-g
options can be provided to have memmon monitor more than one group. If any process in this group exceeds the maximum, it will be restarted.
-
-a
<size>
,
--any
=<size>
¶ A size (suffix-multiplied using “KB”, “MB” or “GB”) that should be considered “too much”. If any program running as a child of supervisor exceeds this maximum, it will be restarted. E.g. 100MB.
-
-s
<command>
,
--sendmail
=<command>
¶ A command that will send mail if passed the email body (including the headers). Defaults to
/usr/sbin/sendmail -t -i
.
Note
Specifying this option doesn’t cause memmon to send mail by itself:
see the -m
/ --email
option.
-
-m
<email address>
,
--email
=<email address>
¶ An email address to which to send email when a process is restarted. By default, memmon will not send any mail unless an email address is specified.
-
-u
<email uptime limit>
,
--uptime
=<email uptime limit>
¶ Only send an email in case the restarted process’ uptime (in seconds) is below this limit. (Useful to only get notified if a processes gets restarted too frequently)
Uptime is given in seconds (suffix-multiplied using “m” for minutes, “h” for hours or “d” for days)
-
-n
<memmon name>
,
--name
=<memmon name>
¶ An optional name that identifies this memmon process. If given, the email subject will start with
memmon [<memmon name>]:
instead ofmemmon:
In case you run multiple supervisors on a single host that control different processes with the same name (eg zopeinstance1) you can use this option to indicate which project the restarted instance belongs to.
Configuring memmon Into the Supervisor Config¶
An [eventlistener:x]
section must be placed in supervisord.conf
in order for memmon to do its work. See the “Events” chapter in the
Supervisor manual for more information about event listeners.
If the [unix_http_server]
or [inet_http_server]
has been configured to use authentication, add the environment variables
SUPERVISOR_USERNAME
and SUPERVISOR_PASSWORD
in the [eventlistener:x]
section as shown in Example Configuration 5.
The following examples assume that memmon is on your system
PATH
.
Example Configuration 1¶
This configuration causes memmon to restart any process which is
a child of supervisord consuming more than 200MB of RSS, and will
send mail to bob@example.com
when it restarts a process using the
default sendmail command.
[eventlistener:memmon]
command=memmon -a 200MB -m bob@example.com
events=TICK_60
Example Configuration 2¶
This configuration causes memmon to restart any process with the
supervisor program name foo
consuming more than 200MB of RSS, and
will send mail to bob@example.com
when it restarts a process using
the default sendmail command.
[eventlistener:memmon]
command=memmon -p foo=200MB -m bob@example.com
events=TICK_60
Example Configuration 3¶
This configuration causes memmon to restart any process in the
process group “bar” consuming more than 200MB of RSS, and will send mail to
bob@example.com
when it restarts a process using the default
sendmail command.
[eventlistener:memmon]
command=memmon -g bar=200MB -m bob@example.com
events=TICK_60
Example Configuration 4¶
This configuration causes memmon to restart any process meeting the same requirements as in Example Configuration 2 with one difference:
The email will only be sent if the process’ uptime is less or equal than 2 days (172800 seconds)
[eventlistener:memmon]
command=memmon -p foo=200MB -m bob@example.com -u 2d
events=TICK_60
Example Configuration 5 (Authentication)¶
This configuration is the same as the one in Example Configuration 1 with the only difference being that the [unix_http_server] or [inet_http_server] has been configured to use authentication.
[eventlistener:memmon]
command=memmon -a 200MB -m bob@example.com
environment=SUPERVISOR_USERNAME="<username>",SUPERVISOR_PASSWORD="<password>"
events=TICK_60