"zeekctl cron" writes node statistics to /nsm/zeek/logs/stats. The CPU and memory
half comes from a helper that shells out to top, which the Zeek container does not
include. The helper's "command not found" output is then parsed as process data, so
every cron run appended a line per node reading "bad output from top", which
so-log-check reports.
Nothing wrote that file before, since log_stats and update_http_stats only run from
"zeekctl cron". Set StatsLogEnable to 0 so neither runs, and mark it read only since
the CPU and memory statistics cannot work with this image. The interface counters it
also collects are not used anywhere in Security Onion, which tracks Zeek packet loss
separately through packetloss.log and Telegraf, so nothing is lost by turning this
off. Note in StatsLogExpireInterval that it does nothing while the stats log is off.
MailTo, MailConnectionSummary and MailHostUpDown do nothing but send mail, and
the Zeek container has no mail program, so nothing they control can happen. Mark
them read only rather than offering knobs in SOC that cannot take effect.
MailConnectionSummary only gates the emailed copy; the connection summary is
generated and archived either way. MailTo also feeds Notice::mail_dest, but
Security Onion never enables the notice email action, so that half is inert too.
MailHostUpDown gates only the notification text - host status detection, the
plugin hook and the stored state all run regardless.
MinDiskSpace stays editable. It is not mail only: setting it to 0 skips the disk
space check entirely, and the warning it produces is not emailed but does appear
in the output of "zeekctl cron". Correct its description, and MailHostUpDown's,
which both claimed these settings have no visible effect.
LogExpireInterval, StatsLogExpireInterval and CrashExpireInterval are only acted
on by "zeekctl cron", which nothing in the grid ran, so setting them in SOC did
nothing. Add so-zeek-cron and run it every 5 minutes, the interval upstream
recommends.
This also restarts a node that died unexpectedly and marks it crashed so a crash
report is written, which is what CrashExpireInterval then reaps.
The crontab runs as root because the script needs the docker socket; it drops to
the zeek user inside the container so the stats logs and zeekctl-config.sh it
writes stay owned by uid 937.
Annotate the five zeekctl settings that were previously undocumented. The regex
on LogExpireInterval matters: a bare number means days, and a value shorter than
LogRotationInterval raises ConfigurationError, which fails the zeekctl deploy in
the container entrypoint. Zeek then never starts while Salt still reports success
and the container still reports healthy. Excluding the min unit keeps that
unreachable at the default 3600 second rotation interval. MinDiskSpace and
MailHostUpDown only send mail and the image has no sendmail, so their
descriptions say they currently have no effect.
- schedule highstate every 2 hours (was 15 minutes); interval lives in
global:push:highstate_interval_hours so the SOC admin UI can tune it and
so-salt-minion-check derives its threshold as (interval + 1) * 3600
- add inotify beacon on the manager + master reactor + orch.push_batch that
writes per-app intent files, with a so-push-drainer schedule on the manager
that debounces, dedupes, and dispatches a single orchestration
- pillar_push_map.yaml allowlists the apps whose pillar changes trigger an
immediate targeted state.apply (targets verified against salt/top.sls);
edits under pillar/minions/ trigger a state.highstate on that one minion
- host-batch every push orchestration (batch: 25%, batch_wait: 15) so rule
changes don't thundering-herd large fleets
- new global:push:enabled kill-switch tears down the beacon, reactor config,
and drainer schedule on the next highstate for operators who want to keep
highstate-only behavior
- set restart_policy: unless-stopped on 23 container states so docker
recovers crashes without waiting for the next highstate; leave registry
(always), strelka/backend (on-failure), kratos, and hydra alone with
inline comments explaining why
Add ulimits as a configurable advanced setting for every container,
allowing customization through the web UI. Move hardcoded ulimits
from elasticsearch and zeek into defaults.yaml and fix elasticsearch
ulimits that were incorrectly nested under the environment key.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Create /opt/so/conf/zeek/zkg directory and sync custom packages
from the manager via file.recurse. Bind mount the directory into
the so-zeek container so the entrypoint can install packages on
startup.
JA4 (BSD licensed) remains always enabled, but JA4+ variants (JA4S,
JA4D, JA4H, JA4L, JA4SSH, JA4T, JA4TS, JA4X) require a FoxIO license
and are now toggleable via the SOC UI. The toggle includes a license
agreement warning and defaults to disabled.
Josh Patterson
Mike Reeves
Mike Reeves
Doug Burks
reyesj2
Josh Brower
Jason Ertel