so-telegraf mounted /var/run/docker.sock and joined the host docker group on
every node type. The :ro flag blocks write() to the inode, not connect() plus
HTTP over the socket, so any code execution inside the container could reach
POST /containers/create with Privileged:true and become root on the host.
No telegraf script used the socket; the only consumer was the native
[[inputs.docker]] plugin, and group_add 920 existed solely to feed it.
Container metrics now come from so-container-stats, a collector that runs on
the host from cron and writes influx line protocol to a file telegraf already
had mounted. This is the pattern so-status, so-raid-status and
so-elasticagent-status already use, so the privileged docker access stays on
the host side where root cron already ran it.
The collector runs as somon, a service account in the docker group with no
login shell and a locked password, rather than root. Docker group membership
is still root-equivalent on the host, so this is defense in depth rather than
a privilege boundary.
The telegraf scripts were root:939 mode 770, letting socore rewrite them for
code execution inside the container; they are now 750, which still allows the
read and execute telegraf needs. The container also ran with no group, giving
it gid 0, and now runs as 939:939. That alone would have broken
lasthighstate.sh, which reached /opt/so/log/salt only via the root group and
could not tell an unreadable file from a missing one, so it silently reported
a 56 year highstate age. Only the lasthighstate file is bind mounted now, and
the script tests readability instead of existence.
Everything inputs.docker collected beyond the five fields the shipped
dashboards query is available per stat under telegraf:container_stats,
annotated for SOC so an operator can enable it without editing files. Defaults
reproduce the previous output exactly. With every stat enabled the emitted
field set matches what inputs.docker wrote, verified by running the plugin
against the live socket and diffing: 53 fields, no type mismatches, no field
present on one side only.
Two deliberate differences: max_usage carries the real cgroup peak where the
daemon reports 0 on cgroup v2, and host-network containers emit no
docker_container_net row, matching inputs.docker. Docker label tags are not
restored, since nothing queries them and they cost significant cardinality.
so-status, the influxdb size cron, so-elasticagent-status, so-raid-status and
so-common-status-check truncated their output in place while telegraf read it,
so telegraf periodically saw an empty file and logged a parse error or emitted
empty values. They now write aside and rename. Measured on a live manager, the
old so-status cron left status.log empty for 215 of 10997 reads.
Tested on a fresh install, a converted grid and a 3.0 upgrade.
Salt installed the so-* scripts into /usr/sbin owned by unprivileged service
UIDs (939/socore, plus 930-960 per service) at mode 755, while root executes
those same files from cron, systemd and state cmd.run. Any file-write
primitive as one of those UIDs was therefore root.
Two mechanisms behind this are not visible in the diff:
file.recurse also manages the destination directory, so /usr/sbin itself was
chowned to whichever service UID ran last. A directory's owner may always
chmod it, so that UID could replace even the scripts already declared
user: root -- so-config-backup, so-suricata-eve-clean, so-nsm-mount-nvme.
usr_sbin_perms now pins the directory to root:root 555, the mode the
filesystem RPM ships.
Omitting user:/group: is a no-op on files that already exist, because
check_perms only chowns when a user is named. Explicit user: root is what
lets upgraded grids self-heal on the next highstate, and what makes a
revert chown back rather than silently do nothing.
The UEK8 rollout installs the new kernel and flips the boot default, but
leaves the stock EL9 (RHCK) packages behind: disk in /boot and a stale
GRUB entry on every upgraded node.
They cannot be removed in the same pass that installs UEK8. dnf's
protect_running_kernel refuses to erase the booted kernel-core, so the
removal has to wait until the node has rebooted onto 6.x. Waiting is the
safer sequencing anyway -- the node proves it comes up on UEK8 before its
fallback is deleted -- so this does not remove RHCK from the uek7 branch
either, where dnf would allow it.
so-kernel-upgrade grows a --cleanup mode that does only the removal and
no-ops (exit 0, with a log line) on a node not yet running UEK8. Its uek8
branch, which previously reported "nothing to do", now runs that cleanup
along with set_default_kernel_conf -- which also closes a gap where a node
that came up on UEK8 straight from a fresh install never had
DEFAULTKERNEL=kernel-uek-core written.
The common highstate calls --cleanup gated on the running kernel, so the
cleanup lands grid-wide as each node reboots: fresh installs reboot at the
end of setup, upgraded nodes whenever the admin schedules it. The rpm
check inside the script is the idempotency guard, so subsequent highstates
cost an rpm query rather than a dnf transaction, and the package list is
not duplicated into the state where it could drift.
Add so-nic-pin, which writes by-MAC persistent-net udev rules pinning each
physical NIC to its current name so a kernel upgrade can't renumber the
interfaces Security Onion binds by name (host:mainint, sensor:mainint, bond0).
Gated by the drop file /opt/so/state/nic_names_pinned: run-once on highstate,
and an admin can pre-create the marker to opt out. Wired into common/init.sls
as pin_nic_names, guarded by a matching unless.
Josh Patterson
Mike Reeves
Jason Ertel
Mike Reeves
defensivedepth
Wes
Jason Ertel
Doug Burks
William Wernert