Commit Graph
18813 Commits
Author SHA1 Message Date
Josh Brower b32aaac290 Merge pull request #16278 from Security-Onion-Solutions/evtx-import-datastreams
EVTX Import cleanup
2026-10-02 13:30:57 -04:00
Josh Brower 1aee3f28dc Merge pull request #16279 from Security-Onion-Solutions/process-caseless-mappings
caseless for non-Defend sources
2026-10-02 13:30:45 -04:00
Josh Patterson 678cb0d5b2 Merge pull request #16286 from Security-Onion-Solutions/fix/docker-29.8.1
upgrade docker 29.8.1 and containerd 2.3.6
2026-10-02 12:05:04 -04:00
Josh Patterson 31c5190a1f add missing comma 2026-10-02 12:03:04 -04:00
Josh Patterson 4ce7a06abe upgrade docker to 29.8.1 and containerd.io to 2.3.6
Latest upstream stable for el9. All four NVRs are already carried by the SO
prod repo, so no repo change is needed -- a so-repo-sync refresh is enough.

Tested on a managersearch and a heavynode (OL 9.8, 3.4.0), upgrading from
29.2.1/2.2.1 both by hand and through the state itself:

- The 29.8.1 RPM ships a byte-identical docker.service, so the full ExecStart
  override in files/iptables-disabled.conf still resolves correctly and the
  hand-written DOCKER/DOCKER-ISOLATION/DOCKER-USER chains came back
  byte-identical on both nodes across upgrade, restart and reboot.
- update_holds re-pinned the versionlock from the old NVRs to the new ones
  without intervention, so soup's path needs no change.
- docker-py 7.1.0 still creates sobridge and soauth (forced by removing both);
  bridges keep their configured kernel names rather than br-<hash>.
- 29.6 changed how dynamic port allocation treats
  net.ipv4.ip_local_reserved_ports; Strelka's 57314 is both published and
  reserved, and docker-proxy still owns it with no bind errors.
- docker ps --format json gained a HealthStatus key. Additive, so so-status,
  so-log-check and so-docker-prune all still parse it.
- containerd 2.3.6 ships the same config.toml, and it is %config(noreplace)
  and unmodified on disk, so no .rpmnew and disabled_plugins=["cri"] survives.
- Zeek/Suricata/Strelka pipeline verified end-to-end with so-test: 111k packets
  replayed, 0 capture loss, file extraction and ES ingest all landed.

The manifest unknown exclusion in so-log-check still fires on 29.8.1 -- it
comes from a tag lookup during the registry-to-registry image copy, not from
the 29.2.1 upgrade the old comment blamed -- so only the comment changes.
2026-10-02 12:03:04 -04:00
defensivedepth 43475452b3 set module 2026-10-01 19:17:52 -04:00
defensivedepth 99322cf26a Add additional mapping 2026-10-01 15:24:52 -04:00
coreyogburn 117548757f Merge pull request #16280 from Security-Onion-Solutions/cogburn/unified-automations
Unified Automations
2026-10-01 10:29:45 -06:00
Corey Ogburn 22bda63847 Unified Automations
Remove the template and mark automations as advanced, readonly, and stored in the DB.
2026-10-01 09:59:12 -06:00
defensivedepth 2a4611df45 Add caseless mappings 2026-10-01 10:48:55 -04:00
defensivedepth 89f8bcd19f evtx-import fixup 2026-10-01 10:28:10 -04:00
Jason Ertel 563269cbac Merge pull request #16277 from Security-Onion-Solutions/jertel/wip
support empty yaml files
2026-10-01 10:10:55 -04:00
Jason Ertel 523c39d4f2 fix flake 2026-10-01 10:09:20 -04:00
Jason Ertel b4557e973c support empty yaml files 2026-10-01 10:03:39 -04:00
Jorge Reyes 0f53a7e0bc Merge pull request #16275 from Security-Onion-Solutions/reyesj2-521
review integration-defaults weird_integrations mappings, removed unus…
2026-10-01 08:36:57 -05:00
reyesj2 d122ee7fea review integration-defaults weird_integrations mappings, removed unused, updated logstash integration naming 2026-09-30 16:49:19 -05:00
coreyogburn 47d74f1ae1 Merge pull request #16270 from Security-Onion-Solutions/cogburn/automation
New Automation Fields
2026-09-30 11:10:51 -06:00
Corey Ogburn 855716846a New Automation Fields 2026-09-29 16:50:04 -06:00
Mike Reeves 8e35d70595 Merge pull request #16266 from Security-Onion-Solutions/mreeves/soai-context-1m
Raise SOAI Sonnet default small context limit to 1M
2026-09-29 12:19:20 -04:00
Jason Ertel e4625cfcae Merge pull request #16267 from Security-Onion-Solutions/jertel/wip
resolve startup errors
2026-09-29 12:07:26 -04:00
Jason Ertel eb803dce0e resolve startup errors 2026-09-29 12:02:48 -04:00
Mike Reeves a06f08217a Raise SOAI Sonnet default small context limit to 1M
Context is now flat-priced, so match contextLimitSmall to contextLimitLarge.
With equal limits the SOC assistant hides the increase-context toggle.
2026-09-29 11:42:29 -04:00
Josh Patterson 29d27cf255 Merge pull request #16261 from Security-Onion-Solutions/fix/telegraf-drop-docker-socket
FIX: remove the docker socket from so-telegraf
2026-09-29 10:46:54 -04:00
Jason Ertel 235a60e587 Merge pull request #16264 from Security-Onion-Solutions/jertel/wip
Metric alarms and more NTF annotations
2026-09-29 08:31:28 -04:00
Jason Ertel a8f7c46b0d Merge branch '3/dev' into jertel/wip 2026-09-28 13:47:19 -04:00
Jason Ertel 26d895ccb7 alarms and ntf 2026-09-28 13:47:16 -04:00
Josh Patterson b43efc458f Merge pull request #16263 from Security-Onion-Solutions/fix/service-account-nologin
FIX: use /sbin/nologin for service accounts
2026-09-28 13:05:26 -04:00
Josh Patterson 21222ff119 FIX: use /sbin/nologin for service accounts
These accounts existed only for container UID mapping and filesystem
ownership, but user.present omitted shell:, so Salt fell through to the
platform useradd default and every one of them got /bin/bash. Pin them to
/sbin/nologin so none can be used as an interactive login or `su -` target.

socore keeps /bin/bash: `su socore -c '/usr/sbin/so-repo-sync'` in soup and
so-kernel-upgrade execs the account's passwd shell, and operator docs tell
users to su to socore. soqemussh keeps /bin/bash as an SSH login account.

elastic-agent, elastic-agent-pr and kafka are included alongside the accounts
named in the issue, being the same class with the same unset shell, so the
default is uniform.

Cron is unaffected: cronie runs jobs via the crontab SHELL (default /bin/sh),
not the passwd shell. suricata is the only account changed here that owns a
crontab, and somon has shipped as nologin with a working cron job already.
The zeek `runuser -l zeek` calls all run inside so-zeek via docker.run/exec,
so they resolve the shell from the image, not the host.

Verified on a 3.4.0 managersearch + sensor grid: highstate converges with the
shell as the only change and no failures, is idempotent on a second run, all
containers stay up, SOC still issues a Kratos login flow, and the suricata
surilogcompress cron job runs post-change ((suricata) CMD/CMDEND in
/var/log/cron) while `su - suricata` is now refused.

Closes #16256
2026-09-25 09:23:09 -04:00
Mike Reeves 88fa7e7fb4 Merge pull request #16262 from Security-Onion-Solutions/TOoSmOotH-patch-4
Add openai_embeddings to the YAML configuration
2026-09-24 15:29:17 -04:00
Mike Reeves efe0581892 Add openai_embeddings to the YAML configuration 2026-09-24 15:27:51 -04:00
Josh Patterson 72f60fcaa9 FIX: remove the docker socket from so-telegraf
so-telegraf mounted /var/run/docker.sock and joined the host docker group on
every node type. The :ro flag blocks write() to the inode, not connect() plus
HTTP over the socket, so any code execution inside the container could reach
POST /containers/create with Privileged:true and become root on the host.

No telegraf script used the socket; the only consumer was the native
[[inputs.docker]] plugin, and group_add 920 existed solely to feed it.
Container metrics now come from so-container-stats, a collector that runs on
the host from cron and writes influx line protocol to a file telegraf already
had mounted. This is the pattern so-status, so-raid-status and
so-elasticagent-status already use, so the privileged docker access stays on
the host side where root cron already ran it.

The collector runs as somon, a service account in the docker group with no
login shell and a locked password, rather than root. Docker group membership
is still root-equivalent on the host, so this is defense in depth rather than
a privilege boundary.

The telegraf scripts were root:939 mode 770, letting socore rewrite them for
code execution inside the container; they are now 750, which still allows the
read and execute telegraf needs. The container also ran with no group, giving
it gid 0, and now runs as 939:939. That alone would have broken
lasthighstate.sh, which reached /opt/so/log/salt only via the root group and
could not tell an unreadable file from a missing one, so it silently reported
a 56 year highstate age. Only the lasthighstate file is bind mounted now, and
the script tests readability instead of existence.

Everything inputs.docker collected beyond the five fields the shipped
dashboards query is available per stat under telegraf:container_stats,
annotated for SOC so an operator can enable it without editing files. Defaults
reproduce the previous output exactly. With every stat enabled the emitted
field set matches what inputs.docker wrote, verified by running the plugin
against the live socket and diffing: 53 fields, no type mismatches, no field
present on one side only.

Two deliberate differences: max_usage carries the real cgroup peak where the
daemon reports 0 on cgroup v2, and host-network containers emit no
docker_container_net row, matching inputs.docker. Docker label tags are not
restored, since nothing queries them and they cost significant cardinality.

so-status, the influxdb size cron, so-elasticagent-status, so-raid-status and
so-common-status-check truncated their output in place while telegraf read it,
so telegraf periodically saw an empty file and logged a parse error or emitted
empty values. They now write aside and rename. Measured on a live manager, the
old so-status cron left status.log empty for 215 of 10997 reads.

Tested on a fresh install, a converted grid and a 3.0 upgrade.
2026-09-24 14:30:25 -04:00
Jorge Reyes 2684a5ca95 Merge pull request #16260 from Security-Onion-Solutions/reyesj2/16254
FIX: Fleet scripts failing when endpoints-initial is missing
2026-09-23 08:58:15 -05:00
reyesj2 bcee63bde5 remove policy precheck 2026-09-23 08:51:11 -05:00
reyesj2 6ce89eb323 jq -e exits 0 with empty input 2026-09-22 21:25:50 -05:00
reyesj2 ebab4b0d90 split between missing token and multiple enrollment tokens 2026-09-22 16:11:56 -05:00
reyesj2 db60c27da2 fail cleanly when endpoints-initial is missing or has multiple enrollment tokens 2026-09-22 12:24:23 -05:00
Jason Ertel f4defdfde0 Merge pull request #16258 from Security-Onion-Solutions/jertel/wip
upgrade whoislookup analyzer deps; notification annotations
2026-09-22 08:41:04 -04:00
Jason Ertel 36652e8f23 fixed malformed desc annotation 2026-09-22 08:30:44 -04:00
Jason Ertel 7bef194540 force transitive dep version 2026-09-22 08:28:44 -04:00
Jason Ertel e2bf2837fe notification annotations 2026-09-22 08:18:59 -04:00
Jason Ertel 06704dad22 upgrade whoislookup analyzer deps 2026-09-22 08:17:18 -04:00
Mike Reeves 47fe0758d0 Merge pull request #16255 from Security-Onion-Solutions/mreeves/fix-so-user-stdin-drain
Fix user creation failing with a valid password
2026-09-18 17:12:09 -04:00
Mike Reeves b71fd93f9d Stop docker exec from consuming so-user's piped password
Adding a user failed with "Password does not meet the minimum requirements"
for passwords that were well over the eight character minimum.

so-user reads the password from stdin in updatePassword, but verifyEnvironment
runs first and calls kratosCurl, which ran docker exec with -i. That attaches
stdin and drains the pipe the caller sent the password on, so the later
read -rs saw EOF, password was empty, and expr length "" tripped the minimum
length check.

No kratosCurl or hydraCurl call sends a body on stdin; every one passes it as
a -d argument, so -i was never needed. Dropping it leaves the admin API
responses unchanged.

so-client had the same wrapper for the Hydra admin API. It does not currently
read stdin, but the flag drains its caller's pipe just the same, so it is
dropped there too.
2026-09-18 17:04:17 -04:00
Mike Reeves d9eff9aa9e Merge pull request #16253 from Security-Onion-Solutions/mreeves/fix-docker-networks-setting-conflict
Fix config tree error from the docker.networks setting
2026-09-18 15:41:25 -04:00
Mike Reeves d8884dbd99 Document sobridge alongside the soauth network settings
sobridge carries most containers but held no annotation, so it was absent from
the config tree entirely. Its pillar entry is an empty map, so annotating it
adds a setting with no children for docker.networks.soauth.range to collide
with, and the tree still builds.

docker.map.jinja fills in sobridge's range and gateway at render time from
docker.range and docker.gateway, and resets the entry when it is not a mapping,
so nothing writes keys underneath it.

The sentence about sobridge is dropped from the soauth range description now
that sobridge documents itself.
2026-09-18 14:37:11 -04:00
Mike Reeves 6c0d4c15e8 Annotate the soauth network leaves instead of the networks map
SOC reported "Malformed config setting (docker.networks.soauth.range): Setting
name 'networks' conflicts with another similarly named setting" and refused to
render the config tree.

soc_docker.yaml annotated docker.networks itself, and every key in that block
was a scalar, so FlattenAnnotations registered docker.networks as a setting.
defaults.yaml holds a nested map there, so FlattenPillar separately registered
docker.networks.soauth.range, .gateway and .manager_only, and
HydrateAnnotations unions the two sets. The config tree gives each id segment
either a value or children, never both, so addToNode pushed docker.networks as
a childless leaf and then threw when docker.networks.soauth.range tried to
descend through it.

The annotations move down to the three leaves that actually carry values, so
docker.networks is a plain branch. This matches docker.containers, which is
nested the same way and has never carried annotations of its own.

docker.ulimits and the per-container networks list are unaffected: their pillar
values are lists rather than maps, so they stay single settings.
2026-09-18 13:42:41 -04:00
Jason Ertel 2e2f62f265 Merge pull request #16252 from Security-Onion-Solutions/jertel/wip
store in db
2026-09-18 12:47:32 -04:00
Jason Ertel b7a11a525c store in db 2026-09-18 12:41:35 -04:00
Mike Reeves 8eef95ea3e Merge pull request #16221 from Security-Onion-Solutions/mreeves/kratos-soauth-network
Isolate the Kratos admin API on a dedicated soauth docker network
2026-09-18 10:24:00 -04:00
Mike Reeves 65e261475d Mention Hydra in the soauth network description
The description was written alongside the Kratos change and named only the
Kratos admin API, but so-hydra now sits on soauth too.

The trailing note about rebuilding the firewall is dropped, since the setting
is readonly and cannot be changed from SOC in the first place.
2026-09-18 10:17:20 -04:00