Commit Graph

1936 Commits

Author SHA1 Message Date
Josh Patterson 66c0a662fc convert wait to script 2026-05-08 09:26:42 -04:00
Josh Patterson f774334b6c Merge remote-tracking branch 'origin/3/dev' into saltthangs 2026-05-06 08:16:41 -04:00
reyesj2 702b3585cc excluding additional integration transform job failures 2026-05-01 12:57:59 -05:00
Josh Patterson d3d30a587c Merge remote-tracking branch 'origin/3/dev' into saltthangs 2026-04-28 15:30:31 -04:00
Mike Reeves 82dac82d15 drop platform/digest pull resolution
The digest-pull logic was added to make `docker push` work for multi-arch
upstream tags. Now that the push step is `docker buildx imagetools create`
pinned to the gpg-verified RepoDigest, the registry-to-registry copy
handles single- and multi-arch sources without help. Reverts the pull
back to the original line and removes the unused PLATFORM_OS/_ARCH
detection.
2026-04-28 14:54:25 -04:00
Mike Reeves 288a823edf push images via buildx imagetools create
Replaces `docker push` with a registry-to-registry copy. On Docker 29.x
with the containerd image store, `docker push` of a freshly-pulled image
hits a path that wraps single-platform manifests in a synthetic index
and then can't push the layers it claims to reference, producing
`NotFound: content digest ...` even when the image is fully present.

Keep the local `docker tag` so so-image-pull's `docker images | grep :5000`
existence check continues to work.
2026-04-28 14:49:02 -04:00
Mike Reeves c86399327b fix so-docker-refresh push for multi-arch source images
docker pull of a multi-arch tag on Docker 29.x leaves the local tag
pointing at the image index rather than the platform-specific manifest.
The subsequent docker push then tries to push every sub-manifest the
index references and fails on layers we never fetched.

Resolve the local-platform manifest digest from the upstream index via
docker buildx imagetools inspect, pull by that digest, and re-tag locally
to the canonical tag. The signing flow and the existing tag/push to the
embedded registry are unchanged.
2026-04-28 14:27:59 -04:00
Josh Patterson 034711d148 Merge remote-tracking branch 'origin/3/dev' into saltthangs 2026-04-28 10:47:29 -04:00
Mike Reeves fa8162de02 Merge pull request #15749 from Security-Onion-Solutions/feature/postgres
Add so-postgres Salt states and infrastructure
2026-04-28 10:15:47 -04:00
Jorge Reyes b22585ca90 Merge pull request #15833 from Security-Onion-Solutions/reyesj2-es933
exclude more transform job errors
2026-04-27 15:05:11 -05:00
reyesj2 9f2ca7012f exclude more transform job errors 2026-04-27 15:02:13 -05:00
Josh Patterson cd6707a566 Merge pull request #15800 from Security-Onion-Solutions/feature/vm-raid-status
monitor raid for vms
2026-04-22 09:42:44 -04:00
Jason Ertel 247091766c more error handling during image updates 2026-04-21 10:18:05 -04:00
Josh Patterson ee437265fc monitor raid for vms 2026-04-20 12:00:02 -04:00
Mike Reeves 5228668be0 Fix Telegraf→Postgres table creation and state.apply race
- Telegraf's partman template passed p_type:='native', which pg_partman
  5.x (the version shipped by postgresql-17-partman on Debian) rejects.
  Switched to 'range' so partman.create_parent() actually creates
  partitions and Telegraf's INSERTs succeed.
- Added a postgres_wait_ready gate in telegraf_users.sls so psql execs
  don't race the init-time restart that docker-entrypoint.sh performs.
- so-verify now ignores the literal "-v ON_ERROR_STOP=1" token in the
  setup log. Dropped the matching entry from so-log-check, which scans
  container stdout where that token never appears.
2026-04-17 13:00:12 -04:00
Mike Reeves c124186989 so-log-check: exclude psql ON_ERROR_STOP flag
The psql invocation flag '-v ON_ERROR_STOP=1' used by the so-postgres
init script gets flagged by so-log-check because the token 'ERROR'
matches its error regex. Add to the exclusion list.
2026-04-15 19:45:42 -04:00
Mike Reeves a0cf0489d6 reduce highstate frequency with active push for rules and pillars
- schedule highstate every 2 hours (was 15 minutes); interval lives in
  global:push:highstate_interval_hours so the SOC admin UI can tune it and
  so-salt-minion-check derives its threshold as (interval + 1) * 3600
- add inotify beacon on the manager + master reactor + orch.push_batch that
  writes per-app intent files, with a so-push-drainer schedule on the manager
  that debounces, dedupes, and dispatches a single orchestration
- pillar_push_map.yaml allowlists the apps whose pillar changes trigger an
  immediate targeted state.apply (targets verified against salt/top.sls);
  edits under pillar/minions/ trigger a state.highstate on that one minion
- host-batch every push orchestration (batch: 25%, batch_wait: 15) so rule
  changes don't thundering-herd large fleets
- new global:push:enabled kill-switch tears down the beacon, reactor config,
  and drainer schedule on the next highstate for operators who want to keep
  highstate-only behavior
- set restart_policy: unless-stopped on 23 container states so docker
  recovers crashes without waiting for the next highstate; leave registry
  (always), strelka/backend (on-failure), kratos, and hydra alone with
  inline comments explaining why
2026-04-10 15:43:16 -04:00
Mike Reeves 358a2e6d3f Add so-postgres to container image pull list
Add to both the import and default manager container lists so
the image gets downloaded during installation.
2026-04-09 10:02:41 -04:00
Jorge Reyes 20c4da50b1 Merge pull request #15632 from Security-Onion-Solutions/reyesj2-15601
fix global override settings affecting non-data stream indices
2026-03-18 10:51:17 -05:00
Jason Ertel 83bd8a025c ignore redis restart warning in logstash log 2026-03-18 10:59:20 -04:00
reyesj2 1a943aefc5 rollover datastreams to get latest index templates + remove existing ilm policies from so-case / so-detection indices 2026-03-17 13:49:20 -05:00
Mike Reeves d2cee468a0 Remove support for non-Oracle Linux 9 operating systems
Security Onion now exclusively supports Oracle Linux 9. This removes
detection, setup, and update logic for Ubuntu, Debian, CentOS, Rocky,
AlmaLinux, and RHEL.
2026-03-16 16:44:07 -04:00
Jason Ertel 7f07c96a2f pcapout still used for extracts 2026-03-09 14:58:27 -04:00
Jason Ertel e8adea3022 restore pcapout since it's still used 2026-03-07 08:20:08 -05:00
Jason Ertel 71839bc87f remove steno 2026-03-06 15:45:36 -05:00
Jason Ertel 2c4d833a5b update 2.4 references to 3 2026-03-05 11:05:19 -05:00
Jason Ertel 863276e24f Merge pull request #15539 from Security-Onion-Solutions/jertel/wip
prepare for nextgen docs
2026-02-27 13:18:47 -05:00
Jason Ertel 9bd5e1897a prepare for nextgen docs 2026-02-27 13:09:55 -05:00
Josh Patterson 78ae6cd84c upgrade docker 2026-02-20 12:29:23 -05:00
reyesj2 6ce6eb95d6 use existing retry 2026-01-29 15:54:36 -06:00
reyesj2 b3d1dd51a4 initialize specific indices as needed 2026-01-29 15:41:39 -06:00
reyesj2 6b1939b827 exclude known issues with 3 integrations 2026-01-27 12:59:17 -06:00
reyesj2 55b3fa389e no dates 2026-01-23 16:33:22 -06:00
reyesj2 b3ae716929 ignore kratos file mapping error 2026-01-23 16:31:30 -06:00
Josh Patterson f6bde3eb04 remove double logging 2026-01-20 11:56:31 -05:00
Josh Patterson a192455fae Merge remote-tracking branch 'origin/2.4/dev' into bravo 2026-01-19 17:17:58 -05:00
reyesj2 d430ed6727 false positive 2026-01-15 15:25:28 -06:00
reyesj2 349d77ffdf exclude kafka restart error 2026-01-15 14:43:57 -06:00
Josh Patterson 152f2e03f1 Merge remote-tracking branch 'origin/2.4/dev' into bravo 2026-01-06 15:15:30 -05:00
Jason Ertel 2d705e7caa exempt kratos online check 2026-01-06 09:47:35 -05:00
Josh Patterson 1475f0fc2f timestamp logging for wait_for_salt_minion 2025-12-12 16:30:42 -05:00
Josh Patterson 8158fee8fc change how we determine if the salt-minion is ready 2025-12-12 15:24:47 -05:00
DefensiveDepth f15a39c153 Add historical hashes 2025-12-03 11:24:04 -05:00
DefensiveDepth 9a6ff75793 Merge remote-tracking branch 'origin/2.4/dev' into idstools-refactor 2025-11-12 08:51:51 -05:00
Josh Patterson e3972dc5af Merge remote-tracking branch 'origin/2.4/dev' into bravo 2025-11-10 13:28:42 -05:00
Josh Patterson 274295bc97 return exit codes 2025-11-07 17:39:13 -05:00
reyesj2 a84df14137 rename forward node -> sensor node 2025-11-06 15:23:55 -06:00
DefensiveDepth 2f6fb717c1 Merge remote-tracking branch 'origin/2.4/dev' into idstools-refactor 2025-11-06 10:38:37 -05:00
reyesj2 635545630b strelka use single master image 2025-11-03 09:36:46 -06:00
Josh Patterson 1949be90c2 allow to preserve files 2025-10-29 16:49:59 -04:00