Commit Graph

11773 Commits

Author SHA1 Message Date
Mike Reeves 64731c73ba Fix psql :var substitution in telegraf role and retention SQL
psql does not substitute :var references inside dollar-quoted strings,
so the DO blocks in the user and retention subcommands were receiving
literal colons and failing (silently for user, via hide_output: True).
Rewrite the conditional CREATE/ALTER ROLE with SELECT format(...) \\gexec
and guard the retention UPDATE with \\gset + \\if.
2026-05-14 17:17:49 -04:00
Mike Reeves 249b126312 Quote telegraf role env vars to survive YAML-special chars in passwords 2026-05-14 17:08:51 -04:00
Mike Reeves 8e38bff0c3 Rename telegraf_postgres.sh to so-telegraf-postgres 2026-05-14 16:55:53 -04:00
Mike Reeves b9f2d56932 Consolidate telegraf postgres SQL into multi-mode script
Replace inline psql heredocs in telegraf_users.sls with subcommand
dispatcher telegraf_postgres.sh: create_db, group_role, user, retention.
2026-05-14 16:37:08 -04:00
Mike Reeves 03fa01a705 Move telegraf_role.sh to postgres tools/sbin 2026-05-14 16:18:01 -04:00
Mike Reeves 450eacca41 Move telegraf role provisioning to external script with env vars 2026-05-14 16:15:54 -04:00
Mike Reeves b7a13899f7 Suppress output logging for postgres telegraf role provisioning 2026-05-14 15:56:04 -04:00
Mike Reeves 6f273d7d97 Rename init-users.sh to init-db.sh and update all references 2026-05-14 15:53:00 -04:00
Josh Brower b328820c01 Merge pull request #15792 from Security-Onion-Solutions/3/strelkalnk
Fix module name
2026-05-14 13:06:26 +02:00
Jorge Reyes 638aca97c8 Merge pull request #15877 from Security-Onion-Solutions/reyesj2-patch-1
update redis index template
2026-05-13 13:44:04 -05:00
Jorge Reyes 74a5c895e8 Merge pull request #15889 from Security-Onion-Solutions/reyesj2/zeek-ja4d
add zeek.ja4d ingest pipeline
2026-05-13 13:43:56 -05:00
reyesj2 d56bf01823 add zeek.ja4d ingest pipeline 2026-05-13 12:32:54 -05:00
Mike Reeves 72327285b2 Change Telegraf output from BOTH to INFLUXDB 2026-05-13 11:58:21 -04:00
Josh Patterson cc7a237457 Merge pull request #15887 from Security-Onion-Solutions/m0duspwnens-patch-1
remove stig from hypervisor and managerhype
2026-05-13 10:57:58 -04:00
Josh Patterson b068ad2b35 remove stig from hypervisor and managerhype 2026-05-13 10:53:11 -04:00
reyesj2 ef79c63858 Merge branch '3/dev' of github.com:Security-Onion-Solutions/securityonion into reyesj2/strelkalnk 2026-05-12 15:20:09 -05:00
reyesj2 01fb1aa156 check pillars for ScanLNK and rename to ScanLnk 2026-05-12 15:19:44 -05:00
Doug Burks f19bdd7aae Merge pull request #15883 from Security-Onion-Solutions/reyesj2/transformhealth
use temp files to prevent jq arg too long
2026-05-12 15:36:12 -04:00
reyesj2 f637dc62d1 use temp files to prevent jq arg too long 2026-05-12 13:29:32 -05:00
Jorge Reyes 081f6fa1fb Merge pull request #15878 from Security-Onion-Solutions/reyesj2/es-ingest-lag
add ingest latency metrics
2026-05-12 10:21:04 -05:00
Josh Brower 125610ed42 Additional test coverage 2026-05-12 10:11:22 -04:00
Josh Brower 306b0af4d0 Initial commit 2026-05-12 09:55:06 -04:00
reyesj2 492ae80da7 add ingest latency metrics 2026-05-11 16:51:38 -05:00
Jorge Reyes 4a2177c827 update redis index template
missing redis integration component templates
2026-05-11 16:15:56 -05:00
Josh Brower 006ac31109 Merge pull request #15579 from marcopedrinazzi/3/dev
New Sigma rules pipeline mapping for M365 and Fortigate
2026-05-11 21:03:53 +02:00
Josh Brower e1d830da76 proc_creation per OS type 2026-05-08 09:11:24 -04:00
Josh Brower 499f7102bd cleanup status code 2026-05-07 11:27:49 -04:00
Josh Patterson 4bc19f91ce Merge pull request #15867 from Security-Onion-Solutions/fixhype
sanitize minion ids for hypervisor reactors / orchestration
2026-05-06 09:46:01 -04:00
reyesj2 dceed421ae update grok type conversion to convert processor 2026-05-05 13:41:00 -05:00
Josh Patterson 652ac5d61f fix regex 2026-05-05 14:26:04 -04:00
Josh Patterson 192f6cfe13 Merge remote-tracking branch 'origin/3/dev' into fixhype 2026-05-05 08:18:26 -04:00
Mike Reeves 5bca81d833 Merge pull request #15858 from Security-Onion-Solutions/security-fix
Fix unsafe PyYAML load in filecheck
2026-05-04 16:16:40 -04:00
Josh Patterson 1c6574c694 ensure minion ids 2026-05-04 14:03:14 -04:00
Mike Reeves b701664e04 Fix unsafe PyYAML load in filecheck 2026-05-04 12:09:35 -04:00
Jorge Reyes bc64f1431d Merge pull request #15857 from Security-Onion-Solutions/reyesj2/package-registry-health
fleet package registry health check
2026-05-04 11:05:23 -05:00
reyesj2 2203037ce7 fleet package registry health check 2026-05-04 10:52:37 -05:00
reyesj2 702b3585cc excluding additional integration transform job failures 2026-05-01 12:57:59 -05:00
reyesj2 86966d2778 reauthorize unhealthy transform jobs using kibana 9.3.3 auth flow 2026-05-01 12:44:08 -05:00
Jorge Reyes ce3ad3a895 Merge pull request #15844 from Security-Onion-Solutions/reyesj2/elastic-agent-warning
update default elastic agent logging level to warning
2026-04-30 09:46:28 -05:00
reyesj2 39d0947102 update default elastic agent logging level to warning 2026-04-29 17:38:40 -05:00
Jorge Reyes 2f01ce3b23 so-elastic-fleet-outputs-update now checks for cert drift. Remove running --cert arg on cert change to prevent highstate from running outputs-update 2x 2026-04-29 12:33:28 -05:00
Mike Reeves 71b19c1b5f Merge pull request #15840 from Security-Onion-Solutions/fix/import-postgres-firewall
Open postgres in DOCKER-USER firewall everywhere influxdb is open
2026-04-29 09:20:03 -04:00
Mike Reeves 82e55ae87f Open postgres on every hostgroup that opens influxdb
The static defaults only listed postgres on each role's self-hostgroup,
leaving sensor/searchnode/heavynode/receiver/fleet/idh/desktop/hypervisor
hostgroups unable to reach the manager's so-postgres in distributed
grids. A dynamic block in firewall/map.jinja added postgres to those
hostgroups only when telegraf.output was switched to POSTGRES/BOTH,
which left postgres unreachable by default.

Mirror influxdb statically across manager/managerhype/managersearch/
standalone for every hostgroup that already lists influxdb, and drop
the now-redundant telegraf-gated dynamic block from firewall/map.jinja.
2026-04-29 09:09:50 -04:00
Mike Reeves 3e02001544 Open postgres port for import role in DOCKER-USER firewall
When so-postgres was wired in (868cd1187), the import role's firewall
defaults were missed while every other manager-class role (manager,
managerhype, managersearch, standalone, eval) had postgres added to
their DOCKER-USER manager-hostgroup portgroups. As a result, on a
fresh import install the so-postgres container starts but tcp/5432 is
dropped at DOCKER-USER, so soc/kratos/telegraf can't reach it.

Add postgres alongside the existing influxdb entry so import nodes
match the other roles.
2026-04-29 08:48:45 -04:00
Mike Reeves 2dcded6cca drop postgres module from soc defaults injection
The soc binary on 3/dev does not register a postgres module, so injecting
postgres into soc.config.server.modules makes soc abort at launch with
'Module does not exist: postgres'. The soc-side module is staged on
feature/postgres but is not landing this release. Drop the injection
until the module ships; salt/postgres state and pillars are unchanged.
2026-04-28 15:46:56 -04:00
Mike Reeves 8ca59e6f0c Merge pull request #15838 from Security-Onion-Solutions/fix/docker-refresh-multiarch-pull
Fix/docker refresh multiarch pull
2026-04-28 15:14:27 -04:00
Mike Reeves 82dac82d15 drop platform/digest pull resolution
The digest-pull logic was added to make `docker push` work for multi-arch
upstream tags. Now that the push step is `docker buildx imagetools create`
pinned to the gpg-verified RepoDigest, the registry-to-registry copy
handles single- and multi-arch sources without help. Reverts the pull
back to the original line and removes the unused PLATFORM_OS/_ARCH
detection.
2026-04-28 14:54:25 -04:00
Mike Reeves 288a823edf push images via buildx imagetools create
Replaces `docker push` with a registry-to-registry copy. On Docker 29.x
with the containerd image store, `docker push` of a freshly-pulled image
hits a path that wraps single-platform manifests in a synthetic index
and then can't push the layers it claims to reference, producing
`NotFound: content digest ...` even when the image is fully present.

Keep the local `docker tag` so so-image-pull's `docker images | grep :5000`
existence check continues to work.
2026-04-28 14:49:02 -04:00
reyesj2 9cec79b299 check current fleet policy cert against cert on disk
Co-authored-by: Copilot <copilot@github.com>
2026-04-28 13:34:39 -05:00
Mike Reeves c86399327b fix so-docker-refresh push for multi-arch source images
docker pull of a multi-arch tag on Docker 29.x leaves the local tag
pointing at the image index rather than the platform-specific manifest.
The subsequent docker push then tries to push every sub-manifest the
index references and fails on layers we never fetched.

Resolve the local-platform manifest digest from the upstream index via
docker buildx imagetools inspect, pull by that digest, and re-tag locally
to the canonical tag. The signing flow and the existing tag/push to the
embedded registry are unchanged.
2026-04-28 14:27:59 -04:00