Commit Graph
552 Commits
Author SHA1 Message Date
Mike Reeves 11c8fe3093 Use the host CA trust store for all grid containers
Anchor the grid CA and blocklist the eMudhra roots in the host trust
store (new ca.trust state, also used by the desktop role), then mount the
extracted store into each container instead of the frozen copies made by
so-catrust. Containers restart when the host trust changes.
2026-10-27 15:08:21 -04:00
Jorge Reyes fc5e66e913 remap enterprisesearch integration components 2026-10-13 09:10:36 -05:00
Josh Brower b32aaac290 Merge pull request #16278 from Security-Onion-Solutions/evtx-import-datastreams
EVTX Import cleanup
2026-10-02 13:30:57 -04:00
defensivedepth 43475452b3 set module 2026-10-01 19:17:52 -04:00
defensivedepth 89f8bcd19f evtx-import fixup 2026-10-01 10:28:10 -04:00
Jorge Reyes 0f53a7e0bc Merge pull request #16275 from Security-Onion-Solutions/reyesj2-521
review integration-defaults weird_integrations mappings, removed unus…
2026-10-01 08:36:57 -05:00
reyesj2 d122ee7fea review integration-defaults weird_integrations mappings, removed unused, updated logstash integration naming 2026-09-30 16:49:19 -05:00
Josh Patterson 21222ff119 FIX: use /sbin/nologin for service accounts
These accounts existed only for container UID mapping and filesystem
ownership, but user.present omitted shell:, so Salt fell through to the
platform useradd default and every one of them got /bin/bash. Pin them to
/sbin/nologin so none can be used as an interactive login or `su -` target.

socore keeps /bin/bash: `su socore -c '/usr/sbin/so-repo-sync'` in soup and
so-kernel-upgrade execs the account's passwd shell, and operator docs tell
users to su to socore. soqemussh keeps /bin/bash as an SSH login account.

elastic-agent, elastic-agent-pr and kafka are included alongside the accounts
named in the issue, being the same class with the same unset shell, so the
default is uniform.

Cron is unaffected: cronie runs jobs via the crontab SHELL (default /bin/sh),
not the passwd shell. suricata is the only account changed here that owns a
crontab, and somon has shipped as nologin with a working cron job already.
The zeek `runuser -l zeek` calls all run inside so-zeek via docker.run/exec,
so they resolve the shell from the image, not the host.

Verified on a 3.4.0 managersearch + sensor grid: highstate converges with the
shell as the only change and no failures, is idempotent on a second run, all
containers stay up, SOC still issues a Kratos login flow, and the suricata
surilogcompress cron job runs post-change ((suricata) CMD/CMDEND in
/var/log/cron) while `su - suricata` is now refused.

Closes #16256
2026-09-25 09:23:09 -04:00
reyesj2 bcee63bde5 remove policy precheck 2026-09-23 08:51:11 -05:00
reyesj2 6ce89eb323 jq -e exits 0 with empty input 2026-09-22 21:25:50 -05:00
reyesj2 ebab4b0d90 split between missing token and multiple enrollment tokens 2026-09-22 16:11:56 -05:00
reyesj2 db60c27da2 fail cleanly when endpoints-initial is missing or has multiple enrollment tokens 2026-09-22 12:24:23 -05:00
Josh Patterson 1e86be11b2 FIX: install /usr/sbin management scripts as root
Salt installed the so-* scripts into /usr/sbin owned by unprivileged service
UIDs (939/socore, plus 930-960 per service) at mode 755, while root executes
those same files from cron, systemd and state cmd.run. Any file-write
primitive as one of those UIDs was therefore root.

Two mechanisms behind this are not visible in the diff:

file.recurse also manages the destination directory, so /usr/sbin itself was
chowned to whichever service UID ran last. A directory's owner may always
chmod it, so that UID could replace even the scripts already declared
user: root -- so-config-backup, so-suricata-eve-clean, so-nsm-mount-nvme.
usr_sbin_perms now pins the directory to root:root 555, the mode the
filesystem RPM ships.

Omitting user:/group: is a no-op on files that already exist, because
check_perms only chowns when a user is named. Explicit user: root is what
lets upgraded grids self-heal on the next highstate, and what makes a
revert chown back rather than silently do nothing.
2026-09-16 10:34:20 -04:00
reyesj2 3d8f86883a after an ES upgrade run a final elasticsearch state to create/regenerate any needed addon index templates 2026-08-28 12:16:08 -05:00
reyesj2 d771fbc444 upgrade integration policies directly after integration package upgrade 2026-08-27 14:18:39 -05:00
reyesj2 fae1754fec clean elasticsearch transform prior to elasticsearch integration package upgrade to prevent fleet automatic rollback 2026-08-27 12:50:32 -05:00
reyesj2 088b761190 9.4.5 policy updates 2026-08-20 16:35:46 -05:00
reyesj2 c9a041ddb4 upstream sentinel_one_cloud_funnel integration patched data stream name
ref: https://github.com/elastic/integrations/commit/9f1513423ca26e6fded414c4f5a3a89409efb736
2026-08-17 22:08:19 -05:00
Jorge Reyes baca444a7e Revert "elastic fleet server persistence" 2026-07-24 13:18:48 -05:00
reyesj2 f542d1e7ce include current version in so-elastic-fleet-package-upgrade attempt 2026-07-20 14:34:03 -05:00
Josh Patterson f958212bea Merge remote-tracking branch 'origin/3/dev' into saltthangs 2026-07-15 15:07:09 -04:00
Jorge Reyes 8167ae3282 Merge pull request #16062 from Security-Onion-Solutions/reyesj2/es937
ES 9.3.7
2026-07-13 14:35:35 -05:00
reyesj2 87a5639643 pipeline updates 2026-07-13 13:05:25 -05:00
reyesj2 99e9fc1c3b ES 9.3.7 2026-07-10 13:50:55 -05:00
Jorge Reyes e5de499bcc Merge pull request #16059 from Security-Onion-Solutions/reyesj2-patch-4 2026-07-10 13:02:18 -05:00
reyesj2 7d17784e96 status messages for so-elastic-fleet-setup 2026-07-10 12:50:21 -05:00
reyesj2 f6a2758321 add so-elastic-agent-install script to minions 2026-07-09 14:44:42 -05:00
reyesj2 0b078c4804 give the elasticfleet state a few chances to complete successfully before exiting 1 causing so-setup to fail 2026-07-09 14:23:45 -05:00
reyesj2 2959dc9564 keep logs for all 3 attempts 2026-07-09 14:18:08 -05:00
reyesj2 8b0759866e verify installers are generated 2026-07-09 14:15:22 -05:00
reyesj2 6fa0d327cb cause so-setup to fail if there are issues setting up fleet 2026-07-09 14:14:58 -05:00
reyesj2 3394e9aab7 increase agent health timeout and add logging for elastic agent status output 2026-07-09 11:29:43 -05:00
Jorge Reyes 3766f74102 Merge pull request #16052 from Security-Onion-Solutions/reyesj2-patch-2
elastic fleet server persistence
2026-07-08 16:40:45 -05:00
reyesj2 c04a30785f make elastic agent state persistent to prevent re-enrollment during soup / reboots 2026-07-08 16:15:02 -05:00
reyesj2 85d7f6bebc independently download so-elastic-agent installer 2026-07-08 08:19:15 -05:00
reyesj2 2a4a7307f7 uninstall agent after downloading new one and getting gridtoken 2026-07-08 07:51:13 -05:00
reyesj2 f8de176f4b uninstall agent on final failed attempt 2026-07-08 07:47:47 -05:00
reyesj2 70af3cec53 avoid using 'failure' until all loops are done so so-verify doesn't flag it 2026-07-07 16:48:10 -05:00
reyesj2 57b7d59387 verify elastic-agent reports healthy status before completing installation 2026-07-07 16:42:47 -05:00
Josh Patterson 61aa963a2d Merge remote-tracking branch 'origin/3/dev' into saltthangs 2026-06-24 08:10:27 -04:00
reyesj2 4f3b57f495 remove duplicate package-upgrade attempts, upgrade only when reported latest version differs from installed version 2026-06-23 15:52:10 -05:00
reyesj2 84228a819b remove heayvnode FleetServer_* directory creation, and skip empty directories during FleetServer policy management 2026-06-23 15:30:49 -05:00
Josh Patterson d71e80cf66 Merge remote-tracking branch 'origin/3/dev' into saltthangs 2026-06-23 10:32:32 -04:00
reyesj2 96fcc0ec38 wip 2026-06-22 14:25:46 -05:00
Josh Patterson d0bea2ebcb Restore grouped per-integration logging and retry 409s in fleet integration loader
elastic_fleet_load_integrations_dir now buffers each concurrent job's
output (header + API response) to a per-job file and prints them in
submission order after wait, restoring the readable serial-style output
while keeping concurrent writes.

Add --retry-all-errors to the integration create/update curl calls so
transient 409 conflicts from concurrent writes to the same agent policy
are retried (curl --retry alone does not retry 409).
2026-06-18 11:19:36 -04:00
Josh Patterson 8e33d0e1e9 Merge remote-tracking branch 'origin/3/dev' into soupmod2 2026-06-16 12:54:18 -04:00
Jorge Reyes b81257bf45 Merge pull request #15973 from Security-Onion-Solutions/reyesj2/dlm-support
Data stream lifecycle management support
2026-06-15 14:47:51 -05:00
Josh Patterson 1ee555957a Speed up so-elastic-fleet-integration-upgrade
Fetch each agent policy once and extract integration name/package/version/id
locally via a single jq pass instead of re-fetching the identical policy JSON
1+3N times. Memoize epm/packages latest-version lookups so each package is
queried once instead of per (policy, integration). Dispatch the per-integration
dry-run+upgrade as throttled background jobs (MAX_FLEET_JOBS) with
flock-serialized output and a FAIL_FILE marker, mirroring
elastic_fleet_load_integrations_dir.

Behavior preserved: same elastic-defend-endpoints/fleet_server skips, same
AUTO_UPGRADE_INTEGRATIONS default-package gating (moved into jq, using $defaults
to avoid the jq $def keyword collision), and exit 1 on any failure so salt
retries.
2026-06-12 15:23:43 -04:00
Josh Patterson ae6a705ce1 Speed up so-elastic-fleet-integration-policy-load
Fetch each agent policy once per group instead of refetching the full
policy (plus a fresh Kibana session cookie) for every integration file,
and dispatch the create/update writes as throttled background jobs.

Adds elastic_fleet_load_integrations_dir and elastic_fleet_throttle to
so-elastic-fleet-common, reusing the bounded-concurrency pattern from
so-elasticsearch-ilm-policy-load. Replaces the four serial loops in the
loader with one call per agent policy.
2026-06-12 09:38:41 -04:00
Josh Patterson b1273573ed Fix jq $def keyword collision in optional-integrations-load
The agent-policy enumeration passed --argjson def, creating a jq
variable $def. 'def' is a reserved keyword in jq and the deployed jq
version rejects it, so the program failed to compile and
in_use_integrations was left empty (silently disabling the in-use
upgrade guard). Rename the arg to $defaults.
2026-06-11 15:50:53 -04:00