Supersedes the pre-install placement (right after secrets_pillar) from
the previous commit, which was broken: salt's ext_pillar overlay
shadowed disk pillar's elasticsearch subtree before so-pillar-import
had populated PG, so elasticsearch.enabled.sls failed rendering on
ELASTICSEARCHMERGED.auth.users.so_elastic_user.pass — that key lives
in elasticsearch/auth.sls, which is on the importer's secrets
allowlist and never makes it into so_pillar.pillar_entry. The install
would then hang forever waiting for the elasticsearch container that
the broken state never deployed.
The new placement is right after the final state.highstate completes:
1. drop adv_postgres.sls flipping the flag to True
2. salt-call saltutil.refresh_pillar so the next state sees it
3. salt-call state.apply postgres.schema_pillar — deploys schema,
ALTERs role login passwords, installs psycopg2 into salt's
bundled python, runs so-pillar-import, writes
/opt/so/conf/so-yaml/mode=postgres
4. salt-call state.apply salt.master — re-renders engines.conf
with the pg_notify_pillar engine block, drops master.d
ext_pillar config, watch_in restarts salt-master and ext_pillar
takes over
verify_setup runs after this so its final checks see PG-canonical
mode in place. Same end state as the previous commit's intent, just
without the bootstrap chicken-and-egg.
Drops a local pillar override (postgres.so_pillar.enabled = True) right
after secrets_pillar so the install-time highstate brings up
schema_pillar, ext_pillar_postgres, and the pg_notify_pillar engine
without operator intervention. Without this the whole PG-canonical
stack stays gated off on the default-False flag and the install lands
in legacy disk-pillar mode — which defeats the point of being on the
postsalt branch at all.
The new enable_so_pillar_postgres() function in so-functions is
idempotent (overwrites adv_postgres.sls with a fixed body) and the
generated file is mode 0644 socore:socore so it merges into pillar
under the existing local-pillar directory ownership convention.
Rollback path: edit /opt/so/saltstack/local/pillar/postgres/adv_postgres.sls
to set enabled: False, or delete the file. The schema and engine
config states will tear themselves down on the next highstate via
their existing else-branch absent states.
The unscoped `umask 077` on postsalt's secrets_pillar path leaked into
every subsequent file write by so-setup (and the salt-call processes
it spawned) for the rest of the install. Every state-rendered config
file under /opt/so/conf landed at mode 0600 instead of 0644, which
broke any container that bind-mounts its config read-only and runs as
a non-root user after the entrypoint's gosu drop. The first concrete
casualty was the influxdb container, which exits with
"failed to load config file: open /conf/config.yaml: permission denied"
after init mode completes and re-execs as the influxdb user.
The chmod 0400 immediately after the printf already enforces the
intended file mode, so the umask was redundant for the key file
itself; scoping it to a subshell preserves the defense-in-depth
between the printf and the chmod without polluting the parent shell.
Hooks every so-yaml.py write through a new so_yaml_postgres helper that
mirrors disk YAML mutations into so_pillar.pillar_entry via docker exec
psql. Disk remains canonical during the transition; PG mirror failures
are logged only when a real write error occurs (skipped paths and
postgres-unreachable cases stay silent so existing callers don't see
new noise on stderr).
Adds a `purge YAML_FILE` verb on so-yaml that deletes the file from
disk and removes the matching pillar_entry rows. For minion files it
also drops the so_pillar.minion row, which CASCADEs to pillar_entry +
role_member. Designed for so-minion's delete path (replaces rm -f) so
the audit log captures the deletion.
setup/so-functions::generate_passwords + secrets_pillar generate
secrets:pillar_master_pass and /opt/so/conf/postgres/so_pillar.key on
fresh installs, and append the password to existing secrets.sls files
on upgrade.
- salt/manager/tools/sbin/so_yaml_postgres.py: locate(), write_yaml(),
purge_yaml(), and a small CLI for diagnostics. Skips bootstrap and
mine-driven paths via the same allowlist used by so-pillar-import.
- salt/manager/tools/sbin/so-yaml.py: import the helper, hook
writeYaml() to mirror after every disk write, add purgeFile() and
the purge verb.
- salt/manager/tools/sbin/so-yaml_test.py: 16 new tests covering the
purge verb and the path-locator / write contract of so_yaml_postgres
without contacting Postgres. All 91 tests pass.
- setup/so-functions: generate_passwords adds PILLARMASTERPASS and
SO_PILLAR_KEY; secrets_pillar writes pillar_master_pass and the
pgcrypto master key file.
- Telegraf's partman template passed p_type:='native', which pg_partman
5.x (the version shipped by postgresql-17-partman on Debian) rejects.
Switched to 'range' so partman.create_parent() actually creates
partitions and Telegraf's INSERTs succeed.
- Added a postgres_wait_ready gate in telegraf_users.sls so psql execs
don't race the init-time restart that docker-entrypoint.sh performs.
- so-verify now ignores the literal "-v ON_ERROR_STOP=1" token in the
setup log. Dropped the matching entry from so-log-check, which scans
container stdout where that token never appears.
Phase 1 of the PostgreSQL central data platform:
- Salt states: init, enabled, disabled, config, ssl, auth, sostatus
- TLS via SO CA-signed certs with postgresql.conf template
- Two-tier auth: postgres superuser + so_postgres application user
- Firewall restricts port 5432 to manager-only (HA-ready)
- Wired into top.sls, pillar/top.sls, allowed_states, firewall
containers map, docker defaults, CA signing policies, and setup
scripts for all manager-type roles
Security Onion now exclusively supports Oracle Linux 9. This removes
detection, setup, and update logic for Ubuntu, Debian, CentOS, Rocky,
AlmaLinux, and RHEL.
Mike Reeves
Mike Reeves
Josh Patterson
Jason Ertel
Jason Ertel
Josh Brower
reyesj2