setup runs so-minion -o=setup before /usr/sbin/so-common is installed, so
valid_ip4 was undefined, every MAINIP was rejected, and no minion pillar
was written. Pillar compile then failed for the new manager and setup gave
up waiting for the salt master. Use an inline IPv4 regex instead.
so-minion exported every line of the minion-controlled /opt/so/install.txt
into its root shell and wrote the values unescaped into a Jinja-rendered
pillar, allowing a rogue node to redefine PILLARFILE or run code on the
master at the next pillar compile. pcapspace also fed minion-returned
disk.usage output into bash arithmetic, which evaluates array subscripts.
- Parse install.txt against an allowlist of known keys; never export
- Validate MINION_ID before building pillar paths; make them readonly
- Validate node type, IP, interface, hostname, heap and core values
before any pillar is written; strip braces and control chars from
the free-text node description
- Require a numeric disk size before pcapspace arithmetic
- Refuse manager node types on add/addVM so a remote node cannot
rewrite the CA pillar; only setup may create them
The old flow had two writers for each per-minion Telegraf password
(so-minion wrote the minion pillar; postgres.auth regenerated any
missing aggregate entries). They drifted on first-boot and there was
no trigger to create DB roles when a new minion joined.
Split responsibilities:
- pillar/postgres/auth.sls (manager-scoped) keeps only the so_postgres
admin cred.
- pillar/telegraf/creds.sls (grid-wide) holds a {minion_id: {user,
pass}} map, shadowed per-install by the local-pillar copy.
- salt/manager/tools/sbin/so-telegraf-cred is the single writer:
flock, atomic YAML write, PyYAML safe_dump so passwords never
round-trip through so-yaml.py's type coercion. Idempotent add, quiet
remove.
- so-minion's add/remove hooks now shell out to so-telegraf-cred
instead of editing pillar files directly.
- postgres.telegraf_users iterates the new pillar key and CREATE/ALTERs
roles from it; telegraf.conf reads its own entry via grains.id.
- orch.deploy_newnode runs postgres.telegraf_users on the manager and
refreshes the new minion's pillar before the new node highstates,
so the DB role is in place the first time telegraf tries to connect.
- soup's post_to_3.1.0 backfills the creds pillar from accepted salt
keys (idempotent) and runs postgres.telegraf_users once to reconcile
the DB.
Paired with the add path in add_telegraf_to_minion: when a minion is
removed, drop its entry from the aggregate postgres pillar and drop the
matching so_telegraf_<safe> role from the database. Without this, stale
entries and DB roles accumulate over time.
Makes rotate-password and compromise-recovery both a clean delete+add:
so-minion -o=delete -m=<id>
so-minion -o=add -m=<id>
The first call drops the role and clears the aggregate pillar; the
second generates a brand-new password.
The cleanup is best-effort — if so-postgres isn't running or the DROP
ROLE fails (e.g., the role owns unexpected objects), we log a warning
and continue so the minion delete itself never gets blocked by postgres
state. Admins can mop up stray roles manually if that happens.
Simpler, race-free replacement for the reactor + orch + fan-out chain.
- salt/manager/tools/sbin/so-minion: expand add_telegraf_to_minion to
generate a random 72-char password, reuse any existing password from
the aggregate pillar, write postgres.telegraf.{user,pass} into the
minion's own pillar file, and update the aggregate pillar so
postgres.telegraf_users can CREATE ROLE on the next manager apply.
Every create<ROLE> function already calls this hook, so add / addVM /
setup dispatches are all covered identically and synchronously.
- salt/postgres/auth.sls: strip the fanout_targets loop and the
postgres_telegraf_minion_pillar_<safe> cmd.run block — it's now
redundant. The state still manages the so_postgres admin user and
writes the aggregate pillar for postgres.telegraf_users to consume.
- salt/reactor/telegraf_user_sync.sls: deleted.
- salt/orch/telegraf_postgres_sync.sls: deleted.
- salt/salt/master.sls: drop the reactor_config_telegraf block that
registered the reactor on /etc/salt/master.d/reactor_telegraf.conf.
- salt/orch/deploy_newnode.sls: drop the manager_fanout_postgres_telegraf
step and the require: it added to the newnode highstate. Back to its
original 3/dev shape.
No more ephemeral postgres_fanout_minion pillar, no more async salt/key
reactor, no more so-minion setupMinionFiles race: the pillar write
happens inline inside setupMinionFiles itself.
Update so-minion to include running kafka.nodes state to ensure nodeid is generated for new brokers
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
Josh Patterson
reyesj2
Mike Reeves
Jason Ertel
Mike Reeves
DefensiveDepth