Commit Graph

30 Commits

Author SHA1 Message Date
Mike Reeves 05f6503d61 Gate postgres telegraf fan-out on reactor-provided minion id
postgres.auth was running an `unless` shell check per up-minion on every
manager highstate, even when nothing had changed — N fork+python starts
of so-yaml.py add up on large grids. The work is only needed when a
specific minion's key is accepted.

- salt/postgres/auth.sls: fan out only when postgres_fanout_minion
  pillar is set (targets that single minion). Manager highstates with
  no pillar take a zero-N code path.
- salt/reactor/telegraf_user_sync.sls: re-pass the accepted minion id
  as postgres_fanout_minion to the orch.
- salt/orch/telegraf_postgres_sync.sls: forward the pillar to the
  salt.state invocation so the state render sees it.
- salt/manager/tools/sbin/soup: for the one-time 3.1.0 backfill, drop
  the per-minion state.apply and do an in-shell loop over the minion
  pillar files using so-yaml.py directly. Skips minions that already
  have postgres.telegraf.user set.
2026-04-21 10:05:08 -04:00
Mike Reeves a902f667ba Target manager by role grain in telegraf_postgres_sync orch
The previous MANAGER resolution used pillar.get('setup:manager') with a
fallback to grains.get('master'). Neither works from the reactor:
setup:manager is only populated by the setup workflow (not by reactor
runs), and grains.master returns the minion's master-hostname setting,
not a targetable minion id.

Match the pattern used by orch/delete_hypervisor.sls: compound-target
whichever minion is the manager via role grain.
2026-04-21 09:37:35 -04:00
Mike Reeves 72105f1f2f Drop telegraf push from new-minion orch; highstate covers it
New minions run highstate as part of onboarding, which already applies
the telegraf state with the fresh pillar entry we just wrote. Pushing
telegraf a second time from the reactor is redundant.

- Remove the MINION-scoped salt.state block from the orch; keep only
  the manager-side postgres.auth + postgres.telegraf_users provisioning.
- Stop passing minion_id as pillar in the reactor; the orch doesn't
  reference it anymore.
2026-04-21 09:31:45 -04:00
Mike Reeves cefbe01333 Add telegraf_output selector for InfluxDB/Postgres dual-write
Introduces global.telegraf_output (INFLUXDB|POSTGRES|BOTH, default BOTH)
so Telegraf can write metrics to Postgres alongside or instead of
InfluxDB. Each minion authenticates with its own so_telegraf_<minion>
role and writes to a matching schema inside a shared so_telegraf
database, keeping blast radius per-credential to that minion's data.

- Per-minion credentials auto-generated and persisted in postgres/auth.sls
- postgres/telegraf_users.sls reconciles roles/schemas on every apply
- Firewall opens 5432 only to minion hostgroups when Postgres output is active
- Reactor on salt/auth + orch/telegraf_postgres_sync.sls provision new
  minions automatically on key accept
- soup post_to_3.1.0 backfills users for existing minions on upgrade
- so-show-stats prints latest CPU/mem/disk/load per minion for sanity checks
- so-telegraf-trim + nightly cron prune rows older than
  postgres.telegraf.retention_days (default 14)
2026-04-15 14:32:10 -04:00
Josh Patterson 453c32df0d handle - in hypervisor hostname 2025-08-04 15:25:26 -04:00
Josh Patterson 6d7066c381 add license 2025-07-02 16:20:30 -04:00
Josh Patterson d003e1380f ensure hypervisor is remove from salt cloud profiles when key is deleted 2025-07-02 16:14:43 -04:00
Josh Patterson f6a0e62853 include managerhype in orch. run hypervisor state before libvirt states 2025-04-08 09:50:26 -04:00
Josh Patterson 445afca6ee use vrt 2025-04-03 13:44:13 -04:00
Josh Patterson ae993c47c1 remove minion pillar files when a vm is destroyed 2025-03-11 11:12:45 -04:00
Josh Patterson c784a6e440 fix setting hypervisor for our custom event tag 2025-03-10 16:55:02 -04:00
Josh Patterson f30938ed59 hypervisor annotation show if base domain is initialized or not 2025-03-06 15:26:08 -05:00
Josh Patterson 2c5861a0c2 ensure local hypervisor dir when new hypervisor key accepted. apply soc.dyanno.hypervisor when hypervisor key accepted 2025-03-05 08:51:10 -05:00
Josh Patterson c896785480 fix vm deletion 2025-02-24 14:20:09 -05:00
Josh Patterson 0006948c29 get hypervisor from dir name 2025-02-24 12:26:28 -05:00
Josh Patterson fd9a4966ec move logic from reactor to orchestration 2025-02-23 14:07:51 -05:00
Josh Patterson 3246176c0a comments 2025-02-21 14:34:08 -05:00
Josh Patterson b68f561e6f progress and hw tracking for soc hypervisor dynamic annotations 2025-02-21 09:50:01 -05:00
m0duspwnens 2e3c1adc63 runner to setup manager for first hypervisor 2025-01-14 16:20:21 -05:00
m0duspwnens 776afa4a36 setup items on manager when hypervisor joins the grid 2025-01-09 16:32:41 -05:00
m0duspwnens 1862deaf5e add copyright 2024-05-08 10:14:08 -04:00
m0duspwnens 0d2e5e0065 need repo and docker first 2024-05-08 09:50:01 -04:00
m0duspwnens dcc1f656ee predownload logstash and elastic for new searchnode and heavynode 2024-05-07 10:13:51 -04:00
m0duspwnens bdf1b45a07 redirect and throw in bg 2024-05-03 14:54:44 -04:00
m0duspwnens 3d4fd59a15 orchit 2024-05-03 13:48:51 -04:00
m0duspwnens 442a717d75 orchit 2024-05-03 12:08:57 -04:00
m0duspwnens fa3522a233 fix requirement 2024-05-03 11:10:21 -04:00
m0duspwnens bbc374b56e add logic in orch 2024-05-03 09:56:52 -04:00
m0duspwnens 2929877042 fix var 2024-05-02 16:37:54 -04:00
m0duspwnens e9b1263249 orchestate searchnode deployment 2024-05-02 16:32:43 -04:00