Compare commits

..

2 Commits

Author SHA1 Message Date
Mike Reeves 81c0f2b464 so-yaml.py: tolerate missing ancestors in removeKey
replace calls removeKey before addKey, so running `so-yaml.py replace`
on a new dotted key whose parent doesn't exist — e.g., postgres.auth
fanning postgres.telegraf.user into a minion pillar file that has
never carried any postgres.* keys — crashed with
    KeyError: 'postgres'
from removeKey recursing into a missing parent dict.

Make removeKey a no-op when an intermediate key is absent so that:
  - `remove` has the natural "remove if exists" semantics, and
  - `replace` works for brand-new nested keys.
2026-04-21 14:43:10 -04:00
Mike Reeves d5dc28e526 Fan postgres telegraf cred for manager on every auth run
The empty-pillar case produced a telegraf.conf with `user= password=`
which libpq misparses ("password=" gets consumed as the user value),
yielding `password authentication failed for user "password="` on
every manager without a prior fan-out (fresh install, not the salt-key
path the reactor handles).

Two fixes:

- salt/postgres/auth.sls: always fan for grains.id in addition to any
  postgres_fanout_minion from the reactor, so the manager's own pillar
  is populated on every postgres.auth run. The existing `unless` guard
  keeps re-runs idempotent.
- salt/telegraf/etc/telegraf.conf: gate the [[outputs.postgresql]]
  block on PG_USER and PG_PASS being non-empty. If a minion hasn't
  received its pillar yet the output block simply isn't rendered — the
  next highstate picks up the creds once the fan-out completes, and in
  the meantime telegraf keeps running the other outputs instead of
  erroring with a malformed connection string.
2026-04-21 14:40:19 -04:00
3 changed files with 26 additions and 12 deletions
+2 -1
View File
@@ -285,7 +285,8 @@ def add(args):
def removeKey(content, key):
pieces = key.split(".", 1)
if len(pieces) > 1:
removeKey(content[pieces[0]], pieces[1])
if pieces[0] in content:
removeKey(content[pieces[0]], pieces[1])
else:
content.pop(key, None)
+23 -10
View File
@@ -50,14 +50,27 @@ postgres_auth_pillar:
{% endfor %}
- show_changes: False
{# Fan a specific minion's telegraf cred out to its own pillar file. Only
runs when postgres_fanout_minion pillar is provided — otherwise this state
is a no-op. That keeps manager highstates from doing N so-yaml.py forks
when nothing changed. The reactor passes postgres_fanout_minion through
the orch on salt-key accept; soup handles bulk backfill separately. #}
{# Fan a specific minion's telegraf cred out to its own pillar file.
Two triggers populate the target list:
- grains.id (always) so the manager's own pillar is populated on every
postgres.auth run — otherwise the manager's telegraf has no cred on
a fresh install and can't write to its own postgres.
- pillar postgres_fanout_minion (when the reactor fires on a new
minion's salt-key accept).
The `unless` guard keeps re-runs idempotent, so this is one so-yaml.py
check per target, not per minion in the grid. Bulk backfill for
already-accepted minions lives in soup. #}
{% set fanout_targets = [] %}
{% if grains.id %}
{%- do fanout_targets.append(grains.id) %}
{% endif %}
{% set fanout_mid = salt['pillar.get']('postgres_fanout_minion') %}
{% if fanout_mid %}
{%- set safe = fanout_mid | replace('.','_') | replace('-','_') | lower %}
{% if fanout_mid and fanout_mid not in fanout_targets %}
{%- do fanout_targets.append(fanout_mid) %}
{% endif %}
{% for mid in fanout_targets %}
{%- set safe = mid | replace('.','_') | replace('-','_') | lower %}
{%- set key = 'telegraf_' ~ safe %}
{%- set entry = telegraf_users.get(key) %}
{%- if entry %}
@@ -66,7 +79,7 @@ postgres_telegraf_minion_pillar_{{ safe }}:
cmd.run:
- name: |
set -e
PILLAR_FILE=/opt/so/saltstack/local/pillar/minions/{{ fanout_mid }}.sls
PILLAR_FILE=/opt/so/saltstack/local/pillar/minions/{{ mid }}.sls
if [ ! -f "$PILLAR_FILE" ]; then
echo '{}' > "$PILLAR_FILE"
chown socore:socore "$PILLAR_FILE" 2>/dev/null || true
@@ -75,12 +88,12 @@ postgres_telegraf_minion_pillar_{{ safe }}:
/usr/sbin/so-yaml.py replace "$PILLAR_FILE" postgres.telegraf.user '{{ entry.user }}'
/usr/sbin/so-yaml.py replace "$PILLAR_FILE" postgres.telegraf.pass '{{ entry.pass }}'
- unless: |
[ "$(/usr/sbin/so-yaml.py get -r /opt/so/saltstack/local/pillar/minions/{{ fanout_mid }}.sls postgres.telegraf.user 2>/dev/null)" = '{{ entry.user }}' ]
[ "$(/usr/sbin/so-yaml.py get -r /opt/so/saltstack/local/pillar/minions/{{ mid }}.sls postgres.telegraf.user 2>/dev/null)" = '{{ entry.user }}' ]
- require:
- file: postgres_auth_pillar
{%- endif %}
{% endif %}
{% endfor %}
{% else %}
{{sls}}_state_not_allowed:
+1 -1
View File
@@ -96,7 +96,7 @@
# insecure_skip_verify = false
{%- endif %}
{%- if TG_OUT in ['POSTGRES', 'BOTH'] %}
{%- if TG_OUT in ['POSTGRES', 'BOTH'] and PG_USER and PG_PASS %}
# Configuration for sending metrics to PostgreSQL.
# options='-c role=so_telegraf' makes every connection SET ROLE to the shared
# group role so tables created on first write are owned by so_telegraf, and