Postgres: pg_cron over the local socket, so_maint_telegraf role, repair and soup 3.5.0

Merge pull request #16389 from Security-Onion-Solutions/reyesj2-patch-1
update elastic-agent artifact repo directory
2026-11-15 00:44:09 +01:00 · 2026-11-12 19:07:57 -05:00 · 2026-11-11 14:32:59 -06:00 · 2026-11-10 14:53:30 -05:00 · 2026-11-10 08:26:19 -05:00 · 2026-11-09 12:32:06 -06:00
250 changed files with 2096 additions and 558 deletions
@@ -14,6 +14,7 @@ body:
        - 3.2.0
        - 3.3.0
        - 3.4.0
+        - 3.5.0
        - Other (please provide detail below)
    validations:
      required: true
@@ -6,6 +6,7 @@ on:
      - "salt/sensoroni/files/analyzers/**"
      - "salt/manager/tools/sbin/**"
      - "salt/_beacons/**"
+      - "salt/elastalert/files/modules/**"
      - "salt/telegraf/tools/sbin_jinja/**"
      - "salt/telegraf/defaults.yaml"
      - "salt/telegraf/soc_telegraf.yaml"
@@ -18,7 +19,7 @@ jobs:
      fail-fast: false
      matrix:
        python-version: ["3.14"]
-        python-code-path: ["salt/sensoroni/files/analyzers", "salt/manager/tools/sbin", "salt/_beacons"]
+        python-code-path: ["salt/sensoroni/files/analyzers", "salt/manager/tools/sbin", "salt/_beacons", "salt/elastalert/files/modules/so"]

    steps:
    - uses: actions/checkout@v3
@@ -1,17 +1,17 @@
-### 3.3.0-20260911 ISO image released on 2026/09/11
+### 3.4.0-20261102 ISO image released on 2026/11/02


 ### Download and Verify

-3.3.0-20260911 ISO image:  
-https://download.securityonion.net/file/securityonion/securityonion-3.3.0-20260911.iso
+3.4.0-20261102 ISO image:  
+https://download.securityonion.net/file/securityonion/securityonion-3.4.0-20261102.iso
 
-MD5: 12B18433D3A2198A185892FF79CF638F  
-SHA1: 2B3C2E1FA7A78ED1F956E7EDCC12E32593C14EEE  
-SHA256: 0938C73B76CE30EC9E4394D312C79EA7CAC721B6818541697279A6221F7D870D  
+MD5: FB0380AE996E175D6CC7E7A35D8CC178  
+SHA1: D8960F1DA352710BCA209773BF0B3AE8791D0D1B  
+SHA256: DFC4C87668836852A328EC7A0B755C8C1511FF0E43D423DEE23047A95F676D99  

 Signature for ISO image:  
-https://github.com/Security-Onion-Solutions/securityonion/raw/3/main/sigs/securityonion-3.3.0-20260911.iso.sig
+https://github.com/Security-Onion-Solutions/securityonion/raw/3/main/sigs/securityonion-3.4.0-20261102.iso.sig

 Signing key:  
 https://raw.githubusercontent.com/Security-Onion-Solutions/securityonion/3/main/KEYS  
@@ -25,22 +25,22 @@ wget https://raw.githubusercontent.com/Security-Onion-Solutions/securityonion/3/

 Download the signature file for the ISO:  
 ```
-wget https://github.com/Security-Onion-Solutions/securityonion/raw/3/main/sigs/securityonion-3.3.0-20260911.iso.sig
+wget https://github.com/Security-Onion-Solutions/securityonion/raw/3/main/sigs/securityonion-3.4.0-20261102.iso.sig
 ```

 Download the ISO image:  
 ```
-wget https://download.securityonion.net/file/securityonion/securityonion-3.3.0-20260911.iso
+wget https://download.securityonion.net/file/securityonion/securityonion-3.4.0-20261102.iso
 ```

 Verify the downloaded ISO image using the signature file:  
 ```
-gpg --verify securityonion-3.3.0-20260911.iso.sig securityonion-3.3.0-20260911.iso
+gpg --verify securityonion-3.4.0-20261102.iso.sig securityonion-3.4.0-20261102.iso
 ```

 The output should show "Good signature" and the Primary key fingerprint should match what's shown below:
 ```
-gpg: Signature made Fri 11 Sep 2026 11:23:56 AM EDT using RSA key ID FE507013
+gpg: Signature made Sat 31 Oct 2026 09:27:50 AM EDT using RSA key ID FE507013
 gpg: Good signature from "Security Onion Solutions, LLC <info@securityonionsolutions.com>"
 gpg: WARNING: This key is not certified with a trusted signature!
 gpg:          There is no indication that the signature belongs to the owner.
@@ -1 +1 @@
-3.4.0
+3.5.0
@@ -0,0 +1,14 @@
+-----BEGIN CERTIFICATE-----
+MIICKzCCAbGgAwIBAgIKe3G2gla4EnycqDAKBggqhkjOPQQDAzBaMQswCQYDVQQG
+EwJVUzETMBEGA1UECxMKZW1TaWduIFBLSTEUMBIGA1UEChMLZU11ZGhyYSBJbmMx
+IDAeBgNVBAMTF2VtU2lnbiBFQ0MgUm9vdCBDQSAtIEMzMB4XDTE4MDIxODE4MzAw
+MFoXDTQzMDIxODE4MzAwMFowWjELMAkGA1UEBhMCVVMxEzARBgNVBAsTCmVtU2ln
+biBQS0kxFDASBgNVBAoTC2VNdWRocmEgSW5jMSAwHgYDVQQDExdlbVNpZ24gRUND
+IFJvb3QgQ0EgLSBDMzB2MBAGByqGSM49AgEGBSuBBAAiA2IABP2lYa57JhAd6bci
+MK4G9IGzsUJxlTm801Ljr6/58pc1kjZGDoeVjbk5Wum739D+yAdBPLtVb4Ojavti
+sIGJAnB9SMVK4+kiVCJNk7tCDK93nCOmfddhEc5lx/h//vXyqaNCMEAwHQYDVR0O
+BBYEFPtaSNCAIEDyqOkAB2kZd6fmw/TPMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMB
+Af8EBTADAQH/MAoGCCqGSM49BAMDA2gAMGUCMQC02C8Cif22TGK6Q04ThHK1rt0c
+3ta13FaPWEBaLd4gTCKDypOofu4SQMfWh0/434UCMBwUZOR8loMRnLDRWmFLpg9J
+0wD8ofzkpf9/rdcw0Md3f76BB1UwUCAU9Vc4CqgxUQ==
+-----END CERTIFICATE-----
@@ -0,0 +1,15 @@
+-----BEGIN CERTIFICATE-----
+MIICTjCCAdOgAwIBAgIKPPYHqWhwDtqLhDAKBggqhkjOPQQDAzBrMQswCQYDVQQG
+EwJJTjETMBEGA1UECxMKZW1TaWduIFBLSTElMCMGA1UEChMcZU11ZGhyYSBUZWNo
+bm9sb2dpZXMgTGltaXRlZDEgMB4GA1UEAxMXZW1TaWduIEVDQyBSb290IENBIC0g
+RzMwHhcNMTgwMjE4MTgzMDAwWhcNNDMwMjE4MTgzMDAwWjBrMQswCQYDVQQGEwJJ
+TjETMBEGA1UECxMKZW1TaWduIFBLSTElMCMGA1UEChMcZU11ZGhyYSBUZWNobm9s
+b2dpZXMgTGltaXRlZDEgMB4GA1UEAxMXZW1TaWduIEVDQyBSb290IENBIC0gRzMw
+djAQBgcqhkjOPQIBBgUrgQQAIgNiAAQjpQy4LRL1KPOxst3iAhKAnjlfSU2fySU0
+WXTsuwYc58Byr+iuL+FBVIcUqEqy6HyC5ltqtdyzdc6LBtCGI79G1Y4PPwT01xyS
+fvalY8L1X44uT6EYGQIrMgqCZH0Wk9GjQjBAMB0GA1UdDgQWBBR8XQKEE9TMipuB
+zhccLikenEhjQjAOBgNVHQ8BAf8EBAMCAQYwDwYDVR0TAQH/BAUwAwEB/zAKBggq
+hkjOPQQDAwNpADBmAjEAvvNhzwIQHWSVB7gYboiFBS+DCBeQyh+KTOgNG3qxrdWB
+CUfvO6wIBHxcmbHtRwfSAjEAnbpV/KlK6O3t5nYBQnvI+GDZjVGLVTv7jHvrZQnD
+JbNR6iC8hZVdyR+EhCVBCyj
+-----END CERTIFICATE-----
@@ -0,0 +1,21 @@
+-----BEGIN CERTIFICATE-----
+MIIDczCCAlugAwIBAgILAK7PALrEzzL4Q7IwDQYJKoZIhvcNAQELBQAwVjELMAkG
+A1UEBhMCVVMxEzARBgNVBAsTCmVtU2lnbiBQS0kxFDASBgNVBAoTC2VNdWRocmEg
+SW5jMRwwGgYDVQQDExNlbVNpZ24gUm9vdCBDQSAtIEMxMB4XDTE4MDIxODE4MzAw
+MFoXDTQzMDIxODE4MzAwMFowVjELMAkGA1UEBhMCVVMxEzARBgNVBAsTCmVtU2ln
+biBQS0kxFDASBgNVBAoTC2VNdWRocmEgSW5jMRwwGgYDVQQDExNlbVNpZ24gUm9v
+dCBDQSAtIEMxMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAz+upufGZ
+BczYKCFK83M0UYRWEPWgTywS4/oTmifQz/l5GnRfHXk5/Fv4cI7gklL35CX5VIPZ
+HdPIWoU/Xse2B+4+wM6ar6xWQio5JXDWv7V7Nq2s9nPczdcdioOl+yuQFTdrHCZH
+3DspVpNqs8FqOp099cGXOFgFixwR4+S0uF2FHYP+eF8LRWgYSKVGczQ7/g/IdrvH
+GPMF0Ybzhe3nudkyrVWIzqa2kbBPrH4VI5b2P/AgNBbeCsbEBEV5f6f9vtKppa+c
+xSMq9zwhbL2vj07FOrLzNBL834AaSaTUqZX3noleoomslMuoaJuvimUnzYnu3Yy1
+aylwQ6BpC+S5DwIDAQABo0IwQDAdBgNVHQ4EFgQU/qHgcB4qAzlSWkK+XJGFehiq
+TbUwDgYDVR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQEL
+BQADggEBAMJKVvoVIXsoounlHfv4LcQ5lkFMOycsxGwYFYDGrK9HWS8mC+M2sO87
+/kOXSTKZEhVb3xEp/6tT+LvBeA+snFOvV71ojD1pM/CjoCNjO2RnIkSt1XHLVip4
+kqNPEjE2NuLe/gDEo2APJ62gsIq1NnpSob0n9CAnYuhNlCQT5AoE6TyrLshDCUrG
+YQTlSTR+08TI9Q/Aqum6VF7zYytPT1DU/rl7mYw9wC68AivTxEDkigcxHpvOJpkT
+xHqmiIMERnHXhuBUDDIlhJu58tBf5E7oke3VIAb3ADMmpDqw8NQBmIMMMAVSKeo
+WXzhriKi4gp6D/piq1JM4fHfyr6DDUI=
+-----END CERTIFICATE-----
@@ -0,0 +1,32 @@
+-----BEGIN CERTIFICATE-----
+MIIFcjCCA1qgAwIBAgIKLwq3aw3LSq8nWDANBgkqhkiG9w0BAQwFADBWMQswCQYD
+VQQGEwJVUzETMBEGA1UECxMKZW1TaWduIFBLSTEUMBIGA1UEChMLZU11ZGhyYSBJ
+bmMxHDAaBgNVBAMTE2VtU2lnbiBSb290IENBIC0gQzIwHhcNMTgwMjE4MTgzMDAw
+WhcNNDMwMjE4MTgzMDAwWjBWMQswCQYDVQQGEwJVUzETMBEGA1UECxMKZW1TaWdu
+IFBLSTEUMBIGA1UEChMLZU11ZGhyYSBJbmMxHDAaBgNVBAMTE2VtU2lnbiBSb290
+IENBIC0gQzIwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCMfX1lA+Tb
+mh9YInmRgOW97IVx4LUJf2DRZfs837Jrml+py64aVnYgWO4t6C78fgjfS7jX+c4T
+inIzEquWcI+zi0fd4Sc8NDf7JONp27VWX0qwUYqzLDRCt+s7zpLcfx1ky0zVIJj6
+L06uPyK3kIr9+YAsrVj+39utm6e2MBQsRNstSI3fCQYAGvoQTQ8fULauTqNWaYAk
+NYFe6HUHHQPp2u1Ua00odMXiD5oRFxLcDnGAcE1I/9E9mLCdkggXijYUmico7+Xw
+ZeFoPhva6eIJ5p03Lt3Du5W3EcHR0cJmmY1pyeA36JaXKWRNM9IRjYMVNCcp4jhB
+2tIYiZ+LVk8bwQ9/1c23txmv3u97taZlV22NF4ttS1qq3J+MOp0oGULBzpKfRx0q
+GVqbPukQNGAjOLIN8KDNQNzbR1iAl2d8H+MSoicBo4Aid8TjLWcNv48oCWL53ZrF
+BMTDjaIA6frG1t4IpbnHadA7qCJJe2qpJN6n2eQKAUn6UiQDHPsSqNBlcUhQ4Y/0
+Y0mU5rghm2OB9rXQS1Fb1JRCfJMNnJIm5AUB2+2RWzq5Tgz7SbSho8NsZk0UbQnF
+xciqQ9uoVTAsK14Sk9oG8Q3zfsM08cdPoRb0WlIZklR6mKD7L8nH/zfGu8PIJv94
+GGB9RZ9U4A69r3ePmy8MvrzfNxHKtH6svwIDAQABo0IwQDAdBgNVHQ4EFgQUs/eK
+pNYPiABZ6FEXT9V+7IYigZ0wDgYDVR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMB
+Af8wDQYJKoZIhvcNAQEMBQADggIBADQlpiWM0cv2nZ0H5jVsBq0x2q62Q0LwqATs
+CFvyub7gxNCytRuoA8stmPOEu/lg8Igxj4FIjoyhIrWUVxyiLU7No4P+WjEUOwUT
+xIpkEOtvGUQ9fiOlcGHtIZDNBlZq7WpktXAxeV55RPPsor26p2FNAMRFfZQh0sLX
+hKgk8iulSSggqx8ezgPye63FaiYEi4c/dzRj3HOCnsZiwZZU02df5YpNFjxSwZvE
+41cjGpsrpWMfQFI2s53RbeXp47lSAxYE4NzjBFMe+EwFuEveBCJBEAH5rvYu3pi2
+orsJ424TqWEQV1tCsCkQz+Yq/Okal7yHAkKDeOXcP7oN4A+TdXc2pdqxuVCnBO0R
+mWz2JpGSSeJjiTk/OPwRsPNWtwG/KXL04o2ta3jiPpJuICVtWDAc9R3auBEgJl5r
+ShRmBdszG0LmzsHuZPCFSYC15RBDCOBsa8bDRJ8pBFU2Wi/CVXCACEuavgoveA4F
+a5bt38o0PWxsBP+MpocCdVtDMqzQhxy9IohKuXWAGresoIvKDg3xFk6rBOrjfVwJ
+elwi/xAisojHPJVQv9W1zVIoHp+EQg/4MQC21NbIX2RoioB+V3hK439b/w7deU8x
+2M8cl1OG0nPfbnARl5GPM7vJgi470jto4SeMg6HMAW3Egb56tQcNLwI9U8mZnNvR
+gUMrkAgL
+-----END CERTIFICATE-----
@@ -0,0 +1,22 @@
+-----BEGIN CERTIFICATE-----
+MIIDlDCCAnygAwIBAgIKMfXkYgxsWO3W2DANBgkqhkiG9w0BAQsFADBnMQswCQYD
+VQQGEwJJTjETMBEGA1UECxMKZW1TaWduIFBLSTElMCMGA1UEChMcZU11ZGhyYSBU
+ZWNobm9sb2dpZXMgTGltaXRlZDEcMBoGA1UEAxMTZW1TaWduIFJvb3QgQ0EgLSBH
+MTAeFw0xODAyMTgxODMwMDBaFw00MzAyMTgxODMwMDBaMGcxCzAJBgNVBAYTAklO
+MRMwEQYDVQQLEwplbVNpZ24gUEtJMSUwIwYDVQQKExxlTXVkaHJhIFRlY2hub2xv
+Z2llcyBMaW1pdGVkMRwwGgYDVQQDExNlbVNpZ24gUm9vdCBDQSAtIEcxMIIBIjAN
+BgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAk0u76WaK7p1b1TST0Bsew+eeuGQz
+f2N4aLTNLnF115sgxk0pvLZoYIr3IZpWNVrzdr3YzZr/k1ZLpVkGoZM0Kd0WNHVO
+8oG0x5ZOrRkVUkr+PHB1cM2vK6sVmjM8qrOLqs1D/fXqcP/tzxE7lM5OMhbTI0Aq
+d7OvPAEsbO2ZLIvZTmmYsvePQbAyeGHWDV/D+qJAkh1cF+ZwPjXnorfCYuKrpDhM
+tTk1b+oDafo6VGiFbdbyL0NVHpENDtjVaqSW0RM8LHhQ6DqS0hdW5TUaQBw+jSzt
+Od9C4INBdN+jzcKGYEho42kLVACL5HZpIQ15TjQIXhTCzLG3rdd8cIrHhQIDAQAB
+o0IwQDAdBgNVHQ4EFgQU++8Nhp6w492pufEhF38+/PB3KxowDgYDVR0PAQH/BAQD
+AgEGMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAFn/8oz1h31x
+PaOfG1vR2vjTnGs2vZupYeveFix0PZ7mddrXuqe8QhfnPZHr5X3dPpzxz5KsbEjM
+wiI/aTvFthUvozXGaCocV685743QNcMYDHsAVhzNixl03r4PEuDQqqE/AjSxcM6d
+GNYIAwlG7mDgfrbESQRRfXBgvKqy/3lyeqYdPV8q+Mri/Tm3R7nrft8EI6/6nAYH
+6ftjk4BAtcZsCjEozgyfz7MjNYBBjWzEN3uBL4ChQEKF6dk4jeihU80Bv2noWgby
+RQuQ+q7hv53yrlc8pa6yVvSLZUDp/TGBLPQ5Cdjua6e0ph0VpZj3AYHYhX3zUVxx
+iN66zB+Afko=
+-----END CERTIFICATE-----
@@ -0,0 +1,32 @@
+-----BEGIN CERTIFICATE-----
+MIIFlTCCA32gAwIBAgILAIZNvw/jXtd9jtgwDQYJKoZIhvcNAQEMBQAwZzELMAkG
+A1UEBhMCSU4xEzARBgNVBAsTCmVtU2lnbiBQS0kxJTAjBgNVBAoTHGVNdWRocmEg
+VGVjaG5vbG9naWVzIExpbWl0ZWQxHDAaBgNVBAMTE2VtU2lnbiBSb290IENBIC0g
+RzIwHhcNMTgwMjE4MTgzMDAwWhcNNDMwMjE4MTgzMDAwWjBnMQswCQYDVQQGEwJJ
+TjETMBEGA1UECxMKZW1TaWduIFBLSTElMCMGA1UEChMcZU11ZGhyYSBUZWNobm9s
+b2dpZXMgTGltaXRlZDEcMBoGA1UEAxMTZW1TaWduIFJvb3QgQ0EgLSBHMjCCAiIw
+DQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAMNwGIWW2kHfHK+sXTNwxF07K+IV
+ySTuyFM2r1v002wUfcdT+zs5OM5QbMYFFnedXQI6gCFLsjKrcaej48Zt37OyEb3i
+aPs7CsP4kAyTwzKH9aZe6gXYHrJq40/ZVMNcQVI2PcIp40B/SAN2gUZ+ZaUtIOvV
+jEx26/ebNaXRIsthlkOG/caB+QRwDw1tl7338Zlv0M2oTBUy4B3e7dGP5pgXH71M
+jqHPCoNo+xv9f0NTBT+hUDa8h8wUtcGQq9CDeJTpjWcD2bP2AMdVG6oVpMAUeUzo
+cCyglvtFdUMjggxBbw4qhau1HXPG8Ot9hwL7ZMi8tkTzrvUIxxb8G9LF/7kKeCE7
+tGZaVzDTnXuifl3msR4ErHsQ4P7lVu2AIjIAhrAXoedDidb7pMcf7TABdrYUT1Jo
+G/AiK+J9jO6GTjeADD4LMDSBZhHMuBK/PJ/g0kGBt+/C1L+/HURzQhJkMlRnM6Rv
+XoCtfKopSlns5trZmTi971Wjbn88QXP61lGpBCUPwCjs7rpOYvSUJtI+lcbF+37q
+kIqOXYkVT3cupDSpw+H89kFtj5GKY+Xny4LxY+3IvDIRiyd6ky1DPj713DI0yqve
+EpsIr3A0PdwuyUI7CS1jg0NnGFT6Xxyr0xB+VDt83FJYW8v16k2pbaQ4kVxA3aXd
+X9dZYyVR1S59KM75AgMBAAGjQjBAMB0GA1UdDgQWBBTt7E1FYRgo57MjKBEcTaUn
+DV7s9DAOBgNVHQ8BAf8EBAMCAQYwDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG9w0B
+AQwFAAOCAgEACFC/ilQg8KTCVBxFJW/sazomkS0kNYbEIZg4B3obqwsJ7SX98z8Z
+gfzBpz0nYClwwJjWbFN1R2zY8pCEot6/dgmA8Vbq0GxhwPM5YN/SZquNyRIxO3cU
+dlAcwf+vSezdVCf9wOzvSAF3q0a5ljvbdbNJNpfScQVp7UUd5sBsZk8jXO1KQ/go
+/Vf/GDPnrIFmxpAIGE3sgnO8lAv9FzUaAeuv7HWe47xN9J7+bQzF93yHuIXACPTL
+pQHhg2zMv5C7BAbuDHfbj1Cu294Z832yhSfBcziWGskOvl3es2EcHytbS9c9P+0z
+Mpka7zGC1FHrvLb/FoduH86TeZt0QjZ6pcplNzoaxDnDvzTJ6CC2Eny+qH/APFCu
+VUv5/wjwF+HPm8Pup2ARj9cEp92+0qcerfHacNq5hMeGZdbA/dzdUR/5z5zXdxAk
+nl8mcfGb0eMNSTXQmmB/i4AecNnr72uYjzlaXUGYN7Nrb6XouG0pnh0/BBtWWp0U
+ShIPpWEAqs7RJBj6+1ZUYXZ4ObrCw962DxhN2p19Hxw9LtuUUcLqqTPrFXYvwO4t
+ouj7KJnAkaTUfXGdEaFVtFig1EA30WzJY2X1vAQ7hVnniCjgaXAGqjsU6sklNM9n
+xDx5rFCCCEtj9Kh8UHjGK2QqgP5kwgttjOApQMaCoezMfK4KD7WpOXU=
+-----END CERTIFICATE-----
@@ -0,0 +1,32 @@
+-----BEGIN CERTIFICATE-----
+MIIFdjCCA16gAwIBAgIQCIYhdXKnxAzDuy7l1y1ubDANBgkqhkiG9w0BAQwFADBV
+MQswCQYDVQQGEwJJTjElMCMGA1UECgwcZU11ZGhyYSBUZWNobm9sb2dpZXMgTGlt
+aXRlZDEfMB0GA1UEAwwWZW1TaWduIFJvb3QgQ1MgQ0EgLSBHMjAeFw0yNDAyMDgw
+ODE1MzlaFw00OTAyMDcwODE1MzlaMFUxCzAJBgNVBAYTAklOMSUwIwYDVQQKDBxl
+TXVkaHJhIFRlY2hub2xvZ2llcyBMaW1pdGVkMR8wHQYDVQQDDBZlbVNpZ24gUm9v
+dCBDUyBDQSAtIEcyMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA4CVO
+xuBjdLJXpkrjosA+7ygTGaYzufUJrhQOeIZEuCqeCPI8hDqIu+MIqEqOlgDFetVM
+3Cor3CGBlGJzfzzlI31vyElvQq88jjCfo12Ghqcf+NAZDFduJk5+NgC0z3lfdiB6
+NNzxbfVkA+X6oDNNRioVUZsVFuIzT9Qt0Sh4KwQPRFBc/wI2yGpDuKHNsLehcECx
+Uw00m33xGYh9UkeL3reOmAaLYWZiO+Pd2vxPtrkXI4wyN4BMlESJ73uP0tyX9lq4
+M4yN2yh3i+ETI8/jzPjxHD7QFaVkCoS+uj2YmBa3mytlqlBgyOoKueNeotX8hfzB
+FNm962abpV4mN0SoC6hjkYv8IayseRPFv9VI0meQXg9OLRoNvBOqG5kw937FH0Li
+q3CIzKyvdyeYRT0LJdWA2iuB9l+4zzIfCwGyT4M4VDmJuPBhrTmjXKae6AUtm1eO
+q2f8gftA6J6AlbD0gRd/t8CrFssFNm+LYAEd36kWRBuolbaOfbSbEww84xNAgj4U
+SPwYjtNDsCkQjtgSOaDtDOmolN+Mbk528Y+LHMlbd4T3Xmp+VIWp5+70GhBDOA91
+hhtims+hHkXjRI2U7WrJLcTlcF8ykiymRbFKhwOBvTmtZNbHmDI7scU4wWUAsmW6
+tWaeeoM50KBl8ai28A6yBGYMO//03Ubvqiqst80CAwEAAaNCMEAwHQYDVR0OBBYE
+FOYcxqontIvyvYVkX6/0unKm0AF1MA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMBAf8E
+BTADAQH/MA0GCSqGSIb3DQEBDAUAA4ICAQCv5Asord7fdQMT30emwb+oLuDUxdmJ
+DQHnzxIXoNvymDTbNHF9LqjeXc+X+XcjIV8PTp4o2I9wyVppHmcaVgRjN/G9FxeN
+5tCQ97v5sdx4eAdPljK1xqg7Vhfmzcp+7rSjJG7Sn9gMlZx81kL3Oa2qZj+W5Fi6
+tN4vaR9OLEyvKKHDG/eFxS/qghD1d/Clkb7UGGjBcWJlaZTWVALMKG+QCxiMoELi
+oeKaoBbZoXooxfF0mWdkIl9RWNU4r9O/yt66LBE9rj49dX19L+rGg71/4bTi5ua+
+Wz60XaOc2CdHJaBSols5hWEx/LBjS7UzfU3uJfnpQXVDi0US5sFzIHIlS6kjh7h
+YuF4+/9ZAl4Bx3w2TNN7pVm7I39pgYl7XN6KZRwfkAv4aRswPqhWbQfr3+qfcC93
+cgTyFy2RG5YE/OfmpSs6VtYKawqsdEODz/p2nhZfSDrVDnEPvERqahttY5+gaWNc
+2fbibgKMfuejTZaJ3yttPslVSko5SDs0xKU423swu34sb/fdISqtOu4NzPCZiHSG
+ladaoITmSA6UWFk4fdqaLBI+xSxY5Et3S1APp5r3j/l1KrWPNbb1/+ydHN+YLbh8
+hCBIecKdk6VbnnFRlulm0d2RSgpfCRrWLomN8JHCxCMVv/hBQXz3AXGM1ZlfYTyl
+E1BpDby0G2eEkw==
+-----END CERTIFICATE-----
@@ -10,3 +10,4 @@ include:
  - ca.server
 {% endif %}
  - ca.trustca
+  - ca.trust
@@ -0,0 +1,12 @@
+# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
+# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at 
+# https://securityonion.net/license; you may not use this file except in compliance with the
+# Elastic License 2.0.
+
+{# Containers mount the extracted directory rather than single files: update-ca-trust replaces
+   files by rename, so a file bind mount would keep pointing at the old inode. #}
+{% set CATRUST = {
+    'dir': '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro',
+    'java': '/etc/pki/ca-trust/extracted/java/cacerts',
+    'pem': '/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'
+} %}
@@ -0,0 +1,37 @@
+# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
+# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at 
+# https://securityonion.net/license; you may not use this file except in compliance with the
+# Elastic License 2.0.
+
+ca_trust_anchor:
+  file.managed:
+    - name: /etc/pki/ca-trust/source/anchors/so-intca.crt
+    - source: salt://ca/files/ca.crt
+    - show_changes: False
+    - makedirs: True
+
+ca_trust_legacy_anchor:
+  file.absent:
+    - name: /etc/pki/ca-trust/source/anchors/ca.crt
+
+ca_trust_legacy_bundles:
+  file.absent:
+    - name: /opt/so/conf/ca
+
+# clean removes blocklist entries that are no longer shipped
+ca_trust_blocklist:
+  file.recurse:
+    - name: /etc/pki/ca-trust/source/blocklist
+    - source: salt://ca/files/blocklist
+    - clean: True
+    - makedirs: True
+    - file_mode: '0644'
+    - dir_mode: '0755'
+
+ca_trust_extract:
+  cmd.run:
+    - name: update-ca-trust extract
+    - onchanges:
+      - file: ca_trust_anchor
+      - file: ca_trust_legacy_anchor
+      - file: ca_trust_blocklist
@@ -178,6 +178,7 @@ if [[ $EXCLUDE_FALSE_POSITIVE_ERRORS == 'Y' ]]; then
    EXCLUDED_ERRORS="$EXCLUDED_ERRORS|Missing ory_kratos_session cookie" # expected WARN log lines indicating invalid auth header
    EXCLUDED_ERRORS="$EXCLUDED_ERRORS|Static assets preprocessor only supports GET and HEAD requests" # expected WARN log lines indicating invalid auth header
    EXCLUDED_ERRORS="$EXCLUDED_ERRORS|respondError"                 # respondError is a function name, output via http middleware as standard request logging
+    EXCLUDED_ERRORS="$EXCLUDED_ERRORS|GET /kibana/"                 # Ignore Kibana queries with triggered words
 fi

 if [[ $EXCLUDE_KNOWN_ERRORS == 'Y' ]]; then
@@ -438,6 +438,10 @@ install_networkminer:
  pkg.latest:
    - name: securityonion-networkminer

+install_so_crates:
+  pkg.latest:
+    - name: securityonion-so-crates
+
 {% else %}

 desktop_packages_os_fail:
@@ -1,18 +1,10 @@
- {% from 'vars/globals.map.jinja' import GLOBALS %}
+{% from 'vars/globals.map.jinja' import GLOBALS %}

 {# we only want this state to run it is CentOS #}
 {% if GLOBALS.os == 'OEL' %}

-trusted_ca:
-  file.managed:
-    - name: /etc/pki/ca-trust/source/anchors/ca.crt
-    - source: salt://ca/files/ca.crt
-
-update_ca_certs:
-  cmd.run:
-    - name: update-ca-trust
-    - onchanges:
-      - file: trusted_ca
+include:
+  - ca.trust

 {% else %}

@@ -10,7 +10,7 @@ elastalert:
    buffer_time:
      minutes: 10
    old_query_limit:
-      minutes: 5
+      minutes: 1440
    es_port: 9200
    es_conn_timeout: 55
    max_query_size: 5000
@@ -6,9 +6,11 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}

 include:
+  - ca.trust
  - elastalert.config
  - elastalert.sostatus

@@ -28,6 +30,7 @@ so-elastalert:
        - ipv4_address: {{ DOCKERMERGED.containers['so-elastalert'].ip }}
    - detach: True
    - binds:
+      - {{ CATRUST.dir }}
      - /opt/so/rules/elastalert:/opt/elastalert/rules/:ro
      - /opt/so/log/elastalert:/var/log/elastalert:rw
      - /opt/so/conf/elastalert/modules/:/opt/elastalert/modules/:ro
@@ -46,8 +49,9 @@ so-elastalert:
      - {{ XTRAHOST }}
        {% endfor %}
      {% endif %}
-      {% if DOCKERMERGED.containers['so-elastalert'].extra_env %}
    - environment:
+      - REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
+      {% if DOCKERMERGED.containers['so-elastalert'].extra_env %}
        {% for XTRAENV in DOCKERMERGED.containers['so-elastalert'].extra_env %}
      - {{ XTRAENV }}
        {% endfor %}
@@ -65,6 +69,7 @@ so-elastalert:
      - file: elastacustmodulesdir
      - file: elastaconf
    - watch:
+      - cmd: ca_trust_extract
      - file: elastaconf
    - onlyif:
      - "so-elasticsearch-query / | jq -r '.version.number[0:1]' | grep -q 9" {# only run this state if elasticsearch is version 9 #}
@@ -0,0 +1,80 @@
+# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
+# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
+# https://securityonion.net/license; you may not use this file except in compliance with the
+# Elastic License 2.0.
+
+from datetime import datetime
+import json
+import logging
+import sys
+import types
+
+# stand-ins when ElastAlert isn't installed (CI)
+try:
+    import elastalert.alerts  # noqa: F401
+except ImportError:
+
+    class Alerter:
+        def __init__(self, rule):
+            self.rule = rule
+
+    class DateTimeEncoder(json.JSONEncoder):
+        def default(self, obj):
+            return obj.isoformat() if hasattr(obj, 'isoformat') else json.JSONEncoder.default(self, obj)
+
+    class EAException(Exception):
+        pass
+
+    def lookup_es_key(doc, term):
+        for part in term.split('.'):
+            if not isinstance(doc, dict) or part not in doc:
+                return None
+            doc = doc[part]
+        return doc
+
+    def ts_to_dt(value):
+        return value if isinstance(value, datetime) else datetime.fromisoformat(value)
+
+    def elasticsearch_client(conf):
+        return None  # tests set the alerter's client
+
+    alerts = types.ModuleType('elastalert.alerts')
+    alerts.Alerter = Alerter
+    alerts.DateTimeEncoder = DateTimeEncoder
+    util = types.ModuleType('elastalert.util')
+    util.EAException = EAException
+    util.elastalert_logger = logging.getLogger('elastalert')
+    util.lookup_es_key = lookup_es_key
+    util.ts_to_dt = ts_to_dt
+    util.elasticsearch_client = elasticsearch_client
+    package = types.ModuleType('elastalert')
+    package.alerts = alerts
+    package.util = util
+    sys.modules.update({'elastalert': package, 'elastalert.alerts': alerts, 'elastalert.util': util})
+
+# stand-ins when elasticsearch-py isn't installed (CI)
+try:
+    import elasticsearch.exceptions  # noqa: F401
+except ImportError:
+
+    class ElasticsearchException(Exception):
+        pass
+
+    class TransportError(ElasticsearchException):
+        pass
+
+    class ConnectionError(TransportError):
+        pass
+
+    class ConflictError(TransportError):
+        pass
+
+    class RequestError(TransportError):
+        pass
+
+    exceptions = types.ModuleType('elasticsearch.exceptions')
+    for cls in (ElasticsearchException, TransportError, ConnectionError, ConflictError, RequestError):
+        setattr(exceptions, cls.__name__, cls)
+    es_package = types.ModuleType('elasticsearch')
+    es_package.exceptions = exceptions
+    sys.modules.update({'elasticsearch': es_package, 'elasticsearch.exceptions': exceptions})
@@ -1,63 +1,269 @@
 # -*- coding: utf-8 -*-

 # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
-# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at 
+# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
 # https://securityonion.net/license; you may not use this file except in compliance with the
 # Elastic License 2.0.

-
-from time import gmtime, strftime
-import requests,json
-from elastalert.alerts import Alerter
+from datetime import datetime, timezone
+import hashlib
+import ipaddress
+import json
+import re
+import uuid

 import urllib3
+from elasticsearch.exceptions import ConflictError, ElasticsearchException, RequestError
+from elastalert.alerts import Alerter, DateTimeEncoder
+from elastalert.util import EAException, elastalert_logger, elasticsearch_client, lookup_es_key, ts_to_dt
+
+# grid runs verify_certs: false; also quiets ElastAlert's own queries
 urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

+ALERT_INDEX = 'logs-detections.alerts-so'
+# ES error text kept in logs and alerts
+ERROR_TEXT_LIMIT = 500
+# a match missing backend columns (window_start, count, @timestamp); the alert is still written
+MATCH_ERRORS = (KeyError, TypeError, ValueError)
+
+
 class SecurityOnionESAlerter(Alerter):
    """
    Use matched data to create alerts in Elasticsearch.
    """

-    required_options = set(['detection_title', 'sigma_level'])
-    optional_fields = ['sigma_category', 'sigma_product', 'sigma_service']
+    required_options = {'detection_title', 'sigma_level'}
+    optional_fields = ['sigma_category', 'sigma_product', 'sigma_service', 'sigma_correlation']
+
+    # count column and default summary per type; stored alert data, so not localized
+    CORRELATION_COUNTS = {
+        'event_count': ('event_count', '%count% events'),
+        'value_count': ('value_count', '%count% distinct values'),
+        'temporal': ('event_type_count', '%count% correlated rules matched'),
+        'value_sum': ('value_sum', 'total %count%'),
+        'value_avg': ('value_avg', 'average %count%'),
+        'value_percentile': ('value_percentile', 'percentile %count%'),
+        'value_median': ('value_median', 'median %count%'),
+    }
+    PLACEHOLDER = re.compile(r'%([^%\s]+)%')
+    # group-by fields copied into ECS related.*
+    RELATED_USERS = {'user.name', 'winlog.event_data.TargetUserName', 'winlog.event_data.SubjectUserName'}
+    RELATED_HOSTS = {'host.name', 'host.hostname', 'winlog.computer_name'}
+
+    def __init__(self, rule):
+        super().__init__(rule)
+        # uses the grid's TLS, auth and timeout settings
+        self.es = elasticsearch_client(rule)
+
+    @property
+    def is_correlation(self):
+        return bool(self.rule.get('sigma_correlation'))
+
+    def query_keys(self):
+        """compound_query_key holds the list; query_key is flattened to a string."""
+        if self.rule.get('compound_query_key'):
+            return self.rule['compound_query_key']
+        if self.rule.get('query_key'):
+            return [self.rule['query_key']]
+        return []
+
+    def alert_id(self, match):
+        """Stable id: window end + group values for correlations, source _id otherwise; random without one."""
+        if self.is_correlation:
+            # ungrouped rows have a hashed _id that changes with the count
+            values = ''.join(f"|{lookup_es_key(match, k)}" for k in self.query_keys())
+            key = f"{self.rule['detection_public_id']}|{ts_to_dt(match['@timestamp']).isoformat()}{values}"
+        elif match.get('_id'):
+            key = f"{self.rule['detection_public_id']}|{match['_id']}"
+        else:
+            return uuid.uuid4().hex
+
+        return hashlib.sha256(key.encode('utf-8')).hexdigest()
+
+    def group(self, match):
+        """Group-by values, joined like ElastAlert's realert key."""
+        return ', '.join(str(lookup_es_key(match, k)) for k in self.query_keys())
+
+    def related_bucket(self, key):
+        if key == 'ip' or key.endswith('.ip'):
+            return 'ip'
+        if key in self.RELATED_USERS or key.endswith('.user.name'):
+            return 'user'
+        if key in self.RELATED_HOSTS:
+            return 'hosts'
+        return None
+
+    @staticmethod
+    def valid_ip(value):
+        try:
+            ipaddress.ip_address(value)
+            return True
+        except ValueError:
+            return False
+
+    def related(self, match):
+        """ECS related.* from group-by values; skips invalid IPs."""
+        related = {}
+        for key in self.query_keys():
+            bucket = self.related_bucket(key)
+            if not bucket:
+                continue
+            # original spellings of a lowercased group
+            value = lookup_es_key(match, f"{key}_spellings")
+            if value is None:
+                value = lookup_es_key(match, key)
+            for v in value if isinstance(value, list) else [value]:
+                if v is None or (bucket == 'ip' and not self.valid_ip(str(v))):
+                    continue
+                # dict: ordered and deduped
+                related.setdefault(bucket, {})[str(v)] = None
+        return {bucket: list(values) for bucket, values in related.items()}
+
+    def event_data(self, match):
+        """The match minus the compound query_key field, which ES would map by its last part."""
+        if not self.rule.get('compound_query_key'):
+            return match
+        return {k: v for k, v in match.items() if k != self.rule['query_key']}
+
+    @staticmethod
+    def format_value(value):
+        if isinstance(value, list):
+            shown = ', '.join(str(v) for v in value[:3])
+            return shown if len(value) <= 3 else f"{shown} and {len(value) - 3} more"
+        return str(value)
+
+    @staticmethod
+    def format_count(value):
+        if isinstance(value, float) and not value.is_integer():
+            return f"{value:,.2f}"
+        if isinstance(value, (int, float)):
+            return f"{int(value):,}"
+        return str(value)
+
+    @staticmethod
+    def format_duration(seconds):
+        for unit, size in (('hour', 3600), ('minute', 60)):
+            if seconds >= 2 * size:
+                return f"{seconds // size} {unit}s"
+        return f"{seconds} second{'' if seconds == 1 else 's'}"
+
+    def summary(self, match):
+        """One-line correlation summary."""
+        column, label = self.CORRELATION_COUNTS.get(self.rule['sigma_correlation'], (None, '%count%'))
+        start = ts_to_dt(match['window_start'])
+        end = ts_to_dt(match['@timestamp'])
+        values = {
+            'count': self.format_count(match.get(column)),
+            'start': start.strftime('%Y-%m-%d %H:%M:%S UTC'),
+            'end': end.strftime('%Y-%m-%d %H:%M:%S UTC'),
+            'duration': self.format_duration(int((end - start).total_seconds())),
+        }
+
+        template = self.rule.get('summary_template')
+        if not template:
+            groups = ', '.join(f"{k} %{k}%" for k in self.query_keys())
+            template = f"{label} for {groups} in %duration%" if groups else f"{label} in %duration%"
+
+        def fill(m):
+            if m[1] in values:
+                return values[m[1]]
+            value = lookup_es_key(match, m[1])
+            # unknown placeholders stay visible so typos show
+            return m[0] if value is None else self.format_value(value)
+
+        return self.PLACEHOLDER.sub(fill, template)

    def alert(self, matches):
        for match in matches:
-            timestamp = strftime("%Y-%m-%d"'T'"%H:%M:%S"'.000Z', gmtime())
-            headers = {"Content-Type": "application/json"}
+            try:
+                alert_id = self.alert_id(match)
+            except MATCH_ERRORS as e:
+                elastalert_logger.warning("Writing alert for rule %s without a stable id, so a retry may duplicate it: %r",
+                                          self.rule['detection_public_id'], e)
+                alert_id = uuid.uuid4().hex
+            try:
+                self.write(alert_id, self.payload(match))
+            except ElasticsearchException as e:
+                # EAException makes ElastAlert retry
+                raise EAException(f"Unable to write the alert to Elasticsearch: {str(e)[:ERROR_TEXT_LIMIT]}") from e

-            creds = None
-            if 'es_username' in self.rule and 'es_password' in self.rule:
-                creds = (self.rule['es_username'], self.rule['es_password'])
+    def payload(self, match):
+        rule_info = {
+            "name": self.rule['detection_title'],
+            "uuid": self.rule['detection_public_id']
+        }

-            # Start building the rule dict
-            rule_info = {
-                "name": self.rule['detection_title'],
-                "uuid": self.rule['detection_public_id']
-            }
+        # Add optional fields if they are present in the rule
+        for field in self.optional_fields:
+            rule_key = field.split('_')[-1]  # Assumes field format "sigma_<key>"
+            if field in self.rule:
+                rule_info[rule_key] = self.rule[field]

-            # Add optional fields if they are present in the rule
-            for field in self.optional_fields:
-                rule_key = field.split('_')[-1]  # Assumes field format "sigma_<key>"
-                if field in self.rule:
-                    rule_info[rule_key] = self.rule[field]
+        event_info = {
+            "kind": "alert",
+            "severity": self.rule['event.severity'],
+            "module": self.rule['event.module'],
+            "dataset": self.rule['event.dataset'],
+            "severity_label": self.rule['sigma_level']
+        }

-            # Construct the payload with the conditional rule_info
-            payload = {
-                "tags": "alert",
-                "rule": rule_info,
-                "event": {
-                    "severity": self.rule['event.severity'],
-                    "module": self.rule['event.module'],
-                    "dataset": self.rule['event.dataset'],
-                    "severity_label": self.rule['sigma_level']
-                },
-                "sigma_level": self.rule['sigma_level'],
-                "event_data": match,
-                "@timestamp": timestamp
-            }
-            url = f"https://{self.rule['es_host']}:{self.rule['es_port']}/logs-detections.alerts-so/_doc/"
-            requests.post(url, data=json.dumps(payload), headers=headers, verify=False, auth=creds)
+        payload = {
+            "tags": ["alert"],
+            "rule": rule_info,
+            "event": event_info,
+            "sigma_level": self.rule['sigma_level'],
+            "event_data": self.event_data(match),
+            "@timestamp": datetime.now(timezone.utc).strftime('%Y-%m-%dT%H:%M:%S.000Z')
+        }
+
+        if self.is_correlation:
+            keys = self.query_keys()
+            try:
+                # built before any is added, so a failure adds none
+                reason = self.summary(match)
+                labels = {"correlation_group_by": ', '.join(keys), "correlation_group": self.group(match)} if keys else None
+                related = self.related(match)
+            except MATCH_ERRORS as e:
+                elastalert_logger.warning("Writing alert for rule %s without its correlation summary: %r",
+                                          self.rule['detection_public_id'], e)
+            else:
+                payload["event"]["reason"] = reason
+                if labels:
+                    payload["labels"] = labels
+                if related:
+                    payload["related"] = related
+
+        return payload
+
+    def write(self, alert_id, payload):
+        try:
+            self.create(alert_id, payload)
+        except RequestError as e:
+            # mapping rejections come from event_data; retry it as text
+            rejection = str(e)[:ERROR_TEXT_LIMIT]
+            try:
+                self.create(alert_id, self.without_event_data(payload, rejection))
+            except RequestError as again:
+                elastalert_logger.error("Dropping alert %s for rule %s, rejected by Elasticsearch even without its event data: %s; first rejection: %s",
+                                        alert_id, self.rule['detection_public_id'], str(again)[:ERROR_TEXT_LIMIT], rejection)
+                return
+            elastalert_logger.warning("Stored alert %s for rule %s with its event data as text, rejected by Elasticsearch: %s",
+                                      alert_id, self.rule['detection_public_id'], rejection)
+
+    def create(self, alert_id, payload):
+        try:
+            self.es.create(index=ALERT_INDEX, id=alert_id, body=payload)
+        except ConflictError:
+            pass  # a repeat id is already stored
+
+    @staticmethod
+    def without_event_data(payload, rejection):
+        """Moves event_data to event.original; the tag keeps Fleet's final pipeline from dropping it."""
+        fallback = {k: v for k, v in payload.items() if k != 'event_data'}
+        fallback['event'] = dict(payload['event'], original=json.dumps(payload['event_data'], cls=DateTimeEncoder))
+        fallback['error'] = {'message': f"event_data rejected by Elasticsearch: {rejection}"}
+        fallback['tags'] = payload['tags'] + ['preserve_original_event']
+        return fallback

    def get_info(self):
        return {'type': 'SecurityOnionESAlerter'}
@@ -0,0 +1,260 @@
+# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
+# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
+# https://securityonion.net/license; you may not use this file except in compliance with the
+# Elastic License 2.0.
+
+import copy
+from datetime import datetime, timezone
+import importlib.util
+import json
+import os
+import unittest
+from unittest.mock import MagicMock
+
+from elasticsearch.exceptions import ConflictError, ConnectionError, RequestError
+
+spec = importlib.util.spec_from_file_location('securityonion_es', os.path.join(os.path.dirname(__file__), 'securityonion-es.py'))
+es = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(es)
+
+BASE_RULE = {
+    'detection_title': 'Many Failed Network Logons To One Host From One Source',
+    'detection_public_id': '35a42db6-6629-45af-b8aa-e1fa33c28ef5',
+    'sigma_level': 'medium',
+    'sigma_correlation': 'event_count',
+    'event.severity': 3,
+    'event.module': 'sigma',
+    'event.dataset': 'sigma.alert',
+    'summary_template': '%count% failed network logons to %host.name% from %source.ip% in %duration%',
+}
+
+PLAIN_RULE = {k: v for k, v in BASE_RULE.items() if k not in ('sigma_correlation', 'summary_template')}
+
+
+def correlation_match():
+    return {
+        'event_count': 3561,
+        'window_start': '2026-09-30T18:05:10+00:00',
+        '@timestamp': '2026-09-30T18:07:53+00:00',
+        'host': {'name': 'host-01'},
+        'source': {'ip': '192.0.2.10'},
+        '_id': '6d1c',
+        'num_hits': 1,
+        'num_matches': 1,
+    }
+
+
+class TestSecurityOnionESAlerter(unittest.TestCase):
+
+    def creates(self, rule, match, effects=None):
+        """Run alert(); return (body, id) of each create."""
+        alerter = es.SecurityOnionESAlerter(rule)
+        alerter.es = MagicMock()
+        alerter.es.create.side_effect = effects
+        alerter.alert([match])
+        calls = alerter.es.create.call_args_list
+        self.assertTrue(all(c.kwargs['index'] == 'logs-detections.alerts-so' for c in calls))
+        return [(json.loads(json.dumps(c.kwargs['body'], cls=es.DateTimeEncoder)), c.kwargs['id']) for c in calls]
+
+    def send(self, rule, match):
+        """Run alert(); return the payload it wrote and its id."""
+        (payload, alert_id), = self.creates(rule, match)
+        return payload, alert_id
+
+    def test_compound_query_key_left_out_of_event_data(self):
+        rule = dict(BASE_RULE, compound_query_key=['host.name', 'source.ip'], query_key='host.name,source.ip')
+        match = correlation_match()
+        match['host.name,source.ip'] = 'host-01, 192.0.2.10'
+        original = copy.deepcopy(match)
+
+        payload, _ = self.send(rule, match)
+
+        self.assertNotIn('host.name,source.ip', payload['event_data'])
+        self.assertEqual(payload['event_data']['host'], {'name': 'host-01'})
+        self.assertEqual(payload['event_data']['source'], {'ip': '192.0.2.10'})
+        self.assertEqual(payload['rule'], {'name': BASE_RULE['detection_title'], 'uuid': BASE_RULE['detection_public_id'], 'correlation': 'event_count'})
+        self.assertEqual(payload['labels'], {'correlation_group_by': 'host.name, source.ip', 'correlation_group': 'host-01, 192.0.2.10'})
+        self.assertEqual(payload['related'], {'hosts': ['host-01'], 'ip': ['192.0.2.10']})
+        self.assertEqual(payload['event']['kind'], 'alert')
+        self.assertEqual(payload['event']['reason'], '3,561 failed network logons to host-01 from 192.0.2.10 in 2 minutes')
+        # ElastAlert reuses the match
+        self.assertEqual(match, original)
+
+    def test_single_query_key(self):
+        rule = dict(BASE_RULE, query_key='user.name', summary_template='%count% failed SOC logins for %user.name%')
+        match = {'event_count': 3, 'window_start': '2026-09-30T16:49:52+00:00', '@timestamp': '2026-09-30T16:50:00+00:00',
+                 'user': {'name': 'user@example.invalid'}}
+
+        payload, _ = self.send(rule, match)
+
+        self.assertEqual(payload['labels'], {'correlation_group_by': 'user.name', 'correlation_group': 'user@example.invalid'})
+        self.assertEqual(payload['related'], {'user': ['user@example.invalid']})
+        self.assertEqual(payload['event']['reason'], '3 failed SOC logins for user@example.invalid')
+        self.assertEqual(payload['event_data'], match)
+
+    def test_related_buckets(self):
+        rule = dict(BASE_RULE, compound_query_key=['winlog.event_data.TargetUserName', 'source.ip', 'dns.highest_registered_domain'],
+                    query_key='winlog.event_data.TargetUserName,source.ip,dns.highest_registered_domain')
+        match = correlation_match()
+        match.update({'winlog': {'event_data': {'TargetUserName': ['admin1', 'svc', 'admin1']}},
+                      'source': {'ip': 'not-an-ip'}, 'dns': {'highest_registered_domain': 'example.com'}})
+
+        payload, _ = self.send(rule, match)
+
+        # deduped; invalid IP skipped; domain stays in the group only
+        self.assertEqual(payload['related'], {'user': ['admin1', 'svc']})
+        self.assertEqual(payload['labels']['correlation_group'], "['admin1', 'svc', 'admin1'], not-an-ip, example.com")
+
+        payload, _ = self.send(dict(BASE_RULE, query_key='dns.highest_registered_domain'), match)
+
+        self.assertNotIn('related', payload)
+
+    def test_related_uses_original_spellings(self):
+        rule = dict(BASE_RULE, query_key='user.name', summary_template=None)
+        match = {'event_count': 3, 'window_start': '2026-09-30T16:49:52+00:00', '@timestamp': '2026-09-30T16:50:00+00:00',
+                 'user': {'name': 'admin', 'name_spellings': ['Admin', 'admin', 'ADMIN']}}
+
+        payload, _ = self.send(rule, match)
+
+        # the group shows the lowercased value; related.user finds every spelling
+        self.assertEqual(payload['labels']['correlation_group'], 'admin')
+        self.assertEqual(payload['related'], {'user': ['Admin', 'admin', 'ADMIN']})
+        self.assertEqual(payload['event']['reason'], '3 events for user.name admin in 8 seconds')
+
+    def test_plain_rule_has_no_correlation_fields(self):
+        # a query_key alone (e.g. from an override) isn't a correlation
+        rule = dict(PLAIN_RULE, query_key='user.name')
+        # ElastAlert parses @timestamp for EQL hits
+        match = {'@timestamp': datetime(2026, 9, 30, 16, 50, tzinfo=timezone.utc), '_id': 'abc',
+                 'process': {'name': 'whoami.exe'}, 'user': {'name': 'user'}}
+
+        payload, alert_id = self.send(rule, match)
+
+        alerter = es.SecurityOnionESAlerter(rule)
+        self.assertEqual(payload['rule'], {'name': PLAIN_RULE['detection_title'], 'uuid': PLAIN_RULE['detection_public_id']})
+        self.assertNotIn('labels', payload)
+        self.assertNotIn('related', payload)
+        self.assertNotIn('reason', payload['event'])
+        self.assertEqual(payload['event']['kind'], 'alert')
+        self.assertEqual(payload['event_data'], dict(match, **{'@timestamp': '2026-09-30T16:50:00+00:00'}))
+        self.assertEqual(alert_id, alerter.alert_id(match))
+        self.assertNotEqual(alerter.alert_id(match), alerter.alert_id(dict(match, _id='abd')))
+        self.assertNotEqual(alerter.alert_id(match), es.SecurityOnionESAlerter(dict(rule, detection_public_id='other')).alert_id(match))
+        # without an _id, never deduplicated
+        self.assertNotEqual(alerter.alert_id({'_id': None}), alerter.alert_id({'_id': None}))
+
+    def test_ungrouped_correlation_id_ignores_row_hash(self):
+        rule = dict(BASE_RULE, summary_template=None)
+        first = {k: v for k, v in correlation_match().items() if k not in ('host', 'source')}
+        # ES|QL hashes the row into _id, so a later count changes it
+        later = dict(first, event_count=3600, _id='9f2a')
+
+        payload, alert_id = self.send(rule, first)
+
+        alerter = es.SecurityOnionESAlerter(rule)
+        self.assertEqual(alerter.alert_id(first), alerter.alert_id(later))
+        self.assertNotEqual(alerter.alert_id(first), alerter.alert_id(dict(first, **{'@timestamp': '2026-09-30T18:09:00+00:00'})))
+        self.assertEqual(alert_id, alerter.alert_id(first))
+        self.assertEqual(payload['event']['reason'], '3,561 events in 2 minutes')
+        self.assertNotIn('labels', payload)
+
+    def test_temporal_count_column(self):
+        rule = dict(BASE_RULE, sigma_correlation='temporal', summary_template=None)
+        match = {'event_type_count': 2, 'window_start': '2026-09-30T16:49:52+00:00', '@timestamp': '2026-09-30T16:50:00+00:00'}
+
+        payload, _ = self.send(rule, match)
+
+        self.assertEqual(payload['event']['reason'], '2 correlated rules matched in 8 seconds')
+
+    def test_grouped_correlation_id_unchanged(self):
+        """Ids of alerts already written must not change."""
+        rule = dict(BASE_RULE, compound_query_key=['host.name', 'source.ip'], query_key='host.name,source.ip')
+        key = f"{BASE_RULE['detection_public_id']}|2026-09-30T18:07:53+00:00|host-01|192.0.2.10"
+
+        self.assertEqual(es.SecurityOnionESAlerter(rule).alert_id(correlation_match()), es.hashlib.sha256(key.encode()).hexdigest())
+
+    def test_rejected_event_data_is_kept_as_text(self):
+        rule = dict(BASE_RULE, query_key='user.name', summary_template='%count% failed SOC logins for %user.name%')
+        match = {'event_count': 3, 'window_start': '2026-09-30T16:49:52+00:00', '@timestamp': '2026-09-30T16:50:00+00:00',
+                 'user': {'name': 'user@example.invalid'}}
+        rejected = RequestError(400, 'document_parsing_exception', {})
+
+        (first, _), (second, _) = self.creates(rule, match, [rejected, None])
+
+        self.assertIn('event_data', first)
+        self.assertNotIn('event_data', second)
+        self.assertEqual(json.loads(second['event']['original']), match)
+        self.assertEqual(second['tags'], ['alert', 'preserve_original_event'])
+        self.assertEqual(first['tags'], ['alert'])
+        self.assertTrue(second['error']['message'].startswith('event_data rejected by Elasticsearch: '))
+        self.assertIn('document_parsing_exception', second['error']['message'])
+        # everything else carries over
+        self.assertEqual({k: v for k, v in second['event'].items() if k != 'original'}, first['event'])
+        changed = ('event_data', 'event', 'error', 'tags')
+        self.assertEqual({k: v for k, v in second.items() if k not in changed}, {k: v for k, v in first.items() if k not in changed})
+
+    def test_rejected_dates_are_kept_as_text(self):
+        match = {'@timestamp': datetime(2026, 9, 30, 16, 50, tzinfo=timezone.utc), '_id': 'abc'}
+
+        _, (second, _) = self.creates(PLAIN_RULE, match, [RequestError(400, 'document_parsing_exception', {}), None])
+
+        self.assertEqual(json.loads(second['event']['original']), {'@timestamp': '2026-09-30T16:50:00+00:00', '_id': 'abc'})
+
+    def test_rejected_twice_is_dropped_without_retry(self):
+        rejected = RequestError(400, 'document_parsing_exception', {})
+        match = {'@timestamp': '2026-09-30T16:50:00+00:00', '_id': 'abc'}
+
+        with self.assertLogs('elastalert', 'ERROR') as logs:
+            self.assertEqual(len(self.creates(PLAIN_RULE, match, [rejected, rejected])), 2)
+
+        self.assertIn('Dropping alert', logs.output[0])
+
+    def test_write_failure_is_retried(self):
+        match = {'@timestamp': '2026-09-30T16:50:00+00:00', '_id': 'abc'}
+
+        with self.assertRaisesRegex(es.EAException, 'Unable to write the alert to Elasticsearch'):
+            self.creates(PLAIN_RULE, match, [ConnectionError('N/A', 'refused', None)])
+
+    def test_repeat_id_is_ignored(self):
+        match = {'@timestamp': '2026-09-30T16:50:00+00:00', '_id': 'abc'}
+
+        self.assertEqual(len(self.creates(PLAIN_RULE, match, [ConflictError(409, 'version_conflict_engine_exception', {})])), 1)
+
+    def test_correlation_fields_are_optional(self):
+        rule = dict(BASE_RULE, query_key='source.ip')
+        # no window_start: the summary cannot be built
+        match = {k: v for k, v in correlation_match().items() if k != 'window_start'}
+
+        with self.assertLogs('elastalert', 'WARNING') as logs:
+            payload, _ = self.send(rule, match)
+
+        self.assertIn('without its correlation summary', logs.output[0])
+        self.assertEqual(payload['event']['kind'], 'alert')
+        self.assertNotIn('reason', payload['event'])
+        self.assertNotIn('labels', payload)
+        self.assertNotIn('related', payload)
+
+    def test_unstable_id_still_writes(self):
+        # no @timestamp: the window end is unknown
+        match = {k: v for k, v in correlation_match().items() if k not in ('@timestamp', 'window_start')}
+
+        with self.assertLogs('elastalert', 'WARNING') as logs:
+            payload, alert_id = self.send(BASE_RULE, match)
+
+        self.assertIn('without a stable id', logs.output[0])
+        self.assertEqual(len(alert_id), 32)
+        self.assertEqual(payload['event_data'], match)
+
+    def test_summary_formats_values(self):
+        rule = dict(BASE_RULE, sigma_correlation='value_avg', query_key='source.ip',
+                    summary_template='%count% for %source.ip% to %destination.port% %no.such.field%')
+        match = {'value_avg': 2.5, 'window_start': '2026-09-30T16:49:52+00:00', '@timestamp': '2026-09-30T16:50:00+00:00',
+                 'source': {'ip': '192.0.2.10'}, 'destination': {'port': [22, 80, 443, 8080, 8443]}}
+
+        payload, _ = self.send(rule, match)
+
+        self.assertEqual(payload['event']['reason'], '2.50 for 192.0.2.10 to 22, 80, 443 and 2 more %no.such.field%')
+        self.assertEqual(es.SecurityOnionESAlerter.format_count('n/a'), 'n/a')
+
+    def test_get_info(self):
+        self.assertEqual(es.SecurityOnionESAlerter(PLAIN_RULE).get_info(), {'type': 'SecurityOnionESAlerter'})
@@ -120,7 +120,7 @@ elastalert:
        helpLink: elastalert
    old_query_limit:
      minutes:
-        description: Amount of time in minutes between queries to start at the most recently run query.
+        description: How long ElastAlert can be down, in minutes, and still resume each rule where it stopped. After a longer outage, rules restart from now and skip the gap.
        global: True
        helpLink: elastalert
    es_conn_timeout:
@@ -6,9 +6,11 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}

 include:
+  - ca.trust
  - elastic-fleet-package-registry.config
  - elastic-fleet-package-registry.sostatus

@@ -34,8 +36,9 @@ so-elastic-fleet-package-registry:
      {% for BINDING in DOCKERMERGED.containers['so-elastic-fleet-package-registry'].port_bindings %}
      - {{ BINDING }}
      {% endfor %}
-    {% if DOCKERMERGED.containers['so-elastic-fleet-package-registry'].custom_bind_mounts %}
    - binds:
+      - {{ CATRUST.pem }}:/etc/ssl/certs/ca-certificates.crt:ro
+    {% if DOCKERMERGED.containers['so-elastic-fleet-package-registry'].custom_bind_mounts %}
        {% for BIND in DOCKERMERGED.containers['so-elastic-fleet-package-registry'].custom_bind_mounts %}
      - {{ BIND }}
        {% endfor %}
@@ -52,6 +55,8 @@ so-elastic-fleet-package-registry:
      - {{ ULIMIT.name }}={{ ULIMIT.soft }}:{{ ULIMIT.hard }}
    {%   endfor %}
    {% endif %}
+    - watch:
+      - cmd: ca_trust_extract

 wait_for_so-elastic-fleet-package-registry:
  http.wait_for_successful_query:
@@ -6,6 +6,7 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}

 include:
@@ -37,6 +38,7 @@ so-elastic-agent:
      - {{ BINDING }}
      {% endfor %}
    - binds:
+      - {{ CATRUST.dir }}
      - /opt/so/conf/elastic-agent/elastic-agent.yml:/usr/share/elastic-agent/elastic-agent.yml:ro
      - /opt/so/log/elasticagent:/usr/share/elastic-agent/logs
      - /etc/pki/tls/certs/intca.crt:/etc/pki/tls/certs/intca.crt:ro 
@@ -65,6 +67,7 @@ so-elastic-agent:
      - file: create-elastic-agent-config
      - file: trusttheca
    - watch:
+      - cmd: ca_trust_extract
      - file: create-elastic-agent-config
      - file: trusttheca

@@ -6,6 +6,7 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'elasticfleet/map.jinja' import ELASTICFLEETMERGED %}

@@ -63,6 +64,7 @@ so-elastic-fleet:
      - {{ BINDING }}
      {% endfor %}
    - binds:
+      - {{ CATRUST.dir }}
      - /etc/pki/elasticfleet-server.crt:/etc/pki/elasticfleet-server.crt:ro
      - /etc/pki/elasticfleet-server.key:/etc/pki/elasticfleet-server.key:ro
      - /etc/pki/tls/certs/intca.crt:/etc/pki/tls/certs/intca.crt:ro
@@ -95,6 +97,7 @@ so-elastic-fleet:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: trusttheca
      - x509: etc_elasticfleet_key
      - x509: etc_elasticfleet_crt
@@ -30,6 +30,8 @@
    'azure_metrics.monitor': 'azure.monitor',
    'azure_metrics.storage_account': 'azure.storage_account',
    'azure_openai.metrics': 'azure.open_ai',
+    'enterprisesearch.health': 'enterprisesearch.stack_monitoring.health',
+    'enterprisesearch.stats': 'enterprisesearch.stack_monitoring.stats',
    'kibana.cluster_actions': 'kibana.stack_monitoring.cluster_actions',
    'kibana.cluster_rules': 'kibana.stack_monitoring.cluster_rules',
    'kibana.node_actions': 'kibana.stack_monitoring.node_actions',
@@ -1,49 +0,0 @@
-# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
-# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at 
-# https://securityonion.net/license; you may not use this file except in compliance with the
-# Elastic License 2.0.
-
-{% from 'allowed_states.map.jinja' import allowed_states %}
-{% if sls.split('.')[0] in allowed_states or sls in allowed_states %}
-{%   from 'vars/globals.map.jinja' import GLOBALS %}
-
-# Move our new CA over so Elastic and Logstash can use SSL with the internal CA
-catrustdir:
-  file.directory:
-    - name: /opt/so/conf/ca
-    - user: 939
-    - group: 939
-    - makedirs: True
-
-{%   if GLOBALS.is_manager %}
-# We have to add the Manager CA to the CA list
-catrustscript:
-  cmd.script:
-    - source: salt://elasticsearch/tools/sbin_jinja/so-catrust
-    - template: jinja
-    - cwd: /opt/so
-    - defaults:
-        GLOBALS: {{ GLOBALS }}
-{%   endif %}
-
-elasticsearch_cacerts:
-  file.managed:
-    - name: /opt/so/conf/ca/cacerts
-    - source: salt://elasticsearch/cacerts
-    - user: 939
-    - group: 939
-
-elasticsearch_capems:
-  file.managed:
-    - name: /opt/so/conf/ca/tls-ca-bundle.pem
-    - source: salt://elasticsearch/tls-ca-bundle.pem
-    - user: 939
-    - group: 939
-
-{% else %}
-
-{{sls}}_state_not_allowed:
-  test.fail_without_changes:
-    - name: {{sls}}_state_not_allowed
-
-{% endif %}
@@ -107,6 +107,12 @@ so-elasticsearch-system-indices-patch:
      - file: so-elasticsearch-system-indices-patch-script
 {%  endif %}

+so-elasticsearch-ilm-policy-reload:
+  file.absent:
+    - name: /opt/so/state/esilm.txt
+    - onchanges:
+      - file: so-elasticsearch-ilm-policy-load-script
+
 # heavynodes will only load ILM policies for SO managed indices. (Indicies defined in elasticsearch/defaults.yaml)
 so-elasticsearch-ilm-policy-load:
  cmd.run:
@@ -115,8 +121,9 @@ so-elasticsearch-ilm-policy-load:
    - require:
      - docker_container: so-elasticsearch
      - file: so-elasticsearch-ilm-policy-load-script
-    - onchanges:
-      - file: so-elasticsearch-ilm-policy-load-script
+      - http: wait_for_so-elasticsearch
+      - file: so-elasticsearch-ilm-policy-reload
+    - unless: test -f /opt/so/state/esilm.txt

 so-elasticsearch-templates-reload:
  file.absent:
@@ -1,7 +1,7 @@
 elasticsearch:
  enabled: false
  esheap: '600m'
-  version: 9.4.5
+  version: 9.4.8
  index_clean: true
  data_retention_method: DLM
  vm:
@@ -1160,6 +1160,7 @@ elasticsearch:
        - so-fleet_agent_id_verification-1
        - so-logs-mappings
        - so-logs-settings
+        - detections-alerts-mappings
        data_stream:
          allow_custom_routing: false
          hidden: false
@@ -6,6 +6,7 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'elasticsearch/config.map.jinja' import ELASTICSEARCH_NODES %}
 {%   from 'elasticsearch/config.map.jinja' import ELASTICSEARCH_SEED_HOSTS %}
@@ -13,7 +14,6 @@

 include:
  - ca
-  - elasticsearch.ca
  - elasticsearch.ssl
  - elasticsearch.config
  - elasticsearch.sostatus
@@ -63,15 +63,14 @@ so-elasticsearch:
      - {{ BINDING }}
      {% endfor %}
    - binds:
+      - {{ CATRUST.dir }}
      - /opt/so/conf/elasticsearch/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml:ro
      - /opt/so/conf/elasticsearch/log4j2.properties:/usr/share/elasticsearch/config/log4j2.properties:ro
      - /nsm/elasticsearch:/usr/share/elasticsearch/data:rw
      - /opt/so/log/elasticsearch:/var/log/elasticsearch:rw
-      - /opt/so/conf/ca/cacerts:/usr/share/elasticsearch/jdk/lib/security/cacerts:ro
      - /etc/pki/tls/certs/intca.crt:/usr/share/elasticsearch/config/ca.crt:ro
      - /etc/pki/elasticsearch.crt:/usr/share/elasticsearch/config/elasticsearch.crt:ro
      - /etc/pki/elasticsearch.key:/usr/share/elasticsearch/config/elasticsearch.key:ro
-      - /etc/pki/elasticsearch.p12:/usr/share/elasticsearch/config/elasticsearch.p12:ro
      - /opt/so/conf/elasticsearch/users_roles:/usr/share/elasticsearch/config/users_roles:ro
      - /opt/so/conf/elasticsearch/users:/usr/share/elasticsearch/config/users:ro
      {% if ELASTICSEARCHMERGED.config.path.get('repo', False) %}
@@ -85,21 +84,19 @@ so-elasticsearch:
        {% endfor %}
      {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: trusttheca
      - x509: elasticsearch_crt
      - x509: elasticsearch_key
-      - file: elasticsearch_cacerts
      - file: esyml
    - require:
      - file: trusttheca
      - x509: elasticsearch_crt
      - x509: elasticsearch_key
-      - file: elasticsearch_cacerts
      - file: esyml
      - file: eslog4jfile
      - file: nsmesdir
      - file: eslogdir
-      - file: elasticp12perms
      - cmd: auth_users_roles_inode
      - cmd: auth_users_inode

@@ -1,5 +1,5 @@
 {
-    "version": 3,
+    "version": 4,
    "_meta": {
        "managed_by": "securityonion",
        "managed": true
@@ -177,6 +177,32 @@
                "description": "Extract IPs from Elastic Agent events (host.ip) and adds them to related.ip"
            }
        },
+        {
+            "trim": {
+                "if": "ctx.event?.dataset == 'system.auth'",
+                "description": "Grok leaves a leading space on 'invalid user' names (elastic/integrations#12174) and, before 2.23.2, sudo padding",
+                "field": "user.name",
+                "ignore_missing": true,
+                "ignore_failure": true
+            }
+        },
+        {
+            "trim": {
+                "if": "ctx.event?.dataset == 'system.auth'",
+                "description": "Appended from the untrimmed user.name",
+                "field": "related.user",
+                "ignore_missing": true,
+                "ignore_failure": true
+            }
+        },
+        {
+            "script": {
+                "if": "ctx.event?.dataset == 'system.auth' && ctx.related?.user instanceof List",
+                "description": "Dedupe after trimming",
+                "source": "ctx.related.user = new ArrayList(new LinkedHashSet(ctx.related.user));",
+                "ignore_failure": true
+            }
+        },
        {
            "script": {
                "description": "Snapshot event.ingested into _tmp.event_ingested_pre_fleet before .fleet_final_pipeline-1 overwrites it with ES ingest time",
@@ -38,10 +38,6 @@ elasticsearch_crt:
    - retry:
        attempts: 5
        interval: 30
-  cmd.run:
-    - name: "/usr/bin/openssl pkcs12 -inkey /etc/pki/elasticsearch.key -in /etc/pki/elasticsearch.crt -export -out /etc/pki/elasticsearch.p12 -nodes -passout pass:"
-    - onchanges:
-      - x509: /etc/pki/elasticsearch.key

 elastickeyperms:
  file.managed:
@@ -50,13 +46,6 @@ elastickeyperms:
    - mode: 640
    - group: 930
    
-elasticp12perms:
-  file.managed:
-    - replace: False
-    - name: /etc/pki/elasticsearch.p12
-    - mode: 640
-    - group: 930
-
 {% else %}

 {{sls}}_state_not_allowed:
@@ -154,6 +154,20 @@
            }
          }
        }
+      },
+      "runtime": {
+        "so_detection.sigmaKind": {
+          "type": "keyword",
+          "script": {
+            "source": "if (doc['so_detection.language'].size() == 0 || doc['so_detection.language'].value != 'sigma') { return; } def d = params._source.so_detection; if (d == null || d.content == null) { return; } String c = d.content; String NL = String.valueOf((char) 10); emit(c.startsWith('correlation:') || c.contains(NL + 'correlation:') ? 'correlation' : 'single');"
+          }
+        },
+        "so_detection.sigmaCorrelationType": {
+          "type": "keyword",
+          "script": {
+            "source": "if (doc['so_detection.language'].size() == 0 || doc['so_detection.language'].value != 'sigma') { return; } def d = params._source.so_detection; if (d == null || d.content == null) { return; } String c = d.content; String NL = String.valueOf((char) 10); int i = c.startsWith('correlation:') ? 0 : c.indexOf(NL + 'correlation:'); if (i < 0) { return; } i = c.indexOf(NL, i + 1); while (i >= 0 && i + 1 < c.length()) { int e = c.indexOf(NL, i + 1); String line = e < 0 ? c.substring(i + 1) : c.substring(i + 1, e); if (line.length() > 0 && !Character.isWhitespace(line.charAt(0)) && line.charAt(0) != (char) 35) { return; } String t = line.trim(); if (t.startsWith('type:')) { String v = t.substring(5).trim(); int h = v.indexOf(' ' + String.valueOf((char) 35)); if (h >= 0) { v = v.substring(0, h).trim(); } if (v.length() > 1 && (v.charAt(0) == (char) 39 || v.charAt(0) == (char) 34)) { v = v.substring(1, v.length() - 1); } emit(v); return; } i = e; }"
+          }
+        }
      }
    }
  },
@@ -0,0 +1,149 @@
+{
+  "template": {
+    "mappings": {
+      "properties": {
+        "tags": {
+          "ignore_above": 1024,
+          "type": "keyword"
+        },
+        "sigma_level": {
+          "ignore_above": 1024,
+          "type": "keyword"
+        },
+        "rule": {
+          "properties": {
+            "name": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
+            "uuid": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
+            "category": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
+            "product": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
+            "service": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
+            "correlation": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            }
+          }
+        },
+        "event": {
+          "properties": {
+            "severity": {
+              "type": "long"
+            },
+            "severity_label": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
+            "module": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
+            "dataset": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
+            "kind": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
+            "reason": {
+              "type": "match_only_text",
+              "fields": {
+                "keyword": {
+                  "ignore_above": 1024,
+                  "type": "keyword"
+                }
+              }
+            },
+            "original": {
+              "type": "keyword",
+              "index": false,
+              "doc_values": false
+            }
+          }
+        },
+        "event_data": {
+          "properties": {
+            "@timestamp": {
+              "type": "date"
+            },
+            "window_start": {
+              "type": "date"
+            },
+            "event_count": {
+              "type": "long"
+            },
+            "value_count": {
+              "type": "long"
+            },
+            "event_type_count": {
+              "type": "long"
+            },
+            "value_sum": {
+              "type": "double"
+            },
+            "value_avg": {
+              "type": "double"
+            },
+            "value_percentile": {
+              "type": "double"
+            },
+            "value_median": {
+              "type": "double"
+            }
+          }
+        },
+        "labels": {
+          "properties": {
+            "correlation_group_by": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
+            "correlation_group": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            }
+          }
+        },
+        "related": {
+          "properties": {
+            "ip": {
+              "type": "ip"
+            },
+            "user": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
+            "hosts": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            }
+          }
+        },
+        "error": {
+          "properties": {
+            "message": {
+              "type": "match_only_text"
+            }
+          }
+        }
+      }
+    }
+  },
+  "_meta": {
+    "description": "Fields written by the ElastAlert SecurityOnionESAlerter to logs-detections.alerts-so"
+  }
+}
@@ -1,26 +0,0 @@
-#!/bin/bash
-#
-# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
-# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at 
-# https://securityonion.net/license; you may not use this file except in compliance with the
-# Elastic License 2.0.
-
-. /usr/sbin/so-common
-get_elastic_agent_vars
-
-# Exit on errors, since all lines must succeed
-set -e
-
-# Check to see if we have extracted the ca cert.
-if [ ! -f /opt/so/saltstack/local/salt/elasticsearch/cacerts ]; then
-    docker run -v /etc/pki/ca.crt:/etc/ssl/ca.crt --name so-elasticsearchca --user root --entrypoint jdk/bin/keytool {{ GLOBALS.registry_host }}:5000/{{ GLOBALS.image_repo }}/so-elasticsearch:$ELASTIC_AGENT_TARBALL_VERSION -keystore /usr/share/elasticsearch/jdk/lib/security/cacerts -alias SOSCA -import -file /etc/ssl/ca.crt -storepass changeit -noprompt
-    # Make sure symbolic links are followed when copying from container
-    docker cp -L so-elasticsearchca:/usr/share/elasticsearch/jdk/lib/security/cacerts /opt/so/saltstack/local/salt/elasticsearch/cacerts
-    docker cp -L so-elasticsearchca:/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem /opt/so/saltstack/local/salt/elasticsearch/tls-ca-bundle.pem
-    docker rm so-elasticsearchca
-    echo "" >> /opt/so/saltstack/local/salt/elasticsearch/tls-ca-bundle.pem
-    echo "sosca" >> /opt/so/saltstack/local/salt/elasticsearch/tls-ca-bundle.pem
-    cat /etc/pki/ca.crt >> /opt/so/saltstack/local/salt/elasticsearch/tls-ca-bundle.pem
-else
-    exit 0
-fi
@@ -7,11 +7,17 @@
 . /usr/sbin/so-common

 MAX_JOBS=${MAX_ILM_JOBS:-10}
+ILM_STATEFILE=/opt/so/state/esilm.txt
+rm -f "$ILM_STATEFILE"

+check_elasticsearch_responsive
+
+ILM_TMP_DIR=$(mktemp -d)
+trap 'rm -rf "$ILM_TMP_DIR"' EXIT
 # Lock used to serialize block writes so concurrent jobs never interleave their output.
-ILM_OUTPUT_LOCK=$(mktemp)
-ILM_FAIL_FILE=$(mktemp)
-trap 'rm -f "$ILM_OUTPUT_LOCK" "$ILM_FAIL_FILE"' EXIT
+ILM_OUTPUT_LOCK="$ILM_TMP_DIR/output.lock"
+ILM_FAIL_FILE="$ILM_TMP_DIR/failed"
+touch "$ILM_FAIL_FILE"

 # Policies are loaded concurrently (up to MAX_JOBS at a time) for speed. Each policy's block is
 # printed the moment its curl returns, so output appears in COMPLETION ORDER, not the order
@@ -20,13 +26,30 @@ echo "Loading ILM policies concurrently; output below appears in completion orde
 echo

 put_policy() {
-  local desc="$1" policyname="$2" data="$3" result rc=0
-  if ! result=$(curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L --fail \
-    -X PUT "https://localhost:9200/_ilm/policy/${policyname}" \
-    -H 'Content-Type: application/json' -d"${data}" 2>&1); then
-    rc=1
-  elif ! jq -e '.acknowledged == true' <<<"$result" >/dev/null 2>&1; then
-    rc=1
+  local desc="$1" policyname="$2" data="$3" rc=1 attempt out http_code body result reason tmpdata
+  tmpdata=$(mktemp "$ILM_TMP_DIR/${policyname}.XXXXXX.json")
+  printf '%s' "$data" >"$tmpdata"
+
+  for attempt in 1 2 3; do
+    out=$(so-elasticsearch-query "_ilm/policy/${policyname}" -d@"${tmpdata}" -XPUT -w '\n%{http_code}' 2>/dev/null)
+    http_code=${out##*$'\n'} body=${out%$'\n'*}
+
+    if [[ "$http_code" == 200 ]] && jq -e '.acknowledged == true' <<<"$body" >/dev/null 2>&1; then
+      rc=0
+      break
+    fi
+    # 429 = ES rejected/throttled (may succeed later); other 4xx will not
+    [[ "$http_code" == 4?? && "$http_code" != 429 ]] && break
+    (( attempt < 3 )) && sleep 3
+  done
+
+  if (( rc == 0 )); then
+    result='{"acknowledged":true}'
+  else
+    result=${body:-"HTTP ${http_code:-000}"}
+    # flatten tabs/newlines so the TAB fail-file stays one record
+    reason=$(jq -r '(if (.error|type) == "object" then (.error.reason // .error.type) elif (.error|type) == "string" then .error else empty end) // empty | tostring | gsub("[\t\r\n]+"; " ")' <<<"$body" 2>/dev/null)
+    reason="HTTP ${http_code:-000}${reason:+: ${reason}}"
  fi

  # curl above ran in parallel; serialize just this block write so concurrent jobs never interleave.
@@ -34,7 +57,7 @@ put_policy() {
    flock 200
    printf 'Setting up %s policy...\n%s\n\n' "${desc}" "${result}"
    if (( rc != 0 )); then
-      printf '%s\n' "${policyname}" >>"$ILM_FAIL_FILE"
+      printf '%s\t%s\n' "${policyname}" "${reason}" >>"$ILM_FAIL_FILE"
    fi
  } 200>>"${ILM_OUTPUT_LOCK}"

@@ -82,10 +105,15 @@ wait || true

 if [[ -s "$ILM_FAIL_FILE" ]]; then
  echo "ERROR: Failed to load ILM policy(s):"
-  while read -r POLICY; do
-    echo " - $POLICY"
+  while IFS=$'\t' read -r POLICY REASON; do
+    if [[ -n "$REASON" ]]; then
+      echo " - ${POLICY}: ${REASON}"
+    else
+      echo " - ${POLICY}"
+    fi
  done < "$ILM_FAIL_FILE"
  exit 1
 else
  echo "Successfully loaded all ILM policies."
+  touch "$ILM_STATEFILE"
 fi
@@ -13,9 +13,11 @@
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   if 'api' in salt['pillar.get']('features', []) %}

 include:
+  - ca.trust
  - hydra.config
  - hydra.sostatus

@@ -29,6 +31,7 @@ so-hydra:
      - soauth:
        - ipv4_address: {{ DOCKERMERGED.containers['so-hydra'].ips['soauth'] }}
    - binds:
+      - {{ CATRUST.dir }}
      - /opt/so/conf/hydra/:/hydra-conf:ro
      - /opt/so/log/hydra/:/hydra-log:rw
      - /nsm/hydra/db:/hydra-data:rw
@@ -60,6 +63,7 @@ so-hydra:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: hydraconfig
    - require:
      - file: hydraconfig
@@ -6,9 +6,11 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}

 include:
+  - ca.trust
  - idh.config
  - idh.sostatus

@@ -20,6 +22,7 @@ so-idh:
    - detach: True
    - network_mode: host
    - binds:
+      - {{ CATRUST.dir }}
      - /nsm/idh:/var/tmp:rw
      - /opt/so/conf/idh/http-skins:/opt/opencanary/http-skins:ro
      - /opt/so/conf/idh/opencanary.conf:/etc/opencanaryd/opencanary.conf:ro
@@ -34,12 +37,13 @@ so-idh:
      - {{ XTRAHOST }}
      {% endfor %}
    {% endif %}
-    {% if DOCKERMERGED.containers['so-idh'].extra_env %}
    - environment:
-      {% for XTRAENV in DOCKERMERGED.containers['so-idh'].extra_env %}
+      - REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
+      {% if DOCKERMERGED.containers['so-idh'].extra_env %}
+        {% for XTRAENV in DOCKERMERGED.containers['so-idh'].extra_env %}
      - {{ XTRAENV }}
-      {% endfor %}
-    {% endif %}
+        {% endfor %}
+      {% endif %}
    {% if DOCKERMERGED.containers['so-idh'].ulimits %}
    - ulimits:
    {%   for ULIMIT in DOCKERMERGED.containers['so-idh'].ulimits %}
@@ -47,6 +51,7 @@ so-idh:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: opencanary_config
    - require:
      - file: opencanary_config
@@ -6,11 +6,13 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   set PASSWORD = salt['pillar.get']('secrets:influx_pass') %}
 {%   set TOKEN = salt['pillar.get']('influxdb:token') %}

 include:
+  - ca.trust
  - influxdb.ssl
  - influxdb.config
  - influxdb.sostatus
@@ -38,6 +40,7 @@ so-influxdb:
        {% endfor %}
      {% endif %}
    - binds:
+      - {{ CATRUST.pem }}:/etc/ssl/certs/ca-certificates.crt:ro
      - /opt/so/log/influxdb/:/log:rw
      - /opt/so/conf/influxdb/config.yaml:/conf/config.yaml:ro
      - /opt/so/conf/influxdb/etc:/etc/influxdb2:rw
@@ -66,6 +69,7 @@ so-influxdb:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: influxdbconf
      - x509: influxdb_key
      - x509: influxdb_crt
@@ -12,12 +12,14 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   set KAFKANODES = salt['pillar.get']('kafka:nodes') %}
 {%   set KAFKA_EXTERNAL_ACCESS = salt['pillar.get']('kafka:config:external_access:enabled', default=False) %}
 {%   if 'gmd' in salt['pillar.get']('features', []) %}

 include:
+  - ca.trust
  - kafka.ca
  - kafka.config
  - kafka.ssl
@@ -51,6 +53,7 @@ so-kafka:
      - {{ BINDING }}
      {% endfor %}
    - binds:
+      - {{ CATRUST.java }}:/opt/java/openjdk/lib/security/cacerts:ro
      - /etc/pki/kafka.p12:/etc/pki/kafka.p12:ro
      - /opt/so/conf/kafka/kafka-truststore.jks:/etc/pki/kafka-truststore.jks:ro
      - /nsm/kafka/data/:/nsm/kafka/data/:rw
@@ -61,6 +64,11 @@ so-kafka:
      {% if KAFKA_EXTERNAL_ACCESS %}
      - /opt/so/conf/kafka/kafka_server_jaas.conf:/opt/kafka/config/kafka_server_jaas.conf:ro
      {% endif %}
+      {% if DOCKERMERGED.containers['so-kafka'].custom_bind_mounts %}
+        {% for BIND in DOCKERMERGED.containers['so-kafka'].custom_bind_mounts %}
+      - {{ BIND }}
+        {% endfor %}
+      {% endif %}
    {% if DOCKERMERGED.containers['so-kafka'].ulimits %}
    - ulimits:
    {%   for ULIMIT in DOCKERMERGED.containers['so-kafka'].ulimits %}
@@ -68,6 +76,7 @@ so-kafka:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      {% for sc in ['server', 'client'] %}
      - file: kafka_kraft_{{sc}}_properties
      {% endfor %}
@@ -22,7 +22,7 @@ kibana:
          - default
          - file
    migrations:
-      discardCorruptObjects: "9.4.5"
+      discardCorruptObjects: "9.4.8"
    telemetry:
      enabled: False
    xpack:
@@ -8,8 +8,10 @@
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'elasticsearch/config.map.jinja' import ELASTICSEARCHMERGED %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}

 include:
+  - ca.trust
  - kibana.config
  - kibana.sostatus

@@ -27,6 +29,7 @@ so-kibana:
      - ELASTICSEARCH_HOST={{ GLOBALS.manager }}
      - ELASTICSEARCH_PORT=9200
      - MANAGER={{ GLOBALS.manager }}
+      - NODE_OPTIONS=--use-openssl-ca
      {% if DOCKERMERGED.containers['so-kibana'].extra_env %}
        {% for XTRAENV in DOCKERMERGED.containers['so-kibana'].extra_env %}
      - {{ XTRAENV }}
@@ -40,6 +43,7 @@ so-kibana:
      {% endfor %}
    {% endif %}
    - binds:
+      - {{ CATRUST.dir }}
      - /opt/so/conf/kibana/etc:/usr/share/kibana/config:rw
      - /opt/so/log/kibana:/var/log/kibana:rw
      - /opt/so/conf/kibana/customdashboards:/usr/share/kibana/custdashboards:ro
@@ -60,6 +64,7 @@ so-kibana:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: kibanaconfig

 wait_for_so-kibana:
@@ -7,8 +7,10 @@
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}

 include:
+  - ca.trust
  - kratos.config
  - kratos.sostatus

@@ -22,6 +24,7 @@ so-kratos:
      - soauth:
        - ipv4_address: {{ DOCKERMERGED.containers['so-kratos'].ips['soauth'] }}
    - binds:
+      - {{ CATRUST.dir }}
      - /opt/so/conf/kratos/:/kratos-conf:ro
      - /opt/so/log/kratos/:/kratos-log:rw
      - /nsm/kratos/db:/kratos-data:rw
@@ -53,6 +56,7 @@ so-kratos:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: kratosschema
      - file: kratosconfig
    - require:
@@ -6,6 +6,7 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'logstash/map.jinja' import LOGSTASH_MERGED %}
 {%   from 'logstash/map.jinja' import LOGSTASH_NODES %}
@@ -13,9 +14,6 @@

 include:
  - ca
-{%   if GLOBALS.role not in ['so-receiver','so-fleet'] %}
-  - elasticsearch.ca
-{%   endif %}
 {# Kafka ca runs on nodes that can run logstash for Kafka input / output. Only when Kafka is global pipeline #}
 {%   if GLOBALS.role in ['so-searchnode', 'so-manager', 'so-managerhype', 'so-managersearch', 'so-receiver', 'so-standalone'] and GLOBALS.pipeline == 'KAFKA' %}
  - kafka.ca
@@ -58,6 +56,8 @@ so-logstash:
      - {{ BINDING }}
      {% endfor %}
    - binds:
+      - {{ CATRUST.dir }}
+      - {{ CATRUST.java }}:/usr/share/logstash/jdk/lib/security/cacerts:ro
      - /opt/so/conf/elasticsearch/templates/:/templates/:ro
      - /opt/so/conf/logstash/etc/:/usr/share/logstash/config/:ro
      - /opt/so/conf/logstash/pipelines:/usr/share/logstash/pipelines:ro
@@ -78,10 +78,6 @@ so-logstash:
      - /etc/pki/filebeat.p8:/usr/share/logstash/filebeat.key:ro
          {% endif %}
      {% endif %}
-      {% if GLOBALS.role not in ['so-receiver','so-fleet'] %}
-      - /opt/so/conf/ca/cacerts:/etc/pki/ca-trust/extracted/java/cacerts:ro
-      - /opt/so/conf/ca/tls-ca-bundle.pem:/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem:ro
-      {% endif %}
      {% if GLOBALS.pipeline == "KAFKA" and GLOBALS.role in ['so-manager', 'so-managerhype', 'so-managersearch', 'so-standalone', 'so-searchnode'] %}
      - /etc/pki/kafka-logstash.p12:/usr/share/logstash/kafka-logstash.p12:ro
      - /opt/so/conf/kafka/kafka-truststore.jks:/etc/pki/kafka-truststore.jks:ro
@@ -104,14 +100,11 @@ so-logstash:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: lsetcsync
      - file: lslog4j2
      - file: lspipelinesyml
      - file: trusttheca
-      {% if GLOBALS.is_manager %}
-      - file: elasticsearch_cacerts
-      - file: elasticsearch_capems
-      {% endif %}
      {% if GLOBALS.is_manager or GLOBALS.role in ['so-fleet', 'so-heavynode', 'so-receiver'] %}
      - x509: etc_elasticfleet_logstash_crt
      - x509: etc_elasticfleet_logstash_key
@@ -133,10 +126,6 @@ so-logstash:
      {% endif %}
    - require:
      - file: trusttheca
-      {% if GLOBALS.is_manager %}
-      - file: elasticsearch_cacerts
-      - file: elasticsearch_capems
-      {% endif %}
      {% if GLOBALS.is_manager or GLOBALS.role in ['so-fleet', 'so-heavynode', 'so-receiver'] %}
      - x509: etc_elasticfleet_logstash_crt
      - x509: etc_elasticfleet_logstash_key
@@ -121,9 +121,6 @@ check_err() {
      161)
        echo 'Required intermediate Elasticsearch upgrade not complete'
      ;;
-      162)
-        echo 'One or more Elastic Agent nodes do not support the x86-64-v3 CPU instruction set'
-      ;;
      170)
        echo "Intermediate upgrade completed successfully to $next_step_so_version, but next soup to Security Onion $originally_requested_so_version could not be started automatically."
        echo "Start soup again manually to continue the upgrade to Security Onion $originally_requested_so_version."
@@ -351,83 +348,6 @@ check_cluster_health() {
  exit 0
 }

-no_soup_for_you() {
-  echo ""
-  echo "No soup for you!"
-  exit 162
-}
-
-check_cpu_compatibility() {
-  # Roles running a container built from the so-elastic-agent image; mirrors the
-  # elasticagent and elasticfleet entries in salt/reactor/pillar_push_map.yaml.
-  local cpu_target='G@role:so-heavynode or G@role:so-eval or G@role:so-fleet or G@role:so-import or G@role:so-manager or G@role:so-managerhype or G@role:so-managersearch or G@role:so-standalone'
-  local expected_nodes cpu_results node result confirm
-  local -a unsupported=() offline=()
-
-  echo "Checking that Elastic Agent nodes support the x86-64-v3 CPU instruction set now required by Elastic."
-
-  if [[ "$SKIP_CPU_CHECK" == "true" ]]; then
-    printf "\nSkipping the x86-64-v3 CPU check because --skip-cpu-check was specified.\n\n"
-    return
-  fi
-
-  # Nodes that never answer are absent from the results, so diff against who should have.
-  expected_nodes=$(salt -C "$cpu_target" --preview-target --out=json 2>/dev/null | jq -r '.[]?') || true
-  if [[ -z "$expected_nodes" ]]; then
-    printf "\nCould not determine which nodes run the Elastic Agent, so the x86-64-v3 CPU check cannot run.\n"
-    no_soup_for_you
-  fi
-
-  cpu_results=$(salt -t 30 -C "$cpu_target" cmd.run "/lib64/ld-linux-x86-64.so.2 --help | grep x86-64-v3" --out=json 2>/dev/null) || true
-
-  while IFS= read -r node; do
-    [[ -z "$node" ]] && continue
-    result=$(jq -r --arg node "$node" '.[$node] // empty' <<< "$cpu_results" 2>/dev/null)
-    if [[ -z "$result" || "$result" == *"did not return"* ]]; then
-      offline+=("$node")
-    elif [[ "$result" != *"x86-64-v3 (supported"* ]]; then
-      # glibc appends "(supported, searched)" only when supported; the open paren keeps
-      # this from matching a future "(unsupported".
-      unsupported+=("$node")
-    fi
-  done <<< "$expected_nodes"
-
-  if [[ ${#unsupported[@]} -eq 0 && ${#offline[@]} -eq 0 ]]; then
-    printf "\nAll Elastic Agent nodes support x86-64-v3. We can proceed with SOUP.\n\n"
-    return
-  fi
-
-  echo ""
-  if [[ ${#unsupported[@]} -gt 0 ]]; then
-    echo "The following node(s) do NOT support the x86-64-v3 CPU instruction set:"
-    printf '  %s\n' "${unsupported[@]}"
-    echo ""
-    echo "Upstream Elastic now builds its binaries for x86-64-v3, so these nodes can no"
-    echo "longer run Elastic. Upgrading them WILL BREAK them."
-    echo ""
-  fi
-  if [[ ${#offline[@]} -gt 0 ]]; then
-    echo "The following node(s) did not respond and could not be checked:"
-    printf '  %s\n' "${offline[@]}"
-    echo ""
-    echo "These nodes are offline, so we cannot confirm they support x86-64-v3, which"
-    echo "upstream Elastic now requires."
-    echo ""
-  fi
-
-  if [[ -n $UNATTENDED ]]; then
-    echo "Unattended mode cannot prompt for an override. Re-run soup interactively, or pass --skip-cpu-check to bypass this check."
-    no_soup_for_you
-  fi
-
-  read -rp "Type 'override' to continue anyway, or press Enter to exit: " confirm
-  if [[ "${confirm,,}" == "override" ]]; then
-    printf "\nOverride accepted. Continuing at your own risk.\n\n"
-  else
-    no_soup_for_you
-  fi
-}
-
 check_fleet_server() {
  echo "Checking that Elastic Fleet Server is responding."
  # Modeled on the wait_for_so-elastic-fleet state check in elasticfleet/enabled.sls,
@@ -626,6 +546,7 @@ postupgrade_changes() {
    [[ "$POSTVERSION" == "3.1.0" ]] && post_to_3.2.0
    [[ "$POSTVERSION" == "3.2.0" ]] && post_to_3.3.0
    [[ "$POSTVERSION" == "3.3.0" ]] && post_to_3.4.0
+    [[ "$POSTVERSION" == "3.4.0" ]] && post_to_3.5.0
    # All applicable post-upgrade steps completed; clear the resume marker.
    rm -f "$POSTVERSION_FILE"
    true
@@ -1131,9 +1052,6 @@ post_to_3.2.0() {

 ### 3.3.0 Scripts ###
 up_to_3.3.0() {
-  # download 9.4.5 elastic agent packages
-  determine_elastic_agent_upgrade
-  
  # remove existing (patched) elasticsearch index template to match integration naming change
  if ! remove_elasticsearch_index_template "so-logs-sentinel_one_cloud_funnel.login" "sentinel_one_cloud_funnel.login changed to sentinel_one_cloud_funnel.logins"; then
    FINAL_MESSAGE_QUEUE+=("WARNING: Unable to automatically remove the so-logs-sentinel_one_cloud_funnel.login index template. This step can be performed manually using the following command:")
@@ -1148,28 +1066,38 @@ telegraf_repair() {
  # when there is something to repair, so everyone else is left alone.
  local repair=/usr/sbin/so-telegraf-repair
  [[ -x "$repair" ]] || return 0
-  docker ps --format '{{.Names}}' | grep -qx so-postgres || return 0
+  local state
+  state=$(docker inspect -f '{{.State.Running}} {{.State.Restarting}}' so-postgres 2>/dev/null) || return 0
+  if [[ "$state" != "true false" ]]; then
+    echo "  so-postgres is not running; skipping the Telegraf partition check."
+    FINAL_MESSAGE_QUEUE+=("WARNING: so-postgres is not running, so Telegraf partitions were not checked. If /nsm is full, follow \"Telegraf Partition Repair\" at:")
+    FINAL_MESSAGE_QUEUE+=("    - https://securityonion.net/docs/postgresql")
+    return 0
+  fi

  echo "Checking Telegraf metric partitions."
-  local status=0
-  "$repair" --check >> "$SOUP_LOG" 2>&1 || status=$?
+  local out status=0
+  out=$("$repair" --check 2>&1) || status=$?
+  echo "$out" >> "$SOUP_LOG"
  case "$status" in
    0) echo "  Telegraf partitions are healthy; nothing to repair." ;;
    1) echo "  Repairing stalled Telegraf partitions."
-       "$repair" --yes \
-         || echo "  warning: so-telegraf-repair failed; run it manually" >&2 ;;
-    *) echo "  Skipping; Telegraf is not storing metrics in Postgres on this host." ;;
+       if ! "$repair" --yes; then
+         echo "  warning: so-telegraf-repair failed; run it manually" >&2
+         FINAL_MESSAGE_QUEUE+=("WARNING: Telegraf partition repair failed; Postgres metrics will keep filling /nsm until it is repaired:")
+         FINAL_MESSAGE_QUEUE+=("    - sudo so-telegraf-repair")
+       fi ;;
+    2) echo "  Skipping: $(tail -n 1 <<<"$out")" ;;
+    *) echo "  warning: could not check Telegraf partitions" >&2
+       FINAL_MESSAGE_QUEUE+=("WARNING: Telegraf partitions could not be checked during the upgrade; run:")
+       FINAL_MESSAGE_QUEUE+=("    - sudo so-telegraf-repair --check") ;;
  esac
 }

 post_to_3.3.0() {
  # Recollate again since some internal DBs were excluded during 3.2.0 soup
  recollate_postgres
-
-  # Generate 9.4.5 elastic agent installers
-  echo "Regenerating Elastic Agent Installers"
-  /sbin/so-elastic-agent-gen-installers
-
+  
  telegraf_repair

  set_postversion 3.3.0
@@ -1178,6 +1106,9 @@ post_to_3.3.0() {

 ### 3.4.0 Scripts ###
 up_to_3.4.0() {
+  # download 9.4.8 elastic agent packages
+  determine_elastic_agent_upgrade
+  
  set_soauth_range

  echo "Removing so-kratos, so-hydra and so-soc so they are recreated on the soauth network."
@@ -1255,14 +1186,46 @@ valid_soauth_range() {
 }

 post_to_3.4.0() {
+  # Generate 9.4.8 elastic agent installers
+  echo "Regenerating Elastic Agent Installers"
+  /sbin/so-elastic-agent-gen-installers
+  
  for idx in "metrics-logstash.node-default" "metrics-logstash.stack_monitoring.node-default"; do
    rollover_index "$idx"
  done

+  add_detection_runtime_fields
+
+  remove_frozen_ca_bundles
+
  set_postversion 3.4.0
 }
+
+remove_frozen_ca_bundles() {
+  rm -f /opt/so/saltstack/local/salt/elasticsearch/cacerts /opt/so/saltstack/local/salt/elasticsearch/tls-ca-bundle.pem >> $SOUP_LOG 2>&1
+}
+
+add_detection_runtime_fields() {
+  local template=$DEFAULT_SALT_DIR/salt/elasticsearch/templates/component/so/detection-mappings.json
+  local runtime
+
+  runtime=$(jq -c '{runtime: .template.mappings.runtime}' "$template")
+  if ! so-elasticsearch-query so-detection/_mapping -XPUT -d "$runtime" --retry 3 --retry-delay 5 --fail >> $SOUP_LOG 2>&1; then
+    FINAL_MESSAGE_QUEUE+=("WARNING: Unable to add the Sigma search fields to the so-detection index. Searches on so_detection.sigmaKind will match nothing until they are added:")
+    FINAL_MESSAGE_QUEUE+=("    - sudo so-elasticsearch-query so-detection/_mapping -XPUT -d \"\$(jq -c '{runtime: .template.mappings.runtime}' $template)\"")
+  fi
+}
 ### 3.4.0 End ###

+### 3.5.0 Scripts ###
+
+post_to_3.5.0() {
+  telegraf_repair
+
+  set_postversion 3.5.0
+}
+### 3.5.0 End ###
+

 repo_sync() {
  echo "Sync the local repo."
@@ -1535,9 +1498,10 @@ verify_es_version_compatibility() {
      ["8.18.4"]="8.18.6 8.18.8 9.0.8"
 	    ["8.18.6"]="8.18.8 9.0.8"
 	    ["8.18.8"]="9.0.8"
-      ["9.0.8"]="9.3.3 9.3.7 9.4.5"
-      ["9.3.3"]="9.3.7 9.4.5"
-      ["9.3.7"]="9.4.5"
+      ["9.0.8"]="9.3.3 9.3.7 9.4.5 9.4.8"
+      ["9.3.3"]="9.3.7 9.4.5 9.4.8"
+      ["9.3.7"]="9.4.5 9.4.8"
+      ["9.4.5"]="9.4.8"
    )

    # Elasticsearch MUST upgrade through these versions
@@ -2148,8 +2112,6 @@ main() {
  echo "Let's see if we need to update Security Onion."
  upgrade_check

-  check_cpu_compatibility
-
  upgrade_space

  echo "Verifying Elasticsearch version compatibility across the grid before upgrading."
@@ -2428,17 +2390,6 @@ fi
  echo "### soup has been served at $(date) ###"
 }

-SKIP_CPU_CHECK=false
-declare -a SOUP_ARGS=()
-for arg in "$@"; do
-  if [[ "$arg" == "--skip-cpu-check" ]]; then
-    SKIP_CPU_CHECK=true
-  else
-    SOUP_ARGS+=("$arg")
-  fi
-done
-set -- "${SOUP_ARGS[@]}"
-
 while getopts ":b:f:y" opt; do
  case ${opt} in
    b )
@@ -2462,7 +2413,7 @@ while getopts ":b:f:y" opt; do
      ISOLOC="$OPTARG"
    ;;
    \? )
-      echo "Usage: soup [-b] [-y] [-f <iso location>] [--skip-cpu-check]"
+      echo "Usage: soup [-b] [-y] [-f <iso location>]"
      exit 1
    ;;
    : )
@@ -6,10 +6,12 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'nginx/map.jinja' import NGINXMERGED %}

 include:
+  - ca.trust
  - nginx.ssl
  - nginx.config
  - nginx.sostatus
@@ -47,13 +49,14 @@ so-nginx:
      {% endfor %}
    {% endif %}
    - binds:
+      - {{ CATRUST.pem }}:/etc/ssl/certs/ca-certificates.crt:ro
      - /opt/so/conf/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
      - /opt/so/log/nginx/:/var/log/nginx:rw
      - /opt/so/tmp/nginx/:/var/lib/nginx:rw
      - /opt/so/tmp/nginx/:/run:rw
-      - /nsm/elastic-fleet/so_agent-installers/:/opt/socore/html/packages
-      - /nsm/elastic-fleet/artifacts/:/opt/socore/html/artifacts 
+      - /nsm/elastic-fleet/artifacts/:/opt/socore/elastic-artifacts/artifacts:ro
      {% if GLOBALS.is_manager %}
+      - /nsm/elastic-fleet/so_agent-installers/:/opt/socore/html/packages
      - /etc/pki/managerssl.crt:/etc/pki/nginx/server.crt:ro
      - /etc/pki/managerssl.key:/etc/pki/nginx/server.key:ro
      # ATT&CK Navigator binds
@@ -89,6 +92,7 @@ so-nginx:
      - {{ BINDING }}
      {% endfor %}
    - watch:
+      - cmd: ca_trust_extract
      - file: nginxconf
      - file: nginxconfdir
      {% if GLOBALS.is_manager %}
@@ -4,6 +4,8 @@
 {%- set role = grains.id.split('_') | last %}
 {%- set influxpass = salt['pillar.get']('secrets:influx_pass') %}
 {%- set influxauth = ('so:' + influxpass) | base64_encode %}
+{%- set all_names = [GLOBALS.hostname, GLOBALS.url_base] + NGINXMERGED.ssl.alt_names %}
+{%- set full_server_name = all_names | unique | join(' ') %}

 worker_processes auto;
 error_log /var/log/nginx/error.log;
@@ -39,52 +41,31 @@ http {

 	include /etc/nginx/conf.d/*.conf;

-	{%- if role in ['fleet'] %}
-
-	server {
-		listen 8443;
-		server_name {{ GLOBALS.hostname }};
-		root /opt/socore/html;
-		location /artifacts/ {
-			try_files $uri =206;
-			proxy_read_timeout    90;
-			proxy_connect_timeout 90;
-			proxy_set_header      Host $host;
-			proxy_set_header      X-Real-IP $remote_addr;
-			proxy_set_header      X-Forwarded-For $proxy_add_x_forwarded_for;
-			proxy_set_header      Proxy "";
-			proxy_set_header      X-Forwarded-Proto $scheme;
-		}
-	}
-
-    {%- endif %}
-
 	{%- if GLOBALS.is_manager %}
-	{%- set all_names = [GLOBALS.hostname, GLOBALS.url_base] + NGINXMERGED.ssl.alt_names %}
-	{%- set full_server_name = all_names | unique | join(' ') %}
-
 	server {
 		listen 80 default_server;
 		server_name _;
 		return 307 https://{{ GLOBALS.url_base }}$request_uri;
 	}
-
+  {%- endif %}
+  {%- if GLOBALS.is_manager or role == 'fleet' %}
 	server {
 		listen 8443;
+    {%- if GLOBALS.is_manager %}
 		server_name {{ full_server_name }};
-		root /opt/socore/html;
+    {%- else %}
+    server_name {{ GLOBALS.hostname }};
+    {%- endif %}
+		root /opt/socore/elastic-artifacts;
 		location /artifacts/ {
-			try_files $uri =206;
-			proxy_read_timeout    90;
-			proxy_connect_timeout 90;
-			proxy_set_header      Host $host;
-			proxy_set_header      X-Real-IP $remote_addr;
-			proxy_set_header      X-Forwarded-For $proxy_add_x_forwarded_for;
-			proxy_set_header      Proxy "";
-			proxy_set_header      X-Forwarded-Proto $scheme;
+			try_files $uri =404;
+		}
+		location / {
+			return 404;
 		}
 	}
-
+  {%- endif %}
+  {%- if GLOBALS.is_manager %}
 	server {
 		listen 443 ssl default_server;
 		http2 on;
@@ -17,3 +17,4 @@ postgres:
    log_min_messages: 'warning'
    shared_preload_libraries: pg_cron
    cron.database_name: postgres
+    cron.host: /run/postgresql
@@ -6,10 +6,12 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   set SO_POSTGRES_USER = salt['pillar.get']('postgres:auth:users:so_postgres_user:user', 'so_postgres') %}

 include:
+  - ca.trust
  - postgres.auth
  - postgres.ssl
  - postgres.config
@@ -42,6 +44,7 @@ so-postgres:
        {% endfor %}
      {% endif %}
    - binds:
+      - {{ CATRUST.dir }}
      - /opt/so/log/postgres/:/log:rw
      - /nsm/postgres:/var/lib/postgresql/data:rw
      - /opt/so/conf/postgres/postgresql.conf:/conf/postgresql.conf:ro
@@ -69,6 +72,7 @@ so-postgres:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: postgresconf
      - file: postgreshba
      - file: postgresinitdb
@@ -87,3 +87,9 @@ postgres:
      global: True
      advanced: True
      helpLink: postgres
+    cron.host:
+      description: Where pg_cron connects to run scheduled jobs such as Telegraf partition maintenance. Must be the container's PostgreSQL socket directory, /run/postgresql, because pg_cron connects without a password and only local socket connections are trusted. Any other value makes every scheduled job fail. Changing it restarts PostgreSQL.
+      readonly: True
+      global: True
+      advanced: True
+      helpLink: postgres
@@ -52,7 +52,7 @@ if [ -n "$FILTER_HOST" ] && ! [[ "$FILTER_HOST" =~ $HOST_RE ]]; then
 fi

 so_psql() {
-  docker exec so-postgres psql -U postgres -d so_telegraf -At -F $'\t' "$@"
+  docker exec -e PGOPTIONS='-c application_name=so-stats-show' so-postgres psql -U so_maint_telegraf -d so_telegraf -At -F $'\t' "$@"
 }

 if ! docker exec so-postgres psql -U postgres -lqt 2>/dev/null | cut -d\| -f1 | grep -qw so_telegraf; then
@@ -60,6 +60,11 @@ if ! docker exec so-postgres psql -U postgres -lqt 2>/dev/null | cut -d\| -f1 |
  exit 2
 fi

+if ! docker exec so-postgres psql -U postgres -tAc "SELECT 1 FROM pg_roles WHERE rolname = 'so_maint_telegraf'" 2>/dev/null | grep -q 1; then
+  echo "The so_maint_telegraf role is missing; run a highstate."
+  exit 1
+fi
+
 table_exists() {
  local table="$1"
  [ -n "$(so_psql -c "SELECT 1 FROM information_schema.tables WHERE table_schema='${SCHEMA}' AND table_name='${table}' LIMIT 1;")" ]
@@ -4,15 +4,18 @@ set -e
 # Provision Telegraf state inside the so-postgres container.
 # Usage: so-telegraf-postgres <subcommand>
 #   create_db    Ensure the so_telegraf database exists.
-#   group_role   Provision the so_telegraf group role, telegraf/partman schemas,
-#                pg_partman, the so_admin maintenance routines, and the hourly
-#                pg_cron maintenance job.
+#   group_role   Provision the so_telegraf group role, the so_maint_telegraf
+#                maintenance login role, telegraf/partman schemas, pg_partman,
+#                the so_admin maintenance routines, and the hourly pg_cron
+#                maintenance job.
 #   user         Create or update a per-minion login role granted to so_telegraf.
 #                Env: ROLE_USER, ROLE_PASS.
 #   retention    Reconcile partman retention and premake on telegraf parents.
 #                Env: RETENTION_DAYS.
-#   maintenance  Drain default partitions and run partman maintenance.
-#   check        Report partition health. Non-zero if any parent is unhealthy.
+#   maintenance  Drain default partitions and run partman maintenance, as
+#                so_maint_telegraf.
+#   check        Report partition health, as so_maint_telegraf. Non-zero if any
+#                parent is unhealthy.
 #
 # A default partition holding rows for a day blocks creating that day's child,
 # so maintenance drains defaults before calling partman. Use so-telegraf-repair
@@ -20,6 +23,10 @@ set -e

 cmd="${1:?subcommand required}"

+# Long enough for an hourly maintenance run to release its locks, short enough
+# that soup and admins are not left hanging.
+LOCK_TIMEOUT=60s
+
 case "$cmd" in
  create_db)
    if ! docker exec so-postgres psql -U postgres -tAc \
@@ -30,7 +37,9 @@ case "$cmd" in
    ;;

  group_role)
-    docker exec -i so-postgres psql -v ON_ERROR_STOP=1 -U postgres -d so_telegraf <<'EOSQL'
+    docker exec -i so-postgres psql -v ON_ERROR_STOP=1 -v lock_timeout="$LOCK_TIMEOUT" \
+      -U postgres -d so_telegraf <<'EOSQL'
+SET lock_timeout = :'lock_timeout';
 DO $$
 BEGIN
    IF NOT EXISTS (SELECT FROM pg_catalog.pg_roles WHERE rolname = 'so_telegraf') THEN
@@ -38,6 +47,17 @@ BEGIN
    END IF;
 END
 $$;
+DO $$
+BEGIN
+    IF NOT EXISTS (SELECT FROM pg_catalog.pg_roles WHERE rolname = 'so_maint_telegraf') THEN
+        CREATE ROLE so_maint_telegraf;
+    END IF;
+END
+$$;
+ALTER ROLE so_maint_telegraf LOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS PASSWORD NULL;
+GRANT so_telegraf TO so_maint_telegraf;
+ALTER ROLE so_maint_telegraf SET role = so_telegraf;
+ALTER ROLE so_maint_telegraf SET search_path = pg_catalog;
 GRANT CONNECT ON DATABASE so_telegraf TO so_telegraf;
 CREATE SCHEMA IF NOT EXISTS telegraf AUTHORIZATION so_telegraf;
 GRANT USAGE, CREATE ON SCHEMA telegraf TO so_telegraf;
@@ -58,44 +78,82 @@ ALTER DEFAULT PRIVILEGES IN SCHEMA partman
 ALTER DEFAULT PRIVILEGES IN SCHEMA partman
    GRANT USAGE, SELECT, UPDATE ON SEQUENCES TO so_telegraf;

-- pg_cron runs these as postgres, so they must not sit in a schema any
-- Telegraf role can create objects in.
+-- Owned by postgres so no Telegraf role can create objects here; the routines
+-- run with the caller's privileges.
 CREATE SCHEMA IF NOT EXISTS so_admin AUTHORIZATION postgres;
 REVOKE ALL ON SCHEMA so_admin FROM PUBLIC;

+-- Resolved through pg_inherits, never by name, so a view or a table that only
+-- carries the _default name is never read or written.
+CREATE OR REPLACE FUNCTION so_admin.telegraf_defaults()
+RETURNS TABLE (
+    parent_table  regclass,
+    default_part  regclass,
+    has_rows      boolean,
+    default_size  bigint
+)
+LANGUAGE plpgsql
+STABLE
+SECURITY INVOKER
+AS $func$
+DECLARE
+    r  record;
+BEGIN
+    FOR r IN
+        SELECT to_regclass(pc.parent_table) AS pt
+        FROM partman.part_config pc
+        WHERE pc.parent_table LIKE 'telegraf.%'
+        ORDER BY pc.parent_table
+    LOOP
+        CONTINUE WHEN r.pt IS NULL;
+        parent_table := r.pt;
+
+        SELECT c.oid::regclass INTO default_part
+        FROM pg_catalog.pg_inherits i
+        JOIN pg_catalog.pg_class c ON c.oid = i.inhrelid
+        WHERE i.inhparent = r.pt
+          AND c.relkind = 'r' AND c.relispartition
+          AND pg_catalog.pg_get_expr(c.relpartbound, c.oid) = 'DEFAULT';
+
+        IF default_part IS NULL THEN
+            has_rows     := false;
+            default_size := NULL;
+        ELSE
+            EXECUTE format('SELECT EXISTS (SELECT 1 FROM %s LIMIT 1)', default_part)
+                INTO has_rows;
+            default_size := pg_catalog.pg_total_relation_size(default_part);
+        END IF;
+
+        RETURN NEXT;
+    END LOOP;
+END;
+$func$;
+
 CREATE OR REPLACE PROCEDURE so_admin.telegraf_maintenance()
 LANGUAGE plpgsql
+SECURITY INVOKER
+-- No SET clause: it would forbid the COMMITs below, so so_maint_telegraf pins search_path instead.
 AS $proc$
 DECLARE
-    r          record;
-    v_default  text;
-    v_rows     bigint;
+    r  record;
 BEGIN
    -- No per-parent EXCEPTION handler: partition_data_proc commits internally,
    -- and COMMIT is illegal while a subtransaction is active. A failing parent
    -- aborts the run and the next pass retries.
    FOR r IN
-        SELECT parent_table, retention
-        FROM partman.part_config
-        WHERE parent_table LIKE 'telegraf.%'
-        ORDER BY parent_table
+        SELECT pc.parent_table, pc.retention, d.default_part, d.default_size
+        FROM so_admin.telegraf_defaults() d
+        JOIN partman.part_config pc ON to_regclass(pc.parent_table) = d.parent_table
+        WHERE d.default_part IS NOT NULL AND d.has_rows
+        ORDER BY pc.parent_table
    LOOP
-        v_default := format('%I.%I',
-                            split_part(r.parent_table, '.', 1),
-                            split_part(r.parent_table, '.', 2) || '_default');
-
-        CONTINUE WHEN to_regclass(v_default) IS NULL;
-
-        EXECUTE format('SELECT count(*) FROM %s', v_default) INTO v_rows;
-        CONTINUE WHEN v_rows = 0;
-
-        RAISE WARNING 'so_admin.telegraf_maintenance: % rows stranded in %, draining',
-                      v_rows, v_default;
+        RAISE WARNING 'so_admin.telegraf_maintenance: % holds rows (%), draining',
+                      r.default_part, pg_size_pretty(r.default_size);

        -- Cheaper to delete expired rows than to repartition and then drop them.
        IF r.retention IS NOT NULL THEN
            EXECUTE format('DELETE FROM %s WHERE "time" < now() - %L::interval',
-                           v_default, r.retention);
+                           r.default_part, r.retention);
            COMMIT;
        END IF;

@@ -103,7 +161,7 @@ BEGIN
        CALL partman.partition_data_proc(
            p_parent_table := r.parent_table,
            p_loop_count   := 200,
-            p_source_table := v_default
+            p_source_table := r.default_part::text
        );
        COMMIT;
    END LOOP;
@@ -112,58 +170,75 @@ BEGIN
 END;
 $proc$;

-CREATE OR REPLACE FUNCTION so_admin.telegraf_partition_status()
+DROP FUNCTION IF EXISTS so_admin.telegraf_partition_status();
+CREATE FUNCTION so_admin.telegraf_partition_status()
 RETURNS TABLE (
-    parent_table  text,
-    oldest_child  date,
-    newest_child  date,
-    days_ahead    int,
-    retention     text,
-    default_rows  bigint,
-    default_size  text
+    parent_table      text,
+    oldest_child      date,
+    newest_child      date,
+    days_ahead        int,
+    retention         text,
+    default_has_rows  boolean,
+    default_size      text
 )
 LANGUAGE plpgsql
+STABLE
 AS $func$
 DECLARE
-    r          record;
-    v_default  regclass;
+    r  record;
 BEGIN
    FOR r IN
-        SELECT pc.parent_table AS pt, pc.retention AS ret
-        FROM partman.part_config pc
-        WHERE pc.parent_table LIKE 'telegraf.%'
-        ORDER BY pc.parent_table
+        SELECT d.parent_table AS pt, d.has_rows, d.default_size AS size, pc.retention AS ret
+        FROM so_admin.telegraf_defaults() d
+        JOIN partman.part_config pc ON to_regclass(pc.parent_table) = d.parent_table
+        ORDER BY d.parent_table::text
    LOOP
-        parent_table := r.pt;
+        parent_table := r.pt::text;
        retention    := r.ret;

-        SELECT min(d), max(d) INTO oldest_child, newest_child
+        SELECT min(x), max(x) INTO oldest_child, newest_child
        FROM (
-            SELECT to_date(substring(c.relname FROM '_p(\d{8})$'), 'YYYYMMDD') AS d
-            FROM pg_inherits i
-            JOIN pg_class c ON c.oid = i.inhrelid
-            WHERE i.inhparent = r.pt::regclass
-              AND pg_get_expr(c.relpartbound, c.oid) <> 'DEFAULT'
+            SELECT to_date(substring(c.relname FROM '_p(\d{8})$'), 'YYYYMMDD') AS x
+            FROM pg_catalog.pg_inherits i
+            JOIN pg_catalog.pg_class c ON c.oid = i.inhrelid
+            WHERE i.inhparent = r.pt
+              AND pg_catalog.pg_get_expr(c.relpartbound, c.oid) <> 'DEFAULT'
        ) s;

-        days_ahead := newest_child - current_date;
-
-        v_default := to_regclass(format('%I.%I',
-                                        split_part(r.pt, '.', 1),
-                                        split_part(r.pt, '.', 2) || '_default'));
-        IF v_default IS NULL THEN
-            default_rows := NULL;
-            default_size := NULL;
-        ELSE
-            EXECUTE format('SELECT count(*) FROM %s', v_default::text) INTO default_rows;
-            default_size := pg_size_pretty(pg_total_relation_size(v_default));
-        END IF;
+        days_ahead       := newest_child - current_date;
+        default_has_rows := r.has_rows;
+        default_size     := pg_size_pretty(r.size);

        RETURN NEXT;
    END LOOP;
 END;
 $func$;

+GRANT USAGE ON SCHEMA so_admin TO so_telegraf;
+GRANT EXECUTE ON FUNCTION so_admin.telegraf_defaults() TO so_telegraf;
+GRANT EXECUTE ON PROCEDURE so_admin.telegraf_maintenance() TO so_telegraf;
+GRANT EXECUTE ON FUNCTION so_admin.telegraf_partition_status() TO so_telegraf;
+
+-- Partitions created by a superuser session (an older repair) belong to
+-- postgres; partman acting as so_telegraf could not drop them.
+DO $$
+DECLARE
+    r  record;
+BEGIN
+    FOR r IN
+        SELECT n.nspname, c.relname
+        FROM pg_catalog.pg_class c
+        JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
+        WHERE c.relkind IN ('r', 'p')
+          AND (n.nspname = 'telegraf'
+               OR (n.nspname = 'partman' AND c.relname LIKE 'template\_telegraf\_%'))
+          AND c.relowner <> 'so_telegraf'::regrole
+    LOOP
+        EXECUTE format('ALTER TABLE %I.%I OWNER TO so_telegraf', r.nspname, r.relname);
+    END LOOP;
+END
+$$;
+
 -- Drop the registration older releases left in so_telegraf.
 SELECT CASE
         WHEN current_setting('cron.database_name', true) IS DISTINCT FROM current_database()
@@ -175,20 +250,69 @@ DROP EXTENSION pg_cron CASCADE;
 \endif
 EOSQL

+    stalled=false
+    probe_failed=false
+    if out=$(docker exec -e PGOPTIONS="-c lock_timeout=$LOCK_TIMEOUT" so-postgres \
+        psql -U so_maint_telegraf -d so_telegraf -tAc \
+        "SELECT coalesce(bool_or(has_rows), false) FROM so_admin.telegraf_defaults()"); then
+      case "$out" in
+        t) stalled=true ;;
+        f) stalled=false ;;
+        *) probe_failed=true ;;
+      esac
+    else
+      probe_failed=true
+    fi
+    if $probe_failed; then
+      echo "ERROR: could not check Telegraf default partitions; the maintenance job was not changed." >&2
+      exit 1
+    fi
+
    # Guarded on the live GUC so applying this before the postgresql.conf change
    # has restarted the container skips rather than failing.
-    docker exec -i so-postgres psql -v ON_ERROR_STOP=1 -U postgres -d postgres <<'EOSQL'
+    docker exec -i so-postgres psql -v ON_ERROR_STOP=1 -v stalled="$stalled" \
+      -U postgres -d postgres <<'EOSQL'
 SELECT CASE WHEN current_setting('cron.database_name', true) = current_database()
            THEN 'true' ELSE 'false' END AS cron_here \gset
 \if :cron_here
 CREATE EXTENSION IF NOT EXISTS pg_cron;
-- cron.schedule_in_database is idempotent by jobname.
+-- A highstate reconciles the job's definition but never its active flag; only so-telegraf-repair turns it on.
+BEGIN;
+SELECT coalesce(max(jobid)::text, '') AS jobid,
+       count(*) > 0 AS job_exists,
+       coalesce(bool_and(active), true) AS job_active
+FROM cron.job
+WHERE jobname = 'telegraf-partman-maintenance' AND username = 'so_maint_telegraf' \gset
+SELECT cron.unschedule(jobid) FROM cron.job
+WHERE jobname = 'telegraf-partman-maintenance' AND username <> 'so_maint_telegraf';
+\if :job_exists
+SELECT cron.alter_job(:jobid,
+  schedule := '17 * * * *',
+  command  := 'CALL so_admin.telegraf_maintenance()',
+  database := 'so_telegraf');
+\else
+-- Registered inactive on a stalled grid so nothing drains the default
+-- partitions before so-telegraf-repair truncates them.
 SELECT cron.schedule_in_database(
  'telegraf-partman-maintenance',
  '17 * * * *',
  'CALL so_admin.telegraf_maintenance()',
-  'so_telegraf'
+  'so_telegraf',
+  'so_maint_telegraf',
+  NOT :'stalled'::boolean
 );
+\endif
+UPDATE cron.job SET nodename = current_setting('cron.host')
+WHERE jobname = 'telegraf-partman-maintenance'
+  AND nodename IS DISTINCT FROM current_setting('cron.host');
+COMMIT;
+SELECT coalesce(bool_and(active), true) AS job_active
+FROM cron.job
+WHERE jobname = 'telegraf-partman-maintenance' AND username = 'so_maint_telegraf' \gset
+\if :job_active
+\else
+\echo 'WARNING: the Telegraf maintenance job is inactive; run ''sudo so-telegraf-repair''.'
+\endif
 \else
 \echo 'pg_cron metadata database is not `postgres` yet; skipping job registration.'
 \endif
@@ -247,7 +371,7 @@ EOSQL
    ;;

  maintenance)
-    docker exec -i so-postgres psql -v ON_ERROR_STOP=1 -U postgres -d so_telegraf <<'EOSQL'
+    docker exec -i so-postgres psql -v ON_ERROR_STOP=1 -U so_maint_telegraf -d so_telegraf <<'EOSQL'
 SELECT CASE WHEN to_regproc('so_admin.telegraf_maintenance') IS NOT NULL
            THEN 'true' ELSE 'false' END AS has_proc \gset
 \if :has_proc
@@ -259,7 +383,7 @@ EOSQL
    ;;

  check)
-    docker exec -i so-postgres psql -U postgres -d so_telegraf <<'EOSQL'
+    docker exec -i so-postgres psql -U so_maint_telegraf -d so_telegraf <<'EOSQL'
 \pset border 2
 SELECT * FROM so_admin.telegraf_partition_status();
 EOSQL
@@ -278,9 +402,9 @@ LIMIT 5;
 \echo 'pg_cron is not installed in this database.'
 \endif
 EOSQL
-    unhealthy=$(docker exec so-postgres psql -U postgres -d so_telegraf -tAc \
+    unhealthy=$(docker exec so-postgres psql -U so_maint_telegraf -d so_telegraf -tAc \
      "SELECT count(*) FROM so_admin.telegraf_partition_status()
-       WHERE coalesce(default_rows, 0) > 0 OR coalesce(days_ahead, -1) < 1")
+       WHERE default_has_rows OR coalesce(days_ahead, -1) < 1")
    if [ "${unhealthy:-1}" != "0" ]; then
      echo "so-telegraf-postgres check: $unhealthy telegraf parent(s) unhealthy" >&2
      exit 1
@@ -11,26 +11,38 @@
 # maintenance once. Healthy grids are reported and left alone.
 #
 # Usage: so-telegraf-repair [--check] [--yes] [--no-restart]
-#   --check       Report health and change nothing.
+#   --check       Report health and change nothing. Also reports a missing
+#                 so_maint_telegraf role and a maintenance job that is missing
+#                 or inactive.
 #   --yes         Skip the confirmation prompt (for soup and other automation).
 #   --no-restart  Never restart so-postgres, even if pg_cron's launcher is dead.
 #
+# Repair mode (without --check) registers the so_maint_telegraf role and the
+# maintenance job if they are missing, pauses the job while it repairs, and
+# always leaves it active afterwards. Data work runs as so_maint_telegraf.
+#
 # Exit status:
 #   0  healthy, or repair completed
-#   1  repair is needed (--check only)
-#   2  cannot run here: so-postgres, so_telegraf or pg_partman is missing
+#   1  repair is needed (--check), or the repair failed after it started
+#   2  skipped: Telegraf output is not Postgres, or so-postgres, so_telegraf or pg_partman is missing
+#   3  could not check, for example a connection error

-set -e
+set -eE

 # Matches p_premake in telegraf.conf's create_parent template.
 PREMAKE=7
 JOB_NAME=telegraf-partman-maintenance
+MAINT_ROLE=so_maint_telegraf
+# Long enough for an hourly maintenance run to release its locks, short enough
+# that soup and admins are not left hanging.
+LOCK_TIMEOUT=60s
+PG_OPTS="-c lock_timeout=$LOCK_TIMEOUT -c application_name=so-telegraf-repair"

 CHECK_ONLY=false
 ASSUME_YES=false
 NO_RESTART=false

-usage() { sed -n '/^# Usage:/,/^#   2 /p' "$0" | sed 's/^# \?//'; }
+usage() { sed -n '/^# Usage:/,/^#   3 /p' "$0" | sed 's/^# \?//'; }

 while [[ $# -gt 0 ]]; do
  case "$1" in
@@ -38,70 +50,153 @@ while [[ $# -gt 0 ]]; do
    --yes|-y)          ASSUME_YES=true ;;
    --no-restart)      NO_RESTART=true ;;
    -h|--help)         usage; exit 0 ;;
-    *) echo "Unknown option: $1" >&2; usage >&2; exit 2 ;;
+    *) echo "Unknown option: $1" >&2; usage >&2; exit 3 ;;
  esac
  shift
 done

-skip() { echo "$*"; exit 2; }
+# Strips C0 (except tab and newline), DEL and C1 characters, so names and
+# messages written with Telegraf credentials can't drive the terminal.
+sanitise() { LC_ALL=C tr -d '\000-\010\013-\037\177' | LC_ALL=C sed 's/\xc2[\x80-\x9f]//g'; }

-psql_tg() { docker exec -i so-postgres psql -U postgres -d so_telegraf "$@"; }
-psql_pg() { docker exec -i so-postgres psql -U postgres -d postgres "$@"; }
+ERR_FILE=$(mktemp)
+ERR_STATUS=3
+ERR_PREFIX="Could not check Telegraf partitions"
+job_id=
+restore_active=
+
+on_err() {
+  # Command substitutions inherit this trap; only the top level reports.
+  [[ $BASH_SUBSHELL -eq 0 ]] || exit "$ERR_STATUS"
+  local line
+  line=$(tail -n 1 "$ERR_FILE" | sanitise)
+  echo "$ERR_PREFIX: ${line:-command failed}" >&2
+  exit "$ERR_STATUS"
+}
+
+on_exit() {
+  if [[ -n "$restore_active" ]]; then
+    psql_cron -tAc "SELECT cron.alter_job($job_id, active := $restore_active)" >/dev/null || true
+  fi
+  rm -f "$ERR_FILE"
+}
+
+trap on_err ERR
+trap on_exit EXIT
+
+finish() { trap - ERR; exit "$1"; }
+skip() { echo "$*"; finish 2; }
+
+# Keeps the last stderr line for on_err while still showing all of it.
+capture() {
+  local rc=0
+  : >"$ERR_FILE"
+  "$@" 2>"$ERR_FILE" || rc=$?
+  sanitise <"$ERR_FILE" >&2
+  return "$rc"
+}
+
+psql_tg() {
+  capture docker exec -i -e PGOPTIONS="$PG_OPTS" so-postgres psql -U "$MAINT_ROLE" -d so_telegraf "$@"
+}
+psql_pg() {
+  capture docker exec -i -e PGOPTIONS="$PG_OPTS" so-postgres psql -U postgres -d postgres "$@"
+}
+psql_cron() {
+  capture docker exec -i -e PGOPTIONS="$PG_OPTS" so-postgres psql -U postgres -d "$cron_db" "$@"
+}

-# query_to_xml so the per-table row counts need no helper function installed.
 REPORT="
-WITH parents AS (
-    SELECT pc.parent_table,
-           pc.premake,
-           split_part(pc.parent_table, '.', 1) AS sch,
-           split_part(pc.parent_table, '.', 2) AS tbl
-    FROM partman.part_config pc
-    WHERE pc.parent_table LIKE 'telegraf.%'
-), children AS (
-    SELECT p.parent_table,
-           max(to_date(substring(c.relname FROM '_p(\d{8})\$'), 'YYYYMMDD')) AS newest_child
-    FROM parents p
-    JOIN pg_class pt ON pt.oid = p.parent_table::regclass
-    JOIN pg_inherits i ON i.inhparent = pt.oid
-    JOIN pg_class c ON c.oid = i.inhrelid
-    WHERE pg_get_expr(c.relpartbound, c.oid) <> 'DEFAULT'
-    GROUP BY p.parent_table
-), defaults AS (
-    SELECT p.parent_table,
-           p.premake,
-           format('%I.%I', p.sch, p.tbl || '_default') AS default_table,
-           to_regclass(format('%I.%I', p.sch, p.tbl || '_default')) AS default_oid
-    FROM parents p
-)
 SELECT d.parent_table,
       c.newest_child,
       (c.newest_child - current_date) AS days_ahead,
-       d.premake,
-       CASE WHEN d.default_oid IS NULL THEN NULL ELSE
-            (xpath('/row/cnt/text()',
-                   query_to_xml(format('SELECT count(*) AS cnt FROM %s', d.default_table),
-                                false, true, '')))[1]::text::bigint
-       END AS default_rows,
-       CASE WHEN d.default_oid IS NULL THEN NULL
-            ELSE pg_size_pretty(pg_total_relation_size(d.default_oid)) END AS default_size
-FROM defaults d
-LEFT JOIN children c ON c.parent_table = d.parent_table
-ORDER BY 1
+       pc.premake,
+       d.has_rows AS default_has_rows,
+       pg_size_pretty(d.default_size) AS default_size
+FROM so_admin.telegraf_defaults() d
+JOIN partman.part_config pc ON to_regclass(pc.parent_table) = d.parent_table
+LEFT JOIN LATERAL (
+    SELECT max(to_date(substring(ch.relname FROM '_p(\d{8})\$'), 'YYYYMMDD')) AS newest_child
+    FROM pg_inherits i
+    JOIN pg_class ch ON ch.oid = i.inhrelid
+    WHERE i.inhparent = d.parent_table
+      AND pg_get_expr(ch.relpartbound, ch.oid) <> 'DEFAULT'
+) c ON true
+ORDER BY d.parent_table::text
 "

-docker ps --format '{{.Names}}' | grep -qx so-postgres \
+print_report() {
+  local report
+  report=$(psql_tg -c "$REPORT")
+  sanitise <<<"$report"
+}
+
+has_cron() {
+  [[ -n "$cron_db" ]] || return 0
+  psql_cron -tAc "SELECT 1 FROM pg_extension WHERE extname = 'pg_cron'"
+}
+
+lookup_job() {
+  local cron
+  cron=$(has_cron)
+  [[ -n "$cron" ]] || return 0
+  psql_cron -tA -F '|' -c \
+    "SELECT jobid, active FROM cron.job
+      WHERE jobname = '$JOB_NAME' AND username = '$MAINT_ROLE'
+      ORDER BY jobid LIMIT 1"
+}
+
+output=$(capture salt-call pillar.get telegraf:output --out=newline_values_only)
+output=$(tr -d '[:space:]' <<<"$output" | tr '[:lower:]' '[:upper:]')
+output=${output:-INFLUXDB}
+case "$output" in
+  POSTGRES|BOTH) ;;
+  *) skip "Telegraf output is $(sanitise <<<"$output"); not checking Postgres partitions." ;;
+esac
+
+containers=$(capture docker ps --format '{{.Names}}')
+grep -qx so-postgres <<<"$containers" \
  || skip "so-postgres is not running; nothing to repair."
-docker exec so-postgres psql -U postgres -tAc \
-  "SELECT 1 FROM pg_database WHERE datname='so_telegraf'" | grep -q 1 \
+has_db=$(psql_pg -tAc "SELECT 1 FROM pg_database WHERE datname = 'so_telegraf'")
+[[ -n "$has_db" ]] \
  || skip "The so_telegraf database does not exist; Telegraf is not writing to Postgres."
-psql_tg -tAc "SELECT 1 FROM pg_extension WHERE extname='pg_partman'" | grep -q 1 \
+
+# so_telegraf before the postgres state lands, postgres after.
+cron_db=$(psql_pg -tAc "SELECT current_setting('cron.database_name', true)")
+cron_db=$(tr -d '[:space:]' <<<"$cron_db")
+
+has_role=$(psql_pg -tAc "SELECT 1 FROM pg_roles WHERE rolname = '$MAINT_ROLE'")
+job=
+[[ "$cron_db" == postgres ]] && job=$(lookup_job)
+
+if [[ -z "$has_role" ]] && $CHECK_ONLY; then
+  echo "Repair is needed:"
+  echo "  * maintenance role $MAINT_ROLE is missing"
+  echo
+  echo "Re-run without --check to repair."
+  finish 1
+fi
+
+if ! $CHECK_ONLY && [[ -z "$has_role" || ( "$cron_db" == postgres && -z "$job" ) ]]; then
+  echo "Registering the maintenance role and job."
+  if ! /usr/sbin/so-telegraf-postgres group_role; then
+    echo "Could not register the maintenance job; run a highstate and try again." >&2
+    finish 3
+  fi
+  [[ "$cron_db" == postgres ]] && job=$(lookup_job)
+fi
+
+has_partman=$(psql_tg -tAc "SELECT 1 FROM pg_extension WHERE extname = 'pg_partman'")
+[[ -n "$has_partman" ]] \
  || skip "pg_partman is not installed in so_telegraf; nothing to repair."

 parents=$(psql_tg -tAc \
  "SELECT count(*) FROM partman.part_config WHERE parent_table LIKE 'telegraf.%'")
-stranded=$(psql_tg -tAc "SELECT coalesce(sum(default_rows), 0) FROM ( $REPORT ) t")
-behind=$(psql_tg -tAc \
-  "SELECT count(*) FROM ( $REPORT ) t WHERE coalesce(days_ahead, -1) < 1")
+counts=$(psql_tg -tAc \
+  "SELECT count(*) FILTER (WHERE default_has_rows) || '|' ||
+          count(*) FILTER (WHERE coalesce(days_ahead, -1) < 1)
+     FROM ( $REPORT ) t")
+IFS='|' read -r stranded behind <<<"$counts"
 # premake < 7, or infinite_time_partitions off: without the latter partman
 # refuses to premake forward across the gap the stall left behind.
 misconfigured=$(psql_tg -tAc \
@@ -115,16 +210,24 @@ launcher=$(psql_pg -tAc \
  "SELECT count(*) FROM pg_stat_activity
    WHERE backend_type ILIKE '%pg_cron%' OR application_name ILIKE '%pg_cron%'")

-# so_telegraf before the postgres state lands, postgres after.
-cron_db=$(docker exec so-postgres psql -U postgres -tAc \
-  "SELECT current_setting('cron.database_name', true)" | tr -d '[:space:]')
 last_run=never
-if [[ -n "$cron_db" ]]; then
-  last_run=$(docker exec so-postgres psql -U postgres -d "$cron_db" -tAc \
-    "SELECT coalesce(max(d.start_time)::text, 'never')
+cron_installed=$(has_cron)
+if [[ -n "$cron_installed" ]]; then
+  run=$(psql_cron -tAc \
+    "SELECT d.status || ' at ' || d.start_time
       FROM cron.job_run_details d JOIN cron.job j USING (jobid)
-      WHERE j.jobname = '$JOB_NAME'" 2>/dev/null | tr -d '[:space:]' || echo unknown)
-  [[ -n "$last_run" ]] || last_run=never
+      WHERE j.jobname = '$JOB_NAME'
+      ORDER BY d.start_time DESC LIMIT 1")
+  if [[ -n "$run" ]]; then
+    message=$(psql_cron -tAc \
+      "SELECT coalesce(d.return_message, '')
+         FROM cron.job_run_details d JOIN cron.job j USING (jobid)
+        WHERE j.jobname = '$JOB_NAME'
+        ORDER BY d.start_time DESC LIMIT 1")
+    message=$(sanitise <<<"$message")
+    message=${message//$'\n'/ }
+    last_run="$(sanitise <<<"$run"): ${message:0:200}"
+  fi
 fi

 # A grid that has never written a metric has nothing to recover, and an empty
@@ -132,43 +235,67 @@ fi
 restart_needed=false
 [[ "$launcher" -eq 0 && "$parents" -gt 0 && -n "$cron_db" ]] && restart_needed=true

+# group_role registers the job only in the postgres database.
+job_inactive=false
+if [[ "$cron_db" == postgres ]]; then
+  [[ -z "$job" || "${job##*|}" == f ]] && job_inactive=true
+fi
+
 repair_needed=false
 [[ "$stranded" -gt 0 ]] && repair_needed=true
 [[ "$behind" -gt 0 ]] && repair_needed=true
 [[ "$misconfigured" -gt 0 ]] && repair_needed=true
 $restart_needed && repair_needed=true
+$job_inactive && repair_needed=true

 echo "Telegraf partition status:"
-psql_tg -c "$REPORT"
-echo "Rows stranded in default partitions: $stranded"
-echo "pg_cron metadata database: ${cron_db:-unset}"
+print_report
+echo "Default partitions with rows: $stranded"
+echo "pg_cron metadata database: $(sanitise <<<"${cron_db:-unset}")"
+[[ "$cron_db" == postgres ]] \
+  || echo "pg_cron metadata database is $(sanitise <<<"${cron_db:-unset}"); not checking the maintenance job."
 echo "pg_cron launcher running: $([[ "$launcher" -gt 0 ]] && echo yes || echo no)"
 echo "Last $JOB_NAME run: $last_run"
 echo

 if ! $repair_needed; then
  echo "Telegraf partitions are healthy. Nothing to do."
-  exit 0
+  finish 0
 fi

 if $CHECK_ONLY; then
  echo "Repair is needed:"
-  [[ "$stranded" -gt 0 ]]    && echo "  * $stranded row(s) stranded in default partitions"
+  [[ "$stranded" -gt 0 ]]    && echo "  * $stranded default partition(s) hold rows"
  [[ "$behind" -gt 0 ]]      && echo "  * $behind parent(s) with no partition for the current window"
  [[ "$misconfigured" -gt 0 ]] && echo "  * $misconfigured parent(s) with stale partman settings"
  $restart_needed            && echo "  * pg_cron's launcher is dead; maintenance is not running at all"
+  $job_inactive              && echo "  * the maintenance job is missing or inactive"
  echo
  echo "Re-run without --check to repair."
-  exit 1
+  finish 1
 fi

 if [[ "$stranded" -gt 0 ]] && ! $ASSUME_YES; then
-  echo "This will permanently discard the $stranded stranded row(s) above."
+  echo "This will permanently discard the rows in the $stranded default partition(s) above."
  $restart_needed && ! $NO_RESTART && \
    echo "so-postgres will also be restarted, which briefly interrupts SOC."
-  [[ -t 0 ]] || { echo "Not a terminal; re-run with --yes to confirm." >&2; exit 2; }
+  [[ -t 0 ]] || { echo "Not a terminal; re-run with --yes to confirm." >&2; finish 1; }
  read -r -p "Continue? [y/N] " answer
-  [[ "$answer" =~ ^[Yy]$ ]] || { echo "Aborted."; exit 0; }
+  [[ "$answer" =~ ^[Yy]$ ]] || { echo "Aborted."; finish 0; }
+fi
+
+ERR_STATUS=1
+ERR_PREFIX="Telegraf partition repair failed"
+
+if [[ "$cron_db" == postgres ]]; then
+  if [[ -z "$job" ]]; then
+    echo "Could not register the maintenance job; run a highstate and try again." >&2
+    finish 3
+  fi
+  job_id=${job%%|*}
+  [[ "${job##*|}" == t ]] && restore_active=true || restore_active=false
+  echo "Pausing the $JOB_NAME job during the repair."
+  psql_cron -tAc "SELECT cron.alter_job($job_id, active := false)" >/dev/null
 fi

 if [[ "$misconfigured" -gt 0 ]]; then
@@ -186,27 +313,24 @@ if [[ "$stranded" -gt 0 ]]; then
  # One transaction: Telegraf is still writing, so a default emptied without
  # its partition in place is refilled before maintenance can attach one.
  psql_tg -v ON_ERROR_STOP=1 <<'EOSQL'
+BEGIN;
 DO $$
 DECLARE
    r record;
 BEGIN
    FOR r IN
-        SELECT pc.parent_table,
-               format('%I.%I', n.nspname, c.relname) AS default_table
-        FROM partman.part_config pc
-        JOIN pg_class p ON p.oid = pc.parent_table::regclass
-        JOIN pg_inherits i ON i.inhparent = p.oid
-        JOIN pg_class c ON c.oid = i.inhrelid
-        JOIN pg_namespace n ON n.oid = c.relnamespace
-        WHERE pc.parent_table LIKE 'telegraf.%'
-          AND pg_get_expr(c.relpartbound, c.oid) = 'DEFAULT'
+        SELECT pc.parent_table, d.default_part
+        FROM so_admin.telegraf_defaults() d
+        JOIN partman.part_config pc ON to_regclass(pc.parent_table) = d.parent_table
+        WHERE d.default_part IS NOT NULL
    LOOP
-        EXECUTE format('TRUNCATE TABLE %s', r.default_table);
+        EXECUTE format('TRUNCATE TABLE %s', r.default_part);
        PERFORM partman.create_partition_time(
            r.parent_table, ARRAY[date_trunc('day', now())]::timestamptz[]);
    END LOOP;
 END
 $$;
+COMMIT;
 EOSQL
 fi

@@ -222,7 +346,7 @@ if $restart_needed; then
      sleep 2
    done
    docker exec so-postgres pg_isready -U postgres -q \
-      || { echo "so-postgres did not come back; check 'docker logs so-postgres'." >&2; exit 1; }
+      || { echo "so-postgres did not come back; check 'docker logs so-postgres'." >&2; finish 1; }
  fi
 fi

@@ -238,8 +362,14 @@ CALL partman.run_maintenance_proc();
 \endif
 EOSQL

+if [[ -n "$job_id" ]]; then
+  psql_cron -tAc "SELECT cron.alter_job($job_id, active := true)" >/dev/null
+  restore_active=
+  echo "The $JOB_NAME job is active."
+fi
+
 echo
 echo "Telegraf partition status after repair:"
-psql_tg -c "$REPORT"
+print_report
 echo "The $JOB_NAME job runs hourly at :17. Confirm it fired with:"
 echo "  so-telegraf-repair --check"
@@ -7,6 +7,7 @@
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}

 include:
  - ca
@@ -28,6 +29,7 @@ so-redis:
      - {{ BINDING }}
      {% endfor %}
    - binds:
+      - {{ CATRUST.pem }}:/etc/ssl/certs/ca-certificates.crt:ro
      - /opt/so/log/redis:/var/log/redis:rw
      - /opt/so/conf/redis/etc/redis.conf:/usr/local/etc/redis/redis.conf:ro
      - /opt/so/conf/redis/working:/redis:rw
@@ -60,6 +62,7 @@ so-redis:
    {% endif %}
    - entrypoint: "redis-server /usr/local/etc/redis/redis.conf"
    - watch:
+      - cmd: ca_trust_extract
      - file: trusttheca
      - x509: redis_crt
      - x509: redis_key
@@ -6,9 +6,11 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}

 include:
+  - ca.trust
  - registry.ssl
  - registry.config
  - registry.sostatus
@@ -29,6 +31,7 @@ so-dockerregistry:
      - {{ BINDING }}
      {% endfor %}
    - binds:
+      - {{ CATRUST.pem }}:/etc/ssl/certs/ca-certificates.crt:ro
      - /opt/so/conf/docker-registry/etc/config.yml:/etc/distribution/config.yml:ro
      - /opt/so/conf/docker-registry:/var/lib/registry:rw
      - /nsm/docker-registry/docker:/var/lib/registry/docker:rw
@@ -63,6 +66,7 @@ so-dockerregistry:
        attempts: 5
        interval: 30
    - watch:
+      - cmd: ca_trust_extract
      - x509: registry_crt
      - x509: registry_key
    - require:
@@ -3,6 +3,31 @@
 # https://securityonion.net/license; you may not use this file except in compliance with the
 # Elastic License 2.0.

+{% set ONEDIR_PACKAGES = {'relenv': '0.22.27', 'pip': '26.2.2', 'certifi': '2026.10.19'} %}
+
+{% if grains.saltversion == '3006.19' %}
+onedir_package:
+  file.recurse:
+    - name: /opt/so/conf/salt/module_packages/onedir
+    - source: salt://salt/module_packages/onedir
+    - clean: True
+    - makedirs: True
+
+# Backport of salt 3006.29 (PR 70466): certifi 2026.10.19 and pip 26.2.2 drop the compromised
+# eMudhra/emSign root. relenv must move with pip: 0.22.2's runtime wrapper hardcodes pre-26.2 pip
+# install() signatures, so a lone pip upgrade breaks every later pip run in the onedir. Installing
+# all three in one invocation under the stock pip avoids ever being in that state.
+onedir_python_module_install:
+  cmd.run:
+    - name: /opt/saltstack/salt/bin/python3.10 -m pip install --no-index --find-links=/opt/so/conf/salt/module_packages/onedir/ --upgrade --root-user-action=ignore {% for pkg, ver in ONEDIR_PACKAGES.items() %}{{ pkg }}=={{ ver }} {% endfor %}
+    - unless: /opt/saltstack/salt/bin/python3.10 -c 'import {{ ONEDIR_PACKAGES.keys() | join(", ") }}; assert {% for pkg, ver in ONEDIR_PACKAGES.items() %}{{ pkg }}.__version__ == "{{ ver }}"{% if not loop.last %} and {% endif %}{% endfor %}'
+    - require:
+      - file: onedir_package
+    - require_in:
+      - cmd: docker_python_module_install
+    - failhard: True
+{% endif %}
+
 docker_module_package:
  file.recurse:
    - name: /opt/so/conf/salt/module_packages/docker
@@ -4,10 +4,12 @@
 # Elastic License 2.0.

 {% from 'vars/globals.map.jinja' import GLOBALS %}
+{% from 'ca/trust.map.jinja' import CATRUST %}
 {% from 'docker/docker.map.jinja' import DOCKERMERGED %}


 include:
+  - ca.trust
  - sensoroni.config
  - sensoroni.sostatus

@@ -17,6 +19,7 @@ so-sensoroni:
    - restart_policy: unless-stopped
    - network_mode: host
    - binds:
+      - {{ CATRUST.dir }}
      - /nsm/import:/nsm/import:rw
      - /nsm/pcapout:/nsm/pcapout:rw
      - /opt/so/conf/sensoroni/sensoroni.json:/opt/sensoroni/sensoroni.json:ro
@@ -48,6 +51,7 @@ so-sensoroni:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: /opt/so/conf/sensoroni/sensoroni.json
    - require:
      - file: sensoroniagentconf
@@ -1,3 +1,5 @@
 requests>=2.31.0
 pyyaml>=6.0
-urllib3>=2.1.0
+urllib3>=2.1.0
+certifi>=2026.10.23
+idna>=3.20
@@ -1,2 +1,4 @@
 requests>=2.31.0
 pyyaml>=6.0
+certifi>=2026.10.23
+idna>=3.20
@@ -1,2 +1,4 @@
 requests>=2.31.0
 pyyaml>=6.0
+certifi>=2026.10.23
+idna>=3.20
@@ -1,2 +1,4 @@
 requests>=2.31.0
 pyyaml>=6.0
+certifi>=2026.10.23
+idna>=3.20
Author	SHA1	Message	Date
Mike Reeves	ae394ce709	Postgres: pg_cron over the local socket, so_maint_telegraf role, repair and soup 3.5.0	2026-11-12 19:07:57 -05:00
Jorge Reyes	40a30f6908	Merge pull request #16389 from Security-Onion-Solutions/reyesj2-patch-1 update elastic-agent artifact repo directory	2026-11-11 14:32:59 -06:00
Mike Reeves	1fd37251c9	Merge pull request #16390 from Security-Onion-Solutions/foundry/mwaq014injcipm Telegraf: fix EPS double counting and duplicate metric rows	2026-11-10 14:53:30 -05:00
Mike Reeves	4a5253f688	Telegraf: fix EPS double counting and duplicate metric rows	2026-11-10 08:26:19 -05:00
reyesj2	ec220572f0	dedup artifact registry config & split ea artifacts out of /opt/socore/html/ into dedicated /opt/socore/elastic-artifacts/artifacts/	2026-11-09 12:32:06 -06:00
Jorge Reyes	b7b00f8233	Merge pull request #16378 from Security-Onion-Solutions/reyesj2/fix-16369 surface ilm policy errors in so-elasticsearch-ilm-policy-load output	2026-11-05 14:05:29 -06:00
Jorge Reyes	45d2b29a42	Merge pull request #16374 from Security-Onion-Solutions/fix/skip-mismatched-iso-registry Skip leftover ISO registry.tar when tags do not match SOVERSION	2026-11-05 11:11:45 -06:00
Doug Burks	7b494439e7	Merge pull request #16377 from Security-Onion-Solutions/feature/desktop-so-crates Install securityonion-so-crates on Security Onion Desktop	2026-11-04 15:56:12 -05:00
Doug Burks	9a8dc097b2	Install securityonion-so-crates on Security Onion Desktop	2026-11-04 15:51:16 -05:00
reyesj2	274da18929	Surface ILM policy validation errors in highstate Fail 4xx ILM PUTs immediately (retry 429/5xx/connect), print the Elasticsearch reason in the salt summary, and retry the load on the next highstate via /opt/so/state/esilm.txt when the last run failed. Fixes https://github.com/Security-Onion-Solutions/sos/issues/277	2026-11-04 13:18:31 -06:00
reyesj2	99a8ad2d4b	Wait for Elasticsearch and retry ILM policy loads Require wait_for_so-elasticsearch, check ES is responsive before dispatch, and retry each policy PUT so soup no longer fails with empty output when ES is not ready yet. Fixes https://github.com/Security-Onion-Solutions/securityonion/issues/16369	2026-11-04 11:14:12 -06:00
Mike Reeves	bd9cdeadef	Merge pull request #16373 from Security-Onion-Solutions/foundry/mw1jl4boa6hshh Remove x86-64-v3 CPU pre-flight check from soup	2026-11-03 22:50:54 -05:00
Mike Reeves	bdf01a5a5a	Remove x86-64-v3 CPU check from soup	2026-11-03 22:13:58 -05:00
reyesj2	1849c74cea	Skip leftover ISO registry.tar when tags do not match SOVERSION Fixes #16372	2026-11-03 21:07:29 -06:00
Mike Reeves	021b069706	Merge pull request #16367 from Security-Onion-Solutions/TOoSmOotH-patch-1 Bump version from 3.4.0 to 3.5.0	2026-11-02 11:52:06 -05:00
Mike Reeves	824d674eec	Add version 3.5.0 to discussion template	2026-11-02 11:50:49 -05:00
Mike Reeves	d43791c63e	Bump version from 3.4.0 to 3.5.0	2026-11-02 11:50:08 -05:00
Mike Reeves	4dc7f2d97c	Merge pull request #16365 from Security-Onion-Solutions/3.4 3.4.0	2026-11-02 10:52:49 -05:00
Mike Reeves	092be7f222	3.4.0	2026-11-02 10:46:19 -05:00
Mike Reeves	340e7928f6	3.4.0	2026-10-30 11:36:47 -04:00
Jason Ertel	f5ff40bd20	Merge pull request #16361 from Security-Onion-Solutions/jertel/wip switch analyzer deps to py 314	2026-10-29 16:15:29 -04:00
Jason Ertel	7c271ea9c1	switch analyzer deps to py 314	2026-10-29 16:06:43 -04:00
coreyogburn	6794af6f68	Merge pull request #16358 from Security-Onion-Solutions/cogburn/sync-timeouts New Configurable Timeouts for Detection Engine Syncs	2026-10-28 11:34:02 -06:00
Corey Ogburn	691bef2cf4	New Configurable Timeouts for Detection Engine Syncs Each engine now has timeouts for how long to allow a sync to go on after detections are modified.	2026-10-28 10:48:03 -06:00
Mike Reeves	1bbaa213cb	Merge pull request #16354 from Security-Onion-Solutions/mreeves/remove-es-sokeys	2026-10-27 17:36:48 -04:00
Mike Reeves	5342765ca8	Merge pull request #16357 from Security-Onion-Solutions/mreeves/host-ca-trust Use the host CA trust store, with the grid CA, for all containers	2026-10-27 16:40:58 -04:00
Mike Reeves	02394acf69	Use the host CA trust store for sensor, strelka and idh containers Mount the host trust store into zeek, suricata, tcpreplay, the strelka containers and idh, and restart them when it changes. Point the Python requests clients in strelka-backend and idh at it instead of certifi. Remove the frozen /opt/so/conf/ca bundles from every node in ca.trust rather than only on the manager in soup.	2026-10-27 16:24:33 -04:00
Josh Patterson	d8f69f7457	Merge pull request #16356 from Security-Onion-Solutions/custombindmounts Add custom_bind_mounts to so-kafka and so-soc	2026-10-27 15:25:36 -04:00
Mike Reeves	11c8fe3093	Use the host CA trust store for all grid containers Anchor the grid CA and blocklist the eMudhra roots in the host trust store (new ca.trust state, also used by the desktop role), then mount the extracted store into each container instead of the frozen copies made by so-catrust. Containers restart when the host trust changes.	2026-10-27 15:08:21 -04:00
Mike Reeves	d6fecc761a	Remove unused elasticsearch.p12/sokeys keystore	2026-10-27 14:43:46 -04:00
Josh Patterson	40e3dafbee	Add custom_bind_mounts to so-kafka and so-soc Every container in docker/defaults.yaml exposes custom_bind_mounts via soc_docker.yaml, but kafka and soc enabled.sls never appended them to binds, so the setting was silently ignored for those two containers.	2026-10-27 14:06:58 -04:00
Jason Ertel	1fb7e1bc79	Merge pull request #16351 from Security-Onion-Solutions/jertel/wip fix false positive scan results	2026-10-26 15:50:37 -04:00
Jason Ertel	aeb4e1c64c	fix false positive scan results	2026-10-26 15:48:08 -04:00
Josh Patterson	a7b4cdd813	Merge pull request #16350 from Security-Onion-Solutions/fixsaltemudhra Backport salt 3006.29 eMudhra/emSign CA fix into the 3006.19 onedir	2026-10-26 15:21:46 -04:00
Josh Patterson	a540e47672	Backport salt 3006.29 eMudhra/emSign CA fix into the 3006.19 onedir Salt 3006.29 (saltstack/salt#70466) answered the eMudhra/emSign incident by bumping certifi to 2026.10.19, which drops the "emSign ECC Root CA - G3" root whose intermediates had recoverable private keys, and by shipping pip 26.2.2, whose vendored certifi gets the same fix. We stay on 3006.19, so apply the same two upgrades inside /opt/saltstack/salt from vendored wheels. relenv is upgraded to 0.22.27 in the same pip invocation. relenv 0.22.2's runtime wrapper hardcodes pre-26.2 pip install() signatures, so a lone pip 26.2.2 upgrade breaks every later pip run with an unexpected 'script_executable' argument. 0.22.27 is what 3006.29 builds against and its runtime diff is otherwise cosmetic. Installing all three at once under the stock pip means no host ever sits in the broken intermediate state. The state lives in salt.python_modules so new installs get it from salt_install_module_deps before salt-master starts, and existing grids get it from the soup highstate plus grid push. It is gated on saltversion 3006.19 so a future Salt bump disables it, and ordered before the docker module install so docker is always installed with the fixed pip. The docker wheel set now carries certifi 2026.10.19 instead of 2024.7.4. Verified on a fresh 3.4.0 manager + sensor grid: state applies on both, reruns are no-ops, full highstate passes on both nodes.	2026-10-26 11:39:47 -04:00
Jason Ertel	6a83ea8f2f	Merge pull request #16341 from Security-Onion-Solutions/jertel/wip analyzer dep updates	2026-10-26 08:50:54 -04:00
Jason Ertel	c584675259	analyzer dep updates	2026-10-23 11:34:51 -04:00
Josh Brower	466e959340	Merge pull request #16335 from Security-Onion-Solutions/esql-default-query Refactor correlation template	2026-10-19 13:05:55 -04:00
defensivedepth	d2251d0cbc	Refactor correlation template	2026-10-19 12:55:51 -04:00
coreyogburn	78013aa5bc	Merge pull request #16333 from Security-Onion-Solutions/cogburn/automation-turn-timeout Automation Turn Timeout	2026-10-15 15:12:52 -06:00
Corey Ogburn	69b2cb2b02	Automation Turn Timeout Simplified to turnTimeoutSeconds as it sits on automationSettings and doesn't need to stutter.	2026-10-15 13:06:09 -06:00
Mike Reeves	c13085ba71	Merge pull request #16332 from Security-Onion-Solutions/TOoSmOotH-patch-6 Change readonly to readonlyUi in YAML configuration	2026-10-15 11:44:49 -04:00
Mike Reeves	da25083837	Change readonly to readonlyUi in YAML configuration	2026-10-15 11:39:59 -04:00
Mike Reeves	76fb1cc479	Merge pull request #16331 from Security-Onion-Solutions/TOoSmOotH-patch-5 Add readonly attribute to earliest alert time field	2026-10-15 11:19:58 -04:00
Mike Reeves	cfb8992a92	Add readonly attribute to earliest alert time field	2026-10-15 11:16:48 -04:00
Josh Patterson	7121531336	Merge pull request #16328 from Security-Onion-Solutions/fix/socqeurymerge combine default and alerts queries added via pillar	2026-10-14 11:13:39 -04:00
Josh Patterson	ff2b46b73e	combine default and alerts queries added via pillar pillar.get replaces lists, so once alerts queries are saved to pillar from the SOC config UI, queries added to defaults later (e.g. the licensed Group By Assessment, Name) never reach soc.json. Merge them by name the same way availableModels are merged by id: pillar order and edits are kept, and default queries missing from pillar are appended.	2026-10-14 09:19:16 -04:00
Mike Reeves	b158a38099	Merge pull request #16323 from Security-Onion-Solutions/mreeves/alert-triage-notify-skill Remove the Notifier agent mapping	2026-10-13 17:45:22 -04:00
Mike Reeves	26f5d07435	Remove the Notifier agent mapping AlertTriage now sends its own notification through the Notify skill, so the Notifier agent no longer exists in SOC.	2026-10-13 17:35:22 -04:00
Mike Reeves	747abeed79	Merge pull request #16302 from Security-Onion-Solutions/mreeves/alert-assessment-groupby Add a licensed Group By Assessment, Name alerts query	2026-10-13 17:16:48 -04:00
Mike Reeves	1bcf80dd10	Group by assessment without a missing bucket The assessment is a number, and SOC's missing bucket only works on keyword fields, so the * made Elasticsearch fail parsing "__missing__" as a long. Untriaged alerts now drop out of this view.	2026-10-13 17:11:49 -04:00
Josh Patterson	e7d7676a90	Merge pull request #16322 from Security-Onion-Solutions/fix/socmerge combine default and availableModels added via pillar	2026-10-13 16:27:57 -04:00
Josh Patterson	2b6566d33b	combine default and availableModels added via pillar	2026-10-13 13:52:30 -04:00
Jorge Reyes	f5641647ab	Merge pull request #16319 from Security-Onion-Solutions/reyesj2-patch-1 remap enterprisesearch integration components	2026-10-13 09:17:30 -05:00
Jorge Reyes	fc5e66e913	remap enterprisesearch integration components	2026-10-13 09:10:36 -05:00
Jason Ertel	5f4fc58416	Merge pull request #16317 from Security-Onion-Solutions/jertel/wip eval to use Suricata for metadata	2026-10-10 11:04:49 -04:00
Jason Ertel	84b2b19698	eval to use Suricata for metadata	2026-10-10 11:02:35 -04:00
Josh Brower	7214f8770c	Merge pull request #16269 from Security-Onion-Solutions/esql-correlations Initial Correlations support	2026-10-09 15:53:10 -04:00
Mike Reeves	0a0062b9ab	Merge pull request #16315 from Security-Onion-Solutions/mreeves/soai-haiku Add Claude Haiku to the SOAI assistant models	2026-10-09 13:51:46 -04:00
Mike Reeves	5d88755429	Increase contextLimitSmall to 1000000	2026-10-09 13:14:26 -04:00
Mike Reeves	ecc75ef65a	Merge branch '3/dev' into mreeves/soai-haiku	2026-10-09 13:13:45 -04:00
Mike Reeves	037f6d3d4f	Update agent mapping for Investigator and DetectionEngineer	2026-10-09 13:09:09 -04:00
Mike Reeves	0cc7e36af5	Reduce contextLimitSmall from 1000000 to 100000	2026-10-09 12:47:46 -04:00
Mike Reeves	946e904434	Merge pull request #16301 from Security-Onion-Solutions/mreeves/alert-triage-agent Map the AlertTriage and Notifier agents to a model	2026-10-09 12:39:24 -04:00
Mike Reeves	b17b596a64	Add Claude Haiku to the SOAI assistant models	2026-10-09 10:11:25 -04:00
defensivedepth	5a24fe8b61	Move pipeline fix	2026-10-08 19:43:22 -04:00
defensivedepth	fed0c5739d	Refactor correlations	2026-10-08 19:31:29 -04:00
Matthew Wright	603949002b	Merge pull request #16312 from Security-Onion-Solutions/mwright/md-img-toggle Add allowExternalMarkdownImages SOC Setting	2026-10-08 12:38:45 -04:00
Matthew Wright	de63f95ab0	turn off advanced	2026-10-08 12:38:13 -04:00
Matthew Wright	a1b76650fb	add external image markdown toggle	2026-10-08 12:10:30 -04:00
Jorge Reyes	0cd8e53832	Merge pull request #16311 from Security-Onion-Solutions/reyesj2-patch-4 regenerate elastic agent installer	2026-10-08 10:13:30 -05:00
reyesj2	0210ccfcc3	elastic agent 9.4.8 regenerate installer	2026-10-08 09:54:41 -05:00
reyesj2	517076330a	stg profile update - breaks so-elastic-agent / so-elastic-fleet containers	2026-10-08 09:54:35 -05:00
Jason Ertel	ad249782fc	Merge pull request #16310 from Security-Onion-Solutions/jertel/wip new destination timeout annotation; fix fp	2026-10-08 10:48:26 -04:00
Jason Ertel	2eab084358	new destination timeout annotation; fix fp	2026-10-08 10:46:15 -04:00
Jorge Reyes	547d2a316b	Merge pull request #16306 from Security-Onion-Solutions/reyesj2/es948 ES 9.4.8	2026-10-07 16:17:52 -05:00
Jorge Reyes	1c4eef4224	Update version from 3.0.0-foxtrot to 3.4.0	2026-10-07 12:31:45 -05:00
defensivedepth	d352bc0384	ES\|QL and Sigma correlation support	2026-10-07 11:26:32 -04:00
Mike Reeves	3304aff62b	Note that an oai query also needs the agentic assistant SOC (securityonion-soc#1363) now hides a query licensed for oai unless the assistant is enabled and agentic, so the query editor's license help says so. Claude-Session: https://claude.ai/code/session_01SZLcwj68A2wLo3F8TGKsLP	2026-10-07 08:56:44 -04:00
Mike Reeves	7469d833d1	Add a licensed Group By Assessment, Name alerts query Groups alerts by the AI triage assessment, keeping untriaged alerts as a missing bucket. Query entries take an optional license feature, which SOC uses to hide this one on grids without OnionAI, and the query editor now shows it.	2026-10-06 21:14:25 -04:00
Mike Reeves	b11fc6257a	Map the AlertTriage and Notifier agents to a model SOC disables a built-in agent that has no agentMapping entry, so the new AlertTriage agent and the Notifier it delegates to need one to run.	2026-10-06 20:43:04 -04:00
Jason Ertel	d1114a0dae	Merge pull request #16300 from Security-Onion-Solutions/jertel/wip update tick interval desc	2026-10-06 18:41:48 -04:00
Jason Ertel	f4b301d71c	update tick interval desc	2026-10-06 18:32:49 -04:00
Jorge Reyes	c6e42131b2	Update version from 3.4.0 to 3.0.0-foxtrot	2026-10-06 13:45:22 -05:00
reyesj2	337dddf596	ES 9.4.8	2026-10-06 13:44:52 -05:00
@@ -1 +1 @@
 .4.0
 .5.0