remove unused localrules config

dedup artifact registry config & split ea artifacts out of /opt/socore/html/ into dedicated /opt/socore/elastic-artifacts/artifacts/
Merge pull request #16378 from Security-Onion-Solutions/reyesj2/fix-16369
2026-11-11 07:00:01 +01:00 · 2026-11-10 17:30:01 -06:00 · 2026-11-09 12:32:06 -06:00 · 2026-11-05 14:05:29 -06:00 · 2026-11-05 11:11:45 -06:00 · 2026-11-04 15:56:12 -05:00
59 changed files with 796 additions and 330 deletions
@@ -14,6 +14,7 @@ body:
        - 3.2.0
        - 3.3.0
        - 3.4.0
+        - 3.5.0
        - Other (please provide detail below)
    validations:
      required: true
@@ -1,17 +1,17 @@
-### 3.3.0-20260911 ISO image released on 2026/09/11
+### 3.4.0-20261102 ISO image released on 2026/11/02


 ### Download and Verify

-3.3.0-20260911 ISO image:  
-https://download.securityonion.net/file/securityonion/securityonion-3.3.0-20260911.iso
+3.4.0-20261102 ISO image:  
+https://download.securityonion.net/file/securityonion/securityonion-3.4.0-20261102.iso
 
-MD5: 12B18433D3A2198A185892FF79CF638F  
-SHA1: 2B3C2E1FA7A78ED1F956E7EDCC12E32593C14EEE  
-SHA256: 0938C73B76CE30EC9E4394D312C79EA7CAC721B6818541697279A6221F7D870D  
+MD5: FB0380AE996E175D6CC7E7A35D8CC178  
+SHA1: D8960F1DA352710BCA209773BF0B3AE8791D0D1B  
+SHA256: DFC4C87668836852A328EC7A0B755C8C1511FF0E43D423DEE23047A95F676D99  

 Signature for ISO image:  
-https://github.com/Security-Onion-Solutions/securityonion/raw/3/main/sigs/securityonion-3.3.0-20260911.iso.sig
+https://github.com/Security-Onion-Solutions/securityonion/raw/3/main/sigs/securityonion-3.4.0-20261102.iso.sig

 Signing key:  
 https://raw.githubusercontent.com/Security-Onion-Solutions/securityonion/3/main/KEYS  
@@ -25,22 +25,22 @@ wget https://raw.githubusercontent.com/Security-Onion-Solutions/securityonion/3/

 Download the signature file for the ISO:  
 ```
-wget https://github.com/Security-Onion-Solutions/securityonion/raw/3/main/sigs/securityonion-3.3.0-20260911.iso.sig
+wget https://github.com/Security-Onion-Solutions/securityonion/raw/3/main/sigs/securityonion-3.4.0-20261102.iso.sig
 ```

 Download the ISO image:  
 ```
-wget https://download.securityonion.net/file/securityonion/securityonion-3.3.0-20260911.iso
+wget https://download.securityonion.net/file/securityonion/securityonion-3.4.0-20261102.iso
 ```

 Verify the downloaded ISO image using the signature file:  
 ```
-gpg --verify securityonion-3.3.0-20260911.iso.sig securityonion-3.3.0-20260911.iso
+gpg --verify securityonion-3.4.0-20261102.iso.sig securityonion-3.4.0-20261102.iso
 ```

 The output should show "Good signature" and the Primary key fingerprint should match what's shown below:
 ```
-gpg: Signature made Fri 11 Sep 2026 11:23:56 AM EDT using RSA key ID FE507013
+gpg: Signature made Sat 31 Oct 2026 09:27:50 AM EDT using RSA key ID FE507013
 gpg: Good signature from "Security Onion Solutions, LLC <info@securityonionsolutions.com>"
 gpg: WARNING: This key is not certified with a trusted signature!
 gpg:          There is no indication that the signature belongs to the owner.
@@ -1 +1 @@
-3.4.0
+3.5.0
@@ -0,0 +1,14 @@
+-----BEGIN CERTIFICATE-----
+MIICKzCCAbGgAwIBAgIKe3G2gla4EnycqDAKBggqhkjOPQQDAzBaMQswCQYDVQQG
+EwJVUzETMBEGA1UECxMKZW1TaWduIFBLSTEUMBIGA1UEChMLZU11ZGhyYSBJbmMx
+IDAeBgNVBAMTF2VtU2lnbiBFQ0MgUm9vdCBDQSAtIEMzMB4XDTE4MDIxODE4MzAw
+MFoXDTQzMDIxODE4MzAwMFowWjELMAkGA1UEBhMCVVMxEzARBgNVBAsTCmVtU2ln
+biBQS0kxFDASBgNVBAoTC2VNdWRocmEgSW5jMSAwHgYDVQQDExdlbVNpZ24gRUND
+IFJvb3QgQ0EgLSBDMzB2MBAGByqGSM49AgEGBSuBBAAiA2IABP2lYa57JhAd6bci
+MK4G9IGzsUJxlTm801Ljr6/58pc1kjZGDoeVjbk5Wum739D+yAdBPLtVb4Ojavti
+sIGJAnB9SMVK4+kiVCJNk7tCDK93nCOmfddhEc5lx/h//vXyqaNCMEAwHQYDVR0O
+BBYEFPtaSNCAIEDyqOkAB2kZd6fmw/TPMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMB
+Af8EBTADAQH/MAoGCCqGSM49BAMDA2gAMGUCMQC02C8Cif22TGK6Q04ThHK1rt0c
+3ta13FaPWEBaLd4gTCKDypOofu4SQMfWh0/434UCMBwUZOR8loMRnLDRWmFLpg9J
+0wD8ofzkpf9/rdcw0Md3f76BB1UwUCAU9Vc4CqgxUQ==
+-----END CERTIFICATE-----
@@ -0,0 +1,15 @@
+-----BEGIN CERTIFICATE-----
+MIICTjCCAdOgAwIBAgIKPPYHqWhwDtqLhDAKBggqhkjOPQQDAzBrMQswCQYDVQQG
+EwJJTjETMBEGA1UECxMKZW1TaWduIFBLSTElMCMGA1UEChMcZU11ZGhyYSBUZWNo
+bm9sb2dpZXMgTGltaXRlZDEgMB4GA1UEAxMXZW1TaWduIEVDQyBSb290IENBIC0g
+RzMwHhcNMTgwMjE4MTgzMDAwWhcNNDMwMjE4MTgzMDAwWjBrMQswCQYDVQQGEwJJ
+TjETMBEGA1UECxMKZW1TaWduIFBLSTElMCMGA1UEChMcZU11ZGhyYSBUZWNobm9s
+b2dpZXMgTGltaXRlZDEgMB4GA1UEAxMXZW1TaWduIEVDQyBSb290IENBIC0gRzMw
+djAQBgcqhkjOPQIBBgUrgQQAIgNiAAQjpQy4LRL1KPOxst3iAhKAnjlfSU2fySU0
+WXTsuwYc58Byr+iuL+FBVIcUqEqy6HyC5ltqtdyzdc6LBtCGI79G1Y4PPwT01xyS
+fvalY8L1X44uT6EYGQIrMgqCZH0Wk9GjQjBAMB0GA1UdDgQWBBR8XQKEE9TMipuB
+zhccLikenEhjQjAOBgNVHQ8BAf8EBAMCAQYwDwYDVR0TAQH/BAUwAwEB/zAKBggq
+hkjOPQQDAwNpADBmAjEAvvNhzwIQHWSVB7gYboiFBS+DCBeQyh+KTOgNG3qxrdWB
+CUfvO6wIBHxcmbHtRwfSAjEAnbpV/KlK6O3t5nYBQnvI+GDZjVGLVTv7jHvrZQnD
+JbNR6iC8hZVdyR+EhCVBCyj
+-----END CERTIFICATE-----
@@ -0,0 +1,21 @@
+-----BEGIN CERTIFICATE-----
+MIIDczCCAlugAwIBAgILAK7PALrEzzL4Q7IwDQYJKoZIhvcNAQELBQAwVjELMAkG
+A1UEBhMCVVMxEzARBgNVBAsTCmVtU2lnbiBQS0kxFDASBgNVBAoTC2VNdWRocmEg
+SW5jMRwwGgYDVQQDExNlbVNpZ24gUm9vdCBDQSAtIEMxMB4XDTE4MDIxODE4MzAw
+MFoXDTQzMDIxODE4MzAwMFowVjELMAkGA1UEBhMCVVMxEzARBgNVBAsTCmVtU2ln
+biBQS0kxFDASBgNVBAoTC2VNdWRocmEgSW5jMRwwGgYDVQQDExNlbVNpZ24gUm9v
+dCBDQSAtIEMxMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAz+upufGZ
+BczYKCFK83M0UYRWEPWgTywS4/oTmifQz/l5GnRfHXk5/Fv4cI7gklL35CX5VIPZ
+HdPIWoU/Xse2B+4+wM6ar6xWQio5JXDWv7V7Nq2s9nPczdcdioOl+yuQFTdrHCZH
+3DspVpNqs8FqOp099cGXOFgFixwR4+S0uF2FHYP+eF8LRWgYSKVGczQ7/g/IdrvH
+GPMF0Ybzhe3nudkyrVWIzqa2kbBPrH4VI5b2P/AgNBbeCsbEBEV5f6f9vtKppa+c
+xSMq9zwhbL2vj07FOrLzNBL834AaSaTUqZX3noleoomslMuoaJuvimUnzYnu3Yy1
+aylwQ6BpC+S5DwIDAQABo0IwQDAdBgNVHQ4EFgQU/qHgcB4qAzlSWkK+XJGFehiq
+TbUwDgYDVR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQEL
+BQADggEBAMJKVvoVIXsoounlHfv4LcQ5lkFMOycsxGwYFYDGrK9HWS8mC+M2sO87
+/kOXSTKZEhVb3xEp/6tT+LvBeA+snFOvV71ojD1pM/CjoCNjO2RnIkSt1XHLVip4
+kqNPEjE2NuLe/gDEo2APJ62gsIq1NnpSob0n9CAnYuhNlCQT5AoE6TyrLshDCUrG
+YQTlSTR+08TI9Q/Aqum6VF7zYytPT1DU/rl7mYw9wC68AivTxEDkigcxHpvOJpkT
+xHqmiIMERnHXhuBUDDIlhJu58tBf5E7oke3VIAb3ADMmpDqw8NQBmIMMMAVSKeo
+WXzhriKi4gp6D/piq1JM4fHfyr6DDUI=
+-----END CERTIFICATE-----
@@ -0,0 +1,32 @@
+-----BEGIN CERTIFICATE-----
+MIIFcjCCA1qgAwIBAgIKLwq3aw3LSq8nWDANBgkqhkiG9w0BAQwFADBWMQswCQYD
+VQQGEwJVUzETMBEGA1UECxMKZW1TaWduIFBLSTEUMBIGA1UEChMLZU11ZGhyYSBJ
+bmMxHDAaBgNVBAMTE2VtU2lnbiBSb290IENBIC0gQzIwHhcNMTgwMjE4MTgzMDAw
+WhcNNDMwMjE4MTgzMDAwWjBWMQswCQYDVQQGEwJVUzETMBEGA1UECxMKZW1TaWdu
+IFBLSTEUMBIGA1UEChMLZU11ZGhyYSBJbmMxHDAaBgNVBAMTE2VtU2lnbiBSb290
+IENBIC0gQzIwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCMfX1lA+Tb
+mh9YInmRgOW97IVx4LUJf2DRZfs837Jrml+py64aVnYgWO4t6C78fgjfS7jX+c4T
+inIzEquWcI+zi0fd4Sc8NDf7JONp27VWX0qwUYqzLDRCt+s7zpLcfx1ky0zVIJj6
+L06uPyK3kIr9+YAsrVj+39utm6e2MBQsRNstSI3fCQYAGvoQTQ8fULauTqNWaYAk
+NYFe6HUHHQPp2u1Ua00odMXiD5oRFxLcDnGAcE1I/9E9mLCdkggXijYUmico7+Xw
+ZeFoPhva6eIJ5p03Lt3Du5W3EcHR0cJmmY1pyeA36JaXKWRNM9IRjYMVNCcp4jhB
+2tIYiZ+LVk8bwQ9/1c23txmv3u97taZlV22NF4ttS1qq3J+MOp0oGULBzpKfRx0q
+GVqbPukQNGAjOLIN8KDNQNzbR1iAl2d8H+MSoicBo4Aid8TjLWcNv48oCWL53ZrF
+BMTDjaIA6frG1t4IpbnHadA7qCJJe2qpJN6n2eQKAUn6UiQDHPsSqNBlcUhQ4Y/0
+Y0mU5rghm2OB9rXQS1Fb1JRCfJMNnJIm5AUB2+2RWzq5Tgz7SbSho8NsZk0UbQnF
+xciqQ9uoVTAsK14Sk9oG8Q3zfsM08cdPoRb0WlIZklR6mKD7L8nH/zfGu8PIJv94
+GGB9RZ9U4A69r3ePmy8MvrzfNxHKtH6svwIDAQABo0IwQDAdBgNVHQ4EFgQUs/eK
+pNYPiABZ6FEXT9V+7IYigZ0wDgYDVR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMB
+Af8wDQYJKoZIhvcNAQEMBQADggIBADQlpiWM0cv2nZ0H5jVsBq0x2q62Q0LwqATs
+CFvyub7gxNCytRuoA8stmPOEu/lg8Igxj4FIjoyhIrWUVxyiLU7No4P+WjEUOwUT
+xIpkEOtvGUQ9fiOlcGHtIZDNBlZq7WpktXAxeV55RPPsor26p2FNAMRFfZQh0sLX
+hKgk8iulSSggqx8ezgPye63FaiYEi4c/dzRj3HOCnsZiwZZU02df5YpNFjxSwZvE
+41cjGpsrpWMfQFI2s53RbeXp47lSAxYE4NzjBFMe+EwFuEveBCJBEAH5rvYu3pi2
+orsJ424TqWEQV1tCsCkQz+Yq/Okal7yHAkKDeOXcP7oN4A+TdXc2pdqxuVCnBO0R
+mWz2JpGSSeJjiTk/OPwRsPNWtwG/KXL04o2ta3jiPpJuICVtWDAc9R3auBEgJl5r
+ShRmBdszG0LmzsHuZPCFSYC15RBDCOBsa8bDRJ8pBFU2Wi/CVXCACEuavgoveA4F
+a5bt38o0PWxsBP+MpocCdVtDMqzQhxy9IohKuXWAGresoIvKDg3xFk6rBOrjfVwJ
+elwi/xAisojHPJVQv9W1zVIoHp+EQg/4MQC21NbIX2RoioB+V3hK439b/w7deU8x
+2M8cl1OG0nPfbnARl5GPM7vJgi470jto4SeMg6HMAW3Egb56tQcNLwI9U8mZnNvR
+gUMrkAgL
+-----END CERTIFICATE-----
@@ -0,0 +1,22 @@
+-----BEGIN CERTIFICATE-----
+MIIDlDCCAnygAwIBAgIKMfXkYgxsWO3W2DANBgkqhkiG9w0BAQsFADBnMQswCQYD
+VQQGEwJJTjETMBEGA1UECxMKZW1TaWduIFBLSTElMCMGA1UEChMcZU11ZGhyYSBU
+ZWNobm9sb2dpZXMgTGltaXRlZDEcMBoGA1UEAxMTZW1TaWduIFJvb3QgQ0EgLSBH
+MTAeFw0xODAyMTgxODMwMDBaFw00MzAyMTgxODMwMDBaMGcxCzAJBgNVBAYTAklO
+MRMwEQYDVQQLEwplbVNpZ24gUEtJMSUwIwYDVQQKExxlTXVkaHJhIFRlY2hub2xv
+Z2llcyBMaW1pdGVkMRwwGgYDVQQDExNlbVNpZ24gUm9vdCBDQSAtIEcxMIIBIjAN
+BgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAk0u76WaK7p1b1TST0Bsew+eeuGQz
+f2N4aLTNLnF115sgxk0pvLZoYIr3IZpWNVrzdr3YzZr/k1ZLpVkGoZM0Kd0WNHVO
+8oG0x5ZOrRkVUkr+PHB1cM2vK6sVmjM8qrOLqs1D/fXqcP/tzxE7lM5OMhbTI0Aq
+d7OvPAEsbO2ZLIvZTmmYsvePQbAyeGHWDV/D+qJAkh1cF+ZwPjXnorfCYuKrpDhM
+tTk1b+oDafo6VGiFbdbyL0NVHpENDtjVaqSW0RM8LHhQ6DqS0hdW5TUaQBw+jSzt
+Od9C4INBdN+jzcKGYEho42kLVACL5HZpIQ15TjQIXhTCzLG3rdd8cIrHhQIDAQAB
+o0IwQDAdBgNVHQ4EFgQU++8Nhp6w492pufEhF38+/PB3KxowDgYDVR0PAQH/BAQD
+AgEGMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAFn/8oz1h31x
+PaOfG1vR2vjTnGs2vZupYeveFix0PZ7mddrXuqe8QhfnPZHr5X3dPpzxz5KsbEjM
+wiI/aTvFthUvozXGaCocV685743QNcMYDHsAVhzNixl03r4PEuDQqqE/AjSxcM6d
+GNYIAwlG7mDgfrbESQRRfXBgvKqy/3lyeqYdPV8q+Mri/Tm3R7nrft8EI6/6nAYH
+6ftjk4BAtcZsCjEozgyfz7MjNYBBjWzEN3uBL4ChQEKF6dk4jeihU80Bv2noWgby
+RQuQ+q7hv53yrlc8pa6yVvSLZUDp/TGBLPQ5Cdjua6e0ph0VpZj3AYHYhX3zUVxx
+iN66zB+Afko=
+-----END CERTIFICATE-----
@@ -0,0 +1,32 @@
+-----BEGIN CERTIFICATE-----
+MIIFlTCCA32gAwIBAgILAIZNvw/jXtd9jtgwDQYJKoZIhvcNAQEMBQAwZzELMAkG
+A1UEBhMCSU4xEzARBgNVBAsTCmVtU2lnbiBQS0kxJTAjBgNVBAoTHGVNdWRocmEg
+VGVjaG5vbG9naWVzIExpbWl0ZWQxHDAaBgNVBAMTE2VtU2lnbiBSb290IENBIC0g
+RzIwHhcNMTgwMjE4MTgzMDAwWhcNNDMwMjE4MTgzMDAwWjBnMQswCQYDVQQGEwJJ
+TjETMBEGA1UECxMKZW1TaWduIFBLSTElMCMGA1UEChMcZU11ZGhyYSBUZWNobm9s
+b2dpZXMgTGltaXRlZDEcMBoGA1UEAxMTZW1TaWduIFJvb3QgQ0EgLSBHMjCCAiIw
+DQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAMNwGIWW2kHfHK+sXTNwxF07K+IV
+ySTuyFM2r1v002wUfcdT+zs5OM5QbMYFFnedXQI6gCFLsjKrcaej48Zt37OyEb3i
+aPs7CsP4kAyTwzKH9aZe6gXYHrJq40/ZVMNcQVI2PcIp40B/SAN2gUZ+ZaUtIOvV
+jEx26/ebNaXRIsthlkOG/caB+QRwDw1tl7338Zlv0M2oTBUy4B3e7dGP5pgXH71M
+jqHPCoNo+xv9f0NTBT+hUDa8h8wUtcGQq9CDeJTpjWcD2bP2AMdVG6oVpMAUeUzo
+cCyglvtFdUMjggxBbw4qhau1HXPG8Ot9hwL7ZMi8tkTzrvUIxxb8G9LF/7kKeCE7
+tGZaVzDTnXuifl3msR4ErHsQ4P7lVu2AIjIAhrAXoedDidb7pMcf7TABdrYUT1Jo
+G/AiK+J9jO6GTjeADD4LMDSBZhHMuBK/PJ/g0kGBt+/C1L+/HURzQhJkMlRnM6Rv
+XoCtfKopSlns5trZmTi971Wjbn88QXP61lGpBCUPwCjs7rpOYvSUJtI+lcbF+37q
+kIqOXYkVT3cupDSpw+H89kFtj5GKY+Xny4LxY+3IvDIRiyd6ky1DPj713DI0yqve
+EpsIr3A0PdwuyUI7CS1jg0NnGFT6Xxyr0xB+VDt83FJYW8v16k2pbaQ4kVxA3aXd
+X9dZYyVR1S59KM75AgMBAAGjQjBAMB0GA1UdDgQWBBTt7E1FYRgo57MjKBEcTaUn
+DV7s9DAOBgNVHQ8BAf8EBAMCAQYwDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG9w0B
+AQwFAAOCAgEACFC/ilQg8KTCVBxFJW/sazomkS0kNYbEIZg4B3obqwsJ7SX98z8Z
+gfzBpz0nYClwwJjWbFN1R2zY8pCEot6/dgmA8Vbq0GxhwPM5YN/SZquNyRIxO3cU
+dlAcwf+vSezdVCf9wOzvSAF3q0a5ljvbdbNJNpfScQVp7UUd5sBsZk8jXO1KQ/go
+/Vf/GDPnrIFmxpAIGE3sgnO8lAv9FzUaAeuv7HWe47xN9J7+bQzF93yHuIXACPTL
+pQHhg2zMv5C7BAbuDHfbj1Cu294Z832yhSfBcziWGskOvl3es2EcHytbS9c9P+0z
+Mpka7zGC1FHrvLb/FoduH86TeZt0QjZ6pcplNzoaxDnDvzTJ6CC2Eny+qH/APFCu
+VUv5/wjwF+HPm8Pup2ARj9cEp92+0qcerfHacNq5hMeGZdbA/dzdUR/5z5zXdxAk
+nl8mcfGb0eMNSTXQmmB/i4AecNnr72uYjzlaXUGYN7Nrb6XouG0pnh0/BBtWWp0U
+ShIPpWEAqs7RJBj6+1ZUYXZ4ObrCw962DxhN2p19Hxw9LtuUUcLqqTPrFXYvwO4t
+ouj7KJnAkaTUfXGdEaFVtFig1EA30WzJY2X1vAQ7hVnniCjgaXAGqjsU6sklNM9n
+xDx5rFCCCEtj9Kh8UHjGK2QqgP5kwgttjOApQMaCoezMfK4KD7WpOXU=
+-----END CERTIFICATE-----
@@ -0,0 +1,32 @@
+-----BEGIN CERTIFICATE-----
+MIIFdjCCA16gAwIBAgIQCIYhdXKnxAzDuy7l1y1ubDANBgkqhkiG9w0BAQwFADBV
+MQswCQYDVQQGEwJJTjElMCMGA1UECgwcZU11ZGhyYSBUZWNobm9sb2dpZXMgTGlt
+aXRlZDEfMB0GA1UEAwwWZW1TaWduIFJvb3QgQ1MgQ0EgLSBHMjAeFw0yNDAyMDgw
+ODE1MzlaFw00OTAyMDcwODE1MzlaMFUxCzAJBgNVBAYTAklOMSUwIwYDVQQKDBxl
+TXVkaHJhIFRlY2hub2xvZ2llcyBMaW1pdGVkMR8wHQYDVQQDDBZlbVNpZ24gUm9v
+dCBDUyBDQSAtIEcyMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA4CVO
+xuBjdLJXpkrjosA+7ygTGaYzufUJrhQOeIZEuCqeCPI8hDqIu+MIqEqOlgDFetVM
+3Cor3CGBlGJzfzzlI31vyElvQq88jjCfo12Ghqcf+NAZDFduJk5+NgC0z3lfdiB6
+NNzxbfVkA+X6oDNNRioVUZsVFuIzT9Qt0Sh4KwQPRFBc/wI2yGpDuKHNsLehcECx
+Uw00m33xGYh9UkeL3reOmAaLYWZiO+Pd2vxPtrkXI4wyN4BMlESJ73uP0tyX9lq4
+M4yN2yh3i+ETI8/jzPjxHD7QFaVkCoS+uj2YmBa3mytlqlBgyOoKueNeotX8hfzB
+FNm962abpV4mN0SoC6hjkYv8IayseRPFv9VI0meQXg9OLRoNvBOqG5kw937FH0Li
+q3CIzKyvdyeYRT0LJdWA2iuB9l+4zzIfCwGyT4M4VDmJuPBhrTmjXKae6AUtm1eO
+q2f8gftA6J6AlbD0gRd/t8CrFssFNm+LYAEd36kWRBuolbaOfbSbEww84xNAgj4U
+SPwYjtNDsCkQjtgSOaDtDOmolN+Mbk528Y+LHMlbd4T3Xmp+VIWp5+70GhBDOA91
+hhtims+hHkXjRI2U7WrJLcTlcF8ykiymRbFKhwOBvTmtZNbHmDI7scU4wWUAsmW6
+tWaeeoM50KBl8ai28A6yBGYMO//03Ubvqiqst80CAwEAAaNCMEAwHQYDVR0OBBYE
+FOYcxqontIvyvYVkX6/0unKm0AF1MA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMBAf8E
+BTADAQH/MA0GCSqGSIb3DQEBDAUAA4ICAQCv5Asord7fdQMT30emwb+oLuDUxdmJ
+DQHnzxIXoNvymDTbNHF9LqjeXc+X+XcjIV8PTp4o2I9wyVppHmcaVgRjN/G9FxeN
+5tCQ97v5sdx4eAdPljK1xqg7Vhfmzcp+7rSjJG7Sn9gMlZx81kL3Oa2qZj+W5Fi6
+tN4vaR9OLEyvKKHDG/eFxS/qghD1d/Clkb7UGGjBcWJlaZTWVALMKG+QCxiMoELi
+oeKaoBbZoXooxfF0mWdkIl9RWNU4r9O/yt66LBE9rj49dX19L+rGg71/4bTi5ua+
+Wz60XaOc2CdHJaBSols5hWEx/LBjS7UzfU3uJfnpQXVDi0US5sFzIHIlS6kjh7h
+YuF4+/9ZAl4Bx3w2TNN7pVm7I39pgYl7XN6KZRwfkAv4aRswPqhWbQfr3+qfcC93
+cgTyFy2RG5YE/OfmpSs6VtYKawqsdEODz/p2nhZfSDrVDnEPvERqahttY5+gaWNc
+2fbibgKMfuejTZaJ3yttPslVSko5SDs0xKU423swu34sb/fdISqtOu4NzPCZiHSG
+ladaoITmSA6UWFk4fdqaLBI+xSxY5Et3S1APp5r3j/l1KrWPNbb1/+ydHN+YLbh8
+hCBIecKdk6VbnnFRlulm0d2RSgpfCRrWLomN8JHCxCMVv/hBQXz3AXGM1ZlfYTyl
+E1BpDby0G2eEkw==
+-----END CERTIFICATE-----
@@ -10,3 +10,4 @@ include:
  - ca.server
 {% endif %}
  - ca.trustca
+  - ca.trust
@@ -0,0 +1,12 @@
+# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
+# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at 
+# https://securityonion.net/license; you may not use this file except in compliance with the
+# Elastic License 2.0.
+
+{# Containers mount the extracted directory rather than single files: update-ca-trust replaces
+   files by rename, so a file bind mount would keep pointing at the old inode. #}
+{% set CATRUST = {
+    'dir': '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro',
+    'java': '/etc/pki/ca-trust/extracted/java/cacerts',
+    'pem': '/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'
+} %}
@@ -0,0 +1,37 @@
+# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
+# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at 
+# https://securityonion.net/license; you may not use this file except in compliance with the
+# Elastic License 2.0.
+
+ca_trust_anchor:
+  file.managed:
+    - name: /etc/pki/ca-trust/source/anchors/so-intca.crt
+    - source: salt://ca/files/ca.crt
+    - show_changes: False
+    - makedirs: True
+
+ca_trust_legacy_anchor:
+  file.absent:
+    - name: /etc/pki/ca-trust/source/anchors/ca.crt
+
+ca_trust_legacy_bundles:
+  file.absent:
+    - name: /opt/so/conf/ca
+
+# clean removes blocklist entries that are no longer shipped
+ca_trust_blocklist:
+  file.recurse:
+    - name: /etc/pki/ca-trust/source/blocklist
+    - source: salt://ca/files/blocklist
+    - clean: True
+    - makedirs: True
+    - file_mode: '0644'
+    - dir_mode: '0755'
+
+ca_trust_extract:
+  cmd.run:
+    - name: update-ca-trust extract
+    - onchanges:
+      - file: ca_trust_anchor
+      - file: ca_trust_legacy_anchor
+      - file: ca_trust_blocklist
@@ -438,6 +438,10 @@ install_networkminer:
  pkg.latest:
    - name: securityonion-networkminer

+install_so_crates:
+  pkg.latest:
+    - name: securityonion-so-crates
+
 {% else %}

 desktop_packages_os_fail:
@@ -1,18 +1,10 @@
- {% from 'vars/globals.map.jinja' import GLOBALS %}
+{% from 'vars/globals.map.jinja' import GLOBALS %}

 {# we only want this state to run it is CentOS #}
 {% if GLOBALS.os == 'OEL' %}

-trusted_ca:
-  file.managed:
-    - name: /etc/pki/ca-trust/source/anchors/ca.crt
-    - source: salt://ca/files/ca.crt
-
-update_ca_certs:
-  cmd.run:
-    - name: update-ca-trust
-    - onchanges:
-      - file: trusted_ca
+include:
+  - ca.trust

 {% else %}

@@ -103,7 +103,6 @@ docker:
        - 80:80
        - 443:443
        - 8443:8443
-        - 7788:7788
        - 7789:7789
      custom_bind_mounts: []
      extra_hosts: []
@@ -6,9 +6,11 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}

 include:
+  - ca.trust
  - elastalert.config
  - elastalert.sostatus

@@ -28,6 +30,7 @@ so-elastalert:
        - ipv4_address: {{ DOCKERMERGED.containers['so-elastalert'].ip }}
    - detach: True
    - binds:
+      - {{ CATRUST.dir }}
      - /opt/so/rules/elastalert:/opt/elastalert/rules/:ro
      - /opt/so/log/elastalert:/var/log/elastalert:rw
      - /opt/so/conf/elastalert/modules/:/opt/elastalert/modules/:ro
@@ -46,8 +49,9 @@ so-elastalert:
      - {{ XTRAHOST }}
        {% endfor %}
      {% endif %}
-      {% if DOCKERMERGED.containers['so-elastalert'].extra_env %}
    - environment:
+      - REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
+      {% if DOCKERMERGED.containers['so-elastalert'].extra_env %}
        {% for XTRAENV in DOCKERMERGED.containers['so-elastalert'].extra_env %}
      - {{ XTRAENV }}
        {% endfor %}
@@ -65,6 +69,7 @@ so-elastalert:
      - file: elastacustmodulesdir
      - file: elastaconf
    - watch:
+      - cmd: ca_trust_extract
      - file: elastaconf
    - onlyif:
      - "so-elasticsearch-query / | jq -r '.version.number[0:1]' | grep -q 9" {# only run this state if elasticsearch is version 9 #}
@@ -6,9 +6,11 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}

 include:
+  - ca.trust
  - elastic-fleet-package-registry.config
  - elastic-fleet-package-registry.sostatus

@@ -34,8 +36,9 @@ so-elastic-fleet-package-registry:
      {% for BINDING in DOCKERMERGED.containers['so-elastic-fleet-package-registry'].port_bindings %}
      - {{ BINDING }}
      {% endfor %}
-    {% if DOCKERMERGED.containers['so-elastic-fleet-package-registry'].custom_bind_mounts %}
    - binds:
+      - {{ CATRUST.pem }}:/etc/ssl/certs/ca-certificates.crt:ro
+    {% if DOCKERMERGED.containers['so-elastic-fleet-package-registry'].custom_bind_mounts %}
        {% for BIND in DOCKERMERGED.containers['so-elastic-fleet-package-registry'].custom_bind_mounts %}
      - {{ BIND }}
        {% endfor %}
@@ -52,6 +55,8 @@ so-elastic-fleet-package-registry:
      - {{ ULIMIT.name }}={{ ULIMIT.soft }}:{{ ULIMIT.hard }}
    {%   endfor %}
    {% endif %}
+    - watch:
+      - cmd: ca_trust_extract

 wait_for_so-elastic-fleet-package-registry:
  http.wait_for_successful_query:
@@ -6,6 +6,7 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}

 include:
@@ -37,6 +38,7 @@ so-elastic-agent:
      - {{ BINDING }}
      {% endfor %}
    - binds:
+      - {{ CATRUST.dir }}
      - /opt/so/conf/elastic-agent/elastic-agent.yml:/usr/share/elastic-agent/elastic-agent.yml:ro
      - /opt/so/log/elasticagent:/usr/share/elastic-agent/logs
      - /etc/pki/tls/certs/intca.crt:/etc/pki/tls/certs/intca.crt:ro 
@@ -65,6 +67,7 @@ so-elastic-agent:
      - file: create-elastic-agent-config
      - file: trusttheca
    - watch:
+      - cmd: ca_trust_extract
      - file: create-elastic-agent-config
      - file: trusttheca

@@ -6,6 +6,7 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'elasticfleet/map.jinja' import ELASTICFLEETMERGED %}

@@ -63,6 +64,7 @@ so-elastic-fleet:
      - {{ BINDING }}
      {% endfor %}
    - binds:
+      - {{ CATRUST.dir }}
      - /etc/pki/elasticfleet-server.crt:/etc/pki/elasticfleet-server.crt:ro
      - /etc/pki/elasticfleet-server.key:/etc/pki/elasticfleet-server.key:ro
      - /etc/pki/tls/certs/intca.crt:/etc/pki/tls/certs/intca.crt:ro
@@ -95,6 +97,7 @@ so-elastic-fleet:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: trusttheca
      - x509: etc_elasticfleet_key
      - x509: etc_elasticfleet_crt
@@ -1,49 +0,0 @@
-# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
-# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at 
-# https://securityonion.net/license; you may not use this file except in compliance with the
-# Elastic License 2.0.
-
-{% from 'allowed_states.map.jinja' import allowed_states %}
-{% if sls.split('.')[0] in allowed_states or sls in allowed_states %}
-{%   from 'vars/globals.map.jinja' import GLOBALS %}
-
-# Move our new CA over so Elastic and Logstash can use SSL with the internal CA
-catrustdir:
-  file.directory:
-    - name: /opt/so/conf/ca
-    - user: 939
-    - group: 939
-    - makedirs: True
-
-{%   if GLOBALS.is_manager %}
-# We have to add the Manager CA to the CA list
-catrustscript:
-  cmd.script:
-    - source: salt://elasticsearch/tools/sbin_jinja/so-catrust
-    - template: jinja
-    - cwd: /opt/so
-    - defaults:
-        GLOBALS: {{ GLOBALS }}
-{%   endif %}
-
-elasticsearch_cacerts:
-  file.managed:
-    - name: /opt/so/conf/ca/cacerts
-    - source: salt://elasticsearch/cacerts
-    - user: 939
-    - group: 939
-
-elasticsearch_capems:
-  file.managed:
-    - name: /opt/so/conf/ca/tls-ca-bundle.pem
-    - source: salt://elasticsearch/tls-ca-bundle.pem
-    - user: 939
-    - group: 939
-
-{% else %}
-
-{{sls}}_state_not_allowed:
-  test.fail_without_changes:
-    - name: {{sls}}_state_not_allowed
-
-{% endif %}
@@ -107,6 +107,12 @@ so-elasticsearch-system-indices-patch:
      - file: so-elasticsearch-system-indices-patch-script
 {%  endif %}

+so-elasticsearch-ilm-policy-reload:
+  file.absent:
+    - name: /opt/so/state/esilm.txt
+    - onchanges:
+      - file: so-elasticsearch-ilm-policy-load-script
+
 # heavynodes will only load ILM policies for SO managed indices. (Indicies defined in elasticsearch/defaults.yaml)
 so-elasticsearch-ilm-policy-load:
  cmd.run:
@@ -115,8 +121,9 @@ so-elasticsearch-ilm-policy-load:
    - require:
      - docker_container: so-elasticsearch
      - file: so-elasticsearch-ilm-policy-load-script
-    - onchanges:
-      - file: so-elasticsearch-ilm-policy-load-script
+      - http: wait_for_so-elasticsearch
+      - file: so-elasticsearch-ilm-policy-reload
+    - unless: test -f /opt/so/state/esilm.txt

 so-elasticsearch-templates-reload:
  file.absent:
@@ -6,6 +6,7 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'elasticsearch/config.map.jinja' import ELASTICSEARCH_NODES %}
 {%   from 'elasticsearch/config.map.jinja' import ELASTICSEARCH_SEED_HOSTS %}
@@ -13,7 +14,6 @@

 include:
  - ca
-  - elasticsearch.ca
  - elasticsearch.ssl
  - elasticsearch.config
  - elasticsearch.sostatus
@@ -63,15 +63,14 @@ so-elasticsearch:
      - {{ BINDING }}
      {% endfor %}
    - binds:
+      - {{ CATRUST.dir }}
      - /opt/so/conf/elasticsearch/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml:ro
      - /opt/so/conf/elasticsearch/log4j2.properties:/usr/share/elasticsearch/config/log4j2.properties:ro
      - /nsm/elasticsearch:/usr/share/elasticsearch/data:rw
      - /opt/so/log/elasticsearch:/var/log/elasticsearch:rw
-      - /opt/so/conf/ca/cacerts:/usr/share/elasticsearch/jdk/lib/security/cacerts:ro
      - /etc/pki/tls/certs/intca.crt:/usr/share/elasticsearch/config/ca.crt:ro
      - /etc/pki/elasticsearch.crt:/usr/share/elasticsearch/config/elasticsearch.crt:ro
      - /etc/pki/elasticsearch.key:/usr/share/elasticsearch/config/elasticsearch.key:ro
-      - /etc/pki/elasticsearch.p12:/usr/share/elasticsearch/config/elasticsearch.p12:ro
      - /opt/so/conf/elasticsearch/users_roles:/usr/share/elasticsearch/config/users_roles:ro
      - /opt/so/conf/elasticsearch/users:/usr/share/elasticsearch/config/users:ro
      {% if ELASTICSEARCHMERGED.config.path.get('repo', False) %}
@@ -85,21 +84,19 @@ so-elasticsearch:
        {% endfor %}
      {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: trusttheca
      - x509: elasticsearch_crt
      - x509: elasticsearch_key
-      - file: elasticsearch_cacerts
      - file: esyml
    - require:
      - file: trusttheca
      - x509: elasticsearch_crt
      - x509: elasticsearch_key
-      - file: elasticsearch_cacerts
      - file: esyml
      - file: eslog4jfile
      - file: nsmesdir
      - file: eslogdir
-      - file: elasticp12perms
      - cmd: auth_users_roles_inode
      - cmd: auth_users_inode

@@ -38,10 +38,6 @@ elasticsearch_crt:
    - retry:
        attempts: 5
        interval: 30
-  cmd.run:
-    - name: "/usr/bin/openssl pkcs12 -inkey /etc/pki/elasticsearch.key -in /etc/pki/elasticsearch.crt -export -out /etc/pki/elasticsearch.p12 -nodes -passout pass:"
-    - onchanges:
-      - x509: /etc/pki/elasticsearch.key

 elastickeyperms:
  file.managed:
@@ -50,13 +46,6 @@ elastickeyperms:
    - mode: 640
    - group: 930
    
-elasticp12perms:
-  file.managed:
-    - replace: False
-    - name: /etc/pki/elasticsearch.p12
-    - mode: 640
-    - group: 930
-
 {% else %}

 {{sls}}_state_not_allowed:
@@ -1,26 +0,0 @@
-#!/bin/bash
-#
-# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
-# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at 
-# https://securityonion.net/license; you may not use this file except in compliance with the
-# Elastic License 2.0.
-
-. /usr/sbin/so-common
-get_elastic_agent_vars
-
-# Exit on errors, since all lines must succeed
-set -e
-
-# Check to see if we have extracted the ca cert.
-if [ ! -f /opt/so/saltstack/local/salt/elasticsearch/cacerts ]; then
-    docker run -v /etc/pki/ca.crt:/etc/ssl/ca.crt --name so-elasticsearchca --user root --entrypoint jdk/bin/keytool {{ GLOBALS.registry_host }}:5000/{{ GLOBALS.image_repo }}/so-elasticsearch:$ELASTIC_AGENT_TARBALL_VERSION -keystore /usr/share/elasticsearch/jdk/lib/security/cacerts -alias SOSCA -import -file /etc/ssl/ca.crt -storepass changeit -noprompt
-    # Make sure symbolic links are followed when copying from container
-    docker cp -L so-elasticsearchca:/usr/share/elasticsearch/jdk/lib/security/cacerts /opt/so/saltstack/local/salt/elasticsearch/cacerts
-    docker cp -L so-elasticsearchca:/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem /opt/so/saltstack/local/salt/elasticsearch/tls-ca-bundle.pem
-    docker rm so-elasticsearchca
-    echo "" >> /opt/so/saltstack/local/salt/elasticsearch/tls-ca-bundle.pem
-    echo "sosca" >> /opt/so/saltstack/local/salt/elasticsearch/tls-ca-bundle.pem
-    cat /etc/pki/ca.crt >> /opt/so/saltstack/local/salt/elasticsearch/tls-ca-bundle.pem
-else
-    exit 0
-fi
@@ -7,11 +7,17 @@
 . /usr/sbin/so-common

 MAX_JOBS=${MAX_ILM_JOBS:-10}
+ILM_STATEFILE=/opt/so/state/esilm.txt
+rm -f "$ILM_STATEFILE"

+check_elasticsearch_responsive
+
+ILM_TMP_DIR=$(mktemp -d)
+trap 'rm -rf "$ILM_TMP_DIR"' EXIT
 # Lock used to serialize block writes so concurrent jobs never interleave their output.
-ILM_OUTPUT_LOCK=$(mktemp)
-ILM_FAIL_FILE=$(mktemp)
-trap 'rm -f "$ILM_OUTPUT_LOCK" "$ILM_FAIL_FILE"' EXIT
+ILM_OUTPUT_LOCK="$ILM_TMP_DIR/output.lock"
+ILM_FAIL_FILE="$ILM_TMP_DIR/failed"
+touch "$ILM_FAIL_FILE"

 # Policies are loaded concurrently (up to MAX_JOBS at a time) for speed. Each policy's block is
 # printed the moment its curl returns, so output appears in COMPLETION ORDER, not the order
@@ -20,13 +26,30 @@ echo "Loading ILM policies concurrently; output below appears in completion orde
 echo

 put_policy() {
-  local desc="$1" policyname="$2" data="$3" result rc=0
-  if ! result=$(curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L --fail \
-    -X PUT "https://localhost:9200/_ilm/policy/${policyname}" \
-    -H 'Content-Type: application/json' -d"${data}" 2>&1); then
-    rc=1
-  elif ! jq -e '.acknowledged == true' <<<"$result" >/dev/null 2>&1; then
-    rc=1
+  local desc="$1" policyname="$2" data="$3" rc=1 attempt out http_code body result reason tmpdata
+  tmpdata=$(mktemp "$ILM_TMP_DIR/${policyname}.XXXXXX.json")
+  printf '%s' "$data" >"$tmpdata"
+
+  for attempt in 1 2 3; do
+    out=$(so-elasticsearch-query "_ilm/policy/${policyname}" -d@"${tmpdata}" -XPUT -w '\n%{http_code}' 2>/dev/null)
+    http_code=${out##*$'\n'} body=${out%$'\n'*}
+
+    if [[ "$http_code" == 200 ]] && jq -e '.acknowledged == true' <<<"$body" >/dev/null 2>&1; then
+      rc=0
+      break
+    fi
+    # 429 = ES rejected/throttled (may succeed later); other 4xx will not
+    [[ "$http_code" == 4?? && "$http_code" != 429 ]] && break
+    (( attempt < 3 )) && sleep 3
+  done
+
+  if (( rc == 0 )); then
+    result='{"acknowledged":true}'
+  else
+    result=${body:-"HTTP ${http_code:-000}"}
+    # flatten tabs/newlines so the TAB fail-file stays one record
+    reason=$(jq -r '(if (.error|type) == "object" then (.error.reason // .error.type) elif (.error|type) == "string" then .error else empty end) // empty | tostring | gsub("[\t\r\n]+"; " ")' <<<"$body" 2>/dev/null)
+    reason="HTTP ${http_code:-000}${reason:+: ${reason}}"
  fi

  # curl above ran in parallel; serialize just this block write so concurrent jobs never interleave.
@@ -34,7 +57,7 @@ put_policy() {
    flock 200
    printf 'Setting up %s policy...\n%s\n\n' "${desc}" "${result}"
    if (( rc != 0 )); then
-      printf '%s\n' "${policyname}" >>"$ILM_FAIL_FILE"
+      printf '%s\t%s\n' "${policyname}" "${reason}" >>"$ILM_FAIL_FILE"
    fi
  } 200>>"${ILM_OUTPUT_LOCK}"

@@ -82,10 +105,15 @@ wait || true

 if [[ -s "$ILM_FAIL_FILE" ]]; then
  echo "ERROR: Failed to load ILM policy(s):"
-  while read -r POLICY; do
-    echo " - $POLICY"
+  while IFS=$'\t' read -r POLICY REASON; do
+    if [[ -n "$REASON" ]]; then
+      echo " - ${POLICY}: ${REASON}"
+    else
+      echo " - ${POLICY}"
+    fi
  done < "$ILM_FAIL_FILE"
  exit 1
 else
  echo "Successfully loaded all ILM policies."
+  touch "$ILM_STATEFILE"
 fi
@@ -118,10 +118,6 @@ firewall:
      tcp:
        - 5601
      udp: []
-    localrules:
-      tcp:
-        - 7788
-      udp: []
    nginx:
      tcp:
        - 80
@@ -200,7 +196,6 @@ firewall:
                - postgres
                - elasticsearch_rest
                - elasticsearch_node
-                - localrules
            sensor:
              portgroups:
                - beats_5044
@@ -391,7 +386,6 @@ firewall:
                - elastic_agent_control
                - elastic_agent_data
                - elastic_agent_update
-                - localrules
                - sensoroni
            fleet:
              portgroups:
@@ -611,7 +605,6 @@ firewall:
                - elastic_agent_control
                - elastic_agent_data
                - elastic_agent_update
-                - localrules
                - sensoroni
            fleet:
              portgroups:
@@ -829,7 +822,6 @@ firewall:
                - elastic_agent_control
                - elastic_agent_data
                - elastic_agent_update
-                - localrules
                - sensoroni
            fleet:
              portgroups:
@@ -1058,7 +1050,6 @@ firewall:
                - elastic_agent_update
                - endgame
                - strelka_frontend
-                - localrules
            fleet:
              portgroups:
                - elasticsearch_rest
@@ -1518,7 +1509,6 @@ firewall:
                - elasticsearch_rest
                - elasticsearch_node
                - elastic_agent_control
-                - localrules
            sensor:
              portgroups:
                - beats_5044
@@ -138,9 +138,6 @@ firewall:
    kibana:
      tcp: *tcpsettings
      udp: *udpsettings
-    localrules:
-      tcp: *tcpsettings
-      udp: *udpsettings
    nginx:
      tcp: *tcpsettings
      udp: *udpsettings
@@ -13,9 +13,11 @@
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   if 'api' in salt['pillar.get']('features', []) %}

 include:
+  - ca.trust
  - hydra.config
  - hydra.sostatus

@@ -29,6 +31,7 @@ so-hydra:
      - soauth:
        - ipv4_address: {{ DOCKERMERGED.containers['so-hydra'].ips['soauth'] }}
    - binds:
+      - {{ CATRUST.dir }}
      - /opt/so/conf/hydra/:/hydra-conf:ro
      - /opt/so/log/hydra/:/hydra-log:rw
      - /nsm/hydra/db:/hydra-data:rw
@@ -60,6 +63,7 @@ so-hydra:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: hydraconfig
    - require:
      - file: hydraconfig
@@ -6,9 +6,11 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}

 include:
+  - ca.trust
  - idh.config
  - idh.sostatus

@@ -20,6 +22,7 @@ so-idh:
    - detach: True
    - network_mode: host
    - binds:
+      - {{ CATRUST.dir }}
      - /nsm/idh:/var/tmp:rw
      - /opt/so/conf/idh/http-skins:/opt/opencanary/http-skins:ro
      - /opt/so/conf/idh/opencanary.conf:/etc/opencanaryd/opencanary.conf:ro
@@ -34,12 +37,13 @@ so-idh:
      - {{ XTRAHOST }}
      {% endfor %}
    {% endif %}
-    {% if DOCKERMERGED.containers['so-idh'].extra_env %}
    - environment:
-      {% for XTRAENV in DOCKERMERGED.containers['so-idh'].extra_env %}
+      - REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
+      {% if DOCKERMERGED.containers['so-idh'].extra_env %}
+        {% for XTRAENV in DOCKERMERGED.containers['so-idh'].extra_env %}
      - {{ XTRAENV }}
-      {% endfor %}
-    {% endif %}
+        {% endfor %}
+      {% endif %}
    {% if DOCKERMERGED.containers['so-idh'].ulimits %}
    - ulimits:
    {%   for ULIMIT in DOCKERMERGED.containers['so-idh'].ulimits %}
@@ -47,6 +51,7 @@ so-idh:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: opencanary_config
    - require:
      - file: opencanary_config
@@ -6,11 +6,13 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   set PASSWORD = salt['pillar.get']('secrets:influx_pass') %}
 {%   set TOKEN = salt['pillar.get']('influxdb:token') %}

 include:
+  - ca.trust
  - influxdb.ssl
  - influxdb.config
  - influxdb.sostatus
@@ -38,6 +40,7 @@ so-influxdb:
        {% endfor %}
      {% endif %}
    - binds:
+      - {{ CATRUST.pem }}:/etc/ssl/certs/ca-certificates.crt:ro
      - /opt/so/log/influxdb/:/log:rw
      - /opt/so/conf/influxdb/config.yaml:/conf/config.yaml:ro
      - /opt/so/conf/influxdb/etc:/etc/influxdb2:rw
@@ -66,6 +69,7 @@ so-influxdb:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: influxdbconf
      - x509: influxdb_key
      - x509: influxdb_crt
@@ -12,12 +12,14 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   set KAFKANODES = salt['pillar.get']('kafka:nodes') %}
 {%   set KAFKA_EXTERNAL_ACCESS = salt['pillar.get']('kafka:config:external_access:enabled', default=False) %}
 {%   if 'gmd' in salt['pillar.get']('features', []) %}

 include:
+  - ca.trust
  - kafka.ca
  - kafka.config
  - kafka.ssl
@@ -51,6 +53,7 @@ so-kafka:
      - {{ BINDING }}
      {% endfor %}
    - binds:
+      - {{ CATRUST.java }}:/opt/java/openjdk/lib/security/cacerts:ro
      - /etc/pki/kafka.p12:/etc/pki/kafka.p12:ro
      - /opt/so/conf/kafka/kafka-truststore.jks:/etc/pki/kafka-truststore.jks:ro
      - /nsm/kafka/data/:/nsm/kafka/data/:rw
@@ -61,6 +64,11 @@ so-kafka:
      {% if KAFKA_EXTERNAL_ACCESS %}
      - /opt/so/conf/kafka/kafka_server_jaas.conf:/opt/kafka/config/kafka_server_jaas.conf:ro
      {% endif %}
+      {% if DOCKERMERGED.containers['so-kafka'].custom_bind_mounts %}
+        {% for BIND in DOCKERMERGED.containers['so-kafka'].custom_bind_mounts %}
+      - {{ BIND }}
+        {% endfor %}
+      {% endif %}
    {% if DOCKERMERGED.containers['so-kafka'].ulimits %}
    - ulimits:
    {%   for ULIMIT in DOCKERMERGED.containers['so-kafka'].ulimits %}
@@ -68,6 +76,7 @@ so-kafka:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      {% for sc in ['server', 'client'] %}
      - file: kafka_kraft_{{sc}}_properties
      {% endfor %}
@@ -8,8 +8,10 @@
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'elasticsearch/config.map.jinja' import ELASTICSEARCHMERGED %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}

 include:
+  - ca.trust
  - kibana.config
  - kibana.sostatus

@@ -27,6 +29,7 @@ so-kibana:
      - ELASTICSEARCH_HOST={{ GLOBALS.manager }}
      - ELASTICSEARCH_PORT=9200
      - MANAGER={{ GLOBALS.manager }}
+      - NODE_OPTIONS=--use-openssl-ca
      {% if DOCKERMERGED.containers['so-kibana'].extra_env %}
        {% for XTRAENV in DOCKERMERGED.containers['so-kibana'].extra_env %}
      - {{ XTRAENV }}
@@ -40,6 +43,7 @@ so-kibana:
      {% endfor %}
    {% endif %}
    - binds:
+      - {{ CATRUST.dir }}
      - /opt/so/conf/kibana/etc:/usr/share/kibana/config:rw
      - /opt/so/log/kibana:/var/log/kibana:rw
      - /opt/so/conf/kibana/customdashboards:/usr/share/kibana/custdashboards:ro
@@ -60,6 +64,7 @@ so-kibana:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: kibanaconfig

 wait_for_so-kibana:
@@ -7,8 +7,10 @@
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}

 include:
+  - ca.trust
  - kratos.config
  - kratos.sostatus

@@ -22,6 +24,7 @@ so-kratos:
      - soauth:
        - ipv4_address: {{ DOCKERMERGED.containers['so-kratos'].ips['soauth'] }}
    - binds:
+      - {{ CATRUST.dir }}
      - /opt/so/conf/kratos/:/kratos-conf:ro
      - /opt/so/log/kratos/:/kratos-log:rw
      - /nsm/kratos/db:/kratos-data:rw
@@ -53,6 +56,7 @@ so-kratos:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: kratosschema
      - file: kratosconfig
    - require:
@@ -6,6 +6,7 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'logstash/map.jinja' import LOGSTASH_MERGED %}
 {%   from 'logstash/map.jinja' import LOGSTASH_NODES %}
@@ -13,9 +14,6 @@

 include:
  - ca
-{%   if GLOBALS.role not in ['so-receiver','so-fleet'] %}
-  - elasticsearch.ca
-{%   endif %}
 {# Kafka ca runs on nodes that can run logstash for Kafka input / output. Only when Kafka is global pipeline #}
 {%   if GLOBALS.role in ['so-searchnode', 'so-manager', 'so-managerhype', 'so-managersearch', 'so-receiver', 'so-standalone'] and GLOBALS.pipeline == 'KAFKA' %}
  - kafka.ca
@@ -58,6 +56,8 @@ so-logstash:
      - {{ BINDING }}
      {% endfor %}
    - binds:
+      - {{ CATRUST.dir }}
+      - {{ CATRUST.java }}:/usr/share/logstash/jdk/lib/security/cacerts:ro
      - /opt/so/conf/elasticsearch/templates/:/templates/:ro
      - /opt/so/conf/logstash/etc/:/usr/share/logstash/config/:ro
      - /opt/so/conf/logstash/pipelines:/usr/share/logstash/pipelines:ro
@@ -78,10 +78,6 @@ so-logstash:
      - /etc/pki/filebeat.p8:/usr/share/logstash/filebeat.key:ro
          {% endif %}
      {% endif %}
-      {% if GLOBALS.role not in ['so-receiver','so-fleet'] %}
-      - /opt/so/conf/ca/cacerts:/etc/pki/ca-trust/extracted/java/cacerts:ro
-      - /opt/so/conf/ca/tls-ca-bundle.pem:/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem:ro
-      {% endif %}
      {% if GLOBALS.pipeline == "KAFKA" and GLOBALS.role in ['so-manager', 'so-managerhype', 'so-managersearch', 'so-standalone', 'so-searchnode'] %}
      - /etc/pki/kafka-logstash.p12:/usr/share/logstash/kafka-logstash.p12:ro
      - /opt/so/conf/kafka/kafka-truststore.jks:/etc/pki/kafka-truststore.jks:ro
@@ -104,14 +100,11 @@ so-logstash:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: lsetcsync
      - file: lslog4j2
      - file: lspipelinesyml
      - file: trusttheca
-      {% if GLOBALS.is_manager %}
-      - file: elasticsearch_cacerts
-      - file: elasticsearch_capems
-      {% endif %}
      {% if GLOBALS.is_manager or GLOBALS.role in ['so-fleet', 'so-heavynode', 'so-receiver'] %}
      - x509: etc_elasticfleet_logstash_crt
      - x509: etc_elasticfleet_logstash_key
@@ -133,10 +126,6 @@ so-logstash:
      {% endif %}
    - require:
      - file: trusttheca
-      {% if GLOBALS.is_manager %}
-      - file: elasticsearch_cacerts
-      - file: elasticsearch_capems
-      {% endif %}
      {% if GLOBALS.is_manager or GLOBALS.role in ['so-fleet', 'so-heavynode', 'so-receiver'] %}
      - x509: etc_elasticfleet_logstash_crt
      - x509: etc_elasticfleet_logstash_key
@@ -20,6 +20,7 @@ def showUsage(args):
    print('    append           - Append a list item to a yaml key, if it exists and is a list. Requires KEY and LISTITEM args.', file=sys.stderr)
    print('    appendlistobject - Append an object to a yaml list key. Requires KEY and JSON_OBJECT args.', file=sys.stderr)
    print('    removelistitem   - Remove a list item from a yaml key, if it exists and is a list. Requires KEY and LISTITEM args.', file=sys.stderr)
+    print('    purgelistitem    - Remove every occurrence of a list item from all lists nested under a yaml key, if it exists. Requires KEY and LISTITEM args.', file=sys.stderr)
    print('    replacelistobject  - Replace a list object based on a condition. Requires KEY, CONDITION_FIELD, CONDITION_VALUE, and JSON_OBJECT args.', file=sys.stderr)
    print('    add              - Add a new key and set its value. Fails if key already exists. Requires KEY and VALUE args.', file=sys.stderr)
    print('    get [-r]         - Displays (to stdout) the value stored in the given key. Requires KEY arg. Use -r for raw output without YAML formatting.', file=sys.stderr)
@@ -199,6 +200,33 @@ def removelistitem(args):
    return 0


+def purgeListItem(content, listItem):
+    if isinstance(content, dict):
+        for value in content.values():
+            purgeListItem(value, listItem)
+    elif isinstance(content, list):
+        content[:] = [item for item in content if item != listItem]
+        for item in content:
+            purgeListItem(item, listItem)
+
+
+def purgelistitem(args):
+    if len(args) != 3:
+        print('Missing filename, key arg, or list item to purge', file=sys.stderr)
+        showUsage(None)
+        return 1
+
+    filename = args[0]
+    key = args[1]
+    listItem = args[2]
+
+    content = loadYaml(filename)
+    purgeListItem(getKeyValue(content, key), convertType(listItem))
+    writeYaml(filename, content)
+
+    return 0
+
+
 def replaceListObjectByCondition(content, key, conditionField, conditionValue, newObject):
    pieces = key.split(".", 1)
    if len(pieces) > 1:
@@ -385,6 +413,7 @@ def main():
        "append": append,
        "appendlistobject": appendlistobject,
        "removelistitem": removelistitem,
+        "purgelistitem": purgelistitem,
        "replacelistobject": replacelistobject,
        "get": get,
        "remove": remove,
@@ -709,6 +709,223 @@ class TestRemoveListItem(unittest.TestCase):
                self.assertEqual("The existing value for the given key is not a list. No action was taken on the file.\n", mock_stderr.getvalue())


+class TestPurgeListItem(unittest.TestCase):
+
+    def test_purgelistitem_missing_arg(self):
+        with patch('sys.exit', new=MagicMock()) as sysmock:
+            with patch('sys.stderr', new=StringIO()) as mock_stderr:
+                sys.argv = ["cmd", "help"]
+                soyaml.purgelistitem(["file", "key"])
+                sysmock.assert_called()
+                self.assertIn("Missing filename, key arg, or list item to purge", mock_stderr.getvalue())
+
+    def test_purgelistitem(self):
+        filename = "/tmp/so-yaml_test-purgelistitem.yaml"
+        file = open(filename, "w")
+        file.write("{key1: [a,b,c], key2: false}")
+        file.close()
+
+        soyaml.purgelistitem([filename, "key1", "b"])
+
+        file = open(filename, "r")
+        actual = file.read()
+        file.close()
+
+        expected = "key1:\n- a\n- c\nkey2: false\n"
+        self.assertEqual(actual, expected)
+
+    def test_purgelistitem_nested_lists(self):
+        filename = "/tmp/so-yaml_test-purgelistitem.yaml"
+        file = open(filename, "w")
+        file.write("{key1: { child1: [a,b,c], child2: { deep1: [b,d], deep2: 45 } }, key2: [b,e]}")
+        file.close()
+
+        soyaml.purgelistitem([filename, "key1", "b"])
+
+        file = open(filename, "r")
+        actual = file.read()
+        file.close()
+
+        expected = "key1:\n  child1:\n  - a\n  - c\n  child2:\n    deep1:\n    - d\n    deep2: 45\nkey2:\n- b\n- e\n"
+        self.assertEqual(actual, expected)
+
+    def test_purgelistitem_duplicates(self):
+        filename = "/tmp/so-yaml_test-purgelistitem.yaml"
+        file = open(filename, "w")
+        file.write("{key1: [b,a,b,c,b]}")
+        file.close()
+
+        soyaml.purgelistitem([filename, "key1", "b"])
+
+        file = open(filename, "r")
+        actual = file.read()
+        file.close()
+
+        expected = "key1:\n- a\n- c\n"
+        self.assertEqual(actual, expected)
+
+    def test_purgelistitem_list_of_objects(self):
+        filename = "/tmp/so-yaml_test-purgelistitem.yaml"
+        file = open(filename, "w")
+        file.write("{key1: [{name: item1, values: [a,b]}, b, {name: item2, values: [b,c]}]}")
+        file.close()
+
+        soyaml.purgelistitem([filename, "key1", "b"])
+
+        file = open(filename, "r")
+        actual = file.read()
+        file.close()
+
+        expected = "key1:\n- name: item1\n  values:\n  - a\n- name: item2\n  values:\n  - c\n"
+        self.assertEqual(actual, expected)
+
+    def test_purgelistitem_typed(self):
+        filename = "/tmp/so-yaml_test-purgelistitem.yaml"
+        file = open(filename, "w")
+        file.write("{key1: [1, '1', 2]}")
+        file.close()
+
+        soyaml.purgelistitem([filename, "key1", "1"])
+
+        file = open(filename, "r")
+        actual = file.read()
+        file.close()
+
+        expected = "key1:\n- '1'\n- 2\n"
+        self.assertEqual(actual, expected)
+
+    def test_purgelistitem_item_not_in_list(self):
+        filename = "/tmp/so-yaml_test-purgelistitem.yaml"
+        file = open(filename, "w")
+        file.write("{key1: { child1: [a,b,c] }}")
+        file.close()
+
+        soyaml.purgelistitem([filename, "key1", "d"])
+
+        file = open(filename, "r")
+        actual = file.read()
+        file.close()
+
+        expected = "key1:\n  child1:\n  - a\n  - b\n  - c\n"
+        self.assertEqual(actual, expected)
+
+    def test_purgelistitem_key_noexist(self):
+        filename = "/tmp/so-yaml_test-purgelistitem.yaml"
+        file = open(filename, "w")
+        file.write("{key1: [a,b,c]}")
+        file.close()
+
+        result = soyaml.purgelistitem([filename, "key2.child1", "b"])
+        self.assertEqual(result, 0)
+
+        file = open(filename, "r")
+        actual = file.read()
+        file.close()
+
+        expected = "key1:\n- a\n- b\n- c\n"
+        self.assertEqual(actual, expected)
+
+    def test_purgelistitem_key_noexist_deep(self):
+        filename = "/tmp/so-yaml_test-purgelistitem.yaml"
+        file = open(filename, "w")
+        file.write("{key1: { child1: [a,b,c] }}")
+        file.close()
+
+        result = soyaml.purgelistitem([filename, "key1.child2.deep1", "b"])
+        self.assertEqual(result, 0)
+
+        file = open(filename, "r")
+        actual = file.read()
+        file.close()
+
+        expected = "key1:\n  child1:\n  - a\n  - b\n  - c\n"
+        self.assertEqual(actual, expected)
+
+    def test_purgelistitem_scalar_and_null_values(self):
+        filename = "/tmp/so-yaml_test-purgelistitem.yaml"
+        file = open(filename, "w")
+        file.write("{key1: { child1: b, child2: null, child3: [b] }}")
+        file.close()
+
+        soyaml.purgelistitem([filename, "key1", "b"])
+
+        file = open(filename, "r")
+        actual = file.read()
+        file.close()
+
+        expected = "key1:\n  child1: b\n  child2: null\n  child3: []\n"
+        self.assertEqual(actual, expected)
+
+    def test_purgelistitem_lists_of_lists(self):
+        filename = "/tmp/so-yaml_test-purgelistitem.yaml"
+        file = open(filename, "w")
+        file.write("{key1: [[a,b], [b,[b,c]], b]}")
+        file.close()
+
+        soyaml.purgelistitem([filename, "key1", "b"])
+
+        file = open(filename, "r")
+        actual = file.read()
+        file.close()
+
+        expected = "key1:\n- - a\n- - - c\n"
+        self.assertEqual(actual, expected)
+
+    def test_purgelistitem_file_object(self):
+        filename = "/tmp/so-yaml_test-purgelistitem.yaml"
+        file = open(filename, "w")
+        file.write("{key1: [{name: x}, a, {name: y}, {name: x}]}")
+        file.close()
+
+        itemfile = "/tmp/so-yaml_test-purgelistitem-item.yaml"
+        file = open(itemfile, "w")
+        file.write("name: x\n")
+        file.close()
+
+        soyaml.purgelistitem([filename, "key1", "file:" + itemfile])
+
+        file = open(filename, "r")
+        actual = file.read()
+        file.close()
+
+        expected = "key1:\n- a\n- name: y\n"
+        self.assertEqual(actual, expected)
+
+    def test_purgelistitem_main(self):
+        filename = "/tmp/so-yaml_test-purgelistitem.yaml"
+        file = open(filename, "w")
+        file.write("{key1: { child1: [a,b,c] }, key2: [b]}")
+        file.close()
+
+        with patch('sys.exit', new=MagicMock()) as sysmock:
+            sys.argv = ["cmd", "purgelistitem", filename, "key1", "b"]
+            soyaml.main()
+            sysmock.assert_called_once_with(0)
+
+        file = open(filename, "r")
+        actual = file.read()
+        file.close()
+
+        expected = "key1:\n  child1:\n  - a\n  - c\nkey2:\n- b\n"
+        self.assertEqual(actual, expected)
+
+    def test_purgelistitem_with_remove(self):
+        filename = "/tmp/so-yaml_test-purgelistitem.yaml"
+        file = open(filename, "w")
+        file.write("{key1: { child1: { deep1: [a] }, child2: [b] }, key2: { group1: { items: [x,child1,y] }, group2: { items: [child1] } }}")
+        file.close()
+
+        soyaml.remove([filename, "key1.child1"])
+        soyaml.purgelistitem([filename, "key2", "child1"])
+
+        file = open(filename, "r")
+        actual = file.read()
+        file.close()
+
+        expected = "key1:\n  child2:\n  - b\nkey2:\n  group1:\n    items:\n    - x\n    - y\n  group2:\n    items: []\n"
+        self.assertEqual(actual, expected)
+
+
 class TestAppendListObject(unittest.TestCase):

    def test_appendlistobject_missing_arg(self):
@@ -121,9 +121,6 @@ check_err() {
      161)
        echo 'Required intermediate Elasticsearch upgrade not complete'
      ;;
-      162)
-        echo 'One or more Elastic Agent nodes do not support the x86-64-v3 CPU instruction set'
-      ;;
      170)
        echo "Intermediate upgrade completed successfully to $next_step_so_version, but next soup to Security Onion $originally_requested_so_version could not be started automatically."
        echo "Start soup again manually to continue the upgrade to Security Onion $originally_requested_so_version."
@@ -351,83 +348,6 @@ check_cluster_health() {
  exit 0
 }

-no_soup_for_you() {
-  echo ""
-  echo "No soup for you!"
-  exit 162
-}
-
-check_cpu_compatibility() {
-  # Roles running a container built from the so-elastic-agent image; mirrors the
-  # elasticagent and elasticfleet entries in salt/reactor/pillar_push_map.yaml.
-  local cpu_target='G@role:so-heavynode or G@role:so-eval or G@role:so-fleet or G@role:so-import or G@role:so-manager or G@role:so-managerhype or G@role:so-managersearch or G@role:so-standalone'
-  local expected_nodes cpu_results node result confirm
-  local -a unsupported=() offline=()
-
-  echo "Checking that Elastic Agent nodes support the x86-64-v3 CPU instruction set now required by Elastic."
-
-  if [[ "$SKIP_CPU_CHECK" == "true" ]]; then
-    printf "\nSkipping the x86-64-v3 CPU check because --skip-cpu-check was specified.\n\n"
-    return
-  fi
-
-  # Nodes that never answer are absent from the results, so diff against who should have.
-  expected_nodes=$(salt -C "$cpu_target" --preview-target --out=json 2>/dev/null | jq -r '.[]?') || true
-  if [[ -z "$expected_nodes" ]]; then
-    printf "\nCould not determine which nodes run the Elastic Agent, so the x86-64-v3 CPU check cannot run.\n"
-    no_soup_for_you
-  fi
-
-  cpu_results=$(salt -t 30 -C "$cpu_target" cmd.run "/lib64/ld-linux-x86-64.so.2 --help | grep x86-64-v3" --out=json 2>/dev/null) || true
-
-  while IFS= read -r node; do
-    [[ -z "$node" ]] && continue
-    result=$(jq -r --arg node "$node" '.[$node] // empty' <<< "$cpu_results" 2>/dev/null)
-    if [[ -z "$result" || "$result" == *"did not return"* ]]; then
-      offline+=("$node")
-    elif [[ "$result" != *"x86-64-v3 (supported"* ]]; then
-      # glibc appends "(supported, searched)" only when supported; the open paren keeps
-      # this from matching a future "(unsupported".
-      unsupported+=("$node")
-    fi
-  done <<< "$expected_nodes"
-
-  if [[ ${#unsupported[@]} -eq 0 && ${#offline[@]} -eq 0 ]]; then
-    printf "\nAll Elastic Agent nodes support x86-64-v3. We can proceed with SOUP.\n\n"
-    return
-  fi
-
-  echo ""
-  if [[ ${#unsupported[@]} -gt 0 ]]; then
-    echo "The following node(s) do NOT support the x86-64-v3 CPU instruction set:"
-    printf '  %s\n' "${unsupported[@]}"
-    echo ""
-    echo "Upstream Elastic now builds its binaries for x86-64-v3, so these nodes can no"
-    echo "longer run Elastic. Upgrading them WILL BREAK them."
-    echo ""
-  fi
-  if [[ ${#offline[@]} -gt 0 ]]; then
-    echo "The following node(s) did not respond and could not be checked:"
-    printf '  %s\n' "${offline[@]}"
-    echo ""
-    echo "These nodes are offline, so we cannot confirm they support x86-64-v3, which"
-    echo "upstream Elastic now requires."
-    echo ""
-  fi
-
-  if [[ -n $UNATTENDED ]]; then
-    echo "Unattended mode cannot prompt for an override. Re-run soup interactively, or pass --skip-cpu-check to bypass this check."
-    no_soup_for_you
-  fi
-
-  read -rp "Type 'override' to continue anyway, or press Enter to exit: " confirm
-  if [[ "${confirm,,}" == "override" ]]; then
-    printf "\nOverride accepted. Continuing at your own risk.\n\n"
-  else
-    no_soup_for_you
-  fi
-}
-
 check_fleet_server() {
  echo "Checking that Elastic Fleet Server is responding."
  # Modeled on the wait_for_so-elastic-fleet state check in elasticfleet/enabled.sls,
@@ -607,6 +527,7 @@ preupgrade_changes() {
    [[ "$INSTALLEDVERSION" == "3.1.0" ]] && up_to_3.2.0
    [[ "$INSTALLEDVERSION" == "3.2.0" ]] && up_to_3.3.0
    [[ "$INSTALLEDVERSION" == "3.3.0" ]] && up_to_3.4.0
+    [[ "$INSTALLEDVERSION" == "3.4.0" ]] && up_to_3.5.0
    true
 }

@@ -1261,9 +1182,15 @@ post_to_3.4.0() {

  add_detection_runtime_fields

+  remove_frozen_ca_bundles
+
  set_postversion 3.4.0
 }

+remove_frozen_ca_bundles() {
+  rm -f /opt/so/saltstack/local/salt/elasticsearch/cacerts /opt/so/saltstack/local/salt/elasticsearch/tls-ca-bundle.pem >> $SOUP_LOG 2>&1
+}
+
 add_detection_runtime_fields() {
  local template=$DEFAULT_SALT_DIR/salt/elasticsearch/templates/component/so/detection-mappings.json
  local runtime
@@ -1276,6 +1203,25 @@ add_detection_runtime_fields() {
 }
 ### 3.4.0 End ###

+### 3.5.0 Scripts ###
+up_to_3.5.0() {
+  remove_localrules_portgroup
+
+  INSTALLEDVERSION=3.5.0
+}
+
+remove_localrules_portgroup() {
+  local pillar_file
+
+  for pillar_file in /opt/so/saltstack/local/pillar/firewall/{soc,adv}_firewall.sls; do
+    grep -qs localrules "$pillar_file" || continue
+    echo "Removing the localrules portgroup from $pillar_file."
+    so-yaml.py remove "$pillar_file" firewall.portgroups.localrules
+    so-yaml.py purgelistitem "$pillar_file" firewall.role localrules
+  done
+}
+### 3.5.0 End ###
+

 repo_sync() {
  echo "Sync the local repo."
@@ -2162,8 +2108,6 @@ main() {
  echo "Let's see if we need to update Security Onion."
  upgrade_check

-  check_cpu_compatibility
-
  upgrade_space

  echo "Verifying Elasticsearch version compatibility across the grid before upgrading."
@@ -2442,17 +2386,6 @@ fi
  echo "### soup has been served at $(date) ###"
 }

-SKIP_CPU_CHECK=false
-declare -a SOUP_ARGS=()
-for arg in "$@"; do
-  if [[ "$arg" == "--skip-cpu-check" ]]; then
-    SKIP_CPU_CHECK=true
-  else
-    SOUP_ARGS+=("$arg")
-  fi
-done
-set -- "${SOUP_ARGS[@]}"
-
 while getopts ":b:f:y" opt; do
  case ${opt} in
    b )
@@ -2476,7 +2409,7 @@ while getopts ":b:f:y" opt; do
      ISOLOC="$OPTARG"
    ;;
    \? )
-      echo "Usage: soup [-b] [-y] [-f <iso location>] [--skip-cpu-check]"
+      echo "Usage: soup [-b] [-y] [-f <iso location>]"
      exit 1
    ;;
    : )
@@ -6,10 +6,12 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'nginx/map.jinja' import NGINXMERGED %}

 include:
+  - ca.trust
  - nginx.ssl
  - nginx.config
  - nginx.sostatus
@@ -47,13 +49,14 @@ so-nginx:
      {% endfor %}
    {% endif %}
    - binds:
+      - {{ CATRUST.pem }}:/etc/ssl/certs/ca-certificates.crt:ro
      - /opt/so/conf/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
      - /opt/so/log/nginx/:/var/log/nginx:rw
      - /opt/so/tmp/nginx/:/var/lib/nginx:rw
      - /opt/so/tmp/nginx/:/run:rw
-      - /nsm/elastic-fleet/so_agent-installers/:/opt/socore/html/packages
-      - /nsm/elastic-fleet/artifacts/:/opt/socore/html/artifacts 
+      - /nsm/elastic-fleet/artifacts/:/opt/socore/elastic-artifacts/artifacts:ro
      {% if GLOBALS.is_manager %}
+      - /nsm/elastic-fleet/so_agent-installers/:/opt/socore/html/packages
      - /etc/pki/managerssl.crt:/etc/pki/nginx/server.crt:ro
      - /etc/pki/managerssl.key:/etc/pki/nginx/server.key:ro
      # ATT&CK Navigator binds
@@ -61,7 +64,6 @@ so-nginx:
      - /opt/so/conf/navigator/config.json:/opt/socore/html/navigator/assets/config.json:ro
      - /nsm/repo:/opt/socore/html/repo:ro
      - /nsm/kernelrepo:/opt/socore/html/kernelrepo:ro
-      - /nsm/rules:/nsm/rules:ro
      {%   if NGINXMERGED.external_suricata %}
      - /opt/so/rules/nids/suri:/surirules:ro
      {%   endif %}
@@ -89,6 +91,7 @@ so-nginx:
      - {{ BINDING }}
      {% endfor %}
    - watch:
+      - cmd: ca_trust_extract
      - file: nginxconf
      - file: nginxconfdir
      {% if GLOBALS.is_manager %}
@@ -4,6 +4,8 @@
 {%- set role = grains.id.split('_') | last %}
 {%- set influxpass = salt['pillar.get']('secrets:influx_pass') %}
 {%- set influxauth = ('so:' + influxpass) | base64_encode %}
+{%- set all_names = [GLOBALS.hostname, GLOBALS.url_base] + NGINXMERGED.ssl.alt_names %}
+{%- set full_server_name = all_names | unique | join(' ') %}

 worker_processes auto;
 error_log /var/log/nginx/error.log;
@@ -39,52 +41,31 @@ http {

 	include /etc/nginx/conf.d/*.conf;

-	{%- if role in ['fleet'] %}
-
-	server {
-		listen 8443;
-		server_name {{ GLOBALS.hostname }};
-		root /opt/socore/html;
-		location /artifacts/ {
-			try_files $uri =206;
-			proxy_read_timeout    90;
-			proxy_connect_timeout 90;
-			proxy_set_header      Host $host;
-			proxy_set_header      X-Real-IP $remote_addr;
-			proxy_set_header      X-Forwarded-For $proxy_add_x_forwarded_for;
-			proxy_set_header      Proxy "";
-			proxy_set_header      X-Forwarded-Proto $scheme;
-		}
-	}
-
-    {%- endif %}
-
 	{%- if GLOBALS.is_manager %}
-	{%- set all_names = [GLOBALS.hostname, GLOBALS.url_base] + NGINXMERGED.ssl.alt_names %}
-	{%- set full_server_name = all_names | unique | join(' ') %}
-
 	server {
 		listen 80 default_server;
 		server_name _;
 		return 307 https://{{ GLOBALS.url_base }}$request_uri;
 	}
-
+  {%- endif %}
+  {%- if GLOBALS.is_manager or role == 'fleet' %}
 	server {
 		listen 8443;
+    {%- if GLOBALS.is_manager %}
 		server_name {{ full_server_name }};
-		root /opt/socore/html;
+    {%- else %}
+    server_name {{ GLOBALS.hostname }};
+    {%- endif %}
+		root /opt/socore/elastic-artifacts;
 		location /artifacts/ {
-			try_files $uri =206;
-			proxy_read_timeout    90;
-			proxy_connect_timeout 90;
-			proxy_set_header      Host $host;
-			proxy_set_header      X-Real-IP $remote_addr;
-			proxy_set_header      X-Forwarded-For $proxy_add_x_forwarded_for;
-			proxy_set_header      Proxy "";
-			proxy_set_header      X-Forwarded-Proto $scheme;
+			try_files $uri =404;
+		}
+		location / {
+			return 404;
 		}
 	}
-
+  {%- endif %}
+  {%- if GLOBALS.is_manager %}
 	server {
 		listen 443 ssl default_server;
 		http2 on;
@@ -111,21 +92,6 @@ http {
 	{%- endif %}

 	{%- if GLOBALS.is_manager %}
-
-	server {
-		listen 7788;
-		server_name {{ full_server_name }};
-		root /nsm/rules;
-		location / {
-			allow all;
-			sendfile on;
-			sendfile_max_chunk 1m;
-			autoindex on;
-			autoindex_exact_size off;
-			autoindex_format html;
-			autoindex_localtime on;
-		}
-	}
 	{%-   if NGINXMERGED.external_suricata %}
    server {
        listen 7789 ssl;
@@ -6,10 +6,12 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   set SO_POSTGRES_USER = salt['pillar.get']('postgres:auth:users:so_postgres_user:user', 'so_postgres') %}

 include:
+  - ca.trust
  - postgres.auth
  - postgres.ssl
  - postgres.config
@@ -42,6 +44,7 @@ so-postgres:
        {% endfor %}
      {% endif %}
    - binds:
+      - {{ CATRUST.dir }}
      - /opt/so/log/postgres/:/log:rw
      - /nsm/postgres:/var/lib/postgresql/data:rw
      - /opt/so/conf/postgres/postgresql.conf:/conf/postgresql.conf:ro
@@ -69,6 +72,7 @@ so-postgres:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: postgresconf
      - file: postgreshba
      - file: postgresinitdb
@@ -7,6 +7,7 @@
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}

 include:
  - ca
@@ -28,6 +29,7 @@ so-redis:
      - {{ BINDING }}
      {% endfor %}
    - binds:
+      - {{ CATRUST.pem }}:/etc/ssl/certs/ca-certificates.crt:ro
      - /opt/so/log/redis:/var/log/redis:rw
      - /opt/so/conf/redis/etc/redis.conf:/usr/local/etc/redis/redis.conf:ro
      - /opt/so/conf/redis/working:/redis:rw
@@ -60,6 +62,7 @@ so-redis:
    {% endif %}
    - entrypoint: "redis-server /usr/local/etc/redis/redis.conf"
    - watch:
+      - cmd: ca_trust_extract
      - file: trusttheca
      - x509: redis_crt
      - x509: redis_key
@@ -6,9 +6,11 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}

 include:
+  - ca.trust
  - registry.ssl
  - registry.config
  - registry.sostatus
@@ -29,6 +31,7 @@ so-dockerregistry:
      - {{ BINDING }}
      {% endfor %}
    - binds:
+      - {{ CATRUST.pem }}:/etc/ssl/certs/ca-certificates.crt:ro
      - /opt/so/conf/docker-registry/etc/config.yml:/etc/distribution/config.yml:ro
      - /opt/so/conf/docker-registry:/var/lib/registry:rw
      - /nsm/docker-registry/docker:/var/lib/registry/docker:rw
@@ -63,6 +66,7 @@ so-dockerregistry:
        attempts: 5
        interval: 30
    - watch:
+      - cmd: ca_trust_extract
      - x509: registry_crt
      - x509: registry_key
    - require:
@@ -4,10 +4,12 @@
 # Elastic License 2.0.

 {% from 'vars/globals.map.jinja' import GLOBALS %}
+{% from 'ca/trust.map.jinja' import CATRUST %}
 {% from 'docker/docker.map.jinja' import DOCKERMERGED %}


 include:
+  - ca.trust
  - sensoroni.config
  - sensoroni.sostatus

@@ -17,6 +19,7 @@ so-sensoroni:
    - restart_policy: unless-stopped
    - network_mode: host
    - binds:
+      - {{ CATRUST.dir }}
      - /nsm/import:/nsm/import:rw
      - /nsm/pcapout:/nsm/pcapout:rw
      - /opt/so/conf/sensoroni/sensoroni.json:/opt/sensoroni/sensoroni.json:ro
@@ -48,6 +51,7 @@ so-sensoroni:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: /opt/so/conf/sensoroni/sensoroni.json
    - require:
      - file: sensoroniagentconf
@@ -1441,6 +1441,8 @@ soc:
          rulesFingerprintFile: /opt/sensoroni/fingerprints/sigma.fingerprint
          stateFilePath: /opt/sensoroni/fingerprints/elastalertengine.state
          integrityCheckFrequencySeconds: 1200
+          partialSyncTimeoutSeconds: 300
+          bulkSyncTimeoutSeconds: 1800
          rulesRepos:
            default:
              - repo: https://github.com/Security-Onion-Solutions/securityonion-resources
@@ -1700,6 +1702,8 @@ soc:
          yaraRulesFolder: /opt/sensoroni/yara/rules
          stateFilePath: /opt/sensoroni/fingerprints/strelkaengine.state
          integrityCheckFrequencySeconds: 1200
+          partialSyncTimeoutSeconds: 300
+          bulkSyncTimeoutSeconds: 1800
        suricataengine:
          aiRepoUrl: https://github.com/Security-Onion-Solutions/securityonion-resources
          aiRepoBranch: generated-summaries-published
@@ -1715,6 +1719,8 @@ soc:
          rulesFingerprintFile: /opt/sensoroni/fingerprints/emerging-all.fingerprint
          stateFilePath: /opt/sensoroni/fingerprints/suricataengine.state
          integrityCheckFrequencySeconds: 1200
+          partialSyncTimeoutSeconds: 300
+          bulkSyncTimeoutSeconds: 1800
          ignoredSidRanges:
            - '1100000-1101000'
          rulesetSources:
@@ -6,6 +6,7 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'soc/merged.map.jinja' import DOCKER_EXTRA_HOSTS %}
 {%   from 'soc/merged.map.jinja' import SOCMERGED %}
@@ -27,6 +28,7 @@ so-soc:
      - soauth:
        - ipv4_address: {{ DOCKERMERGED.containers['so-soc'].ips['soauth'] }}
    - binds:
+      - {{ CATRUST.dir }}
      - /nsm/rules:/nsm/rules:rw
      - /opt/so/conf/strelka:/opt/sensoroni/yara:rw
      - /opt/so/conf/sigma:/opt/sensoroni/sigma:rw
@@ -62,6 +64,11 @@ so-soc:
      - /nsm/backup/detections-migration:/nsm/backup/detections-migration:ro
      - /opt/so/state:/opt/so/state:rw
      - /etc/pki/tls/certs/intca.crt:/opt/sensoroni/html/so-ca.crt:ro
+      {% if DOCKERMERGED.containers['so-soc'].custom_bind_mounts %}
+        {% for BIND in DOCKERMERGED.containers['so-soc'].custom_bind_mounts %}
+      - {{ BIND }}
+        {% endfor %}
+      {% endif %}
    - extra_hosts:
    {% for node in DOCKER_EXTRA_HOSTS %}
    {%   for hostname, ip in node.items() %}
@@ -90,6 +97,7 @@ so-soc:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: trusttheca
      - file: /opt/so/conf/soc/*
    - require:
@@ -370,6 +370,18 @@ soc:
            description: 'How often the ElastAlert integrity checker runs (in seconds). This verifies the integrity of deployed rules.'
            global: True
            advanced: True
+          partialSyncTimeoutSeconds:
+            description: 'How long to wait for Sigma rules to deploy after a detection is created, updated, or deleted. Must allow time for a scheduled rule sync already in progress to finish. Increase on systems with limited resources.'
+            global: True
+            advanced: True
+            forcedType: int
+            helpLink: sigma
+          bulkSyncTimeoutSeconds:
+            description: 'How long to wait for Sigma rules to deploy after a bulk enable, disable, or delete has saved its detections. Must allow time for a scheduled rule sync already in progress to finish. Increase on systems with limited resources.'
+            global: True
+            advanced: True
+            forcedType: int
+            helpLink: sigma
          rulesRepos:
            default: &eerulesRepos
              description: "Custom Git repositories to pull Sigma rules from. 'license' field is required, 'folder' is optional. 'community' disables some management options for the imported rules - they can't be deleted or edited, just tuned, duplicated and Enabled | Disabled. The new settings will be applied within 15 minutes. At that point, you will need to wait for the scheduled rule update to take place (by default, every 24 hours), or you can force the update by nagivating to Detections -->  Options dropdown menu --> Elastalert --> Full Update."
@@ -649,6 +661,18 @@ soc:
            description: 'How often the Strelka integrity checker runs (in seconds). This verifies the integrity of deployed rules.'
            global: True
            advanced: True
+          partialSyncTimeoutSeconds:
+            description: 'How long to wait for YARA rules to deploy after a detection is created, updated, or deleted. Must allow time for a scheduled rule sync already in progress to finish. Increase on systems with limited resources.'
+            global: True
+            advanced: True
+            forcedType: int
+            helpLink: yara
+          bulkSyncTimeoutSeconds:
+            description: 'How long to wait for YARA rules to deploy after a bulk enable, disable, or delete has saved its detections. Must allow time for a scheduled rule sync already in progress to finish. Increase on systems with limited resources.'
+            global: True
+            advanced: True
+            forcedType: int
+            helpLink: yara
          rulesRepos:
            default: &serulesRepos
              description: "Custom Git repositories to pull YARA rules from. 'license' field is required, 'folder' is optional. 'community' disables some management options for the imported rules - they can't be deleted or edited, just tuned, duplicated and Enabled | Disabled. The new settings will be applied within 15 minutes. At that point, you will need to wait for the scheduled rule update to take place (by default, every 24 hours), or you can force the update by nagivating to Detections -->  Options dropdown menu --> Strelka --> Full Update."
@@ -713,6 +737,18 @@ soc:
            description: 'How often the Suricata integrity checker runs (in seconds). This verifies the integrity of deployed rules.'
            global: True
            advanced: True
+          partialSyncTimeoutSeconds:
+            description: 'How long to wait for Suricata rules to deploy after a detection is created, updated, or deleted. Must allow time for a scheduled rule sync already in progress to finish. Increase on systems with limited resources.'
+            global: True
+            advanced: True
+            forcedType: int
+            helpLink: suricata
+          bulkSyncTimeoutSeconds:
+            description: 'How long to wait for Suricata rules to deploy after a bulk enable, disable, or delete has saved its detections. Must allow time for a scheduled rule sync already in progress to finish. Increase on systems with limited resources.'
+            global: True
+            advanced: True
+            forcedType: int
+            helpLink: suricata
          customRulesets:
            description: 'This setting is no longer used. Use Ruleset Sources setting instead.'
            global: True
@@ -7,8 +7,10 @@
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}

 include:
+  - ca.trust
  - strelka.backend.config
  - strelka.backend.sostatus

@@ -17,6 +19,7 @@ strelka_backend:
    - image: {{ GLOBALS.registry_host }}:5000/{{ GLOBALS.image_repo }}/so-strelka-backend:{{ GLOBALS.so_version }}
    - restart_policy: unless-stopped
    - binds:
+      - {{ CATRUST.dir }}
      - /opt/so/conf/strelka/backend/:/etc/strelka/:ro
      - /opt/so/conf/strelka/rules/compiled/:/etc/yara/:ro
      {% if DOCKERMERGED.containers['so-strelka-backend'].custom_bind_mounts %}
@@ -36,12 +39,14 @@ strelka_backend:
      - {{ XTRAHOST }}
        {% endfor %}
      {% endif %}
-    {% if DOCKERMERGED.containers['so-strelka-backend'].extra_env %}
    - environment:
-      {% for XTRAENV in DOCKERMERGED.containers['so-strelka-backend'].extra_env %}
+      - REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
+      - SSL_CERT_FILE=/etc/pki/tls/certs/ca-bundle.crt
+      {% if DOCKERMERGED.containers['so-strelka-backend'].extra_env %}
+        {% for XTRAENV in DOCKERMERGED.containers['so-strelka-backend'].extra_env %}
      - {{ XTRAENV }}
-      {% endfor %}
-    {% endif %}
+        {% endfor %}
+      {% endif %}
    {% if DOCKERMERGED.containers['so-strelka-backend'].ulimits %}
    - ulimits:
    {%   for ULIMIT in DOCKERMERGED.containers['so-strelka-backend'].ulimits %}
@@ -49,6 +54,7 @@ strelka_backend:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: strelkasensorcompiledrules
      - file: backend_backend_config
      - file: backend_logging_config
@@ -7,8 +7,10 @@
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}

 include:
+  - ca.trust
  - strelka.coordinator.config
  - strelka.coordinator.sostatus

@@ -39,6 +41,7 @@ strelka_coordinator:
      {% endfor %}
    {% endif %}
    - binds:
+      - {{ CATRUST.pem }}:/etc/ssl/certs/ca-certificates.crt:ro
      - /nsm/strelka/coord-redis-data:/data:rw
      {% if DOCKERMERGED.containers['so-strelka-coordinator'].custom_bind_mounts %}
        {% for BIND in DOCKERMERGED.containers['so-strelka-coordinator'].custom_bind_mounts %}
@@ -51,6 +54,9 @@ strelka_coordinator:
      - {{ ULIMIT.name }}={{ ULIMIT.soft }}:{{ ULIMIT.hard }}
    {%   endfor %}
    {% endif %}
+    - watch:
+      - cmd: ca_trust_extract
+
 delete_so-strelka-coordinator_so-status.disabled:
  file.uncomment:
    - name: /opt/so/conf/so-status/so-status.conf
@@ -7,8 +7,10 @@
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}

 include:
+  - ca.trust
  - strelka.filestream.config
  - strelka.filestream.sostatus

@@ -17,6 +19,7 @@ strelka_filestream:
    - image: {{ GLOBALS.registry_host }}:5000/{{ GLOBALS.image_repo }}/so-strelka-manager:{{ GLOBALS.so_version }}
    - restart_policy: unless-stopped
    - binds:
+      - {{ CATRUST.pem }}:/etc/ssl/certs/ca-certificates.crt:ro
      - /opt/so/conf/strelka/filestream/:/etc/strelka/:ro
      - /nsm/strelka:/nsm/strelka
      {% if DOCKERMERGED.containers['so-strelka-filestream'].custom_bind_mounts %}
@@ -49,6 +52,7 @@ strelka_filestream:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: filestream_config

 delete_so-strelka-filestream_so-status.disabled:
@@ -7,8 +7,10 @@
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}

 include:
+  - ca.trust
  - strelka.frontend.config
  - strelka.frontend.sostatus

@@ -17,6 +19,7 @@ strelka_frontend:
    - image: {{ GLOBALS.registry_host }}:5000/{{ GLOBALS.image_repo }}/so-strelka-manager:{{ GLOBALS.so_version }}
    - restart_policy: unless-stopped
    - binds:
+      - {{ CATRUST.pem }}:/etc/ssl/certs/ca-certificates.crt:ro
      - /opt/so/conf/strelka/frontend/:/etc/strelka/:ro
      - /nsm/strelka/log/:/var/log/strelka/:rw
      {% if DOCKERMERGED.containers['so-strelka-frontend'].custom_bind_mounts %}
@@ -54,6 +57,7 @@ strelka_frontend:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: frontend_config

 delete_so-strelka-frontend_so-status.disabled:
@@ -7,8 +7,10 @@
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}

 include:
+  - ca.trust
  - strelka.gatekeeper.config
  - strelka.gatekeeper.sostatus

@@ -33,6 +35,7 @@ strelka_gatekeeper:
      - {{ BINDING }}
      {% endfor %}
    - binds:
+      - {{ CATRUST.pem }}:/etc/ssl/certs/ca-certificates.crt:ro
      - /nsm/strelka/gk-redis-data:/data:rw
      {% if DOCKERMERGED.containers['so-strelka-gatekeeper'].custom_bind_mounts %}
        {% for BIND in DOCKERMERGED.containers['so-strelka-gatekeeper'].custom_bind_mounts %}
@@ -51,6 +54,8 @@ strelka_gatekeeper:
      - {{ ULIMIT.name }}={{ ULIMIT.soft }}:{{ ULIMIT.hard }}
    {%   endfor %}
    {% endif %}
+    - watch:
+      - cmd: ca_trust_extract

 delete_so-strelka-gatekeeper_so-status.disabled:
  file.uncomment:
@@ -7,8 +7,10 @@
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}

 include:
+  - ca.trust
  - strelka.manager.config
  - strelka.manager.sostatus

@@ -17,6 +19,7 @@ strelka_manager:
    - image: {{ GLOBALS.registry_host }}:5000/{{ GLOBALS.image_repo }}/so-strelka-manager:{{ GLOBALS.so_version }}
    - restart_policy: unless-stopped
    - binds:
+      - {{ CATRUST.pem }}:/etc/ssl/certs/ca-certificates.crt:ro
      - /opt/so/conf/strelka/manager/:/etc/strelka/:ro
      {% if DOCKERMERGED.containers['so-strelka-manager'].custom_bind_mounts %}
        {% for BIND in DOCKERMERGED.containers['so-strelka-manager'].custom_bind_mounts %}
@@ -48,6 +51,7 @@ strelka_manager:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: manager_config

 delete_so-strelka-manager_so-status.disabled:
@@ -6,11 +6,13 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'suricata/map.jinja' import SURICATAMERGED %}


 include:
+  - ca.trust
  - suricata.config
  - suricata.sostatus

@@ -33,6 +35,7 @@ so-suricata:
    {%   endfor %}
    {% endif %}
    - binds:
+      - {{ CATRUST.dir }}
      - /opt/so/conf/suricata/suricata.yaml:/etc/suricata/suricata.yaml:ro
      - /opt/so/conf/suricata/threshold.conf:/etc/suricata/threshold.conf:ro
      - /opt/so/conf/suricata/classification.config:/etc/suricata/classification.config:ro
@@ -55,6 +58,7 @@ so-suricata:
      {% endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: suriconfig
      - file: surithresholding
      - file: /opt/so/conf/suricata/bpf
@@ -2,6 +2,10 @@
 {% if sls in allowed_states %}

 {% from 'vars/globals.map.jinja' import GLOBALS %}
+{% from 'ca/trust.map.jinja' import CATRUST %}
+
+include:
+  - ca.trust

 so-tcpreplay:
  docker_container.running:
@@ -13,7 +17,10 @@ so-tcpreplay:
    - interactive: True
    - tty: True
    - binds:
+      - {{ CATRUST.dir }}
      - /opt/so/samples:/opt/so/samples:ro
+    - watch:
+      - cmd: ca_trust_extract


 {% else %}
@@ -6,6 +6,7 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}
 {%   from 'telegraf/map.jinja' import TELEGRAFMERGED %}

@@ -33,6 +34,7 @@ so-telegraf:
    - network_mode: host
    - init: True
    - binds:
+      - {{ CATRUST.pem }}:/etc/ssl/certs/ca-certificates.crt:ro
      - /opt/so/log/telegraf:/var/log/telegraf:rw
      - /opt/so/conf/telegraf/etc/telegraf.conf:/etc/telegraf/telegraf.conf:ro
      - /opt/so/conf/telegraf/node_config.json:/etc/telegraf/node_config.json:ro
@@ -73,6 +75,7 @@ so-telegraf:
    {%   endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: tgraf_sbin_jinja
      - file: lasthighstate_placeholder
      - file: trusttheca
@@ -6,10 +6,12 @@
 {% from 'allowed_states.map.jinja' import allowed_states %}
 {% if sls.split('.')[0] in allowed_states %}
 {%   from 'vars/globals.map.jinja' import GLOBALS %}
+{%   from 'ca/trust.map.jinja' import CATRUST %}
 {%   from 'docker/docker.map.jinja' import DOCKERMERGED %}


 include:
+  - ca.trust
  - zeek.config
  - zeek.sostatus

@@ -30,6 +32,7 @@ so-zeek:
    {%   endfor %}
    {% endif %}
    - binds:
+      - {{ CATRUST.dir }}
      - /nsm/zeek/logs:/nsm/zeek/logs:rw
      - /nsm/zeek/spool:/nsm/zeek/spool:rw
      - /nsm/zeek/extracted:/nsm/zeek/extracted:rw
@@ -63,6 +66,7 @@ so-zeek:
      {% endfor %}
    {% endif %}
    - watch:
+      - cmd: ca_trust_extract
      - file: /opt/so/conf/zeek/local.zeek
      - file: /opt/so/conf/zeek/node.cfg
      - file: /opt/so/conf/zeek/networks.cfg
@@ -976,27 +976,50 @@ docker_seed_update() {
 	set_progress_str "$docker_seed_update_percent" "Downloading $name"
 }

+registry_tar_has_soversion() {
+	local tarfile=$1
+	local version=$2
+
+	[ -n "$version" ] || return 1
+	[ -f "$tarfile" ] || return 1
+	# grep -q in a pipe SIGPIPEs tar; under pipefail a match would look like failure.
+	grep -qF "/_manifests/tags/${version}/" < <(tar -tf "$tarfile" 2>/dev/null)
+}
+
 docker_seed_registry() {
 	local VERSION="$SOVERSION"
+	local REGISTRY_TAR=/nsm/docker-registry/docker/registry.tar

-	if [ -f /nsm/docker-registry/docker/registry.tar ]; then
-		logCmd "tar xvf /nsm/docker-registry/docker/registry.tar -C /nsm/docker-registry/docker"
-		logCmd "rm /nsm/docker-registry/docker/registry.tar"
+	if [ -f "$REGISTRY_TAR" ] && registry_tar_has_soversion "$REGISTRY_TAR" "$VERSION"; then
+		info "Extracting $REGISTRY_TAR"
+		if ! tar xvf "$REGISTRY_TAR" -C /nsm/docker-registry/docker >> "$setup_log" 2>&1; then
+			error "Failed to extract $REGISTRY_TAR"
+			return 1
+		fi
+		logCmd "rm -f $REGISTRY_TAR"
+		return 0
+	elif [ -f "$REGISTRY_TAR" ]; then
+		info "Not using $REGISTRY_TAR; it has no images tagged '${VERSION}'"
+		if [[ "$is_airgap" == true ]]; then
+			error "Airgap install: $REGISTRY_TAR does not match Security Onion version '${VERSION}'. Use the ISO for this version."
+			return 1
+		fi
 	elif [[ -d /nsm/docker-registry/docker/registry && ( -f /etc/SOCLOUD || "$is_airgap" == true ) ]]; then
 		echo "Using existing docker registry content"
-	else
-		if [ "$install_type" == 'IMPORT' ]; then
-			container_list 'so-import'	
-		else
-			container_list
-		fi
-
-		docker_seed_update_percent=25
-
-		update_docker_containers 'netinstall' '' 'docker_seed_update' '/dev/stdout' 2>&1 | tee -a "$setup_log"
-        # Use pipe exit status of 'update_docker_containers' for return code
-		return ${PIPESTATUS[0]}
+		return 0
 	fi
+
+	if [ "$install_type" == 'IMPORT' ]; then
+		container_list 'so-import'	
+	else
+		container_list
+	fi
+
+	docker_seed_update_percent=25
+
+	update_docker_containers 'netinstall' '' 'docker_seed_update' '/dev/stdout' 2>&1 | tee -a "$setup_log"
+        # Use pipe exit status of 'update_docker_containers' for return code
+	return ${PIPESTATUS[0]}
 }

 elasticfleet_pillar() {
Author	SHA1	Message	Date
reyesj2	4e0558d2d3	remove unused localrules config	2026-11-10 17:30:01 -06:00
reyesj2	ec220572f0	dedup artifact registry config & split ea artifacts out of /opt/socore/html/ into dedicated /opt/socore/elastic-artifacts/artifacts/	2026-11-09 12:32:06 -06:00
Jorge Reyes	b7b00f8233	Merge pull request #16378 from Security-Onion-Solutions/reyesj2/fix-16369 surface ilm policy errors in so-elasticsearch-ilm-policy-load output	2026-11-05 14:05:29 -06:00
Jorge Reyes	45d2b29a42	Merge pull request #16374 from Security-Onion-Solutions/fix/skip-mismatched-iso-registry Skip leftover ISO registry.tar when tags do not match SOVERSION	2026-11-05 11:11:45 -06:00
Doug Burks	7b494439e7	Merge pull request #16377 from Security-Onion-Solutions/feature/desktop-so-crates Install securityonion-so-crates on Security Onion Desktop	2026-11-04 15:56:12 -05:00
Doug Burks	9a8dc097b2	Install securityonion-so-crates on Security Onion Desktop	2026-11-04 15:51:16 -05:00
reyesj2	274da18929	Surface ILM policy validation errors in highstate Fail 4xx ILM PUTs immediately (retry 429/5xx/connect), print the Elasticsearch reason in the salt summary, and retry the load on the next highstate via /opt/so/state/esilm.txt when the last run failed. Fixes https://github.com/Security-Onion-Solutions/sos/issues/277	2026-11-04 13:18:31 -06:00
reyesj2	99a8ad2d4b	Wait for Elasticsearch and retry ILM policy loads Require wait_for_so-elasticsearch, check ES is responsive before dispatch, and retry each policy PUT so soup no longer fails with empty output when ES is not ready yet. Fixes https://github.com/Security-Onion-Solutions/securityonion/issues/16369	2026-11-04 11:14:12 -06:00
Mike Reeves	bd9cdeadef	Merge pull request #16373 from Security-Onion-Solutions/foundry/mw1jl4boa6hshh Remove x86-64-v3 CPU pre-flight check from soup	2026-11-03 22:50:54 -05:00
Mike Reeves	bdf01a5a5a	Remove x86-64-v3 CPU check from soup	2026-11-03 22:13:58 -05:00
reyesj2	1849c74cea	Skip leftover ISO registry.tar when tags do not match SOVERSION Fixes #16372	2026-11-03 21:07:29 -06:00
Mike Reeves	021b069706	Merge pull request #16367 from Security-Onion-Solutions/TOoSmOotH-patch-1 Bump version from 3.4.0 to 3.5.0	2026-11-02 11:52:06 -05:00
Mike Reeves	824d674eec	Add version 3.5.0 to discussion template	2026-11-02 11:50:49 -05:00
Mike Reeves	d43791c63e	Bump version from 3.4.0 to 3.5.0	2026-11-02 11:50:08 -05:00
Mike Reeves	4dc7f2d97c	Merge pull request #16365 from Security-Onion-Solutions/3.4 3.4.0	2026-11-02 10:52:49 -05:00
Mike Reeves	092be7f222	3.4.0	2026-11-02 10:46:19 -05:00
Mike Reeves	340e7928f6	3.4.0	2026-10-30 11:36:47 -04:00
Jason Ertel	f5ff40bd20	Merge pull request #16361 from Security-Onion-Solutions/jertel/wip switch analyzer deps to py 314	2026-10-29 16:15:29 -04:00
coreyogburn	6794af6f68	Merge pull request #16358 from Security-Onion-Solutions/cogburn/sync-timeouts New Configurable Timeouts for Detection Engine Syncs	2026-10-28 11:34:02 -06:00
Corey Ogburn	691bef2cf4	New Configurable Timeouts for Detection Engine Syncs Each engine now has timeouts for how long to allow a sync to go on after detections are modified.	2026-10-28 10:48:03 -06:00
Mike Reeves	1bbaa213cb	Merge pull request #16354 from Security-Onion-Solutions/mreeves/remove-es-sokeys	2026-10-27 17:36:48 -04:00
Mike Reeves	5342765ca8	Merge pull request #16357 from Security-Onion-Solutions/mreeves/host-ca-trust Use the host CA trust store, with the grid CA, for all containers	2026-10-27 16:40:58 -04:00
Mike Reeves	02394acf69	Use the host CA trust store for sensor, strelka and idh containers Mount the host trust store into zeek, suricata, tcpreplay, the strelka containers and idh, and restart them when it changes. Point the Python requests clients in strelka-backend and idh at it instead of certifi. Remove the frozen /opt/so/conf/ca bundles from every node in ca.trust rather than only on the manager in soup.	2026-10-27 16:24:33 -04:00
Josh Patterson	d8f69f7457	Merge pull request #16356 from Security-Onion-Solutions/custombindmounts Add custom_bind_mounts to so-kafka and so-soc	2026-10-27 15:25:36 -04:00
Mike Reeves	11c8fe3093	Use the host CA trust store for all grid containers Anchor the grid CA and blocklist the eMudhra roots in the host trust store (new ca.trust state, also used by the desktop role), then mount the extracted store into each container instead of the frozen copies made by so-catrust. Containers restart when the host trust changes.	2026-10-27 15:08:21 -04:00
Mike Reeves	d6fecc761a	Remove unused elasticsearch.p12/sokeys keystore	2026-10-27 14:43:46 -04:00
Josh Patterson	40e3dafbee	Add custom_bind_mounts to so-kafka and so-soc Every container in docker/defaults.yaml exposes custom_bind_mounts via soc_docker.yaml, but kafka and soc enabled.sls never appended them to binds, so the setting was silently ignored for those two containers.	2026-10-27 14:06:58 -04:00
Jason Ertel	1fb7e1bc79	Merge pull request #16351 from Security-Onion-Solutions/jertel/wip fix false positive scan results	2026-10-26 15:50:37 -04:00
@@ -1 +1 @@
 .4.0
 .5.0