Compare commits

..
Author SHA1 Message Date
Mike Reeves 52037314be Remove outdated HOTFIX version number 2026-09-11 16:47:13 -04:00
Mike Reeves 9c12c10f96 Merge pull request #16239 from Security-Onion-Solutions/hotfix/3.3.0
Hotfix/3.3.0
2026-09-11 16:46:56 -04:00
Mike Reeves 9fc9be2cc9 Merge pull request #16237 from Security-Onion-Solutions/hf33
3.3.0 hotfix
2026-09-11 16:20:23 -04:00
Mike Reeves 7245843a3c 3.3.0 hotfix 2026-09-11 16:17:34 -04:00
Mike Reeves a1d17417ea 3.3.0 hotfix 2026-09-11 16:16:01 -04:00
Mike Reeves bee03d5bae 3.3.0 hotfix 2026-09-11 16:13:09 -04:00
coreyogburn 32d1274b80 Merge pull request #16236 from Security-Onion-Solutions/cogburn/move-fields
filterEventFields
2026-09-11 11:23:03 -06:00
Corey Ogburn 1624e8c094 filterEventFields
Previously hard coded array of fields is now a config option with the hard coded value as the default value.
2026-09-11 10:54:20 -06:00
Mike Reeves 3f3f091a7f Merge pull request #16235 from Security-Onion-Solutions/soup-x86-64-v3-check
Add x86-64-v3 CPU pre-flight check to soup
2026-09-11 11:00:34 -04:00
Mike Reeves cb48909578 Add x86-64-v3 CPU pre-flight check to soup
Upstream Elastic now ships binaries built for the x86-64-v3
micro-architecture level. This is not a Security Onion choice: nodes whose
CPUs predate x86-64-v3 can no longer run Elastic's own builds, so those
nodes break once they are upgraded.

Check for support before soup modifies anything, and require the operator
to type "override" to proceed when a node is unsupported or offline.

Runs after upgrade_check so a grid that is already current exits without
prompting. Targets only the roles that run a container built from the
so-elastic-agent image, using the role lists already maintained in
salt/reactor/pillar_push_map.yaml. Adds --skip-cpu-check to bypass the gate
for automation, and exit code 162 when the operator declines to override.
2026-09-11 10:51:25 -04:00
Mike Reeves 3057775770 Merge pull request #16234 from Security-Onion-Solutions/TOoSmOotH-patch-3
Update version number in HOTFIX file
2026-09-11 09:04:04 -04:00
Mike Reeves e4e8b90b9c Update version number in HOTFIX file 2026-09-11 09:03:24 -04:00
Mike Reeves 223ace6ff3 Merge pull request #16233 from Security-Onion-Solutions/TOoSmOotH-patch-2
Update HOTFIX version to 3.3.0-20260911
2026-09-11 08:57:31 -04:00
Mike Reeves 66e7863336 Update HOTFIX version to 3.3.0-20260911 2026-09-11 08:56:00 -04:00
Corey Ogburn 8f253d17a6 Default Memory to Disabled
Gives users a chance to reconfigure embed model before messages they send to the OnionAI get sent to SOAI by default.
2026-09-10 15:18:35 -06:00
Jason Ertel 191ee159ef Merge pull request #16229 from Security-Onion-Solutions/jertel/wip
fix location typo
2026-09-10 07:36:32 -04:00
Jason Ertel a8bfe955a5 fix location typo 2026-09-10 07:27:25 -04:00
Jason Ertel bd354abe83 Merge pull request #16225 from Security-Onion-Solutions/jertel/wip
/login is showing an nginx failure
2026-09-09 16:54:44 -04:00
Jason Ertel 37782fb45c /login is showing an nginx failure 2026-09-09 16:44:27 -04:00
Mike Reeves 8f14e96215 Merge pull request #16215 from Security-Onion-Solutions/3/dev
3.3.0
2026-09-08 15:00:13 -04:00
Mike Reeves 23a9daf7a2 Merge pull request #16125 from Security-Onion-Solutions/3/dev
3.2.0
2026-07-29 14:10:31 -04:00
21 changed files with 231 additions and 285 deletions

No files matched your search

+11 -11
View File
@@ -1,17 +1,17 @@
### 3.3.0-20260908 ISO image released on 2026/09/08 ### 3.3.0-20260911 ISO image released on 2026/09/11
### Download and Verify ### Download and Verify
3.3.0-20260908 ISO image: 3.3.0-20260911 ISO image:
https://download.securityonion.net/file/securityonion/securityonion-3.3.0-20260908.iso https://download.securityonion.net/file/securityonion/securityonion-3.3.0-20260911.iso
MD5: 5A2C42D0083F2D7B4DC2178C30EBC05F MD5: 12B18433D3A2198A185892FF79CF638F
SHA1: 505220A8A3315AFEE601C13772996018425CCD29 SHA1: 2B3C2E1FA7A78ED1F956E7EDCC12E32593C14EEE
SHA256: 6EB8401296A1D051FEC558C520D2D4AB1912A72D351A2426FBF8C87FEE2BA844 SHA256: 0938C73B76CE30EC9E4394D312C79EA7CAC721B6818541697279A6221F7D870D
Signature for ISO image: Signature for ISO image:
https://github.com/Security-Onion-Solutions/securityonion/raw/3/main/sigs/securityonion-3.3.0-20260908.iso.sig https://github.com/Security-Onion-Solutions/securityonion/raw/3/main/sigs/securityonion-3.3.0-20260911.iso.sig
Signing key: Signing key:
https://raw.githubusercontent.com/Security-Onion-Solutions/securityonion/3/main/KEYS https://raw.githubusercontent.com/Security-Onion-Solutions/securityonion/3/main/KEYS
@@ -25,22 +25,22 @@ wget https://raw.githubusercontent.com/Security-Onion-Solutions/securityonion/3/
Download the signature file for the ISO: Download the signature file for the ISO:
``` ```
wget https://github.com/Security-Onion-Solutions/securityonion/raw/3/main/sigs/securityonion-3.3.0-20260908.iso.sig wget https://github.com/Security-Onion-Solutions/securityonion/raw/3/main/sigs/securityonion-3.3.0-20260911.iso.sig
``` ```
Download the ISO image: Download the ISO image:
``` ```
wget https://download.securityonion.net/file/securityonion/securityonion-3.3.0-20260908.iso wget https://download.securityonion.net/file/securityonion/securityonion-3.3.0-20260911.iso
``` ```
Verify the downloaded ISO image using the signature file: Verify the downloaded ISO image using the signature file:
``` ```
gpg --verify securityonion-3.3.0-20260908.iso.sig securityonion-3.3.0-20260908.iso gpg --verify securityonion-3.3.0-20260911.iso.sig securityonion-3.3.0-20260911.iso
``` ```
The output should show "Good signature" and the Primary key fingerprint should match what's shown below: The output should show "Good signature" and the Primary key fingerprint should match what's shown below:
``` ```
gpg: Signature made Tue 08 Sep 2026 10:07:12 AM EDT using RSA key ID FE507013 gpg: Signature made Fri 11 Sep 2026 11:23:56 AM EDT using RSA key ID FE507013
gpg: Good signature from "Security Onion Solutions, LLC <info@securityonionsolutions.com>" gpg: Good signature from "Security Onion Solutions, LLC <info@securityonionsolutions.com>"
gpg: WARNING: This key is not certified with a trusted signature! gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner. gpg: There is no indication that the signature belongs to the owner.
+2 -9
View File
@@ -1,12 +1,6 @@
docker: docker:
range: '172.17.1.0/24' range: '172.17.1.0/24'
gateway: '172.17.1.1' gateway: '172.17.1.1'
networks:
sobridge: {}
soauth:
range: '172.17.2.0/24'
gateway: '172.17.2.1'
manager_only: True
ulimits: ulimits:
- name: nofile - name: nofile
soft: 1048576 soft: 1048576
@@ -64,18 +58,18 @@ docker:
ulimits: [] ulimits: []
'so-kratos': 'so-kratos':
final_octet: 28 final_octet: 28
networks: ['soauth']
port_bindings: port_bindings:
- 0.0.0.0:4433:4433 - 0.0.0.0:4433:4433
- 0.0.0.0:4434:4434
custom_bind_mounts: [] custom_bind_mounts: []
extra_hosts: [] extra_hosts: []
extra_env: [] extra_env: []
ulimits: [] ulimits: []
'so-hydra': 'so-hydra':
final_octet: 30 final_octet: 30
networks: ['soauth']
port_bindings: port_bindings:
- 0.0.0.0:4444:4444 - 0.0.0.0:4444:4444
- 0.0.0.0:4445:4445
custom_bind_mounts: [] custom_bind_mounts: []
extra_hosts: [] extra_hosts: []
extra_env: [] extra_env: []
@@ -134,7 +128,6 @@ docker:
ulimits: [] ulimits: []
'so-soc': 'so-soc':
final_octet: 34 final_octet: 34
networks: ['sobridge', 'soauth']
port_bindings: port_bindings:
- 0.0.0.0:9822:9822 - 0.0.0.0:9822:9822
custom_bind_mounts: [] custom_bind_mounts: []
+3 -21
View File
@@ -1,26 +1,8 @@
{% import_yaml 'docker/defaults.yaml' as DOCKERDEFAULTS %} {% import_yaml 'docker/defaults.yaml' as DOCKERDEFAULTS %}
{% set DOCKERMERGED = salt['pillar.get']('docker', DOCKERDEFAULTS.docker, merge=True) %} {% set DOCKERMERGED = salt['pillar.get']('docker', DOCKERDEFAULTS.docker, merge=True) %}
{% set RANGESPLIT = DOCKERMERGED.range.split('.') %}
{% if DOCKERMERGED.networks.sobridge is not mapping %} {% set FIRSTTHREE = RANGESPLIT[0] ~ '.' ~ RANGESPLIT[1] ~ '.' ~ RANGESPLIT[2] ~ '.' %}
{% do DOCKERMERGED.networks.update({'sobridge': {}}) %}
{% endif %}
{% do DOCKERMERGED.networks['sobridge'].update({'range': DOCKERMERGED.range, 'gateway': DOCKERMERGED.gateway}) %}
{% for netname, net in DOCKERMERGED.networks.items() %}
{% set RANGESPLIT = net.range.split('.') %}
{% do net.update({'prefix': RANGESPLIT[0] ~ '.' ~ RANGESPLIT[1] ~ '.' ~ RANGESPLIT[2] ~ '.'}) %}
{% endfor %}
{% for container, vals in DOCKERMERGED.containers.items() %} {% for container, vals in DOCKERMERGED.containers.items() %}
{% set CONTAINER_NETS = vals.get('networks', ['sobridge']) %} {% do DOCKERMERGED.containers[container].update({'ip': FIRSTTHREE ~ DOCKERMERGED.containers[container].final_octet}) %}
{% set IPS = {} %}
{% for netname in CONTAINER_NETS %}
{% do IPS.update({netname: DOCKERMERGED.networks[netname].prefix ~ vals.final_octet}) %}
{% endfor %}
{% do DOCKERMERGED.containers[container].update({
'networks': CONTAINER_NETS,
'ips': IPS,
'network': CONTAINER_NETS[0],
'ip': IPS[CONTAINER_NETS[0]]
}) %}
{% endfor %} {% endfor %}
+6 -10
View File
@@ -71,19 +71,15 @@ dockerreserveports:
- source: salt://common/files/99-reserved-ports.conf - source: salt://common/files/99-reserved-ports.conf
- name: /etc/sysctl.d/99-reserved-ports.conf - name: /etc/sysctl.d/99-reserved-ports.conf
{% for NETNAME, NETWORK in DOCKERMERGED.networks.items() %} sos_docker_net:
{% if not NETWORK.get('manager_only') or GLOBALS.get('is_manager', False) %}
sos_docker_net_{{ NETNAME }}:
docker_network.present: docker_network.present:
- name: {{ NETNAME }} - name: sobridge
- subnet: {{ NETWORK.range }} - subnet: {{ DOCKERMERGED.range }}
- gateway: {{ NETWORK.gateway }} - gateway: {{ DOCKERMERGED.gateway }}
- options: - options:
com.docker.network.bridge.name: '{{ NETNAME }}' com.docker.network.bridge.name: 'sobridge'
com.docker.network.driver.mtu: '1500' com.docker.network.driver.mtu: '1500'
com.docker.network.bridge.enable_ip_masquerade: 'true' com.docker.network.bridge.enable_ip_masquerade: 'true'
com.docker.network.bridge.enable_icc: 'true' com.docker.network.bridge.enable_icc: 'true'
com.docker.network.bridge.host_binding_ipv4: '0.0.0.0' com.docker.network.bridge.host_binding_ipv4: '0.0.0.0'
- unless: ip l | grep {{ NETNAME }} - unless: ip l | grep sobridge
{% endif %}
{% endfor %}
-37
View File
@@ -7,33 +7,6 @@ docker:
description: Default docker IP range for containers. description: Default docker IP range for containers.
helpLink: docker helpLink: docker
advanced: True advanced: True
networks:
soauth:
range:
description: |
IP range for the soauth docker network. soauth is an isolated network for the
authentication services, so that the Kratos and Hydra admin APIs are only reachable
from the containers placed on it. Most containers instead use sobridge, which takes
its range and gateway from the docker.range and docker.gateway settings above.
helpLink: docker
readonly: True
advanced: True
global: True
gateway:
description: Gateway for the soauth docker network.
helpLink: docker
readonly: True
advanced: True
global: True
manager_only:
description: |
Limits the soauth network to grid members running the authentication containers,
instead of creating it on every node.
helpLink: docker
readonly: True
advanced: True
global: True
forcedType: bool
ulimits: ulimits:
description: | description: |
Default ulimit settings applied to all containers via the Docker daemon. Each entry specifies a resource name (e.g. nofile, memlock, core, nproc) with soft and hard limits. Individual container ulimits override these defaults. Valid resource names include: cpu, fsize, data, stack, core, rss, nproc, nofile, memlock, as, locks, sigpending, msgqueue, nice, rtprio, rttime. Default ulimit settings applied to all containers via the Docker daemon. Each entry specifies a resource name (e.g. nofile, memlock, core, nproc) with soft and hard limits. Individual container ulimits override these defaults. Valid resource names include: cpu, fsize, data, stack, core, rss, nproc, nofile, memlock, as, locks, sigpending, msgqueue, nice, rtprio, rttime.
@@ -61,16 +34,6 @@ docker:
readonly: True readonly: True
advanced: True advanced: True
global: True global: True
networks:
description: |
Docker networks this container is attached to. The first entry is the container's
primary network and determines the address its published ports are forwarded to.
Defaults to sobridge when unset.
helpLink: docker
readonly: True
advanced: True
global: True
forcedType: "[]string"
port_bindings: port_bindings:
description: List of port bindings for the container. description: List of port bindings for the container.
helpLink: docker helpLink: docker
+14 -33
View File
@@ -4,19 +4,11 @@
{%- set role = GLOBALS.role.split('-')[1] %} {%- set role = GLOBALS.role.split('-')[1] %}
{%- from 'firewall/containers.map.jinja' import NODE_CONTAINERS %} {%- from 'firewall/containers.map.jinja' import NODE_CONTAINERS %}
{%- set NODE_NETWORKS = [] %}
{%- for NETNAME, NETWORK in DOCKERMERGED.networks.items() %}
{%- if not NETWORK.get('manager_only') or GLOBALS.get('is_manager', False) %}
{%- do NODE_NETWORKS.append(NETNAME) %}
{%- endif %}
{%- endfor %}
{%- set PR = [] %} {%- set PR = [] %}
{%- set D1 = [] %} {%- set D1 = [] %}
{%- set D2 = [] %} {%- set D2 = [] %}
{%- for container in NODE_CONTAINERS %} {%- for container in NODE_CONTAINERS %}
{%- set IP = DOCKERMERGED.containers[container].ip %} {%- set IP = DOCKERMERGED.containers[container].ip %}
{%- set BRIDGE = DOCKERMERGED.containers[container].network %}
{%- if DOCKERMERGED.containers[container].port_bindings is defined %} {%- if DOCKERMERGED.containers[container].port_bindings is defined %}
{%- for binding in DOCKERMERGED.containers[container].port_bindings %} {%- for binding in DOCKERMERGED.containers[container].port_bindings %}
{#- cant split int so we convert to string #} {#- cant split int so we convert to string #}
@@ -43,11 +35,11 @@
{%- endif %} {%- endif %}
{%- do PR.append("-A POSTROUTING -s " ~ DOCKERMERGED.containers[container].ip ~ "/32 -d " ~ DOCKERMERGED.containers[container].ip ~ "/32 -p " ~ proto ~ " -m " ~ proto ~ " --dport " ~ containerPort ~ " -j MASQUERADE") %} {%- do PR.append("-A POSTROUTING -s " ~ DOCKERMERGED.containers[container].ip ~ "/32 -d " ~ DOCKERMERGED.containers[container].ip ~ "/32 -p " ~ proto ~ " -m " ~ proto ~ " --dport " ~ containerPort ~ " -j MASQUERADE") %}
{%- if bindip | length and bindip != '0.0.0.0' %} {%- if bindip | length and bindip != '0.0.0.0' %}
{%- do D1.append("-A DOCKER -d " ~ bindip ~ "/32 ! -i " ~ BRIDGE ~ " -p " ~ proto ~ " -m " ~ proto ~ " --dport " ~ hostPort ~ " -j DNAT --to-destination " ~ DOCKERMERGED.containers[container].ip ~ ":" ~ containerPort) %} {%- do D1.append("-A DOCKER -d " ~ bindip ~ "/32 ! -i sobridge -p " ~ proto ~ " -m " ~ proto ~ " --dport " ~ hostPort ~ " -j DNAT --to-destination " ~ DOCKERMERGED.containers[container].ip ~ ":" ~ containerPort) %}
{%- else %} {%- else %}
{%- do D1.append("-A DOCKER ! -i " ~ BRIDGE ~ " -p " ~ proto ~ " -m " ~ proto ~ " --dport " ~ hostPort ~ " -j DNAT --to-destination " ~ DOCKERMERGED.containers[container].ip ~ ":" ~ containerPort) %} {%- do D1.append("-A DOCKER ! -i sobridge -p " ~ proto ~ " -m " ~ proto ~ " --dport " ~ hostPort ~ " -j DNAT --to-destination " ~ DOCKERMERGED.containers[container].ip ~ ":" ~ containerPort) %}
{%- endif %} {%- endif %}
{%- do D2.append("-A DOCKER -d " ~ DOCKERMERGED.containers[container].ip ~ "/32 ! -i " ~ BRIDGE ~ " -o " ~ BRIDGE ~ " -p " ~ proto ~ " -m " ~ proto ~ " --dport " ~ containerPort ~ " -j ACCEPT") %} {%- do D2.append("-A DOCKER -d " ~ DOCKERMERGED.containers[container].ip ~ "/32 ! -i sobridge -o sobridge -p " ~ proto ~ " -m " ~ proto ~ " --dport " ~ containerPort ~ " -j ACCEPT") %}
{%- endfor %} {%- endfor %}
{%- endif %} {%- endif %}
{%- endfor %} {%- endfor %}
@@ -60,15 +52,11 @@
:DOCKER - [0:0] :DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER -A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
{%- for NETNAME in NODE_NETWORKS %} -A POSTROUTING -s {{DOCKERMERGED.range}} ! -o sobridge -j MASQUERADE
-A POSTROUTING -s {{ DOCKERMERGED.networks[NETNAME].range }} ! -o {{ NETNAME }} -j MASQUERADE
{%- endfor %}
{%- for rule in PR %} {%- for rule in PR %}
{{ rule }} {{ rule }}
{%- endfor %} {%- endfor %}
{%- for NETNAME in NODE_NETWORKS %} -A DOCKER -i sobridge -j RETURN
-A DOCKER -i {{ NETNAME }} -j RETURN
{%- endfor %}
{%- for rule in D1 %} {%- for rule in D1 %}
{{ rule }} {{ rule }}
{%- endfor %} {%- endfor %}
@@ -109,12 +97,10 @@ COMMIT
{%- endif %} {%- endif %}
-A FORWARD -j DOCKER-USER -A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1 -A FORWARD -j DOCKER-ISOLATION-STAGE-1
{%- for NETNAME in NODE_NETWORKS %} -A FORWARD -o sobridge -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o {{ NETNAME }} -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A FORWARD -o sobridge -j DOCKER
-A FORWARD -o {{ NETNAME }} -j DOCKER -A FORWARD -i sobridge ! -o sobridge -j ACCEPT
-A FORWARD -i {{ NETNAME }} ! -o {{ NETNAME }} -j ACCEPT -A FORWARD -i sobridge -o sobridge -j ACCEPT
-A FORWARD -i {{ NETNAME }} -o {{ NETNAME }} -j ACCEPT
{%- endfor %}
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i lo -j ACCEPT -A FORWARD -i lo -j ACCEPT
-A FORWARD -m conntrack --ctstate INVALID -j DROP -A FORWARD -m conntrack --ctstate INVALID -j DROP
@@ -126,18 +112,13 @@ COMMIT
{%- for rule in D2 %} {%- for rule in D2 %}
{{ rule }} {{ rule }}
{%- endfor %} {%- endfor %}
{% for NETNAME in NODE_NETWORKS %}
-A DOCKER-ISOLATION-STAGE-1 -i {{ NETNAME }} ! -o {{ NETNAME }} -j DOCKER-ISOLATION-STAGE-2 -A DOCKER-ISOLATION-STAGE-1 -i sobridge ! -o sobridge -j DOCKER-ISOLATION-STAGE-2
{%- endfor %}
-A DOCKER-ISOLATION-STAGE-1 -j RETURN -A DOCKER-ISOLATION-STAGE-1 -j RETURN
{%- for NETNAME in NODE_NETWORKS %} -A DOCKER-ISOLATION-STAGE-2 -o sobridge -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o {{ NETNAME }} -j DROP
{%- endfor %}
-A DOCKER-ISOLATION-STAGE-2 -j RETURN -A DOCKER-ISOLATION-STAGE-2 -j RETURN
{%- for NETNAME in NODE_NETWORKS %} -A DOCKER-USER ! -i sobridge -o sobridge -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-USER ! -i {{ NETNAME }} -o {{ NETNAME }} -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A DOCKER-USER ! -i sobridge -o sobridge -j LOGGING
-A DOCKER-USER ! -i {{ NETNAME }} -o {{ NETNAME }} -j LOGGING
{%- endfor %}
-A DOCKER-USER -j RETURN -A DOCKER-USER -j RETURN
-A LOGGING -m limit --limit 2/min -j LOG --log-prefix "IPTables-dropped: " -A LOGGING -m limit --limit 2/min -j LOG --log-prefix "IPTables-dropped: "
-A LOGGING -j DROP -A LOGGING -j DROP
+2 -6
View File
@@ -4,12 +4,8 @@
{# add our ip to self #} {# add our ip to self #}
{% do FIREWALL_DEFAULT.firewall.hostgroups.self.append(GLOBALS.node_ip) %} {% do FIREWALL_DEFAULT.firewall.hostgroups.self.append(GLOBALS.node_ip) %}
{# add dockernet ranges #} {# add dockernet range #}
{% for NETNAME, NETWORK in DOCKERMERGED.networks.items() %} {% do FIREWALL_DEFAULT.firewall.hostgroups.dockernet.append(DOCKERMERGED.range) %}
{% if not NETWORK.get('manager_only') or GLOBALS.get('is_manager', False) %}
{% do FIREWALL_DEFAULT.firewall.hostgroups.dockernet.append(NETWORK.range) %}
{% endif %}
{% endfor %}
{% if GLOBALS.role == 'so-idh' %} {% if GLOBALS.role == 'so-idh' %}
{% from 'idh/opencanary_config.map.jinja' import IDH_PORTGROUPS %} {% from 'idh/opencanary_config.map.jinja' import IDH_PORTGROUPS %}
+3 -3
View File
@@ -26,8 +26,8 @@ so-hydra:
- hostname: hydra - hostname: hydra
- name: so-hydra - name: so-hydra
- networks: - networks:
- soauth: - sobridge:
- ipv4_address: {{ DOCKERMERGED.containers['so-hydra'].ips['soauth'] }} - ipv4_address: {{ DOCKERMERGED.containers['so-hydra'].ip }}
- binds: - binds:
- /opt/so/conf/hydra/:/hydra-conf:ro - /opt/so/conf/hydra/:/hydra-conf:ro
- /opt/so/log/hydra/:/hydra-log:rw - /opt/so/log/hydra/:/hydra-log:rw
@@ -73,7 +73,7 @@ delete_so-hydra_so-status.disabled:
wait_for_hydra: wait_for_hydra:
http.wait_for_successful_query: http.wait_for_successful_query:
- name: 'http://{{ DOCKERMERGED.containers['so-hydra'].ips['soauth'] }}:4444/health/alive' - name: 'http://{{ GLOBALS.manager }}:4444/health/alive'
- ssl: True - ssl: True
- verify_ssl: False - verify_ssl: False
- status: - status:
+3 -3
View File
@@ -19,8 +19,8 @@ so-kratos:
- hostname: kratos - hostname: kratos
- name: so-kratos - name: so-kratos
- networks: - networks:
- soauth: - sobridge:
- ipv4_address: {{ DOCKERMERGED.containers['so-kratos'].ips['soauth'] }} - ipv4_address: {{ DOCKERMERGED.containers['so-kratos'].ip }}
- binds: - binds:
- /opt/so/conf/kratos/:/kratos-conf:ro - /opt/so/conf/kratos/:/kratos-conf:ro
- /opt/so/log/kratos/:/kratos-log:rw - /opt/so/log/kratos/:/kratos-log:rw
@@ -71,7 +71,7 @@ delete_so-kratos_so-status.disabled:
wait_for_kratos: wait_for_kratos:
http.wait_for_successful_query: http.wait_for_successful_query:
- name: 'http://{{ DOCKERMERGED.containers['so-kratos'].ips['soauth'] }}:4434/' - name: 'http://{{ GLOBALS.manager }}:4434/'
- ssl: True - ssl: True
- verify_ssl: False - verify_ssl: False
- status: - status:
+8 -13
View File
@@ -106,8 +106,7 @@ while [[ $# -gt 0 ]]; do
esac esac
done done
hydraContainer=${HYDRA_CONTAINER:-so-hydra} hydraUrl=${HYDRA_URL:-http://127.0.0.1:4445}
hydraUrl=${HYDRA_URL:-http://localhost:4445}
socRolesFile=${SOC_ROLES_FILE:-/opt/so/conf/soc/soc_clients_roles} socRolesFile=${SOC_ROLES_FILE:-/opt/so/conf/soc/soc_clients_roles}
soUID=${SOCORE_UID:-939} soUID=${SOCORE_UID:-939}
soGID=${SOCORE_GID:-939} soGID=${SOCORE_GID:-939}
@@ -125,10 +124,6 @@ function fail() {
exit 1 exit 1
} }
function hydraCurl() {
docker exec -i "$hydraContainer" curl "$@"
}
function require() { function require() {
cmd=$1 cmd=$1
which "$1" 2>&1 > /dev/null which "$1" 2>&1 > /dev/null
@@ -138,8 +133,8 @@ function require() {
# Verify this environment is capable of running this script # Verify this environment is capable of running this script
function verifyEnvironment() { function verifyEnvironment() {
require "jq" require "jq"
require "docker" require "curl"
response=$(hydraCurl -Ss -L ${hydraUrl}/health/alive) response=$(curl -Ss -L ${hydraUrl}/health/alive)
[[ "$response" != '{"status":"ok"}' ]] && fail "Unable to communicate with Hydra; specify URL via HYDRA_URL environment variable" [[ "$response" != '{"status":"ok"}' ]] && fail "Unable to communicate with Hydra; specify URL via HYDRA_URL environment variable"
} }
@@ -169,7 +164,7 @@ function ensureRoleFileExists() {
} }
function listClients() { function listClients() {
response=$(hydraCurl -Ss -L -f ${hydraUrl}/admin/clients) response=$(curl -Ss -L -f ${hydraUrl}/admin/clients)
[[ $? != 0 ]] && fail "Unable to communicate with Hydra" [[ $? != 0 ]] && fail "Unable to communicate with Hydra"
clientIds=$(echo "${response}" | jq -r ".[] | .client_id" | sort) clientIds=$(echo "${response}" | jq -r ".[] | .client_id" | sort)
@@ -256,7 +251,7 @@ function createClient() {
EOF EOF
) )
response=$(hydraCurl -Ss -L --fail-with-body -X POST ${hydraUrl}/admin/clients -d "$body") response=$(curl -Ss -L --fail-with-body -X POST ${hydraUrl}/admin/clients -d "$body")
if [[ $? != 0 ]]; then if [[ $? != 0 ]]; then
error=$(echo $response | jq .error) error=$(echo $response | jq .error)
fail "Failed to submit request to Hydra: $error" fail "Failed to submit request to Hydra: $error"
@@ -288,7 +283,7 @@ function update() {
EOF EOF
) )
response=$(hydraCurl -Ss -L --fail-with-body -X PATCH ${hydraUrl}/admin/clients/$id -d "$body") response=$(curl -Ss -L --fail-with-body -X PATCH ${hydraUrl}/admin/clients/$id -d "$body")
if [[ $? != 0 ]]; then if [[ $? != 0 ]]; then
error=$(echo $response | jq .error) error=$(echo $response | jq .error)
fail "Failed to submit request to Hydra: $error" fail "Failed to submit request to Hydra: $error"
@@ -310,7 +305,7 @@ function generateSecret() {
EOF EOF
) )
response=$(hydraCurl -Ss -L --fail-with-body -X PATCH ${hydraUrl}/admin/clients/$id -d "$body") response=$(curl -Ss -L --fail-with-body -X PATCH ${hydraUrl}/admin/clients/$id -d "$body")
if [[ $? != 0 ]]; then if [[ $? != 0 ]]; then
error=$(echo $response | jq .error) error=$(echo $response | jq .error)
fail "Failed to submit request to Hydra: $error" fail "Failed to submit request to Hydra: $error"
@@ -322,7 +317,7 @@ function deleteClient() {
[[ ${identityId} == "" ]] && fail "Client not found" [[ ${identityId} == "" ]] && fail "Client not found"
response=$(hydraCurl -Ss -XDELETE -L --fail-with-body "${hydraUrl}/admin/clients/$identityId") response=$(curl -Ss -XDELETE -L --fail-with-body "${hydraUrl}/admin/clients/$identityId")
if [[ $? != 0 ]]; then if [[ $? != 0 ]]; then
error=$(echo $response | jq .error) error=$(echo $response | jq .error)
fail "Failed to submit request to Hydra: $error" fail "Failed to submit request to Hydra: $error"
+11 -16
View File
@@ -129,8 +129,7 @@ while [[ $# -gt 0 ]]; do
esac esac
done done
kratosContainer=${KRATOS_CONTAINER:-so-kratos} kratosUrl=${KRATOS_URL:-http://127.0.0.1:4434/admin}
kratosUrl=${KRATOS_URL:-http://localhost:4434/admin}
databasePath=${KRATOS_DB_PATH:-/nsm/kratos/db/db.sqlite} databasePath=${KRATOS_DB_PATH:-/nsm/kratos/db/db.sqlite}
databaseTimeout=${KRATOS_DB_TIMEOUT:-5000} databaseTimeout=${KRATOS_DB_TIMEOUT:-5000}
bcryptRounds=${BCRYPT_ROUNDS:-12} bcryptRounds=${BCRYPT_ROUNDS:-12}
@@ -155,10 +154,6 @@ function fail() {
exit 1 exit 1
} }
function kratosCurl() {
docker exec -i "$kratosContainer" curl "$@"
}
function require() { function require() {
cmd=$1 cmd=$1
which "$1" 2>&1 > /dev/null which "$1" 2>&1 > /dev/null
@@ -169,18 +164,18 @@ function require() {
function verifyEnvironment() { function verifyEnvironment() {
require "htpasswd" require "htpasswd"
require "jq" require "jq"
require "docker" require "curl"
require "openssl" require "openssl"
require "sqlite3" require "sqlite3"
[[ ! -f $databasePath ]] && fail "Unable to find database file; specify path via KRATOS_DB_PATH environment variable" [[ ! -f $databasePath ]] && fail "Unable to find database file; specify path via KRATOS_DB_PATH environment variable"
response=$(kratosCurl -Ss -L ${kratosUrl}/) response=$(curl -Ss -L ${kratosUrl}/)
[[ "$response" != "404 page not found" ]] && fail "Unable to communicate with Kratos; specify URL via KRATOS_URL environment variable" [[ "$response" != "404 page not found" ]] && fail "Unable to communicate with Kratos; specify URL via KRATOS_URL environment variable"
} }
function findIdByEmail() { function findIdByEmail() {
email=${1,,} email=${1,,}
response=$(kratosCurl -Ss -L ${kratosUrl}/identities) response=$(curl -Ss -L ${kratosUrl}/identities)
identityId=$(echo "${response}" | jq -r ".[] | select(.verifiable_addresses[0].value == \"$email\") | .id") identityId=$(echo "${response}" | jq -r ".[] | select(.verifiable_addresses[0].value == \"$email\") | .id")
echo $identityId echo $identityId
} }
@@ -421,7 +416,7 @@ function syncAll() {
} }
function listUsers() { function listUsers() {
response=$(kratosCurl -Ss -L ${kratosUrl}/identities) response=$(curl -Ss -L ${kratosUrl}/identities)
[[ $? != 0 ]] && fail "Unable to communicate with Kratos" [[ $? != 0 ]] && fail "Unable to communicate with Kratos"
users=$(echo "${response}" | jq -r ".[] | .verifiable_addresses[0].value" | sort) users=$(echo "${response}" | jq -r ".[] | .verifiable_addresses[0].value" | sort)
@@ -500,7 +495,7 @@ function createUser() {
EOF EOF
) )
response=$(kratosCurl -Ss -L ${kratosUrl}/identities -d "$addUserJson") response=$(curl -Ss -L ${kratosUrl}/identities -d "$addUserJson")
[[ $? != 0 ]] && fail "Unable to communicate with Kratos" [[ $? != 0 ]] && fail "Unable to communicate with Kratos"
identityId=$(echo "${response}" | jq -r ".id") identityId=$(echo "${response}" | jq -r ".id")
@@ -523,7 +518,7 @@ function updateStatus() {
identityId=$(findIdByEmail "$email") identityId=$(findIdByEmail "$email")
[[ ${identityId} == "" ]] && fail "User not found" [[ ${identityId} == "" ]] && fail "User not found"
response=$(kratosCurl -Ss -L "${kratosUrl}/identities/$identityId") response=$(curl -Ss -L "${kratosUrl}/identities/$identityId")
[[ $? != 0 ]] && fail "Unable to communicate with Kratos" [[ $? != 0 ]] && fail "Unable to communicate with Kratos"
schemaId=$(echo "$response" | jq -r .schema_id) schemaId=$(echo "$response" | jq -r .schema_id)
@@ -536,7 +531,7 @@ function updateStatus() {
state="inactive" state="inactive"
fi fi
body="{ \"schema_id\": \"$schemaId\", \"state\": \"$state\", \"traits\": $traitBlock }" body="{ \"schema_id\": \"$schemaId\", \"state\": \"$state\", \"traits\": $traitBlock }"
response=$(kratosCurl -fSsL -XPUT -H "Content-Type: application/json" "${kratosUrl}/identities/$identityId" -d "$body") response=$(curl -fSsL -XPUT -H "Content-Type: application/json" "${kratosUrl}/identities/$identityId" -d "$body")
[[ $? != 0 ]] && fail "Unable to update user" [[ $? != 0 ]] && fail "Unable to update user"
} }
@@ -555,7 +550,7 @@ function updateUserProfile() {
identityId=$(findIdByEmail "$email") identityId=$(findIdByEmail "$email")
[[ ${identityId} == "" ]] && fail "User not found" [[ ${identityId} == "" ]] && fail "User not found"
response=$(kratosCurl -Ss -L "${kratosUrl}/identities/$identityId") response=$(curl -Ss -L "${kratosUrl}/identities/$identityId")
[[ $? != 0 ]] && fail "Unable to communicate with Kratos" [[ $? != 0 ]] && fail "Unable to communicate with Kratos"
schemaId=$(echo "$response" | jq -r .schema_id) schemaId=$(echo "$response" | jq -r .schema_id)
@@ -564,7 +559,7 @@ function updateUserProfile() {
traitBlock="{\"email\":\"$email\",\"firstName\":\"$firstName\",\"lastName\":\"$lastName\",\"note\":\"$note\"}" traitBlock="{\"email\":\"$email\",\"firstName\":\"$firstName\",\"lastName\":\"$lastName\",\"note\":\"$note\"}"
body="{ \"schema_id\": \"$schemaId\", \"state\": \"$state\", \"traits\": $traitBlock }" body="{ \"schema_id\": \"$schemaId\", \"state\": \"$state\", \"traits\": $traitBlock }"
response=$(kratosCurl -fSsL -XPUT -H "Content-Type: application/json" "${kratosUrl}/identities/$identityId" -d "$body") response=$(curl -fSsL -XPUT -H "Content-Type: application/json" "${kratosUrl}/identities/$identityId" -d "$body")
[[ $? != 0 ]] && fail "Unable to update user" [[ $? != 0 ]] && fail "Unable to update user"
} }
@@ -574,7 +569,7 @@ function deleteUser() {
identityId=$(findIdByEmail "$email") identityId=$(findIdByEmail "$email")
[[ ${identityId} == "" ]] && fail "User not found" [[ ${identityId} == "" ]] && fail "User not found"
response=$(kratosCurl -Ss -XDELETE -L "${kratosUrl}/identities/$identityId") response=$(curl -Ss -XDELETE -L "${kratosUrl}/identities/$identityId")
[[ $? != 0 ]] && fail "Unable to communicate with Kratos" [[ $? != 0 ]] && fail "Unable to communicate with Kratos"
rolesTmpFile="${socRolesFile}.tmp" rolesTmpFile="${socRolesFile}.tmp"
+95 -80
View File
@@ -28,7 +28,6 @@ INSTALLEDSALTVERSION=$(salt --versions-report | grep Salt: | awk '{print $2}')
# percentage like "25%"). Empty means so-soup-grid-highstate uses the salt:auto_apply:batch # percentage like "25%"). Empty means so-soup-grid-highstate uses the salt:auto_apply:batch
# pillar default. # pillar default.
BATCHSIZE= BATCHSIZE=
DEFAULT_DOCKER_RANGE='172.17.1.0/24'
SOUP_LOG=/root/soup.log SOUP_LOG=/root/soup.log
SOUP_DEBUG_LOG=/root/soup-debug.log SOUP_DEBUG_LOG=/root/soup-debug.log
WHATWOULDYOUSAYYAHDOHERE=soup WHATWOULDYOUSAYYAHDOHERE=soup
@@ -121,6 +120,9 @@ check_err() {
161) 161)
echo 'Required intermediate Elasticsearch upgrade not complete' echo 'Required intermediate Elasticsearch upgrade not complete'
;; ;;
162)
echo 'One or more Elastic Agent nodes do not support the x86-64-v3 CPU instruction set'
;;
170) 170)
echo "Intermediate upgrade completed successfully to $next_step_so_version, but next soup to Security Onion $originally_requested_so_version could not be started automatically." echo "Intermediate upgrade completed successfully to $next_step_so_version, but next soup to Security Onion $originally_requested_so_version could not be started automatically."
echo "Start soup again manually to continue the upgrade to Security Onion $originally_requested_so_version." echo "Start soup again manually to continue the upgrade to Security Onion $originally_requested_so_version."
@@ -348,6 +350,83 @@ check_cluster_health() {
exit 0 exit 0
} }
no_soup_for_you() {
echo ""
echo "No soup for you!"
exit 162
}
check_cpu_compatibility() {
# Roles running a container built from the so-elastic-agent image; mirrors the
# elasticagent and elasticfleet entries in salt/reactor/pillar_push_map.yaml.
local cpu_target='G@role:so-heavynode or G@role:so-eval or G@role:so-fleet or G@role:so-import or G@role:so-manager or G@role:so-managerhype or G@role:so-managersearch or G@role:so-standalone'
local expected_nodes cpu_results node result confirm
local -a unsupported=() offline=()
echo "Checking that Elastic Agent nodes support the x86-64-v3 CPU instruction set now required by Elastic."
if [[ "$SKIP_CPU_CHECK" == "true" ]]; then
printf "\nSkipping the x86-64-v3 CPU check because --skip-cpu-check was specified.\n\n"
return
fi
# Nodes that never answer are absent from the results, so diff against who should have.
expected_nodes=$(salt -C "$cpu_target" --preview-target --out=json 2>/dev/null | jq -r '.[]?') || true
if [[ -z "$expected_nodes" ]]; then
printf "\nCould not determine which nodes run the Elastic Agent, so the x86-64-v3 CPU check cannot run.\n"
no_soup_for_you
fi
cpu_results=$(salt -t 30 -C "$cpu_target" cmd.run "/lib64/ld-linux-x86-64.so.2 --help | grep x86-64-v3" --out=json 2>/dev/null) || true
while IFS= read -r node; do
[[ -z "$node" ]] && continue
result=$(jq -r --arg node "$node" '.[$node] // empty' <<< "$cpu_results" 2>/dev/null)
if [[ -z "$result" || "$result" == *"did not return"* ]]; then
offline+=("$node")
elif [[ "$result" != *"x86-64-v3 (supported"* ]]; then
# glibc appends "(supported, searched)" only when supported; the open paren keeps
# this from matching a future "(unsupported".
unsupported+=("$node")
fi
done <<< "$expected_nodes"
if [[ ${#unsupported[@]} -eq 0 && ${#offline[@]} -eq 0 ]]; then
printf "\nAll Elastic Agent nodes support x86-64-v3. We can proceed with SOUP.\n\n"
return
fi
echo ""
if [[ ${#unsupported[@]} -gt 0 ]]; then
echo "The following node(s) do NOT support the x86-64-v3 CPU instruction set:"
printf ' %s\n' "${unsupported[@]}"
echo ""
echo "Upstream Elastic now builds its binaries for x86-64-v3, so these nodes can no"
echo "longer run Elastic. Upgrading them WILL BREAK them."
echo ""
fi
if [[ ${#offline[@]} -gt 0 ]]; then
echo "The following node(s) did not respond and could not be checked:"
printf ' %s\n' "${offline[@]}"
echo ""
echo "These nodes are offline, so we cannot confirm they support x86-64-v3, which"
echo "upstream Elastic now requires."
echo ""
fi
if [[ -n $UNATTENDED ]]; then
echo "Unattended mode cannot prompt for an override. Re-run soup interactively, or pass --skip-cpu-check to bypass this check."
no_soup_for_you
fi
read -rp "Type 'override' to continue anyway, or press Enter to exit: " confirm
if [[ "${confirm,,}" == "override" ]]; then
printf "\nOverride accepted. Continuing at your own risk.\n\n"
else
no_soup_for_you
fi
}
check_fleet_server() { check_fleet_server() {
echo "Checking that Elastic Fleet Server is responding." echo "Checking that Elastic Fleet Server is responding."
# Modeled on the wait_for_so-elastic-fleet state check in elasticfleet/enabled.sls, # Modeled on the wait_for_so-elastic-fleet state check in elasticfleet/enabled.sls,
@@ -526,7 +605,6 @@ preupgrade_changes() {
[[ "$INSTALLEDVERSION" == "3.0.0" ]] && up_to_3.1.0 [[ "$INSTALLEDVERSION" == "3.0.0" ]] && up_to_3.1.0
[[ "$INSTALLEDVERSION" == "3.1.0" ]] && up_to_3.2.0 [[ "$INSTALLEDVERSION" == "3.1.0" ]] && up_to_3.2.0
[[ "$INSTALLEDVERSION" == "3.2.0" ]] && up_to_3.3.0 [[ "$INSTALLEDVERSION" == "3.2.0" ]] && up_to_3.3.0
[[ "$INSTALLEDVERSION" == "3.3.0" ]] && up_to_3.4.0
true true
} }
@@ -545,7 +623,6 @@ postupgrade_changes() {
[[ "$POSTVERSION" == "3.0.0" ]] && post_to_3.1.0 [[ "$POSTVERSION" == "3.0.0" ]] && post_to_3.1.0
[[ "$POSTVERSION" == "3.1.0" ]] && post_to_3.2.0 [[ "$POSTVERSION" == "3.1.0" ]] && post_to_3.2.0
[[ "$POSTVERSION" == "3.2.0" ]] && post_to_3.3.0 [[ "$POSTVERSION" == "3.2.0" ]] && post_to_3.3.0
[[ "$POSTVERSION" == "3.3.0" ]] && post_to_3.4.0
# All applicable post-upgrade steps completed; clear the resume marker. # All applicable post-upgrade steps completed; clear the resume marker.
rm -f "$POSTVERSION_FILE" rm -f "$POSTVERSION_FILE"
true true
@@ -1096,82 +1173,6 @@ post_to_3.3.0() {
} }
### 3.3.0 End ### ### 3.3.0 End ###
### 3.4.0 Scripts ###
up_to_3.4.0() {
set_soauth_range
echo "Removing so-kratos, so-hydra and so-soc so they are recreated on the soauth network."
docker rm -f so-kratos so-hydra so-soc >> $SOUP_LOG 2>&1
INSTALLEDVERSION=3.4.0
}
set_soauth_range() {
local pillar_file=/opt/so/saltstack/local/pillar/docker/soc_docker.sls
local current_range suggested authnet authgw input
[[ -f "$pillar_file" ]] || return 0
current_range=$(so-yaml.py get -r "$pillar_file" docker.range 2>/dev/null) || return 0
# A default range gets the 172.17.2.0/24 from docker/defaults.yaml, same as a fresh
# install, so there is nothing to ask about.
[[ -n "$current_range" && "$current_range" != "$DEFAULT_DOCKER_RANGE" ]] || return 0
if so-yaml.py get -r "$pillar_file" docker.networks.soauth.range >/dev/null 2>&1; then
return 0
fi
suggested=$(echo "${current_range%%/*}" | awk -F'.' '{ printf "%s.%s.%s.%s", $1, $2, ($3 + 1) % 256, $4 }')
if [[ -z $UNATTENDED ]]; then
echo ""
echo "This grid uses a custom Docker range ($current_range). The authentication"
echo "services are moving to their own isolated network, which needs a second /24"
echo "that does not overlap it."
echo ""
while :; do
read -rp "Enter the network without the /24 suffix, or press Enter for ${suggested}: " input
[[ -z "$input" ]] && input="$suggested"
if valid_soauth_range "$input" "$current_range"; then
authnet="$input"
break
fi
echo "That range must be a valid IPv4 network, must not be within 172.17.0.0/24, and must not overlap ${current_range}."
done
else
if ! valid_soauth_range "$suggested" "$current_range"; then
FINAL_MESSAGE_QUEUE+=("WARNING: Unable to pick a range for the authentication network alongside $current_range. Set it manually before the next highstate:")
FINAL_MESSAGE_QUEUE+=(" - so-yaml.py add $pillar_file docker.networks.soauth.range <network>/24")
FINAL_MESSAGE_QUEUE+=(" - so-yaml.py add $pillar_file docker.networks.soauth.gateway <gateway>")
return 0
fi
authnet="$suggested"
FINAL_MESSAGE_QUEUE+=("NOTE: The authentication services moved to an isolated Docker network and were assigned ${authnet}/24.")
FINAL_MESSAGE_QUEUE+=(" - If that conflicts with your environment, update docker.networks.soauth in $pillar_file and run so-checkin.")
fi
authgw=$(echo "$authnet" | awk -F'.' '{print $1,$2,$3,1}' OFS='.')
echo "Assigning the authentication network the range ${authnet}/24."
so-yaml.py add "$pillar_file" docker.networks.soauth.range "${authnet}/24" >> $SOUP_LOG 2>&1
so-yaml.py add "$pillar_file" docker.networks.soauth.gateway "$authgw" >> $SOUP_LOG 2>&1
}
valid_soauth_range() {
local candidate=$1 docker_range=$2
valid_ip4 "$candidate" || return 1
[[ $candidate =~ ^172\.17\.0\. ]] && return 1
[[ "${candidate}/24" == "$docker_range" ]] && return 1
return 0
}
post_to_3.4.0() {
set_postversion 3.4.0
}
### 3.4.0 End ###
repo_sync() { repo_sync() {
echo "Sync the local repo." echo "Sync the local repo."
@@ -2056,6 +2057,9 @@ main() {
echo "Let's see if we need to update Security Onion." echo "Let's see if we need to update Security Onion."
upgrade_check upgrade_check
check_cpu_compatibility
upgrade_space upgrade_space
echo "Verifying Elasticsearch version compatibility across the grid before upgrading." echo "Verifying Elasticsearch version compatibility across the grid before upgrading."
@@ -2334,6 +2338,17 @@ fi
echo "### soup has been served at $(date) ###" echo "### soup has been served at $(date) ###"
} }
SKIP_CPU_CHECK=false
declare -a SOUP_ARGS=()
for arg in "$@"; do
if [[ "$arg" == "--skip-cpu-check" ]]; then
SKIP_CPU_CHECK=true
else
SOUP_ARGS+=("$arg")
fi
done
set -- "${SOUP_ARGS[@]}"
while getopts ":b:f:y" opt; do while getopts ":b:f:y" opt; do
case ${opt} in case ${opt} in
b ) b )
@@ -2357,7 +2372,7 @@ while getopts ":b:f:y" opt; do
ISOLOC="$OPTARG" ISOLOC="$OPTARG"
;; ;;
\? ) \? )
echo "Usage: soup [-b] [-y] [-f <iso location>]" echo "Usage: soup [-b] [-y] [-f <iso location>] [--skip-cpu-check]"
exit 1 exit 1
;; ;;
: ) : )
+1 -1
View File
@@ -183,7 +183,7 @@ http {
ssl_prefer_server_ciphers on; ssl_prefer_server_ciphers on;
ssl_protocols TLSv1.2 TLSv1.3; ssl_protocols TLSv1.2 TLSv1.3;
location ~* (^/login/.*|^/js/.*|^/css/.*|^/images/.*|^/pages/.*|^/docs/.*) { location ~* (^/login|^/login/.*|^/js/.*|^/css/.*|^/images/.*|^/pages/.*|^/docs/.*) {
proxy_pass http://{{ GLOBALS.manager }}:9822; proxy_pass http://{{ GLOBALS.manager }}:9822;
proxy_read_timeout 90; proxy_read_timeout 90;
proxy_connect_timeout 90; proxy_connect_timeout 90;
-2
View File
@@ -14,8 +14,6 @@
{% do SOCDEFAULTS.soc.config.server.modules[module].update({'hostUrl': application_url}) %} {% do SOCDEFAULTS.soc.config.server.modules[module].update({'hostUrl': application_url}) %}
{% endfor %} {% endfor %}
{% do SOCDEFAULTS.soc.config.server.modules.kratos.update({'publicHostUrl': 'http://' ~ DOCKERMERGED.containers['so-kratos'].ips['soauth'] ~ ':4433/'}) %}
{# add all grid heavy nodes to soc.server.modules.elastic.remoteHostUrls #} {# add all grid heavy nodes to soc.server.modules.elastic.remoteHostUrls #}
{% for node_type, minions in salt['pillar.get']('elasticsearch:nodes', {}).items() %} {% for node_type, minions in salt['pillar.get']('elasticsearch:nodes', {}).items() %}
{% if node_type in ['heavynode'] %} {% if node_type in ['heavynode'] %}
+64 -2
View File
@@ -1380,7 +1380,6 @@ soc:
retryFailureMaxAttempts: 5 retryFailureMaxAttempts: 5
kratos: kratos:
hostUrl: hostUrl:
publicHostUrl:
hydra: hydra:
hostUrl: hostUrl:
elastalertengine: elastalertengine:
@@ -1538,7 +1537,7 @@ soc:
Orchestrator: sonnet@SOAI Orchestrator: sonnet@SOAI
Investigator: gemma@SOAI Investigator: gemma@SOAI
DetectionEngineer: gemma@SOAI DetectionEngineer: gemma@SOAI
useMemory: true useMemory: false
useMemoryScanner: false useMemoryScanner: false
dontScanBefore: "" dontScanBefore: ""
memoryScanIntervalSeconds: 300 memoryScanIntervalSeconds: 300
@@ -1557,6 +1556,69 @@ soc:
reconcilePersona: "" reconcilePersona: ""
toolUseTurnAttempts: 12 toolUseTurnAttempts: 12
toolUseTurnDelayMs: 175 toolUseTurnDelayMs: 175
tools:
filterEventFields:
- "@timestamp"
- "client.name"
- "destination.ip"
- "destination.port"
- "destination.geo.country_name"
- "dns.query.name"
- "dns.query_name"
- "event.action"
- "event.category"
- "event.module"
- "event.dataset"
- "event.outcome"
- "event.severity"
- "event.severity_label"
- "event.type"
- "event_data.agent.name"
- "event_data.host.os.name"
- "file.mime_type"
- "file.name"
- "hash.md5"
- "hash.sha1"
- "host.mac"
- "host.name"
- "host.os.name"
- "http.method"
- "http.useragent"
- "http.virtual_host"
- "log.id.uid"
- "network.community_id"
- "network.protocol"
- "network.transport"
- "notice.message"
- "observer.name"
- "process.name"
- "process.executable"
- "process.entity_id"
- "process.command_line"
- "process.Ext.ancestry"
- "process.parent.entity_id"
- "process.parent.command_line"
- "rule.category"
- "rule.name"
- "rule.uuid"
- "software.name"
- "software.type"
- "software.version.unparsed"
- "source.ip"
- "source.port"
- "source.geo.country_name"
- "ssh.cypher_algorithm"
- "ssh.client"
- "ssh.server"
- "ssl.cipher"
- "ssl.server_name"
- "ssl.version"
- "system.auth.sudo.command"
- "user.name"
- "user.domain"
- "user.effective.name"
- "weird.name"
- "tags"
onionconfig: onionconfig:
saltstackDir: /opt/so/saltstack saltstackDir: /opt/so/saltstack
bypassEnabled: false bypassEnabled: false
+1 -3
View File
@@ -23,9 +23,7 @@ so-soc:
- name: so-soc - name: so-soc
- networks: - networks:
- sobridge: - sobridge:
- ipv4_address: {{ DOCKERMERGED.containers['so-soc'].ips['sobridge'] }} - ipv4_address: {{ DOCKERMERGED.containers['so-soc'].ip }}
- soauth:
- ipv4_address: {{ DOCKERMERGED.containers['so-soc'].ips['soauth'] }}
- binds: - binds:
- /nsm/rules:/nsm/rules:rw - /nsm/rules:/nsm/rules:rw
- /opt/so/conf/strelka:/opt/sensoroni/yara:rw - /opt/so/conf/strelka:/opt/sensoroni/yara:rw
+5
View File
@@ -916,6 +916,11 @@ soc:
description: The number of times to retry extracting memories from a session if errors occur. description: The number of times to retry extracting memories from a session if errors occur.
global: True global: True
advanced: True advanced: True
tools:
filterEventFields:
description: A whitelist of fields to return when OnionAI uses the query_events tool. All other fields are removed. One field per line.
global: True
multiline: True
client: client:
assistant: assistant:
enabled: enabled:
+2 -2
View File
@@ -54,8 +54,8 @@
{% {%
do GLOBALS.update({ do GLOBALS.update({
'application_urls': { 'application_urls': {
'hydra': 'http://' ~ DOCKERMERGED.containers['so-hydra'].ips['soauth'] ~ ':4445/', 'hydra': 'http://' ~ GLOBALS.manager ~ ':4445/',
'kratos': 'http://' ~ DOCKERMERGED.containers['so-kratos'].ips['soauth'] ~ ':4434/', 'kratos': 'http://' ~ GLOBALS.manager ~ ':4434/',
'elastic': 'https://' ~ GLOBALS.manager ~ ':9200/', 'elastic': 'https://' ~ GLOBALS.manager ~ ':9200/',
'influxdb': 'https://' ~ GLOBALS.manager ~ ':8086/' 'influxdb': 'https://' ~ GLOBALS.manager ~ ':8086/'
} }
-20
View File
@@ -276,20 +276,9 @@ collect_dockernet() {
whiptail_invalid_input whiptail_invalid_input
whiptail_dockernet_sosnet "$DOCKERNET" whiptail_dockernet_sosnet "$DOCKERNET"
done done
whiptail_authnet_sosnet "$(adjacent_net "$DOCKERNET")"
while ! valid_ip4 "$AUTHNET" || [[ $AUTHNET =~ "172.17.0." ]] || [[ "$AUTHNET" == "$DOCKERNET" ]]; do
whiptail_invalid_input
whiptail_authnet_sosnet "$AUTHNET"
done
fi fi
} }
adjacent_net() {
echo "$1" | awk -F'.' '{ printf "%s.%s.%s.%s", $1, $2, ($3 + 1) % 256, $4 }'
}
collect_gateway() { collect_gateway() {
whiptail_management_interface_gateway whiptail_management_interface_gateway
@@ -1410,15 +1399,6 @@ docker_pillar() {
"docker:"\ "docker:"\
" range: '$DOCKERNET/24'"\ " range: '$DOCKERNET/24'"\
" gateway: '$DOCKERGATEWAY'" > $docker_pillar_file " gateway: '$DOCKERGATEWAY'" > $docker_pillar_file
if [ ! -z "$AUTHNET" ]; then
AUTHGATEWAY=$(echo $AUTHNET | awk -F'.' '{print $1,$2,$3,1}' OFS='.')
printf '%s\n'\
" networks:"\
" soauth:"\
" range: '$AUTHNET/24'"\
" gateway: '$AUTHGATEWAY'" >> $docker_pillar_file
fi
fi fi
} }
-13
View File
@@ -365,18 +365,6 @@ whiptail_dockernet_sosnet() {
} }
whiptail_authnet_sosnet() {
[ -n "$TESTING" ] && return
AUTHNET=$(whiptail --title "$whiptail_title" --inputbox \
"\nEnter a second /24 size network range WITHOUT the /24 suffix. The authentication services are isolated on their own network so that the identity provider is not reachable from other containers. It must not overlap the range you just entered, and any range within 172.17.0.0/24 cannot be used." 13 65 "$1" 3>&1 1>&2 2>&3)
local exitstatus=$?
whiptail_check_exitstatus $exitstatus
}
whiptail_end_settings() { whiptail_end_settings() {
[ -n "$TESTING" ] && return [ -n "$TESTING" ] && return
@@ -439,7 +427,6 @@ whiptail_end_settings() {
[[ -n $WEBUSER ]] && __append_end_msg "Web User: $WEBUSER" [[ -n $WEBUSER ]] && __append_end_msg "Web User: $WEBUSER"
[[ -n $DOCKERNET ]] && __append_end_msg "Docker network: $DOCKERNET/24" [[ -n $DOCKERNET ]] && __append_end_msg "Docker network: $DOCKERNET/24"
[[ -n $AUTHNET ]] && __append_end_msg "Authentication network: $AUTHNET/24"
if [[ ${#ntp_servers[@]} -gt 0 ]]; then if [[ ${#ntp_servers[@]} -gt 0 ]]; then
__append_end_msg "NTP Servers:" __append_end_msg "NTP Servers:"
for server in "${ntp_servers[@]}"; do for server in "${ntp_servers[@]}"; do
Binary file not shown.