Compare commits

..
Author SHA1 Message Date
Mike Reeves 866a046853 Annotate the soauth network leaves instead of the networks map
SOC reported "Malformed config setting (docker.networks.soauth.range): Setting
name 'networks' conflicts with another similarly named setting" and refused to
render the config tree.

soc_docker.yaml annotated docker.networks itself, and every key in that block
was a scalar, so FlattenAnnotations registered docker.networks as a setting.
defaults.yaml holds a nested map there, so FlattenPillar separately registered
docker.networks.soauth.range, .gateway and .manager_only, and
HydrateAnnotations unions the two sets. The config tree gives each id segment
either a value or children, never both, so addToNode pushed docker.networks as
a childless leaf and then threw when docker.networks.soauth.range tried to
descend through it.

The annotations move down to the three leaves that actually carry values, so
docker.networks is a plain branch. This matches docker.containers, which is
nested the same way and has never carried annotations of its own.

docker.ulimits and the per-container networks list are unaffected: their pillar
values are lists rather than maps, so they stay single settings.
2026-09-18 13:40:05 -04:00
Mike Reeves 65e261475d Mention Hydra in the soauth network description
The description was written alongside the Kratos change and named only the
Kratos admin API, but so-hydra now sits on soauth too.

The trailing note about rebuilding the firewall is dropped, since the setting
is readonly and cannot be changed from SOC in the first place.
2026-09-18 10:17:20 -04:00
Mike Reeves bbc28c88b7 Isolate the Hydra admin API on the soauth docker network
The Hydra admin API creates OAuth clients and introspects tokens without
authentication, relying on network isolation the same way Kratos does, but
so-hydra published 0.0.0.0:4445:4445 and sat on sobridge. With userland-proxy
left at its default of true, Docker ran a proxy listener on the host, so any
container reached the admin API through the manager IP or the bridge gateway
regardless of its docker network.

so-hydra now sits alone on soauth and no longer publishes 4445. 4444 is still
published, so the nginx proxy for /oauth2/token and the well-known endpoints is
unchanged. so-soc is already dual homed from the Kratos change and reaches the
admin API over soauth, so only its hostUrl moves. wait_for_hydra polls the
container address instead of the host port.

so-client reaches the admin API through docker exec, mirroring so-user. Exit
codes still propagate, so the --fail-with-body error handling is unchanged.

so-hydra is removed in up_to_3.4.0 alongside so-kratos and so-soc so the
highstate recreates it with the correct network membership. The auth network
range is already handled by set_soauth_range.
2026-09-17 16:58:49 -04:00
Mike Reeves cf3a4ebc27 Isolate the Kratos admin API on a dedicated soauth docker network
The Kratos admin API is unauthenticated by design and relies on network
isolation, but so-kratos published 0.0.0.0:4434:4434 and daemon.json leaves
userland-proxy at its default of true. Docker therefore ran a proxy listener
on the host, and any container reached the admin API through the manager IP
or the bridge gateway regardless of its docker network.

so-kratos now sits alone on a new soauth network and no longer publishes
4434. 4433 is still published, so nginx is unchanged. so-soc is dual homed
and reaches the admin API over soauth, with publicHostUrl set explicitly.
so-user reaches it through docker exec, and wait_for_kratos polls the
container address instead of the host port.

firewall/iptables.jinja hardcoded sobridge in every generated DNAT, ACCEPT,
masquerade and isolation rule, so it is now driven by a docker:networks map
and a per-container networks list. Containers that do not declare one
default to sobridge, and .ip still resolves to the primary network's
address. soauth is only created on the roles that run so-kratos.

Setup already allows a custom docker range, so it now also prompts for the
auth network range and writes it to the docker pillar. Grids on the default
range need no decision during soup, since they get the same 172.17.2.0/24 a
fresh install would. Grids that set a custom range are prompted, defaulting
to the adjacent /24, and unattended upgrades take that default with a notice
rather than blocking.

so-kratos and so-soc are removed in up_to_3.4.0 so the highstate recreates
them with the correct network membership.
2026-09-09 16:09:00 -04:00
21 changed files with 285 additions and 231 deletions

No files matched your search

+11 -11
View File
@@ -1,17 +1,17 @@
### 3.3.0-20260911 ISO image released on 2026/09/11
### 3.3.0-20260908 ISO image released on 2026/09/08
### Download and Verify
3.3.0-20260911 ISO image:
https://download.securityonion.net/file/securityonion/securityonion-3.3.0-20260911.iso
3.3.0-20260908 ISO image:
https://download.securityonion.net/file/securityonion/securityonion-3.3.0-20260908.iso
MD5: 12B18433D3A2198A185892FF79CF638F
SHA1: 2B3C2E1FA7A78ED1F956E7EDCC12E32593C14EEE
SHA256: 0938C73B76CE30EC9E4394D312C79EA7CAC721B6818541697279A6221F7D870D
MD5: 5A2C42D0083F2D7B4DC2178C30EBC05F
SHA1: 505220A8A3315AFEE601C13772996018425CCD29
SHA256: 6EB8401296A1D051FEC558C520D2D4AB1912A72D351A2426FBF8C87FEE2BA844
Signature for ISO image:
https://github.com/Security-Onion-Solutions/securityonion/raw/3/main/sigs/securityonion-3.3.0-20260911.iso.sig
https://github.com/Security-Onion-Solutions/securityonion/raw/3/main/sigs/securityonion-3.3.0-20260908.iso.sig
Signing key:
https://raw.githubusercontent.com/Security-Onion-Solutions/securityonion/3/main/KEYS
@@ -25,22 +25,22 @@ wget https://raw.githubusercontent.com/Security-Onion-Solutions/securityonion/3/
Download the signature file for the ISO:
```
wget https://github.com/Security-Onion-Solutions/securityonion/raw/3/main/sigs/securityonion-3.3.0-20260911.iso.sig
wget https://github.com/Security-Onion-Solutions/securityonion/raw/3/main/sigs/securityonion-3.3.0-20260908.iso.sig
```
Download the ISO image:
```
wget https://download.securityonion.net/file/securityonion/securityonion-3.3.0-20260911.iso
wget https://download.securityonion.net/file/securityonion/securityonion-3.3.0-20260908.iso
```
Verify the downloaded ISO image using the signature file:
```
gpg --verify securityonion-3.3.0-20260911.iso.sig securityonion-3.3.0-20260911.iso
gpg --verify securityonion-3.3.0-20260908.iso.sig securityonion-3.3.0-20260908.iso
```
The output should show "Good signature" and the Primary key fingerprint should match what's shown below:
```
gpg: Signature made Fri 11 Sep 2026 11:23:56 AM EDT using RSA key ID FE507013
gpg: Signature made Tue 08 Sep 2026 10:07:12 AM EDT using RSA key ID FE507013
gpg: Good signature from "Security Onion Solutions, LLC <info@securityonionsolutions.com>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
+9 -2
View File
@@ -1,6 +1,12 @@
docker:
range: '172.17.1.0/24'
gateway: '172.17.1.1'
networks:
sobridge: {}
soauth:
range: '172.17.2.0/24'
gateway: '172.17.2.1'
manager_only: True
ulimits:
- name: nofile
soft: 1048576
@@ -58,18 +64,18 @@ docker:
ulimits: []
'so-kratos':
final_octet: 28
networks: ['soauth']
port_bindings:
- 0.0.0.0:4433:4433
- 0.0.0.0:4434:4434
custom_bind_mounts: []
extra_hosts: []
extra_env: []
ulimits: []
'so-hydra':
final_octet: 30
networks: ['soauth']
port_bindings:
- 0.0.0.0:4444:4444
- 0.0.0.0:4445:4445
custom_bind_mounts: []
extra_hosts: []
extra_env: []
@@ -128,6 +134,7 @@ docker:
ulimits: []
'so-soc':
final_octet: 34
networks: ['sobridge', 'soauth']
port_bindings:
- 0.0.0.0:9822:9822
custom_bind_mounts: []
+21 -3
View File
@@ -1,8 +1,26 @@
{% import_yaml 'docker/defaults.yaml' as DOCKERDEFAULTS %}
{% set DOCKERMERGED = salt['pillar.get']('docker', DOCKERDEFAULTS.docker, merge=True) %}
{% set RANGESPLIT = DOCKERMERGED.range.split('.') %}
{% set FIRSTTHREE = RANGESPLIT[0] ~ '.' ~ RANGESPLIT[1] ~ '.' ~ RANGESPLIT[2] ~ '.' %}
{% if DOCKERMERGED.networks.sobridge is not mapping %}
{% do DOCKERMERGED.networks.update({'sobridge': {}}) %}
{% endif %}
{% do DOCKERMERGED.networks['sobridge'].update({'range': DOCKERMERGED.range, 'gateway': DOCKERMERGED.gateway}) %}
{% for netname, net in DOCKERMERGED.networks.items() %}
{% set RANGESPLIT = net.range.split('.') %}
{% do net.update({'prefix': RANGESPLIT[0] ~ '.' ~ RANGESPLIT[1] ~ '.' ~ RANGESPLIT[2] ~ '.'}) %}
{% endfor %}
{% for container, vals in DOCKERMERGED.containers.items() %}
{% do DOCKERMERGED.containers[container].update({'ip': FIRSTTHREE ~ DOCKERMERGED.containers[container].final_octet}) %}
{% set CONTAINER_NETS = vals.get('networks', ['sobridge']) %}
{% set IPS = {} %}
{% for netname in CONTAINER_NETS %}
{% do IPS.update({netname: DOCKERMERGED.networks[netname].prefix ~ vals.final_octet}) %}
{% endfor %}
{% do DOCKERMERGED.containers[container].update({
'networks': CONTAINER_NETS,
'ips': IPS,
'network': CONTAINER_NETS[0],
'ip': IPS[CONTAINER_NETS[0]]
}) %}
{% endfor %}
+10 -6
View File
@@ -71,15 +71,19 @@ dockerreserveports:
- source: salt://common/files/99-reserved-ports.conf
- name: /etc/sysctl.d/99-reserved-ports.conf
sos_docker_net:
{% for NETNAME, NETWORK in DOCKERMERGED.networks.items() %}
{% if not NETWORK.get('manager_only') or GLOBALS.get('is_manager', False) %}
sos_docker_net_{{ NETNAME }}:
docker_network.present:
- name: sobridge
- subnet: {{ DOCKERMERGED.range }}
- gateway: {{ DOCKERMERGED.gateway }}
- name: {{ NETNAME }}
- subnet: {{ NETWORK.range }}
- gateway: {{ NETWORK.gateway }}
- options:
com.docker.network.bridge.name: 'sobridge'
com.docker.network.bridge.name: '{{ NETNAME }}'
com.docker.network.driver.mtu: '1500'
com.docker.network.bridge.enable_ip_masquerade: 'true'
com.docker.network.bridge.enable_icc: 'true'
com.docker.network.bridge.host_binding_ipv4: '0.0.0.0'
- unless: ip l | grep sobridge
- unless: ip l | grep {{ NETNAME }}
{% endif %}
{% endfor %}
+37
View File
@@ -7,6 +7,33 @@ docker:
description: Default docker IP range for containers.
helpLink: docker
advanced: True
networks:
soauth:
range:
description: |
IP range for the soauth docker network. soauth is an isolated network for the
authentication services, so that the Kratos and Hydra admin APIs are only reachable
from the containers placed on it. Most containers instead use sobridge, which takes
its range and gateway from the docker.range and docker.gateway settings above.
helpLink: docker
readonly: True
advanced: True
global: True
gateway:
description: Gateway for the soauth docker network.
helpLink: docker
readonly: True
advanced: True
global: True
manager_only:
description: |
Limits the soauth network to grid members running the authentication containers,
instead of creating it on every node.
helpLink: docker
readonly: True
advanced: True
global: True
forcedType: bool
ulimits:
description: |
Default ulimit settings applied to all containers via the Docker daemon. Each entry specifies a resource name (e.g. nofile, memlock, core, nproc) with soft and hard limits. Individual container ulimits override these defaults. Valid resource names include: cpu, fsize, data, stack, core, rss, nproc, nofile, memlock, as, locks, sigpending, msgqueue, nice, rtprio, rttime.
@@ -34,6 +61,16 @@ docker:
readonly: True
advanced: True
global: True
networks:
description: |
Docker networks this container is attached to. The first entry is the container's
primary network and determines the address its published ports are forwarded to.
Defaults to sobridge when unset.
helpLink: docker
readonly: True
advanced: True
global: True
forcedType: "[]string"
port_bindings:
description: List of port bindings for the container.
helpLink: docker
+33 -14
View File
@@ -4,11 +4,19 @@
{%- set role = GLOBALS.role.split('-')[1] %}
{%- from 'firewall/containers.map.jinja' import NODE_CONTAINERS %}
{%- set NODE_NETWORKS = [] %}
{%- for NETNAME, NETWORK in DOCKERMERGED.networks.items() %}
{%- if not NETWORK.get('manager_only') or GLOBALS.get('is_manager', False) %}
{%- do NODE_NETWORKS.append(NETNAME) %}
{%- endif %}
{%- endfor %}
{%- set PR = [] %}
{%- set D1 = [] %}
{%- set D2 = [] %}
{%- for container in NODE_CONTAINERS %}
{%- set IP = DOCKERMERGED.containers[container].ip %}
{%- set BRIDGE = DOCKERMERGED.containers[container].network %}
{%- if DOCKERMERGED.containers[container].port_bindings is defined %}
{%- for binding in DOCKERMERGED.containers[container].port_bindings %}
{#- cant split int so we convert to string #}
@@ -35,11 +43,11 @@
{%- endif %}
{%- do PR.append("-A POSTROUTING -s " ~ DOCKERMERGED.containers[container].ip ~ "/32 -d " ~ DOCKERMERGED.containers[container].ip ~ "/32 -p " ~ proto ~ " -m " ~ proto ~ " --dport " ~ containerPort ~ " -j MASQUERADE") %}
{%- if bindip | length and bindip != '0.0.0.0' %}
{%- do D1.append("-A DOCKER -d " ~ bindip ~ "/32 ! -i sobridge -p " ~ proto ~ " -m " ~ proto ~ " --dport " ~ hostPort ~ " -j DNAT --to-destination " ~ DOCKERMERGED.containers[container].ip ~ ":" ~ containerPort) %}
{%- do D1.append("-A DOCKER -d " ~ bindip ~ "/32 ! -i " ~ BRIDGE ~ " -p " ~ proto ~ " -m " ~ proto ~ " --dport " ~ hostPort ~ " -j DNAT --to-destination " ~ DOCKERMERGED.containers[container].ip ~ ":" ~ containerPort) %}
{%- else %}
{%- do D1.append("-A DOCKER ! -i sobridge -p " ~ proto ~ " -m " ~ proto ~ " --dport " ~ hostPort ~ " -j DNAT --to-destination " ~ DOCKERMERGED.containers[container].ip ~ ":" ~ containerPort) %}
{%- do D1.append("-A DOCKER ! -i " ~ BRIDGE ~ " -p " ~ proto ~ " -m " ~ proto ~ " --dport " ~ hostPort ~ " -j DNAT --to-destination " ~ DOCKERMERGED.containers[container].ip ~ ":" ~ containerPort) %}
{%- endif %}
{%- do D2.append("-A DOCKER -d " ~ DOCKERMERGED.containers[container].ip ~ "/32 ! -i sobridge -o sobridge -p " ~ proto ~ " -m " ~ proto ~ " --dport " ~ containerPort ~ " -j ACCEPT") %}
{%- do D2.append("-A DOCKER -d " ~ DOCKERMERGED.containers[container].ip ~ "/32 ! -i " ~ BRIDGE ~ " -o " ~ BRIDGE ~ " -p " ~ proto ~ " -m " ~ proto ~ " --dport " ~ containerPort ~ " -j ACCEPT") %}
{%- endfor %}
{%- endif %}
{%- endfor %}
@@ -52,11 +60,15 @@
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s {{DOCKERMERGED.range}} ! -o sobridge -j MASQUERADE
{%- for NETNAME in NODE_NETWORKS %}
-A POSTROUTING -s {{ DOCKERMERGED.networks[NETNAME].range }} ! -o {{ NETNAME }} -j MASQUERADE
{%- endfor %}
{%- for rule in PR %}
{{ rule }}
{%- endfor %}
-A DOCKER -i sobridge -j RETURN
{%- for NETNAME in NODE_NETWORKS %}
-A DOCKER -i {{ NETNAME }} -j RETURN
{%- endfor %}
{%- for rule in D1 %}
{{ rule }}
{%- endfor %}
@@ -97,10 +109,12 @@ COMMIT
{%- endif %}
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o sobridge -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o sobridge -j DOCKER
-A FORWARD -i sobridge ! -o sobridge -j ACCEPT
-A FORWARD -i sobridge -o sobridge -j ACCEPT
{%- for NETNAME in NODE_NETWORKS %}
-A FORWARD -o {{ NETNAME }} -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o {{ NETNAME }} -j DOCKER
-A FORWARD -i {{ NETNAME }} ! -o {{ NETNAME }} -j ACCEPT
-A FORWARD -i {{ NETNAME }} -o {{ NETNAME }} -j ACCEPT
{%- endfor %}
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i lo -j ACCEPT
-A FORWARD -m conntrack --ctstate INVALID -j DROP
@@ -112,13 +126,18 @@ COMMIT
{%- for rule in D2 %}
{{ rule }}
{%- endfor %}
-A DOCKER-ISOLATION-STAGE-1 -i sobridge ! -o sobridge -j DOCKER-ISOLATION-STAGE-2
{% for NETNAME in NODE_NETWORKS %}
-A DOCKER-ISOLATION-STAGE-1 -i {{ NETNAME }} ! -o {{ NETNAME }} -j DOCKER-ISOLATION-STAGE-2
{%- endfor %}
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o sobridge -j DROP
{%- for NETNAME in NODE_NETWORKS %}
-A DOCKER-ISOLATION-STAGE-2 -o {{ NETNAME }} -j DROP
{%- endfor %}
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER ! -i sobridge -o sobridge -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-USER ! -i sobridge -o sobridge -j LOGGING
{%- for NETNAME in NODE_NETWORKS %}
-A DOCKER-USER ! -i {{ NETNAME }} -o {{ NETNAME }} -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-USER ! -i {{ NETNAME }} -o {{ NETNAME }} -j LOGGING
{%- endfor %}
-A DOCKER-USER -j RETURN
-A LOGGING -m limit --limit 2/min -j LOG --log-prefix "IPTables-dropped: "
-A LOGGING -j DROP
+6 -2
View File
@@ -4,8 +4,12 @@
{# add our ip to self #}
{% do FIREWALL_DEFAULT.firewall.hostgroups.self.append(GLOBALS.node_ip) %}
{# add dockernet range #}
{% do FIREWALL_DEFAULT.firewall.hostgroups.dockernet.append(DOCKERMERGED.range) %}
{# add dockernet ranges #}
{% for NETNAME, NETWORK in DOCKERMERGED.networks.items() %}
{% if not NETWORK.get('manager_only') or GLOBALS.get('is_manager', False) %}
{% do FIREWALL_DEFAULT.firewall.hostgroups.dockernet.append(NETWORK.range) %}
{% endif %}
{% endfor %}
{% if GLOBALS.role == 'so-idh' %}
{% from 'idh/opencanary_config.map.jinja' import IDH_PORTGROUPS %}
+3 -3
View File
@@ -26,8 +26,8 @@ so-hydra:
- hostname: hydra
- name: so-hydra
- networks:
- sobridge:
- ipv4_address: {{ DOCKERMERGED.containers['so-hydra'].ip }}
- soauth:
- ipv4_address: {{ DOCKERMERGED.containers['so-hydra'].ips['soauth'] }}
- binds:
- /opt/so/conf/hydra/:/hydra-conf:ro
- /opt/so/log/hydra/:/hydra-log:rw
@@ -73,7 +73,7 @@ delete_so-hydra_so-status.disabled:
wait_for_hydra:
http.wait_for_successful_query:
- name: 'http://{{ GLOBALS.manager }}:4444/health/alive'
- name: 'http://{{ DOCKERMERGED.containers['so-hydra'].ips['soauth'] }}:4444/health/alive'
- ssl: True
- verify_ssl: False
- status:
+3 -3
View File
@@ -19,8 +19,8 @@ so-kratos:
- hostname: kratos
- name: so-kratos
- networks:
- sobridge:
- ipv4_address: {{ DOCKERMERGED.containers['so-kratos'].ip }}
- soauth:
- ipv4_address: {{ DOCKERMERGED.containers['so-kratos'].ips['soauth'] }}
- binds:
- /opt/so/conf/kratos/:/kratos-conf:ro
- /opt/so/log/kratos/:/kratos-log:rw
@@ -71,7 +71,7 @@ delete_so-kratos_so-status.disabled:
wait_for_kratos:
http.wait_for_successful_query:
- name: 'http://{{ GLOBALS.manager }}:4434/'
- name: 'http://{{ DOCKERMERGED.containers['so-kratos'].ips['soauth'] }}:4434/'
- ssl: True
- verify_ssl: False
- status:
+13 -8
View File
@@ -106,7 +106,8 @@ while [[ $# -gt 0 ]]; do
esac
done
hydraUrl=${HYDRA_URL:-http://127.0.0.1:4445}
hydraContainer=${HYDRA_CONTAINER:-so-hydra}
hydraUrl=${HYDRA_URL:-http://localhost:4445}
socRolesFile=${SOC_ROLES_FILE:-/opt/so/conf/soc/soc_clients_roles}
soUID=${SOCORE_UID:-939}
soGID=${SOCORE_GID:-939}
@@ -124,6 +125,10 @@ function fail() {
exit 1
}
function hydraCurl() {
docker exec -i "$hydraContainer" curl "$@"
}
function require() {
cmd=$1
which "$1" 2>&1 > /dev/null
@@ -133,8 +138,8 @@ function require() {
# Verify this environment is capable of running this script
function verifyEnvironment() {
require "jq"
require "curl"
response=$(curl -Ss -L ${hydraUrl}/health/alive)
require "docker"
response=$(hydraCurl -Ss -L ${hydraUrl}/health/alive)
[[ "$response" != '{"status":"ok"}' ]] && fail "Unable to communicate with Hydra; specify URL via HYDRA_URL environment variable"
}
@@ -164,7 +169,7 @@ function ensureRoleFileExists() {
}
function listClients() {
response=$(curl -Ss -L -f ${hydraUrl}/admin/clients)
response=$(hydraCurl -Ss -L -f ${hydraUrl}/admin/clients)
[[ $? != 0 ]] && fail "Unable to communicate with Hydra"
clientIds=$(echo "${response}" | jq -r ".[] | .client_id" | sort)
@@ -251,7 +256,7 @@ function createClient() {
EOF
)
response=$(curl -Ss -L --fail-with-body -X POST ${hydraUrl}/admin/clients -d "$body")
response=$(hydraCurl -Ss -L --fail-with-body -X POST ${hydraUrl}/admin/clients -d "$body")
if [[ $? != 0 ]]; then
error=$(echo $response | jq .error)
fail "Failed to submit request to Hydra: $error"
@@ -283,7 +288,7 @@ function update() {
EOF
)
response=$(curl -Ss -L --fail-with-body -X PATCH ${hydraUrl}/admin/clients/$id -d "$body")
response=$(hydraCurl -Ss -L --fail-with-body -X PATCH ${hydraUrl}/admin/clients/$id -d "$body")
if [[ $? != 0 ]]; then
error=$(echo $response | jq .error)
fail "Failed to submit request to Hydra: $error"
@@ -305,7 +310,7 @@ function generateSecret() {
EOF
)
response=$(curl -Ss -L --fail-with-body -X PATCH ${hydraUrl}/admin/clients/$id -d "$body")
response=$(hydraCurl -Ss -L --fail-with-body -X PATCH ${hydraUrl}/admin/clients/$id -d "$body")
if [[ $? != 0 ]]; then
error=$(echo $response | jq .error)
fail "Failed to submit request to Hydra: $error"
@@ -317,7 +322,7 @@ function deleteClient() {
[[ ${identityId} == "" ]] && fail "Client not found"
response=$(curl -Ss -XDELETE -L --fail-with-body "${hydraUrl}/admin/clients/$identityId")
response=$(hydraCurl -Ss -XDELETE -L --fail-with-body "${hydraUrl}/admin/clients/$identityId")
if [[ $? != 0 ]]; then
error=$(echo $response | jq .error)
fail "Failed to submit request to Hydra: $error"
+16 -11
View File
@@ -129,7 +129,8 @@ while [[ $# -gt 0 ]]; do
esac
done
kratosUrl=${KRATOS_URL:-http://127.0.0.1:4434/admin}
kratosContainer=${KRATOS_CONTAINER:-so-kratos}
kratosUrl=${KRATOS_URL:-http://localhost:4434/admin}
databasePath=${KRATOS_DB_PATH:-/nsm/kratos/db/db.sqlite}
databaseTimeout=${KRATOS_DB_TIMEOUT:-5000}
bcryptRounds=${BCRYPT_ROUNDS:-12}
@@ -154,6 +155,10 @@ function fail() {
exit 1
}
function kratosCurl() {
docker exec -i "$kratosContainer" curl "$@"
}
function require() {
cmd=$1
which "$1" 2>&1 > /dev/null
@@ -164,18 +169,18 @@ function require() {
function verifyEnvironment() {
require "htpasswd"
require "jq"
require "curl"
require "docker"
require "openssl"
require "sqlite3"
[[ ! -f $databasePath ]] && fail "Unable to find database file; specify path via KRATOS_DB_PATH environment variable"
response=$(curl -Ss -L ${kratosUrl}/)
response=$(kratosCurl -Ss -L ${kratosUrl}/)
[[ "$response" != "404 page not found" ]] && fail "Unable to communicate with Kratos; specify URL via KRATOS_URL environment variable"
}
function findIdByEmail() {
email=${1,,}
response=$(curl -Ss -L ${kratosUrl}/identities)
response=$(kratosCurl -Ss -L ${kratosUrl}/identities)
identityId=$(echo "${response}" | jq -r ".[] | select(.verifiable_addresses[0].value == \"$email\") | .id")
echo $identityId
}
@@ -416,7 +421,7 @@ function syncAll() {
}
function listUsers() {
response=$(curl -Ss -L ${kratosUrl}/identities)
response=$(kratosCurl -Ss -L ${kratosUrl}/identities)
[[ $? != 0 ]] && fail "Unable to communicate with Kratos"
users=$(echo "${response}" | jq -r ".[] | .verifiable_addresses[0].value" | sort)
@@ -495,7 +500,7 @@ function createUser() {
EOF
)
response=$(curl -Ss -L ${kratosUrl}/identities -d "$addUserJson")
response=$(kratosCurl -Ss -L ${kratosUrl}/identities -d "$addUserJson")
[[ $? != 0 ]] && fail "Unable to communicate with Kratos"
identityId=$(echo "${response}" | jq -r ".id")
@@ -518,7 +523,7 @@ function updateStatus() {
identityId=$(findIdByEmail "$email")
[[ ${identityId} == "" ]] && fail "User not found"
response=$(curl -Ss -L "${kratosUrl}/identities/$identityId")
response=$(kratosCurl -Ss -L "${kratosUrl}/identities/$identityId")
[[ $? != 0 ]] && fail "Unable to communicate with Kratos"
schemaId=$(echo "$response" | jq -r .schema_id)
@@ -531,7 +536,7 @@ function updateStatus() {
state="inactive"
fi
body="{ \"schema_id\": \"$schemaId\", \"state\": \"$state\", \"traits\": $traitBlock }"
response=$(curl -fSsL -XPUT -H "Content-Type: application/json" "${kratosUrl}/identities/$identityId" -d "$body")
response=$(kratosCurl -fSsL -XPUT -H "Content-Type: application/json" "${kratosUrl}/identities/$identityId" -d "$body")
[[ $? != 0 ]] && fail "Unable to update user"
}
@@ -550,7 +555,7 @@ function updateUserProfile() {
identityId=$(findIdByEmail "$email")
[[ ${identityId} == "" ]] && fail "User not found"
response=$(curl -Ss -L "${kratosUrl}/identities/$identityId")
response=$(kratosCurl -Ss -L "${kratosUrl}/identities/$identityId")
[[ $? != 0 ]] && fail "Unable to communicate with Kratos"
schemaId=$(echo "$response" | jq -r .schema_id)
@@ -559,7 +564,7 @@ function updateUserProfile() {
traitBlock="{\"email\":\"$email\",\"firstName\":\"$firstName\",\"lastName\":\"$lastName\",\"note\":\"$note\"}"
body="{ \"schema_id\": \"$schemaId\", \"state\": \"$state\", \"traits\": $traitBlock }"
response=$(curl -fSsL -XPUT -H "Content-Type: application/json" "${kratosUrl}/identities/$identityId" -d "$body")
response=$(kratosCurl -fSsL -XPUT -H "Content-Type: application/json" "${kratosUrl}/identities/$identityId" -d "$body")
[[ $? != 0 ]] && fail "Unable to update user"
}
@@ -569,7 +574,7 @@ function deleteUser() {
identityId=$(findIdByEmail "$email")
[[ ${identityId} == "" ]] && fail "User not found"
response=$(curl -Ss -XDELETE -L "${kratosUrl}/identities/$identityId")
response=$(kratosCurl -Ss -XDELETE -L "${kratosUrl}/identities/$identityId")
[[ $? != 0 ]] && fail "Unable to communicate with Kratos"
rolesTmpFile="${socRolesFile}.tmp"
+80 -95
View File
@@ -28,6 +28,7 @@ INSTALLEDSALTVERSION=$(salt --versions-report | grep Salt: | awk '{print $2}')
# percentage like "25%"). Empty means so-soup-grid-highstate uses the salt:auto_apply:batch
# pillar default.
BATCHSIZE=
DEFAULT_DOCKER_RANGE='172.17.1.0/24'
SOUP_LOG=/root/soup.log
SOUP_DEBUG_LOG=/root/soup-debug.log
WHATWOULDYOUSAYYAHDOHERE=soup
@@ -120,9 +121,6 @@ check_err() {
161)
echo 'Required intermediate Elasticsearch upgrade not complete'
;;
162)
echo 'One or more Elastic Agent nodes do not support the x86-64-v3 CPU instruction set'
;;
170)
echo "Intermediate upgrade completed successfully to $next_step_so_version, but next soup to Security Onion $originally_requested_so_version could not be started automatically."
echo "Start soup again manually to continue the upgrade to Security Onion $originally_requested_so_version."
@@ -350,83 +348,6 @@ check_cluster_health() {
exit 0
}
no_soup_for_you() {
echo ""
echo "No soup for you!"
exit 162
}
check_cpu_compatibility() {
# Roles running a container built from the so-elastic-agent image; mirrors the
# elasticagent and elasticfleet entries in salt/reactor/pillar_push_map.yaml.
local cpu_target='G@role:so-heavynode or G@role:so-eval or G@role:so-fleet or G@role:so-import or G@role:so-manager or G@role:so-managerhype or G@role:so-managersearch or G@role:so-standalone'
local expected_nodes cpu_results node result confirm
local -a unsupported=() offline=()
echo "Checking that Elastic Agent nodes support the x86-64-v3 CPU instruction set now required by Elastic."
if [[ "$SKIP_CPU_CHECK" == "true" ]]; then
printf "\nSkipping the x86-64-v3 CPU check because --skip-cpu-check was specified.\n\n"
return
fi
# Nodes that never answer are absent from the results, so diff against who should have.
expected_nodes=$(salt -C "$cpu_target" --preview-target --out=json 2>/dev/null | jq -r '.[]?') || true
if [[ -z "$expected_nodes" ]]; then
printf "\nCould not determine which nodes run the Elastic Agent, so the x86-64-v3 CPU check cannot run.\n"
no_soup_for_you
fi
cpu_results=$(salt -t 30 -C "$cpu_target" cmd.run "/lib64/ld-linux-x86-64.so.2 --help | grep x86-64-v3" --out=json 2>/dev/null) || true
while IFS= read -r node; do
[[ -z "$node" ]] && continue
result=$(jq -r --arg node "$node" '.[$node] // empty' <<< "$cpu_results" 2>/dev/null)
if [[ -z "$result" || "$result" == *"did not return"* ]]; then
offline+=("$node")
elif [[ "$result" != *"x86-64-v3 (supported"* ]]; then
# glibc appends "(supported, searched)" only when supported; the open paren keeps
# this from matching a future "(unsupported".
unsupported+=("$node")
fi
done <<< "$expected_nodes"
if [[ ${#unsupported[@]} -eq 0 && ${#offline[@]} -eq 0 ]]; then
printf "\nAll Elastic Agent nodes support x86-64-v3. We can proceed with SOUP.\n\n"
return
fi
echo ""
if [[ ${#unsupported[@]} -gt 0 ]]; then
echo "The following node(s) do NOT support the x86-64-v3 CPU instruction set:"
printf ' %s\n' "${unsupported[@]}"
echo ""
echo "Upstream Elastic now builds its binaries for x86-64-v3, so these nodes can no"
echo "longer run Elastic. Upgrading them WILL BREAK them."
echo ""
fi
if [[ ${#offline[@]} -gt 0 ]]; then
echo "The following node(s) did not respond and could not be checked:"
printf ' %s\n' "${offline[@]}"
echo ""
echo "These nodes are offline, so we cannot confirm they support x86-64-v3, which"
echo "upstream Elastic now requires."
echo ""
fi
if [[ -n $UNATTENDED ]]; then
echo "Unattended mode cannot prompt for an override. Re-run soup interactively, or pass --skip-cpu-check to bypass this check."
no_soup_for_you
fi
read -rp "Type 'override' to continue anyway, or press Enter to exit: " confirm
if [[ "${confirm,,}" == "override" ]]; then
printf "\nOverride accepted. Continuing at your own risk.\n\n"
else
no_soup_for_you
fi
}
check_fleet_server() {
echo "Checking that Elastic Fleet Server is responding."
# Modeled on the wait_for_so-elastic-fleet state check in elasticfleet/enabled.sls,
@@ -605,6 +526,7 @@ preupgrade_changes() {
[[ "$INSTALLEDVERSION" == "3.0.0" ]] && up_to_3.1.0
[[ "$INSTALLEDVERSION" == "3.1.0" ]] && up_to_3.2.0
[[ "$INSTALLEDVERSION" == "3.2.0" ]] && up_to_3.3.0
[[ "$INSTALLEDVERSION" == "3.3.0" ]] && up_to_3.4.0
true
}
@@ -623,6 +545,7 @@ postupgrade_changes() {
[[ "$POSTVERSION" == "3.0.0" ]] && post_to_3.1.0
[[ "$POSTVERSION" == "3.1.0" ]] && post_to_3.2.0
[[ "$POSTVERSION" == "3.2.0" ]] && post_to_3.3.0
[[ "$POSTVERSION" == "3.3.0" ]] && post_to_3.4.0
# All applicable post-upgrade steps completed; clear the resume marker.
rm -f "$POSTVERSION_FILE"
true
@@ -1173,6 +1096,82 @@ post_to_3.3.0() {
}
### 3.3.0 End ###
### 3.4.0 Scripts ###
up_to_3.4.0() {
set_soauth_range
echo "Removing so-kratos, so-hydra and so-soc so they are recreated on the soauth network."
docker rm -f so-kratos so-hydra so-soc >> $SOUP_LOG 2>&1
INSTALLEDVERSION=3.4.0
}
set_soauth_range() {
local pillar_file=/opt/so/saltstack/local/pillar/docker/soc_docker.sls
local current_range suggested authnet authgw input
[[ -f "$pillar_file" ]] || return 0
current_range=$(so-yaml.py get -r "$pillar_file" docker.range 2>/dev/null) || return 0
# A default range gets the 172.17.2.0/24 from docker/defaults.yaml, same as a fresh
# install, so there is nothing to ask about.
[[ -n "$current_range" && "$current_range" != "$DEFAULT_DOCKER_RANGE" ]] || return 0
if so-yaml.py get -r "$pillar_file" docker.networks.soauth.range >/dev/null 2>&1; then
return 0
fi
suggested=$(echo "${current_range%%/*}" | awk -F'.' '{ printf "%s.%s.%s.%s", $1, $2, ($3 + 1) % 256, $4 }')
if [[ -z $UNATTENDED ]]; then
echo ""
echo "This grid uses a custom Docker range ($current_range). The authentication"
echo "services are moving to their own isolated network, which needs a second /24"
echo "that does not overlap it."
echo ""
while :; do
read -rp "Enter the network without the /24 suffix, or press Enter for ${suggested}: " input
[[ -z "$input" ]] && input="$suggested"
if valid_soauth_range "$input" "$current_range"; then
authnet="$input"
break
fi
echo "That range must be a valid IPv4 network, must not be within 172.17.0.0/24, and must not overlap ${current_range}."
done
else
if ! valid_soauth_range "$suggested" "$current_range"; then
FINAL_MESSAGE_QUEUE+=("WARNING: Unable to pick a range for the authentication network alongside $current_range. Set it manually before the next highstate:")
FINAL_MESSAGE_QUEUE+=(" - so-yaml.py add $pillar_file docker.networks.soauth.range <network>/24")
FINAL_MESSAGE_QUEUE+=(" - so-yaml.py add $pillar_file docker.networks.soauth.gateway <gateway>")
return 0
fi
authnet="$suggested"
FINAL_MESSAGE_QUEUE+=("NOTE: The authentication services moved to an isolated Docker network and were assigned ${authnet}/24.")
FINAL_MESSAGE_QUEUE+=(" - If that conflicts with your environment, update docker.networks.soauth in $pillar_file and run so-checkin.")
fi
authgw=$(echo "$authnet" | awk -F'.' '{print $1,$2,$3,1}' OFS='.')
echo "Assigning the authentication network the range ${authnet}/24."
so-yaml.py add "$pillar_file" docker.networks.soauth.range "${authnet}/24" >> $SOUP_LOG 2>&1
so-yaml.py add "$pillar_file" docker.networks.soauth.gateway "$authgw" >> $SOUP_LOG 2>&1
}
valid_soauth_range() {
local candidate=$1 docker_range=$2
valid_ip4 "$candidate" || return 1
[[ $candidate =~ ^172\.17\.0\. ]] && return 1
[[ "${candidate}/24" == "$docker_range" ]] && return 1
return 0
}
post_to_3.4.0() {
set_postversion 3.4.0
}
### 3.4.0 End ###
repo_sync() {
echo "Sync the local repo."
@@ -2057,9 +2056,6 @@ main() {
echo "Let's see if we need to update Security Onion."
upgrade_check
check_cpu_compatibility
upgrade_space
echo "Verifying Elasticsearch version compatibility across the grid before upgrading."
@@ -2338,17 +2334,6 @@ fi
echo "### soup has been served at $(date) ###"
}
SKIP_CPU_CHECK=false
declare -a SOUP_ARGS=()
for arg in "$@"; do
if [[ "$arg" == "--skip-cpu-check" ]]; then
SKIP_CPU_CHECK=true
else
SOUP_ARGS+=("$arg")
fi
done
set -- "${SOUP_ARGS[@]}"
while getopts ":b:f:y" opt; do
case ${opt} in
b )
@@ -2372,7 +2357,7 @@ while getopts ":b:f:y" opt; do
ISOLOC="$OPTARG"
;;
\? )
echo "Usage: soup [-b] [-y] [-f <iso location>] [--skip-cpu-check]"
echo "Usage: soup [-b] [-y] [-f <iso location>]"
exit 1
;;
: )
+1 -1
View File
@@ -183,7 +183,7 @@ http {
ssl_prefer_server_ciphers on;
ssl_protocols TLSv1.2 TLSv1.3;
location ~* (^/login|^/login/.*|^/js/.*|^/css/.*|^/images/.*|^/pages/.*|^/docs/.*) {
location ~* (^/login/.*|^/js/.*|^/css/.*|^/images/.*|^/pages/.*|^/docs/.*) {
proxy_pass http://{{ GLOBALS.manager }}:9822;
proxy_read_timeout 90;
proxy_connect_timeout 90;
+2
View File
@@ -14,6 +14,8 @@
{% do SOCDEFAULTS.soc.config.server.modules[module].update({'hostUrl': application_url}) %}
{% endfor %}
{% do SOCDEFAULTS.soc.config.server.modules.kratos.update({'publicHostUrl': 'http://' ~ DOCKERMERGED.containers['so-kratos'].ips['soauth'] ~ ':4433/'}) %}
{# add all grid heavy nodes to soc.server.modules.elastic.remoteHostUrls #}
{% for node_type, minions in salt['pillar.get']('elasticsearch:nodes', {}).items() %}
{% if node_type in ['heavynode'] %}
+2 -64
View File
@@ -1380,6 +1380,7 @@ soc:
retryFailureMaxAttempts: 5
kratos:
hostUrl:
publicHostUrl:
hydra:
hostUrl:
elastalertengine:
@@ -1537,7 +1538,7 @@ soc:
Orchestrator: sonnet@SOAI
Investigator: gemma@SOAI
DetectionEngineer: gemma@SOAI
useMemory: false
useMemory: true
useMemoryScanner: false
dontScanBefore: ""
memoryScanIntervalSeconds: 300
@@ -1556,69 +1557,6 @@ soc:
reconcilePersona: ""
toolUseTurnAttempts: 12
toolUseTurnDelayMs: 175
tools:
filterEventFields:
- "@timestamp"
- "client.name"
- "destination.ip"
- "destination.port"
- "destination.geo.country_name"
- "dns.query.name"
- "dns.query_name"
- "event.action"
- "event.category"
- "event.module"
- "event.dataset"
- "event.outcome"
- "event.severity"
- "event.severity_label"
- "event.type"
- "event_data.agent.name"
- "event_data.host.os.name"
- "file.mime_type"
- "file.name"
- "hash.md5"
- "hash.sha1"
- "host.mac"
- "host.name"
- "host.os.name"
- "http.method"
- "http.useragent"
- "http.virtual_host"
- "log.id.uid"
- "network.community_id"
- "network.protocol"
- "network.transport"
- "notice.message"
- "observer.name"
- "process.name"
- "process.executable"
- "process.entity_id"
- "process.command_line"
- "process.Ext.ancestry"
- "process.parent.entity_id"
- "process.parent.command_line"
- "rule.category"
- "rule.name"
- "rule.uuid"
- "software.name"
- "software.type"
- "software.version.unparsed"
- "source.ip"
- "source.port"
- "source.geo.country_name"
- "ssh.cypher_algorithm"
- "ssh.client"
- "ssh.server"
- "ssl.cipher"
- "ssl.server_name"
- "ssl.version"
- "system.auth.sudo.command"
- "user.name"
- "user.domain"
- "user.effective.name"
- "weird.name"
- "tags"
onionconfig:
saltstackDir: /opt/so/saltstack
bypassEnabled: false
+3 -1
View File
@@ -23,7 +23,9 @@ so-soc:
- name: so-soc
- networks:
- sobridge:
- ipv4_address: {{ DOCKERMERGED.containers['so-soc'].ip }}
- ipv4_address: {{ DOCKERMERGED.containers['so-soc'].ips['sobridge'] }}
- soauth:
- ipv4_address: {{ DOCKERMERGED.containers['so-soc'].ips['soauth'] }}
- binds:
- /nsm/rules:/nsm/rules:rw
- /opt/so/conf/strelka:/opt/sensoroni/yara:rw
-5
View File
@@ -916,11 +916,6 @@ soc:
description: The number of times to retry extracting memories from a session if errors occur.
global: True
advanced: True
tools:
filterEventFields:
description: A whitelist of fields to return when OnionAI uses the query_events tool. All other fields are removed. One field per line.
global: True
multiline: True
client:
assistant:
enabled:
+2 -2
View File
@@ -54,8 +54,8 @@
{%
do GLOBALS.update({
'application_urls': {
'hydra': 'http://' ~ GLOBALS.manager ~ ':4445/',
'kratos': 'http://' ~ GLOBALS.manager ~ ':4434/',
'hydra': 'http://' ~ DOCKERMERGED.containers['so-hydra'].ips['soauth'] ~ ':4445/',
'kratos': 'http://' ~ DOCKERMERGED.containers['so-kratos'].ips['soauth'] ~ ':4434/',
'elastic': 'https://' ~ GLOBALS.manager ~ ':9200/',
'influxdb': 'https://' ~ GLOBALS.manager ~ ':8086/'
}
+20
View File
@@ -276,9 +276,20 @@ collect_dockernet() {
whiptail_invalid_input
whiptail_dockernet_sosnet "$DOCKERNET"
done
whiptail_authnet_sosnet "$(adjacent_net "$DOCKERNET")"
while ! valid_ip4 "$AUTHNET" || [[ $AUTHNET =~ "172.17.0." ]] || [[ "$AUTHNET" == "$DOCKERNET" ]]; do
whiptail_invalid_input
whiptail_authnet_sosnet "$AUTHNET"
done
fi
}
adjacent_net() {
echo "$1" | awk -F'.' '{ printf "%s.%s.%s.%s", $1, $2, ($3 + 1) % 256, $4 }'
}
collect_gateway() {
whiptail_management_interface_gateway
@@ -1399,6 +1410,15 @@ docker_pillar() {
"docker:"\
" range: '$DOCKERNET/24'"\
" gateway: '$DOCKERGATEWAY'" > $docker_pillar_file
if [ ! -z "$AUTHNET" ]; then
AUTHGATEWAY=$(echo $AUTHNET | awk -F'.' '{print $1,$2,$3,1}' OFS='.')
printf '%s\n'\
" networks:"\
" soauth:"\
" range: '$AUTHNET/24'"\
" gateway: '$AUTHGATEWAY'" >> $docker_pillar_file
fi
fi
}
+13
View File
@@ -365,6 +365,18 @@ whiptail_dockernet_sosnet() {
}
whiptail_authnet_sosnet() {
[ -n "$TESTING" ] && return
AUTHNET=$(whiptail --title "$whiptail_title" --inputbox \
"\nEnter a second /24 size network range WITHOUT the /24 suffix. The authentication services are isolated on their own network so that the identity provider is not reachable from other containers. It must not overlap the range you just entered, and any range within 172.17.0.0/24 cannot be used." 13 65 "$1" 3>&1 1>&2 2>&3)
local exitstatus=$?
whiptail_check_exitstatus $exitstatus
}
whiptail_end_settings() {
[ -n "$TESTING" ] && return
@@ -427,6 +439,7 @@ whiptail_end_settings() {
[[ -n $WEBUSER ]] && __append_end_msg "Web User: $WEBUSER"
[[ -n $DOCKERNET ]] && __append_end_msg "Docker network: $DOCKERNET/24"
[[ -n $AUTHNET ]] && __append_end_msg "Authentication network: $AUTHNET/24"
if [[ ${#ntp_servers[@]} -gt 0 ]]; then
__append_end_msg "NTP Servers:"
for server in "${ntp_servers[@]}"; do
Binary file not shown.