CSEC_PUBLIC/securityonion
1
0
Fork 0
mirror of https://github.com/Security-Onion-Solutions/securityonion.git synced 2025-12-06 09:12:45 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #862 from Security-Onion-Solutions/dev

Browse Source 
1.4.0
This commit is contained in:
Mike Reeves
2020-06-16 14:47:08 -04:00
committed by GitHub
parent 9eb4d6c1d5 6410087994
commit ada0d7b8cd
172 changed files with 10805 additions and 4270 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
README.md
View File

@@ -1,3 +1,37 @@
## Hybrid Hunter Beta 1.4.0 - Beta 3
- Complete overhaul of the way we handle custom and default settings and data. You will now see a default and local directory under the saltstack directory. All customizations are stored in local.
- The way firewall rules are handled has been completely revamped. This will allow the user to customize firewall rules much easier.
- Users can now change their own password in SOC.
- Hunt now allows users to enable auto-hunt. This is a toggle which, when enabled, automatically submits a new hunt when filtering, grouping, etc.
- Title bar now reflects current Hunt query. This will assist users in locating a previous query from their browser history.
- Zeek 3.0.7
- Elastic 7.7.1
- Suricata can now be used for meta data generation.
- Suricata eve.json has been moved to `/nsm` to align with storage of other data.
- Suricata will now properly rotate its logs.
- Grafana dashboards now work properly in standalone mode.
- Kibana Dashboard updates including osquery, community_id.
- New Elasticsearch Ingest processor to generate community_id from any log that includes the required fields.
- Community_id generated for additional logs: Zeek HTTP/SMTP/ , Sysmon shipped with Osquery or Winlogbeat.
- Major streamlining of Fleet setup & configuration - no need to run a secondary setup script anymore.
- Fleet Standalone node now includes the ability to set a FQDN to point osquery endpoints to.
- Distributed installs now support ingesting Windows Eventlogs via Winlogbeat - includes full parsing support for Sysmon.
- SOC Downloads section now includes a link to the supported version of Winlogbeat.
- Basic syslog ingestion capability now included.
- Elasticsearch index name transition fixes for various components.
- Updated URLs for pivot fields in Kibana.
- Instances of `hive` renamed to `thehive`.
### Known Issues:
- The Hunt feature is currently considered "Preview" and although very useful in its current state, not everything works. We wanted to get this out as soon as possible to get the feedback from you! Let us know what you want to see! Let us know what you think we should call it!
- You cannot pivot to PCAP from Suricata alerts in Kibana or Hunt.
- Navigator is currently not working when using hostname to access SOC. IP mode works correctly.
- Due to the move to ECS, the current Playbook plays may not alert correctly at this time.
- The osquery MacOS package does not install correctly.
## Hybrid Hunter Beta 1.3.0 - Beta 2
### Changes:

2
VERSION
View File

@@ -1 +1 @@
1.3.0
1.4.0

20
files/firewall/assigned_hostgroups.local.map.yaml Normal file
View File

@@ -0,0 +1,20 @@
{% import_yaml 'firewall/portgroups.yaml' as default_portgroups %}
{% set default_portgroups = default_portgroups.firewall.aliases.ports %}
{% import_yaml 'firewall/portgroups.local.yaml' as local_portgroups %}
{% if local_portgroups.firewall.aliases.ports %}
{% set local_portgroups = local_portgroups.firewall.aliases.ports %}
{% else %}
{% set local_portgroups = {} %}
{% endif %}
{% set portgroups = salt['defaults.merge'](default_portgroups, local_portgroups, in_place=False) %}
role:
eval:
fleet:
heavynode:
helixsensor:
master:
mastersearch:
standalone:
searchnode:
sensor:

62
files/firewall/hostgroups.local.yaml Normal file
View File

@@ -0,0 +1,62 @@
firewall:
hostgroups:
analyst:
ips:
delete:
insert:
beats_endpoint:
ips:
delete:
insert:
beats_endpoint_ssl:
ips:
delete:
insert:
fleet:
ips:
delete:
insert:
heavy_node:
ips:
delete:
insert:
master:
ips:
delete:
insert:
minion:
ips:
delete:
insert:
node:
ips:
delete:
insert:
osquery_endpoint:
ips:
delete:
insert:
search_node:
ips:
delete:
insert:
sensor:
ips:
delete:
insert:
syslog:
ips:
delete:
insert:
wazuh_agent:
ips:
delete:
insert:
wazuh_api:
ips:
delete:
insert:
wazuh_authd:
ips:
delete:
insert:

3
files/firewall/portgroups.local.yaml Normal file
View File

@@ -0,0 +1,3 @@
firewall:
aliases:
ports:

7
files/master
View File

@@ -37,7 +37,9 @@ log_file: /opt/so/log/salt/master
#
file_roots:
base:
- /opt/so/saltstack/salt
- /opt/so/saltstack/local/salt
- /opt/so/saltstack/default/salt
# The master_roots setting configures a master-only copy of the file_roots dictionary,
# used by the state compiler.
@@ -53,7 +55,8 @@ file_roots:
pillar_roots:
base:
- /opt/so/saltstack/pillar
- /opt/so/saltstack/local/pillar
- /opt/so/saltstack/default/pillar
peer:
.*:

41
pillar/data/addtotab.sh
View File

@@ -1,7 +1,8 @@
#!/usr/bin/env bash
# This script adds sensors/nodes/etc to the nodes tab
default_salt_dir=/opt/so/saltstack/default
local_salt_dir=/opt/so/saltstack/local
TYPE=$1
NAME=$2
IPADDRESS=$3
@@ -15,7 +16,7 @@ MONINT=$9
#HOTNAME=$11
echo "Seeing if this host is already in here. If so delete it"
if grep -q $NAME "/opt/so/saltstack/pillar/data/$TYPE.sls"; then
if grep -q $NAME "$local_salt_dir/pillar/data/$TYPE.sls"; then
echo "Node Already Present - Let's re-add it"
awk -v blah=" $NAME:" 'BEGIN{ print_flag=1 }
{
@@ -31,27 +32,29 @@ if grep -q $NAME "/opt/so/saltstack/pillar/data/$TYPE.sls"; then
if ( print_flag == 1 )
print $0
} ' /opt/so/saltstack/pillar/data/$TYPE.sls > /opt/so/saltstack/pillar/data/tmp.$TYPE.sls
mv /opt/so/saltstack/pillar/data/tmp.$TYPE.sls /opt/so/saltstack/pillar/data/$TYPE.sls
} ' $local_salt_dir/pillar/data/$TYPE.sls > $local_salt_dir/pillar/data/tmp.$TYPE.sls
mv $local_salt_dir/pillar/data/tmp.$TYPE.sls $local_salt_dir/pillar/data/$TYPE.sls
echo "Deleted $NAME from the tab. Now adding it in again with updated info"
fi
echo " $NAME:" >> /opt/so/saltstack/pillar/data/$TYPE.sls
echo " ip: $IPADDRESS" >> /opt/so/saltstack/pillar/data/$TYPE.sls
echo " manint: $MANINT" >> /opt/so/saltstack/pillar/data/$TYPE.sls
echo " totalcpus: $CPUS" >> /opt/so/saltstack/pillar/data/$TYPE.sls
echo " guid: $GUID" >> /opt/so/saltstack/pillar/data/$TYPE.sls
echo " rootfs: $ROOTFS" >> /opt/so/saltstack/pillar/data/$TYPE.sls
echo " nsmfs: $NSM" >> /opt/so/saltstack/pillar/data/$TYPE.sls
echo " $NAME:" >> $local_salt_dir/pillar/data/$TYPE.sls
echo " ip: $IPADDRESS" >> $local_salt_dir/pillar/data/$TYPE.sls
echo " manint: $MANINT" >> $local_salt_dir/pillar/data/$TYPE.sls
echo " totalcpus: $CPUS" >> $local_salt_dir/pillar/data/$TYPE.sls
echo " guid: $GUID" >> $local_salt_dir/pillar/data/$TYPE.sls
echo " rootfs: $ROOTFS" >> $local_salt_dir/pillar/data/$TYPE.sls
echo " nsmfs: $NSM" >> $local_salt_dir/pillar/data/$TYPE.sls
if [ $TYPE == 'sensorstab' ]; then
echo " monint: $MONINT" >> /opt/so/saltstack/pillar/data/$TYPE.sls
salt-call state.apply common queue=True
echo " monint: $MONINT" >> $local_salt_dir/pillar/data/$TYPE.sls
salt-call state.apply grafana queue=True
fi
if [ $TYPE == 'evaltab' ]; then
echo " monint: $MONINT" >> /opt/so/saltstack/pillar/data/$TYPE.sls
salt-call state.apply common queue=True
salt-call state.apply utility queue=True
if [ $TYPE == 'evaltab' ] || [ $TYPE == 'standalonetab' ]; then
echo " monint: $MONINT" >> $local_salt_dir/pillar/data/$TYPE.sls
if [ ! $10 ]; then
salt-call state.apply grafana queue=True
salt-call state.apply utility queue=True
fi
fi
#if [ $TYPE == 'nodestab' ]; then
# echo " nodetype: $NODETYPE" >> /opt/so/saltstack/pillar/data/$TYPE.sls
# echo " hotname: $HOTNAME" >> /opt/so/saltstack/pillar/data/$TYPE.sls
# echo " nodetype: $NODETYPE" >> $local_salt_dir/pillar/data/$TYPE.sls
# echo " hotname: $HOTNAME" >> $local_salt_dir/pillar/data/$TYPE.sls
#fi

1
pillar/data/evaltab.sls
View File

@@ -1 +0,0 @@
evaltab:

1
pillar/data/mastersearchtab.sls
View File

@@ -1 +0,0 @@
mastersearchtab:

1
pillar/data/mastertab.sls
View File

@@ -1 +0,0 @@
mastertab:

1
pillar/data/nodestab.sls
View File

@@ -1 +0,0 @@
nodestab:

1
pillar/data/sensorstab.sls
View File

@@ -1 +0,0 @@
sensorstab:

6
pillar/firewall/addfirewall.sh
View File

@@ -1,13 +1,13 @@
#!/usr/bin/env bash
# This script adds ip addresses to specific rule sets defined by the user
local_salt_dir=/opt/so/saltstack/local
POLICY=$1
IPADDRESS=$2
if grep -q $2 "/opt/so/saltstack/pillar/firewall/$1.sls"; then
if grep -q $2 "$local_salt_dir/pillar/firewall/$1.sls"; then
echo "Firewall Rule Already There"
else
echo " - $2" >> /opt/so/saltstack/pillar/firewall/$1.sls
echo " - $2" >> $local_salt_dir/pillar/firewall/$1.sls
salt-call state.apply firewall queue=True
fi

62
pillar/firewall/ports.sls Normal file
View File

@@ -0,0 +1,62 @@
firewall:
analyst:
ports:
tcp:
- 80
- 443
udp:
beats_endpoint:
ports:
tcp:
- 5044
forward_nodes:
ports:
tcp:
- 443
- 5044
- 5644
- 9822
udp:
master:
ports:
tcp:
- 1514
- 3200
- 3306
- 4200
- 5601
- 6379
- 8086
- 8090
- 9001
- 9200
- 9300
- 9400
- 9500
udp:
- 1514
minions:
ports:
tcp:
- 3142
- 4505
- 4506
- 5000
- 8080
- 8086
- 55000
osquery_endpoint:
ports:
tcp:
- 8090
search_nodes:
ports:
tcp:
- 6379
- 9300
wazuh_endpoint:
ports:
tcp:
- 1514
udp:
-1514

1
pillar/logstash/master.sls
View File

@@ -2,5 +2,6 @@ logstash:
pipelines:
master:
config:
- so/0009_input_beats.conf
- so/0010_input_hhbeats.conf
- so/9999_output_redis.conf.jinja

2
pillar/logstash/search.sls
View File

@@ -5,12 +5,12 @@ logstash:
- so/0900_input_redis.conf.jinja
- so/9000_output_zeek.conf.jinja
- so/9002_output_import.conf.jinja
- so/9034_output_syslog.conf.jinja
- so/9100_output_osquery.conf.jinja
- so/9400_output_suricata.conf.jinja
- so/9500_output_beats.conf.jinja
- so/9600_output_ossec.conf.jinja
- so/9700_output_strelka.conf.jinja
templates:
- so/so-beats-template.json
- so/so-common-template.json
- so/so-zeek-template.json

9
pillar/top.sls
View File

@@ -14,7 +14,6 @@ base:
'*_sensor':
- static
- firewall.*
- brologs
- healthcheck.sensor
- minions.{{ grains.id }}
@@ -22,7 +21,6 @@ base:
'*_master or *_mastersearch':
- match: compound
- static
- firewall.*
- data.*
- secrets
- minions.{{ grains.id }}
@@ -33,7 +31,6 @@ base:
'*_eval':
- static
- firewall.*
- data.*
- brologs
- secrets
@@ -44,7 +41,6 @@ base:
- logstash
- logstash.master
- logstash.search
- firewall.*
- data.*
- brologs
- secrets
@@ -54,18 +50,15 @@ base:
'*_node':
- static
- firewall.*
- minions.{{ grains.id }}
'*_heavynode':
- static
- firewall.*
- brologs
- minions.{{ grains.id }}
'*_helix':
- static
- firewall.*
- fireeye
- brologs
- logstash
@@ -74,14 +67,12 @@ base:
'*_fleet':
- static
- firewall.*
- data.*
- secrets
- minions.{{ grains.id }}
'*_searchnode':
- static
- firewall.*
- logstash
- logstash.search
- minions.{{ grains.id }}

2
salt/ca/init.sls
View File

@@ -44,5 +44,3 @@ send_x509_pem_entries_to_mine:
- mine.send:
- func: x509.get_pem_entries
- glob_path: /etc/pki/ca.crt
- onchanges:
- x509: /etc/pki/ca.crt

81
salt/common/init.sls
View File

@@ -28,20 +28,85 @@ salttmp:
- group: 939
- makedirs: True
# Install packages needed for the sensor
sensorpkgs:
# Install epel
{% if grains['os'] == 'CentOS' %}
epel:
pkg.installed:
- skip_suggestions: False
- skip_suggestions: True
- pkgs:
- epel-release
{% endif %}
# Install common packages
{% if grains['os'] != 'CentOS' %}
commonpkgs:
pkg.installed:
- skip_suggestions: True
- pkgs:
- apache2-utils
- wget
- ntpdate
- jq
- python3-docker
- docker-ce
- curl
- ca-certificates
- software-properties-common
- apt-transport-https
- openssl
- netcat
- python3-mysqldb
- sqlite3
- argon2
- libssl-dev
- python3-dateutil
- python3-m2crypto
- python3-mysqldb
- git
heldpackages:
pkg.installed:
- pkgs:
- containerd.io: 1.2.13-2
- docker-ce: 5:19.03.9~3-0~ubuntu-bionic
- hold: True
- update_holds: True
{% else %}
commonpkgs:
pkg.installed:
- skip_suggestions: True
- pkgs:
- wget
- ntpdate
- bind-utils
- jq
{% if grains['os'] != 'CentOS' %}
- apache2-utils
{% else %}
- net-tools
- tcpdump
- httpd-tools
{% endif %}
- net-tools
- curl
- sqlite
- argon2
- mariadb-devel
- nmap-ncat
- python3
- python36-docker
- python36-dateutil
- python36-m2crypto
- python36-mysql
- yum-utils
- device-mapper-persistent-data
- lvm2
- openssl
- git
heldpackages:
pkg.installed:
- pkgs:
- containerd.io: 1.2.13-3.2.el7
- docker-ce: 3:19.03.11-3.el7
- hold: True
- update_holds: True
{% endif %}
# Always keep these packages up to date

1
salt/common/maps/eval.map.jinja
View File

@@ -1,5 +1,6 @@
{% set docker = {
'containers': [
'so-filebeat',
'so-nginx',
'so-telegraf',
'so-dockerregistry',

37
salt/common/tools/sbin/so-allow
View File

@@ -17,6 +17,9 @@
. /usr/sbin/so-common
default_salt_dir=/opt/so/saltstack/default
local_salt_dir=/opt/so/saltstack/local
SKIP=0
while getopts "abowi:" OPTION
@@ -42,9 +45,22 @@ do
SKIP=1
;;
w)
FULLROLE="wazuh_endpoint"
FULLROLE="wazuh_agent"
SKIP=1
;;
s)
FULLROLE="syslog"
SKIP=1
;;
p)
FULLROLE="wazuh_api"
SKIP=1
;;
r)
FULLROLE="wazuh_authd"
SKIP=1
;;
esac
done
@@ -57,8 +73,10 @@ if [ "$SKIP" -eq 0 ]; then
echo "[a] - Analyst - ports 80/tcp and 443/tcp"
echo "[b] - Logstash Beat - port 5044/tcp"
echo "[o] - Osquery endpoint - port 8090/tcp"
echo "[w] - Wazuh endpoint - port 1514"
echo ""
echo "[s] - Syslog device - 514/tcp/udp"
echo "[w] - Wazuh agent - port 1514/tcp/udp"
echo "[p] - Wazuh API - port 55000/tcp"
echo "[r] - Wazuh registration service - 1515/tcp"
echo "Please enter your selection (a - analyst, b - beats, o - osquery, w - wazuh):"
read ROLE
echo "Enter a single ip address or range to allow (example: 10.10.10.10 or 10.10.0.0/16):"
@@ -71,7 +89,13 @@ if [ "$SKIP" -eq 0 ]; then
elif [ "$ROLE" == "o" ]; then
FULLROLE=osquery_endpoint
elif [ "$ROLE" == "w" ]; then
FULLROLE=wazuh_endpoint
FULLROLE=wazuh_agent
elif [ "$ROLE" == "s" ]; then
FULLROLE=syslog
elif [ "$ROLE" == "p" ]; then
FULLROLE=wazuh_api
elif [ "$ROLE" == "r" ]; then
FULLROLE=wazuh_authd
else
echo "I don't recognize that role"
exit 1
@@ -80,10 +104,11 @@ if [ "$SKIP" -eq 0 ]; then
fi
echo "Adding $IP to the $FULLROLE role. This can take a few seconds"
/opt/so/saltstack/pillar/firewall/addfirewall.sh $FULLROLE $IP
/usr/sbin/so-firewall includehost $FULLROLE $IP
salt-call state.apply firewall queue=True
# Check if Wazuh enabled
if grep -q -R "wazuh: 1" /opt/so/saltstack/pillar/*; then
if grep -q -R "wazuh: 1" $local_salt_dir/pillar/*; then
# If analyst, add to Wazuh AR whitelist
if [ "$FULLROLE" == "analyst" ]; then
WAZUH_MGR_CFG="/opt/so/wazuh/etc/ossec.conf"

7
salt/common/tools/sbin/so-bro-logs
View File

@@ -1,11 +1,12 @@
#!/bin/bash
local_salt_dir=/opt/so/saltstack/local
bro_logs_enabled() {
echo "brologs:" > /opt/so/saltstack/pillar/brologs.sls
echo " enabled:" >> /opt/so/saltstack/pillar/brologs.sls
echo "brologs:" > $local_salt_dir/pillar/brologs.sls
echo " enabled:" >> $local_salt_dir/pillar/brologs.sls
for BLOG in ${BLOGS[@]}; do
echo " - $BLOG" | tr -d '"' >> /opt/so/saltstack/pillar/brologs.sls
echo " - $BLOG" | tr -d '"' >> $local_salt_dir/pillar/brologs.sls
done
}

5
salt/common/tools/sbin/so-cortex-restart
View File

@@ -1,5 +1,5 @@
#!/bin/bash
#
# Copyright 2014,2015,2016,2017,2018,2019,2020 Security Onion Solutions, LLC
#
# This program is free software: you can redistribute it and/or modify
@@ -17,4 +17,5 @@
. /usr/sbin/so-common
/usr/sbin/so-restart cortex $1
/usr/sbin/so-stop cortex $1
/usr/sbin/so-start thehive $1

2
salt/common/tools/sbin/so-cortex-start
View File

@@ -17,4 +17,4 @@
. /usr/sbin/so-common
/usr/sbin/so-start cortex $1
/usr/sbin/so-start thehive $1

2
salt/common/tools/sbin/so-cortex-stop
View File

@@ -1,5 +1,5 @@
#!/bin/bash
#
# Copyright 2014,2015,2016,2017,2018,2019,2020 Security Onion Solutions, LLC
#
# This program is free software: you can redistribute it and/or modify

112
salt/common/tools/sbin/so-docker-refresh Normal file
View File

@@ -0,0 +1,112 @@
#!/bin/bash
# Copyright 2014,2015,2016,2017,2018,2019,2020 Security Onion Solutions, LLC
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
got_root(){
if [ "$(id -u)" -ne 0 ]; then
echo "This script must be run using sudo!"
exit 1
fi
}
master_check() {
# Check to see if this is a master
MASTERCHECK=$(cat /etc/salt/grains | grep role | awk '{print $2}')
if [ $MASTERCHECK == 'so-eval' ] || [ $MASTERCHECK == 'so-master' ] || [ $MASTERCHECK == 'so-mastersearch' ] || [ $MASTERCHECK == 'so-standalone' ] || [ $MASTERCHECK == 'so-helix' ]; then
echo "This is a master. We can proceed"
else
echo "Please run soup on the master. The master controls all updates."
exit 1
fi
}
update_docker_containers() {
# Download the containers from the interwebs
for i in "${TRUSTED_CONTAINERS[@]}"
do
# Pull down the trusted docker image
echo "Downloading $i"
docker pull --disable-content-trust=false docker.io/soshybridhunter/$i
# Tag it with the new registry destination
docker tag soshybridhunter/$i $HOSTNAME:5000/soshybridhunter/$i
docker push $HOSTNAME:5000/soshybridhunter/$i
done
}
version_check() {
if [ -f /etc/soversion ]; then
VERSION=$(cat /etc/soversion)
else
echo "Unable to detect version. I will now terminate."
exit 1
fi
}
got_root
master_check
version_check
# Use the hostname
HOSTNAME=$(hostname)
BUILD=HH
# List all the containers
if [ $MASTERCHECK != 'so-helix' ]; then
TRUSTED_CONTAINERS=( \
"so-acng:$BUILD$VERSION" \
"so-thehive-cortex:$BUILD$VERSION" \
"so-curator:$BUILD$VERSION" \
"so-domainstats:$BUILD$VERSION" \
"so-elastalert:$BUILD$VERSION" \
"so-elasticsearch:$BUILD$VERSION" \
"so-filebeat:$BUILD$VERSION" \
"so-fleet:$BUILD$VERSION" \
"so-fleet-launcher:$BUILD$VERSION" \
"so-freqserver:$BUILD$VERSION" \
"so-grafana:$BUILD$VERSION" \
"so-idstools:$BUILD$VERSION" \
"so-influxdb:$BUILD$VERSION" \
"so-kibana:$BUILD$VERSION" \
"so-kratos:$BUILD$VERSION" \
"so-logstash:$BUILD$VERSION" \
"so-mysql:$BUILD$VERSION" \
"so-navigator:$BUILD$VERSION" \
"so-nginx:$BUILD$VERSION" \
"so-playbook:$BUILD$VERSION" \
"so-redis:$BUILD$VERSION" \
"so-soc:$BUILD$VERSION" \
"so-soctopus:$BUILD$VERSION" \
"so-steno:$BUILD$VERSION" \
"so-strelka:$BUILD$VERSION" \
"so-suricata:$BUILD$VERSION" \
"so-telegraf:$BUILD$VERSION" \
"so-thehive:$BUILD$VERSION" \
"so-thehive-es:$BUILD$VERSION" \
"so-wazuh:$BUILD$VERSION" \
"so-zeek:$BUILD$VERSION" )
else
TRUSTED_CONTAINERS=( \
"so-filebeat:$BUILD$VERSION" \
"so-idstools:$BUILD$VERSION" \
"so-logstash:$BUILD$VERSION" \
"so-nginx:$BUILD$VERSION" \
"so-redis:$BUILD$VERSION" \
"so-steno:$BUILD$VERSION" \
"so-suricata:$BUILD$VERSION" \
"so-telegraf:$BUILD$VERSION" \
"so-zeek:$BUILD$VERSION" )
fi
update_docker_containers

3
salt/common/tools/sbin/so-elastalert-create
View File

@@ -166,8 +166,7 @@ cat << EOF
What elasticsearch index do you want to use?
Below are the default Index Patterns used in Security Onion:
*:logstash-*
*:logstash-beats-*
*:so-ids-*
*:elastalert_status*
EOF

3
salt/common/tools/sbin/so-elasticsearch-templates
View File

@@ -15,12 +15,13 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
default_salt_dir=/opt/so/saltstack/default
ELASTICSEARCH_HOST="{{ MASTERIP}}"
ELASTICSEARCH_PORT=9200
#ELASTICSEARCH_AUTH=""
# Define a default directory to load pipelines from
ELASTICSEARCH_TEMPLATES="/opt/so/saltstack/salt/logstash/pipelines/templates/so/"
ELASTICSEARCH_TEMPLATES="$default_salt_dir/salt/logstash/pipelines/templates/so/"
# Wait for ElasticSearch to initialize
echo -n "Waiting for ElasticSearch..."

5
salt/common/tools/sbin/so-features-enable
View File

@@ -15,10 +15,11 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common
local_salt_dir=/opt/so/saltstack/local
VERSION=$(grep soversion /opt/so/saltstack/pillar/static.sls | cut -d':' -f2|sed 's/ //g')
VERSION=$(grep soversion $local_salt_dir/pillar/static.sls | cut -d':' -f2|sed 's/ //g')
# Modify static.sls to enable Features
sed -i 's/features: False/features: True/' /opt/so/saltstack/pillar/static.sls
sed -i 's/features: False/features: True/' $local_salt_dir/pillar/static.sls
SUFFIX="-features"
TRUSTED_CONTAINERS=( \
"so-elasticsearch:$VERSION$SUFFIX" \

305
salt/common/tools/sbin/so-firewall Executable file
View File

@@ -0,0 +1,305 @@
#!/usr/bin/env python3
# Copyright 2014,2015,2016,2017,2018,2019,2020 Security Onion Solutions, LLC
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import subprocess
import sys
import yaml
hostgroupsFilename = "/opt/so/saltstack/local/salt/firewall/hostgroups.local.yaml"
portgroupsFilename = "/opt/so/saltstack/local/salt/firewall/portgroups.local.yaml"
supportedProtocols = ['tcp', 'udp']
def showUsage(args):
print('Usage: {} [OPTIONS] <COMMAND> [ARGS...]'.format(sys.argv[0]))
print(' Options:')
print(' --apply - After updating the firewall configuration files, apply the new firewall state')
print('')
print(' Available commands:')
print(' help - Prints this usage information.')
print(' includedhosts - Lists the IPs included in the given group. Args: <GROUP_NAME>')
print(' excludedhosts - Lists the IPs excluded from the given group. Args: <GROUP_NAME>')
print(' includehost - Includes the given IP in the given group. Args: <GROUP_NAME> <IP>')
print(' excludehost - Excludes the given IP from the given group. Args: <GROUP_NAME> <IP>')
print(' removehost - Removes an excluded IP from the given group. Args: <GROUP_NAME> <IP>')
print(' addhostgroup - Adds a new, custom host group. Args: <GROUP_NAME>')
print(' listports - Lists ports in the given group and protocol. Args: <GROUP_NAME> <PORT_PROTOCOL>')
print(' addport - Adds a PORT to the given group. Args: <GROUP_NAME> <PORT_PROTOCOL> <PORT>')
print(' removeport - Removes a PORT from the given group. Args: <GROUP_NAME> <PORT_PROTOCOL> <PORT>')
print(' addportgroup - Adds a new, custom port group. Args: <GROUP_NAME>')
print('')
print(' Where:')
print(' GROUP_NAME - The name of an alias group (Ex: analyst)')
print(' IP - Either a single IP address (Ex: 8.8.8.8) or a CIDR block (Ex: 10.23.0.0/16).')
print(' PORT_PROTOCOL - Must be one of the following: ' + str(supportedProtocols))
print(' PORT - Either a single numeric port (Ex: 443), or a port range (Ex: 8000:8002).')
sys.exit(1)
def loadYaml(filename):
file = open(filename, "r")
return yaml.load(file.read())
def writeYaml(filename, content):
file = open(filename, "w")
return yaml.dump(content, file)
def listIps(name, mode):
content = loadYaml(hostgroupsFilename)
if name not in content['firewall']['hostgroups']:
print('Host group does not exist', file=sys.stderr)
return 4
hostgroup = content['firewall']['hostgroups'][name]
ips = hostgroup['ips'][mode]
if ips is not None:
for ip in ips:
print(ip)
return 0
def addIp(name, ip, mode):
content = loadYaml(hostgroupsFilename)
if name not in content['firewall']['hostgroups']:
print('Host group does not exist', file=sys.stderr)
return 4
hostgroup = content['firewall']['hostgroups'][name]
ips = hostgroup['ips'][mode]
if ips is None:
ips = []
hostgroup['ips'][mode] = ips
if ip not in ips:
ips.append(ip)
else:
print('Already exists', file=sys.stderr)
return 3
writeYaml(hostgroupsFilename, content)
return 0
def removeIp(name, ip, mode, silence = False):
content = loadYaml(hostgroupsFilename)
if name not in content['firewall']['hostgroups']:
print('Host group does not exist', file=sys.stderr)
return 4
hostgroup = content['firewall']['hostgroups'][name]
ips = hostgroup['ips'][mode]
if ips is None:
ips = []
hostgroup['ips'][mode] = ips
if ip in ips:
ips.remove(ip)
else:
if not silence:
print('IP does not exist', file=sys.stderr)
return 3
writeYaml(hostgroupsFilename, content)
return 0
def createProtocolMap():
map = {}
for protocol in supportedProtocols:
map[protocol] = []
return map
def addhostgroup(args):
if len(args) != 1:
print('Missing host group name argument', file=sys.stderr)
showUsage(args)
name = args[1]
content = loadYaml(hostgroupsFilename)
if name in content['firewall']['hostgroups']:
print('Already exists', file=sys.stderr)
return 3
content['firewall']['hostgroups'][name] = { 'ips': { 'insert': [], 'delete': [] }}
writeYaml(hostgroupsFilename, content)
return 0
def addportgroup(args):
if len(args) != 1:
print('Missing port group name argument', file=sys.stderr)
showUsage(args)
name = args[0]
content = loadYaml(portgroupsFilename)
ports = content['firewall']['aliases']['ports']
if ports is None:
ports = {}
content['firewall']['aliases']['ports'] = ports
if name in ports:
print('Already exists', file=sys.stderr)
return 3
ports[name] = createProtocolMap()
writeYaml(portgroupsFilename, content)
return 0
def listports(args):
if len(args) != 2:
print('Missing port group name or port protocol', file=sys.stderr)
showUsage(args)
name = args[0]
protocol = args[1]
if protocol not in supportedProtocols:
print('Port protocol is not supported', file=sys.stderr)
return 5
content = loadYaml(portgroupsFilename)
ports = content['firewall']['aliases']['ports']
if ports is None:
ports = {}
content['firewall']['aliases']['ports'] = ports
if name not in ports:
print('Port group does not exist', file=sys.stderr)
return 3
ports = ports[name][protocol]
if ports is not None:
for port in ports:
print(port)
return 0
def addport(args):
if len(args) != 3:
print('Missing port group name or port protocol, or port argument', file=sys.stderr)
showUsage(args)
name = args[0]
protocol = args[1]
port = args[2]
if protocol not in supportedProtocols:
print('Port protocol is not supported', file=sys.stderr)
return 5
content = loadYaml(portgroupsFilename)
ports = content['firewall']['aliases']['ports']
if ports is None:
ports = {}
content['firewall']['aliases']['ports'] = ports
if name not in ports:
print('Port group does not exist', file=sys.stderr)
return 3
ports = ports[name][protocol]
if ports is None:
ports = []
content['firewall']['aliases']['ports'][name][protocol] = ports
if port in ports:
print('Already exists', file=sys.stderr)
return 3
ports.append(port)
writeYaml(portgroupsFilename, content)
return 0
def removeport(args):
if len(args) != 3:
print('Missing port group name or port protocol, or port argument', file=sys.stderr)
showUsage(args)
name = args[0]
protocol = args[1]
port = args[2]
if protocol not in supportedProtocols:
print('Port protocol is not supported', file=sys.stderr)
return 5
content = loadYaml(portgroupsFilename)
ports = content['firewall']['aliases']['ports']
if ports is None:
ports = {}
content['firewall']['aliases']['ports'] = ports
if name not in ports:
print('Port group does not exist', file=sys.stderr)
return 3
ports = ports[name][protocol]
if ports is None or port not in ports:
print('Port does not exist', file=sys.stderr)
return 3
ports.remove(port)
writeYaml(portgroupsFilename, content)
return 0
def includedhosts(args):
if len(args) != 1:
print('Missing host group name argument', file=sys.stderr)
showUsage(args)
return listIps(args[0], 'insert')
def excludedhosts(args):
if len(args) != 1:
print('Missing host group name argument', file=sys.stderr)
showUsage(args)
return listIps(args[0], 'delete')
def includehost(args):
if len(args) != 2:
print('Missing host group name or ip argument', file=sys.stderr)
showUsage(args)
result = addIp(args[0], args[1], 'insert')
if result == 0:
removeIp(args[0], args[1], 'delete', True)
return result
def excludehost(args):
if len(args) != 2:
print('Missing host group name or ip argument', file=sys.stderr)
showUsage(args)
result = addIp(args[0], args[1], 'delete')
if result == 0:
removeIp(args[0], args[1], 'insert', True)
return result
def removehost(args):
if len(args) != 2:
print('Missing host group name or ip argument', file=sys.stderr)
showUsage(args)
return removeIp(args[0], args[1], 'delete')
def apply():
proc = subprocess.run(['salt-call', 'state.apply', 'firewall', 'queue=True'])
return proc.returncode
def main():
options = []
args = sys.argv[1:]
for option in args:
if option.startswith("--"):
options.append(option)
args.remove(option)
if len(args) == 0:
showUsage(None)
commands = {
"help": showUsage,
"includedhosts": includedhosts,
"excludedhosts": excludedhosts,
"includehost": includehost,
"excludehost": excludehost,
"removehost": removehost,
"listports": listports,
"addport": addport,
"removeport": removeport,
"addhostgroup": addhostgroup,
"addportgroup": addportgroup
}
cmd = commands.get(args[0], showUsage)
code = cmd(args[1:])
if code == 0 and "--apply" in options:
code = apply()
sys.exit(code)
if __name__ == "__main__":
main()

44
salt/common/tools/sbin/so-fleet-setup Normal file
View File

@@ -0,0 +1,44 @@
#!/bin/bash
#so-fleet-setup $FleetEmail $FleetPassword
if [[ $# -ne 2 ]] ; then
echo "Username or Password was not set - exiting now."
exit 1
fi
# Checking to see if required containers are started...
if [ ! "$(docker ps -q -f name=so-fleet)" ]; then
echo "Starting Docker Containers..."
salt-call state.apply mysql queue=True >> /root/fleet-setup.log
salt-call state.apply fleet queue=True >> /root/fleet-setup.log
salt-call state.apply redis queue=True >> /root/fleet-setup.log
fi
docker exec so-fleet fleetctl config set --address https://localhost:8080 --tls-skip-verify --url-prefix /fleet
docker exec so-fleet fleetctl setup --email $1 --password $2
docker exec so-fleet fleetctl apply -f /packs/palantir/Fleet/Endpoints/MacOS/osquery.yaml
docker exec so-fleet fleetctl apply -f /packs/palantir/Fleet/Endpoints/Windows/osquery.yaml
docker exec so-fleet fleetctl apply -f /packs/so/so-default.yml
docker exec so-fleet /bin/sh -c 'for pack in /packs/palantir/Fleet/Endpoints/packs/*.yaml; do fleetctl apply -f "$pack"; done'
docker exec so-fleet fleetctl apply -f /packs/osquery-config.conf
# Enable Fleet
echo "Enabling Fleet..."
salt-call state.apply fleet.event_enable-fleet queue=True >> /root/fleet-setup.log
salt-call state.apply nginx queue=True >> /root/fleet-setup.log
# Generate osquery install packages
echo "Generating osquery install packages - this will take some time..."
salt-call state.apply fleet.event_gen-packages queue=True >> /root/fleet-setup.log
sleep 120
echo "Installing launcher via salt..."
salt-call state.apply fleet.install_package queue=True >> /root/fleet-setup.log
salt-call state.apply filebeat queue=True >> /root/fleet-setup.log
docker stop so-nginx
salt-call state.apply nginx queue=True >> /root/fleet-setup.log
echo "Fleet Setup Complete - Login with the username and password you ran the script with."

7
salt/common/tools/sbin/so-helix-apikey
View File

@@ -1,4 +1,7 @@
#!/bin/bash
local_salt_dir=/opt/so/saltstack/local
got_root() {
# Make sure you are root
@@ -10,13 +13,13 @@ got_root() {
}
got_root
if [ ! -f /opt/so/saltstack/pillar/fireeye/init.sls ]; then
if [ ! -f $local_salt_dir/pillar/fireeye/init.sls ]; then
echo "This is nto configured for Helix Mode. Please re-install."
exit
else
echo "Enter your Helix API Key: "
read APIKEY
sed -i "s/^ api_key.*/ api_key: $APIKEY/g" /opt/so/saltstack/pillar/fireeye/init.sls
sed -i "s/^ api_key.*/ api_key: $APIKEY/g" $local_salt_dir/pillar/fireeye/init.sls
docker stop so-logstash
docker rm so-logstash
echo "Restarting Logstash for updated key"

2
salt/common/tools/sbin/so-kibana-config-export
View File

@@ -31,5 +31,5 @@ sed -i "s/$KIBANA_HOST/PLACEHOLDER/g" $OUTFILE
# Clean up for Fleet, if applicable
# {% if FLEET_NODE or FLEET_MASTER %}
# Fleet IP
sed -i "s/{{ FLEET_IP }}/FLEETPLACEHOLDER/g" $OUTFILE
sed -i "s/{{ MASTER }}/FLEETPLACEHOLDER/g" $OUTFILE
# {% endif %}

57
salt/common/tools/sbin/so-saltstack-update Normal file
View File

@@ -0,0 +1,57 @@
#!/bin/bash
# Copyright 2014,2015,2016,2017,2018,2019,2020 Security Onion Solutions, LLC
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
default_salt_dir=/opt/so/saltstack/default
clone_to_tmp() {
# TODO Need to add a air gap option
# Make a temp location for the files
mkdir /tmp/sogh
cd /tmp/sogh
#git clone -b dev https://github.com/Security-Onion-Solutions/securityonion-saltstack.git
git clone https://github.com/Security-Onion-Solutions/securityonion-saltstack.git
cd /tmp
}
copy_new_files() {
# Copy new files over to the salt dir
cd /tmp/sogh/securityonion-saltstack
git checkout $BRANCH
rsync -a --exclude-from 'exclude-list.txt' salt $default_salt_dir/
rsync -a --exclude-from 'exclude-list.txt' pillar $default_salt_dir/
chown -R socore:socore $default_salt_dir/salt
chown -R socore:socore $default_salt_dir/pillar
chmod 755 $default_salt_dir/pillar/firewall/addfirewall.sh
rm -rf /tmp/sogh
}
got_root(){
if [ "$(id -u)" -ne 0 ]; then
echo "This script must be run using sudo!"
exit 1
fi
}
got_root
if [ $# -ne 1 ] ; then
BRANCH=master
else
BRANCH=$1
fi
clone_to_tmp
copy_new_files

2
salt/common/tools/sbin/so-start
View File

@@ -32,5 +32,5 @@ fi
case $1 in
"all") salt-call state.highstate queue=True;;
"steno") if docker ps | grep -q so-$1; then printf "\n$1 is already running!\n\n"; else docker rm so-$1 >/dev/null 2>&1 ; salt-call state.apply pcap queue=True; fi ;;
*) if docker ps | grep -q so-$1; then printf "\n$1 is already running\n\n"; else docker rm so-$1 >/dev/null 2>&1 ; salt-call state.apply $1 queue=True; fi ;;
*) if docker ps | grep -E -q '^so-$1$'; then printf "\n$1 is already running\n\n"; else docker rm so-$1 >/dev/null 2>&1 ; salt-call state.apply $1 queue=True; fi ;;
esac

2
salt/common/tools/sbin/so-status
View File

@@ -15,7 +15,7 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
{%- from 'common/maps/so-status.map.jinja' import docker with context %}
{%- set container_list = docker['containers'] %}
{%- set container_list = docker['containers'] | sort %}
if ! [ "$(id -u)" = 0 ]; then
echo "This command must be run as root"

21
salt/common/tools/sbin/so-thehive-es-restart Executable file
View File

@@ -0,0 +1,21 @@
#!/bin/bash
#
# Copyright 2014,2015,2016,2017,2018,2019,2020 Security Onion Solutions, LLC
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common
/usr/sbin/so-stop thehive-es $1
/usr/sbin/so-start thehive $1

20
salt/common/tools/sbin/so-thehive-es-start Executable file
View File

@@ -0,0 +1,20 @@
#!/bin/bash
# Copyright 2014,2015,2016,2017,2018,2019,2020 Security Onion Solutions, LLC
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common
/usr/sbin/so-start thehive $1

20
salt/common/tools/sbin/so-thehive-es-stop Executable file
View File

@@ -0,0 +1,20 @@
#!/bin/bash
#
# Copyright 2014,2015,2016,2017,2018,2019,2020 Security Onion Solutions, LLC
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common
/usr/sbin/so-stop thehive-es $1

2
salt/common/tools/sbin/so-thehive-restart
View File

@@ -1,5 +1,5 @@
#!/bin/bash
#
# Copyright 2014,2015,2016,2017,2018,2019,2020 Security Onion Solutions, LLC
#
# This program is free software: you can redistribute it and/or modify

2
salt/common/tools/sbin/so-thehive-stop
View File

@@ -1,5 +1,5 @@
#!/bin/bash
#
# Copyright 2014,2015,2016,2017,2018,2019,2020 Security Onion Solutions, LLC
#
# This program is free software: you can redistribute it and/or modify

39
salt/common/tools/sbin/so-zeek-stats Normal file
View File

@@ -0,0 +1,39 @@
#!/bin/bash
# Copyright 2014,2015,2016,2017,2018,2019,2020 Security Onion Solutions, LLC
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Show Zeek stats (capstats, netstats)
show_stats() {
echo '##############'
echo '# Zeek Stats #'
echo '##############'
echo
echo "Average throughput:"
echo
docker exec -it so-zeek /opt/zeek/bin/zeekctl capstats
echo
echo "Average packet loss:"
echo
docker exec -it so-zeek /opt/zeek/bin/zeekctl netstats
echo
}
if docker ps | grep -q zeek; then
show_stats
else
echo "Zeek is not running! Try starting it with 'so-zeek-start'." && exit 1;
fi

5
salt/curator/files/action/close.yml
View File

@@ -24,9 +24,8 @@ actions:
disable_action: False
filters:
- filtertype: pattern
kind: prefix
value: logstash-
exclude:
kind: regex
value: '^(logstash-.*|so-.*)$'
- filtertype: age
source: name
direction: older

4
salt/curator/files/action/delete.yml
View File

@@ -20,8 +20,8 @@ actions:
disable_action: False
filters:
- filtertype: pattern
kind: prefix
value: logstash-
kind: regex
value: '^(logstash-.*|so-.*)$'
- filtertype: space
source: creation_date
use_age: True

8
salt/curator/files/bin/so-curator-closed-delete-delete
View File

@@ -33,17 +33,17 @@ LOG="/opt/so/log/curator/so-curator-closed-delete.log"
# Check for 2 conditions:
# 1. Are Elasticsearch indices using more disk space than LOG_SIZE_LIMIT?
# 2. Are there any closed logstash- indices that we can delete?
# 2. Are there any closed logstash-, or so- indices that we can delete?
# If both conditions are true, keep on looping until one of the conditions is false.
while [[ $(du -hs --block-size=1GB /nsm/elasticsearch/nodes | awk '{print $1}' ) -gt "{{LOG_SIZE_LIMIT}}" ]] &&
curl -s {{ELASTICSEARCH_HOST}}:{{ELASTICSEARCH_PORT}}/_cat/indices | grep "^ close logstash-" > /dev/null; do
curl -s {{ELASTICSEARCH_HOST}}:{{ELASTICSEARCH_PORT}}/_cat/indices | grep -E "^ close (logstash-|so-)" > /dev/null; do
# We need to determine OLDEST_INDEX.
# First, get the list of closed indices that are prefixed with "logstash-".
# First, get the list of closed indices that are prefixed with "logstash-" or "so-".
# For example: logstash-ids-YYYY.MM.DD
# Then, sort by date by telling sort to use hyphen as delimiter and then sort on the third field.
# Finally, select the first entry in that sorted list.
OLDEST_INDEX=$(curl -s {{ELASTICSEARCH_HOST}}:{{ELASTICSEARCH_PORT}}/_cat/indices | grep "^ close logstash-" | awk '{print $2}' | sort -t- -k3 | head -1)
OLDEST_INDEX=$(curl -s {{ELASTICSEARCH_HOST}}:{{ELASTICSEARCH_PORT}}/_cat/indices | grep -E "^ close (logstash-|so-)" | awk '{print $2}' | sort -t- -k3 | head -1)
# Now that we've determined OLDEST_INDEX, ask Elasticsearch to delete it.
curl -XDELETE {{ELASTICSEARCH_HOST}}:{{ELASTICSEARCH_PORT}}/${OLDEST_INDEX}

2
salt/curator/files/curator.yml
View File

@@ -1,4 +1,4 @@
{% if grains['role'] in ['so-node', 'so-searchnode', 'so-heavynode'] %}
{% if grains['role'] in ['so-node', 'so-heavynode'] %}
{%- set elasticsearch = salt['pillar.get']('node:mainip', '') -%}
{% elif grains['role'] in ['so-eval', 'so-mastersearch', 'so-standalone'] %}
{%- set elasticsearch = salt['pillar.get']('master:mainip', '') -%}

8
salt/curator/init.sls
View File

@@ -1,6 +1,6 @@
{% set VERSION = salt['pillar.get']('static:soversion', 'HH1.2.2') %}
{% set MASTER = salt['grains.get']('master') %}
{% if grains['role'] in ['so-searchnode', 'so-eval', 'so-node', 'so-mastersearch', 'so-heavynode', 'so-standalone'] %}
{% if grains['role'] in ['so-eval', 'so-node', 'so-mastersearch', 'so-heavynode', 'so-standalone'] %}
# Curator
# Create the group
curatorgroup:
@@ -89,7 +89,7 @@ curdel:
so-curatorcloseddeletecron:
cron.present:
- name: /usr/sbin/so-curator-closed-delete
- name: /usr/sbin/so-curator-closed-delete > /opt/so/log/curator/cron-closed-delete.log 2>&1
- user: root
- minute: '*'
- hour: '*'
@@ -99,7 +99,7 @@ so-curatorcloseddeletecron:
so-curatorclosecron:
cron.present:
- name: /usr/sbin/so-curator-close
- name: /usr/sbin/so-curator-close > /opt/so/log/curator/cron-close.log 2>&1
- user: root
- minute: '*'
- hour: '*'
@@ -109,7 +109,7 @@ so-curatorclosecron:
so-curatordeletecron:
cron.present:
- name: /usr/sbin/so-curator-delete
- name: /usr/sbin/so-curator-delete > /opt/so/log/curator/cron-delete.log 2>&1
- user: root
- minute: '*'
- hour: '*'

2
salt/elastalert/files/rules/so/nids2hive.yaml
View File

@@ -40,7 +40,7 @@ hive_alert_config:
title: '{match[rule][name]}'
type: 'NIDS'
source: 'SecurityOnion'
description: "`Hunting Pivot:` \n\n <https://{{MASTER}}/#/hunt?q=event.module%3A%20suricata%20AND%20rule.uuid%3A{match[rule][uuid]}%20%7C%20groupby%20source.ip%20destination.ip%20rule.name> \n\n `Kibana Dashboard:` \n\n <https://{{MASTER}}/kibana/app/kibana#/dashboard/ed6f7e20-e060-11e9-8f0c-2ddbf5ed9290?_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-24h,mode:quick,to:now))&_a=(columns:!(_source),index:'*:logstash-*',interval:auto,query:(query_string:(analyze_wildcard:!t,query:'sid:')),sort:!('@timestamp',desc))> \n\n `IPs: `{match[source][ip]}:{match[source][port]} --> {match[destination][ip]}:{match[destination][port]} \n\n `Signature:`{match[rule][rule]}"
description: "`Hunting Pivot:` \n\n <https://{{MASTER}}/#/hunt?q=event.module%3A%20suricata%20AND%20rule.uuid%3A{match[rule][uuid]}%20%7C%20groupby%20source.ip%20destination.ip%20rule.name> \n\n `Kibana Dashboard - Signature Drilldown:` \n\n <https://{{MASTER}}/kibana/app/kibana#/dashboard/ed6f7e20-e060-11e9-8f0c-2ddbf5ed9290?_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-24h,mode:quick,to:now))&_a=(columns:!(_source),index:'*:logstash-*',interval:auto,query:(query_string:(analyze_wildcard:!t,query:'sid:')),sort:!('@timestamp',desc))> \n\n `Kibana Dashboard - Community_ID:` \n\n <https://{{MASTER}}/kibana/app/kibana#/dashboard/30d0ac90-729f-11ea-8dd2-9d8795a1200b?_g=(filters:!(('$state':(store:globalState),meta:(alias:!n,disabled:!f,index:'*:so-*',key:network.community_id,negate:!f,params:(query:'{match[network][community_id]}'),type:phrase),query:(match_phrase:(network.community_id:'{match[network][community_id]}')))),refreshInterval:(pause:!t,value:0),time:(from:now-7d,to:now))> \n\n `IPs: `{match[source][ip]}:{match[source][port]} --> {match[destination][ip]}:{match[destination][port]} \n\n `Signature:`{match[rule][rule]}"
severity: 2
tags: ['{match[rule][uuid]}','{match[source][ip]}','{match[destination][ip]}']
tlp: 3

4
salt/elasticsearch/files/elasticsearch.yml
View File

@@ -22,3 +22,7 @@ transport.bind_host: 0.0.0.0
transport.publish_host: {{ nodeip }}
transport.publish_port: 9300
{%- endif %}
cluster.routing.allocation.disk.threshold_enabled: true
cluster.routing.allocation.disk.watermark.low: 95%
cluster.routing.allocation.disk.watermark.high: 98%
cluster.routing.allocation.disk.watermark.flood_stage: 98%

35
salt/elasticsearch/files/ingest/beats.common Normal file
View File

@@ -0,0 +1,35 @@
{
"description" : "beats.common",
"processors" : [
{"community_id": {"if": "ctx.winlog.event_data?.Protocol != null", "field":["winlog.event_data.SourceIp","winlog.event_data.SourcePort","winlog.event_data.DestinationIp","winlog.event_data.DestinationPort","winlog.event_data.Protocol"],"target_field":"network.community_id"}},
{ "set": { "if": "ctx.winlog?.channel != null", "field": "dataset", "value": "wel-{{winlog.channel}}", "override": true } },
{ "set": { "if": "ctx.agent?.type != null", "field": "module", "value": "{{agent.type}}", "override": true } },
{ "set": { "if": "ctx.winlog?.channel == 'Microsoft-Windows-Sysmon/Operational' && ctx.event?.code == 3", "field": "event.category", "value": "host,process,network", "override": true } },
{ "set": { "if": "ctx.winlog?.channel == 'Microsoft-Windows-Sysmon/Operational' && ctx.event?.code == 1", "field": "event.category", "value": "host,process", "override": true } },
{ "rename": { "field": "agent.hostname", "target_field": "agent.name", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.SubjectUserName", "target_field": "user.name", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.DestinationHostname", "target_field": "destination.hostname", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.DestinationIp", "target_field": "destination.ip", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.DestinationPort", "target_field": "destination.port", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.Image", "target_field": "process.executable", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.ProcessID", "target_field": "process.pid", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.ProcessGuid", "target_field": "process.entity_id", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.CommandLine", "target_field": "process.command_line", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.CurrentDirectory", "target_field": "process.working_directory", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.Description", "target_field": "process.pe.description", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.Product", "target_field": "process.pe.product", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.OriginalFileName", "target_field": "process.pe.original_file_name", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.FileVersion", "target_field": "process.pe.file_version", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.ParentCommandLine", "target_field": "process.parent.command_line", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.ParentImage", "target_field": "process.parent.executable", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.ParentProcessGuid", "target_field": "process.parent.entity_id", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.ParentProcessId", "target_field": "process.ppid", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.Protocol", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.User", "target_field": "user.name", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.SourceHostname", "target_field": "source.hostname", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.SourceIp", "target_field": "source.ip", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.SourcePort", "target_field": "source.port", "ignore_missing": true } },
{ "rename": { "field": "winlog.event_data.targetFilename", "target_field": "file.target", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

5
salt/elasticsearch/files/ingest/common
View File

@@ -38,7 +38,10 @@
{ "rename": { "field": "module", "target_field": "event.module", "ignore_missing": true } },
{ "rename": { "field": "dataset", "target_field": "event.dataset", "ignore_missing": true } },
{ "rename": { "field": "category", "target_field": "event.category", "ignore_missing": true } },
{ "rename": { "field": "message2.community_id", "target_field": "network.community_id", "ignore_missing": true } },
{ "rename": { "field": "message2.community_id", "target_field": "network.community_id", "ignore_failure": true, "ignore_missing": true } },
{ "convert": { "field": "destination.port", "type": "integer", "ignore_failure": true, "ignore_missing": true } },
{ "convert": { "field": "source.port", "type": "integer", "ignore_failure": true, "ignore_missing": true } },
{ "convert": { "field": "log.id.uid", "type": "string", "ignore_failure": true, "ignore_missing": true } },
{
"remove": {
"field": [ "index_name_prefix", "message2", "type" ],

2
salt/elasticsearch/files/ingest/osquery.query_result
View File

@@ -31,7 +31,7 @@
{ "rename": { "field": "message3.columns.remote_port", "target_field": "remote.port", "ignore_missing": true } },
{ "rename": { "field": "message3.columns.process_name", "target_field": "process.name", "ignore_missing": true } },
{ "rename": { "field": "message3.columns.eventid", "target_field": "event.code", "ignore_missing": true } },
{ "set": { "if": "ctx.message3.columns.?data != null", "field": "dataset", "value": "wel-{{message3.columns.source}}", "override": true } },
{ "set": { "if": "ctx.message3.columns?.data != null", "field": "dataset", "value": "wel-{{message3.columns.source}}", "override": true } },
{ "rename": { "field": "message3.columns.winlog.EventData.SubjectUserName", "target_field": "user.name", "ignore_missing": true } },
{ "rename": { "field": "message3.columns.winlog.EventData.destinationHostname", "target_field": "destination.hostname", "ignore_missing": true } },
{ "rename": { "field": "message3.columns.winlog.EventData.destinationIp", "target_field": "destination.ip", "ignore_missing": true } },

5
salt/elasticsearch/files/ingest/suricata.alert
View File

@@ -1,13 +1,12 @@
{
"description" : "suricata.alert",
"processors" : [
{ "json": { "field": "message", "target_field": "message2", "ignore_failure": true } },
{ "rename":{ "field": "message2.comunity_id", "target_field": "network.comunity_id", "ignore_failure": true } },
{ "rename":{ "field": "message2.alert", "target_field": "rule", "ignore_failure": true } },
{ "rename":{ "field": "rule.signature", "target_field": "rule.name", "ignore_failure": true } },
{ "rename":{ "field": "rule.ref", "target_field": "rule.version", "ignore_failure": true } },
{ "rename":{ "field": "rule.signature_id", "target_field": "rule.uuid", "ignore_failure": true } },
{ "rename":{ "field": "rule.signature_id", "target_field": "rule.signature", "ignore_failure": true } },
{ "pipeline": { "name": "suricata.common" } }
{ "rename":{ "field": "message2.payload_printable", "target_field": "network.data.decoded", "ignore_failure": true } },
{ "pipeline": { "name": "common" } }
]
}

9
salt/elasticsearch/files/ingest/suricata.common
View File

@@ -1,15 +1,18 @@
{
"description" : "suricata.common",
"processors" : [
{ "json": { "field": "message", "target_field": "message2", "ignore_failure": true } },
{ "rename":{ "field": "message2.proto", "target_field": "network.transport", "ignore_failure": true } },
{ "rename":{ "field": "message2.flow_id", "target_field": "event.id", "ignore_failure": true } },
{ "rename":{ "field": "message2.flow_id", "target_field": "log.id.uid", "ignore_failure": true } },
{ "rename":{ "field": "message2.src_ip", "target_field": "source.ip", "ignore_failure": true } },
{ "rename":{ "field": "message2.src_port", "target_field": "source.port", "ignore_failure": true } },
{ "rename":{ "field": "message2.dest_ip", "target_field": "destination.ip", "ignore_failure": true } },
{ "rename":{ "field": "message2.dest_port", "target_field": "destination.port", "ignore_failure": true } },
{ "rename": { "field": "message2.community_id", "target_field": "network.community_id", "ignore_missing": true } },
{ "remove":{ "field": "dataset", "ignore_failure": true } },
{ "rename":{ "field": "message2.event_type", "target_field": "dataset", "ignore_failure": true } },
{ "set": { "field": "observer.name", "value": "{{agent.name}}" } },
{ "remove": { "field": ["agent"], "ignore_failure": true } },
{ "pipeline": { "name": "common" } }
{ "remove":{ "field": "agent", "ignore_failure": true } },
{ "pipeline": { "name": "suricata.{{dataset}}" } }
]
}

14
salt/elasticsearch/files/ingest/suricata.dhcp Normal file
View File

@@ -0,0 +1,14 @@
{
"description" : "suricata.dhcp",
"processors" : [
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.app_proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "rename": { "field": "message2.dhcp.assigned_ip", "target_field": "dhcp.assigned_ip", "ignore_missing": true } },
{ "rename": { "field": "message2.dhcp.client_mac", "target_field": "host.mac", "ignore_missing": true } },
{ "rename": { "field": "message2.dhcp.dhcp_type", "target_field": "dhcp.message_types", "ignore_missing": true } },
{ "rename": { "field": "message2.dhcp.assigned_ip", "target_field": "dhcp.assigned_ip", "ignore_missing": true } },
{ "rename": { "field": "message2.dhcp.type", "target_field": "dhcp.type", "ignore_missing": true } },
{ "rename": { "field": "message2.dhcp.id", "target_field": "dhcp.id", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

8
salt/elasticsearch/files/ingest/suricata.dnp3 Normal file
View File

@@ -0,0 +1,8 @@
{
"description" : "suricata.dnp3",
"processors" : [
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.app_proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

14
salt/elasticsearch/files/ingest/suricata.dns Normal file
View File

@@ -0,0 +1,14 @@
{
"description" : "suricata.dns",
"processors" : [
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.app_proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "rename": { "field": "message2.dns.type", "target_field": "dns.type", "ignore_missing": true } },
{ "rename": { "field": "message2.dns.tx_id", "target_field": "dns.id", "ignore_missing": true } },
{ "rename": { "field": "message2.dns.version", "target_field": "dns.version", "ignore_missing": true } },
{ "rename": { "field": "message2.dns.rrname", "target_field": "dns.query.name", "ignore_missing": true } },
{ "rename": { "field": "message2.grouped.A", "target_field": "dns.answers", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

8
salt/elasticsearch/files/ingest/suricata.fileinfo Normal file
View File

@@ -0,0 +1,8 @@
{
"description" : "suricata.fileinfo",
"processors" : [
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.app_proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

14
salt/elasticsearch/files/ingest/suricata.flow Normal file
View File

@@ -0,0 +1,14 @@
{
"description" : "suricata.flow",
"processors" : [
{ "set": { "field": "dataset", "value": "conn" } },
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.app_proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "rename": { "field": "message2.flow.state", "target_field": "connection.state", "ignore_missing": true } },
{ "rename": { "field": "message2.flow.bytes_toclient", "target_field": "server.ip_bytes", "ignore_missing": true } },
{ "rename": { "field": "message2.flow.bytes_toserver", "target_field": "client.ip_bytes", "ignore_missing": true } },
{ "rename": { "field": "message2.flow.start", "target_field": "connection.start", "ignore_missing": true } },
{ "rename": { "field": "message2.flow.end", "target_field": "connection.end", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

14
salt/elasticsearch/files/ingest/suricata.ftp Normal file
View File

@@ -0,0 +1,14 @@
{
"description" : "suricata.ftp",
"processors" : [
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.app_proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "rename": { "field": "message2.ftp.reply", "target_field": "server.reply_message", "ignore_missing": true } },
{ "rename": { "field": "message2.ftp.completion_code", "target_field": "server.reply_code", "ignore_missing": true } },
{ "rename": { "field": "message2.ftp.reply_received", "target_field": "server.reply_received", "ignore_missing": true } },
{ "rename": { "field": "message2.ftp.command", "target_field": "ftp.command", "ignore_missing": true } },
{ "rename": { "field": "message2.ftp.command_data", "target_field": "ftp.command_data", "ignore_missing": true } },
{ "rename": { "field": "message2.ftp.dynamic_port", "target_field": "ftp.data_channel_destination.port", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

17
salt/elasticsearch/files/ingest/suricata.http Normal file
View File

@@ -0,0 +1,17 @@
{
"description" : "suricata.http",
"processors" : [
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.app_proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "rename": { "field": "message2.http.hostname", "target_field": "http.virtual_host", "ignore_missing": true } },
{ "rename": { "field": "message2.http.http_user_agent", "target_field": "http.useragent", "ignore_missing": true } },
{ "rename": { "field": "message2.http.url", "target_field": "http.uri", "ignore_missing": true } },
{ "rename": { "field": "message2.http.http_content_type", "target_field": "file.resp_mime_types", "ignore_missing": true } },
{ "rename": { "field": "message2.http.http_refer", "target_field": "http.referrer", "ignore_missing": true } },
{ "rename": { "field": "message2.http.http_method", "target_field": "http.method", "ignore_missing": true } },
{ "rename": { "field": "message2.http.protocol", "target_field": "http.version", "ignore_missing": true } },
{ "rename": { "field": "message2.http.status", "target_field": "http.status_code", "ignore_missing": true } },
{ "rename": { "field": "message2.http.length", "target_field": "http.request.body.length", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

8
salt/elasticsearch/files/ingest/suricata.ikev2 Normal file
View File

@@ -0,0 +1,8 @@
{
"description" : "suricata.ikev2",
"processors" : [
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.app_proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

8
salt/elasticsearch/files/ingest/suricata.krb5 Normal file
View File

@@ -0,0 +1,8 @@
{
"description" : "suricata.krb5",
"processors" : [
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.app_proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

8
salt/elasticsearch/files/ingest/suricata.nfs Normal file
View File

@@ -0,0 +1,8 @@
{
"description" : "suricata.nfs",
"processors" : [
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.app_proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

8
salt/elasticsearch/files/ingest/suricata.rdp Normal file
View File

@@ -0,0 +1,8 @@
{
"description" : "suricata.rdp",
"processors" : [
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.app_proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

8
salt/elasticsearch/files/ingest/suricata.sip Normal file
View File

@@ -0,0 +1,8 @@
{
"description" : "suricata.sip",
"processors" : [
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.app_proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

8
salt/elasticsearch/files/ingest/suricata.smb Normal file
View File

@@ -0,0 +1,8 @@
{
"description" : "suricata.smb",
"processors" : [
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.app_proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

13
salt/elasticsearch/files/ingest/suricata.smtp Normal file
View File

@@ -0,0 +1,13 @@
{
"description" : "suricata.smtp",
"processors" : [
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.app_proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "rename": { "field": "message2.smtp.helo", "target_field": "smtp.helo", "ignore_missing": true } },
{ "rename": { "field": "message2.email.from", "target_field": "smtp.from", "ignore_missing": true } },
{ "rename": { "field": "message2.email.to", "target_field": "smtp.to", "ignore_missing": true } },
{ "rename": { "field": "message2.email.cc", "target_field": "smtp.cc", "ignore_missing": true } },
{ "rename": { "field": "message2.email.attachment", "target_field": "smtp.attachment", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

10
salt/elasticsearch/files/ingest/suricata.snmp Normal file
View File

@@ -0,0 +1,10 @@
{
"description" : "suricata.snmp",
"processors" : [
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.app_proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "rename": { "field": "message2.snmp.version", "target_field": "snmp.version", "ignore_missing": true } },
{ "rename": { "field": "message2.snmp.community", "target_field": "snmp.community", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

11
salt/elasticsearch/files/ingest/suricata.ssh Normal file
View File

@@ -0,0 +1,11 @@
{
"description" : "suricata.ssh",
"processors" : [
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.app_proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "rename": { "field": "message2.ssh.client.proto_version", "target_field": "ssh.version", "ignore_missing": true } },
{ "rename": { "field": "message2.ssh.client.software_version", "target_field": "ssh.client", "ignore_missing": true } },
{ "rename": { "field": "message2.ssh.server.proto_version", "target_field": "ssh.server", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

8
salt/elasticsearch/files/ingest/suricata.tftp Normal file
View File

@@ -0,0 +1,8 @@
{
"description" : "suricata.tftp",
"processors" : [
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.app_proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

8
salt/elasticsearch/files/ingest/suricata.tls Normal file
View File

@@ -0,0 +1,8 @@
{
"description" : "suricata.tls",
"processors" : [
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.app_proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "pipeline": { "name": "common" } }
]
}

17
salt/elasticsearch/files/ingest/syslog Normal file
View File

@@ -0,0 +1,17 @@
{
"description" : "syslog",
"processors" : [
{
"dissect": {
"field": "message",
"pattern" : "%{message}",
"on_failure": [ { "drop" : { } } ]
},
"remove": {
"field": [ "type", "agent" ],
"ignore_failure": true
}
},
{ "pipeline": { "name": "common" } }
]
}

1
salt/elasticsearch/files/ingest/zeek.common
View File

@@ -7,6 +7,7 @@
{ "dot_expander": { "field": "id.orig_p", "path": "message2", "ignore_failure": true } },
{ "dot_expander": { "field": "id.resp_h", "path": "message2", "ignore_failure": true } },
{ "dot_expander": { "field": "id.resp_p", "path": "message2", "ignore_failure": true } },
{"community_id": {"if": "ctx.network?.transport != null", "field":["message2.id.orig_h","message2.id.orig_p","message2.id.resp_h","message2.id.resp_p","network.transport"],"target_field":"network.community_id"}},
{ "rename": { "field": "message2.id.orig_h", "target_field": "source.ip", "ignore_missing": true } },
{ "rename": { "field": "message2.id.orig_p", "target_field": "source.port", "ignore_missing": true } },
{ "rename": { "field": "message2.id.resp_h", "target_field": "destination.ip", "ignore_missing": true } },

2
salt/elasticsearch/files/ingest/zeek.conn
View File

@@ -17,7 +17,7 @@
{ "rename": { "field": "message2.orig_ip_bytes", "target_field": "client.ip_bytes", "ignore_missing": true } },
{ "rename": { "field": "message2.resp_pkts", "target_field": "server.packets", "ignore_missing": true } },
{ "rename": { "field": "message2.resp_ip_bytes", "target_field": "server.ip_bytes", "ignore_missing": true } },
{ "rename": { "field": "message2.tunnel_parents", "target_field": "connection.tunnel_parents", "ignore_missing": true } },
{ "rename": { "field": "message2.tunnel_parents", "target_field": "log.id.tunnel_parents", "ignore_missing": true } },
{ "rename": { "field": "message2.orig_cc", "target_field": "client.country_code","ignore_missing": true } },
{ "rename": { "field": "message2.resp_cc", "target_field": "server.country_code", "ignore_missing": true } },
{ "rename": { "field": "message2.sensorname", "target_field": "observer.name", "ignore_missing": true } },

1
salt/elasticsearch/files/ingest/zeek.http
View File

@@ -29,6 +29,7 @@
{ "script": { "lang": "painless", "source": "ctx.uri_length = ctx.uri.length()", "ignore_failure": true } },
{ "script": { "lang": "painless", "source": "ctx.useragent_length = ctx.useragent.length()", "ignore_failure": true } },
{ "script": { "lang": "painless", "source": "ctx.virtual_host_length = ctx.virtual_host.length()", "ignore_failure": true } },
{ "set": { "field": "network.transport", "value": "tcp" } },
{ "pipeline": { "name": "zeek.common" } }
]
}

2
salt/elasticsearch/files/ingest/zeek.notice
View File

@@ -6,7 +6,7 @@
{ "rename": { "field": "message2.fuid", "target_field": "log.id.fuid", "ignore_missing": true } },
{ "rename": { "field": "message2.mime", "target_field": "file.mimetype", "ignore_missing": true } },
{ "rename": { "field": "message2.desc", "target_field": "file.description", "ignore_missing": true } },
{ "rename": { "field": "message2.proto", "target_field": "network.protocol", "ignore_missing": true } },
{ "rename": { "field": "message2.proto", "target_field": "network.transport", "ignore_missing": true } },
{ "rename": { "field": "message2.note", "target_field": "notice.note", "ignore_missing": true } },
{ "rename": { "field": "message2.msg", "target_field": "notice.message", "ignore_missing": true } },
{ "rename": { "field": "message2.sub", "target_field": "notice.sub_message", "ignore_missing": true } },

2
salt/elasticsearch/files/ingest/zeek.radius
View File

@@ -5,7 +5,7 @@
{ "json": { "field": "message", "target_field": "message2", "ignore_failure": true } },
{ "rename": { "field": "message2.username", "target_field": "user.name", "ignore_missing": true } },
{ "rename": { "field": "message2.mac", "target_field": "host.mac", "ignore_missing": true } },
{ "rename": { "field": "message2.framed_addr", "target_field": "framed_addr", "ignore_missing": true } },
{ "rename": { "field": "message2.framed_addr", "target_field": "radius.framed_address", "ignore_missing": true } },
{ "rename": { "field": "message2.remote_ip", "target_field": "destination.ip", "ignore_missing": true } },
{ "rename": { "field": "message2.connect_info", "target_field": "radius.connect_info", "ignore_missing": true } },
{ "rename": { "field": "message2.reply_msg", "target_field": "radius.reply_message", "ignore_missing": true } },

1
salt/elasticsearch/files/ingest/zeek.smtp
View File

@@ -25,6 +25,7 @@
{ "rename": { "field": "message2.tls", "target_field": "smtp.tls", "ignore_missing": true } },
{ "rename": { "field": "message2.fuids", "target_field": "log.id.fuids", "ignore_missing": true } },
{ "rename": { "field": "message2.is_webmail", "target_field": "smtp.is_webmail", "ignore_missing": true } },
{ "set": { "field": "network.transport", "value": "tcp" } },
{ "pipeline": { "name": "zeek.common" } }
]
}

2
salt/elasticsearch/files/ingest/zeek.tunnels
View File

@@ -3,7 +3,7 @@
"processors" : [
{ "remove": { "field": ["host"], "ignore_failure": true } },
{ "json": { "field": "message", "target_field": "message2", "ignore_failure": true } },
{ "rename": { "field": "message2.uid", "target_field": "uid", "ignore_missing": true } },
{ "rename": { "field": "message2.uid", "target_field": "log.id.uid", "ignore_missing": true } },
{ "dot_expander": { "field": "id.orig_h", "path": "message2", "ignore_failure": true } },
{ "rename": { "field": "message2.id.orig_h", "target_field": "source.ip", "ignore_missing": true } },
{ "dot_expander": { "field": "id.orig_p", "path": "message2", "ignore_failure": true } },

2
salt/elasticsearch/files/ingest/zeek.x509
View File

@@ -3,7 +3,7 @@
"processors" : [
{ "remove": { "field": ["host"], "ignore_failure": true } },
{ "json": { "field": "message", "target_field": "message2", "ignore_failure": true } },
{ "rename": { "field": "message2.id", "target_field": "id", "ignore_missing": true } },
{ "rename": { "field": "message2.id", "target_field": "log.id.fuid", "ignore_missing": true } },
{ "dot_expander": { "field": "certificate.version", "path": "message2", "ignore_failure": true } },
{ "rename": { "field": "message2.certificate.version", "target_field": "x509.certificate.version", "ignore_missing": true } },
{ "dot_expander": { "field": "certificate.serial", "path": "message2", "ignore_failure": true } },

5
salt/elasticsearch/files/so-elasticsearch-pipelines
View File

@@ -15,6 +15,7 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
RETURN_CODE=0
ELASTICSEARCH_HOST=$1
ELASTICSEARCH_PORT=9200
@@ -46,7 +47,9 @@ fi
cd ${ELASTICSEARCH_INGEST_PIPELINES}
echo "Loading pipelines..."
for i in *; do echo $i; curl ${ELASTICSEARCH_AUTH} -XPUT http://${ELASTICSEARCH_HOST}:${ELASTICSEARCH_PORT}/_ingest/pipeline/$i -H 'Content-Type: application/json' -d@$i 2>/dev/null; echo; done
for i in *; do echo $i; RESPONSE=$(curl ${ELASTICSEARCH_AUTH} -XPUT http://${ELASTICSEARCH_HOST}:${ELASTICSEARCH_PORT}/_ingest/pipeline/$i -H 'Content-Type: application/json' -d@$i 2>/dev/null); echo $RESPONSE; if [[ "$RESPONSE" == *"error"* ]]; then RETURN_CODE=1; fi; done
echo
cd - >/dev/null
exit $RETURN_CODE

30
salt/filebeat/etc/filebeat.yml
View File

@@ -75,6 +75,32 @@ filebeat.modules:
filebeat.inputs:
#------------------------------ Log prospector --------------------------------
{%- if grains['role'] == 'so-sensor' or grains['role'] == "so-eval" or grains['role'] == "so-helix" or grains['role'] == "so-heavynode" or grains['role'] == "so-standalone" %}
- type: udp
enabled: true
host: "0.0.0.0:514"
fields:
module: syslog
dataset: syslog
pipeline: "syslog"
index: "so-syslog-%{+yyyy.MM.dd}"
processors:
- drop_fields:
fields: ["source", "prospector", "input", "offset", "beat"]
fields_under_root: true
- type: tcp
enabled: true
host: "0.0.0.0:514"
fields:
module: syslog
dataset: syslog
pipeline: "syslog"
index: "so-syslog-%{+yyyy.MM.dd}"
processors:
- drop_fields:
fields: ["source", "prospector", "input", "offset", "beat"]
fields_under_root: true
{%- if BROVER != 'SURICATA' %}
{%- for LOGNAME in salt['pillar.get']('brologs:enabled', '') %}
- type: log
@@ -97,10 +123,10 @@ filebeat.inputs:
- type: log
paths:
- /suricata/eve.json
- /suricata/eve*.json
fields:
module: suricata
dataset: alert
dataset: common
category: network
processors:

4
salt/filebeat/init.sls
View File

@@ -57,12 +57,14 @@ so-filebeat:
- /opt/so/conf/filebeat/etc/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro
- /nsm/zeek:/nsm/zeek:ro
- /nsm/strelka/log:/nsm/strelka/log:ro
- /opt/so/log/suricata:/suricata:ro
- /nsm/suricata:/suricata:ro
- /opt/so/wazuh/logs/alerts:/wazuh/alerts:ro
- /opt/so/wazuh/logs/archives:/wazuh/archives:ro
- /nsm/osquery/fleet/:/nsm/osquery/fleet:ro
- /opt/so/conf/filebeat/etc/pki/filebeat.crt:/usr/share/filebeat/filebeat.crt:ro
- /opt/so/conf/filebeat/etc/pki/filebeat.key:/usr/share/filebeat/filebeat.key:ro
- /etc/ssl/certs/intca.crt:/usr/share/filebeat/intraca.crt:ro
- port_bindings:
- 0.0.0.0:514:514/udp
- watch:
- file: /opt/so/conf/filebeat/etc/filebeat.yml

469
salt/firewall/assigned_hostgroups.map.yaml Normal file
View File

@@ -0,0 +1,469 @@
{% import_yaml 'firewall/portgroups.yaml' as portgroups %}
{% set portgroups = portgroups.firewall.aliases.ports %}
role:
eval:
chain:
DOCKER-USER:
hostgroups:
master:
portgroups:
- {{ portgroups.wazuh_agent }}
- {{ portgroups.wazuh_api }}
- {{ portgroups.wazuh_authd }}
- {{ portgroups.playbook }}
- {{ portgroups.mysql }}
- {{ portgroups.navigator }}
- {{ portgroups.kibana }}
- {{ portgroups.redis }}
- {{ portgroups.influxdb }}
- {{ portgroups.fleet_api }}
- {{ portgroups.cortex }}
- {{ portgroups.elasticsearch_rest }}
- {{ portgroups.elasticsearch_node }}
- {{ portgroups.cortex_es_rest }}
- {{ portgroups.cortex_es_node }}
minion:
portgroups:
- {{ portgroups.acng }}
- {{ portgroups.docker_registry }}
- {{ portgroups.osquery_8080 }}
- {{ portgroups.influxdb }}
- {{ portgroups.wazuh_api }}
- {{ portgroups.fleet_api }}
sensor:
portgroups:
- {{ portgroups.sensoroni }}
- {{ portgroups.beats_5044 }}
- {{ portgroups.beats_5644 }}
search_node:
portgroups:
- {{ portgroups.redis }}
- {{ portgroups.elasticsearch_node }}
self:
portgroups:
- {{ portgroups.syslog}}
beats_endpoint:
portgroups:
- {{ portgroups.beats_5044 }}
beats_endpoint_ssl:
portgroups:
- {{ portgroups.beats_5644 }}
osquery_endpoint:
portgroups:
- {{ portgroups.fleet_api }}
syslog:
portgroups:
- {{ portgroups.syslog }}
wazuh_agent:
portgroups:
- {{ portgroups.wazuh_agent }}
wazuh_api:
portgroups:
- {{ portgroups.wazuh_api }}
wazuh_authd:
portgroups:
- {{ portgroups.wazuh_authd }}
analyst:
portgroups:
- {{ portgroups.nginx }}
INPUT:
hostgroups:
anywhere:
portgroups:
- {{ portgroups.ssh }}
dockernet:
portgroups:
- {{ portgroups.all }}
localhost:
portgroups:
- {{ portgroups.all }}
minion:
portgroups:
- {{ portgroups.salt_master }}
master:
chain:
DOCKER-USER:
hostgroups:
master:
portgroups:
- {{ portgroups.wazuh_agent }}
- {{ portgroups.wazuh_api }}
- {{ portgroups.wazuh_authd }}
- {{ portgroups.playbook }}
- {{ portgroups.mysql }}
- {{ portgroups.navigator }}
- {{ portgroups.kibana }}
- {{ portgroups.redis }}
- {{ portgroups.influxdb }}
- {{ portgroups.fleet_api }}
- {{ portgroups.cortex }}
- {{ portgroups.elasticsearch_rest }}
- {{ portgroups.elasticsearch_node }}
- {{ portgroups.cortex_es_rest }}
- {{ portgroups.cortex_es_node }}
minion:
portgroups:
- {{ portgroups.acng }}
- {{ portgroups.docker_registry }}
- {{ portgroups.osquery_8080 }}
- {{ portgroups.influxdb }}
- {{ portgroups.wazuh_api }}
- {{ portgroups.fleet_api }}
sensor:
portgroups:
- {{ portgroups.sensoroni }}
- {{ portgroups.beats_5044 }}
- {{ portgroups.beats_5644 }}
search_node:
portgroups:
- {{ portgroups.redis }}
- {{ portgroups.elasticsearch_node }}
self:
portgroups:
- {{ portgroups.syslog}}
syslog:
portgroups:
- {{ portgroups.syslog }}
beats_endpoint:
portgroups:
- {{ portgroups.beats_5044 }}
beats_endpoint_ssl:
portgroups:
- {{ portgroups.beats_5644 }}
osquery_endpoint:
portgroups:
- {{ portgroups.fleet_api }}
wazuh_agent:
portgroups:
- {{ portgroups.wazuh_agent }}
wazuh_api:
portgroups:
- {{ portgroups.wazuh_api }}
wazuh_authd:
portgroups:
- {{ portgroups.wazuh_authd }}
analyst:
portgroups:
- {{ portgroups.nginx }}
INPUT:
hostgroups:
anywhere:
portgroups:
- {{ portgroups.ssh }}
dockernet:
portgroups:
- {{ portgroups.all }}
localhost:
portgroups:
- {{ portgroups.all }}
minion:
portgroups:
- {{ portgroups.salt_master }}
mastersearch:
chain:
DOCKER-USER:
hostgroups:
master:
portgroups:
- {{ portgroups.wazuh_agent }}
- {{ portgroups.wazuh_api }}
- {{ portgroups.wazuh_authd }}
- {{ portgroups.playbook }}
- {{ portgroups.mysql }}
- {{ portgroups.navigator }}
- {{ portgroups.kibana }}
- {{ portgroups.redis }}
- {{ portgroups.influxdb }}
- {{ portgroups.fleet_api }}
- {{ portgroups.cortex }}
- {{ portgroups.elasticsearch_rest }}
- {{ portgroups.elasticsearch_node }}
- {{ portgroups.cortex_es_rest }}
- {{ portgroups.cortex_es_node }}
minion:
portgroups:
- {{ portgroups.acng }}
- {{ portgroups.docker_registry }}
- {{ portgroups.osquery_8080 }}
- {{ portgroups.influxdb }}
- {{ portgroups.wazuh_api }}
- {{ portgroups.fleet_api }}
sensor:
portgroups:
- {{ portgroups.sensoroni }}
- {{ portgroups.beats_5044 }}
- {{ portgroups.beats_5644 }}
search_node:
portgroups:
- {{ portgroups.redis }}
- {{ portgroups.elasticsearch_node }}
self:
portgroups:
- {{ portgroups.syslog}}
beats_endpoint:
portgroups:
- {{ portgroups.beats_5044 }}
beats_endpoint_ssl:
portgroups:
- {{ portgroups.beats_5644 }}
osquery_endpoint:
portgroups:
- {{ portgroups.fleet_api }}
syslog:
portgroups:
- {{ portgroups.syslog }}
wazuh_agent:
portgroups:
- {{ portgroups.wazuh_agent }}
wazuh_api:
portgroups:
- {{ portgroups.wazuh_api }}
wazuh_authd:
portgroups:
- {{ portgroups.wazuh_authd }}
analyst:
portgroups:
- {{ portgroups.nginx }}
INPUT:
hostgroups:
anywhere:
portgroups:
- {{ portgroups.ssh }}
dockernet:
portgroups:
- {{ portgroups.all }}
localhost:
portgroups:
- {{ portgroups.all }}
minion:
portgroups:
- {{ portgroups.salt_master }}
standalone:
chain:
DOCKER-USER:
hostgroups:
master:
portgroups:
- {{ portgroups.wazuh_agent }}
- {{ portgroups.wazuh_api }}
- {{ portgroups.wazuh_authd }}
- {{ portgroups.playbook }}
- {{ portgroups.mysql }}
- {{ portgroups.navigator }}
- {{ portgroups.kibana }}
- {{ portgroups.redis }}
- {{ portgroups.influxdb }}
- {{ portgroups.fleet_api }}
- {{ portgroups.cortex }}
- {{ portgroups.elasticsearch_rest }}
- {{ portgroups.elasticsearch_node }}
- {{ portgroups.cortex_es_rest }}
- {{ portgroups.cortex_es_node }}
minion:
portgroups:
- {{ portgroups.acng }}
- {{ portgroups.docker_registry }}
- {{ portgroups.osquery_8080 }}
- {{ portgroups.influxdb }}
- {{ portgroups.wazuh_api }}
- {{ portgroups.fleet_api }}
sensor:
portgroups:
- {{ portgroups.sensoroni }}
- {{ portgroups.beats_5044 }}
- {{ portgroups.beats_5644 }}
search_node:
portgroups:
- {{ portgroups.redis }}
- {{ portgroups.elasticsearch_node }}
self:
portgroups:
- {{ portgroups.syslog}}
beats_endpoint:
portgroups:
- {{ portgroups.beats_5044 }}
beats_endpoint_ssl:
portgroups:
- {{ portgroups.beats_5644 }}
osquery_endpoint:
portgroups:
- {{ portgroups.fleet_api }}
syslog:
portgroups:
- {{ portgroups.syslog }}
wazuh_agent:
portgroups:
- {{ portgroups.wazuh_agent }}
wazuh_api:
portgroups:
- {{ portgroups.wazuh_api }}
wazuh_authd:
portgroups:
- {{ portgroups.wazuh_authd }}
analyst:
portgroups:
- {{ portgroups.nginx }}
INPUT:
hostgroups:
anywhere:
portgroups:
- {{ portgroups.ssh }}
dockernet:
portgroups:
- {{ portgroups.all }}
localhost:
portgroups:
- {{ portgroups.all }}
minion:
portgroups:
- {{ portgroups.salt_master }}
helixsensor:
chain:
DOCKER-USER:
hostgroups:
master:
portgroups:
- {{ portgroups.wazuh_agent }}
- {{ portgroups.playbook }}
- {{ portgroups.mysql }}
- {{ portgroups.navigator }}
- {{ portgroups.kibana }}
- {{ portgroups.redis }}
- {{ portgroups.influxdb }}
- {{ portgroups.fleet_api }}
- {{ portgroups.cortex }}
- {{ portgroups.elasticsearch_rest }}
- {{ portgroups.elasticsearch_node }}
- {{ portgroups.cortex_es_rest }}
- {{ portgroups.cortex_es_node }}
minion:
portgroups:
- {{ portgroups.acng }}
- {{ portgroups.docker_registry }}
- {{ portgroups.osquery_8080 }}
- {{ portgroups.influxdb }}
- {{ portgroups.wazuh_api }}
sensor:
portgroups:
- {{ portgroups.sensoroni }}
- {{ portgroups.beats_5044 }}
- {{ portgroups.beats_5644 }}
search_node:
portgroups:
- {{ portgroups.redis }}
- {{ portgroups.elasticsearch_node }}
self:
portgroups:
- {{ portgroups.syslog}}
beats_endpoint:
portgroups:
- {{ portgroups.beats_5044 }}
osquery_endpoint:
portgroups:
- {{ portgroups.fleet_api }}
wazuh_agent:
portgroups:
- {{ portgroups.wazuh_agent }}
analyst:
portgroups:
- {{ portgroups.nginx }}
INPUT:
hostgroups:
anywhere:
portgroups:
- {{ portgroups.ssh }}
dockernet:
portgroups:
- {{ portgroups.all }}
localhost:
portgroups:
- {{ portgroups.all }}
minion:
portgroups:
- {{ portgroups.salt_master }}
searchnode:
chain:
DOCKER-USER:
hostgroups:
master:
portgroups:
- {{ portgroups.elasticsearch_node }}
dockernet:
portgroups:
- {{ portgroups.elasticsearch_node }}
- {{ portgroups.elasticsearch_rest }}
INPUT:
hostgroups:
anywhere:
portgroups:
- {{ portgroups.ssh }}
dockernet:
portgroups:
- {{ portgroups.all }}
localhost:
portgroups:
- {{ portgroups.all }}
sensor:
chain:
INPUT:
hostgroups:
anywhere:
portgroups:
- {{ portgroups.ssh }}
dockernet:
portgroups:
- {{ portgroups.all }}
localhost:
portgroups:
- {{ portgroups.all }}
heavynode:
chain:
DOCKER-USER:
hostgroups:
self:
portgroups:
- {{ portgroups.redis }}
- {{ portgroups.beats_5044 }}
- {{ portgroups.beats_5644 }}
INPUT:
hostgroups:
anywhere:
portgroups:
- {{ portgroups.ssh }}
localhost:
portgroups:
- {{ portgroups.all }}
fleet:
chain:
DOCKER-USER:
hostgroups:
self:
portgroups:
- {{ portgroups.redis }}
- {{ portgroups.mysql }}
- {{ portgroups.osquery_8080 }}
localhost:
portgroups:
- {{ portgroups.mysql }}
- {{ portgroups.osquery_8080 }}
analyst:
portgroups:
- {{ portgroups.fleet_webui }}
minion:
portgroups:
- {{ portgroups.fleet_api }}
osquery_endpoint:
portgroups:
- {{ portgroups.fleet_api}}
INPUT:
hostgroups:
anywhere:
portgroups:
- {{ portgroups.ssh }}
dockernet:
portgroups:
- {{ portgroups.all }}
localhost:
portgroups:
- {{ portgroups.all }}

22
salt/firewall/hostgroups.yaml Normal file
View File

@@ -0,0 +1,22 @@
firewall:
hostgroups:
anywhere:
ips:
delete:
insert:
- 0.0.0.0/0
dockernet:
ips:
delete:
insert:
- 172.17.0.0/24
localhost:
ips:
delete:
insert:
- 127.0.0.1
self:
ips:
delete:
insert:
- {{ salt['grains.get']('ip_interfaces').get(salt['pillar.get']('sensor:mainint', salt['pillar.get']('master:mainint', salt['pillar.get']('node:mainint', salt['pillar.get']('host:mainint')))))[0] }}

779
salt/firewall/init.sls
View File

@@ -1,16 +1,12 @@
# Firewall Magic for the grid
{% if grains['role'] in ['so-eval','so-master','so-helix','so-mastersearch', 'so-standalone'] %}
{% set ip = salt['pillar.get']('static:masterip', '') %}
{% elif grains['role'] == 'so-node' or grains['role'] == 'so-heavynode' %}
{% set ip = salt['pillar.get']('node:mainip', '') %}
{% elif grains['role'] == 'so-sensor' %}
{% set ip = salt['pillar.get']('sensor:mainip', '') %}
{% elif grains['role'] == 'so-fleet' %}
{% set ip = salt['pillar.get']('node:mainip', '') %}
{% endif %}
{% from 'firewall/map.jinja' import hostgroups with context %}
{% from 'firewall/map.jinja' import assigned_hostgroups with context %}
{% set FLEET_NODE = salt['pillar.get']('static:fleet_node') %}
{% set FLEET_NODE_IP = salt['pillar.get']('static:fleet_ip') %}
create_sysconfig_iptables:
file.touch:
- name: /etc/sysconfig/iptables
- makedirs: True
- unless: 'ls /etc/sysconfig/iptables'
# Quick Fix for Docker being difficult
iptables_fix_docker:
@@ -27,15 +23,6 @@ iptables_fix_fwd:
- position: 1
- target: DOCKER-USER
# Keep localhost in the game
iptables_allow_localhost:
iptables.append:
- table: filter
- chain: INPUT
- jump: ACCEPT
- source: 127.0.0.1
- save: True
# Allow related/established sessions
iptables_allow_established:
iptables.append:
@@ -46,16 +33,6 @@ iptables_allow_established:
- ctstate: 'RELATED,ESTABLISHED'
- save: True
# Always allow SSH so we can like log in
iptables_allow_ssh:
iptables.append:
- table: filter
- chain: INPUT
- jump: ACCEPT
- dport: 22
- proto: tcp
- save: True
# I like pings
iptables_allow_pings:
iptables.append:
@@ -113,592 +90,37 @@ enable_docker_user_established:
- match: conntrack
- ctstate: 'RELATED,ESTABLISHED'
# Add rule(s) for Wazuh manager
enable_wazuh_manager_1514_tcp_{{ip}}:
iptables.insert:
{% for chain, hg in assigned_hostgroups.chain.items() %}
{% for hostgroup, portgroups in assigned_hostgroups.chain[chain].hostgroups.items() %}
{% for action in ['insert', 'delete' ] %}
{% if hostgroups[hostgroup].ips[action] %}
{% for ip in hostgroups[hostgroup].ips[action] %}
{% for portgroup in portgroups.portgroups %}
{% for proto, ports in portgroup.items() %}
{% for port in ports %}
{{action}}_{{chain}}_{{hostgroup}}_{{ip}}_{{port}}_{{proto}}:
iptables.{{action}}:
- table: filter
- chain: DOCKER-USER
- chain: {{ chain }}
- jump: ACCEPT
- proto: tcp
- proto: {{ proto }}
- source: {{ ip }}
- dport: 1514
- dport: {{ port }}
{% if action == 'insert' %}
- position: 1
{% endif %}
- save: True
enable_wazuh_manager_1514_udp_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: udp
- source: {{ ip }}
- dport: 1514
- position: 1
- save: True
# Rules if you are a Master
{% if grains['role'] in ['so-master', 'so-eval', 'so-helix', 'so-mastersearch', 'so-standalone'] %}
#This should be more granular
iptables_allow_master_docker:
iptables.insert:
- table: filter
- chain: INPUT
- jump: ACCEPT
- source: 172.17.0.0/24
- position: 1
- save: True
{% for ip in pillar.get('masterfw') %}
# Allow Redis
enable_maternode_redis_6379_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 6379
- position: 1
- save: True
enable_masternode_kibana_5601_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 5601
- position: 1
- save: True
enable_masternode_ES_9200_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 9200
- position: 1
- save: True
enable_masternode_ES_9300_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 9300
- position: 1
- save: True
enable_masternode_ES_9400_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 9400
- position: 1
- save: True
enable_masternode_ES_9500_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 9500
- position: 1
- save: True
enable_masternode_influxdb_8086_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 8086
- position: 1
- save: True
enable_masternode_mysql_3306_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 3306
- position: 1
- save: True
enable_master_osquery_8090_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 8090
- position: 1
- save: True
enable_master_playbook_3200_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 3200
- position: 1
- save: True
enable_master_navigator_4200_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 4200
- position: 1
- save: True
enable_master_cortex_9001_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 9001
- position: 1
- save: True
enable_master_cyberchef_9080_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 9080
- position: 1
- save: True
{% endfor %}
{% endfor %}
{% endfor %}
{% endfor %}
{% endif %}
{% endfor %}
{% endfor %}
{% endfor %}
# Make it so all the minions can talk to salt and update etc.
{% for ip in pillar.get('minions') %}
enable_salt_minions_4505_{{ip}}:
iptables.insert:
- table: filter
- chain: INPUT
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 4505
- position: 1
- save: True
enable_salt_minions_4506_{{ip}}:
iptables.insert:
- table: filter
- chain: INPUT
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 4506
- position: 1
- save: True
enable_salt_minions_5000_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 5000
- position: 1
- save: True
enable_salt_minions_3142_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 3142
- position: 1
- save: True
enable_minions_influxdb_8086_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 8086
- position: 1
- save: True
enable_minion_osquery_8080_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 8080
- position: 1
- save: True
enable_minion_osquery_8090_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 8090
- position: 1
- save: True
enable_minion_wazuh_55000_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 55000
- position: 1
- save: True
{% endfor %}
# Allow Forward Nodes to send their beats traffic
{% for ip in pillar.get('forward_nodes') %}
enable_forwardnode_beats_5044_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 5044
- position: 1
- save: True
enable_forwardnode_beats_5644_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 5644
- position: 1
- save: True
enable_forwardnode_sensoroni_443_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 443
- position: 1
- save: True
enable_forwardnode_sensoroni_9822_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 9822
- position: 1
- save: True
{% endfor %}
# Allow Fleet Node to send its beats traffic
{% if FLEET_NODE %}
enable_fleetnode_beats_5644_{{FLEET_NODE_IP}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ FLEET_NODE_IP }}
- dport: 5644
- position: 1
- save: True
{% endif %}
{% for ip in pillar.get('search_nodes') %}
enable_searchnode_redis_6379_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 6379
- position: 1
- save: True
enable_searchnode_ES_9300_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 9300
- position: 1
- save: True
{% endfor %}
# Allow Beats Endpoints to send their beats traffic
{% for ip in pillar.get('beats_endpoint') %}
enable_standard_beats_5044_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 5044
- position: 1
- save: True
{% endfor %}
# Allow OSQuery Endpoints to send their traffic
{% for ip in pillar.get('osquery_endpoint') %}
enable_standard_osquery_8090_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 8090
- position: 1
- save: True
{% endfor %}
# Allow Wazuh Endpoints to send their traffic
{% for ip in pillar.get('wazuh_endpoint') %}
enable_wazuh_endpoint_tcp_1514_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 1514
- position: 1
- save: True
enable_wazuh_endpoint_udp_1514_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: udp
- source: {{ ip }}
- dport: 1514
- position: 1
- save: True
{% endfor %}
# Allow Analysts
{% for ip in pillar.get('analyst') %}
enable_standard_analyst_80_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 80
- position: 1
- save: True
enable_standard_analyst_443_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 443
- position: 1
- save: True
#enable_standard_analyst_3000_{{ip}}:
# iptables.insert:
# - table: filter
# - chain: DOCKER-USER
# - jump: ACCEPT
# - proto: tcp
# - source: {{ ip }}
# - dport: 3000
# - position: 1
# - save: True
#enable_standard_analyst_7000_{{ip}}:
# iptables.insert:
# - table: filter
# - chain: DOCKER-USER
# - jump: ACCEPT
# - proto: tcp
# - source: {{ ip }}
# - dport: 7000
# - position: 1
# - save: True
#enable_standard_analyst_9000_{{ip}}:
# iptables.insert:
# - table: filter
# - chain: DOCKER-USER
# - jump: ACCEPT
# - proto: tcp
# - source: {{ ip }}
# - dport: 9000
# - position: 1
# - save: True
#enable_standard_analyst_9001_{{ip}}:
# iptables.insert:
# - table: filter
# - chain: DOCKER-USER
# - jump: ACCEPT
# - proto: tcp
# - source: {{ ip }}
# - dport: 9001
# - position: 1
# - save: True
# This is temporary for sensoroni testing
#enable_standard_analyst_9822_{{ip}}:
# iptables.insert:
# - table: filter
# - chain: DOCKER-USER
# - jump: ACCEPT
# - proto: tcp
# - source: {{ ip }}
# - dport: 9822
# - position: 1
# - save: True
{% endfor %}
# Rules for search nodes connecting to master
{% endif %}
# Rules if you are a Node
{% if 'node' in grains['role'] %}
#This should be more granular
iptables_allow_docker:
iptables.insert:
- table: filter
- chain: INPUT
- jump: ACCEPT
- source: 172.17.0.0/24
- position: 1
- save: True
enable_docker_ES_9200:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: 172.17.0.0/24
- dport: 9200
- position: 1
- save: True
enable_docker_ES_9300:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: 172.17.0.0/24
- dport: 9300
- position: 1
- save: True
{% for ip in pillar.get('masterfw') %}
enable_cluster_ES_9300_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 9300
- position: 1
- save: True
{% endfor %}
{% endif %}
# Rules if you are a Sensor
{% if grains['role'] == 'so-sensor' %}
iptables_allow_sensor_docker:
iptables.insert:
- table: filter
- chain: INPUT
- jump: ACCEPT
- source: 172.17.0.0/24
- position: 1
- save: True
{% endif %}
# Rules if you are a Hot Node
# Rules if you are a Warm Node
# Some Fixer upper type rules
# Drop it like it's hot
# Make the input policy send stuff that doesn't match to be logged and dropped
iptables_drop_all_the_things:
iptables.append:
@@ -706,144 +128,3 @@ iptables_drop_all_the_things:
- chain: LOGGING
- jump: DROP
- save: True
{% if grains['role'] == 'so-heavynode' %}
# Allow Redis
enable_heavynode_redis_6379_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 6379
- position: 1
- save: True
enable_forwardnode_beats_5044_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 5044
- position: 1
- save: True
enable_forwardnode_beats_5644_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 5644
- position: 1
- save: True
{% endif %}
# Rules if you are a Standalone Fleet node
{% if grains['role'] == 'so-fleet' %}
#This should be more granular
iptables_allow_fleetnode_docker:
iptables.insert:
- table: filter
- chain: INPUT
- jump: ACCEPT
- source: 172.17.0.0/24
- position: 1
- save: True
# Allow Redis
enable_fleetnode_redis_6379_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 6379
- position: 1
- save: True
enable_fleetnode_mysql_3306_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 3306
- position: 1
- save: True
enable_fleet_osquery_8080_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 8080
- position: 1
- save: True
enable_fleetnodetemp_mysql_3306_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: 127.0.0.1
- dport: 3306
- position: 1
- save: True
enable_fleettemp_osquery_8080_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: 127.0.0.1
- dport: 8080
- position: 1
- save: True
# Allow Analysts to access Fleet WebUI
{% for ip in pillar.get('analyst') %}
enable_fleetnode_fleet_443_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 443
- position: 1
- save: True
{% endfor %}
# Needed for osquery endpoints to checkin to Fleet API for mgt
{% for ip in pillar.get('osquery_endpoint') %}
enable_fleetnode_8090_{{ip}}:
iptables.insert:
- table: filter
- chain: DOCKER-USER
- jump: ACCEPT
- proto: tcp
- source: {{ ip }}
- dport: 8090
- position: 1
- save: True
{% endfor %}
{% endif %}

45
salt/firewall/map.jinja Normal file
View File

@@ -0,0 +1,45 @@
{% set role = grains.id.split('_') | last %}
{% set translated_pillar_assigned_hostgroups = {} %}
{% import_yaml 'firewall/portgroups.yaml' as default_portgroups %}
{% set default_portgroups = default_portgroups.firewall.aliases.ports %}
{% import_yaml 'firewall/portgroups.local.yaml' as local_portgroups %}
{% if local_portgroups.firewall.aliases.ports %}
{% set local_portgroups = local_portgroups.firewall.aliases.ports %}
{% else %}
{% set local_portgroups = {} %}
{% endif %}
{% set portgroups = salt['defaults.merge'](default_portgroups, local_portgroups, in_place=False) %}
{% set defined_portgroups = portgroups %}
{% import_yaml 'firewall/hostgroups.yaml' as default_hostgroups %}
{% import_yaml 'firewall/hostgroups.local.yaml' as local_hostgroups %}
{% set hostgroups = salt['defaults.merge'](default_hostgroups.firewall.hostgroups, local_hostgroups.firewall.hostgroups, in_place=False) %}
{# This block translate the portgroups defined in the pillar to what is defined my portgroups.yaml and portgroups.local.yaml #}
{% if salt['pillar.get']('firewall:assigned_hostgroups:chain') %}
{% for chain, hg in salt['pillar.get']('firewall:assigned_hostgroups:chain').items() %}
{% for pillar_hostgroup, pillar_portgroups in salt['pillar.get']('firewall:assigned_hostgroups:chain')[chain].hostgroups.items() %}
{% do translated_pillar_assigned_hostgroups.update({"chain": {chain: {"hostgroups": {pillar_hostgroup: {"portgroups": []}}}}}) %}
{% for pillar_portgroup in pillar_portgroups.portgroups %}
{% set pillar_portgroup = pillar_portgroup.split('.') | last %}
{% do translated_pillar_assigned_hostgroups.chain[chain].hostgroups[pillar_hostgroup].portgroups.append(defined_portgroups[pillar_portgroup]) %}
{% endfor %}
{% endfor %}
{% endfor %}
{% endif %}
{% import_yaml 'firewall/assigned_hostgroups.map.yaml' as default_assigned_hostgroups %}
{% import_yaml 'firewall/assigned_hostgroups.local.map.yaml' as local_assigned_hostgroups %}
{% if local_assigned_hostgroups.role[role] %}
{% set assigned_hostgroups = salt['defaults.merge'](local_assigned_hostgroups.role[role], default_assigned_hostgroups.role[role], merge_lists=False, in_place=False) %}
{% else %}
{% set assigned_hostgroups = default_assigned_hostgroups.role[role] %}
{% endif %}
{% if translated_pillar_assigned_hostgroups %}
{% do salt['defaults.merge'](assigned_hostgroups, translated_pillar_assigned_hostgroups, merge_lists=True, in_place=True) %}
{% endif %}

92
salt/firewall/portgroups.yaml Normal file
View File

@@ -0,0 +1,92 @@
firewall:
aliases:
ports:
all:
tcp:
- '0:65535'
udp:
- '0:65535'
acng:
tcp:
- 3142
beats_5044:
tcp:
- 5044
beats_5644:
tcp:
- 5644
cortex:
tcp:
- 9001
cortex_es_node:
tcp:
- 9500
cortex_es_rest:
tcp:
- 9400
docker_registry:
tcp:
- 5000
elasticsearch_node:
tcp:
- 9300
elasticsearch_rest:
tcp:
- 9200
fleet_api:
tcp:
- 8090
fleet_webui:
tcp:
- 443
influxdb:
tcp:
- 8086
kibana:
tcp:
- 5601
mysql:
tcp:
- 3306
navigator:
tcp:
- 4200
nginx:
tcp:
- 80
- 443
osquery_8080:
tcp:
- 8080
playbook:
tcp:
- 3200
redis:
tcp:
- 6379
salt_master:
tcp:
- 4505
- 4506
sensoroni:
tcp:
- 443
ssh:
tcp:
- 22
syslog:
tcp:
- 514
udp:
- 514
wazuh_agent:
tcp:
- 1514
udp:
- 1514
wazuh_api:
tcp:
- 55000
wazuh_authd:
tcp:
- 1515

3
salt/fleet/event_enable-fleet.sls
View File

@@ -1,5 +1,6 @@
{% set ENROLLSECRET = salt['cmd.run']('docker exec so-fleet fleetctl get enroll-secret') %}
{%- set MAINIP = salt['pillar.get']('node:mainip') -%}
{% set MAININT = salt['pillar.get']('host:mainint') %}
{% set MAINIP = salt['grains.get']('ip_interfaces').get(MAININT)[0] %}
so/fleet:
event.send:

11
salt/fleet/event_gen-packages.sls
View File

@@ -1,15 +1,24 @@
{% set MASTER = salt['grains.get']('master') %}
{% set ENROLLSECRET = salt['pillar.get']('secrets:fleet_enroll-secret') %}
{% set CURRENTPACKAGEVERSION = salt['pillar.get']('static:fleet_packages-version') %}
{% set VERSION = salt['pillar.get']('static:soversion') %}
{% set CUSTOM_FLEET_HOSTNAME = salt['pillar.get']('static:fleet_custom_hostname', None) %}
{% if CUSTOM_FLEET_HOSTNAME != None and CUSTOM_FLEET_HOSTNAME != '' %}
{% set HOSTNAME = CUSTOM_FLEET_HOSTNAME %}
{% else %}
{% set HOSTNAME = grains.host %}
{% endif %}
so/fleet:
event.send:
- data:
action: 'genpackages'
hostname: {{ grains.host }}
package-hostname: {{ HOSTNAME }}
role: {{ grains.role }}
mainip: {{ grains.host }}
enroll-secret: {{ ENROLLSECRET }}
current-package-version: {{ CURRENTPACKAGEVERSION }}
master: {{ MASTER }}
version: {{ VERSION }}

9
salt/fleet/event_update-custom-hostname.sls Normal file
View File

@@ -0,0 +1,9 @@
{% set CUSTOM_FLEET_HOSTNAME = salt['pillar.get']('static:fleet_custom_hostname', None) %}
so/fleet:
event.send:
- data:
action: 'update_custom_hostname'
custom_hostname: {{ CUSTOM_FLEET_HOSTNAME }}
role: {{ grains.role }}

96
salt/fleet/files/dedicated-index.html
View File

@@ -1,96 +0,0 @@
{%- set PACKAGESTS = salt['pillar.get']('static:fleet_packages-timestamp:', 'N/A') -%}
<!DOCTYPE html>
<html lang="en">
<head>
<title>Security Onion - Hybrid Hunter</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="icon" type="image/png" href="favicon-32x32.png" sizes="32x32" />
<link rel="icon" type="image/png" href="favicon-16x16.png" sizes="16x16" />
<style>
* {
box-sizing: border-box;
font-family: Arial, Helvetica, sans-serif;
padding-left: 30px;
padding-right: 30px;
}
body {
font-family: Arial, Helvetica, sans-serif;
background-color: #2a2a2a;
}
a {
color: #f2f2f2;
text-align: left;
padding: 0px;
}
.center-content {
margin: 0 auto;
}
/* Style the top navigation bar */
.topnav {
overflow: hidden;
background-color: #333;
width: 1080px;
display: flex;
align-content: center;
}
/* Style the topnav links */
.topnav a {
margin: auto;
color: #f2f2f2;
text-align: center;
padding: 14px 16px;
text-decoration: none;
}
/* Change color on hover */
.topnav a:hover {
background-color: #ddd;
color: black;
}
/* Style the content */
.content {
background-color: #2a2a2a;
padding: 10px;
padding-top: 20px;
padding-left: 60px;
color: #E3DBCC;
width: 1080px;
}
/* Style the footer */
.footer {
background-color: #2a2a2a;
padding: 60px;
color: #E3DBCC;
width: 1080px;
}
</style>
</head>
<body>
<div class="center-content">
<div class="topnav center-content">
<a href="/fleet/" target="_blank">Fleet</a>
<a href="https://github.com/Security-Onion-Solutions/securityonion-saltstack/wiki/Configuring-Osquery-with-Security-Onion" target="_blank">Osquery/Fleet Docs</a>
<a href="https://www.securityonionsolutions.com" target="_blank">Security Onion Solutions</a>
</div>
<div class="content center-content">
<p>
<div style="text-align: center;">
<h1>Security Onion - Dedicated Fleet Node</h1>
</div>
<br/>
<br/>
</p>
</div>
</div>
</body>
</html>

0
salt/fleet/files/packs/PUT.PACKS.IN.HERE
View File

13
salt/fleet/files/packs/hh/hh-post-login.sh
View File

@@ -1,13 +0,0 @@
#!/bin/sh
echo "Applying Post Configuration for Osquery"
#fleetctl apply -f /packs/hh/osquery.conf
fleetctl apply -f /packs/palantir/Fleet/Endpoints/options.yaml
fleetctl apply -f /packs/palantir/Fleet/Endpoints/MacOS/osquery.yaml
fleetctl apply -f /packs/palantir/Fleet/Endpoints/Windows/osquery.yaml
fleetctl apply -f /packs/hh/hhdefault.yml
for pack in /packs/palantir/Fleet/Endpoints/packs/*.yaml;
do fleetctl apply -f "$pack"
done
echo ""
echo "You can now exit the container by typing exit"

0
salt/fleet/files/packs/hh/osquery.conf → salt/fleet/files/packs/osquery-config.conf
View File

0
salt/fleet/files/packs/hh/hhdefault.yml → salt/fleet/files/packs/so/so-default.yml
View File

33
salt/fleet/files/scripts/so-fleet-packages
View File

@@ -1,33 +0,0 @@
#!/bin/bash
{% set MAIN_HOSTNAME = salt['grains.get']('host') %}
{% set MAIN_IP = salt['pillar.get']('node:mainip') %}
#so-fleet-packages $FleetHostname/IP
#if [ ! "$(docker ps -q -f name=so-fleet)" ]; then
# echo "so-fleet container not running... Exiting..."
# exit 1
#fi
#docker exec so-fleet /bin/ash -c "echo {{ MAIN_IP }} {{ MAIN_HOSTNAME }} >> /etc/hosts"
#esecret=$(docker exec so-fleet fleetctl get enroll-secret)
#Concat fleet.crt & ca.crt - this is required for launcher connectivity
#cat /etc/pki/fleet.crt /etc/pki/ca.crt > /etc/pki/launcher.crt
#Actually only need to use /etc/ssl/certs/intca.crt
#Create the output directory
#mkdir /opt/so/conf/fleet/packages
docker run \
--rm \
--mount type=bind,source=/opt/so/conf/fleet/packages,target=/output \
--mount type=bind,source=/etc/ssl/certs/intca.crt,target=/var/launcher/launcher.crt \
docker.io/soshybridhunter/so-fleet-launcher:HH1.1.0 "$esecret" "$1":8090
cp /opt/so/conf/fleet/packages/launcher.* /opt/so/saltstack/salt/launcher/packages/
#Update timestamp on packages webpage
sed -i "s@.*Generated.*@Generated: $(date '+%m%d%Y')@g" /opt/so/conf/fleet/packages/index.html
sed -i "s@.*Generated.*@Generated: $(date '+%m%d%Y')@g" /opt/so/saltstack/salt/fleet/files/dedicated-index.html

48
salt/fleet/files/scripts/so-fleet-setup
View File

@@ -1,48 +0,0 @@
#!/bin/bash
{% set MAIN_HOSTNAME = salt['grains.get']('host') %}
{% set MAIN_IP = salt['pillar.get']('node:mainip') %}
#so-fleet-setup.sh $FleetEmail
# Enable Fleet
echo "Starting Docker Containers..."
salt-call state.apply mysql queue=True >> /root/fleet-setup.log
salt-call state.apply fleet queue=True >> /root/fleet-setup.log
salt-call state.apply redis queue=True >> /root/fleet-setup.log
if [ ! "$(docker ps -q -f name=so-fleet)" ]; then
echo "so-fleet container not running... Exiting..."
exit 1
fi
initpw=$(date +%s | sha256sum | base64 | head -c 16 ; echo)
docker exec so-fleet /bin/ash -c "echo {{ MAIN_IP }} {{ MAIN_HOSTNAME }} >> /etc/hosts"
docker exec so-fleet fleetctl config set --address https://{{ MAIN_HOSTNAME }}:443 --tls-skip-verify --url-prefix /fleet
docker exec so-fleet fleetctl setup --email $1 --password $initpw
docker exec so-fleet fleetctl apply -f /packs/palantir/Fleet/Endpoints/MacOS/osquery.yaml
docker exec so-fleet fleetctl apply -f /packs/palantir/Fleet/Endpoints/Windows/osquery.yaml
docker exec so-fleet fleetctl apply -f /packs/hh/hhdefault.yml
docker exec so-fleet /bin/sh -c 'for pack in /packs/palantir/Fleet/Endpoints/packs/*.yaml; do fleetctl apply -f "$pack"; done'
docker exec so-fleet fleetctl apply -f /packs/hh/osquery.conf
# Enable Fleet
echo "Enabling Fleet..."
salt-call state.apply fleet.event_enable-fleet queue=True >> /root/fleet-setup.log
salt-call state.apply nginx queue=True >> /root/fleet-setup.log
# Generate osquery install packages
echo "Generating osquery install packages - this will take some time..."
salt-call state.apply fleet.event_gen-packages queue=True >> /root/fleet-setup.log
sleep 120
echo "Installing launcher via salt..."
salt-call state.apply fleet.install_package queue=True >> /root/fleet-setup.log
salt-call state.apply filebeat queue=True >> /root/fleet-setup.log
docker stop so-nginx
salt-call state.apply nginx queue=True >> /root/fleet-setup.log
echo "Fleet Setup Complete - Login here: https://{{ MAIN_HOSTNAME }}"
echo "Your username is $2 and your password is $initpw"

28
salt/fleet/init.sls
View File

@@ -3,12 +3,11 @@
{%- set FLEETJWT = salt['pillar.get']('secrets:fleet_jwt', None) -%}
{% set VERSION = salt['pillar.get']('static:soversion', 'HH1.2.2') %}
{% set MASTER = salt['grains.get']('master') %}
{% set MAINIP = salt['pillar.get']('node:mainip') %}
{% set FLEETARCH = salt['grains.get']('role') %}
{% if FLEETARCH == "so-fleet" %}
{% set MAINIP = salt['pillar.get']('node:mainip') %}
{% set MAININT = salt['pillar.get']('host:mainint') %}
{% set MAINIP = salt['grains.get']('ip_interfaces').get(MAININT)[0] %}
{% else %}
{% set MAINIP = salt['pillar.get']('static:masterip') %}
{% endif %}
@@ -16,14 +15,6 @@
include:
- mysql
#{% if grains.id.split('_')|last in ['master', 'eval', 'fleet'] %}
#so/fleet:
# event.send:
# - data:
# action: 'enablefleet'
# hostname: {{ grains.host }}
#{% endif %}
# Fleet Setup
fleetcdir:
file.directory:
@@ -67,21 +58,6 @@ fleetlogdir:
- group: 939
- makedirs: True
fleetsetupscripts:
file.recurse:
- name: /usr/sbin
- user: 0
- group: 0
- file_mode: 755
- template: jinja
- source: salt://fleet/files/scripts
osquerypackageswebpage:
file.managed:
- name: /opt/so/conf/fleet/packages/index.html
- source: salt://fleet/files/dedicated-index.html
- template: jinja
fleetdb:
mysql_database.present:
- name: fleet

Some files were not shown because too many files have changed in this diff Show More