CSEC_PUBLIC/securityonion
1
0
Fork 0
mirror of https://github.com/Security-Onion-Solutions/securityonion.git synced 2025-12-06 17:22:49 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #10037 from Security-Onion-Solutions/2.4/dev

Browse Source 
2.4.0 Beta 1
This commit is contained in:
Mike Reeves
2023-03-28 16:08:01 -04:00
committed by GitHub
parent fbcbfaf7c3 72078848d3
commit b2a2dc5aea
1301 changed files with 24894 additions and 58649 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.github/.gitleaks.toml vendored
View File

@@ -536,7 +536,7 @@ secretGroup = 4
[allowlist] [allowlist]
description = "global allow lists" description = "global allow lists"
regexes = ['''219-09-9999''', '''078-05-1120''', '''(9[0-9]{2}|666)-\d{2}-\d{4}'''] regexes = ['''219-09-9999''', '''078-05-1120''', '''(9[0-9]{2}|666)-\d{2}-\d{4}''', '''RPM-GPG-KEY.*''']
paths = [ paths = [
'''gitleaks.toml''', '''gitleaks.toml''',
'''(.*?)(jpg|gif|doc|pdf|bin|svg|socket)$''', '''(.*?)(jpg|gif|doc|pdf|bin|svg|socket)$''',

8
.github/workflows/pythontest.yml vendored
View File

@@ -1,6 +1,12 @@
name: python-test name: python-test
on: [push, pull_request] on:
push:
paths:
- "salt/sensoroni/files/analyzers/**"
pull_request:
paths:
- "salt/sensoroni/files/analyzers/**"
jobs: jobs:
build: build:

16
README.md
View File

@@ -1,6 +1,6 @@
## Security Onion 2.3.140 ## Security Onion 2.4
Security Onion 2.3.140 is here! Security Onion 2.4 is here!
## Screenshots ## Screenshots
@@ -18,24 +18,24 @@ Cases
### Release Notes ### Release Notes
https://docs.securityonion.net/en/2.3/release-notes.html https://docs.securityonion.net/en/2.4/release-notes.html
### Requirements ### Requirements
https://docs.securityonion.net/en/2.3/hardware.html https://docs.securityonion.net/en/2.4/hardware.html
### Download ### Download
https://docs.securityonion.net/en/2.3/download.html https://docs.securityonion.net/en/2.4/download.html
### Installation ### Installation
https://docs.securityonion.net/en/2.3/installation.html https://docs.securityonion.net/en/2.4/installation.html
### FAQ ### FAQ
https://docs.securityonion.net/en/2.3/faq.html https://docs.securityonion.net/en/2.4/faq.html
### Feedback ### Feedback
https://docs.securityonion.net/en/2.3/community-support.html https://docs.securityonion.net/en/2.4/community-support.html

3
SECURITY.md
View File

@@ -4,7 +4,8 @@
| Version | Supported | | Version | Supported |
| ------- | ------------------ | | ------- | ------------------ |
| 2.x.x | :white_check_mark: | | 2.4.x | :white_check_mark: |
| 2.3.x | :white_check_mark: |
| 16.04.x | :x: | | 16.04.x | :x: |
Security Onion 16.04 has reached End Of Life and is no longer supported. Security Onion 16.04 has reached End Of Life and is no longer supported.

22
VERIFY_ISO.md
View File

@@ -1,18 +1,18 @@
### 2.3.140-20220718 ISO image built on 2022/07/18 ### 2.3.120-20220425 ISO image built on 2022/04/25
### Download and Verify ### Download and Verify
2.3.140-20220718 ISO image: 2.3.120-20220425 ISO image:
https://download.securityonion.net/file/securityonion/securityonion-2.3.140-20220718.iso https://download.securityonion.net/file/securityonion/securityonion-2.3.120-20220425.iso
MD5: 9570065548DBFA6230F28FF623A8B61A MD5: C99729E452B064C471BEF04532F28556
SHA1: D48B2CC81DF459C3EBBC0C54BD9AAFAB4327CB75 SHA1: 60BF07D5347C24568C7B793BFA9792E98479CFBF
SHA256: 0E31E15EDFD3392B9569FCCAF1E4518432ECB0D7A174CCA745F2F22CDAC4A034 SHA256: CD17D0D7CABE21D45FA45E1CF91C5F24EB9608C79FF88480134E5592AFDD696E
Signature for ISO image: Signature for ISO image:
https://github.com/Security-Onion-Solutions/securityonion/raw/master/sigs/securityonion-2.3.140-20220718.iso.sig https://github.com/Security-Onion-Solutions/securityonion/raw/master/sigs/securityonion-2.3.120-20220425.iso.sig
Signing key: Signing key:
https://raw.githubusercontent.com/Security-Onion-Solutions/securityonion/master/KEYS https://raw.githubusercontent.com/Security-Onion-Solutions/securityonion/master/KEYS
@@ -26,22 +26,22 @@ wget https://raw.githubusercontent.com/Security-Onion-Solutions/securityonion/ma
Download the signature file for the ISO: Download the signature file for the ISO:
``` ```
wget https://github.com/Security-Onion-Solutions/securityonion/raw/master/sigs/securityonion-2.3.140-20220718.iso.sig wget https://github.com/Security-Onion-Solutions/securityonion/raw/master/sigs/securityonion-2.3.120-20220425.iso.sig
``` ```
Download the ISO image: Download the ISO image:
``` ```
wget https://download.securityonion.net/file/securityonion/securityonion-2.3.140-20220718.iso wget https://download.securityonion.net/file/securityonion/securityonion-2.3.120-20220425.iso
``` ```
Verify the downloaded ISO image using the signature file: Verify the downloaded ISO image using the signature file:
``` ```
gpg --verify securityonion-2.3.140-20220718.iso.sig securityonion-2.3.140-20220718.iso gpg --verify securityonion-2.3.120-20220425.iso.sig securityonion-2.3.120-20220425.iso
``` ```
The output should show "Good signature" and the Primary key fingerprint should match what's shown below: The output should show "Good signature" and the Primary key fingerprint should match what's shown below:
``` ```
gpg: Signature made Mon 18 Jul 2022 10:16:05 AM EDT using RSA key ID FE507013 gpg: Signature made Mon 25 Apr 2022 08:20:40 AM EDT using RSA key ID FE507013
gpg: Good signature from "Security Onion Solutions, LLC <info@securityonionsolutions.com>" gpg: Good signature from "Security Onion Solutions, LLC <info@securityonionsolutions.com>"
gpg: WARNING: This key is not certified with a trusted signature! gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner. gpg: There is no indication that the signature belongs to the owner.

2
VERSION
View File

@@ -1 +1 @@
2.3.140 2.4.0

10
files/firewall/assigned_hostgroups.local.map.yaml
View File

@@ -1,8 +1,8 @@
{% import_yaml 'firewall/portgroups.yaml' as default_portgroups %} {% import_yaml 'firewall/ports/ports.yaml' as default_portgroups %}
{% set default_portgroups = default_portgroups.firewall.aliases.ports %} {% set default_portgroups = default_portgroups.firewall.ports %}
{% import_yaml 'firewall/portgroups.local.yaml' as local_portgroups %} {% import_yaml 'firewall/ports/ports.local.yaml' as local_portgroups %}
{% if local_portgroups.firewall.aliases.ports %} {% if local_portgroups.firewall.ports %}
{% set local_portgroups = local_portgroups.firewall.aliases.ports %} {% set local_portgroups = local_portgroups.firewall.ports %}
{% else %} {% else %}
{% set local_portgroups = {} %} {% set local_portgroups = {} %}
{% endif %} {% endif %}

82
files/firewall/hostgroups.local.yaml
View File

@@ -1,82 +0,0 @@
firewall:
hostgroups:
analyst:
ips:
delete:
insert:
beats_endpoint:
ips:
delete:
insert:
beats_endpoint_ssl:
ips:
delete:
insert:
elasticsearch_rest:
ips:
delete:
insert:
endgame:
ips:
delete:
insert:
fleet:
ips:
delete:
insert:
heavy_node:
ips:
delete:
insert:
idh:
ips:
delete:
insert:
manager:
ips:
delete:
insert:
minion:
ips:
delete:
insert:
node:
ips:
delete:
insert:
osquery_endpoint:
ips:
delete:
insert:
receiver:
ips:
delete:
insert:
search_node:
ips:
delete:
insert:
sensor:
ips:
delete:
insert:
strelka_frontend:
ips:
delete:
insert:
syslog:
ips:
delete:
insert:
wazuh_agent:
ips:
delete:
insert:
wazuh_api:
ips:
delete:
insert:
wazuh_authd:
ips:
delete:
insert:

3
files/firewall/portgroups.local.yaml
View File

@@ -1,3 +0,0 @@
firewall:
aliases:
ports:

2
files/firewall/ports/ports.local.yaml Normal file
View File

@@ -0,0 +1,2 @@
firewall:
ports:

6
files/salt/master/master
View File

@@ -64,10 +64,4 @@ peer:
.*: .*:
- x509.sign_remote_certificate - x509.sign_remote_certificate
reactor:
- 'so/fleet':
- salt://reactor/fleet.sls
- 'salt/beacon/*/watch_sqlite_db//opt/so/conf/kratos/db/sqlite.db':
- salt://reactor/kratos.sls

2
pillar/data/addtotab.sh
View File

@@ -45,12 +45,10 @@ echo " rootfs: $ROOTFS" >> $local_salt_dir/pillar/data/$TYPE.sls
echo " nsmfs: $NSM" >> $local_salt_dir/pillar/data/$TYPE.sls echo " nsmfs: $NSM" >> $local_salt_dir/pillar/data/$TYPE.sls
if [ $TYPE == 'sensorstab' ]; then if [ $TYPE == 'sensorstab' ]; then
echo " monint: bond0" >> $local_salt_dir/pillar/data/$TYPE.sls echo " monint: bond0" >> $local_salt_dir/pillar/data/$TYPE.sls
salt-call state.apply grafana queue=True
fi fi
if [ $TYPE == 'evaltab' ] || [ $TYPE == 'standalonetab' ]; then if [ $TYPE == 'evaltab' ] || [ $TYPE == 'standalonetab' ]; then
echo " monint: bond0" >> $local_salt_dir/pillar/data/$TYPE.sls echo " monint: bond0" >> $local_salt_dir/pillar/data/$TYPE.sls
if [ ! $10 ]; then if [ ! $10 ]; then
salt-call state.apply grafana queue=True
salt-call state.apply utility queue=True salt-call state.apply utility queue=True
fi fi
fi fi

1
pillar/logstash/init.sls
View File

@@ -3,6 +3,7 @@ logstash:
port_bindings: port_bindings:
- 0.0.0.0:3765:3765 - 0.0.0.0:3765:3765
- 0.0.0.0:5044:5044 - 0.0.0.0:5044:5044
- 0.0.0.0:5055:5055
- 0.0.0.0:5644:5644 - 0.0.0.0:5644:5644
- 0.0.0.0:6050:6050 - 0.0.0.0:6050:6050
- 0.0.0.0:6051:6051 - 0.0.0.0:6051:6051

3
pillar/logstash/manager.sls
View File

@@ -2,8 +2,7 @@ logstash:
pipelines: pipelines:
manager: manager:
config: config:
- so/0009_input_beats.conf
- so/0010_input_hhbeats.conf
- so/0011_input_endgame.conf - so/0011_input_endgame.conf
- so/0012_input_elastic_agent.conf
- so/9999_output_redis.conf.jinja - so/9999_output_redis.conf.jinja

2
pillar/logstash/nodes.sls
View File

@@ -2,7 +2,7 @@
{% set cached_grains = salt.saltutil.runner('cache.grains', tgt='*') %} {% set cached_grains = salt.saltutil.runner('cache.grains', tgt='*') %}
{% for minionid, ip in salt.saltutil.runner( {% for minionid, ip in salt.saltutil.runner(
'mine.get', 'mine.get',
tgt='G@role:so-manager or G@role:so-managersearch or G@role:so-standalone or G@role:so-node or G@role:so-heavynode or G@role:so-receiver or G@role:so-helix', tgt='G@role:so-manager or G@role:so-managersearch or G@role:so-standalone or G@role:so-searchnode or G@role:so-heavynode or G@role:so-receiver or G@role:so-helix ',
fun='network.ip_addrs', fun='network.ip_addrs',
tgt_type='compound') | dictsort() tgt_type='compound') | dictsort()
%} %}

3
pillar/logstash/receiver.sls
View File

@@ -2,8 +2,7 @@ logstash:
pipelines: pipelines:
receiver: receiver:
config: config:
- so/0009_input_beats.conf
- so/0010_input_hhbeats.conf
- so/0011_input_endgame.conf - so/0011_input_endgame.conf
- so/0012_input_elastic_agent.conf
- so/9999_output_redis.conf.jinja - so/9999_output_redis.conf.jinja

13
pillar/logstash/search.sls
View File

@@ -3,16 +3,5 @@ logstash:
search: search:
config: config:
- so/0900_input_redis.conf.jinja - so/0900_input_redis.conf.jinja
- so/9000_output_zeek.conf.jinja - so/9805_output_elastic_agent.conf.jinja
- so/9002_output_import.conf.jinja
- so/9034_output_syslog.conf.jinja
- so/9050_output_filebeatmodules.conf.jinja
- so/9100_output_osquery.conf.jinja
- so/9400_output_suricata.conf.jinja
- so/9500_output_beats.conf.jinja
- so/9600_output_ossec.conf.jinja
- so/9700_output_strelka.conf.jinja
- so/9800_output_logscan.conf.jinja
- so/9801_output_rita.conf.jinja
- so/9802_output_kratos.conf.jinja
- so/9900_output_endgame.conf.jinja - so/9900_output_endgame.conf.jinja

10
pillar/node_data/ips.sls
View File

@@ -1,7 +1,5 @@
{% set node_types = {} %} {% set node_types = {} %}
{% set manage_alived = salt.saltutil.runner('manage.alived', show_ip=True) %} {% set manage_alived = salt.saltutil.runner('manage.alived', show_ip=True) %}
{% set manager = grains.master %}
{% set manager_type = manager.split('_')|last %}
{% for minionid, ip in salt.saltutil.runner('mine.get', tgt='*', fun='network.ip_addrs', tgt_type='glob') | dictsort() %} {% for minionid, ip in salt.saltutil.runner('mine.get', tgt='*', fun='network.ip_addrs', tgt_type='glob') | dictsort() %}
{% set hostname = minionid.split('_')[0] %} {% set hostname = minionid.split('_')[0] %}
{% set node_type = minionid.split('_')[1] %} {% set node_type = minionid.split('_')[1] %}
@@ -24,10 +22,10 @@
node_data: node_data:
{% for node_type, host_values in node_types.items() %} {% for node_type, host_values in node_types.items() %}
{{node_type}}:
{% for hostname, details in host_values.items() %} {% for hostname, details in host_values.items() %}
{{hostname}}: {{hostname}}:
ip: {{details.ip}} ip: {{details.ip}}
alive: {{ details.alive }} alive: {{ details.alive }}
role: {{node_type}}
{% endfor %} {% endfor %}
{% endfor %} {% endfor %}

201
pillar/top.sls
View File

@@ -1,136 +1,229 @@
base: base:
'*': '*':
- patch.needs_restarting - patch.needs_restarting
- ntp.soc_ntp
- ntp.adv_ntp
- logrotate - logrotate
- docker.soc_docker
- docker.adv_docker
- sensoroni.soc_sensoroni
- sensoroni.adv_sensoroni
- telegraf.soc_telegraf
- telegraf.adv_telegraf
- influxdb.token
- node_data.ips
'* and not *_eval and not *_import': '* and not *_eval and not *_import':
- logstash.nodes - logstash.nodes
'*_eval or *_helixsensor or *_heavynode or *_sensor or *_standalone or *_import': '*_eval or *_heavynode or *_sensor or *_standalone or *_import':
- match: compound - match: compound
- zeek - zeek
- bpf.soc_bpf
- bpf.adv_bpf
'*_managersearch or *_heavynode': '*_managersearch or *_heavynode':
- match: compound - match: compound
- logstash - logstash
- logstash.manager - logstash.manager
- logstash.search - logstash.search
- logstash.soc_logstash
- logstash.adv_logstash
- elasticsearch.index_templates - elasticsearch.index_templates
- elasticsearch.soc_elasticsearch
- elasticsearch.adv_elasticsearch
'*_manager': '*_manager':
- logstash - logstash
- logstash.manager - logstash.manager
- logstash.soc_logstash
- logstash.adv_logstash
- elasticsearch.index_templates - elasticsearch.index_templates
'*_manager or *_managersearch': '*_manager or *_managersearch':
- match: compound - match: compound
- data.* {% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/elasticsearch/auth.sls') %}
{% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/elasticsearch/auth.sls') %}
- elasticsearch.auth - elasticsearch.auth
{% endif %} {% endif %}
{% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/kibana/secrets.sls') %} {% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/kibana/secrets.sls') %}
- kibana.secrets - kibana.secrets
{% endif %} {% endif %}
- secrets - secrets
- global - soc_global
- adv_global
- manager.soc_manager
- manager.adv_manager
- idstools.soc_idstools
- idstools.adv_idstools
- soc.soc_soc
- soc.adv_soc
- kratos.soc_kratos
- kratos.adv_kratos
- redis.soc_redis
- redis.adv_redis
- influxdb.soc_influxdb
- influxdb.adv_influxdb
- elasticsearch.soc_elasticsearch
- elasticsearch.adv_elasticsearch
- backup.soc_backup
- backup.adv_backup
- firewall.soc_firewall
- firewall.adv_firewall
- minions.{{ grains.id }} - minions.{{ grains.id }}
- minions.adv_{{ grains.id }}
'*_sensor': '*_sensor':
- zeeklogs
- healthcheck.sensor - healthcheck.sensor
- global - soc_global
- adv_global
- minions.{{ grains.id }} - minions.{{ grains.id }}
- minions.adv_{{ grains.id }}
'*_eval': '*_eval':
- data.*
- zeeklogs
- secrets - secrets
- healthcheck.eval - healthcheck.eval
- elasticsearch.index_templates - elasticsearch.index_templates
{% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/elasticsearch/auth.sls') %} {% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/elasticsearch/auth.sls') %}
- elasticsearch.auth - elasticsearch.auth
{% endif %} {% endif %}
{% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/kibana/secrets.sls') %} {% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/kibana/secrets.sls') %}
- kibana.secrets - kibana.secrets
{% endif %} {% endif %}
- global - soc_global
- kratos.soc_kratos
- elasticsearch.soc_elasticsearch
- elasticsearch.adv_elasticsearch
- manager.soc_manager
- manager.adv_manager
- idstools.soc_idstools
- idstools.adv_idstools
- soc.soc_soc
- kratos.soc_kratos
- kratos.adv_kratos
- redis.soc_redis
- redis.adv_redis
- influxdb.soc_influxdb
- influxdb.adv_influxdb
- backup.soc_backup
- backup.adv_backup
- firewall.soc_firewall
- firewall.adv_firewall
- minions.{{ grains.id }} - minions.{{ grains.id }}
- minions.adv_{{ grains.id }}
'*_standalone': '*_standalone':
- logstash - logstash
- logstash.manager - logstash.manager
- logstash.search - logstash.search
- logstash.soc_logstash
- logstash.adv_logstash
- elasticsearch.index_templates - elasticsearch.index_templates
{% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/elasticsearch/auth.sls') %} {% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/elasticsearch/auth.sls') %}
- elasticsearch.auth - elasticsearch.auth
{% endif %} {% endif %}
{% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/kibana/secrets.sls') %} {% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/kibana/secrets.sls') %}
- kibana.secrets - kibana.secrets
{% endif %} {% endif %}
- data.*
- zeeklogs
- secrets - secrets
- healthcheck.standalone - healthcheck.standalone
- global - soc_global
- minions.{{ grains.id }} - idstools.soc_idstools
- idstools.adv_idstools
'*_node': - kratos.soc_kratos
- global - kratos.adv_kratos
- redis.soc_redis
- redis.adv_redis
- influxdb.soc_influxdb
- influxdb.adv_influxdb
- elasticsearch.soc_elasticsearch
- elasticsearch.adv_elasticsearch
- manager.soc_manager
- manager.adv_manager
- soc.soc_soc
- backup.soc_backup
- backup.adv_backup
- firewall.soc_firewall
- firewall.adv_firewall
- minions.{{ grains.id }} - minions.{{ grains.id }}
- minions.adv_{{ grains.id }}
'*_heavynode': '*_heavynode':
- zeeklogs
- elasticsearch.auth - elasticsearch.auth
- global - soc_global
- minions.{{ grains.id }} - redis.soc_redis
'*_helixsensor':
- fireeye
- zeeklogs
- logstash
- logstash.helix
- global
- minions.{{ grains.id }}
'*_fleet':
- data.*
- secrets
- global
- minions.{{ grains.id }} - minions.{{ grains.id }}
- minions.adv_{{ grains.id }}
'*_idh': '*_idh':
- data.* - soc_global
- global - adv_global
- idh.soc_idh
- idh.adv_idh
- minions.{{ grains.id }} - minions.{{ grains.id }}
- minions.adv_{{ grains.id }}
'*_searchnode': '*_searchnode':
- logstash - logstash
- logstash.search - logstash.search
- logstash.soc_logstash
- logstash.adv_logstash
- elasticsearch.index_templates - elasticsearch.index_templates
- elasticsearch.soc_elasticsearch
- elasticsearch.adv_elasticsearch
{% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/elasticsearch/auth.sls') %}
- elasticsearch.auth - elasticsearch.auth
- global {% endif %}
- redis.soc_redis
- soc_global
- adv_global
- minions.{{ grains.id }} - minions.{{ grains.id }}
- data.nodestab - minions.adv_{{ grains.id }}
'*_receiver': '*_receiver':
- logstash - logstash
- logstash.receiver - logstash.receiver
- logstash.soc_logstash
- logstash.adv_logstash
{% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/elasticsearch/auth.sls') %}
- elasticsearch.auth - elasticsearch.auth
- global {% endif %}
- redis.soc_redis
- redis.adv_redis
- soc_global
- adv_global
- minions.{{ grains.id }} - minions.{{ grains.id }}
- minions.adv_{{ grains.id }}
'*_import': '*_import':
- zeeklogs
- secrets - secrets
- elasticsearch.index_templates - elasticsearch.index_templates
{% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/elasticsearch/auth.sls') %} {% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/elasticsearch/auth.sls') %}
- elasticsearch.auth - elasticsearch.auth
{% endif %} {% endif %}
{% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/kibana/secrets.sls') %} {% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/kibana/secrets.sls') %}
- kibana.secrets - kibana.secrets
{% endif %} {% endif %}
- global - kratos.soc_kratos
- elasticsearch.soc_elasticsearch
- elasticsearch.adv_elasticsearch
- manager.soc_manager
- manager.adv_manager
- soc.soc_soc
- soc_global
- adv_global
- backup.soc_backup
- backup.adv_backup
- kratos.soc_kratos
- kratos.adv_kratos
- redis.soc_redis
- redis.adv_redis
- influxdb.soc_influxdb
- influxdb.adv_influxdb
- firewall.soc_firewall
- firewall.adv_firewall
- minions.{{ grains.id }} - minions.{{ grains.id }}
- minions.adv_{{ grains.id }}
'*_workstation': '*_workstation':
- minions.{{ grains.id }} - minions.{{ grains.id }}
- minions.adv_{{ grains.id }}

54
pillar/zeek/init.sls
View File

@@ -1,55 +1 @@
zeek: zeek:
zeekctl:
MailTo: root@localhost
MailConnectionSummary: 1
MinDiskSpace: 5
MailHostUpDown: 1
LogRotationInterval: 3600
LogExpireInterval: 0
StatsLogEnable: 1
StatsLogExpireInterval: 0
StatusCmdShowAll: 0
CrashExpireInterval: 0
SitePolicyScripts: local.zeek
LogDir: /nsm/zeek/logs
SpoolDir: /nsm/zeek/spool
CfgDir: /opt/zeek/etc
CompressLogs: 1
local:
'@load':
- misc/loaded-scripts
- tuning/defaults
- misc/capture-loss
- misc/stats
- frameworks/software/vulnerable
- frameworks/software/version-changes
- protocols/ftp/software
- protocols/smtp/software
- protocols/ssh/software
- protocols/http/software
- protocols/dns/detect-external-names
- protocols/ftp/detect
- protocols/conn/known-hosts
- protocols/conn/known-services
- protocols/ssl/known-certs
- protocols/ssl/validate-certs
- protocols/ssl/log-hostcerts-only
- protocols/ssh/geo-data
- protocols/ssh/detect-bruteforcing
- protocols/ssh/interesting-hostnames
- protocols/http/detect-sqli
- frameworks/files/hash-all-files
- frameworks/files/detect-MHR
- policy/frameworks/notice/extend-email/hostnames
- ja3
- hassh
- intel
- cve-2020-0601
- securityonion/bpfconf
- securityonion/communityid
- securityonion/file-extraction
'@load-sigs':
- frameworks/signatures/detect-windows-shells
redef:
- LogAscii::use_json = T;
- CaptureLoss::watch_interval = 5 mins;

2
salt/_modules/needs_restarting.py
View File

@@ -10,7 +10,7 @@ def check():
if path.exists('/var/run/reboot-required'): if path.exists('/var/run/reboot-required'):
retval = 'True' retval = 'True'
elif os == 'CentOS': elif os == 'Rocky':
cmd = 'needs-restarting -r > /dev/null 2>&1' cmd = 'needs-restarting -r > /dev/null 2>&1'
try: try:

2
salt/_modules/so.py
View File

@@ -5,6 +5,8 @@ import logging
def status(): def status():
return __salt__['cmd.run']('/usr/sbin/so-status') return __salt__['cmd.run']('/usr/sbin/so-status')
def version():
return __salt__['cp.get_file_str']('/etc/soversion')
def mysql_conn(retry): def mysql_conn(retry):
log = logging.getLogger(__name__) log = logging.getLogger(__name__)

98
salt/allowed_states.map.jinja
View File

@@ -1,13 +1,13 @@
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
{% set ZEEKVER = salt['pillar.get']('global:mdengine', '') %} {% set ZEEKVER = salt['pillar.get']('global:mdengine', '') %}
{% set WAZUH = salt['pillar.get']('global:wazuh', '0') %}
{% set PLAYBOOK = salt['pillar.get']('manager:playbook', '0') %} {% set PLAYBOOK = salt['pillar.get']('manager:playbook', '0') %}
{% set FREQSERVER = salt['pillar.get']('manager:freq', '0') %}
{% set DOMAINSTATS = salt['pillar.get']('manager:domainstats', '0') %}
{% set FLEETMANAGER = salt['pillar.get']('global:fleet_manager', False) %}
{% set FLEETNODE = salt['pillar.get']('global:fleet_node', False) %}
{% set ELASTALERT = salt['pillar.get']('elastalert:enabled', True) %} {% set ELASTALERT = salt['pillar.get']('elastalert:enabled', True) %}
{% set ELASTICSEARCH = salt['pillar.get']('elasticsearch:enabled', True) %} {% set ELASTICSEARCH = salt['pillar.get']('elasticsearch:enabled', True) %}
{% set FILEBEAT = salt['pillar.get']('filebeat:enabled', True) %}
{% set KIBANA = salt['pillar.get']('kibana:enabled', True) %} {% set KIBANA = salt['pillar.get']('kibana:enabled', True) %}
{% set LOGSTASH = salt['pillar.get']('logstash:enabled', True) %} {% set LOGSTASH = salt['pillar.get']('logstash:enabled', True) %}
{% set CURATOR = salt['pillar.get']('curator:enabled', True) %} {% set CURATOR = salt['pillar.get']('curator:enabled', True) %}
@@ -32,9 +32,9 @@
'nginx', 'nginx',
'telegraf', 'telegraf',
'influxdb', 'influxdb',
'grafana',
'soc', 'soc',
'kratos', 'kratos',
'elasticfleet',
'firewall', 'firewall',
'idstools', 'idstools',
'suricata.manager', 'suricata.manager',
@@ -45,8 +45,7 @@
'schedule', 'schedule',
'soctopus', 'soctopus',
'tcpreplay', 'tcpreplay',
'docker_clean', 'docker_clean'
'learn'
], ],
'so-heavynode': [ 'so-heavynode': [
'ssl', 'ssl',
@@ -77,25 +76,10 @@
'tcpreplay', 'tcpreplay',
'docker_clean' 'docker_clean'
], ],
'so-fleet': [
'ssl',
'nginx',
'telegraf',
'firewall',
'mysql',
'redis',
'fleet',
'fleet.install_package',
'filebeat',
'schedule',
'docker_clean'
],
'so-idh': [ 'so-idh': [
'ssl', 'ssl',
'telegraf', 'telegraf',
'firewall', 'firewall',
'fleet.install_package',
'filebeat',
'idh', 'idh',
'schedule', 'schedule',
'docker_clean' 'docker_clean'
@@ -109,6 +93,8 @@
'nginx', 'nginx',
'soc', 'soc',
'kratos', 'kratos',
'influxdb',
'telegraf',
'firewall', 'firewall',
'idstools', 'idstools',
'suricata.manager', 'suricata.manager',
@@ -119,7 +105,7 @@
'schedule', 'schedule',
'tcpreplay', 'tcpreplay',
'docker_clean', 'docker_clean',
'learn' 'elasticfleet'
], ],
'so-manager': [ 'so-manager': [
'salt.master', 'salt.master',
@@ -130,17 +116,16 @@
'nginx', 'nginx',
'telegraf', 'telegraf',
'influxdb', 'influxdb',
'grafana',
'soc', 'soc',
'kratos', 'kratos',
'elasticfleet',
'firewall', 'firewall',
'idstools', 'idstools',
'suricata.manager', 'suricata.manager',
'utility', 'utility',
'schedule', 'schedule',
'soctopus', 'soctopus',
'docker_clean', 'docker_clean'
'learn'
], ],
'so-managersearch': [ 'so-managersearch': [
'salt.master', 'salt.master',
@@ -150,9 +135,9 @@
'nginx', 'nginx',
'telegraf', 'telegraf',
'influxdb', 'influxdb',
'grafana',
'soc', 'soc',
'kratos', 'kratos',
'elasticfleet',
'firewall', 'firewall',
'manager', 'manager',
'idstools', 'idstools',
@@ -160,10 +145,9 @@
'utility', 'utility',
'schedule', 'schedule',
'soctopus', 'soctopus',
'docker_clean', 'docker_clean'
'learn'
], ],
'so-node': [ 'so-searchnode': [
'ssl', 'ssl',
'nginx', 'nginx',
'telegraf', 'telegraf',
@@ -180,9 +164,9 @@
'nginx', 'nginx',
'telegraf', 'telegraf',
'influxdb', 'influxdb',
'grafana',
'soc', 'soc',
'kratos', 'kratos',
'elasticfleet',
'firewall', 'firewall',
'idstools', 'idstools',
'suricata.manager', 'suricata.manager',
@@ -193,8 +177,7 @@
'schedule', 'schedule',
'soctopus', 'soctopus',
'tcpreplay', 'tcpreplay',
'docker_clean', 'docker_clean'
'learn'
], ],
'so-sensor': [ 'so-sensor': [
'ssl', 'ssl',
@@ -204,8 +187,6 @@
'pcap', 'pcap',
'suricata', 'suricata',
'healthcheck', 'healthcheck',
'wazuh',
'filebeat',
'schedule', 'schedule',
'tcpreplay', 'tcpreplay',
'docker_clean' 'docker_clean'
@@ -221,26 +202,10 @@
], ],
}, grain='role') %} }, grain='role') %}
{% if FILEBEAT and grains.role in ['so-helixsensor', 'so-eval', 'so-manager', 'so-standalone', 'so-node', 'so-managersearch', 'so-heavynode', 'so-import', 'so-receiver'] %} {% if (PLAYBOOK != 0) and grains.role in ['so-eval', 'so-manager', 'so-managersearch', 'so-standalone'] %}
{% do allowed_states.append('filebeat') %}
{% endif %}
{% if ((FLEETMANAGER or FLEETNODE) or PLAYBOOK != 0) and grains.role in ['so-eval', 'so-manager', 'so-managersearch', 'so-standalone'] %}
{% do allowed_states.append('mysql') %} {% do allowed_states.append('mysql') %}
{% endif %} {% endif %}
{% if (FLEETMANAGER or FLEETNODE) and grains.role in ['so-sensor', 'so-eval', 'so-manager', 'so-standalone', 'so-node', 'so-managersearch', 'so-heavynode', 'so-receiver'] %}
{% do allowed_states.append('fleet.install_package') %}
{% endif %}
{% if (FLEETMANAGER or FLEETNODE) and grains.role in ['so-eval', 'so-manager', 'so-standalone', 'so-managersearch', 'so-heavynode'] %}
{% do allowed_states.append('fleet') %}
{% endif %}
{% if (FLEETMANAGER or FLEETNODE) and grains.role in ['so-eval'] %}
{% do allowed_states.append('redis') %}
{% endif %}
{%- if ZEEKVER != 'SURICATA' and grains.role in ['so-sensor', 'so-eval', 'so-standalone', 'so-heavynode'] %} {%- if ZEEKVER != 'SURICATA' and grains.role in ['so-sensor', 'so-eval', 'so-standalone', 'so-heavynode'] %}
{% do allowed_states.append('zeek') %} {% do allowed_states.append('zeek') %}
{%- endif %} {%- endif %}
@@ -249,11 +214,7 @@
{% do allowed_states.append('strelka') %} {% do allowed_states.append('strelka') %}
{% endif %} {% endif %}
{% if WAZUH and grains.role in ['so-eval', 'so-manager', 'so-standalone', 'so-node', 'so-managersearch', 'so-heavynode', 'so-receiver','so-idh']%} {% if ELASTICSEARCH and grains.role in ['so-eval', 'so-manager', 'so-standalone', 'so-searchnode', 'so-managersearch', 'so-heavynode', 'so-import'] %}
{% do allowed_states.append('wazuh') %}
{% endif %}
{% if ELASTICSEARCH and grains.role in ['so-eval', 'so-manager', 'so-standalone', 'so-node', 'so-managersearch', 'so-heavynode', 'so-import'] %}
{% do allowed_states.append('elasticsearch') %} {% do allowed_states.append('elasticsearch') %}
{% endif %} {% endif %}
@@ -266,7 +227,7 @@
{% do allowed_states.append('kibana.secrets') %} {% do allowed_states.append('kibana.secrets') %}
{% endif %} {% endif %}
{% if grains.role in ['so-eval', 'so-standalone', 'so-node', 'so-managersearch', 'so-heavynode', 'so-manager'] %} {% if grains.role in ['so-eval', 'so-standalone', 'so-managersearch', 'so-heavynode', 'so-manager'] %}
{% do allowed_states.append('curator') %} {% do allowed_states.append('curator') %}
{% endif %} {% endif %}
@@ -282,15 +243,7 @@
{% do allowed_states.append('redis') %} {% do allowed_states.append('redis') %}
{% endif %} {% endif %}
{% if (FREQSERVER !=0) and grains.role in ['so-eval', 'so-manager', 'so-standalone', 'so-managersearch'] %} {% if LOGSTASH and grains.role in ['so-helixsensor', 'so-manager', 'so-standalone', 'so-searchnode', 'so-managersearch', 'so-heavynode', 'so-receiver'] %}
{% do allowed_states.append('freqserver') %}
{% endif %}
{% if (DOMAINSTATS !=0) and grains.role in ['so-eval', 'so-manager', 'so-standalone', 'so-managersearch'] %}
{% do allowed_states.append('domainstats') %}
{% endif %}
{% if LOGSTASH and grains.role in ['so-helixsensor', 'so-manager', 'so-standalone', 'so-node', 'so-managersearch', 'so-heavynode', 'so-receiver'] %}
{% do allowed_states.append('logstash') %} {% do allowed_states.append('logstash') %}
{% endif %} {% endif %}
@@ -298,13 +251,6 @@
{% do allowed_states.append('redis') %} {% do allowed_states.append('redis') %}
{% endif %} {% endif %}
{% if grains.os == 'CentOS' %}
{% if not ISAIRGAP %}
{% do allowed_states.append('yum') %}
{% endif %}
{% do allowed_states.append('yum.packages') %}
{% endif %}
{# all nodes on the right salt version can run the following states #} {# all nodes on the right salt version can run the following states #}
{% do allowed_states.append('common') %} {% do allowed_states.append('common') %}
{% do allowed_states.append('patch.os.schedule') %} {% do allowed_states.append('patch.os.schedule') %}

33
salt/backup/config_backup.sls Normal file
View File

@@ -0,0 +1,33 @@
{% from 'backup/map.jinja' import BACKUP_MERGED %}
# Lock permissions on the backup directory
backupdir:
file.directory:
- name: /nsm/backup
- user: 0
- group: 0
- makedirs: True
- mode: 700
config_backup_script:
file.managed:
- name: /usr/sbin/so-config-backup
- user: root
- group: root
- mode: 755
- template: jinja
- source: salt://backup/tools/sbin/so-config-backup.jinja
- defaults:
BACKUPLOCATIONS: {{ BACKUP_MERGED.locations }}
DESTINATION: {{ BACKUP_MERGED.destination }}
# Add config backup
so_config_backup:
cron.present:
- name: /usr/sbin/so-config-backup > /dev/null 2>&1
- user: root
- minute: '1'
- hour: '0'
- daymonth: '*'
- month: '*'
- dayweek: '*'

7
salt/backup/defaults.yaml Normal file
View File

@@ -0,0 +1,7 @@
backup:
locations:
- /opt/so/saltstack/local
- /etc/pki
- /etc/salt
- /nsm/kratos
destination: "/nsm/backup"

2
salt/backup/map.jinja Normal file
View File

@@ -0,0 +1,2 @@
{% import_yaml 'backup/defaults.yaml' as BACKUP_DEFAULTS %}
{% set BACKUP_MERGED = salt['pillar.get']('backup', BACKUP_DEFAULTS.backup, merge=true, merge_nested_lists=true) %}

10
salt/backup/soc_backup.yaml Normal file
View File

@@ -0,0 +1,10 @@
backup:
locations:
description: List of locations to back up to the destination.
helpLink: backup.html
global: True
destination:
description: Directory to store the configuration backups in.
helpLink: backup.html
global: True

37
salt/backup/tools/sbin/so-config-backup.jinja Executable file
View File

@@ -0,0 +1,37 @@
#!/bin/bash
#
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
TODAY=$(date '+%Y_%m_%d')
BACKUPDIR={{ DESTINATION }}
BACKUPFILE="$BACKUPDIR/so-config-backup-$TODAY.tar"
MAXBACKUPS=7
# Create backup dir if it does not exist
mkdir -p /nsm/backup
# If we haven't already written a backup file for today, let's do so
if [ ! -f $BACKUPFILE ]; then
# Create empty backup file
tar -cf $BACKUPFILE -T /dev/null
# Loop through all paths defined in global.sls, and append them to backup file
{%- for LOCATION in BACKUPLOCATIONS %}
tar -rf $BACKUPFILE {{ LOCATION }}
{%- endfor %}
fi
# Find oldest backup files and remove them
NUMBACKUPS=$(find /nsm/backup/ -type f -name "so-config-backup*" | wc -l)
while [ "$NUMBACKUPS" -gt "$MAXBACKUPS" ]; do
OLDESTBACKUP=$(find /nsm/backup/ -type f -name "so-config-backup*" -type f -printf '%T+ %p\n' | sort | head -n 1 | awk -F" " '{print $2}')
rm -f $OLDESTBACKUP
NUMBACKUPS=$(find /nsm/backup/ -type f -name "so-config-backup*" | wc -l)
done

4
salt/bpf/defaults.yaml Normal file
View File

@@ -0,0 +1,4 @@
bpf:
pcap: []
suricata: []
zeek: []

4
salt/bpf/pcap.map.jinja Normal file
View File

@@ -0,0 +1,4 @@
{% import_yaml 'bpf/defaults.yaml' as BPFDEFAULTS %}
{% set BPFMERGED = salt['pillar.get']('bpf', BPFDEFAULTS.bpf, merge=True) %}
{% set PCAPBPF = BPFMERGED.pcap %}

16
salt/bpf/soc_bpf.yaml Normal file
View File

@@ -0,0 +1,16 @@
bpf:
pcap:
description: List of BPF filters to apply to PCAP.
multiline: True
forcedType: "[]string"
helpLink: bpf.html
suricata:
description: List of BPF filters to apply to Suricata.
multiline: True
forcedType: "[]string"
helpLink: bpf.html
zeek:
description: List of BPF filters to apply to Zeek.
multiline: True
forcedType: "[]string"
helpLink: bpf.html

4
salt/bpf/suricata.map.jinja Normal file
View File

@@ -0,0 +1,4 @@
{% import_yaml 'bpf/defaults.yaml' as BPFDEFAULTS %}
{% set BPFMERGED = salt['pillar.get']('bpf', BPFDEFAULTS.bpf, merge=True) %}
{% set SURICATABPF = BPFMERGED.suricata %}

4
salt/bpf/zeek.map.jinja Normal file
View File

@@ -0,0 +1,4 @@
{% import_yaml 'bpf/defaults.yaml' as BPFDEFAULTS %}
{% set BPFMERGED = salt['pillar.get']('bpf', BPFDEFAULTS.bpf, merge=True) %}
{% set ZEEKBPF = BPFMERGED.zeek %}

5
salt/ca/files/signing_policies.conf
View File

@@ -57,7 +57,7 @@ x509_signing_policies:
- extendedKeyUsage: serverAuth - extendedKeyUsage: serverAuth
- days_valid: 820 - days_valid: 820
- copypath: /etc/pki/issued_certs/ - copypath: /etc/pki/issued_certs/
fleet: elasticfleet:
- minions: '*' - minions: '*'
- signing_private_key: /etc/pki/ca.key - signing_private_key: /etc/pki/ca.key
- signing_cert: /etc/pki/ca.crt - signing_cert: /etc/pki/ca.crt
@@ -65,9 +65,8 @@ x509_signing_policies:
- ST: Utah - ST: Utah
- L: Salt Lake City - L: Salt Lake City
- basicConstraints: "critical CA:false" - basicConstraints: "critical CA:false"
- keyUsage: "critical keyEncipherment" - keyUsage: "digitalSignature, nonRepudiation"
- subjectKeyIdentifier: hash - subjectKeyIdentifier: hash
- authorityKeyIdentifier: keyid,issuer:always - authorityKeyIdentifier: keyid,issuer:always
- extendedKeyUsage: serverAuth
- days_valid: 820 - days_valid: 820
- copypath: /etc/pki/issued_certs/ - copypath: /etc/pki/issued_certs/

10
salt/ca/init.sls
View File

@@ -1,10 +1,16 @@
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
{% from 'allowed_states.map.jinja' import allowed_states %} {% from 'allowed_states.map.jinja' import allowed_states %}
{% if sls in allowed_states %} {% if sls in allowed_states %}
{% from 'vars/globals.map.jinja' import GLOBALS %}
include: include:
- ca.dirs - ca.dirs
{% set manager = salt['grains.get']('master') %}
/etc/salt/minion.d/signing_policies.conf: /etc/salt/minion.d/signing_policies.conf:
file.managed: file.managed:
- source: salt://ca/files/signing_policies.conf - source: salt://ca/files/signing_policies.conf
@@ -25,7 +31,7 @@ pki_public_ca_crt:
x509.certificate_managed: x509.certificate_managed:
- name: /etc/pki/ca.crt - name: /etc/pki/ca.crt
- signing_private_key: /etc/pki/ca.key - signing_private_key: /etc/pki/ca.key
- CN: {{ manager }} - CN: {{ GLOBALS.manager }}
- C: US - C: US
- ST: Utah - ST: Utah
- L: Salt Lake City - L: Salt Lake City

18
salt/common/files/daemon.json
View File

@@ -1,12 +1,14 @@
{%- set DOCKERRANGE = salt['pillar.get']('docker:range', '172.17.0.0/24') %} {%- set DOCKERRANGE = salt['pillar.get']('docker:range', '172.17.0.0/24') %}
{%- set DOCKERBIND = salt['pillar.get']('docker:bip', '172.17.0.1/24') %} {%- set DOCKERBIND = salt['pillar.get']('docker:bip', '172.17.0.1/24') %}
{ {
"registry-mirrors": [ "https://:5000" ], "registry-mirrors": [
"bip": "{{ DOCKERBIND }}", "https://:5000"
"default-address-pools": [ ],
{ "bip": "{{ DOCKERBIND }}",
"base" : "{{ DOCKERRANGE }}", "default-address-pools": [
"size" : 24 {
} "base": "{{ DOCKERRANGE }}",
] "size": 24
}
]
} }

1
salt/common/files/vimrc
View File

@@ -3,4 +3,3 @@ filetype plugin indent on
" Sets .sls files to use YAML syntax highlighting " Sets .sls files to use YAML syntax highlighting
autocmd BufNewFile,BufRead *.sls set syntax=yaml autocmd BufNewFile,BufRead *.sls set syntax=yaml
set number

202
salt/common/init.sls
View File

@@ -1,12 +1,12 @@
{% from 'allowed_states.map.jinja' import allowed_states %} {% from 'allowed_states.map.jinja' import allowed_states %}
{% if sls in allowed_states %} {% if sls in allowed_states %}
{% set role = grains.id.split('_') | last %} {% from 'vars/globals.map.jinja' import GLOBALS %}
{% from 'elasticsearch/auth.map.jinja' import ELASTICAUTH with context %}
include: include:
- common.soup_scripts - common.soup_scripts
{% if grains.role in ['so-eval', 'so-manager', 'so-standalone', 'so-managersearch', 'so-import'] %} - common.packages
{% if GLOBALS.role in GLOBALS.manager_roles %}
- manager.elasticsearch # needed for elastic_curl_config state - manager.elasticsearch # needed for elastic_curl_config state
{% endif %} {% endif %}
@@ -15,11 +15,6 @@ rmvariablesfile:
file.absent: file.absent:
- name: /tmp/variables.txt - name: /tmp/variables.txt
dockergroup:
group.present:
- name: docker
- gid: 920
# Add socore Group # Add socore Group
socoregroup: socoregroup:
group.present: group.present:
@@ -38,15 +33,15 @@ socore:
soconfperms: soconfperms:
file.directory: file.directory:
- name: /opt/so/conf - name: /opt/so/conf
- uid: 939 - user: 939
- gid: 939 - group: 939
- dir_mode: 770 - dir_mode: 770
sostatusconf: sostatusconf:
file.directory: file.directory:
- name: /opt/so/conf/so-status - name: /opt/so/conf/so-status
- uid: 939 - user: 939
- gid: 939 - group: 939
- dir_mode: 770 - dir_mode: 770
so-status.conf: so-status.conf:
@@ -57,8 +52,8 @@ so-status.conf:
sosaltstackperms: sosaltstackperms:
file.directory: file.directory:
- name: /opt/so/saltstack - name: /opt/so/saltstack
- uid: 939 - user: 939
- gid: 939 - group: 939
- dir_mode: 770 - dir_mode: 770
so_log_perms: so_log_perms:
@@ -88,92 +83,6 @@ vimconfig:
- source: salt://common/files/vimrc - source: salt://common/files/vimrc
- replace: False - replace: False
# Install common packages
{% if grains['os'] != 'CentOS' %}
commonpkgs:
pkg.installed:
- skip_suggestions: True
- pkgs:
- apache2-utils
- wget
- ntpdate
- jq
- python3-docker
- curl
- ca-certificates
- software-properties-common
- apt-transport-https
- openssl
- netcat
- python3-mysqldb
- sqlite3
- libssl-dev
- python3-dateutil
- python3-m2crypto
- python3-mysqldb
- python3-packaging
- python3-lxml
- git
- vim
heldpackages:
pkg.installed:
- pkgs:
{% if grains['oscodename'] == 'bionic' %}
- containerd.io: 1.4.4-1
- docker-ce: 5:20.10.5~3-0~ubuntu-bionic
- docker-ce-cli: 5:20.10.5~3-0~ubuntu-bionic
- docker-ce-rootless-extras: 5:20.10.5~3-0~ubuntu-bionic
{% elif grains['oscodename'] == 'focal' %}
- containerd.io: 1.4.9-1
- docker-ce: 5:20.10.8~3-0~ubuntu-focal
- docker-ce-cli: 5:20.10.5~3-0~ubuntu-focal
- docker-ce-rootless-extras: 5:20.10.5~3-0~ubuntu-focal
{% endif %}
- hold: True
- update_holds: True
{% else %}
commonpkgs:
pkg.installed:
- skip_suggestions: True
- pkgs:
- wget
- ntpdate
- bind-utils
- jq
- tcpdump
- httpd-tools
- net-tools
- curl
- sqlite
- mariadb-devel
- nmap-ncat
- python3
- python36-docker
- python36-dateutil
- python36-m2crypto
- python36-mysql
- python36-packaging
- python36-lxml
- yum-utils
- device-mapper-persistent-data
- lvm2
- openssl
- git
- vim-enhanced
heldpackages:
pkg.installed:
- pkgs:
- containerd.io: 1.4.4-3.1.el7
- docker-ce: 3:20.10.5-3.el7
- docker-ce-cli: 1:20.10.5-3.el7
- docker-ce-rootless-extras: 20.10.5-3.el7
- hold: True
- update_holds: True
{% endif %}
# Always keep these packages up to date # Always keep these packages up to date
alwaysupdated: alwaysupdated:
@@ -188,7 +97,6 @@ alwaysupdated:
Etc/UTC: Etc/UTC:
timezone.system timezone.system
{% if salt['pillar.get']('elasticsearch:auth:enabled', False) %}
elastic_curl_config: elastic_curl_config:
file.managed: file.managed:
- name: /opt/so/conf/elasticsearch/curl.config - name: /opt/so/conf/elasticsearch/curl.config
@@ -196,11 +104,10 @@ elastic_curl_config:
- mode: 600 - mode: 600
- show_changes: False - show_changes: False
- makedirs: True - makedirs: True
{% if grains.role in ['so-eval', 'so-manager', 'so-standalone', 'so-managersearch', 'so-import'] %} {% if GLOBALS.role in GLOBALS.manager_roles %}
- require: - require:
- file: elastic_curl_config_distributed - file: elastic_curl_config_distributed
{% endif %} {% endif %}
{% endif %}
# Sync some Utilities # Sync some Utilities
utilsyncscripts: utilsyncscripts:
@@ -211,17 +118,20 @@ utilsyncscripts:
- file_mode: 755 - file_mode: 755
- template: jinja - template: jinja
- source: salt://common/tools/sbin - source: salt://common/tools/sbin
- defaults:
ELASTICCURL: 'curl'
- context:
ELASTICCURL: {{ ELASTICAUTH.elasticcurl }}
- exclude_pat: - exclude_pat:
- so-common - so-common
- so-firewall - so-firewall
- so-image-common - so-image-common
- soup - soup
- so-status
{% if role in ['eval', 'standalone', 'sensor', 'heavynode'] %} so-status_script:
file.managed:
- name: /usr/sbin/so-status
- source: salt://common/tools/sbin/so-status
- mode: 755
{% if GLOBALS.role in GLOBALS.sensor_roles %}
# Add sensor cleanup # Add sensor cleanup
/usr/sbin/so-sensor-clean: /usr/sbin/so-sensor-clean:
cron.present: cron.present:
@@ -290,9 +200,17 @@ sostatus_log:
- name: /opt/so/log/sostatus/status.log - name: /opt/so/log/sostatus/status.log
- mode: 644 - mode: 644
common_pip_dependencies:
pip.installed:
- user: root
- pkgs:
- rich
- target: /usr/lib64/python3.6/site-packages
# Install sostatus check cron # Install sostatus check cron
'/usr/sbin/so-status -q; echo $? > /opt/so/log/sostatus/status.log 2>&1': sostatus_check_cron:
cron.present: cron.present:
- name: '/usr/sbin/so-status -j > /opt/so/log/sostatus/status.log 2>&1'
- user: root - user: root
- minute: '*/1' - minute: '*/1'
- hour: '*' - hour: '*'
@@ -300,36 +218,13 @@ sostatus_log:
- month: '*' - month: '*'
- dayweek: '*' - dayweek: '*'
{% if role in ['eval', 'manager', 'managersearch', 'standalone'] %} remove_post_setup_cron:
# Install cron job to determine size of influxdb for telegraf cron.absent:
'du -s -k /nsm/influxdb | cut -f1 > /opt/so/log/telegraf/influxdb_size.log 2>&1': - name: 'salt-call state.highstate'
cron.present: - identifier: post_setup_cron
- user: root
- minute: '*/1'
- hour: '*'
- daymonth: '*'
- month: '*'
- dayweek: '*'
# Lock permissions on the backup directory {% if GLOBALS.role not in ['eval', 'manager', 'managersearch', 'standalone'] %}
backupdir:
file.directory:
- name: /nsm/backup
- user: 0
- group: 0
- makedirs: True
- mode: 700
# Add config backup
/usr/sbin/so-config-backup > /dev/null 2>&1:
cron.present:
- user: root
- minute: '1'
- hour: '0'
- daymonth: '*'
- month: '*'
- dayweek: '*'
{% else %}
soversionfile: soversionfile:
file.managed: file.managed:
- name: /etc/soversion - name: /etc/soversion
@@ -339,34 +234,8 @@ soversionfile:
{% endif %} {% endif %}
# Manager daemon.json {% if GLOBALS.so_model %}
docker_daemon: {% if GLOBALS.os == 'Rocky' %}
file.managed:
- source: salt://common/files/daemon.json
- name: /etc/docker/daemon.json
- template: jinja
# Make sure Docker is always running
docker:
service.running:
- enable: True
- watch:
- file: docker_daemon
# Reserve OS ports for Docker proxy in case boot settings are not already applied/present
# 55000 = Wazuh, 57314 = Strelka, 47760-47860 = Zeek
dockerapplyports:
cmd.run:
- name: if [ ! -s /etc/sysctl.d/99-reserved-ports.conf ]; then sysctl -w net.ipv4.ip_local_reserved_ports="55000,57314,47760-47860"; fi
# Reserve OS ports for Docker proxy
dockerreserveports:
file.managed:
- source: salt://common/files/99-reserved-ports.conf
- name: /etc/sysctl.d/99-reserved-ports.conf
{% if salt['grains.get']('sosmodel', '') %}
{% if grains['os'] == 'CentOS' %}
# Install Raid tools # Install Raid tools
raidpkgs: raidpkgs:
pkg.installed: pkg.installed:
@@ -377,8 +246,9 @@ raidpkgs:
{% endif %} {% endif %}
# Install raid check cron # Install raid check cron
/usr/sbin/so-raid-status > /dev/null 2>&1: so_raid_status:
cron.present: cron.present:
- name: '/usr/sbin/so-raid-status > /dev/null 2>&1'
- user: root - user: root
- minute: '*/15' - minute: '*/15'
- hour: '*' - hour: '*'

56
salt/common/packages.sls Normal file
View File

@@ -0,0 +1,56 @@
{% from 'vars/globals.map.jinja' import GLOBALS %}
{% if GLOBALS.os == 'Ubuntu' %}
commonpkgs:
pkg.installed:
- skip_suggestions: True
- pkgs:
- chrony
- apache2-utils
- wget
- ntpdate
- jq
- python3-docker
- curl
- ca-certificates
- software-properties-common
- apt-transport-https
- openssl
- netcat
- python3-mysqldb
- sqlite3
- libssl-dev
- python3-dateutil
- python3-m2crypto
- python3-mysqldb
- python3-packaging
- python3-lxml
- git
- vim
{% elif GLOBALS.os == 'Rocky' %}
commonpkgs:
pkg.installed:
- skip_suggestions: True
- pkgs:
- wget
- jq
- tcpdump
- httpd-tools
- net-tools
- curl
- sqlite
- mariadb-devel
- python3-dnf-plugin-versionlock
- nmap-ncat
- yum-utils
- device-mapper-persistent-data
- lvm2
- openssl
- git
- python3-docker
- python3-m2crypto
- rsync
- python3-rich
- python3-watchdog
- unzip
{% endif %}

212
salt/common/tools/sbin/so-allow
View File

@@ -1,207 +1,11 @@
#!/usr/bin/env python3 #!/usr/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import ipaddress echo "Please use the Configuration section in SOC to allow hosts"
import textwrap echo ""
import os echo "If you need command line options on adding hosts please run so-firewall"
import subprocess
import sys
import argparse
import re
from lxml import etree as ET
from datetime import datetime as dt
from datetime import timezone as tz
LOCAL_SALT_DIR='/opt/so/saltstack/local'
WAZUH_CONF='/nsm/wazuh/etc/ossec.conf'
VALID_ROLES = {
'a': { 'role': 'analyst','desc': 'Analyst - 80/tcp, 443/tcp' },
'b': { 'role': 'beats_endpoint', 'desc': 'Logstash Beat - 5044/tcp' },
'e': { 'role': 'elasticsearch_rest', 'desc': 'Elasticsearch REST API - 9200/tcp' },
'f': { 'role': 'strelka_frontend', 'desc': 'Strelka frontend - 57314/tcp' },
'o': { 'role': 'osquery_endpoint', 'desc': 'Osquery endpoint - 8090/tcp' },
's': { 'role': 'syslog', 'desc': 'Syslog device - 514/tcp/udp' },
'w': { 'role': 'wazuh_agent', 'desc': 'Wazuh agent - 1514/tcp/udp' },
'p': { 'role': 'wazuh_api', 'desc': 'Wazuh API - 55000/tcp' },
'r': { 'role': 'wazuh_authd', 'desc': 'Wazuh registration service - 1515/tcp' }
}
def validate_ip_cidr(ip_cidr: str) -> bool:
try:
ipaddress.ip_address(ip_cidr)
except ValueError:
try:
ipaddress.ip_network(ip_cidr)
except ValueError:
return False
return True
def role_prompt() -> str:
print()
print('Choose the role for the IP or Range you would like to allow')
print()
for role in VALID_ROLES:
print(f'[{role}] - {VALID_ROLES[role]["desc"]}')
print()
role = input('Please enter your selection: ')
if role in VALID_ROLES.keys():
return VALID_ROLES[role]['role']
else:
print(f'Invalid role \'{role}\', please try again.', file=sys.stderr)
sys.exit(1)
def ip_prompt() -> str:
ip = input('Enter a single ip address or range to allow (ex: 10.10.10.10 or 10.10.0.0/16): ')
if validate_ip_cidr(ip):
return ip
else:
print(f'Invalid IP address or CIDR block \'{ip}\', please try again.', file=sys.stderr)
sys.exit(1)
def wazuh_enabled() -> bool:
file = f'{LOCAL_SALT_DIR}/pillar/global.sls'
with open(file, 'r') as pillar:
if 'wazuh: 1' in pillar.read():
return True
return False
def root_to_str(root: ET.ElementTree) -> str:
return ET.tostring(root, encoding='unicode', method='xml', xml_declaration=False, pretty_print=True)
def add_wl(ip):
parser = ET.XMLParser(remove_blank_text=True)
with open(WAZUH_CONF, 'rb') as wazuh_conf:
tree = ET.parse(wazuh_conf, parser)
root = tree.getroot()
source_comment = ET.Comment(f'Address {ip} added by /usr/sbin/so-allow on {dt.utcnow().replace(tzinfo=tz.utc).strftime("%a %b %e %H:%M:%S %Z %Y")}')
new_global = ET.Element("global")
new_wl = ET.SubElement(new_global, 'white_list')
new_wl.text = ip
root.append(source_comment)
root.append(new_global)
with open(WAZUH_CONF, 'w') as add_out:
add_out.write(root_to_str(root))
def apply(role: str, ip: str) -> int:
firewall_cmd = ['so-firewall', 'includehost', role, ip]
salt_cmd = ['salt-call', 'state.apply', '-l', 'quiet', 'firewall', 'queue=True']
restart_wazuh_cmd = ['so-wazuh-restart']
print(f'Adding {ip} to the {role} role. This can take a few seconds...')
cmd = subprocess.run(firewall_cmd)
if cmd.returncode == 0:
cmd = subprocess.run(salt_cmd, stdout=subprocess.DEVNULL)
else:
return cmd.returncode
if cmd.returncode == 0:
if wazuh_enabled() and role=='analyst':
try:
add_wl(ip)
print(f'Added whitelist entry for {ip} from {WAZUH_CONF}', file=sys.stderr)
except Exception as e:
print(f'Failed to add whitelist entry for {ip} from {WAZUH_CONF}', file=sys.stderr)
print(e)
return 1
print('Restarting OSSEC Server...')
cmd = subprocess.run(restart_wazuh_cmd)
else:
return cmd.returncode
else:
print(f'Commmand \'{" ".join(salt_cmd)}\' failed.', file=sys.stderr)
return cmd.returncode
if cmd.returncode != 0:
print('Failed to restart OSSEC server.')
return cmd.returncode
def main():
if os.geteuid() != 0:
print('You must run this script as root', file=sys.stderr)
sys.exit(1)
main_parser = argparse.ArgumentParser(
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=textwrap.dedent(f'''\
additional information:
To use this script in interactive mode call it with no arguments
'''
))
group = main_parser.add_argument_group(title='roles')
group.add_argument('-a', dest='roles', action='append_const', const=VALID_ROLES['a']['role'], help="Analyst - 80/tcp, 443/tcp")
group.add_argument('-b', dest='roles', action='append_const', const=VALID_ROLES['b']['role'], help="Logstash Beat - 5044/tcp")
group.add_argument('-e', dest='roles', action='append_const', const=VALID_ROLES['e']['role'], help="Elasticsearch REST API - 9200/tcp")
group.add_argument('-f', dest='roles', action='append_const', const=VALID_ROLES['f']['role'], help="Strelka frontend - 57314/tcp")
group.add_argument('-o', dest='roles', action='append_const', const=VALID_ROLES['o']['role'], help="Osquery endpoint - 8090/tcp")
group.add_argument('-s', dest='roles', action='append_const', const=VALID_ROLES['s']['role'], help="Syslog device - 514/tcp/udp")
group.add_argument('-w', dest='roles', action='append_const', const=VALID_ROLES['w']['role'], help="Wazuh agent - 1514/tcp/udp")
group.add_argument('-p', dest='roles', action='append_const', const=VALID_ROLES['p']['role'], help="Wazuh API - 55000/tcp")
group.add_argument('-r', dest='roles', action='append_const', const=VALID_ROLES['r']['role'], help="Wazuh registration service - 1515/tcp")
ip_g = main_parser.add_argument_group(title='allow')
ip_g.add_argument('-i', help="IP or CIDR block to disallow connections from, requires at least one role argument", metavar='', dest='ip')
args = main_parser.parse_args(sys.argv[1:])
if args.roles is None:
role = role_prompt()
ip = ip_prompt()
try:
return_code = apply(role, ip)
except Exception as e:
print(f'Unexpected exception occurred: {e}', file=sys.stderr)
return_code = e.errno
sys.exit(return_code)
elif args.roles is not None and args.ip is None:
if os.environ.get('IP') is None:
main_parser.print_help()
sys.exit(1)
else:
args.ip = os.environ['IP']
if validate_ip_cidr(args.ip):
try:
for role in args.roles:
return_code = apply(role, args.ip)
if return_code > 0:
break
except Exception as e:
print(f'Unexpected exception occurred: {e}', file=sys.stderr)
return_code = e.errno
else:
print(f'Invalid IP address or CIDR block \'{args.ip}\', please try again.', file=sys.stderr)
return_code = 1
sys.exit(return_code)
if __name__ == '__main__':
try:
main()
except KeyboardInterrupt:
sys.exit(1)

20
salt/common/tools/sbin/so-allow-view
View File

@@ -1,19 +1,11 @@
#!/bin/bash #!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common . /usr/sbin/so-common

29
salt/common/tools/sbin/so-analyst-install
View File

@@ -1,27 +1,18 @@
#!/bin/bash #!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
doc_workstation_url="https://docs.securityonion.net/en/2.3/analyst-vm.html" {# we only want the script to install the workstation if it is Rocky -#}
{# we only want the script to install the workstation if it is CentOS -#} {% if grains.os == 'Rocky' -%}
{% if grains.os == 'CentOS' -%}
{# if this is a manager -#} {# if this is a manager -#}
{% if grains.master == grains.id.split('_')|first -%} {% if grains.master == grains.id.split('_')|first -%}
source /usr/sbin/so-common source /usr/sbin/so-common
doc_workstation_url="$DOC_BASE_URL/analyst-vm.html"
pillar_file="/opt/so/saltstack/local/pillar/minions/{{grains.id}}.sls" pillar_file="/opt/so/saltstack/local/pillar/minions/{{grains.id}}.sls"
if [ -f "$pillar_file" ]; then if [ -f "$pillar_file" ]; then
@@ -89,12 +80,12 @@ echo "Since this is not a manager, the pillar values to enable analyst workstati
{#- endif if this is a manager #} {#- endif if this is a manager #}
{% endif -%} {% endif -%}
{#- if not CentOS #} {#- if not Rocky #}
{%- else %} {%- else %}
echo "The Analyst Workstation can only be installed on CentOS. Please view the documentation at $doc_workstation_url." echo "The Analyst Workstation can only be installed on Rocky. Please view the documentation at $doc_workstation_url."
{#- endif grains.os == CentOS #} {#- endif grains.os == Rocky #}
{% endif -%} {% endif -%}
exit 0 exit 0

20
salt/common/tools/sbin/so-checkin
View File

@@ -1,19 +1,11 @@
#!/bin/bash #!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common . /usr/sbin/so-common

103
salt/common/tools/sbin/so-common
View File

@@ -1,26 +1,24 @@
#!/bin/bash #!/bin/bash
# #
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
DEFAULT_SALT_DIR=/opt/so/saltstack/default DEFAULT_SALT_DIR=/opt/so/saltstack/default
DOC_BASE_URL="https://docs.securityonion.net/en/2.4"
# Check for prerequisites if [ -z $NOROOT ]; then
if [ "$(id -u)" -ne 0 ]; then # Check for prerequisites
echo "This script must be run using sudo!" if [ "$(id -u)" -ne 0 ]; then
exit 1 echo "This script must be run using sudo!"
exit 1
fi
fi
# Ensure /usr/sbin is in path
if ! echo "$PATH" | grep -q "/usr/sbin"; then
export PATH="$PATH:/usr/sbin"
fi fi
# Define a banner to separate sections # Define a banner to separate sections
@@ -162,15 +160,12 @@ elastic_license() {
read -r -d '' message <<- EOM read -r -d '' message <<- EOM
\n \n
Starting in Elastic Stack version 7.11, the Elastic Stack binaries are only available under the Elastic License: Elastic Stack binaries and Security Onion components are only available under the Elastic License version 2 (ELv2):
https://securityonion.net/elastic-license https://securityonion.net/license/
Please review the Elastic License: Do you agree to the terms of ELv2?
https://www.elastic.co/licensing/elastic-license
Do you agree to the terms of the Elastic License? If so, type AGREE to accept ELv2 and continue. Otherwise, press Enter to exit this program without making any changes.
If so, type AGREE to accept the Elastic License and continue. Otherwise, press Enter to exit this program without making any changes.
EOM EOM
AGREED=$(whiptail --title "$whiptail_title" --inputbox \ AGREED=$(whiptail --title "$whiptail_title" --inputbox \
@@ -199,14 +194,14 @@ get_random_value() {
} }
gpg_rpm_import() { gpg_rpm_import() {
if [[ "$OS" == "centos" ]]; then if [[ "$OS" == "rocky" ]]; then
if [[ "$WHATWOULDYOUSAYYAHDOHERE" == "setup" ]]; then if [[ "$WHATWOULDYOUSAYYAHDOHERE" == "setup" ]]; then
local RPMKEYSLOC="../salt/repo/client/files/centos/keys" local RPMKEYSLOC="../salt/repo/client/files/rocky/keys"
else else
local RPMKEYSLOC="$UPDATE_DIR/salt/repo/client/files/centos/keys" local RPMKEYSLOC="$UPDATE_DIR/salt/repo/client/files/rocky/keys"
fi fi
RPMKEYS=('RPM-GPG-KEY-EPEL-7' 'GPG-KEY-WAZUH' 'docker.pub' 'SALTSTACK-GPG-KEY.pub' 'securityonion.pub') RPMKEYS=('RPM-GPG-KEY-EPEL-9' 'SALTSTACK-GPG-KEY2.pub' 'docker.pub' 'securityonion.pub')
for RPMKEY in "${RPMKEYS[@]}"; do for RPMKEY in "${RPMKEYS[@]}"; do
rpm --import $RPMKEYSLOC/$RPMKEY rpm --import $RPMKEYSLOC/$RPMKEY
@@ -237,31 +232,17 @@ init_monitor() {
} }
is_manager_node() { is_manager_node() {
# Check to see if this is a manager node grep "role: so-" /etc/salt/grains | grep -E "manager|eval|managersearch|standalone|import" &> /dev/null
role=$(lookup_role)
is_single_node_grid && return 0
[ $role == 'manager' ] && return 0
[ $role == 'managersearch' ] && return 0
[ $role == 'helix' ] && return 0
return 1
} }
is_sensor_node() { is_sensor_node() {
# Check to see if this is a sensor (forward) node # Check to see if this is a sensor (forward) node
role=$(lookup_role)
is_single_node_grid && return 0 is_single_node_grid && return 0
[ $role == 'sensor' ] && return 0 grep "role: so-" /etc/salt/grains | grep -E "sensor|heavynode|helix" &> /dev/null
[ $role == 'heavynode' ] && return 0
[ $role == 'helix' ] && return 0
return 1
} }
is_single_node_grid() { is_single_node_grid() {
role=$(lookup_role) grep "role: so-" /etc/salt/grains | grep -E "eval|standalone|import" &> /dev/null
[ $role == 'eval' ] && return 0
[ $role == 'standalone' ] && return 0
[ $role == 'import' ] && return 0
return 1
} }
lookup_bond_interfaces() { lookup_bond_interfaces() {
@@ -392,17 +373,23 @@ run_check_net_err() {
fi fi
} }
salt_minion_count() {
local MINIONDIR="/opt/so/saltstack/local/pillar/minions"
MINIONCOUNT=$(ls -la $MINIONDIR/*.sls | grep -v adv_ | wc -l)
}
set_cron_service_name() { set_cron_service_name() {
if [[ "$OS" == "centos" ]]; then if [[ "$OS" == "rocky" ]]; then
cron_service_name="crond" cron_service_name="crond"
else else
cron_service_name="cron" cron_service_name="cron"
fi fi
} }
set_os() { set_os() {
if [ -f /etc/redhat-release ]; then if [ -f /etc/redhat-release ]; then
OS=centos OS=rocky
else else
OS=ubuntu OS=ubuntu
fi fi
@@ -518,6 +505,18 @@ valid_hostname() {
[[ $hostname =~ ^[a-zA-Z0-9\-]+$ ]] && [[ $hostname != 'localhost' ]] && return 0 || return 1 [[ $hostname =~ ^[a-zA-Z0-9\-]+$ ]] && [[ $hostname != 'localhost' ]] && return 0 || return 1
} }
verify_ip4() {
local ip=$1
# Is this an IP or CIDR?
if grep -qP "^[^/]+/[^/]+$" <<< $ip; then
# Looks like a CIDR
valid_ip4_cidr_mask "$ip"
else
# We know this is not a CIDR - Is it an IP?
valid_ip4 "$ip"
fi
}
valid_ip4() { valid_ip4() {
local ip=$1 local ip=$1

48
salt/common/tools/sbin/so-config-backup
View File

@@ -1,48 +0,0 @@
#!/bin/bash
#
# Copyright 2014-2022 Security Onion Solutions, LLC
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.. /usr/sbin/so-common
{% set BACKUPLOCATIONS = salt['pillar.get']('backup:locations', {}) %}
TODAY=$(date '+%Y_%m_%d')
BACKUPFILE="/nsm/backup/so-config-backup-$TODAY.tar"
MAXBACKUPS=7
# Create backup dir if it does not exist
mkdir -p /nsm/backup
# If we haven't already written a backup file for today, let's do so
if [ ! -f $BACKUPFILE ]; then
# Create empty backup file
tar -cf $BACKUPFILE -T /dev/null
# Loop through all paths defined in global.sls, and append them to backup file
{%- for LOCATION in BACKUPLOCATIONS %}
tar -rf $BACKUPFILE {{ LOCATION }}
{%- endfor %}
tar -rf $BACKUPFILE /etc/pki
tar -rf $BACKUPFILE /etc/salt
tar -rf $BACKUPFILE /opt/so/conf/kratos
fi
# Find oldest backup files and remove them
NUMBACKUPS=$(find /nsm/backup/ -type f -name "so-config-backup*" | wc -l)
while [ "$NUMBACKUPS" -gt "$MAXBACKUPS" ]; do
OLDESTBACKUP=$(find /nsm/backup/ -type f -name "so-config-backup*" -type f -printf '%T+ %p\n' | sort | head -n 1 | awk -F" " '{print $2}')
rm -f $OLDESTBACKUP
NUMBACKUPS=$(find /nsm/backup/ -type f -name "so-config-backup*" | wc -l)
done

20
salt/common/tools/sbin/so-cortex-restart
View File

@@ -1,20 +0,0 @@
#!/bin/bash
#
# Copyright 2014-2022 Security Onion Solutions, LLC
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common
echo "TheHive and its components are no longer part of Security Onion"

20
salt/common/tools/sbin/so-cortex-start
View File

@@ -1,20 +0,0 @@
#!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common
echo "TheHive and its components are no longer part of Security Onion"

20
salt/common/tools/sbin/so-cortex-stop
View File

@@ -1,20 +0,0 @@
#!/bin/bash
#
# Copyright 2014-2022 Security Onion Solutions, LLC
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common
echo "TheHive and its components are no longer part of Security Onion"

20
salt/common/tools/sbin/so-cortex-user-add
View File

@@ -1,20 +0,0 @@
#!/bin/bash
#
# Copyright 2014-2022 Security Onion Solutions, LLC
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common
echo "TheHive and its components are no longer part of Security Onion"

20
salt/common/tools/sbin/so-cortex-user-enable
View File

@@ -1,20 +0,0 @@
#!/bin/bash
#
# Copyright 2014-2022 Security Onion Solutions, LLC
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common
echo "TheHive and its components are no longer part of Security Onion"

20
salt/common/tools/sbin/so-curator-restart
View File

@@ -1,19 +1,11 @@
#!/bin/bash #!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common . /usr/sbin/so-common

20
salt/common/tools/sbin/so-curator-start
View File

@@ -1,19 +1,11 @@
#!/bin/bash #!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common . /usr/sbin/so-common

20
salt/common/tools/sbin/so-curator-stop
View File

@@ -1,19 +1,11 @@
#!/bin/bash #!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common . /usr/sbin/so-common

87
salt/common/tools/sbin/so-deny
View File

@@ -1,19 +1,11 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import ipaddress import ipaddress
import textwrap import textwrap
@@ -27,17 +19,12 @@ from xml.dom import minidom
LOCAL_SALT_DIR='/opt/so/saltstack/local' LOCAL_SALT_DIR='/opt/so/saltstack/local'
WAZUH_CONF='/nsm/wazuh/etc/ossec.conf'
VALID_ROLES = { VALID_ROLES = {
'a': { 'role': 'analyst','desc': 'Analyst - 80/tcp, 443/tcp' }, 'a': { 'role': 'analyst','desc': 'Analyst - 80/tcp, 443/tcp' },
'b': { 'role': 'beats_endpoint', 'desc': 'Logstash Beat - 5044/tcp' }, 'b': { 'role': 'beats_endpoint', 'desc': 'Logstash Beat - 5044/tcp' },
'e': { 'role': 'elasticsearch_rest', 'desc': 'Elasticsearch REST API - 9200/tcp' }, 'e': { 'role': 'elasticsearch_rest', 'desc': 'Elasticsearch REST API - 9200/tcp' },
'f': { 'role': 'strelka_frontend', 'desc': 'Strelka frontend - 57314/tcp' }, 'f': { 'role': 'strelka_frontend', 'desc': 'Strelka frontend - 57314/tcp' },
'o': { 'role': 'osquery_endpoint', 'desc': 'Osquery endpoint - 8090/tcp' },
's': { 'role': 'syslog', 'desc': 'Syslog device - 514/tcp/udp' }, 's': { 'role': 'syslog', 'desc': 'Syslog device - 514/tcp/udp' },
'w': { 'role': 'wazuh_agent', 'desc': 'Wazuh agent - 1514/tcp/udp' },
'p': { 'role': 'wazuh_api', 'desc': 'Wazuh API - 55000/tcp' },
'r': { 'role': 'wazuh_authd', 'desc': 'Wazuh registration service - 1515/tcp' }
} }
@@ -76,73 +63,15 @@ def ip_prompt() -> str:
sys.exit(1) sys.exit(1)
def wazuh_enabled() -> bool:
for file in os.listdir(f'{LOCAL_SALT_DIR}/pillar'):
with open(file, 'r') as pillar:
if 'wazuh: 1' in pillar.read():
return True
return False
def root_to_str(root: ET.ElementTree) -> str:
xml_str = ET.tostring(root, encoding='unicode', method='xml').replace('\n', '')
xml_str = re.sub(r'(?:(?<=>) *)', '', xml_str)
# Remove specific substrings to better format comments on intial parse/write
xml_str = re.sub(r' -', '', xml_str)
xml_str = re.sub(r' -->', ' -->', xml_str)
dom = minidom.parseString(xml_str)
return dom.toprettyxml(indent=" ")
def rem_wl(ip):
parser = ET.XMLParser(remove_blank_text=True)
with open(WAZUH_CONF, 'rb') as wazuh_conf:
tree = ET.parse(wazuh_conf, parser)
root = tree.getroot()
global_elems = root.findall(f"global/white_list[. = '{ip}']/..")
if len(global_elems) > 0:
for g_elem in global_elems:
ge_index = list(root).index(g_elem)
if ge_index > 0 and root[list(root).index(g_elem) - 1].tag == ET.Comment:
root.remove(root[ge_index - 1])
root.remove(g_elem)
with open(WAZUH_CONF, 'w') as out:
out.write(root_to_str(root))
def apply(role: str, ip: str) -> int: def apply(role: str, ip: str) -> int:
firewall_cmd = ['so-firewall', 'excludehost', role, ip] firewall_cmd = ['so-firewall', 'excludehost', role, ip]
salt_cmd = ['salt-call', 'state.apply', '-l', 'quiet', 'firewall', 'queue=True'] salt_cmd = ['salt-call', 'state.apply', '-l', 'quiet', 'firewall', 'queue=True']
restart_wazuh_cmd = ['so-wazuh-restart']
print(f'Removing {ip} from the {role} role. This can take a few seconds...') print(f'Removing {ip} from the {role} role. This can take a few seconds...')
cmd = subprocess.run(firewall_cmd) cmd = subprocess.run(firewall_cmd)
if cmd.returncode == 0: if cmd.returncode == 0:
cmd = subprocess.run(salt_cmd, stdout=subprocess.DEVNULL) cmd = subprocess.run(salt_cmd, stdout=subprocess.DEVNULL)
else: else:
return cmd.returncode return cmd.returncode
if cmd.returncode == 0:
if wazuh_enabled and role=='analyst':
try:
rem_wl(ip)
print(f'Removed whitelist entry for {ip} from {WAZUH_CONF}', file=sys.stderr)
except Exception as e:
print(f'Failed to remove whitelist entry for {ip} from {WAZUH_CONF}', file=sys.stderr)
print(e)
return 1
print('Restarting OSSEC Server...')
cmd = subprocess.run(restart_wazuh_cmd)
else:
return cmd.returncode
else:
print(f'Commmand \'{" ".join(salt_cmd)}\' failed.', file=sys.stderr)
return cmd.returncode
if cmd.returncode != 0:
print('Failed to restart OSSEC server.')
return cmd.returncode
def main(): def main():
@@ -163,11 +92,7 @@ def main():
group.add_argument('-b', dest='roles', action='append_const', const=VALID_ROLES['b']['role'], help="Logstash Beat - 5044/tcp") group.add_argument('-b', dest='roles', action='append_const', const=VALID_ROLES['b']['role'], help="Logstash Beat - 5044/tcp")
group.add_argument('-e', dest='roles', action='append_const', const=VALID_ROLES['e']['role'], help="Elasticsearch REST API - 9200/tcp") group.add_argument('-e', dest='roles', action='append_const', const=VALID_ROLES['e']['role'], help="Elasticsearch REST API - 9200/tcp")
group.add_argument('-f', dest='roles', action='append_const', const=VALID_ROLES['f']['role'], help="Strelka frontend - 57314/tcp") group.add_argument('-f', dest='roles', action='append_const', const=VALID_ROLES['f']['role'], help="Strelka frontend - 57314/tcp")
group.add_argument('-o', dest='roles', action='append_const', const=VALID_ROLES['o']['role'], help="Osquery endpoint - 8090/tcp")
group.add_argument('-s', dest='roles', action='append_const', const=VALID_ROLES['s']['role'], help="Syslog device - 514/tcp/udp") group.add_argument('-s', dest='roles', action='append_const', const=VALID_ROLES['s']['role'], help="Syslog device - 514/tcp/udp")
group.add_argument('-w', dest='roles', action='append_const', const=VALID_ROLES['w']['role'], help="Wazuh agent - 1514/tcp/udp")
group.add_argument('-p', dest='roles', action='append_const', const=VALID_ROLES['p']['role'], help="Wazuh API - 55000/tcp")
group.add_argument('-r', dest='roles', action='append_const', const=VALID_ROLES['r']['role'], help="Wazuh registration service - 1515/tcp")
ip_g = main_parser.add_argument_group(title='allow') ip_g = main_parser.add_argument_group(title='allow')
ip_g.add_argument('-i', help="IP or CIDR block to disallow connections from, requires at least one role argument", metavar='', dest='ip') ip_g.add_argument('-i', help="IP or CIDR block to disallow connections from, requires at least one role argument", metavar='', dest='ip')

20
salt/common/tools/sbin/so-docker-prune
View File

@@ -1,19 +1,11 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import sys, argparse, re, docker import sys, argparse, re, docker
from packaging.version import Version, InvalidVersion from packaging.version import Version, InvalidVersion

20
salt/common/tools/sbin/so-docker-refresh
View File

@@ -1,19 +1,11 @@
#!/bin/bash #!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common . /usr/sbin/so-common
. /usr/sbin/so-image-common . /usr/sbin/so-image-common

20
salt/common/tools/sbin/so-elastalert-restart
View File

@@ -1,19 +1,11 @@
#!/bin/bash #!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common . /usr/sbin/so-common

20
salt/common/tools/sbin/so-elastalert-start
View File

@@ -1,19 +1,11 @@
#!/bin/bash #!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common . /usr/sbin/so-common

20
salt/common/tools/sbin/so-elastalert-stop
View File

@@ -1,19 +1,11 @@
#!/bin/bash #!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common . /usr/sbin/so-common

34
salt/common/tools/sbin/so-elastic-agent-gen-installers Executable file
View File

@@ -0,0 +1,34 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0; you may not use
# this file except in compliance with the Elastic License 2.0.
#so-elastic-agent-gen-installers $FleetHost $EnrollmentToken
{% from 'vars/globals.map.jinja' import GLOBALS %}
. /usr/sbin/so-common
ENROLLMENTOKEN=$(curl -K /opt/so/conf/elasticsearch/curl.config -L "localhost:5601/api/fleet/enrollment_api_keys" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' | jq .list | jq -r -c '.[] | select(.policy_id | contains("endpoints")) | .api_key')
FLEETHOST=$(lookup_pillar "server:url" "elasticfleet")
#FLEETHOST=$1
#ENROLLMENTOKEN=$2
CONTAINERGOOS=( "linux" "darwin" "windows" )
rm -rf /tmp/elastic-agent-workspace
mkdir -p /tmp/elastic-agent-workspace
for OS in "${CONTAINERGOOS[@]}"
do
printf "\n\nGenerating $OS Installer..."
cp /opt/so/saltstack/default/salt/elasticfleet/files/elastic-agent/so-elastic-agent-*-$OS-x86_64.tar.gz /tmp/elastic-agent-workspace/$OS.tar.gz
docker run -e CGO_ENABLED=0 -e GOOS=$OS \
--mount type=bind,source=/etc/ssl/certs/,target=/workspace/files/cert/ \
--mount type=bind,source=/tmp/elastic-agent-workspace/,target=/workspace/files/elastic-agent/ \
--mount type=bind,source=/opt/so/saltstack/local/salt/elasticfleet/files/so_agent-installers/,target=/output/ \
{{ GLOBALS.registry_host }}:5000/{{ GLOBALS.image_repo }}/so-elastic-agent-builder:{{ GLOBALS.so_version }} go build -ldflags "-X main.fleetHost=$FLEETHOST -X main.enrollmentToken=$ENROLLMENTOKEN" -o /output/so-elastic-agent_$OS
printf "\n $OS Installer Generated..."
done

67
salt/common/tools/sbin/so-elastic-auth
View File

@@ -1,67 +0,0 @@
#!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
if [ -f "/usr/sbin/so-common" ]; then
. /usr/sbin/so-common
fi
ES_AUTH_PILLAR=${ELASTIC_AUTH_PILLAR:-/opt/so/saltstack/local/pillar/elasticsearch/auth.sls}
ES_USERS_FILE=${ELASTIC_USERS_FILE:-/opt/so/saltstack/local/salt/elasticsearch/files/users}
authEnable=$1
if ! grep -q "enabled: " "$ES_AUTH_PILLAR"; then
echo "Elastic auth pillar file is invalid. Unable to proceed."
exit 1
fi
function restart() {
if [[ -z "$ELASTIC_AUTH_SKIP_HIGHSTATE" ]]; then
echo "Elasticsearch on all affected minions will now be stopped and then restarted..."
salt -C 'G@role:so-standalone or G@role:so-eval or G@role:so-import or G@role:so-manager or G@role:so-managersearch or G@role:so-node or G@role:so-heavynode' cmd.run so-elastic-stop queue=True
echo "Applying highstate to all affected minions..."
salt -C 'G@role:so-standalone or G@role:so-eval or G@role:so-import or G@role:so-manager or G@role:so-managersearch or G@role:so-node or G@role:so-heavynode' state.highstate queue=True
fi
}
if [[ "$authEnable" == "true" ]]; then
if grep -q "enabled: False" "$ES_AUTH_PILLAR"; then
sed -i 's/enabled: False/enabled: True/g' "$ES_AUTH_PILLAR"
restart
echo "Elastic auth is now enabled."
if grep -q "argon" "$ES_USERS_FILE"; then
echo ""
echo "IMPORTANT: The following users will need to change their password, after logging into SOC, in order to access Kibana:"
grep argon "$ES_USERS_FILE" | cut -d ":" -f 1
fi
else
echo "Auth is already enabled."
fi
elif [[ "$authEnable" == "false" ]]; then
if grep -q "enabled: True" "$ES_AUTH_PILLAR"; then
sed -i 's/enabled: True/enabled: False/g' "$ES_AUTH_PILLAR"
restart
echo "Elastic auth is now disabled."
else
echo "Auth is already disabled."
fi
else
echo "Usage: $0 <true|false>"
echo ""
echo "Toggles Elastic authentication. Elasticsearch will be restarted on each affected minion."
echo ""
fi

27
salt/common/tools/sbin/so-elastic-auth-password-reset Normal file → Executable file
View File

@@ -1,19 +1,10 @@
#!/bin/bash #!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
source $(dirname $0)/so-common source $(dirname $0)/so-common
require_manager require_manager
@@ -98,18 +89,16 @@ function killAllSaltJobs() {
function soUserSync() { function soUserSync() {
# apply this state to update /opt/so/saltstack/local/salt/elasticsearch/curl.config on the manager # apply this state to update /opt/so/saltstack/local/salt/elasticsearch/curl.config on the manager
salt-call state.sls_id elastic_curl_config_distributed manager queue=True salt-call state.sls_id elastic_curl_config_distributed manager queue=True
salt -C 'G@role:so-standalone or G@role:so-eval or G@role:so-import or G@role:so-manager or G@role:so-managersearch or G@role:so-node or G@role:so-heavynode' saltutil.kill_all_jobs salt -C 'G@role:so-standalone or G@role:so-eval or G@role:so-import or G@role:so-manager or G@role:so-managersearch or G@role:so-searchnode or G@role:so-heavynode' saltutil.kill_all_jobs
# apply this state to get the curl.config # apply this state to get the curl.config
salt -C 'G@role:so-standalone or G@role:so-eval or G@role:so-import or G@role:so-manager or G@role:so-managersearch or G@role:so-node or G@role:so-heavynode' state.sls_id elastic_curl_config common queue=True salt -C 'G@role:so-standalone or G@role:so-eval or G@role:so-import or G@role:so-manager or G@role:so-managersearch or G@role:so-searchnode or G@role:so-heavynode' state.sls_id elastic_curl_config common queue=True
$(dirname $0)/so-user sync $(dirname $0)/so-user sync
printf "\nApplying logstash state to the appropriate nodes.\n\n" printf "\nApplying logstash state to the appropriate nodes.\n\n"
salt -C 'G@role:so-standalone or G@role:so-eval or G@role:so-import or G@role:so-manager or G@role:so-managersearch or G@role:so-node or G@role:so-heavynode' state.apply logstash queue=True salt -C 'G@role:so-standalone or G@role:so-eval or G@role:so-import or G@role:so-manager or G@role:so-managersearch or G@role:so-searchnode or G@role:so-heavynode' state.apply logstash queue=True
printf "\nApplying filebeat state to the appropriate nodes.\n\n"
salt -C 'G@role:so-standalone or G@role:so-eval or G@role:so-import or G@role:so-manager or G@role:so-managersearch or G@role:so-node or G@role:so-heavynode or G@role:so-sensor or G@role:so-fleet' state.apply filebeat queue=True
printf "\nApplying kibana state to the appropriate nodes.\n\n" printf "\nApplying kibana state to the appropriate nodes.\n\n"
salt -C 'G@role:so-standalone or G@role:so-eval or G@role:so-import or G@role:so-manager or G@role:so-managersearch' state.apply kibana queue=True salt -C 'G@role:so-standalone or G@role:so-eval or G@role:so-import or G@role:so-manager or G@role:so-managersearch' state.apply kibana queue=True
printf "\nApplying curator state to the appropriate nodes.\n\n" printf "\nApplying curator state to the appropriate nodes.\n\n"
salt -C 'G@role:so-standalone or G@role:so-eval or G@role:so-import or G@role:so-manager or G@role:so-managersearch or G@role:so-node or G@role:so-heavynode' state.apply curator queue=True salt -C 'G@role:so-standalone or G@role:so-eval or G@role:so-import or G@role:so-manager or G@role:so-managersearch or G@role:so-searchnode or G@role:so-heavynode' state.apply curator queue=True
} }
function highstateManager() { function highstateManager() {

44
salt/common/tools/sbin/so-elastic-clear
View File

@@ -1,20 +1,11 @@
#!/bin/bash #!/bin/bash
# #
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version. {%- set NODEIP = salt['pillar.get']('host:mainip', '') %}
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
{%- set NODEIP = salt['pillar.get']('elasticsearch:mainip', '') -%}
. /usr/sbin/so-common . /usr/sbin/so-common
SKIP=0 SKIP=0
@@ -50,7 +41,7 @@ done
if [ $SKIP -ne 1 ]; then if [ $SKIP -ne 1 ]; then
# List indices # List indices
echo echo
{{ ELASTICCURL }} -k -L https://{{ NODEIP }}:9200/_cat/indices?v curl -K /opt/so/conf/elasticsearch/curl.config -k -L https://{{ NODEIP }}:9200/_cat/indices?v
echo echo
# Inform user we are about to delete all data # Inform user we are about to delete all data
echo echo
@@ -63,17 +54,10 @@ if [ $SKIP -ne 1 ]; then
if [ "$INPUT" != "AGREE" ] ; then exit 0; fi if [ "$INPUT" != "AGREE" ] ; then exit 0; fi
fi fi
# Check to see if Logstash/Filebeat are running # Check to see if Logstash are running
LS_ENABLED=$(so-status | grep logstash) LS_ENABLED=$(so-status | grep logstash)
FB_ENABLED=$(so-status | grep filebeat)
EA_ENABLED=$(so-status | grep elastalert) EA_ENABLED=$(so-status | grep elastalert)
if [ ! -z "$FB_ENABLED" ]; then
/usr/sbin/so-filebeat-stop
fi
if [ ! -z "$LS_ENABLED" ]; then if [ ! -z "$LS_ENABLED" ]; then
/usr/sbin/so-logstash-stop /usr/sbin/so-logstash-stop
@@ -89,19 +73,13 @@ fi
# Delete data # Delete data
echo "Deleting data..." echo "Deleting data..."
INDXS=$({{ ELASTICCURL }} -s -XGET -k -L https://{{ NODEIP }}:9200/_cat/indices?v | egrep 'logstash|elastalert|so-' | awk '{ print $3 }') INDXS=$(curl -K /opt/so/conf/elasticsearch/curl.config -s -XGET -k -L https://{{ NODEIP }}:9200/_cat/indices?v | egrep 'logstash|elastalert|so-' | awk '{ print $3 }')
for INDX in ${INDXS} for INDX in ${INDXS}
do do
{{ ELASTICCURL }} -XDELETE -k -L https://"{{ NODEIP }}:9200/${INDX}" > /dev/null 2>&1 curl -K /opt/so/conf/elasticsearch/curl.config-XDELETE -k -L https://"{{ NODEIP }}:9200/${INDX}" > /dev/null 2>&1
done done
#Start Logstash/Filebeat #Start Logstash
if [ ! -z "$FB_ENABLED" ]; then
/usr/sbin/so-filebeat-start
fi
if [ ! -z "$LS_ENABLED" ]; then if [ ! -z "$LS_ENABLED" ]; then
/usr/sbin/so-logstash-start /usr/sbin/so-logstash-start

20
salt/common/tools/sbin/so-elastic-diagnose
View File

@@ -1,19 +1,11 @@
#!/bin/bash #!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Source common settings # Source common settings
. /usr/sbin/so-common . /usr/sbin/so-common

19
salt/common/tools/sbin/so-elastic-fleet-agent-policy-delete Executable file
View File

@@ -0,0 +1,19 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
POLICY_ID=$1
# Let's snag a cookie from Kibana
SESSIONCOOKIE=$(curl -K /opt/so/conf/elasticsearch/curl.config -c - -X GET http://localhost:5601/ | grep sid | awk '{print $7}')
echo "Deleting agent policy $POLICY_ID..."
# Delete agent policy
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/agent_policies/delete" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d"{\"agentPolicyId\": \"$POLICY_ID\"}"
echo

17
salt/common/tools/sbin/so-elastic-fleet-agent-policy-list Executable file
View File

@@ -0,0 +1,17 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
# Let's snag a cookie from Kibana
SESSIONCOOKIE=$(curl -K /opt/so/conf/elasticsearch/curl.config -c - -X GET http://localhost:5601/ | grep sid | awk '{print $7}')
echo "Setting up default Security Onion package policies for Elastic Agent..."
# List configured agent policies
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X GET "localhost:5601/api/fleet/agent_policies" | jq
echo

19
salt/common/tools/sbin/so-elastic-fleet-agent-policy-view Executable file
View File

@@ -0,0 +1,19 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
POLICY_ID=$1
# Let's snag a cookie from Kibana
SESSIONCOOKIE=$(curl -K /opt/so/conf/elasticsearch/curl.config -c - -X GET http://localhost:5601/ | grep sid | awk '{print $7}')
echo "Viewing agent policy $POLICY_ID"
# View agent policy
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X GET "localhost:5601/api/fleet/agent_policies/$POLICY_ID/full" | jq
echo

17
salt/common/tools/sbin/so-elastic-fleet-data-streams-list Executable file
View File

@@ -0,0 +1,17 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
# Let's snag a cookie from Kibana
SESSIONCOOKIE=$(curl -K /opt/so/conf/elasticsearch/curl.config -c - -X GET http://localhost:5601/ | grep sid | awk '{print $7}')
echo "Retrieving data stream information..."
# Retrieve data stream information
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X GET "localhost:5601/api/fleet/data_streams" | jq
echo

23
salt/common/tools/sbin/so-elastic-fleet-integration-policy-bulk-delete Executable file
View File

@@ -0,0 +1,23 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
POLICY_ID=$1
# Let's snag a cookie from Kibana
SESSIONCOOKIE=$(curl -q -K /opt/so/conf/elasticsearch/curl.config -c - -X GET http://localhost:5601/ | grep sid | awk '{print $7}')
# Get integration policies relative to agent policy
INTEGRATION_POLICY_IDS=$(curl -q -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X GET "localhost:5601/api/fleet/agent_policies/$POLICY_ID" | jq -r '.item.package_policies[].id')
for i in $INTEGRATION_POLICY_IDS; do
# Delete integration policies
echo "Deleting integration policy: $i..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies/delete" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d"{\"packagePolicyIds\": [\"$i\"], \"force\":true}";
echo
echo
done

19
salt/common/tools/sbin/so-elastic-fleet-integration-policy-delete Executable file
View File

@@ -0,0 +1,19 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
POLICY_ID=$1
# Let's snag a cookie from Kibana
SESSIONCOOKIE=$(curl -K /opt/so/conf/elasticsearch/curl.config -c - -X GET http://localhost:5601/ | grep sid | awk '{print $7}')
echo "Deleting integration policy $POLICY_ID..."
# List configured package policies
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies/delete" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d"{\"packagePolicyIds\": [\"$POLICY_ID\"]}"
echo

17
salt/common/tools/sbin/so-elastic-fleet-integration-policy-list Executable file
View File

@@ -0,0 +1,17 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
# Let's snag a cookie from Kibana
SESSIONCOOKIE=$(curl -K /opt/so/conf/elasticsearch/curl.config -c - -X GET http://localhost:5601/ | grep sid | awk '{print $7}')
echo "Setting up default Security Onion package policies for Elastic Agent..."
# List configured package policies
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X GET "localhost:5601/api/fleet/package_policies" | jq
echo

137
salt/common/tools/sbin/so-elastic-fleet-integration-policy-load Executable file
View File

@@ -0,0 +1,137 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
{%- set ZEEKVER = salt['pillar.get']('global:mdengine', '') %}
{%- set STRELKAENABLED = salt['pillar.get']('strelka:enabled', '0') %}
{%- set RITAENABLED = salt['pillar.get']('rita:enabled', False) %}
wait_for_web_response "http://localhost:5601/api/spaces/space/default" "default" 300 "curl -K /opt/so/conf/elasticsearch/curl.config"
# Let's snag a cookie from Kibana
SESSIONCOOKIE=$(curl -K /opt/so/conf/elasticsearch/curl.config -c - -X GET http://localhost:5601/ | grep sid | awk '{print $7}')
# Disable certain Features from showing up in the Kibana UI
echo
echo "Disable certain Features from showing up in the Kibana UI"
so-kibana-space-defaults
echo
# Suricata logs
echo
echo "Setting up Suricata package policy..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d'{ "policy_id": "so-grid-nodes", "package": { "name": "log", "version": "1.1.0" }, "id": "suricata-logs", "name": "suricata-logs", "description": "Suricata integration", "namespace": "so", "inputs": { "logs-logfile": { "enabled": true, "streams": { "log.log": { "enabled": true, "vars": { "paths": [ "/nsm/suricata/eve*.json" ], "data_stream.dataset": "suricata", "tags": [],"processors": "- add_fields:\n target: event\n fields:\n category: network\n module: suricata", "custom": "pipeline: suricata.common" }}}}}}'
echo
# Zeek logs
echo
echo "Setting up Zeek package policy..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d'{ "policy_id": "so-grid-nodes", "package": { "name": "log", "version": "1.1.0" }, "name": "zeek-logs", "description": "Zeek logs", "namespace": "so", "inputs": { "logs-logfile": { "enabled": true, "streams": { "log.log": { "enabled": true, "vars": { "paths": ["/nsm/zeek/logs/current/*.log"], "data_stream.dataset": "zeek", "tags": [], "processors": "- dissect:\n tokenizer: \"/nsm/zeek/logs/current/%{pipeline}.log\"\n field: \"log.file.path\"\n trim_chars: \".log\"\n target_prefix: \"\"\n- script:\n lang: javascript\n source: >\n function process(event) {\n var pl = event.Get(\"pipeline\");\n event.Put(\"@metadata.pipeline\", \"zeek.\" + pl);\n }\n- add_fields:\n target: event\n fields:\n category: network\n module: zeek\n- add_tags:\n tags: \"ics\"\n when:\n regexp:\n pipeline: \"^bacnet*|^bsap*|^cip*|^cotp*|^dnp3*|^ecat*|^enip*|^modbus*|^opcua*|^profinet*|^s7comm*\"", "custom": "exclude_files: [\"broker|capture_loss|ecat_arp_info|loaded_scripts|packet_filter|stats|stderr|stdout.log$\"]\n" } } } } } }'
echo
# Import - EVTX
echo
echo "Setting up EVTX import package policy..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d '{ "package": { "name": "log", "version": "1.1.0" }, "name": "import-evtx-logs", "namespace": "so", "description": "Import Windows EVTX logs", "policy_id": "so-grid-nodes", "inputs": { "logs-logfile": { "enabled": true, "streams": { "log.log": { "enabled": true, "vars": { "paths": [ "/nsm/import/*/evtx/data.json" ], "data_stream.dataset": "import", "custom": "pipeline: import.wel", "processors": "- dissect:\n tokenizer: \"/nsm/import/%{import.id}/evtx/%{import.file}\"\n field: \"log.file.path\"\n target_prefix: \"\"\n- decode_json_fields:\n fields: [\"message\"]\n target: \"\"\n- add_fields:\n target: event\n fields:\n module: windows_eventlog\n imported: true", "tags": [] } } } } } }'
echo
# Import - Suricata logs
echo
echo "Setting up Suricata import package policy..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d '{ "policy_id": "so-grid-nodes", "package": { "name": "log", "version": "1.1.0" }, "id": "import-suricata-logs", "name": "import-suricata-logs", "description": "Import Suricata logs", "namespace": "so", "inputs": { "logs-logfile": { "enabled": true, "streams": { "log.log": { "enabled": true, "vars": { "paths": ["/nsm/import/*/suricata/eve*.json"], "data_stream.dataset": "import", "tags": [], "processors": "- add_fields:\n target: event\n fields:\n category: network\n module: suricata\n imported: true\n- dissect:\n tokenizer: \"/nsm/import/%{import.id}/suricata/%{import.file}\"\n field: \"log.file.path\"\n target_prefix: \"\"", "custom": "pipeline: suricata.common" } } } } } }'
echo
# Import - Zeek logs
echo
echo "Setting up Zeek import package policy..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d'{ "policy_id": "so-grid-nodes", "package": { "name": "log", "version": "1.1.0" }, "name": "import-zeek-logs", "description": "Zeek Import logs", "namespace": "so", "inputs": { "logs-logfile": { "enabled": true, "streams": { "log.log": { "enabled": true, "vars": { "paths": ["/nsm/import/*/zeek/logs/*.log"], "data_stream.dataset": "import", "tags": [], "processors": "- dissect:\n tokenizer: \"/nsm/import/%{import.id}/zeek/logs/%{import.file}\"\n field: \"log.file.path\"\n target_prefix: \"\"\n- script:\n lang: javascript\n source: >\n function process(event) {\n var pl = event.Get(\"import.file\").slice(0,-4);\n event.Put(\"@metadata.pipeline\", \"zeek.\" + pl);\n }\n- add_fields:\n target: event\n fields:\n category: network\n module: zeek\n imported: true\n- add_tags:\n tags: \"ics\"\n when:\n regexp:\n import.file: \"^bacnet*|^bsap*|^cip*|^cotp*|^dnp3*|^ecat*|^enip*|^modbus*|^opcua*|^profinet*|^s7comm*\"", "custom": "exclude_files: [\"broker|capture_loss|ecat_arp_info|loaded_scripts|packet_filter|stats|stderr|stdout.log$\"]\n" } } } } } }'
echo
# Strelka logs
echo
echo "Setting up Strelka package policy..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d'{ "policy_id": "so-grid-nodes", "package": { "name": "log", "version": "1.1.0" }, "id": "strelka-logs", "name": "strelka-logs", "description": "Strelka logs", "namespace": "so", "inputs": { "logs-logfile": { "enabled": true, "streams": { "log.log": { "enabled": true, "vars": { "paths": [ "/nsm/strelka/log/strelka.log" ], "data_stream.dataset": "strelka", "tags": [],"processors": "- add_fields:\n target: event\n fields:\n category: file\n module: strelka", "custom": "pipeline: strelka.file" }}}}}}'
echo
# Syslog TCP Port 514
echo
echo "Setting up Syslog TCP package policy..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d '{ "policy_id": "so-grid-nodes", "package": { "name": "tcp", "version": "1.5.0" }, "id": "syslog-tcp-514", "name": "syslog-tcp-514", "description": "Syslog Over TCP Port 514", "namespace": "so", "inputs": { "tcp-tcp": { "enabled": true, "streams": { "tcp.generic": { "enabled": true, "vars": { "listen_address": "0.0.0.0", "listen_port": "514", "data_stream.dataset": "syslog", "pipeline": "syslog", "processors": "- add_fields:\n target: event\n fields:\n module: syslog", "tags": [ "syslog" ], "syslog_options": "field: message\n#format: auto\n#timezone: Local" } } } } } }'
echo
# Syslog UDP Port 514
echo
echo "Setting up Syslog UDP package policy..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d '{ "policy_id": "so-grid-nodes", "package": { "name": "udp", "version": "1.5.0" }, "id": "syslog-udp-514", "name": "syslog-udp-514", "description": "Syslog over UDP Port 514", "namespace": "so", "inputs": { "udp-udp": { "enabled": true, "streams": { "udp.generic": { "enabled": true, "vars": { "listen_address": "0.0.0.0", "listen_port": "514", "data_stream.dataset": "syslog", "pipeline": "syslog", "max_message_size": "10KiB", "keep_null": false, "processors": "- add_fields:\n target: event\n fields: \n module: syslog\n", "tags": [ "syslog" ], "syslog_options": "field: message\n#format: auto\n#timezone: Local" } } } } } }'
echo
# Kratos logs
echo
echo "Setting up Kratos package policy..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d'{ "policy_id": "so-grid-nodes", "package": { "name": "log", "version": "1.1.0" }, "id": "kratos-logs", "name": "kratos-logs", "description": "Kratos logs", "namespace": "so", "inputs": { "logs-logfile": { "enabled": true, "streams": { "log.log": { "enabled": true, "vars": { "paths": [ "/opt/so/log/kratos/kratos.log" ], "data_stream.dataset": "kratos", "tags": [],"custom":"pipeline: kratos","processors": "- decode_json_fields:\n fields: [\"message\"]\n target: \"\"\n add_error_key: true \n- add_fields:\n target: event\n fields:\n category: iam\n module: kratos" }}}}}}'
echo
# RITA Logs
#echo
#echo "Setting up RITA package policy..."
#curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d'{ "policy_id": "so-grid-nodes", "package": { "name": "log", "version": "1.1.0" }, "id": "rita-logs", "name": "rita-logs", "description": "RITA Beacon logs", "namespace": "so", "inputs": { "logs-logfile": { "enabled": true, "streams": { "log.log": { "enabled": true, "vars": { "paths": [ "/nsm/rita/beacons.csv", "/nsm/rita/long-connections.csv", "/nsm/rita/short-connections.csv", "/nsm/rita/exploded-dns.csv" ], "data_stream.dataset": "rita", "tags": [], "processors": "- add_fields:\n target: event\n fields:\n category: network\n module: rita\n- if:\n log.file.path: beacons.csv\n then: \n - add_fields:\n target: \"@metadata\"\n fields:\n pipeline: rita.beacon\n- if:\n regexp:\n log.file.path: \"*connections.csv\"\n then: \n - add_fields:\n target: \"@metadata\"\n fields:\n pipeline: rita.connection\n- if:\n log.file.path: \"exploded-dns.csv\"\n then: \n - add_fields:\n target: \"@metadata\"\n fields:\n pipeline: rita.dns" }}}}}}'
#echo
# Elasticsearch logs
echo
echo "Setting up Elasticsearch package policy..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d'{ "policy_id": "so-grid-nodes", "package": { "name": "elasticsearch", "version": "1.0.0" }, "id": "elasticsearch-logs", "name": "elasticsearch-logs", "description": "Elasticsearch Logs", "namespace": "default", "inputs": { "elasticsearch-logfile": { "enabled": true, "streams": { "elasticsearch.audit": { "enabled": false, "vars": { "paths": [ "/var/log/elasticsearch/*_audit.json" ] } }, "elasticsearch.deprecation": { "enabled": false, "vars": { "paths": [ "/var/log/elasticsearch/*_deprecation.json" ] } }, "elasticsearch.gc": { "enabled": false, "vars": { "paths": [ "/var/log/elasticsearch/gc.log.[0-9]*", "/var/log/elasticsearch/gc.log" ] } }, "elasticsearch.server": { "enabled": true, "vars": { "paths": [ "/opt/so/log/elasticsearch/*.log" ] } }, "elasticsearch.slowlog": { "enabled": false, "vars": { "paths": [ "/var/log/elasticsearch/*_index_search_slowlog.json", "/var/log/elasticsearch/*_index_indexing_slowlog.json" ] } } } }, "elasticsearch-elasticsearch/metrics": { "enabled": false, "vars": { "hosts": [ "http://localhost:9200" ], "scope": "node" }, "streams": { "elasticsearch.stack_monitoring.ccr": { "enabled": false }, "elasticsearch.stack_monitoring.cluster_stats": { "enabled": false }, "elasticsearch.stack_monitoring.enrich": { "enabled": false }, "elasticsearch.stack_monitoring.index": { "enabled": false }, "elasticsearch.stack_monitoring.index_recovery": { "enabled": false, "vars": { "active.only": true } }, "elasticsearch.stack_monitoring.index_summary": { "enabled": false }, "elasticsearch.stack_monitoring.ml_job": { "enabled": false }, "elasticsearch.stack_monitoring.node": { "enabled": false }, "elasticsearch.stack_monitoring.node_stats": { "enabled": false }, "elasticsearch.stack_monitoring.pending_tasks": { "enabled": false }, "elasticsearch.stack_monitoring.shard": { "enabled": false } } } } }'
echo
# Logstash logs
#echo
#echo "Setting up Logstash package policy..."
#curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d'{ "policy_id": "so-grid-nodes", "package": { "name": "logstash", "version": "2.0.0" }, "id": "logstash-logs", "name": "logstash-logs", "description": "Logstash logs", "namespace": "default", "inputs": { "logstash-logfile": { "enabled": true, "streams": { "logstash.log": { "enabled": true, "vars": { "paths": [ "/opt/so/logs/logstash/logstash.log" ] } }, "logstash.slowlog": { "enabled": false, "vars": { "paths": [ "/var/log/logstash/logstash-slowlog-plain*.log", "/var/log/logstash/logstash-slowlog-json*.log" ] } } } }, "logstash-logstash/metrics": { "enabled": false, "vars": { "hosts": [ "http://localhost:9600" ], "period": "10s" }, "streams": { "logstash.stack_monitoring.node": { "enabled": false }, "logstash.stack_monitoring.node_stats": { "enabled": false } } } } }'
#echo
# Kibana logs
#echo
#echo "Setting up Kibana package policy..."
#curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d'{ "policy_id": "so-grid-nodes", "package": { "name": "kibana", "version": "2.0.0" }, "id": "kibana-logs", "name": "kibana-logs", "description": "Kibana logs", "namespace": "default", "inputs": { "kibana-logfile": { "enabled": true, "streams": { "kibana.audit": { "enabled": false, "vars": { "paths": [ "/opt/so/log/kibana/kibana.log" ] } }, "kibana.log": { "enabled": true, "vars": { "paths": [ "/opt/so/log/kibana/kibana.log" ] } } } }, "kibana-kibana/metrics": { "enabled": false, "vars": { "hosts": [ "http://localhost:5601" ] }, "streams": { "kibana.stack_monitoring.cluster_actions": { "enabled": false }, "kibana.stack_monitoring.cluster_rules": { "enabled": false }, "kibana.stack_monitoring.node_actions": { "enabled": false }, "kibana.stack_monitoring.node_rules": { "enabled": false }, "kibana.stack_monitoring.stats": { "enabled": false }, "kibana.stack_monitoring.status": { "enabled": false } } } } }'
#echo
# Redis logs
echo
echo "Setting up Redis package policy..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d'{ "policy_id": "so-grid-nodes", "package": { "name": "redis", "version": "1.4.0" }, "id": "redis-logs", "name": "redis-logs", "description": "Redis logs", "namespace": "default", "inputs": { "redis-logfile": { "enabled": true, "streams": { "redis.log": { "enabled": true, "vars": { "paths": [ "/opt/so/log/redis/redis.log" ], "tags": [ "redis-log" ], "preserve_original_event": false } } } }, "redis-redis": { "enabled": false, "streams": { "redis.slowlog": { "enabled": false, "vars": { "hosts": [ "127.0.0.1:6379" ], "password": "" } } } }, "redis-redis/metrics": { "enabled": false, "vars": { "hosts": [ "127.0.0.1:6379" ], "idle_timeout": "20s", "maxconn": 10, "network": "tcp", "password": "" }, "streams": { "redis.info": { "enabled": false, "vars": { "period": "10s" } }, "redis.key": { "enabled": false, "vars": { "key.patterns": "- limit: 20\n pattern: '*'\n", "period": "10s" } }, "redis.keyspace": { "enabled": false, "vars": { "period": "10s" } } } } } }'
echo
# IDH logs
echo
echo "Setting up IDH package policy..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d'{"policy_id":"so-grid-nodes","package":{"name":"log","version":"1.1.1"},"id":"idh-logs","name":"idh-logs","namespace":"so","description":"IDH integration","inputs":{"logs-logfile":{"enabled":true,"streams":{"log.log":{"enabled":true,"vars":{"paths":["/nsm/idh/opencanary.log"],"data_stream.dataset":"idh","custom":"pipeline: common","processors": "\n- decode_json_fields:\n fields: [\"message\"]\n target: \"\"\n add_error_key: true\n- drop_fields:\n when:\n equals:\n logtype: \"1001\"\n fields: [\"src_host\", \"src_port\", \"dst_host\", \"dst_port\" ]\n ignore_missing: true\n- rename:\n fields:\n - from: \"src_host\"\n to: \"source.ip\"\n - from: \"src_port\"\n to: \"source.port\"\n - from: \"dst_host\"\n to: \"destination.host\"\n - from: \"dst_port\"\n to: \"destination.port\"\n ignore_missing: true\n- convert:\n fields:\n - {from: \"logtype\", to: \"event.code\", type: \"string\"}\n ignore_missing: true\n- drop_fields:\n fields: '\''[\"prospector\", \"input\", \"offset\", \"beat\"]'\''\n- add_fields:\n target: event\n fields:\n category: host\n module: opencanary","tags":[]}}}}}}'
echo
# SOC - Server logs
echo
echo "Setting up SOC - Server Logs package policy..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d'{"package":{"name":"log","version":"1.1.2"},"name":"soc-server-logs","namespace":"so","description":"Security Onion Console Logs","policy_id":"so-grid-nodes","inputs":{"logs-logfile":{"enabled":true,"streams":{"log.log":{"enabled":true,"vars":{"paths":["/opt/so/log/soc/sensoroni-server.log"],"data_stream.dataset":"soc","custom":"pipeline: common","processors": "- decode_json_fields:\n fields: [\"message\"]\n target: \"soc\"\n process_array: true\n max_depth: 2\n add_error_key: true \n- add_fields:\n target: event\n fields:\n category: host\n module: soc\n dataset_temp: server\n- rename:\n fields:\n - from: \"soc.fields.sourceIp\"\n to: \"source.ip\"\n - from: \"soc.fields.status\"\n to: \"http.response.status_code\"\n - from: \"soc.fields.method\"\n to: \"http.request.method\"\n - from: \"soc.fields.path\"\n to: \"url.path\"\n - from: \"soc.message\"\n to: \"event.action\"\n - from: \"soc.level\"\n to: \"log.level\"\n ignore_missing: true","tags":[]}}}}}}'
echo
# SOC - Sensoroni logs
echo
echo "Setting up SOC - Sensoroni Logs package policy..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d'{"package":{"name":"log","version":"1.1.2"},"name":"soc-sensoroni-logs","namespace":"so","description":"Security Onion - Sensoroni - Logs","policy_id":"so-grid-nodes","inputs":{"logs-logfile":{"enabled":true,"streams":{"log.log":{"enabled":true,"vars":{"paths":["/opt/so/log/sensoroni/sensoroni.log"],"data_stream.dataset":"soc","custom":"pipeline: common","processors": "- decode_json_fields:\n fields: [\"message\"]\n target: \"sensoroni\"\n process_array: true\n max_depth: 2\n add_error_key: true \n- add_fields:\n target: event\n fields:\n category: host\n module: soc\n dataset_temp: sensoroni\n- rename:\n fields:\n - from: \"sensoroni.fields.sourceIp\"\n to: \"source.ip\"\n - from: \"sensoroni.fields.status\"\n to: \"http.response.status_code\"\n - from: \"sensoroni.fields.method\"\n to: \"http.request.method\"\n - from: \"sensoroni.fields.path\"\n to: \"url.path\"\n - from: \"sensoroni.message\"\n to: \"event.action\"\n - from: \"sensoroni.level\"\n to: \"log.level\"\n ignore_missing: true","tags":[]}}}}}}'
echo
# SOC - Elastic Auth Sync logs
echo
echo "Setting up SOC - Elastic Auth Sync Logs package policy..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d'{"package":{"name":"log","version":"1.1.2"},"name":"soc-auth-sync-logs","namespace":"so","description":"Security Onion - Elastic Auth Sync - Logs","policy_id":"so-grid-nodes","inputs":{"logs-logfile":{"enabled":true,"streams":{"log.log":{"enabled":true,"vars":{"paths":["/opt/so/log/soc/sync.log"],"data_stream.dataset":"soc","custom":"pipeline: common","processors": "- dissect:\n tokenizer: \"%{event.action}\"\n field: \"message\"\n target_prefix: \"\"\n- add_fields:\n target: event\n fields:\n category: host\n module: soc\n dataset_temp: auth_sync","tags":[]}}}}}}'
echo
# SOC - Salt Relay logs
echo
echo "Setting up SOC - Salt_Relay Logs package policy..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X POST "localhost:5601/api/fleet/package_policies" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d'{"package":{"name":"log","version":"1.1.2"},"name":"soc-salt-relay-logs","namespace":"so","description":"Security Onion - Salt Relay - Logs","policy_id":"so-grid-nodes","inputs":{"logs-logfile":{"enabled":true,"streams":{"log.log":{"enabled":true,"vars":{"paths":["/opt/so/log/soc/salt-relay.log"],"data_stream.dataset":"soc","custom":"pipeline: common","processors": "- dissect:\n tokenizer: \"%{soc.ts} | %{event.action}\"\n field: \"message\"\n target_prefix: \"\"\n- add_fields:\n target: event\n fields:\n category: host\n module: soc\n dataset_temp: salt_relay","tags":[]}}}}}}'
echo

12
salt/common/tools/sbin/so-elastic-fleet-restart Executable file
View File

@@ -0,0 +1,12 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
/usr/sbin/so-restart elastic-fleet $1

109
salt/common/tools/sbin/so-elastic-fleet-setup Executable file
View File

@@ -0,0 +1,109 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0; you may not use
# this file except in compliance with the Elastic License 2.0.
{% from 'vars/globals.map.jinja' import GLOBALS %}
. /usr/sbin/so-common
# Create ES Token
ESTOKEN=$(curl -K /opt/so/conf/elasticsearch/curl.config -L -X POST "localhost:5601/api/fleet/service_tokens" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' | jq -r .value)
printf "ESTOKEN = $ESTOKEN \n"
# Add SO-Manager Fleet URL
## This array replaces whatever URLs are currently configured
printf "\n"
curl -K /opt/so/conf/elasticsearch/curl.config -L -X PUT "localhost:5601/api/fleet/settings" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d '{"fleet_server_hosts":["https://{{ GLOBALS.manager_ip }}:8220"]}'
printf "\n\n"
# Configure certificates
mkdir -p /opt/so/conf/elastic-fleet/certs
cp /etc/ssl/certs/intca.crt /opt/so/conf/elastic-fleet/certs
cp /etc/pki/elasticfleet* /opt/so/conf/elastic-fleet/certs
{% if grains.role in ['so-import', 'so-standalone', 'so-eval', 'so-manager', 'so-managersearch'] %}
# Add SO-Manager Elasticsearch Ouput
ESCACRT=$(openssl x509 -in /opt/so/conf/elastic-fleet/certs/intca.crt)
JSON_STRING=$( jq -n \
--arg ESCACRT "$ESCACRT" \
'{"name":"so-manager_elasticsearch","id":"so-manager_elasticsearch","type":"elasticsearch","hosts":["https://{{ GLOBALS.manager_ip }}:9200"],"is_default":true,"is_default_monitoring":true,"config_yaml":"","ssl":{"certificate_authorities": [$ESCACRT]}}' )
curl -K /opt/so/conf/elasticsearch/curl.config -L -X POST "localhost:5601/api/fleet/outputs" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d "$JSON_STRING"
printf "\n\n"
{% else %}
# Create Logstash Output payload
LOGSTASHCRT=$(openssl x509 -in /opt/so/conf/elastic-fleet/certs/elasticfleet.crt)
LOGSTASHKEY=$(openssl rsa -in /opt/so/conf/elastic-fleet/certs/elasticfleet.key)
LOGSTASHCA=$(openssl x509 -in /opt/so/conf/elastic-fleet/certs/intca.crt)
JSON_STRING=$( jq -n \
--arg LOGSTASHCRT "$LOGSTASHCRT" \
--arg LOGSTASHKEY "$LOGSTASHKEY" \
--arg LOGSTASHCA "$LOGSTASHCA" \
'{"name":"so-manager_logstash","id":"so-manager_logstash","type":"logstash","hosts":["{{ GLOBALS.manager_ip }}:5055"],"is_default":true,"is_default_monitoring":true,"config_yaml":"","ssl":{"certificate": $LOGSTASHCRT,"key": $LOGSTASHKEY,"certificate_authorities":[ $LOGSTASHCA ]}}'
)
# Add SO-Manager Logstash Ouput
curl -K /opt/so/conf/elasticsearch/curl.config -L -X POST "localhost:5601/api/fleet/outputs" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d "$JSON_STRING"
printf "\n\n"
{%- endif %}
# Add Elastic Fleet Integrations
# Add Elastic Fleet Server Agent Policy
#curl -vv -K /opt/so/conf/elasticsearch/curl.config -L \
#-X POST "localhost:5601/api/fleet/agent_policies" \
#-H 'kbn-xsrf: true' -H 'Content-Type: application/json' \
#-d '{"name":"SO-Manager","id":"so-manager","description":"SO Manager Fleet Server Policy","namespace":"default","monitoring_enabled":["logs"],"has_fleet_server":true}'
# Add Agent Policy - SOS Grid Nodes
#curl -vv -K /opt/so/conf/elasticsearch/curl.config -L \
#-X POST "localhost:5601/api/fleet/agent_policies" \
#-H 'kbn-xsrf: true' -H 'Content-Type: application/json' \
#-d '{"name":"SO-Grid","id":"so-grid","description":"SO Grid Endpoint Policy","namespace":"default","monitoring_enabled":["logs"]}'
# Add Agent Policy - Default endpoints
#curl -vv -K /opt/so/conf/elasticsearch/curl.config -L \
#-X POST "localhost:5601/api/fleet/agent_policies" \
#-H 'kbn-xsrf: true' -H 'Content-Type: application/json' \
#-d '{"name":"Endpoints-Initalization","id":"endpoints","description":"Initial Endpoint Policy","namespace":"default","monitoring_enabled":["logs"]}'
ENDPOINTSENROLLMENTOKEN=$(curl -K /opt/so/conf/elasticsearch/curl.config -L "localhost:5601/api/fleet/enrollment_api_keys" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' | jq .list | jq -r -c '.[] | select(.policy_id | contains("endpoints-default")) | .api_key')
GRIDNODESENROLLMENTOKEN=$(curl -K /opt/so/conf/elasticsearch/curl.config -L "localhost:5601/api/fleet/enrollment_api_keys" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' | jq .list | jq -r -c '.[] | select(.policy_id | contains("so-grid-nodes")) | .api_key')
# Store needed data in minion pillar
pillar_file=/opt/so/saltstack/local/pillar/minions/{{ GLOBALS.minion_id }}.sls
printf '%s\n'\
"elasticfleet:"\
" server:"\
" es_token: '$ESTOKEN'"\
" endpoints_enrollment: '$ENDPOINTSENROLLMENTOKEN'"\
" grid_enrollment: '$GRIDNODESENROLLMENTOKEN'"\
" url: '{{ GLOBALS.manager_ip }}'"\
"" >> "$pillar_file"
#Store Grid Nodes Enrollment token in Global pillar
global_pillar_file=/opt/so/saltstack/local/pillar/soc_global.sls
printf '%s\n'\
" fleet_grid_enrollment_token: '$GRIDNODESENROLLMENTOKEN'"\
"" >> "$global_pillar_file"
# Call Elastic-Fleet Salt State
salt-call state.apply elasticfleet queue=True
# Load Elastic Fleet integrations
/usr/sbin/so-elastic-fleet-integration-policy-load
# Temp
wget --progress=bar:force:noscroll -P /opt/so/saltstack/default/salt/elasticfleet/files/elastic-agent/ https://github.com/Security-Onion-Solutions/securityonion-docker-rpm/releases/download/so_elastic-agent-8.6.2/so-elastic-agent-8.6.2-darwin-x86_64.tar.gz
wget --progress=bar:force:noscroll -P /opt/so/saltstack/default/salt/elasticfleet/files/elastic-agent/ https://github.com/Security-Onion-Solutions/securityonion-docker-rpm/releases/download/so_elastic-agent-8.6.2/so-elastic-agent-8.6.2-linux-x86_64.tar.gz
wget --progress=bar:force:noscroll -P /opt/so/saltstack/default/salt/elasticfleet/files/elastic-agent/ https://github.com/Security-Onion-Solutions/securityonion-docker-rpm/releases/download/so_elastic-agent-8.6.2/so-elastic-agent-8.6.2-windows-x86_64.tar.gz
#git clone -b 2.4-so-elastic-agent https://github.com/Security-Onion-Solutions/securityonion-image.git
#cd securityonion-image/so-elastic-agent-builder
#docker build -t so-elastic-agent-builder .
so-elastic-agent-gen-installers
salt-call state.apply elasticfleet.install_agent_grid queue=True

12
salt/common/tools/sbin/so-elastic-fleet-start Executable file
View File

@@ -0,0 +1,12 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
/usr/sbin/so-start elastic-fleet $1

12
salt/common/tools/sbin/so-elastic-fleet-stop Executable file
View File

@@ -0,0 +1,12 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
/usr/sbin/so-stop elastic-fleet $1

30
salt/common/tools/sbin/so-elastic-restart
View File

@@ -1,24 +1,16 @@
#!/bin/bash #!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common . /usr/sbin/so-common
{%- if grains['role'] in ['so-eval','so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-node', 'so-import']%} {%- if grains['role'] in ['so-eval','so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-searchnode', 'so-import']%}
/usr/sbin/so-restart elasticsearch $1 /usr/sbin/so-restart elasticsearch $1
{%- endif %} {%- endif %}
@@ -26,15 +18,11 @@
/usr/sbin/so-restart kibana $1 /usr/sbin/so-restart kibana $1
{%- endif %} {%- endif %}
{%- if grains['role'] in ['so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-node']%} {%- if grains['role'] in ['so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-searchnode']%}
/usr/sbin/so-restart logstash $1 /usr/sbin/so-restart logstash $1
{%- endif %} {%- endif %}
{%- if grains['role'] in ['so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-node', 'so-sensor']%} {%- if grains['role'] in ['so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-searchnode']%}
/usr/sbin/so-restart filebeat $1
{%- endif %}
{%- if grains['role'] in ['so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-node']%}
/usr/sbin/so-restart curator $1 /usr/sbin/so-restart curator $1
{%- endif %} {%- endif %}

30
salt/common/tools/sbin/so-elastic-start
View File

@@ -1,24 +1,16 @@
#!/bin/bash #!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common . /usr/sbin/so-common
{%- if grains['role'] in ['so-eval','so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-node', 'so-import']%} {%- if grains['role'] in ['so-eval','so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-searchnode', 'so-import']%}
/usr/sbin/so-start elasticsearch $1 /usr/sbin/so-start elasticsearch $1
{%- endif %} {%- endif %}
@@ -26,15 +18,11 @@
/usr/sbin/so-start kibana $1 /usr/sbin/so-start kibana $1
{%- endif %} {%- endif %}
{%- if grains['role'] in ['so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-node']%} {%- if grains['role'] in ['so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-searchnode']%}
/usr/sbin/so-start logstash $1 /usr/sbin/so-start logstash $1
{%- endif %} {%- endif %}
{%- if grains['role'] in ['so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-node', 'so-sensor']%} {%- if grains['role'] in ['so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-searchnode']%}
/usr/sbin/so-start filebeat $1
{%- endif %}
{%- if grains['role'] in ['so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-node']%}
/usr/sbin/so-start curator $1 /usr/sbin/so-start curator $1
{%- endif %} {%- endif %}

30
salt/common/tools/sbin/so-elastic-stop
View File

@@ -1,24 +1,16 @@
#!/bin/bash #!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common . /usr/sbin/so-common
{%- if grains['role'] in ['so-eval','so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-node', 'so-import']%} {%- if grains['role'] in ['so-eval','so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-searchnode', 'so-import']%}
/usr/sbin/so-stop elasticsearch $1 /usr/sbin/so-stop elasticsearch $1
{%- endif %} {%- endif %}
@@ -26,15 +18,11 @@
/usr/sbin/so-stop kibana $1 /usr/sbin/so-stop kibana $1
{%- endif %} {%- endif %}
{%- if grains['role'] in ['so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-node']%} {%- if grains['role'] in ['so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-searchnode']%}
/usr/sbin/so-stop logstash $1 /usr/sbin/so-stop logstash $1
{%- endif %} {%- endif %}
{%- if grains['role'] in ['so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-node', 'so-sensor']%} {%- if grains['role'] in ['so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-searchnode']%}
/usr/sbin/so-stop filebeat $1
{%- endif %}
{%- if grains['role'] in ['so-manager', 'so-managersearch', 'so-standalone', 'so-heavynode', 'so-node']%}
/usr/sbin/so-stop curator $1 /usr/sbin/so-stop curator $1
{%- endif %} {%- endif %}

57
salt/common/tools/sbin/so-elasticsearch-cluster-space-total Executable file
View File

@@ -0,0 +1,57 @@
#!/bin/bash
#
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
{% from 'vars/globals.map.jinja' import GLOBALS %}
TOTAL_AVAILABLE_SPACE=0
# Wait for ElasticSearch to initialize
COUNT=0
ELASTICSEARCH_CONNECTED="no"
while [[ "$COUNT" -le 240 ]]; do
/usr/sbin/so-elasticsearch-query / -k --output /dev/null --silent --head --fail
if [ $? -eq 0 ]; then
ELASTICSEARCH_CONNECTED="yes"
break
else
((COUNT+=1))
sleep 1
fi
done
if [ "$ELASTICSEARCH_CONNECTED" == "no" ]; then
echo
echo -e "Connection attempt timed out. Unable to connect to ElasticSearch. \nPlease try: \n -checking log(s) in /var/log/elasticsearch/\n -running 'sudo docker ps' \n -running 'sudo so-elastic-restart'"
echo
exit 1
fi
# Set percentage of space to desired value, otherwise use a default value of 80 percent
if [[ "$1" != "" ]]; then
PERCENTAGE=$1
else
PERCENTAGE=80
fi
# Iterate through the output of _cat/allocation for each node in the cluster to determine the total available space
{% if GLOBALS.role == 'so-manager' %}
for i in $(/usr/sbin/so-elasticsearch-query _cat/allocation | grep -v {{ GLOBALS.manager }} | awk '{print $5}'); do
{% else %}
for i in $(/usr/sbin/so-elasticsearch-query _cat/allocation | awk '{print $5}'); do
{% endif %}
size=$(echo $i | grep -oE '[0-9].*' | awk '{print int($1+0.5)}')
unit=$(echo $i | grep -oE '[A-Za-z]+')
if [ $unit = "tb" ]; then
size=$(( size * 1024 ))
fi
TOTAL_AVAILABLE_SPACE=$(( TOTAL_AVAILABLE_SPACE + size ))
done
# Calculate the percentage of available space based on our previously defined value
PERCENTAGE_AVAILABLE_SPACE=$(( TOTAL_AVAILABLE_SPACE*PERCENTAGE/100 ))
echo "$PERCENTAGE_AVAILABLE_SPACE"

28
salt/common/tools/sbin/so-elasticsearch-cluster-space-used Executable file
View File

@@ -0,0 +1,28 @@
#!/bin/bash
#
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
{% from 'vars/globals.map.jinja' import GLOBALS %}
TOTAL_AVAILABLE_SPACE=0
# Iterate through the output of _cat/allocation for each node in the cluster to determine the total available space
{% if GLOBALS.role == 'so-manager' %}
for i in $(/usr/sbin/so-elasticsearch-query _cat/allocation | grep -v {{ GLOBALS.manager }} | awk '{print $3}'); do
{% else %}
for i in $(/usr/sbin/so-elasticsearch-query _cat/allocation | awk '{print $3}'); do
{% endif %}
size=$(echo $i | grep -oE '[0-9].*' | awk '{print int($1+0.5)}')
unit=$(echo $i | grep -oE '[A-Za-z]+')
if [ $unit = "tb" ]; then
size=$(( size * 1024 ))
fi
TOTAL_AVAILABLE_SPACE=$(( TOTAL_AVAILABLE_SPACE + size ))
done
# Calculate the percentage of available space based on our previously defined value
echo "$TOTAL_AVAILABLE_SPACE"

25
salt/common/tools/sbin/so-elasticsearch-component-templates-list
View File

@@ -1,23 +1,14 @@
#!/bin/bash #!/bin/bash
# #
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version. {%- set NODEIP = salt['pillar.get']('host:mainip', '') %}
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
{%- set NODEIP = salt['pillar.get']('elasticsearch:mainip', '') -%}
. /usr/sbin/so-common . /usr/sbin/so-common
if [ "$1" == "" ]; then if [ "$1" == "" ]; then
{{ ELASTICCURL }} -s -k -L https://{{ NODEIP }}:9200/_component_template | jq '.component_templates[] |.name'| sort curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L https://{{ NODEIP }}:9200/_component_template | jq '.component_templates[] |.name'| sort
else else
{{ ELASTICCURL }} -s -k -L https://{{ NODEIP }}:9200/_component_template/$1 | jq curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L https://{{ NODEIP }}:9200/_component_template/$1 | jq
fi fi

15
salt/common/tools/sbin/so-elasticsearch-ilm-lifecycle-status Executable file
View File

@@ -0,0 +1,15 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
{%- set NODEIP = salt['pillar.get']('host:mainip', '') %}
if [ "$1" == "" ]; then
curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L https://{{ NODEIP }}:9200/_all/_ilm/explain | jq .
else
curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L https://{{ NODEIP }}:9200/$1/_ilm/explain | jq .[]
fi

11
salt/common/tools/sbin/so-elasticsearch-ilm-policy-delete Executable file
View File

@@ -0,0 +1,11 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
{%- set NODEIP = salt['pillar.get']('host:mainip', '') %}
curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L -X DELETE https://{{ NODEIP }}:9200/_ilm/policy/$1

21
salt/common/tools/sbin/so-elasticsearch-ilm-policy-load Executable file
View File

@@ -0,0 +1,21 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
{% import_yaml 'elasticsearch/defaults.yaml' as ESCONFIG with context %}
{%- set ES_INDEX_SETTINGS = salt['pillar.get']('elasticsearch:index_settings', default=ESCONFIG.elasticsearch.index_settings, merge=True) %}
{%- set NODEIP = salt['pillar.get']('host:mainip', '') %}
{%- for index, settings in ES_INDEX_SETTINGS.items() %}
{%- if settings.policy is defined %}
echo
echo "Setting up {{ index }}-logs policy..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -s -k -L -X PUT "https://{{ NODEIP }}:9200/_ilm/policy/{{ index }}-logs" -H 'Content-Type: application/json' -d'{ "policy": {{ settings.policy | tojson(true) }} }'
echo
{%- endif %}
{%- endfor %}
echo

15
salt/common/tools/sbin/so-elasticsearch-ilm-policy-view Executable file
View File

@@ -0,0 +1,15 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
{%- set NODEIP = salt['pillar.get']('host:mainip', '') %}
if [ "$1" == "" ]; then
curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L https://{{ NODEIP }}:9200/_ilm/policy | jq .
else
curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L https://{{ NODEIP }}:9200/_ilm/policy/$1 | jq .[]
fi

10
salt/common/tools/sbin/so-elasticsearch-ilm-restart Executable file
View File

@@ -0,0 +1,10 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
so-elasticsearch-ilm-stop
so-elasticsearch-ilm-start

12
salt/common/tools/sbin/so-elasticsearch-ilm-start Executable file
View File

@@ -0,0 +1,12 @@
/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
{%- set NODEIP = salt['pillar.get']('host:mainip', '') %}
echo "Starting ILM..."
curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L -X POST https://{{ NODEIP }}:9200/_ilm/start

11
salt/common/tools/sbin/so-elasticsearch-ilm-status Executable file
View File

@@ -0,0 +1,11 @@
/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
{%- set NODEIP = salt['pillar.get']('host:mainip', '') %}
curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L https://{{ NODEIP }}:9200/_ilm/status | jq .

12
salt/common/tools/sbin/so-elasticsearch-ilm-stop Executable file
View File

@@ -0,0 +1,12 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
{%- set NODEIP = salt['pillar.get']('host:mainip', '') %}
echo "Stopping ILM..."
curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L -X POST https://{{ NODEIP }}:9200/_ilm/stop

25
salt/common/tools/sbin/so-elasticsearch-index-templates-list
View File

@@ -1,23 +1,14 @@
#!/bin/bash #!/bin/bash
# #
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version. {%- set NODEIP = salt['pillar.get']('host:mainip', '') %}
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
{%- set NODEIP = salt['pillar.get']('elasticsearch:mainip', '') -%}
. /usr/sbin/so-common . /usr/sbin/so-common
if [ "$1" == "" ]; then if [ "$1" == "" ]; then
{{ ELASTICCURL }} -s -k -L https://{{ NODEIP }}:9200/_index_template | jq '.index_templates[] |.name'| sort curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L https://{{ NODEIP }}:9200/_index_template | jq '.index_templates[] |.name'| sort
else else
{{ ELASTICCURL }} -s -k -L https://{{ NODEIP }}:9200/_index_template/$1 | jq curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L https://{{ NODEIP }}:9200/_index_template/$1 | jq
fi fi

23
salt/common/tools/sbin/so-elasticsearch-indices-list
View File

@@ -1,21 +1,12 @@
#!/bin/bash #!/bin/bash
# #
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version. {%- set NODEIP = salt['pillar.get']('host:mainip', '') %}
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>
{%- set NODEIP = salt['pillar.get']('elasticsearch:mainip', '') -%}
. /usr/sbin/so-common . /usr/sbin/so-common
{{ ELASTICCURL }} -s -k -L "https://{{ NODEIP }}:9200/_cat/indices?pretty&v&s=index" curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L "https://{{ NODEIP }}:9200/_cat/indices?pretty&v&s=index"

22
salt/common/tools/sbin/so-elasticsearch-indices-rw
View File

@@ -1,23 +1,15 @@
#!/bin/bash #!/bin/bash
# #
# #
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
IP={{ salt['grains.get']('ip_interfaces').get(salt['pillar.get']('sensor:mainint', salt['pillar.get']('manager:mainint', salt['pillar.get']('elasticsearch:mainint', salt['pillar.get']('host:mainint')))))[0] }} IP={{ salt['grains.get']('ip_interfaces').get(salt['pillar.get']('sensor:mainint', salt['pillar.get']('manager:mainint', salt['pillar.get']('elasticsearch:mainint', salt['pillar.get']('host:mainint')))))[0] }}
ESPORT=9200 ESPORT=9200
echo "Removing read only attributes for indices..." echo "Removing read only attributes for indices..."
echo echo
{{ ELASTICCURL }} -s -k -XPUT -H "Content-Type: application/json" -L https://$IP:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}' 2>&1 | if grep -q ack; then echo "Index settings updated..."; else echo "There was any issue updating the read-only attribute. Please ensure Elasticsearch is running.";fi; curl -K /opt/so/conf/elasticsearch/curl.config -s -k -XPUT -H "Content-Type: application/json" -L https://$IP:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}' 2>&1 | if grep -q ack; then echo "Index settings updated..."; else echo "There was any issue updating the read-only attribute. Please ensure Elasticsearch is running.";fi;

25
salt/common/tools/sbin/so-elasticsearch-pipeline-stats
View File

@@ -1,25 +1,16 @@
#!/bin/bash #!/bin/bash
# #
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version. {%- set NODEIP = salt['pillar.get']('host:mainip', '') %}
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>
{%- set NODEIP = salt['pillar.get']('elasticsearch:mainip', '') -%}
. /usr/sbin/so-common . /usr/sbin/so-common
if [ "$1" == "" ]; then if [ "$1" == "" ]; then
{{ ELASTICCURL }} -s -k -L https://{{ NODEIP }}:9200/_nodes/stats | jq .nodes | jq ".[] | .ingest.pipelines" curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L https://{{ NODEIP }}:9200/_nodes/stats | jq .nodes | jq ".[] | .ingest.pipelines"
else else
{{ ELASTICCURL }} -s -k -L https://{{ NODEIP }}:9200/_nodes/stats | jq .nodes | jq ".[] | .ingest.pipelines.\"$1\"" curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L https://{{ NODEIP }}:9200/_nodes/stats | jq .nodes | jq ".[] | .ingest.pipelines.\"$1\""
fi fi

25
salt/common/tools/sbin/so-elasticsearch-pipeline-view
View File

@@ -1,25 +1,16 @@
#!/bin/bash #!/bin/bash
# #
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version. {%- set NODEIP = salt['pillar.get']('host:mainip', '') %}
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>
{%- set NODEIP = salt['pillar.get']('elasticsearch:mainip', '') -%}
. /usr/sbin/so-common . /usr/sbin/so-common
if [ "$1" == "" ]; then if [ "$1" == "" ]; then
{{ ELASTICCURL }} -s -k -L https://{{ NODEIP }}:9200/_ingest/pipeline/* | jq . curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L https://{{ NODEIP }}:9200/_ingest/pipeline/* | jq .
else else
{{ ELASTICCURL }} -s -k -L https://{{ NODEIP }}:9200/_ingest/pipeline/$1 | jq .[] curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L https://{{ NODEIP }}:9200/_ingest/pipeline/$1 | jq .[]
fi fi

25
salt/common/tools/sbin/so-elasticsearch-pipelines-list
View File

@@ -1,23 +1,14 @@
#!/bin/bash #!/bin/bash
# #
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version. {%- set NODEIP = salt['pillar.get']('host:mainip', '') %}
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
{%- set NODEIP = salt['pillar.get']('elasticsearch:mainip', '') -%}
. /usr/sbin/so-common . /usr/sbin/so-common
if [ "$1" == "" ]; then if [ "$1" == "" ]; then
{{ ELASTICCURL }} -s -k -L https://{{ NODEIP }}:9200/_ingest/pipeline/* | jq 'keys' curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L https://{{ NODEIP }}:9200/_ingest/pipeline/* | jq 'keys'
else else
{{ ELASTICCURL }} -s -k -L https://{{ NODEIP }}:9200/_ingest/pipeline/$1 | jq curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L https://{{ NODEIP }}:9200/_ingest/pipeline/$1 | jq
fi fi

21
salt/common/tools/sbin/so-elasticsearch-query
View File

@@ -1,19 +1,10 @@
#!/bin/bash #!/bin/bash
# #
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>
. /usr/sbin/so-common . /usr/sbin/so-common
if [[ $# -lt 1 ]]; then if [[ $# -lt 1 ]]; then
@@ -34,4 +25,4 @@ fi
QUERYPATH=$1 QUERYPATH=$1
shift shift
{{ ELASTICCURL }} -s -k -L -H "Content-Type: application/json" "https://localhost:9200/${QUERYPATH}" "$@" curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L -H "Content-Type: application/json" "https://localhost:9200/${QUERYPATH}" "$@"

20
salt/common/tools/sbin/so-elasticsearch-restart
View File

@@ -1,19 +1,11 @@
#!/bin/bash #!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common . /usr/sbin/so-common

23
salt/common/tools/sbin/so-elasticsearch-shards-list
View File

@@ -1,21 +1,12 @@
#!/bin/bash #!/bin/bash
# #
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version. {%- set NODEIP = salt['pillar.get']('host:mainip', '') %}
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>
{%- set NODEIP = salt['pillar.get']('elasticsearch:mainip', '') -%}
. /usr/sbin/so-common . /usr/sbin/so-common
{{ ELASTICCURL }} -s -k -L https://{{ NODEIP }}:9200/_cat/shards?pretty curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L https://{{ NODEIP }}:9200/_cat/shards?pretty

20
salt/common/tools/sbin/so-elasticsearch-start
View File

@@ -1,19 +1,11 @@
#!/bin/bash #!/bin/bash
# Copyright 2014-2022 Security Onion Solutions, LLC # Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# # or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# This program is free software: you can redistribute it and/or modify # https://securityonion.net/license; you may not use this file except in compliance with the
# it under the terms of the GNU General Public License as published by # Elastic License 2.0.
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
. /usr/sbin/so-common . /usr/sbin/so-common

Some files were not shown because too many files have changed in this diff Show More