Compare commits

..

54 Commits

Author SHA1 Message Date
reyesj2
16a4a42faf check for addon-index templates dir before attempting to load addon index templates 2026-04-14 19:26:37 -05:00
reyesj2
d598e20fbb soup 3.1.0 2026-04-14 14:55:33 -05:00
reyesj2
0405a66c72 enable elastic agent patch release for 9.3.3 2026-04-13 16:27:28 -05:00
reyesj2
a232cd89cc ES 9.3.3 2026-04-13 13:36:51 -05:00
reyesj2
dd40e44530 show when addon integrations are already loaded 2026-04-13 12:36:42 -05:00
reyesj2
29e13b2c0b elasticsearch ilm policy load script 2026-04-13 10:00:17 -05:00
reyesj2
abcad9fde0 addon statefile 2026-04-12 00:36:30 -05:00
reyesj2
a43947cca5 elasticsearch template load script -- for addon index templates 2026-04-12 00:23:26 -05:00
reyesj2
b0584a4dc5 only append "-mappings" to component template names as needed 2026-04-11 15:22:50 -05:00
reyesj2
6298397534 rework elasticsearch template load script -- for core templates 2026-04-11 04:40:47 -05:00
reyesj2
378d1ec81b initialize vars 2026-04-09 18:41:40 -05:00
reyesj2
6b8a6267da remove unused elasticsearch:index_template pillar references 2026-04-09 16:45:26 -05:00
reyesj2
89e49d0bf3 rework elasticsearch index template generation 2026-04-09 16:44:51 -05:00
reyesj2
f0b67a415a more filestream integration policy updates 2026-04-09 12:40:55 -05:00
reyesj2
dd56e7f1ac filestream integration policy updates 2026-04-07 11:08:10 -05:00
reyesj2
51a3c04c3d foxtrot version 2026-04-06 17:35:08 -05:00
reyesj2
8101bc4941 ES 9.3.2 2026-04-06 15:08:30 -05:00
reyesj2
51e0ca2602 Merge branch '3/main' of github.com:Security-Onion-Solutions/securityonion into reyesj2-es932 2026-04-01 14:46:05 -05:00
Jason Ertel
2a37ad82b2 Merge pull request #15704 from Security-Onion-Solutions/jertel/mainpr
pr/workflow changes
2026-04-01 10:55:57 -04:00
Jason Ertel
80540da52f pr/workflow changes 2026-04-01 10:48:47 -04:00
Jason Ertel
e4ba3d6a2a pr/workflow changes 2026-04-01 10:47:59 -04:00
Mike Reeves
bbfb58ea4e Merge pull request #15701 from Security-Onion-Solutions/TOoSmOotH-patch-1
Update SOUP_BRANCH to use 3/main instead of 2.4/main
2026-03-31 15:09:34 -04:00
Mike Reeves
c91deb97b1 Update SOUP_BRANCH to use 3/main instead of 2.4/main 2026-03-31 15:07:23 -04:00
reyesj2
dc2598d5cf Merge branch '3/main' of github.com:Security-Onion-Solutions/securityonion into HEAD 2026-03-31 14:01:58 -05:00
Mike Reeves
ff45e5ebc6 Merge pull request #15699 from Security-Onion-Solutions/TOoSmOotH-patch-4
Version Bump
2026-03-31 13:55:55 -04:00
Mike Reeves
1e2b51eae6 Add version 3.1.0 to discussion template options 2026-03-31 13:53:00 -04:00
Mike Reeves
58d332ea94 Bump version from 3.0.0 to 3.1.0 2026-03-31 13:52:07 -04:00
Mike Reeves
dcc67b9b8f Merge pull request #15696 from Security-Onion-Solutions/3/dev
3.0.0
2026-03-31 13:47:03 -04:00
Mike Reeves
cd886dd0f9 Merge pull request #15698 from Security-Onion-Solutions/merge-main-into-dev
Merge 3/main into 3/dev
2026-03-31 09:49:36 -04:00
Mike Reeves
37a6e28a6c Merge remote-tracking branch 'origin/3/dev' into merge-main-into-dev 2026-03-31 09:48:06 -04:00
Mike Reeves
434a2e7866 Merge pull request #15695 from Security-Onion-Solutions/3.0.0
3.0.0
2026-03-31 09:33:34 -04:00
Mike Reeves
79707db6ee 3.0.0 2026-03-31 09:17:08 -04:00
Josh Brower
0707507412 Merge pull request #15694 from Security-Onion-Solutions/fixpath
Remove hardcoded index
2026-03-30 12:47:55 -04:00
Josh Brower
c7e865aa1c Remove hardcoded index 2026-03-30 12:42:48 -04:00
Josh Brower
a89db79854 Merge pull request #15691 from Security-Onion-Solutions/jertel/wip
revisit workflows
2026-03-27 16:24:30 -04:00
Jason Ertel
812f65eee8 revisit workflows 2026-03-27 16:11:31 -04:00
Josh Patterson
cfa530ba9c Merge pull request #15690 from Security-Onion-Solutions/delta
ensure bool sliders soc
2026-03-27 15:19:30 -04:00
Josh Patterson
922c008b11 ensure bool sliders soc 2026-03-27 15:02:54 -04:00
Mike Reeves
ea30749512 Merge pull request #15676 from Security-Onion-Solutions/TOoSmOotH-patch-3
Make AI adapter settings visible
2026-03-26 09:43:58 -04:00
Mike Reeves
0a55592d7e Make AI adapter settings visible
Changed 'advanced' field from True to False for AI adapters and available models.
2026-03-26 09:37:39 -04:00
Josh Brower
115ca2c41d Merge pull request #15672 from Security-Onion-Solutions/yaracomments
update yara template
2026-03-24 15:59:48 -04:00
Josh Brower
9e53bd3f2d update yara template 2026-03-24 15:56:26 -04:00
Josh Brower
d4f1078f84 Merge pull request #15669 from Security-Onion-Solutions/lowercasefix
Lowercase network transport
2026-03-24 11:30:13 -04:00
Josh Brower
1f9bf45b66 Lowercase network transport 2026-03-24 11:24:59 -04:00
Mike Reeves
271de757e7 Merge pull request #15667 from Security-Onion-Solutions/TOoSmOotH-patch-1
Enable clean option for Zeek configuration
2026-03-24 09:56:03 -04:00
Mike Reeves
d4ac352b5a Enable clean option for Zeek configuration 2026-03-24 09:54:49 -04:00
Jorge Reyes
afcef1d0e7 Merge pull request #15661 from Security-Onion-Solutions/reyesj2-361
update stig profile v1r3
2026-03-23 18:09:33 -05:00
Josh Patterson
91b164b728 Merge pull request #15665 from Security-Onion-Solutions/delta
allow negation in suricata address-group vars
2026-03-23 17:34:21 -04:00
Josh Patterson
6a4501241d allow negation in suricata address-group vars 2026-03-23 17:24:12 -04:00
Josh Brower
c6978f9037 Merge pull request #15663 from Security-Onion-Solutions/fix/idh-skins
Remove hardcoded path
2026-03-23 16:30:51 -04:00
Jorge Reyes
fb7b73c601 Merge pull request #15662 from Security-Onion-Solutions/reyesj2-patch-1
exclude oscap profile from gitleaks
2026-03-23 14:23:24 -05:00
Jorge Reyes
f2b6d59c65 exclude oscap profile from gitleaks 2026-03-23 14:17:39 -05:00
reyesj2
67162357a3 update stig profile v1r3 2026-03-23 14:04:48 -05:00
Mike Reeves
cd0d88e2c0 Merge pull request #15440 from Security-Onion-Solutions/3/dev
Change version from 2.4.201 to UNRELEASED
2026-01-29 12:56:54 -05:00
90 changed files with 75965 additions and 44821 deletions

546
.github/.gitleaks.toml vendored
View File

@@ -1,546 +0,0 @@
title = "gitleaks config"
# Gitleaks rules are defined by regular expressions and entropy ranges.
# Some secrets have unique signatures which make detecting those secrets easy.
# Examples of those secrets would be GitLab Personal Access Tokens, AWS keys, and GitHub Access Tokens.
# All these examples have defined prefixes like `glpat`, `AKIA`, `ghp_`, etc.
#
# Other secrets might just be a hash which means we need to write more complex rules to verify
# that what we are matching is a secret.
#
# Here is an example of a semi-generic secret
#
# discord_client_secret = "8dyfuiRyq=vVc3RRr_edRk-fK__JItpZ"
#
# We can write a regular expression to capture the variable name (identifier),
# the assignment symbol (like '=' or ':='), and finally the actual secret.
# The structure of a rule to match this example secret is below:
#
# Beginning string
# quotation
# │ End string quotation
# │ │
# ▼ ▼
# (?i)(discord[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-z0-9=_\-]{32})['\"]
#
# ▲ ▲ ▲
# │ │ │
# │ │ │
# identifier assignment symbol
# Secret
#
[[rules]]
id = "gitlab-pat"
description = "GitLab Personal Access Token"
regex = '''glpat-[0-9a-zA-Z\-\_]{20}'''
[[rules]]
id = "aws-access-token"
description = "AWS"
regex = '''(A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}'''
# Cryptographic keys
[[rules]]
id = "PKCS8-PK"
description = "PKCS8 private key"
regex = '''-----BEGIN PRIVATE KEY-----'''
[[rules]]
id = "RSA-PK"
description = "RSA private key"
regex = '''-----BEGIN RSA PRIVATE KEY-----'''
[[rules]]
id = "OPENSSH-PK"
description = "SSH private key"
regex = '''-----BEGIN OPENSSH PRIVATE KEY-----'''
[[rules]]
id = "PGP-PK"
description = "PGP private key"
regex = '''-----BEGIN PGP PRIVATE KEY BLOCK-----'''
[[rules]]
id = "github-pat"
description = "GitHub Personal Access Token"
regex = '''ghp_[0-9a-zA-Z]{36}'''
[[rules]]
id = "github-oauth"
description = "GitHub OAuth Access Token"
regex = '''gho_[0-9a-zA-Z]{36}'''
[[rules]]
id = "SSH-DSA-PK"
description = "SSH (DSA) private key"
regex = '''-----BEGIN DSA PRIVATE KEY-----'''
[[rules]]
id = "SSH-EC-PK"
description = "SSH (EC) private key"
regex = '''-----BEGIN EC PRIVATE KEY-----'''
[[rules]]
id = "github-app-token"
description = "GitHub App Token"
regex = '''(ghu|ghs)_[0-9a-zA-Z]{36}'''
[[rules]]
id = "github-refresh-token"
description = "GitHub Refresh Token"
regex = '''ghr_[0-9a-zA-Z]{76}'''
[[rules]]
id = "shopify-shared-secret"
description = "Shopify shared secret"
regex = '''shpss_[a-fA-F0-9]{32}'''
[[rules]]
id = "shopify-access-token"
description = "Shopify access token"
regex = '''shpat_[a-fA-F0-9]{32}'''
[[rules]]
id = "shopify-custom-access-token"
description = "Shopify custom app access token"
regex = '''shpca_[a-fA-F0-9]{32}'''
[[rules]]
id = "shopify-private-app-access-token"
description = "Shopify private app access token"
regex = '''shppa_[a-fA-F0-9]{32}'''
[[rules]]
id = "slack-access-token"
description = "Slack token"
regex = '''xox[baprs]-([0-9a-zA-Z]{10,48})?'''
[[rules]]
id = "stripe-access-token"
description = "Stripe"
regex = '''(?i)(sk|pk)_(test|live)_[0-9a-z]{10,32}'''
[[rules]]
id = "pypi-upload-token"
description = "PyPI upload token"
regex = '''pypi-AgEIcHlwaS5vcmc[A-Za-z0-9\-_]{50,1000}'''
[[rules]]
id = "gcp-service-account"
description = "Google (GCP) Service-account"
regex = '''\"type\": \"service_account\"'''
[[rules]]
id = "heroku-api-key"
description = "Heroku API Key"
regex = ''' (?i)(heroku[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12})['\"]'''
secretGroup = 3
[[rules]]
id = "slack-web-hook"
description = "Slack Webhook"
regex = '''https://hooks.slack.com/services/T[a-zA-Z0-9_]{8}/B[a-zA-Z0-9_]{8,12}/[a-zA-Z0-9_]{24}'''
[[rules]]
id = "twilio-api-key"
description = "Twilio API Key"
regex = '''SK[0-9a-fA-F]{32}'''
[[rules]]
id = "age-secret-key"
description = "Age secret key"
regex = '''AGE-SECRET-KEY-1[QPZRY9X8GF2TVDW0S3JN54KHCE6MUA7L]{58}'''
[[rules]]
id = "facebook-token"
description = "Facebook token"
regex = '''(?i)(facebook[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-f0-9]{32})['\"]'''
secretGroup = 3
[[rules]]
id = "twitter-token"
description = "Twitter token"
regex = '''(?i)(twitter[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-f0-9]{35,44})['\"]'''
secretGroup = 3
[[rules]]
id = "adobe-client-id"
description = "Adobe Client ID (Oauth Web)"
regex = '''(?i)(adobe[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-f0-9]{32})['\"]'''
secretGroup = 3
[[rules]]
id = "adobe-client-secret"
description = "Adobe Client Secret"
regex = '''(p8e-)(?i)[a-z0-9]{32}'''
[[rules]]
id = "alibaba-access-key-id"
description = "Alibaba AccessKey ID"
regex = '''(LTAI)(?i)[a-z0-9]{20}'''
[[rules]]
id = "alibaba-secret-key"
description = "Alibaba Secret Key"
regex = '''(?i)(alibaba[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-z0-9]{30})['\"]'''
secretGroup = 3
[[rules]]
id = "asana-client-id"
description = "Asana Client ID"
regex = '''(?i)(asana[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([0-9]{16})['\"]'''
secretGroup = 3
[[rules]]
id = "asana-client-secret"
description = "Asana Client Secret"
regex = '''(?i)(asana[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-z0-9]{32})['\"]'''
secretGroup = 3
[[rules]]
id = "atlassian-api-token"
description = "Atlassian API token"
regex = '''(?i)(atlassian[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-z0-9]{24})['\"]'''
secretGroup = 3
[[rules]]
id = "bitbucket-client-id"
description = "Bitbucket client ID"
regex = '''(?i)(bitbucket[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-z0-9]{32})['\"]'''
secretGroup = 3
[[rules]]
id = "bitbucket-client-secret"
description = "Bitbucket client secret"
regex = '''(?i)(bitbucket[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-z0-9_\-]{64})['\"]'''
secretGroup = 3
[[rules]]
id = "beamer-api-token"
description = "Beamer API token"
regex = '''(?i)(beamer[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](b_[a-z0-9=_\-]{44})['\"]'''
secretGroup = 3
[[rules]]
id = "clojars-api-token"
description = "Clojars API token"
regex = '''(CLOJARS_)(?i)[a-z0-9]{60}'''
[[rules]]
id = "contentful-delivery-api-token"
description = "Contentful delivery API token"
regex = '''(?i)(contentful[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-z0-9\-=_]{43})['\"]'''
secretGroup = 3
[[rules]]
id = "databricks-api-token"
description = "Databricks API token"
regex = '''dapi[a-h0-9]{32}'''
[[rules]]
id = "discord-api-token"
description = "Discord API key"
regex = '''(?i)(discord[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-h0-9]{64})['\"]'''
secretGroup = 3
[[rules]]
id = "discord-client-id"
description = "Discord client ID"
regex = '''(?i)(discord[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([0-9]{18})['\"]'''
secretGroup = 3
[[rules]]
id = "discord-client-secret"
description = "Discord client secret"
regex = '''(?i)(discord[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-z0-9=_\-]{32})['\"]'''
secretGroup = 3
[[rules]]
id = "doppler-api-token"
description = "Doppler API token"
regex = '''['\"](dp\.pt\.)(?i)[a-z0-9]{43}['\"]'''
[[rules]]
id = "dropbox-api-secret"
description = "Dropbox API secret/key"
regex = '''(?i)(dropbox[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-z0-9]{15})['\"]'''
[[rules]]
id = "dropbox--api-key"
description = "Dropbox API secret/key"
regex = '''(?i)(dropbox[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-z0-9]{15})['\"]'''
[[rules]]
id = "dropbox-short-lived-api-token"
description = "Dropbox short lived API token"
regex = '''(?i)(dropbox[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](sl\.[a-z0-9\-=_]{135})['\"]'''
[[rules]]
id = "dropbox-long-lived-api-token"
description = "Dropbox long lived API token"
regex = '''(?i)(dropbox[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"][a-z0-9]{11}(AAAAAAAAAA)[a-z0-9\-_=]{43}['\"]'''
[[rules]]
id = "duffel-api-token"
description = "Duffel API token"
regex = '''['\"]duffel_(test|live)_(?i)[a-z0-9_-]{43}['\"]'''
[[rules]]
id = "dynatrace-api-token"
description = "Dynatrace API token"
regex = '''['\"]dt0c01\.(?i)[a-z0-9]{24}\.[a-z0-9]{64}['\"]'''
[[rules]]
id = "easypost-api-token"
description = "EasyPost API token"
regex = '''['\"]EZAK(?i)[a-z0-9]{54}['\"]'''
[[rules]]
id = "easypost-test-api-token"
description = "EasyPost test API token"
regex = '''['\"]EZTK(?i)[a-z0-9]{54}['\"]'''
[[rules]]
id = "fastly-api-token"
description = "Fastly API token"
regex = '''(?i)(fastly[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-z0-9\-=_]{32})['\"]'''
secretGroup = 3
[[rules]]
id = "finicity-client-secret"
description = "Finicity client secret"
regex = '''(?i)(finicity[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-z0-9]{20})['\"]'''
secretGroup = 3
[[rules]]
id = "finicity-api-token"
description = "Finicity API token"
regex = '''(?i)(finicity[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-f0-9]{32})['\"]'''
secretGroup = 3
[[rules]]
id = "flutterwave-public-key"
description = "Flutterwave public key"
regex = '''FLWPUBK_TEST-(?i)[a-h0-9]{32}-X'''
[[rules]]
id = "flutterwave-secret-key"
description = "Flutterwave secret key"
regex = '''FLWSECK_TEST-(?i)[a-h0-9]{32}-X'''
[[rules]]
id = "flutterwave-enc-key"
description = "Flutterwave encrypted key"
regex = '''FLWSECK_TEST[a-h0-9]{12}'''
[[rules]]
id = "frameio-api-token"
description = "Frame.io API token"
regex = '''fio-u-(?i)[a-z0-9\-_=]{64}'''
[[rules]]
id = "gocardless-api-token"
description = "GoCardless API token"
regex = '''['\"]live_(?i)[a-z0-9\-_=]{40}['\"]'''
[[rules]]
id = "grafana-api-token"
description = "Grafana API token"
regex = '''['\"]eyJrIjoi(?i)[a-z0-9\-_=]{72,92}['\"]'''
[[rules]]
id = "hashicorp-tf-api-token"
description = "HashiCorp Terraform user/org API token"
regex = '''['\"](?i)[a-z0-9]{14}\.atlasv1\.[a-z0-9\-_=]{60,70}['\"]'''
[[rules]]
id = "hubspot-api-token"
description = "HubSpot API token"
regex = '''(?i)(hubspot[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-h0-9]{8}-[a-h0-9]{4}-[a-h0-9]{4}-[a-h0-9]{4}-[a-h0-9]{12})['\"]'''
secretGroup = 3
[[rules]]
id = "intercom-api-token"
description = "Intercom API token"
regex = '''(?i)(intercom[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-z0-9=_]{60})['\"]'''
secretGroup = 3
[[rules]]
id = "intercom-client-secret"
description = "Intercom client secret/ID"
regex = '''(?i)(intercom[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-h0-9]{8}-[a-h0-9]{4}-[a-h0-9]{4}-[a-h0-9]{4}-[a-h0-9]{12})['\"]'''
secretGroup = 3
[[rules]]
id = "ionic-api-token"
description = "Ionic API token"
regex = '''(?i)(ionic[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](ion_[a-z0-9]{42})['\"]'''
[[rules]]
id = "linear-api-token"
description = "Linear API token"
regex = '''lin_api_(?i)[a-z0-9]{40}'''
[[rules]]
id = "linear-client-secret"
description = "Linear client secret/ID"
regex = '''(?i)(linear[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-f0-9]{32})['\"]'''
secretGroup = 3
[[rules]]
id = "lob-api-key"
description = "Lob API Key"
regex = '''(?i)(lob[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]((live|test)_[a-f0-9]{35})['\"]'''
secretGroup = 3
[[rules]]
id = "lob-pub-api-key"
description = "Lob Publishable API Key"
regex = '''(?i)(lob[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]((test|live)_pub_[a-f0-9]{31})['\"]'''
secretGroup = 3
[[rules]]
id = "mailchimp-api-key"
description = "Mailchimp API key"
regex = '''(?i)(mailchimp[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-f0-9]{32}-us20)['\"]'''
secretGroup = 3
[[rules]]
id = "mailgun-private-api-token"
description = "Mailgun private API token"
regex = '''(?i)(mailgun[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](key-[a-f0-9]{32})['\"]'''
secretGroup = 3
[[rules]]
id = "mailgun-pub-key"
description = "Mailgun public validation key"
regex = '''(?i)(mailgun[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](pubkey-[a-f0-9]{32})['\"]'''
secretGroup = 3
[[rules]]
id = "mailgun-signing-key"
description = "Mailgun webhook signing key"
regex = '''(?i)(mailgun[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-h0-9]{32}-[a-h0-9]{8}-[a-h0-9]{8})['\"]'''
secretGroup = 3
[[rules]]
id = "mapbox-api-token"
description = "Mapbox API token"
regex = '''(?i)(pk\.[a-z0-9]{60}\.[a-z0-9]{22})'''
[[rules]]
id = "messagebird-api-token"
description = "MessageBird API token"
regex = '''(?i)(messagebird[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-z0-9]{25})['\"]'''
secretGroup = 3
[[rules]]
id = "messagebird-client-id"
description = "MessageBird API client ID"
regex = '''(?i)(messagebird[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-h0-9]{8}-[a-h0-9]{4}-[a-h0-9]{4}-[a-h0-9]{4}-[a-h0-9]{12})['\"]'''
secretGroup = 3
[[rules]]
id = "new-relic-user-api-key"
description = "New Relic user API Key"
regex = '''['\"](NRAK-[A-Z0-9]{27})['\"]'''
[[rules]]
id = "new-relic-user-api-id"
description = "New Relic user API ID"
regex = '''(?i)(newrelic[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([A-Z0-9]{64})['\"]'''
secretGroup = 3
[[rules]]
id = "new-relic-browser-api-token"
description = "New Relic ingest browser API token"
regex = '''['\"](NRJS-[a-f0-9]{19})['\"]'''
[[rules]]
id = "npm-access-token"
description = "npm access token"
regex = '''['\"](npm_(?i)[a-z0-9]{36})['\"]'''
[[rules]]
id = "planetscale-password"
description = "PlanetScale password"
regex = '''pscale_pw_(?i)[a-z0-9\-_\.]{43}'''
[[rules]]
id = "planetscale-api-token"
description = "PlanetScale API token"
regex = '''pscale_tkn_(?i)[a-z0-9\-_\.]{43}'''
[[rules]]
id = "postman-api-token"
description = "Postman API token"
regex = '''PMAK-(?i)[a-f0-9]{24}\-[a-f0-9]{34}'''
[[rules]]
id = "pulumi-api-token"
description = "Pulumi API token"
regex = '''pul-[a-f0-9]{40}'''
[[rules]]
id = "rubygems-api-token"
description = "Rubygem API token"
regex = '''rubygems_[a-f0-9]{48}'''
[[rules]]
id = "sendgrid-api-token"
description = "SendGrid API token"
regex = '''SG\.(?i)[a-z0-9_\-\.]{66}'''
[[rules]]
id = "sendinblue-api-token"
description = "Sendinblue API token"
regex = '''xkeysib-[a-f0-9]{64}\-(?i)[a-z0-9]{16}'''
[[rules]]
id = "shippo-api-token"
description = "Shippo API token"
regex = '''shippo_(live|test)_[a-f0-9]{40}'''
[[rules]]
id = "linkedin-client-secret"
description = "LinkedIn Client secret"
regex = '''(?i)(linkedin[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-z]{16})['\"]'''
secretGroup = 3
[[rules]]
id = "linkedin-client-id"
description = "LinkedIn Client ID"
regex = '''(?i)(linkedin[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-z0-9]{14})['\"]'''
secretGroup = 3
[[rules]]
id = "twitch-api-token"
description = "Twitch API token"
regex = '''(?i)(twitch[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-z0-9]{30})['\"]'''
secretGroup = 3
[[rules]]
id = "typeform-api-token"
description = "Typeform API token"
regex = '''(?i)(typeform[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}(tfp_[a-z0-9\-_\.=]{59})'''
secretGroup = 3
[[rules]]
id = "generic-api-key"
description = "Generic API Key"
regex = '''(?i)((key|api[^Version]|token|secret|password)[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([0-9a-zA-Z\-_=]{8,64})['\"]'''
entropy = 3.7
secretGroup = 4
[allowlist]
description = "global allow lists"
regexes = ['''219-09-9999''', '''078-05-1120''', '''(9[0-9]{2}|666)-\d{2}-\d{4}''', '''RPM-GPG-KEY.*''', '''.*:.*StrelkaHexDump.*''', '''.*:.*PLACEHOLDER.*''', '''ssl_.*password''', '''integration_key\s=\s"so-logs-"''']
paths = [
'''gitleaks.toml''',
'''(.*?)(jpg|gif|doc|pdf|bin|svg|socket)$''',
'''(go.mod|go.sum)$''',
'''salt/nginx/files/enterprise-attack.json''',
'''(.*?)whl$'''
]

View File

@@ -10,6 +10,7 @@ body:
options:
-
- 3.0.0
- 3.1.0
- Other (please provide detail below)
validations:
required: true

22
.github/pull_request_template.md vendored Normal file
View File

@@ -0,0 +1,22 @@
## Description
<!--
Explain the purpose of the pull request. Be brief or detailed depending on the scope of the changes.
-->
## Related Issues
<!--
Optionally, list any related issues that this pull request addresses.
-->
## Checklist
- [ ] I have read and followed the [CONTRIBUTING.md](https://github.com/Security-Onion-Solutions/securityonion/blob/3/main/CONTRIBUTING.md) file.
- [ ] I have read and agree to the terms of the [Contributor License Agreement](https://securityonionsolutions.com/cla)
## Questions or Comments
<!--
If you have any questions or comments about this pull request, add them here.
-->

View File

@@ -1,24 +0,0 @@
name: contrib
on:
issue_comment:
types: [created]
pull_request_target:
types: [opened,closed,synchronize]
jobs:
CLAssistant:
runs-on: ubuntu-latest
steps:
- name: "Contributor Check"
if: (github.event.comment.body == 'recheck' || github.event.comment.body == 'I have read the CLA Document and I hereby sign the CLA') || github.event_name == 'pull_request_target'
uses: cla-assistant/github-action@v2.3.1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PERSONAL_ACCESS_TOKEN : ${{ secrets.PERSONAL_ACCESS_TOKEN }}
with:
path-to-signatures: 'signatures_v1.json'
path-to-document: 'https://securityonionsolutions.com/cla'
allowlist: dependabot[bot],jertel,dougburks,TOoSmOotH,defensivedepth,m0duspwnens
remote-organization-name: Security-Onion-Solutions
remote-repository-name: licensing

View File

@@ -1,17 +0,0 @@
name: leak-test
on: [pull_request]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: '0'
- name: Gitleaks
uses: gitleaks/gitleaks-action@v1.6.0
with:
config-path: .github/.gitleaks.toml

View File

@@ -23,7 +23,7 @@
* Link the PR to the related issue, either using [keywords](https://docs.github.com/en/issues/tracking-your-work-with-issues/creating-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword) in the PR description, or [manually](https://docs.github.com/en/issues/tracking-your-work-with-issues/creating-issues/linking-a-pull-request-to-an-issue#manually-linking-a-pull-request-to-an-issue).
* **Pull requests should be opened against the `dev` branch of this repo**, and should clearly describe the problem and solution.
* **Pull requests should be opened against the current `?/dev` branch of this repo**, and should clearly describe the problem and solution.
* Be sure you have tested your changes and are confident they will not break other parts of the product.

View File

@@ -1,46 +1,46 @@
### 2.4.210-20260302 ISO image released on 2026/03/02
### 3.0.0-20260331 ISO image released on 2026/03/31
### Download and Verify
2.4.210-20260302 ISO image:
https://download.securityonion.net/file/securityonion/securityonion-2.4.210-20260302.iso
3.0.0-20260331 ISO image:
https://download.securityonion.net/file/securityonion/securityonion-3.0.0-20260331.iso
MD5: 575F316981891EBED2EE4E1F42A1F016
SHA1: 600945E8823221CBC5F1C056084A71355308227E
SHA256: A6AA6471125F07FA6E2796430E94BEAFDEF728E833E9728FDFA7106351EBC47E
MD5: ECD318A1662A6FDE0EF213F5A9BD4B07
SHA1: E55BE314440CCF3392DC0B06BC5E270B43176D9C
SHA256: 7FC47405E335CBE5C2B6C51FE7AC60248F35CBE504907B8B5A33822B23F8F4D5
Signature for ISO image:
https://github.com/Security-Onion-Solutions/securityonion/raw/2.4/main/sigs/securityonion-2.4.210-20260302.iso.sig
https://github.com/Security-Onion-Solutions/securityonion/raw/3/main/sigs/securityonion-3.0.0-20260331.iso.sig
Signing key:
https://raw.githubusercontent.com/Security-Onion-Solutions/securityonion/2.4/main/KEYS
https://raw.githubusercontent.com/Security-Onion-Solutions/securityonion/3/main/KEYS
For example, here are the steps you can use on most Linux distributions to download and verify our Security Onion ISO image.
Download and import the signing key:
```
wget https://raw.githubusercontent.com/Security-Onion-Solutions/securityonion/2.4/main/KEYS -O - | gpg --import -
wget https://raw.githubusercontent.com/Security-Onion-Solutions/securityonion/3/main/KEYS -O - | gpg --import -
```
Download the signature file for the ISO:
```
wget https://github.com/Security-Onion-Solutions/securityonion/raw/2.4/main/sigs/securityonion-2.4.210-20260302.iso.sig
wget https://github.com/Security-Onion-Solutions/securityonion/raw/3/main/sigs/securityonion-3.0.0-20260331.iso.sig
```
Download the ISO image:
```
wget https://download.securityonion.net/file/securityonion/securityonion-2.4.210-20260302.iso
wget https://download.securityonion.net/file/securityonion/securityonion-3.0.0-20260331.iso
```
Verify the downloaded ISO image using the signature file:
```
gpg --verify securityonion-2.4.210-20260302.iso.sig securityonion-2.4.210-20260302.iso
gpg --verify securityonion-3.0.0-20260331.iso.sig securityonion-3.0.0-20260331.iso
```
The output should show "Good signature" and the Primary key fingerprint should match what's shown below:
```
gpg: Signature made Mon 02 Mar 2026 11:55:24 AM EST using RSA key ID FE507013
gpg: Signature made Mon 30 Mar 2026 06:22:14 PM EDT using RSA key ID FE507013
gpg: Good signature from "Security Onion Solutions, LLC <info@securityonionsolutions.com>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.

View File

@@ -1 +1 @@
3.0.0
3.0.0-foxtrot

View File

@@ -1,2 +0,0 @@
elasticsearch:
index_settings:

View File

@@ -97,7 +97,6 @@ base:
- node_data.ips
- secrets
- healthcheck.eval
- elasticsearch.index_templates
{% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/elasticsearch/auth.sls') %}
- elasticsearch.auth
{% endif %}
@@ -142,7 +141,6 @@ base:
- logstash.nodes
- logstash.soc_logstash
- logstash.adv_logstash
- elasticsearch.index_templates
{% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/elasticsearch/auth.sls') %}
- elasticsearch.auth
{% endif %}
@@ -256,7 +254,6 @@ base:
'*_import':
- node_data.ips
- secrets
- elasticsearch.index_templates
{% if salt['file.file_exists']('/opt/so/saltstack/local/pillar/elasticsearch/auth.sls') %}
- elasticsearch.auth
{% endif %}

View File

@@ -0,0 +1,123 @@
{# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
or more contributor license agreements. Licensed under the Elastic License 2.0; you may not use
this file except in compliance with the Elastic License 2.0. #}
{% import_json '/opt/so/state/esfleet_content_package_components.json' as ADDON_CONTENT_PACKAGE_COMPONENTS %}
{% import_json '/opt/so/state/esfleet_component_templates.json' as INSTALLED_COMPONENT_TEMPLATES %}
{% import_yaml 'elasticfleet/defaults.yaml' as ELASTICFLEETDEFAULTS %}
{% set CORE_ESFLEET_PACKAGES = ELASTICFLEETDEFAULTS.get('elasticfleet', {}).get('packages', {}) %}
{% set ADDON_CONTENT_INTEGRATION_DEFAULTS = {} %}
{% set DEBUG_STUFF = {} %}
{% for pkg in ADDON_CONTENT_PACKAGE_COMPONENTS %}
{% if pkg.name in CORE_ESFLEET_PACKAGES %}
{# skip core content packages #}
{% elif pkg.name not in CORE_ESFLEET_PACKAGES %}
{# generate defaults for each content package #}
{% if pkg.dataStreams is defined and pkg.dataStreams is not none and pkg.dataStreams | length > 0%}
{% for pattern in pkg.dataStreams %}
{# in ES 9.3.2 'input' type integrations no longer create default component templates and instead they wait for user input during 'integration' setup (fleet ui config)
title: generic is an artifact of that and is not in use #}
{% if pattern.title == "generic" %}
{% continue %}
{% endif %}
{% if "metrics-" in pattern.name %}
{% set integration_type = "metrics-" %}
{% elif "logs-" in pattern.name %}
{% set integration_type = "logs-" %}
{% else %}
{% set integration_type = "" %}
{% endif %}
{# on content integrations the component name is user defined at the time it is added to an agent policy #}
{% set component_name = pattern.title %}
{% set index_pattern = pattern.name %}
{# component_name_x maintains the functionality of merging local pillar changes with generated 'defaults' via SOC UI #}
{% set component_name_x = component_name.replace(".","_x_") %}
{# pillar overrides/merge expects the key names to follow the naming in elasticsearch/defaults.yaml eg. so-logs-1password_x_item_usages . The _x_ is replaced later on in elasticsearch/template.map.jinja #}
{% set integration_key = "so-" ~ integration_type ~ pkg.name + '_x_' ~ component_name_x %}
{# Default integration settings #}
{% set integration_defaults = {
"index_sorting": false,
"index_template": {
"composed_of": [integration_type ~ component_name ~ "@package", integration_type ~ component_name ~ "@custom", "so-fleet_integrations.ip_mappings-1", "so-fleet_globals-1", "so-fleet_agent_id_verification-1"],
"data_stream": {
"allow_custom_routing": false,
"hidden": false
},
"ignore_missing_component_templates": [integration_type ~ component_name ~ "@custom"],
"index_patterns": [index_pattern],
"priority": 501,
"template": {
"settings": {
"index": {
"lifecycle": {"name": "so-" ~ integration_type ~ component_name ~ "-logs"},
"number_of_replicas": 0
}
}
}
},
"policy": {
"phases": {
"cold": {
"actions": {
"allocate":{
"number_of_replicas": ""
},
"set_priority": {"priority": 0}
},
"min_age": "60d"
},
"delete": {
"actions": {
"delete": {}
},
"min_age": "365d"
},
"hot": {
"actions": {
"rollover": {
"max_age": "30d",
"max_primary_shard_size": "50gb"
},
"forcemerge":{
"max_num_segments": ""
},
"shrink":{
"max_primary_shard_size": "",
"method": "COUNT",
"number_of_shards": ""
},
"set_priority": {"priority": 100}
},
"min_age": "0ms"
},
"warm": {
"actions": {
"allocate": {
"number_of_replicas": ""
},
"forcemerge": {
"max_num_segments": ""
},
"shrink":{
"max_primary_shard_size": "",
"method": "COUNT",
"number_of_shards": ""
},
"set_priority": {"priority": 50}
},
"min_age": "30d"
}
}
}
} %}
{% do ADDON_CONTENT_INTEGRATION_DEFAULTS.update({integration_key: integration_defaults}) %}
{% endfor %}
{% else %}
{% endif %}
{% endif %}
{% endfor %}

View File

@@ -1,5 +1,6 @@
elasticfleet:
enabled: False
patch_version: 9.3.3+build202604082258 # Elastic Agent specific patch release.
enable_manager_output: True
config:
server:

View File

@@ -9,16 +9,22 @@
"namespace": "so",
"description": "Zeek Import logs",
"policy_id": "so-grid-nodes_general",
"policy_ids": [
"so-grid-nodes_general"
],
"vars": {},
"inputs": {
"filestream-filestream": {
"enabled": true,
"streams": {
"filestream.generic": {
"filestream.filestream": {
"enabled": true,
"vars": {
"paths": [
"/nsm/import/*/zeek/logs/*.log"
],
"compression_gzip": false,
"use_logs_stream": false,
"data_stream.dataset": "import",
"pipeline": "",
"parsers": "#- ndjson:\n# target: \"\"\n# message_key: msg\n#- multiline:\n# type: count\n# count_lines: 3\n",
@@ -34,7 +40,8 @@
"fingerprint_length": "64",
"file_identity_native": true,
"exclude_lines": [],
"include_lines": []
"include_lines": [],
"delete_enabled": false
}
}
}

View File

@@ -15,19 +15,25 @@
"version": ""
},
"name": "kratos-logs",
"namespace": "so",
"description": "Kratos logs",
"policy_id": "so-grid-nodes_general",
"namespace": "so",
"policy_ids": [
"so-grid-nodes_general"
],
"vars": {},
"inputs": {
"filestream-filestream": {
"enabled": true,
"streams": {
"filestream.generic": {
"filestream.filestream": {
"enabled": true,
"vars": {
"paths": [
"/opt/so/log/kratos/kratos.log"
],
"compression_gzip": false,
"use_logs_stream": false,
"data_stream.dataset": "kratos",
"pipeline": "kratos",
"parsers": "#- ndjson:\n# target: \"\"\n# message_key: msg\n#- multiline:\n# type: count\n# count_lines: 3\n",
@@ -48,10 +54,10 @@
"harvester_limit": 0,
"fingerprint": false,
"fingerprint_offset": 0,
"fingerprint_length": "64",
"file_identity_native": true,
"exclude_lines": [],
"include_lines": []
"include_lines": [],
"delete_enabled": false
}
}
}

View File

@@ -9,16 +9,22 @@
"namespace": "so",
"description": "Zeek logs",
"policy_id": "so-grid-nodes_general",
"policy_ids": [
"so-grid-nodes_general"
],
"vars": {},
"inputs": {
"filestream-filestream": {
"enabled": true,
"streams": {
"filestream.generic": {
"filestream.filestream": {
"enabled": true,
"vars": {
"paths": [
"/nsm/zeek/logs/current/*.log"
],
"compression_gzip": false,
"use_logs_stream": false,
"data_stream.dataset": "zeek",
"parsers": "#- ndjson:\n# target: \"\"\n# message_key: msg\n#- multiline:\n# type: count\n# count_lines: 3\n",
"exclude_files": ["({%- endraw -%}{{ ELASTICFLEETMERGED.logging.zeek.excluded | join('|') }}{%- raw -%})(\\..+)?\\.log$"],
@@ -30,10 +36,10 @@
"harvester_limit": 0,
"fingerprint": false,
"fingerprint_offset": 0,
"fingerprint_length": "64",
"file_identity_native": true,
"exclude_lines": [],
"include_lines": []
"include_lines": [],
"delete_enabled": false
}
}
}

View File

@@ -5,7 +5,7 @@
"package": {
"name": "endpoint",
"title": "Elastic Defend",
"version": "9.0.2",
"version": "9.3.0",
"requires_root": true
},
"enabled": true,

View File

@@ -6,21 +6,23 @@
"name": "agent-monitor",
"namespace": "",
"description": "",
"policy_id": "so-grid-nodes_general",
"policy_ids": [
"so-grid-nodes_general"
],
"output_id": null,
"vars": {},
"inputs": {
"filestream-filestream": {
"enabled": true,
"streams": {
"filestream.generic": {
"filestream.filestream": {
"enabled": true,
"vars": {
"paths": [
"/opt/so/log/agents/agent-monitor.log"
],
"compression_gzip": false,
"use_logs_stream": false,
"data_stream.dataset": "agentmonitor",
"pipeline": "elasticagent.monitor",
"parsers": "",
@@ -34,15 +36,16 @@
"ignore_older": "72h",
"clean_inactive": -1,
"harvester_limit": 0,
"fingerprint": true,
"fingerprint": false,
"fingerprint_offset": 0,
"fingerprint_length": 64,
"file_identity_native": false,
"file_identity_native": true,
"exclude_lines": [],
"include_lines": []
"include_lines": [],
"delete_enabled": false
}
}
}
}
}
},
"force": true
}

View File

@@ -4,19 +4,25 @@
"version": ""
},
"name": "hydra-logs",
"namespace": "so",
"description": "Hydra logs",
"policy_id": "so-grid-nodes_general",
"namespace": "so",
"policy_ids": [
"so-grid-nodes_general"
],
"vars": {},
"inputs": {
"filestream-filestream": {
"enabled": true,
"streams": {
"filestream.generic": {
"filestream.filestream": {
"enabled": true,
"vars": {
"paths": [
"/opt/so/log/hydra/hydra.log"
],
"compression_gzip": false,
"use_logs_stream": false,
"data_stream.dataset": "hydra",
"pipeline": "hydra",
"parsers": "#- ndjson:\n# target: \"\"\n# message_key: msg\n#- multiline:\n# type: count\n# count_lines: 3\n",
@@ -34,10 +40,10 @@
"harvester_limit": 0,
"fingerprint": false,
"fingerprint_offset": 0,
"fingerprint_length": "64",
"file_identity_native": true,
"exclude_lines": [],
"include_lines": []
"include_lines": [],
"delete_enabled": false
}
}
}

View File

@@ -4,19 +4,25 @@
"version": ""
},
"name": "idh-logs",
"namespace": "so",
"description": "IDH integration",
"policy_id": "so-grid-nodes_general",
"namespace": "so",
"policy_ids": [
"so-grid-nodes_general"
],
"vars": {},
"inputs": {
"filestream-filestream": {
"enabled": true,
"streams": {
"filestream.generic": {
"filestream.filestream": {
"enabled": true,
"vars": {
"paths": [
"/nsm/idh/opencanary.log"
],
"compression_gzip": false,
"use_logs_stream": false,
"data_stream.dataset": "idh",
"pipeline": "common",
"parsers": "#- ndjson:\n# target: \"\"\n# message_key: msg\n#- multiline:\n# type: count\n# count_lines: 3\n",
@@ -31,10 +37,10 @@
"harvester_limit": 0,
"fingerprint": false,
"fingerprint_offset": 0,
"fingerprint_length": "64",
"file_identity_native": true,
"exclude_lines": [],
"include_lines": []
"include_lines": [],
"delete_enabled": false
}
}
}

View File

@@ -4,26 +4,32 @@
"version": ""
},
"name": "import-evtx-logs",
"namespace": "so",
"description": "Import Windows EVTX logs",
"policy_id": "so-grid-nodes_general",
"namespace": "so",
"policy_ids": [
"so-grid-nodes_general"
],
"vars": {},
"inputs": {
"filestream-filestream": {
"enabled": true,
"streams": {
"filestream.generic": {
"filestream.filestream": {
"enabled": true,
"vars": {
"paths": [
"/nsm/import/*/evtx/*.json"
],
"compression_gzip": false,
"use_logs_stream": false,
"data_stream.dataset": "import",
"parsers": "#- ndjson:\n# target: \"\"\n# message_key: msg\n#- multiline:\n# type: count\n# count_lines: 3\n",
"exclude_files": [
"\\.gz$"
],
"include_files": [],
"processors": "- dissect:\n tokenizer: \"/nsm/import/%{import.id}/evtx/%{import.file}\"\n field: \"log.file.path\"\n target_prefix: \"\"\n- decode_json_fields:\n fields: [\"message\"]\n target: \"\"\n- drop_fields:\n fields: [\"host\"]\n ignore_missing: true\n- add_fields:\n target: data_stream\n fields:\n type: logs\n dataset: system.security\n- add_fields:\n target: event\n fields:\n dataset: system.security\n module: system\n imported: true\n- add_fields:\n target: \"@metadata\"\n fields:\n pipeline: logs-system.security-2.6.1\n- if:\n equals:\n winlog.channel: 'Microsoft-Windows-Sysmon/Operational'\n then: \n - add_fields:\n target: data_stream\n fields:\n dataset: windows.sysmon_operational\n - add_fields:\n target: event\n fields:\n dataset: windows.sysmon_operational\n module: windows\n imported: true\n - add_fields:\n target: \"@metadata\"\n fields:\n pipeline: logs-windows.sysmon_operational-3.1.2\n- if:\n equals:\n winlog.channel: 'Application'\n then: \n - add_fields:\n target: data_stream\n fields:\n dataset: system.application\n - add_fields:\n target: event\n fields:\n dataset: system.application\n - add_fields:\n target: \"@metadata\"\n fields:\n pipeline: logs-system.application-2.6.1\n- if:\n equals:\n winlog.channel: 'System'\n then: \n - add_fields:\n target: data_stream\n fields:\n dataset: system.system\n - add_fields:\n target: event\n fields:\n dataset: system.system\n - add_fields:\n target: \"@metadata\"\n fields:\n pipeline: logs-system.system-2.6.1\n \n- if:\n equals:\n winlog.channel: 'Microsoft-Windows-PowerShell/Operational'\n then: \n - add_fields:\n target: data_stream\n fields:\n dataset: windows.powershell_operational\n - add_fields:\n target: event\n fields:\n dataset: windows.powershell_operational\n module: windows\n - add_fields:\n target: \"@metadata\"\n fields:\n pipeline: logs-windows.powershell_operational-3.1.2\n- add_fields:\n target: data_stream\n fields:\n dataset: import",
"processors": "- dissect:\n tokenizer: \"/nsm/import/%{import.id}/evtx/%{import.file}\"\n field: \"log.file.path\"\n target_prefix: \"\"\n- decode_json_fields:\n fields: [\"message\"]\n target: \"\"\n- drop_fields:\n fields: [\"host\"]\n ignore_missing: true\n- add_fields:\n target: data_stream\n fields:\n type: logs\n dataset: system.security\n- add_fields:\n target: event\n fields:\n dataset: system.security\n module: system\n imported: true\n- add_fields:\n target: \"@metadata\"\n fields:\n pipeline: logs-system.security-2.15.0\n- if:\n equals:\n winlog.channel: 'Microsoft-Windows-Sysmon/Operational'\n then: \n - add_fields:\n target: data_stream\n fields:\n dataset: windows.sysmon_operational\n - add_fields:\n target: event\n fields:\n dataset: windows.sysmon_operational\n module: windows\n imported: true\n - add_fields:\n target: \"@metadata\"\n fields:\n pipeline: logs-windows.sysmon_operational-3.8.0\n- if:\n equals:\n winlog.channel: 'Application'\n then: \n - add_fields:\n target: data_stream\n fields:\n dataset: system.application\n - add_fields:\n target: event\n fields:\n dataset: system.application\n - add_fields:\n target: \"@metadata\"\n fields:\n pipeline: logs-system.application-2.15.0\n- if:\n equals:\n winlog.channel: 'System'\n then: \n - add_fields:\n target: data_stream\n fields:\n dataset: system.system\n - add_fields:\n target: event\n fields:\n dataset: system.system\n - add_fields:\n target: \"@metadata\"\n fields:\n pipeline: logs-system.system-2.15.0\n \n- if:\n equals:\n winlog.channel: 'Microsoft-Windows-PowerShell/Operational'\n then: \n - add_fields:\n target: data_stream\n fields:\n dataset: windows.powershell_operational\n - add_fields:\n target: event\n fields:\n dataset: windows.powershell_operational\n module: windows\n - add_fields:\n target: \"@metadata\"\n fields:\n pipeline: logs-windows.powershell_operational-3.8.0\n- add_fields:\n target: data_stream\n fields:\n dataset: import",
"tags": [
"import"
],
@@ -33,10 +39,10 @@
"harvester_limit": 0,
"fingerprint": false,
"fingerprint_offset": 0,
"fingerprint_length": "64",
"file_identity_native": true,
"exclude_lines": [],
"include_lines": []
"include_lines": [],
"delete_enabled": false
}
}
}

View File

@@ -4,19 +4,25 @@
"version": ""
},
"name": "import-suricata-logs",
"namespace": "so",
"description": "Import Suricata logs",
"policy_id": "so-grid-nodes_general",
"namespace": "so",
"policy_ids": [
"so-grid-nodes_general"
],
"vars": {},
"inputs": {
"filestream-filestream": {
"enabled": true,
"streams": {
"filestream.generic": {
"filestream.filestream": {
"enabled": true,
"vars": {
"paths": [
"/nsm/import/*/suricata/eve*.json"
],
"compression_gzip": false,
"use_logs_stream": false,
"data_stream.dataset": "import",
"pipeline": "suricata.common",
"parsers": "#- ndjson:\n# target: \"\"\n# message_key: msg\n#- multiline:\n# type: count\n# count_lines: 3\n",
@@ -32,10 +38,10 @@
"harvester_limit": 0,
"fingerprint": false,
"fingerprint_offset": 0,
"fingerprint_length": "64",
"file_identity_native": true,
"exclude_lines": [],
"include_lines": []
"include_lines": [],
"delete_enabled": false
}
}
}

View File

@@ -4,14 +4,18 @@
"version": ""
},
"name": "rita-logs",
"namespace": "so",
"description": "RITA Logs",
"policy_id": "so-grid-nodes_general",
"namespace": "so",
"policy_ids": [
"so-grid-nodes_general"
],
"vars": {},
"inputs": {
"filestream-filestream": {
"enabled": true,
"streams": {
"filestream.generic": {
"filestream.filestream": {
"enabled": true,
"vars": {
"paths": [
@@ -19,6 +23,8 @@
"/nsm/rita/exploded-dns.csv",
"/nsm/rita/long-connections.csv"
],
"compression_gzip": false,
"use_logs_stream": false,
"data_stream.dataset": "rita",
"parsers": "#- ndjson:\n# target: \"\"\n# message_key: msg\n#- multiline:\n# type: count\n# count_lines: 3\n",
"exclude_files": [
@@ -33,10 +39,10 @@
"harvester_limit": 0,
"fingerprint": false,
"fingerprint_offset": 0,
"fingerprint_length": "64",
"file_identity_native": true,
"exclude_lines": [],
"include_lines": []
"include_lines": [],
"delete_enabled": false
}
}
}

View File

@@ -4,19 +4,25 @@
"version": ""
},
"name": "so-ip-mappings",
"namespace": "so",
"description": "IP Description mappings",
"policy_id": "so-grid-nodes_general",
"namespace": "so",
"policy_ids": [
"so-grid-nodes_general"
],
"vars": {},
"inputs": {
"filestream-filestream": {
"enabled": true,
"streams": {
"filestream.generic": {
"filestream.filestream": {
"enabled": true,
"vars": {
"paths": [
"/nsm/custom-mappings/ip-descriptions.csv"
],
"compression_gzip": false,
"use_logs_stream": false,
"data_stream.dataset": "hostnamemappings",
"parsers": "#- ndjson:\n# target: \"\"\n# message_key: msg\n#- multiline:\n# type: count\n# count_lines: 3\n",
"exclude_files": [
@@ -32,10 +38,10 @@
"harvester_limit": 0,
"fingerprint": false,
"fingerprint_offset": 0,
"fingerprint_length": "64",
"file_identity_native": true,
"exclude_lines": [],
"include_lines": []
"include_lines": [],
"delete_enabled": false
}
}
}

View File

@@ -4,19 +4,25 @@
"version": ""
},
"name": "soc-auth-sync-logs",
"namespace": "so",
"description": "Security Onion - Elastic Auth Sync - Logs",
"policy_id": "so-grid-nodes_general",
"namespace": "so",
"policy_ids": [
"so-grid-nodes_general"
],
"vars": {},
"inputs": {
"filestream-filestream": {
"enabled": true,
"streams": {
"filestream.generic": {
"filestream.filestream": {
"enabled": true,
"vars": {
"paths": [
"/opt/so/log/soc/sync.log"
],
"compression_gzip": false,
"use_logs_stream": false,
"data_stream.dataset": "soc",
"pipeline": "common",
"parsers": "#- ndjson:\n# target: \"\"\n# message_key: msg\n#- multiline:\n# type: count\n# count_lines: 3\n",
@@ -31,10 +37,10 @@
"harvester_limit": 0,
"fingerprint": false,
"fingerprint_offset": 0,
"fingerprint_length": "64",
"file_identity_native": true,
"exclude_lines": [],
"include_lines": []
"include_lines": [],
"delete_enabled": false
}
}
}

View File

@@ -4,20 +4,26 @@
"version": ""
},
"name": "soc-detections-logs",
"namespace": "so",
"description": "Security Onion Console - Detections Logs",
"policy_id": "so-grid-nodes_general",
"namespace": "so",
"policy_ids": [
"so-grid-nodes_general"
],
"vars": {},
"inputs": {
"filestream-filestream": {
"enabled": true,
"streams": {
"filestream.generic": {
"filestream.filestream": {
"enabled": true,
"vars": {
"paths": [
"/opt/so/log/soc/detections_runtime-status_sigma.log",
"/opt/so/log/soc/detections_runtime-status_yara.log"
],
"compression_gzip": false,
"use_logs_stream": false,
"data_stream.dataset": "soc",
"pipeline": "common",
"parsers": "#- ndjson:\n# target: \"\"\n# message_key: msg\n#- multiline:\n# type: count\n# count_lines: 3\n",
@@ -35,10 +41,10 @@
"harvester_limit": 0,
"fingerprint": false,
"fingerprint_offset": 0,
"fingerprint_length": "64",
"file_identity_native": true,
"exclude_lines": [],
"include_lines": []
"include_lines": [],
"delete_enabled": false
}
}
}

View File

@@ -4,19 +4,25 @@
"version": ""
},
"name": "soc-salt-relay-logs",
"namespace": "so",
"description": "Security Onion - Salt Relay - Logs",
"policy_id": "so-grid-nodes_general",
"namespace": "so",
"policy_ids": [
"so-grid-nodes_general"
],
"vars": {},
"inputs": {
"filestream-filestream": {
"enabled": true,
"streams": {
"filestream.generic": {
"filestream.filestream": {
"enabled": true,
"vars": {
"paths": [
"/opt/so/log/soc/salt-relay.log"
],
"compression_gzip": false,
"use_logs_stream": false,
"data_stream.dataset": "soc",
"pipeline": "common",
"parsers": "#- ndjson:\n# target: \"\"\n# message_key: msg\n#- multiline:\n# type: count\n# count_lines: 3\n",
@@ -33,10 +39,10 @@
"harvester_limit": 0,
"fingerprint": false,
"fingerprint_offset": 0,
"fingerprint_length": "64",
"file_identity_native": true,
"exclude_lines": [],
"include_lines": []
"include_lines": [],
"delete_enabled": false
}
}
}

View File

@@ -4,19 +4,25 @@
"version": ""
},
"name": "soc-sensoroni-logs",
"namespace": "so",
"description": "Security Onion - Sensoroni - Logs",
"policy_id": "so-grid-nodes_general",
"namespace": "so",
"policy_ids": [
"so-grid-nodes_general"
],
"vars": {},
"inputs": {
"filestream-filestream": {
"enabled": true,
"streams": {
"filestream.generic": {
"filestream.filestream": {
"enabled": true,
"vars": {
"paths": [
"/opt/so/log/sensoroni/sensoroni.log"
],
"compression_gzip": false,
"use_logs_stream": false,
"data_stream.dataset": "soc",
"pipeline": "common",
"parsers": "#- ndjson:\n# target: \"\"\n# message_key: msg\n#- multiline:\n# type: count\n# count_lines: 3\n",
@@ -31,10 +37,10 @@
"harvester_limit": 0,
"fingerprint": false,
"fingerprint_offset": 0,
"fingerprint_length": "64",
"file_identity_native": true,
"exclude_lines": [],
"include_lines": []
"include_lines": [],
"delete_enabled": false
}
}
}

View File

@@ -4,19 +4,25 @@
"version": ""
},
"name": "soc-server-logs",
"namespace": "so",
"description": "Security Onion Console Logs",
"policy_id": "so-grid-nodes_general",
"namespace": "so",
"policy_ids": [
"so-grid-nodes_general"
],
"vars": {},
"inputs": {
"filestream-filestream": {
"enabled": true,
"streams": {
"filestream.generic": {
"filestream.filestream": {
"enabled": true,
"vars": {
"paths": [
"/opt/so/log/soc/sensoroni-server.log"
],
"compression_gzip": false,
"use_logs_stream": false,
"data_stream.dataset": "soc",
"pipeline": "common",
"parsers": "#- ndjson:\n# target: \"\"\n# message_key: msg\n#- multiline:\n# type: count\n# count_lines: 3\n",
@@ -33,10 +39,10 @@
"harvester_limit": 0,
"fingerprint": false,
"fingerprint_offset": 0,
"fingerprint_length": "64",
"file_identity_native": true,
"exclude_lines": [],
"include_lines": []
"include_lines": [],
"delete_enabled": false
}
}
}

View File

@@ -4,19 +4,25 @@
"version": ""
},
"name": "strelka-logs",
"namespace": "so",
"description": "Strelka Logs",
"policy_id": "so-grid-nodes_general",
"namespace": "so",
"policy_ids": [
"so-grid-nodes_general"
],
"vars": {},
"inputs": {
"filestream-filestream": {
"enabled": true,
"streams": {
"filestream.generic": {
"filestream.filestream": {
"enabled": true,
"vars": {
"paths": [
"/nsm/strelka/log/strelka.log"
],
"compression_gzip": false,
"use_logs_stream": false,
"data_stream.dataset": "strelka",
"pipeline": "strelka.file",
"parsers": "#- ndjson:\n# target: \"\"\n# message_key: msg\n#- multiline:\n# type: count\n# count_lines: 3\n",
@@ -31,10 +37,10 @@
"harvester_limit": 0,
"fingerprint": false,
"fingerprint_offset": 0,
"fingerprint_length": "64",
"file_identity_native": true,
"exclude_lines": [],
"include_lines": []
"include_lines": [],
"delete_enabled": false
}
}
}

View File

@@ -4,19 +4,25 @@
"version": ""
},
"name": "suricata-logs",
"namespace": "so",
"description": "Suricata integration",
"policy_id": "so-grid-nodes_general",
"namespace": "so",
"policy_ids": [
"so-grid-nodes_general"
],
"vars": {},
"inputs": {
"filestream-filestream": {
"enabled": true,
"streams": {
"filestream.generic": {
"filestream.filestream": {
"enabled": true,
"vars": {
"paths": [
"/nsm/suricata/eve*.json"
],
"compression_gzip": false,
"use_logs_stream": false,
"data_stream.dataset": "suricata",
"pipeline": "suricata.common",
"parsers": "#- ndjson:\n# target: \"\"\n# message_key: msg\n#- multiline:\n# type: count\n# count_lines: 3\n",
@@ -31,10 +37,10 @@
"harvester_limit": 0,
"fingerprint": false,
"fingerprint_offset": 0,
"fingerprint_length": "64",
"file_identity_native": true,
"exclude_lines": [],
"include_lines": []
"include_lines": [],
"delete_enabled": false
}
}
}

View File

@@ -0,0 +1,123 @@
{# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
or more contributor license agreements. Licensed under the Elastic License 2.0; you may not use
this file except in compliance with the Elastic License 2.0. #}
{% import_json '/opt/so/state/esfleet_input_package_components.json' as ADDON_INPUT_PACKAGE_COMPONENTS %}
{% import_json '/opt/so/state/esfleet_component_templates.json' as INSTALLED_COMPONENT_TEMPLATES %}
{% import_yaml 'elasticfleet/defaults.yaml' as ELASTICFLEETDEFAULTS %}
{% set CORE_ESFLEET_PACKAGES = ELASTICFLEETDEFAULTS.get('elasticfleet', {}).get('packages', {}) %}
{% set ADDON_INPUT_INTEGRATION_DEFAULTS = {} %}
{% set DEBUG_STUFF = {} %}
{% for pkg in ADDON_INPUT_PACKAGE_COMPONENTS %}
{% if pkg.name in CORE_ESFLEET_PACKAGES %}
{# skip core input packages #}
{% elif pkg.name not in CORE_ESFLEET_PACKAGES %}
{# generate defaults for each input package #}
{% if pkg.dataStreams is defined and pkg.dataStreams is not none and pkg.dataStreams | length > 0 %}
{% for pattern in pkg.dataStreams %}
{# in ES 9.3.2 'input' type integrations no longer create default component templates and instead they wait for user input during 'integration' setup (fleet ui config)
title: generic is an artifact of that and is not in use #}
{% if pattern.title == "generic" %}
{% continue %}
{% endif %}
{% if "metrics-" in pattern.name %}
{% set integration_type = "metrics-" %}
{% elif "logs-" in pattern.name %}
{% set integration_type = "logs-" %}
{% else %}
{% set integration_type = "" %}
{% endif %}
{# on input integrations the component name is user defined at the time it is added to an agent policy #}
{% set component_name = pattern.title %}
{% set index_pattern = pattern.name %}
{# component_name_x maintains the functionality of merging local pillar changes with generated 'defaults' via SOC UI #}
{% set component_name_x = component_name.replace(".","_x_") %}
{# pillar overrides/merge expects the key names to follow the naming in elasticsearch/defaults.yaml eg. so-logs-1password_x_item_usages . The _x_ is replaced later on in elasticsearch/template.map.jinja #}
{% set integration_key = "so-" ~ integration_type ~ pkg.name + '_x_' ~ component_name_x %}
{# Default integration settings #}
{% set integration_defaults = {
"index_sorting": false,
"index_template": {
"composed_of": [integration_type ~ component_name ~ "@package", integration_type ~ component_name ~ "@custom", "so-fleet_integrations.ip_mappings-1", "so-fleet_globals-1", "so-fleet_agent_id_verification-1"],
"data_stream": {
"allow_custom_routing": false,
"hidden": false
},
"ignore_missing_component_templates": [integration_type ~ component_name ~ "@custom"],
"index_patterns": [index_pattern],
"priority": 501,
"template": {
"settings": {
"index": {
"lifecycle": {"name": "so-" ~ integration_type ~ component_name ~ "-logs"},
"number_of_replicas": 0
}
}
}
},
"policy": {
"phases": {
"cold": {
"actions": {
"allocate":{
"number_of_replicas": ""
},
"set_priority": {"priority": 0}
},
"min_age": "60d"
},
"delete": {
"actions": {
"delete": {}
},
"min_age": "365d"
},
"hot": {
"actions": {
"rollover": {
"max_age": "30d",
"max_primary_shard_size": "50gb"
},
"forcemerge":{
"max_num_segments": ""
},
"shrink":{
"max_primary_shard_size": "",
"method": "COUNT",
"number_of_shards": ""
},
"set_priority": {"priority": 100}
},
"min_age": "0ms"
},
"warm": {
"actions": {
"allocate": {
"number_of_replicas": ""
},
"forcemerge": {
"max_num_segments": ""
},
"shrink":{
"max_primary_shard_size": "",
"method": "COUNT",
"number_of_shards": ""
},
"set_priority": {"priority": 50}
},
"min_age": "30d"
}
}
}
} %}
{% do ADDON_INPUT_INTEGRATION_DEFAULTS.update({integration_key: integration_defaults}) %}
{% do DEBUG_STUFF.update({integration_key: "Generating defaults for "+ pkg.name })%}
{% endfor %}
{% endif %}
{% endif %}
{% endfor %}

View File

@@ -59,8 +59,8 @@
{# skip core integrations #}
{% elif pkg.name not in CORE_ESFLEET_PACKAGES %}
{# generate defaults for each integration #}
{% if pkg.es_index_patterns is defined and pkg.es_index_patterns is not none %}
{% for pattern in pkg.es_index_patterns %}
{% if pkg.dataStreams is defined and pkg.dataStreams is not none and pkg.dataStreams | length > 0 %}
{% for pattern in pkg.dataStreams %}
{% if "metrics-" in pattern.name %}
{% set integration_type = "metrics-" %}
{% elif "logs-" in pattern.name %}
@@ -75,44 +75,27 @@
{% if component_name in WEIRD_INTEGRATIONS %}
{% set component_name = WEIRD_INTEGRATIONS[component_name] %}
{% endif %}
{# create duplicate of component_name, so we can split generics from @custom component templates in the index template below and overwrite the default @package when needed
eg. having to replace unifiedlogs.generic@package with filestream.generic@package, but keep the ability to customize unifiedlogs.generic@custom and its ILM policy #}
{% set custom_component_name = component_name %}
{# duplicate integration_type to assist with sometimes needing to overwrite component templates with 'logs-filestream.generic@package' (there is no metrics-filestream.generic@package) #}
{% set generic_integration_type = integration_type %}
{# component_name_x maintains the functionality of merging local pillar changes with generated 'defaults' via SOC UI #}
{% set component_name_x = component_name.replace(".","_x_") %}
{# pillar overrides/merge expects the key names to follow the naming in elasticsearch/defaults.yaml eg. so-logs-1password_x_item_usages . The _x_ is replaced later on in elasticsearch/template.map.jinja #}
{% set integration_key = "so-" ~ integration_type ~ component_name_x %}
{# if its a .generic template make sure that a .generic@package for the integration exists. Else default to logs-filestream.generic@package #}
{% if ".generic" in component_name and integration_type ~ component_name ~ "@package" not in INSTALLED_COMPONENT_TEMPLATES %}
{# these generic templates by default are directed to index_pattern of 'logs-generic-*', overwrite that here to point to eg gcp_pubsub.generic-* #}
{% set index_pattern = integration_type ~ component_name ~ "-*" %}
{# includes use of .generic component template, but it doesn't exist in installed component templates. Redirect it to filestream.generic@package #}
{% set component_name = "filestream.generic" %}
{% set generic_integration_type = "logs-" %}
{% endif %}
{# Default integration settings #}
{% set integration_defaults = {
"index_sorting": false,
"index_template": {
"composed_of": [generic_integration_type ~ component_name ~ "@package", integration_type ~ custom_component_name ~ "@custom", "so-fleet_integrations.ip_mappings-1", "so-fleet_globals-1", "so-fleet_agent_id_verification-1"],
"composed_of": [integration_type ~ component_name ~ "@package", integration_type ~ component_name ~ "@custom", "so-fleet_integrations.ip_mappings-1", "so-fleet_globals-1", "so-fleet_agent_id_verification-1"],
"data_stream": {
"allow_custom_routing": false,
"hidden": false
},
"ignore_missing_component_templates": [integration_type ~ custom_component_name ~ "@custom"],
"ignore_missing_component_templates": [integration_type ~ component_name ~ "@custom"],
"index_patterns": [index_pattern],
"priority": 501,
"template": {
"settings": {
"index": {
"lifecycle": {"name": "so-" ~ integration_type ~ custom_component_name ~ "-logs"},
"lifecycle": {"name": "so-" ~ integration_type ~ component_name ~ "-logs"},
"number_of_replicas": 0
}
}

View File

@@ -135,9 +135,33 @@ elastic_fleet_bulk_package_install() {
fi
}
elastic_fleet_installed_packages() {
if ! fleet_api "epm/packages/installed?perPage=500"; then
elastic_fleet_get_package_list_by_type() {
if ! output=$(fleet_api "epm/packages"); then
return 1
else
is_integration=$(jq '[.items[] | select(.type=="integration") | .name ]' <<< "$output")
is_input=$(jq '[.items[] | select(.type=="input") | .name ]' <<< "$output")
is_content=$(jq '[.items[] | select(.type=="content") | .name ]' <<< "$output")
jq -n --argjson is_integration "${is_integration:-[]}" \
--argjson is_input "${is_input:-[]}" \
--argjson is_content "${is_content:-[]}" \
'{"integration": $is_integration,"input": $is_input, "content": $is_content}'
fi
}
elastic_fleet_installed_packages_components() {
package_type=${1,,}
if [[ "$package_type" != "integration" && "$package_type" != "input" && "$package_type" != "content" ]]; then
echo "Error: Invalid package type ${package_type}. Valid types are 'integration', 'input', or 'content'."
return 1
fi
packages_by_type=$(elastic_fleet_get_package_list_by_type)
packages=$(jq --arg package_type "$package_type" '.[$package_type]' <<< "$packages_by_type")
if ! output=$(fleet_api "epm/packages/installed?perPage=500"); then
return 1
else
jq -c --argjson packages "$packages" '[.items[] | select(.name | IN($packages[])) | {name: .name, dataStreams: .dataStreams}]' <<< "$output"
fi
}

View File

@@ -6,6 +6,11 @@
. /usr/sbin/so-common
{%- import_yaml 'elasticsearch/defaults.yaml' as ELASTICSEARCHDEFAULTS %}
{%- import_yaml 'elasticfleet/defaults.yaml' as ELASTICFLEETDEFAULTS %}
{# Optionally override Elasticsearch version for Elastic Agent patch releases #}
{%- if ELASTICFLEETDEFAULTS.elasticfleet.patch_version is defined %}
{%- do ELASTICSEARCHDEFAULTS.update({'elasticsearch': {'version': ELASTICFLEETDEFAULTS.elasticfleet.patch_version}}) %}
{%- endif %}
# Only run on Managers
if ! is_manager_node; then

View File

@@ -18,7 +18,9 @@ INSTALLED_PACKAGE_LIST=/tmp/esfleet_installed_packages.json
BULK_INSTALL_PACKAGE_LIST=/tmp/esfleet_bulk_install.json
BULK_INSTALL_PACKAGE_TMP=/tmp/esfleet_bulk_install_tmp.json
BULK_INSTALL_OUTPUT=/opt/so/state/esfleet_bulk_install_results.json
PACKAGE_COMPONENTS=/opt/so/state/esfleet_package_components.json
INTEGRATION_PACKAGE_COMPONENTS=/opt/so/state/esfleet_package_components.json
INPUT_PACKAGE_COMPONENTS=/opt/so/state/esfleet_input_package_components.json
CONTENT_PACKAGE_COMPONENTS=/opt/so/state/esfleet_content_package_components.json
COMPONENT_TEMPLATES=/opt/so/state/esfleet_component_templates.json
PENDING_UPDATE=false
@@ -179,10 +181,13 @@ if [[ -f $STATE_FILE_SUCCESS ]]; then
else
echo "Elastic integrations don't appear to need installation/updating..."
fi
# Write out file for generating index/component/ilm templates
if latest_installed_package_list=$(elastic_fleet_installed_packages); then
echo $latest_installed_package_list | jq '[.items[] | {name: .name, es_index_patterns: .dataStreams}]' > $PACKAGE_COMPONENTS
fi
# Write out file for generating index/component/ilm templates, keeping each package type separate
for package_type in "INTEGRATION" "INPUT" "CONTENT"; do
if latest_installed_package_list=$(elastic_fleet_installed_packages_components "$package_type"); then
outfile="${package_type}_PACKAGE_COMPONENTS"
echo $latest_installed_package_list > "${!outfile}"
fi
done
if retry 3 1 "so-elasticsearch-query / --fail --output /dev/null"; then
# Refresh installed component template list
latest_component_templates_list=$(so-elasticsearch-query _component_template | jq '.component_templates[] | .name' | jq -s '.')

View File

@@ -66,6 +66,8 @@ so-elasticsearch-ilm-policy-load-script:
- group: 939
- mode: 754
- template: jinja
- defaults:
GLOBALS: {{ GLOBALS }}
- show_changes: False
so-elasticsearch-pipelines-script:
@@ -91,6 +93,13 @@ estemplatedir:
- group: 939
- makedirs: True
esaddontemplatedir:
file.directory:
- name: /opt/so/conf/elasticsearch/templates/addon-index
- user: 930
- group: 939
- makedirs: True
esrolesdir:
file.directory:
- name: /opt/so/conf/elasticsearch/roles

View File

@@ -1,6 +1,6 @@
elasticsearch:
enabled: false
version: 9.0.8
version: 9.3.3
index_clean: true
vm:
max_map_count: 1048576

View File

@@ -10,8 +10,10 @@
{% from 'elasticsearch/config.map.jinja' import ELASTICSEARCH_NODES %}
{% from 'elasticsearch/config.map.jinja' import ELASTICSEARCH_SEED_HOSTS %}
{% from 'elasticsearch/config.map.jinja' import ELASTICSEARCHMERGED %}
{% set TEMPLATES = salt['pillar.get']('elasticsearch:templates', {}) %}
{% from 'elasticsearch/template.map.jinja' import ES_INDEX_SETTINGS %}
{% from 'elasticsearch/template.map.jinja' import ES_INDEX_SETTINGS, SO_MANAGED_INDICES %}
{% if GLOBALS.role != 'so-heavynode' %}
{% from 'elasticsearch/template.map.jinja' import ALL_ADDON_SETTINGS %}
{% endif %}
include:
- ca
@@ -117,40 +119,52 @@ escomponenttemplates:
- onchanges_in:
- file: so-elasticsearch-templates-reload
- show_changes: False
# Auto-generate templates from defaults file
# Clean up legacy and non-SO managed templates from the elasticsearch/templates/index/ directory
so_index_template_dir:
file.directory:
- name: /opt/so/conf/elasticsearch/templates/index
- clean: True
{%- if SO_MANAGED_INDICES %}
- require:
{%- for index in SO_MANAGED_INDICES %}
- file: so_index_template_{{index}}
{%- endfor %}
{%- endif %}
# Auto-generate index templates for SO managed indices (directly defined in elasticsearch/defaults.yaml)
# These index templates are for the core SO datasets and are always required
{% for index, settings in ES_INDEX_SETTINGS.items() %}
{% if settings.index_template is defined %}
es_index_template_{{index}}:
{% if settings.index_template is defined %}
so_index_template_{{index}}:
file.managed:
- name: /opt/so/conf/elasticsearch/templates/index/{{ index }}-template.json
- source: salt://elasticsearch/base-template.json.jinja
- defaults:
TEMPLATE_CONFIG: {{ settings.index_template }}
TEMPLATE_CONFIG: {{ settings.index_template }}
- template: jinja
- show_changes: False
- onchanges_in:
- file: so-elasticsearch-templates-reload
{% endif %}
{% endfor %}
{% if TEMPLATES %}
# Sync custom templates to /opt/so/conf/elasticsearch/templates
{% for TEMPLATE in TEMPLATES %}
es_template_{{TEMPLATE.split('.')[0] | replace("/","_") }}:
{% if GLOBALS.role != "so-heavynode" %}
# Auto-generate optional index templates for integration | input | content packages
# These index templates are not used by default (until user adds package to an agent policy).
# Pre-configured with standard defaults, and incorporated into SOC configuration for user customization.
{% for index,settings in ALL_ADDON_SETTINGS.items() %}
{% if settings.index_template is defined %}
addon_index_template_{{index}}:
file.managed:
- source: salt://elasticsearch/templates/index/{{TEMPLATE}}
{% if 'jinja' in TEMPLATE.split('.')[-1] %}
- name: /opt/so/conf/elasticsearch/templates/index/{{TEMPLATE.split('/')[1] | replace(".jinja", "")}}
- name: /opt/so/conf/elasticsearch/templates/addon-index/{{ index }}-template.json
- source: salt://elasticsearch/base-template.json.jinja
- defaults:
TEMPLATE_CONFIG: {{ settings.index_template }}
- template: jinja
{% else %}
- name: /opt/so/conf/elasticsearch/templates/index/{{TEMPLATE.split('/')[1]}}
{% endif %}
- user: 930
- group: 939
- show_changes: False
- onchanges_in:
- file: so-elasticsearch-templates-reload
- file: addon-elasticsearch-templates-reload
{% endif %}
{% endfor %}
{% endif %}
@@ -165,6 +179,7 @@ so-es-cluster-settings:
- file: elasticsearch_sbin_jinja
{% endif %}
# heavynodes will only load ILM policies for SO managed indices. (Indicies defined in elasticsearch/defaults.yaml)
so-elasticsearch-ilm-policy-load:
cmd.run:
- name: /usr/sbin/so-elasticsearch-ilm-policy-load
@@ -179,9 +194,18 @@ so-elasticsearch-templates-reload:
file.absent:
- name: /opt/so/state/estemplates.txt
addon-elasticsearch-templates-reload:
file.absent:
- name: /opt/so/state/addon_estemplates.txt
# so-elasticsearch-templates-load will have its first successful run during the 'so-elastic-fleet-setup' script
so-elasticsearch-templates:
cmd.run:
{%- if GLOBALS.role == "so-heavynode" %}
- name: /usr/sbin/so-elasticsearch-templates-load --heavynode
{%- else %}
- name: /usr/sbin/so-elasticsearch-templates-load
{%- endif %}
- cwd: /opt/so
- template: jinja
- require:

View File

@@ -10,24 +10,28 @@
"processors": [
{
"set": {
"tag": "set_ecs_version_f5923549",
"field": "ecs.version",
"value": "8.17.0"
}
},
{
"set": {
"tag": "set_observer_vendor_ad9d35cc",
"field": "observer.vendor",
"value": "netgate"
}
},
{
"set": {
"tag": "set_observer_type_5dddf3ba",
"field": "observer.type",
"value": "firewall"
}
},
{
"rename": {
"tag": "rename_message_to_event_original_56a77271",
"field": "message",
"target_field": "event.original",
"ignore_missing": true,
@@ -36,12 +40,14 @@
},
{
"set": {
"tag": "set_event_kind_de80643c",
"field": "event.kind",
"value": "event"
}
},
{
"set": {
"tag": "set_event_timezone_4ca44cac",
"field": "event.timezone",
"value": "{{{_tmp.tz_offset}}}",
"if": "ctx._tmp?.tz_offset != null && ctx._tmp?.tz_offset != 'local'"
@@ -49,6 +55,7 @@
},
{
"grok": {
"tag": "grok_event_original_27d9c8c7",
"description": "Parse syslog header",
"field": "event.original",
"patterns": [
@@ -72,6 +79,7 @@
},
{
"date": {
"tag": "date__tmp_timestamp8601_to_timestamp_6ac9d3ce",
"if": "ctx._tmp.timestamp8601 != null",
"field": "_tmp.timestamp8601",
"target_field": "@timestamp",
@@ -82,6 +90,7 @@
},
{
"date": {
"tag": "date__tmp_timestamp_to_timestamp_f21e536e",
"if": "ctx.event?.timezone != null && ctx._tmp?.timestamp != null",
"field": "_tmp.timestamp",
"target_field": "@timestamp",
@@ -95,6 +104,7 @@
},
{
"grok": {
"tag": "grok_process_name_cef3d489",
"description": "Set Event Provider",
"field": "process.name",
"patterns": [
@@ -107,71 +117,83 @@
},
{
"pipeline": {
"name": "logs-pfsense.log-1.23.1-firewall",
"tag": "pipeline_e16851a7",
"name": "logs-pfsense.log-1.25.2-firewall",
"if": "ctx.event.provider == 'filterlog'"
}
},
{
"pipeline": {
"name": "logs-pfsense.log-1.23.1-openvpn",
"tag": "pipeline_828590b5",
"name": "logs-pfsense.log-1.25.2-openvpn",
"if": "ctx.event.provider == 'openvpn'"
}
},
{
"pipeline": {
"name": "logs-pfsense.log-1.23.1-ipsec",
"tag": "pipeline_9d37039c",
"name": "logs-pfsense.log-1.25.2-ipsec",
"if": "ctx.event.provider == 'charon'"
}
},
{
"pipeline": {
"name": "logs-pfsense.log-1.23.1-dhcp",
"if": "[\"dhcpd\", \"dhclient\", \"dhcp6c\"].contains(ctx.event.provider)"
"tag": "pipeline_ad56bbca",
"name": "logs-pfsense.log-1.25.2-dhcp",
"if": "[\"dhcpd\", \"dhclient\", \"dhcp6c\", \"dnsmasq-dhcp\"].contains(ctx.event.provider)"
}
},
{
"pipeline": {
"name": "logs-pfsense.log-1.23.1-unbound",
"tag": "pipeline_dd85553d",
"name": "logs-pfsense.log-1.25.2-unbound",
"if": "ctx.event.provider == 'unbound'"
}
},
{
"pipeline": {
"name": "logs-pfsense.log-1.23.1-haproxy",
"tag": "pipeline_720ed255",
"name": "logs-pfsense.log-1.25.2-haproxy",
"if": "ctx.event.provider == 'haproxy'"
}
},
{
"pipeline": {
"name": "logs-pfsense.log-1.23.1-php-fpm",
"tag": "pipeline_456beba5",
"name": "logs-pfsense.log-1.25.2-php-fpm",
"if": "ctx.event.provider == 'php-fpm'"
}
},
{
"pipeline": {
"name": "logs-pfsense.log-1.23.1-squid",
"tag": "pipeline_a0d89375",
"name": "logs-pfsense.log-1.25.2-squid",
"if": "ctx.event.provider == 'squid'"
}
},
{
"pipeline": {
"name": "logs-pfsense.log-1.23.1-snort",
"tag": "pipeline_c2f1ed55",
"name": "logs-pfsense.log-1.25.2-snort",
"if": "ctx.event.provider == 'snort'"
}
},
{
"pipeline": {
"name": "logs-pfsense.log-1.23.1-suricata",
"tag":"pipeline_33db1c9e",
"name": "logs-pfsense.log-1.25.2-suricata",
"if": "ctx.event.provider == 'suricata'"
}
},
{
"drop": {
"if": "![\"filterlog\", \"openvpn\", \"charon\", \"dhcpd\", \"dhclient\", \"dhcp6c\", \"unbound\", \"haproxy\", \"php-fpm\", \"squid\", \"snort\", \"suricata\"].contains(ctx.event?.provider)"
"tag": "drop_9d7c46f8",
"if": "![\"filterlog\", \"openvpn\", \"charon\", \"dhcpd\", \"dnsmasq-dhcp\", \"dhclient\", \"dhcp6c\", \"unbound\", \"haproxy\", \"php-fpm\", \"squid\", \"snort\", \"suricata\"].contains(ctx.event?.provider)"
}
},
{
"append": {
"tag": "append_event_category_4780a983",
"field": "event.category",
"value": "network",
"if": "ctx.network != null"
@@ -179,6 +201,7 @@
},
{
"convert": {
"tag": "convert_source_address_to_source_ip_f5632a20",
"field": "source.address",
"target_field": "source.ip",
"type": "ip",
@@ -188,6 +211,7 @@
},
{
"convert": {
"tag": "convert_destination_address_to_destination_ip_f1388f0c",
"field": "destination.address",
"target_field": "destination.ip",
"type": "ip",
@@ -197,6 +221,7 @@
},
{
"set": {
"tag": "set_network_type_1f1d940a",
"field": "network.type",
"value": "ipv6",
"if": "ctx.source?.ip != null && ctx.source.ip.contains(\":\")"
@@ -204,6 +229,7 @@
},
{
"set": {
"tag": "set_network_type_69deca38",
"field": "network.type",
"value": "ipv4",
"if": "ctx.source?.ip != null && ctx.source.ip.contains(\".\")"
@@ -211,6 +237,7 @@
},
{
"geoip": {
"tag": "geoip_source_ip_to_source_geo_da2e41b2",
"field": "source.ip",
"target_field": "source.geo",
"ignore_missing": true
@@ -218,6 +245,7 @@
},
{
"geoip": {
"tag": "geoip_destination_ip_to_destination_geo_ab5e2968",
"field": "destination.ip",
"target_field": "destination.geo",
"ignore_missing": true
@@ -225,6 +253,7 @@
},
{
"geoip": {
"tag": "geoip_source_ip_to_source_as_28d69883",
"ignore_missing": true,
"database_file": "GeoLite2-ASN.mmdb",
"field": "source.ip",
@@ -237,6 +266,7 @@
},
{
"geoip": {
"tag": "geoip_destination_ip_to_destination_as_8a007787",
"database_file": "GeoLite2-ASN.mmdb",
"field": "destination.ip",
"target_field": "destination.as",
@@ -249,6 +279,7 @@
},
{
"rename": {
"tag": "rename_source_as_asn_to_source_as_number_a917047d",
"field": "source.as.asn",
"target_field": "source.as.number",
"ignore_missing": true
@@ -256,6 +287,7 @@
},
{
"rename": {
"tag": "rename_source_as_organization_name_to_source_as_organization_name_f1362d0b",
"field": "source.as.organization_name",
"target_field": "source.as.organization.name",
"ignore_missing": true
@@ -263,6 +295,7 @@
},
{
"rename": {
"tag": "rename_destination_as_asn_to_destination_as_number_3b459fcd",
"field": "destination.as.asn",
"target_field": "destination.as.number",
"ignore_missing": true
@@ -270,6 +303,7 @@
},
{
"rename": {
"tag": "rename_destination_as_organization_name_to_destination_as_organization_name_814bd459",
"field": "destination.as.organization_name",
"target_field": "destination.as.organization.name",
"ignore_missing": true
@@ -277,12 +311,14 @@
},
{
"community_id": {
"tag": "community_id_d2308e7a",
"target_field": "network.community_id",
"ignore_failure": true
}
},
{
"grok": {
"tag": "grok_observer_ingress_interface_name_968018d3",
"field": "observer.ingress.interface.name",
"patterns": [
"%{DATA}.%{NONNEGINT:observer.ingress.vlan.id}"
@@ -293,6 +329,7 @@
},
{
"set": {
"tag": "set_network_vlan_id_efd4d96a",
"field": "network.vlan.id",
"copy_from": "observer.ingress.vlan.id",
"ignore_empty_value": true
@@ -300,6 +337,7 @@
},
{
"append": {
"tag": "append_related_ip_c1a6356b",
"field": "related.ip",
"value": "{{{destination.ip}}}",
"allow_duplicates": false,
@@ -308,6 +346,7 @@
},
{
"append": {
"tag": "append_related_ip_8121c591",
"field": "related.ip",
"value": "{{{source.ip}}}",
"allow_duplicates": false,
@@ -316,6 +355,7 @@
},
{
"append": {
"tag": "append_related_ip_53b62ed8",
"field": "related.ip",
"value": "{{{source.nat.ip}}}",
"allow_duplicates": false,
@@ -324,6 +364,7 @@
},
{
"append": {
"tag": "append_related_hosts_6f162628",
"field": "related.hosts",
"value": "{{{destination.domain}}}",
"if": "ctx.destination?.domain != null"
@@ -331,6 +372,7 @@
},
{
"append": {
"tag": "append_related_user_c036eec2",
"field": "related.user",
"value": "{{{user.name}}}",
"if": "ctx.user?.name != null"
@@ -338,6 +380,7 @@
},
{
"set": {
"tag": "set_network_direction_cb1e3125",
"field": "network.direction",
"value": "{{{network.direction}}}bound",
"if": "ctx.network?.direction != null && ctx.network?.direction =~ /^(in|out)$/"
@@ -345,6 +388,7 @@
},
{
"remove": {
"tag": "remove_a82e20f2",
"field": [
"_tmp"
],
@@ -353,11 +397,21 @@
},
{
"script": {
"tag": "script_a7f2c062",
"lang": "painless",
"description": "This script processor iterates over the whole document to remove fields with null values.",
"source": "void handleMap(Map map) {\n for (def x : map.values()) {\n if (x instanceof Map) {\n handleMap(x);\n } else if (x instanceof List) {\n handleList(x);\n }\n }\n map.values().removeIf(v -> v == null || (v instanceof String && v == \"-\"));\n}\nvoid handleList(List list) {\n for (def x : list) {\n if (x instanceof Map) {\n handleMap(x);\n } else if (x instanceof List) {\n handleList(x);\n }\n }\n}\nhandleMap(ctx);\n"
}
},
{
"append": {
"tag": "append_preserve_original_event_on_error",
"field": "tags",
"value": "preserve_original_event",
"allow_duplicates": false,
"if": "ctx.error?.message != null"
}
},
{
"pipeline": {
"name": "global@custom",
@@ -405,7 +459,14 @@
{
"append": {
"field": "error.message",
"value": "{{{ _ingest.on_failure_message }}}"
"value": "Processor '{{{ _ingest.on_failure_processor_type }}}' {{#_ingest.on_failure_processor_tag}}with tag '{{{ _ingest.on_failure_processor_tag }}}' {{/_ingest.on_failure_processor_tag}}in pipeline '{{{ _ingest.pipeline }}}' failed with message '{{{ _ingest.on_failure_message }}}'"
}
},
{
"append": {
"field": "tags",
"value": "preserve_original_event",
"allow_duplicates": false
}
}
]

View File

@@ -22,6 +22,12 @@
"ignore_failure": true
}
},
{
"lowercase": {
"field": "network.transport",
"ignore_failure": true
}
},
{
"rename": {
"field": "message2.in_iface",

View File

@@ -14,15 +14,42 @@
{% set ES_INDEX_SETTINGS_ORIG = ELASTICSEARCHDEFAULTS.elasticsearch.index_settings %}
{% set ALL_ADDON_INTEGRATION_DEFAULTS = {} %}
{% set ALL_ADDON_SETTINGS_ORIG = {} %}
{% set ALL_ADDON_SETTINGS_GLOBAL_OVERRIDES = {} %}
{% set ALL_ADDON_SETTINGS = {} %}
{# start generation of integration default index_settings #}
{% if salt['file.file_exists']('/opt/so/state/esfleet_package_components.json') and salt['file.file_exists']('/opt/so/state/esfleet_component_templates.json') %}
{% set check_package_components = salt['file.stats']('/opt/so/state/esfleet_package_components.json') %}
{% if check_package_components.size > 1 %}
{% from 'elasticfleet/integration-defaults.map.jinja' import ADDON_INTEGRATION_DEFAULTS %}
{% for index, settings in ADDON_INTEGRATION_DEFAULTS.items() %}
{% do ES_INDEX_SETTINGS_ORIG.update({index: settings}) %}
{% endfor %}
{% endif%}
{% if salt['file.file_exists']('/opt/so/state/esfleet_component_templates.json') %}
{# import integration type defaults #}
{% if salt['file.file_exists']('/opt/so/state/esfleet_package_components.json') %}
{% set check_integration_package_components = salt['file.stats']('/opt/so/state/esfleet_package_components.json') %}
{% if check_integration_package_components.size > 1 %}
{% from 'elasticfleet/integration-defaults.map.jinja' import ADDON_INTEGRATION_DEFAULTS %}
{% do ALL_ADDON_INTEGRATION_DEFAULTS.update(ADDON_INTEGRATION_DEFAULTS) %}
{% endif %}
{% endif %}
{# import input type defaults #}
{% if salt['file.file_exists']('/opt/so/state/esfleet_input_package_components.json') %}
{% set check_input_package_components = salt['file.stats']('/opt/so/state/esfleet_input_package_components.json') %}
{% if check_input_package_components.size > 1 %}
{% from 'elasticfleet/input-defaults.map.jinja' import ADDON_INPUT_INTEGRATION_DEFAULTS %}
{% do ALL_ADDON_INTEGRATION_DEFAULTS.update(ADDON_INPUT_INTEGRATION_DEFAULTS) %}
{% endif %}
{% endif %}
{# import content type defaults #}
{% if salt['file.file_exists']('/opt/so/state/esfleet_content_package_components.json') %}
{% set check_content_package_components = salt['file.stats']('/opt/so/state/esfleet_content_package_components.json') %}
{% if check_content_package_components.size > 1 %}
{% from 'elasticfleet/content-defaults.map.jinja' import ADDON_CONTENT_INTEGRATION_DEFAULTS %}
{% do ALL_ADDON_INTEGRATION_DEFAULTS.update(ADDON_CONTENT_INTEGRATION_DEFAULTS) %}
{% endif %}
{% endif %}
{% for index, settings in ALL_ADDON_INTEGRATION_DEFAULTS.items() %}
{% do ALL_ADDON_SETTINGS_ORIG.update({index: settings}) %}
{% endfor %}
{% endif %}
{# end generation of integration default index_settings #}
@@ -31,25 +58,33 @@
{% do ES_INDEX_SETTINGS_GLOBAL_OVERRIDES.update({index: salt['defaults.merge'](ELASTICSEARCHDEFAULTS.elasticsearch.index_settings[index], PILLAR_GLOBAL_OVERRIDES, in_place=False)}) %}
{% endfor %}
{% if ALL_ADDON_SETTINGS_ORIG.keys() | length > 0 %}
{% for index in ALL_ADDON_SETTINGS_ORIG.keys() %}
{% do ALL_ADDON_SETTINGS_GLOBAL_OVERRIDES.update({index: salt['defaults.merge'](ALL_ADDON_SETTINGS_ORIG[index], PILLAR_GLOBAL_OVERRIDES, in_place=False)}) %}
{% endfor %}
{% endif %}
{% set ES_INDEX_SETTINGS = {} %}
{% do ES_INDEX_SETTINGS_GLOBAL_OVERRIDES.update(salt['defaults.merge'](ES_INDEX_SETTINGS_GLOBAL_OVERRIDES, ES_INDEX_PILLAR, in_place=False)) %}
{% for index, settings in ES_INDEX_SETTINGS_GLOBAL_OVERRIDES.items() %}
{% macro create_final_index_template(DEFINED_SETTINGS, GLOBAL_OVERRIDES, FINAL_INDEX_SETTINGS) %}
{% do GLOBAL_OVERRIDES.update(salt['defaults.merge'](GLOBAL_OVERRIDES, ES_INDEX_PILLAR, in_place=False)) %}
{% for index, settings in GLOBAL_OVERRIDES.items() %}
{# prevent this action from being performed on custom defined indices. #}
{# the custom defined index is not present in either of the dictionaries and fails to reder. #}
{% if index in ES_INDEX_SETTINGS_ORIG and index in ES_INDEX_SETTINGS_GLOBAL_OVERRIDES %}
{% if index in DEFINED_SETTINGS and index in GLOBAL_OVERRIDES %}
{# dont merge policy from the global_overrides if policy isn't defined in the original index settingss #}
{# this will prevent so-elasticsearch-ilm-policy-load from trying to load policy on non ILM manged indices #}
{% if not ES_INDEX_SETTINGS_ORIG[index].policy is defined and ES_INDEX_SETTINGS_GLOBAL_OVERRIDES[index].policy is defined %}
{% do ES_INDEX_SETTINGS_GLOBAL_OVERRIDES[index].pop('policy') %}
{% if not DEFINED_SETTINGS[index].policy is defined and GLOBAL_OVERRIDES[index].policy is defined %}
{% do GLOBAL_OVERRIDES[index].pop('policy') %}
{% endif %}
{# this prevents and index from inderiting a policy phase from global overrides if it wasnt defined in the defaults. #}
{% if ES_INDEX_SETTINGS_GLOBAL_OVERRIDES[index].policy is defined %}
{% for phase in ES_INDEX_SETTINGS_GLOBAL_OVERRIDES[index].policy.phases.copy() %}
{% if ES_INDEX_SETTINGS_ORIG[index].policy.phases[phase] is not defined %}
{% do ES_INDEX_SETTINGS_GLOBAL_OVERRIDES[index].policy.phases.pop(phase) %}
{% if GLOBAL_OVERRIDES[index].policy is defined %}
{% for phase in GLOBAL_OVERRIDES[index].policy.phases.copy() %}
{% if DEFINED_SETTINGS[index].policy.phases[phase] is not defined %}
{% do GLOBAL_OVERRIDES[index].policy.phases.pop(phase) %}
{% endif %}
{% endfor %}
{% endif %}
@@ -111,5 +146,14 @@
{% endfor %}
{% endif %}
{% do ES_INDEX_SETTINGS.update({index | replace("_x_", "."): ES_INDEX_SETTINGS_GLOBAL_OVERRIDES[index]}) %}
{% do FINAL_INDEX_SETTINGS.update({index | replace("_x_", "."): GLOBAL_OVERRIDES[index]}) %}
{% endfor %}
{% endmacro %}
{{ create_final_index_template(ES_INDEX_SETTINGS_ORIG, ES_INDEX_SETTINGS_GLOBAL_OVERRIDES, ES_INDEX_SETTINGS) }}
{{ create_final_index_template(ALL_ADDON_SETTINGS_ORIG, ALL_ADDON_SETTINGS_GLOBAL_OVERRIDES, ALL_ADDON_SETTINGS) }}
{% set SO_MANAGED_INDICES = [] %}
{% for index, settings in ES_INDEX_SETTINGS.items() %}
{% do SO_MANAGED_INDICES.append(index) %}
{% endfor %}

View File

@@ -6,8 +6,19 @@
# Elastic License 2.0.
. /usr/sbin/so-common
if [ "$1" == "" ]; then
curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L https://localhost:9200/_component_template | jq '.component_templates[] |.name'| sort
if [[ -z "$1" ]]; then
if output=$(so-elasticsearch-query "_component_template" --retry 3 --retry-delay 1 --fail); then
jq '[.component_templates[] | .name] | sort' <<< "$output"
else
echo "Failed to retrieve component templates from Elasticsearch."
exit 1
fi
else
curl -K /opt/so/conf/elasticsearch/curl.config -s -k -L https://localhost:9200/_component_template/$1 | jq
fi
if output=$(so-elasticsearch-query "_component_template/$1" --retry 3 --retry-delay 1 --fail); then
jq <<< "$output"
else
echo "Failed to retrieve component template '$1' from Elasticsearch."
exit 1
fi
fi

View File

@@ -0,0 +1,253 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
. /usr/sbin/so-common
SO_STATEFILE_SUCCESS=/opt/so/state/estemplates.txt
ADDON_STATEFILE_SUCCESS=/opt/so/state/addon_estemplates.txt
ELASTICSEARCH_TEMPLATES_DIR="/opt/so/conf/elasticsearch/templates"
SO_TEMPLATES_DIR="${ELASTICSEARCH_TEMPLATES_DIR}/index"
ADDON_TEMPLATES_DIR="${ELASTICSEARCH_TEMPLATES_DIR}/addon-index"
SO_LOAD_FAILURES=0
ADDON_LOAD_FAILURES=0
SO_LOAD_FAILURES_NAMES=()
ADDON_LOAD_FAILURES_NAMES=()
IS_HEAVYNODE="false"
FORCE="false"
VERBOSE="false"
SHOULD_EXIT_ON_FAILURE="true"
# If soup is running, ignore errors
pgrep soup >/dev/null && SHOULD_EXIT_ON_FAILURE="false"
while [[ $# -gt 0 ]]; do
case "$1" in
--heavynode)
IS_HEAVYNODE="true"
;;
--force)
FORCE="true"
;;
--verbose)
VERBOSE="true"
;;
*)
echo "Usage: $0 [options]"
echo "Options:"
echo " --heavynode Only loads index templates specific to heavynodes"
echo " --force Force reload all templates regardless of statefiles (default: false)"
echo " --verbose Enable verbose output"
exit 1
;;
esac
shift
done
load_template() {
local uri="$1"
local file="$2"
echo "Loading template file $file"
if ! output=$(retry 3 3 "so-elasticsearch-query $uri -d@$file -XPUT" "{\"acknowledged\":true}"); then
echo "$output"
return 1
elif [[ "$VERBOSE" == "true" ]]; then
echo "$output"
fi
}
check_required_component_template_exists() {
local required
local missing
local file=$1
required=$(jq '[((.composed_of //[]) - (.ignore_missing_component_templates // []))[]]' "$file")
missing=$(jq -n --argjson required "$required" --argjson component_templates "$component_templates" '(($required) - ($component_templates))')
if [[ $(jq length <<<"$missing") -gt 0 ]]; then
return 1
fi
}
check_heavynode_compatiable_index_template() {
# The only templates that are relevant to heavynodes are from datasets defined in elasticagent/files/elastic-agent.yml.jinja.
# Heavynodes do not have fleet server packages installed and do not support elastic agents reporting directly to them.
local -A heavynode_index_templates=(
["so-import"]=1
["so-syslog"]=1
["so-logs-soc"]=1
["so-suricata"]=1
["so-suricata.alerts"]=1
["so-zeek"]=1
["so-strelka"]=1
)
local template_name="$1"
if [[ ! -v heavynode_index_templates["$template_name"] ]]; then
return 1
fi
}
load_component_templates() {
local printed_name="$1"
local pattern="${ELASTICSEARCH_TEMPLATES_DIR}/component/$2"
local append_mappings="${3:-"false"}"
# current state of nullglob shell option
shopt -q nullglob && nullglob_set=1 || nullglob_set=0
shopt -s nullglob
echo -e "\nLoading $printed_name component templates...\n"
for component in "$pattern"/*.json; do
tmpl_name=$(basename "${component%.json}")
if [[ "$append_mappings" == "true" ]]; then
# avoid duplicating "-mappings" if it already exists in the component template filename
tmpl_name="${tmpl_name%-mappings}-mappings"
fi
if ! load_template "_component_template/${tmpl_name}" "$component"; then
SO_LOAD_FAILURES=$((SO_LOAD_FAILURES + 1))
SO_LOAD_FAILURES_NAMES+=("$component")
fi
done
# restore nullglob shell option if needed
if [[ $nullglob_set -eq 1 ]]; then
shopt -u nullglob
fi
}
check_elasticsearch_responsive() {
# Cannot load templates if Elasticsearch is not responding.
# NOTE: Slightly faster exit w/ failure than previous "retry 240 1" if there is a problem with Elasticsearch the
# script should exit sooner rather than hang at the 'so-elasticsearch-templates' salt state.
retry 3 15 "so-elasticsearch-query / --output /dev/null --fail" ||
fail "Elasticsearch is not responding. Please review Elasticsearch logs /opt/so/log/elasticsearch/securityonion.log for more details. Additionally, consider running so-elasticsearch-troubleshoot."
}
if [[ "$FORCE" == "true" || ! -f "$SO_STATEFILE_SUCCESS" ]]; then
check_elasticsearch_responsive
if [[ "$IS_HEAVYNODE" == "false" ]]; then
# TODO: Better way to check if fleet server is installed vs checking for Elastic Defend component template.
fleet_check="logs-endpoint.alerts@package"
if ! so-elasticsearch-query "_component_template/$fleet_check" --output /dev/null --retry 5 --retry-delay 3 --fail; then
# This check prevents so-elasticsearch-templates-load from running before so-elastic-fleet-setup has run.
echo -e "\nPackage $fleet_check not yet installed. Fleet Server may not be fully configured yet."
# Fleet Server is required because some SO index templates depend on components installed via
# specific integrations eg Elastic Defend. These are components that we do not manually create / manage
# via /opt/so/saltstack/salt/elasticsearch/templates/component/
exit 0
fi
fi
# load_component_templates "Name" "directory" "append '-mappings'?"
load_component_templates "ECS" "ecs" "true"
load_component_templates "Elastic Agent" "elastic-agent"
load_component_templates "Security Onion" "so"
component_templates=$(so-elasticsearch-component-templates-list)
echo -e "Loading Security Onion index templates...\n"
for so_idx_tmpl in "${SO_TEMPLATES_DIR}"/*.json; do
tmpl_name=$(basename "${so_idx_tmpl%-template.json}")
if [[ "$IS_HEAVYNODE" == "true" ]]; then
# TODO: Better way to load only heavynode specific templates
if ! check_heavynode_compatiable_index_template "$tmpl_name"; then
if [[ "$VERBOSE" == "true" ]]; then
echo "Skipping over $so_idx_tmpl, template is not a heavynode specific index template."
fi
continue
fi
fi
if check_required_component_template_exists "$so_idx_tmpl"; then
if ! load_template "_index_template/$tmpl_name" "$so_idx_tmpl"; then
SO_LOAD_FAILURES=$((SO_LOAD_FAILURES + 1))
SO_LOAD_FAILURES_NAMES+=("$so_idx_tmpl")
fi
else
echo "Skipping over $so_idx_tmpl due to missing required component template(s)."
SO_LOAD_FAILURES=$((SO_LOAD_FAILURES + 1))
SO_LOAD_FAILURES_NAMES+=("$so_idx_tmpl")
continue
fi
done
if [[ $SO_LOAD_FAILURES -eq 0 ]]; then
echo "All Security Onion core templates loaded successfully."
touch "$SO_STATEFILE_SUCCESS"
else
echo "Encountered $SO_LOAD_FAILURES failure(s) loading templates:"
for failed_template in "${SO_LOAD_FAILURES_NAMES[@]}"; do
echo " - $failed_template"
done
if [[ "$SHOULD_EXIT_ON_FAILURE" == "true" ]]; then
fail "Failed to load all Security Onion core templates successfully."
fi
fi
else
echo "Security Onion core templates already loaded"
fi
# Start loading addon templates
if [[ (-d "$ADDON_TEMPLATES_DIR" && -f "$SO_STATEFILE_SUCCESS" && "$IS_HEAVYNODE" == "false" && ! -f "$ADDON_STATEFILE_SUCCESS") || (-d "$ADDON_TEMPLATES_DIR" && "$IS_HEAVYNODE" == "false" && "$FORCE" == "true") ]]; then
check_elasticsearch_responsive
echo -e "\nLoading addon integration index templates...\n"
component_templates=$(so-elasticsearch-component-templates-list)
for addon_idx_tmpl in "${ADDON_TEMPLATES_DIR}"/*.json; do
tmpl_name=$(basename "${addon_idx_tmpl%-template.json}")
if check_required_component_template_exists "$addon_idx_tmpl"; then
if ! load_template "_index_template/${tmpl_name}" "$addon_idx_tmpl"; then
ADDON_LOAD_FAILURES=$((ADDON_LOAD_FAILURES + 1))
ADDON_LOAD_FAILURES_NAMES+=("$addon_idx_tmpl")
fi
else
echo "Skipping over $addon_idx_tmpl due to missing required component template(s)."
ADDON_LOAD_FAILURES=$((ADDON_LOAD_FAILURES + 1))
ADDON_LOAD_FAILURES_NAMES+=("$addon_idx_tmpl")
continue
fi
done
if [[ $ADDON_LOAD_FAILURES -eq 0 ]]; then
echo "All addon integration templates loaded successfully."
touch "$ADDON_STATEFILE_SUCCESS"
else
echo "Encountered $ADDON_LOAD_FAILURES failure(s) loading addon integration templates:"
for failed_template in "${ADDON_LOAD_FAILURES_NAMES[@]}"; do
echo " - $failed_template"
done
if [[ "$SHOULD_EXIT_ON_FAILURE" == "true" ]]; then
fail "Failed to load all addon integration templates successfully."
fi
fi
elif [[ ! -f "$SO_STATEFILE_SUCCESS" && "$IS_HEAVYNODE" == "false" ]]; then
echo "Skipping loading addon integration templates until Security Onion core templates have been loaded."
elif [[ -f "$ADDON_STATEFILE_SUCCESS" && "$IS_HEAVYNODE" == "false" && "$FORCE" == "false" ]]; then
echo "Addon integration templates already loaded"
fi

View File

@@ -7,6 +7,9 @@
. /usr/sbin/so-common
{%- from 'elasticsearch/template.map.jinja' import ES_INDEX_SETTINGS %}
{%- if GLOBALS.role != "so-heavynode" %}
{%- from 'elasticsearch/template.map.jinja' import ALL_ADDON_SETTINGS %}
{%- endif %}
{%- for index, settings in ES_INDEX_SETTINGS.items() %}
{%- if settings.policy is defined %}
@@ -33,3 +36,13 @@
{%- endif %}
{%- endfor %}
echo
{%- if GLOBALS.role != "so-heavynode" %}
{%- for index, settings in ALL_ADDON_SETTINGS.items() %}
{%- if settings.policy is defined %}
echo
echo "Setting up {{ index }}-logs policy..."
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -s -k -L -X PUT "https://localhost:9200/_ilm/policy/{{ index }}-logs" -H 'Content-Type: application/json' -d'{ "policy": {{ settings.policy | tojson(true) }} }'
echo
{%- endif %}
{%- endfor %}
{%- endif %}

View File

@@ -1,165 +0,0 @@
#!/bin/bash
# Copyright Security Onion Solutions LLC and/or licensed to Security Onion Solutions LLC under one
# or more contributor license agreements. Licensed under the Elastic License 2.0 as shown at
# https://securityonion.net/license; you may not use this file except in compliance with the
# Elastic License 2.0.
{%- import_yaml 'elasticfleet/defaults.yaml' as ELASTICFLEETDEFAULTS %}
{% from 'vars/globals.map.jinja' import GLOBALS %}
STATE_FILE_INITIAL=/opt/so/state/estemplates_initial_load_attempt.txt
STATE_FILE_SUCCESS=/opt/so/state/estemplates.txt
if [[ -f $STATE_FILE_INITIAL ]]; then
# The initial template load has already run. As this is a subsequent load, all dependencies should
# already be satisified. Therefore, immediately exit/abort this script upon any template load failure
# since this is an unrecoverable failure.
should_exit_on_failure=1
else
# This is the initial template load, and there likely are some components not yet setup in Elasticsearch.
# Therefore load as many templates as possible at this time and if an error occurs proceed to the next
# template. But if at least one template fails to load do not mark the templates as having been loaded.
# This will allow the next load to resume the load of the templates that failed to load initially.
should_exit_on_failure=0
echo "This is the initial template load"
fi
# If soup is running, ignore errors
pgrep soup > /dev/null && should_exit_on_failure=0
load_failures=0
load_template() {
uri=$1
file=$2
echo "Loading template file $i"
if ! retry 3 1 "so-elasticsearch-query $uri -d@$file -XPUT" "{\"acknowledged\":true}"; then
if [[ $should_exit_on_failure -eq 1 ]]; then
fail "Could not load template file: $file"
else
load_failures=$((load_failures+1))
echo "Incremented load failure counter: $load_failures"
fi
fi
}
if [ ! -f $STATE_FILE_SUCCESS ]; then
echo "State file $STATE_FILE_SUCCESS not found. Running so-elasticsearch-templates-load."
. /usr/sbin/so-common
{% if GLOBALS.role != 'so-heavynode' %}
if [ -f /usr/sbin/so-elastic-fleet-common ]; then
. /usr/sbin/so-elastic-fleet-common
fi
{% endif %}
default_conf_dir=/opt/so/conf
# Define a default directory to load pipelines from
ELASTICSEARCH_TEMPLATES="$default_conf_dir/elasticsearch/templates/"
{% if GLOBALS.role == 'so-heavynode' %}
file="/opt/so/conf/elasticsearch/templates/index/so-common-template.json"
{% else %}
file="/usr/sbin/so-elastic-fleet-common"
{% endif %}
if [ -f "$file" ]; then
# Wait for ElasticSearch to initialize
echo -n "Waiting for ElasticSearch..."
retry 240 1 "so-elasticsearch-query / -k --output /dev/null --silent --head --fail" || fail "Connection attempt timed out. Unable to connect to ElasticSearch. \nPlease try: \n -checking log(s) in /var/log/elasticsearch/\n -running 'sudo docker ps' \n -running 'sudo so-elastic-restart'"
{% if GLOBALS.role != 'so-heavynode' %}
TEMPLATE="logs-endpoint.alerts@package"
INSTALLED=$(so-elasticsearch-query _component_template/$TEMPLATE | jq -r .component_templates[0].name)
if [ "$INSTALLED" != "$TEMPLATE" ]; then
echo
echo "Packages not yet installed."
echo
exit 0
fi
{% endif %}
touch $STATE_FILE_INITIAL
cd ${ELASTICSEARCH_TEMPLATES}/component/ecs
echo "Loading ECS component templates..."
for i in *; do
TEMPLATE=$(echo $i | cut -d '.' -f1)
load_template "_component_template/${TEMPLATE}-mappings" "$i"
done
echo
cd ${ELASTICSEARCH_TEMPLATES}/component/elastic-agent
echo "Loading Elastic Agent component templates..."
{% if GLOBALS.role == 'so-heavynode' %}
component_pattern="so-*"
{% else %}
component_pattern="*"
{% endif %}
for i in $component_pattern; do
TEMPLATE=${i::-5}
load_template "_component_template/$TEMPLATE" "$i"
done
echo
# Load SO-specific component templates
cd ${ELASTICSEARCH_TEMPLATES}/component/so
echo "Loading Security Onion component templates..."
for i in *; do
TEMPLATE=$(echo $i | cut -d '.' -f1);
load_template "_component_template/$TEMPLATE" "$i"
done
echo
# Load SO index templates
cd ${ELASTICSEARCH_TEMPLATES}/index
echo "Loading Security Onion index templates..."
shopt -s extglob
{% if GLOBALS.role == 'so-heavynode' %}
pattern="!(*1password*|*aws*|*azure*|*cloudflare*|*elastic_agent*|*fim*|*github*|*google*|*osquery*|*system*|*windows*|*endpoint*|*elasticsearch*|*generic*|*fleet_server*|*soc*)"
{% else %}
pattern="*"
{% endif %}
# Index templates will be skipped if the following conditions are met:
# 1. The template is part of the "so-logs-" template group
# 2. The template name does not correlate to at least one existing component template
# In this situation, the script will treat the skipped template as a temporary failure
# and allow the templates to be loaded again on the next run or highstate, whichever
# comes first.
COMPONENT_LIST=$(so-elasticsearch-component-templates-list)
for i in $pattern; do
TEMPLATE=${i::-14}
COMPONENT_PATTERN=${TEMPLATE:3}
MATCH=$(echo "$TEMPLATE" | grep -E "^so-logs-|^so-metrics" | grep -vE "detections|osquery")
if [[ -n "$MATCH" && ! "$COMPONENT_LIST" =~ "$COMPONENT_PATTERN" && ! "$COMPONENT_PATTERN" =~ \.generic|logs-winlog\.winlog ]]; then
load_failures=$((load_failures+1))
echo "Component template does not exist for $COMPONENT_PATTERN. The index template will not be loaded. Load failures: $load_failures"
else
load_template "_index_template/$TEMPLATE" "$i"
fi
done
else
{% if GLOBALS.role == 'so-heavynode' %}
echo "Common template does not exist. Exiting..."
{% else %}
echo "Elastic Fleet not configured. Exiting..."
{% endif %}
exit 0
fi
cd - >/dev/null
if [[ $load_failures -eq 0 ]]; then
echo "All templates loaded successfully"
touch $STATE_FILE_SUCCESS
else
echo "Encountered $load_failures templates that were unable to load, likely due to missing dependencies that will be available later; will retry on next highstate"
fi
else
echo "Templates already loaded"
fi

View File

@@ -9,5 +9,5 @@ SESSIONCOOKIE=$(curl -K /opt/so/conf/elasticsearch/curl.config -c - -X GET http:
# Disable certain Features from showing up in the Kibana UI
echo
echo "Setting up default Kibana Space:"
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X PUT "localhost:5601/api/spaces/space/default" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d' {"id":"default","name":"Default","disabledFeatures":["ml","enterpriseSearch","logs","infrastructure","apm","uptime","monitoring","stackAlerts","actions","securitySolutionCasesV3","inventory","dataQuality","searchSynonyms","enterpriseSearchApplications","enterpriseSearchAnalytics","securitySolutionTimeline","securitySolutionNotes","entityManager"]} ' >> /opt/so/log/kibana/misc.log
curl -K /opt/so/conf/elasticsearch/curl.config -b "sid=$SESSIONCOOKIE" -L -X PUT "localhost:5601/api/spaces/space/default" -H 'kbn-xsrf: true' -H 'Content-Type: application/json' -d' {"id":"default","name":"Default","disabledFeatures":["ml","enterpriseSearch","logs","infrastructure","apm","uptime","monitoring","stackAlerts","actions","securitySolutionCasesV3","inventory","dataQuality","searchSynonyms","searchQueryRules","enterpriseSearchApplications","enterpriseSearchAnalytics","securitySolutionTimeline","securitySolutionNotes","securitySolutionRulesV1","entityManager","streams","cloudConnect","slo"]} ' >> /opt/so/log/kibana/misc.log
echo

View File

@@ -305,7 +305,7 @@ clone_to_tmp() {
# Make a temp location for the files
mkdir -p /tmp/sogh
cd /tmp/sogh
SOUP_BRANCH="-b 2.4/main"
SOUP_BRANCH="-b 3/main"
if [ -n "$BRANCH" ]; then
SOUP_BRANCH="-b $BRANCH"
fi
@@ -363,6 +363,7 @@ preupgrade_changes() {
echo "Checking to see if changes are needed."
[[ "$INSTALLEDVERSION" =~ ^2\.4\.21[0-9]+$ ]] && up_to_3.0.0
[[ "$INSTALLEDVERSION" == "3.0.0" ]] && up_to_3.1.0
true
}
@@ -371,6 +372,7 @@ postupgrade_changes() {
echo "Running post upgrade processes."
[[ "$POSTVERSION" =~ ^2\.4\.21[0-9]+$ ]] && post_to_3.0.0
[[ "$POSTVERSION" == "3.0.0" ]] && post_to_3.1.0
true
}
@@ -445,7 +447,6 @@ migrate_pcap_to_suricata() {
}
up_to_3.0.0() {
determine_elastic_agent_upgrade
migrate_pcap_to_suricata
INSTALLEDVERSION=3.0.0
@@ -469,6 +470,32 @@ post_to_3.0.0() {
### 3.0.0 End ###
### 3.1.0 Scripts ###
elasticsearch_backup_index_templates() {
echo "Backing up current elasticsearch index templates in /opt/so/conf/elasticsearch/templates/index/ to /nsm/backup/3.0.0_elasticsearch_index_templates.tar.gz"
tar -czf /nsm/backup/3.0.0_elasticsearch_index_templates.tar.gz -C /opt/so/conf/elasticsearch/templates/index/ .
}
up_to_3.1.0() {
determine_elastic_agent_upgrade
elasticsearch_backup_index_templates
# Clear existing component template state file.
rm -f /opt/so/state/esfleet_component_templates.json
INSTALLEDVERSION=3.1.0
}
post_to_3.1.0() {
/usr/sbin/so-kibana-space-defaults
POSTVERSION=3.1.0
}
### 3.1.0 End ###
repo_sync() {
echo "Sync the local repo."
su socore -c '/usr/sbin/so-repo-sync' || fail "Unable to complete so-repo-sync."
@@ -728,12 +755,12 @@ verify_es_version_compatibility() {
local is_active_intermediate_upgrade=1
# supported upgrade paths for SO-ES versions
declare -A es_upgrade_map=(
["8.18.8"]="9.0.8"
["9.0.8"]="9.3.3"
)
# Elasticsearch MUST upgrade through these versions
declare -A es_to_so_version=(
["8.18.8"]="2.4.190-20251024"
["9.0.8"]="3.0.0-20260331"
)
# Get current Elasticsearch version
@@ -745,26 +772,17 @@ verify_es_version_compatibility() {
exit 160
fi
if ! target_es_version_raw=$(so-yaml.py get $UPDATE_DIR/salt/elasticsearch/defaults.yaml elasticsearch.version); then
# so-yaml.py failed to get the ES version from upgrade versions elasticsearch/defaults.yaml file. Likely they are upgrading to an SO version older than 2.4.110 prior to the ES version pinning and should be OKAY to continue with the upgrade.
if ! target_es_version=$(so-yaml.py get -r $UPDATE_DIR/salt/elasticsearch/defaults.yaml elasticsearch.version); then
echo "Couldn't determine the target Elasticsearch version (post soup version) to ensure compatibility with current Elasticsearch version. Exiting"
# if so-yaml.py failed to get the ES version AND the version we are upgrading to is newer than 2.4.110 then we should bail
if [[ $(cat $UPDATE_DIR/VERSION | cut -d'.' -f3) > 110 ]]; then
echo "Couldn't determine the target Elasticsearch version (post soup version) to ensure compatibility with current Elasticsearch version. Exiting"
exit 160
fi
# allow upgrade to version < 2.4.110 without checking ES version compatibility
return 0
else
target_es_version=$(sed -n '1p' <<< "$target_es_version_raw")
exit 160
fi
for statefile in "${es_required_version_statefile_base}"-*; do
[[ -f $statefile ]] || continue
local es_required_version_statefile_value=$(cat "$statefile")
local es_required_version_statefile_value
es_required_version_statefile_value=$(cat "$statefile")
if [[ "$es_required_version_statefile_value" == "$target_es_version" ]]; then
echo "Intermediate upgrade to ES $target_es_version is in progress. Skipping Elasticsearch version compatibility check."
@@ -773,7 +791,7 @@ verify_es_version_compatibility() {
fi
# use sort to check if es_required_statefile_value is < the current es_version.
if [[ "$(printf '%s\n' $es_required_version_statefile_value $es_version | sort -V | head -n1)" == "$es_required_version_statefile_value" ]]; then
if [[ "$(printf '%s\n' "$es_required_version_statefile_value" "$es_version" | sort -V | head -n1)" == "$es_required_version_statefile_value" ]]; then
rm -f "$statefile"
continue
fi
@@ -784,8 +802,7 @@ verify_es_version_compatibility() {
echo -e "\n##############################################################################################################################\n"
echo "A previously required intermediate Elasticsearch upgrade was detected. Verifying that all Searchnodes/Heavynodes have successfully upgraded Elasticsearch to $es_required_version_statefile_value before proceeding with soup to avoid potential data loss! This command can take up to an hour to complete."
timeout --foreground 4000 bash "$es_verification_script" "$es_required_version_statefile_value" "$statefile"
if [[ $? -ne 0 ]]; then
if ! timeout --foreground 4000 bash "$es_verification_script" "$es_required_version_statefile_value" "$statefile"; then
echo -e "\n!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n"
echo "A previous required intermediate Elasticsearch upgrade to $es_required_version_statefile_value has yet to successfully complete across the grid. Please allow time for all Searchnodes/Heavynodes to have upgraded Elasticsearch to $es_required_version_statefile_value before running soup again to avoid potential data loss!"
@@ -802,6 +819,7 @@ verify_es_version_compatibility() {
return 0
fi
# shellcheck disable=SC2076 # Do not want a regex here eg usage " 8.18.8 9.0.8 " =~ " 9.0.8 "
if [[ " ${es_upgrade_map[$es_version]} " =~ " $target_es_version " || "$es_version" == "$target_es_version" ]]; then
# supported upgrade
return 0
@@ -810,7 +828,7 @@ verify_es_version_compatibility() {
if [[ -z "$compatible_versions" ]]; then
# If current ES version is not explicitly defined in the upgrade map, we know they have an intermediate upgrade to do.
# We default to the lowest ES version defined in es_to_so_version as $first_es_required_version
local first_es_required_version=$(printf '%s\n' "${!es_to_so_version[@]}" | sort -V | head -n1)
first_es_required_version=$(printf '%s\n' "${!es_to_so_version[@]}" | sort -V | head -n1)
next_step_so_version=${es_to_so_version[$first_es_required_version]}
required_es_upgrade_version="$first_es_required_version"
else
@@ -829,7 +847,7 @@ verify_es_version_compatibility() {
if [[ $is_airgap -eq 0 ]]; then
run_airgap_intermediate_upgrade
else
if [[ ! -z $ISOLOC ]]; then
if [[ -n $ISOLOC ]]; then
originally_requested_iso_location="$ISOLOC"
fi
# Make sure ISOLOC is not set. Network installs that used soup -f would have ISOLOC set.
@@ -861,7 +879,8 @@ wait_for_salt_minion_with_restart() {
}
run_airgap_intermediate_upgrade() {
local originally_requested_so_version=$(cat $UPDATE_DIR/VERSION)
local originally_requested_so_version
originally_requested_so_version=$(cat "$UPDATE_DIR/VERSION")
# preserve ISOLOC value, so we can try to use it post intermediate upgrade
local originally_requested_iso_location="$ISOLOC"
@@ -873,7 +892,8 @@ run_airgap_intermediate_upgrade() {
while [[ -z "$next_iso_location" ]] || [[ ! -f "$next_iso_location" && ! -b "$next_iso_location" ]]; do
# List removable devices if any are present
local removable_devices=$(lsblk -no PATH,SIZE,TYPE,MOUNTPOINTS,RM | awk '$NF==1')
local removable_devices
removable_devices=$(lsblk -no PATH,SIZE,TYPE,MOUNTPOINTS,RM | awk '$NF==1')
if [[ -n "$removable_devices" ]]; then
echo "PATH SIZE TYPE MOUNTPOINTS RM"
echo "$removable_devices"
@@ -894,21 +914,21 @@ run_airgap_intermediate_upgrade() {
echo "Using $next_iso_location for required intermediary upgrade."
exec bash <<EOF
ISOLOC=$next_iso_location soup -y && \
ISOLOC=$next_iso_location soup -y && \
ISOLOC="$next_iso_location" soup -y && \
ISOLOC="$next_iso_location" soup -y && \
echo -e "\n##############################################################################################################################\n" && \
echo -e "Verifying Elasticsearch was successfully upgraded to $required_es_upgrade_version across the grid. This part can take a while as Searchnodes/Heavynodes sync up with the Manager! \n\nOnce verification completes the next soup will begin automatically. If verification takes longer than 1 hour it will stop waiting and your grid will remain at $next_step_so_version. Allowing for all Searchnodes/Heavynodes to upgrade Elasticsearch to the required version on their own time.\n" && \
timeout --foreground 4000 bash /tmp/so_intermediate_upgrade_verification.sh $required_es_upgrade_version $es_required_version_statefile && \
timeout --foreground 4000 bash /tmp/so_intermediate_upgrade_verification.sh "$required_es_upgrade_version" "$es_required_version_statefile" && \
echo -e "\n##############################################################################################################################\n" && \
# automatically start the next soup if the original ISO isn't using the same block device we just used
if [[ -n "$originally_requested_iso_location" ]] && [[ "$originally_requested_iso_location" != "$next_iso_location" ]]; then
umount /tmp/soagupdate
ISOLOC=$originally_requested_iso_location soup -y && \
ISOLOC=$originally_requested_iso_location soup -y
ISOLOC="$originally_requested_iso_location" soup -y && \
ISOLOC="$originally_requested_iso_location" soup -y
else
echo "Could not automatically start next soup to $originally_requested_so_version. Soup will now exit here at $(cat /etc/soversion)" && \
@@ -924,29 +944,29 @@ run_network_intermediate_upgrade() {
if [[ -n "$BRANCH" ]]; then
local originally_requested_so_branch="$BRANCH"
else
local originally_requested_so_branch="2.4/main"
local originally_requested_so_branch="3/main"
fi
echo "Starting automated intermediate upgrade to $next_step_so_version."
echo "After completion, the system will automatically attempt to upgrade to the latest version."
echo -e "\n##############################################################################################################################\n"
exec bash << EOF
BRANCH=$next_step_so_version soup -y && \
BRANCH=$next_step_so_version soup -y && \
BRANCH="$next_step_so_version" soup -y && \
BRANCH="$next_step_so_version" soup -y && \
echo -e "\n##############################################################################################################################\n" && \
echo -e "Verifying Elasticsearch was successfully upgraded to $required_es_upgrade_version across the grid. This part can take a while as Searchnodes/Heavynodes sync up with the Manager! \n\nOnce verification completes the next soup will begin automatically. If verification takes longer than 1 hour it will stop waiting and your grid will remain at $next_step_so_version. Allowing for all Searchnodes/Heavynodes to upgrade Elasticsearch to the required version on their own time.\n" && \
timeout --foreground 4000 bash /tmp/so_intermediate_upgrade_verification.sh $required_es_upgrade_version $es_required_version_statefile && \
timeout --foreground 4000 bash /tmp/so_intermediate_upgrade_verification.sh "$required_es_upgrade_version" "$es_required_version_statefile" && \
echo -e "\n##############################################################################################################################\n" && \
if [[ -n "$originally_requested_iso_location" ]]; then
# nonairgap soup that used -f originally, runs intermediate upgrade using network + BRANCH, later coming back to the original ISO for the last soup
ISOLOC=$originally_requested_iso_location soup -y && \
ISOLOC=$originally_requested_iso_location soup -y
ISOLOC="$originally_requested_iso_location" soup -y && \
ISOLOC="$originally_requested_iso_location" soup -y
else
BRANCH=$originally_requested_so_branch soup -y && \
BRANCH=$originally_requested_so_branch soup -y
BRANCH="$originally_requested_so_branch" soup -y && \
BRANCH="$originally_requested_so_branch" soup -y
fi
echo -e "\n##############################################################################################################################\n"
EOF

View File

@@ -2622,6 +2622,7 @@ soc:
This is a YARA rule template. Replace all template values with your own values.
The YARA rule name is the unique identifier for the rule.
Docs: https://yara.readthedocs.io/en/stable/writingrules.html#writing-yara-rules
Delete these comments before attempting to "Create" the rule
*/
rule Example // This identifier _must_ be unique

View File

@@ -8,6 +8,7 @@ soc:
description: When this setting is enabled and the grid is not in airgap mode, SOC will provide feature usage data to the Security Onion development team via Google Analytics. This data helps Security Onion developers determine which product features are being used and can also provide insight into improving the user interface. When changing this setting, wait for the grid to fully synchronize and then perform a hard browser refresh on SOC, to force the browser cache to update and reflect the new setting.
global: True
helpLink: telemetry
forcedType: bool
files:
soc:
banner__md:
@@ -139,6 +140,7 @@ soc:
title: Require TOTP
description: Require all users to enable Time-based One Time Passwords (MFA) upon login to SOC.
global: True
forcedType: bool
customReportsPath:
title: Custom Reports Path
description: Path to custom markdown templates for PDF report generation. All markdown files in this directory will be available as custom reports in the SOC Reports interface.
@@ -185,6 +187,7 @@ soc:
description: "Set to true to enable reverse DNS lookups for IP addresses in the SOC UI. To add your own local lookups, create a CSV file at /nsm/custom-mappings/ip-descriptions.csv on your Manager and populate the file with IP addresses and descriptions as follows: IP, Description. Elasticsearch will then ingest the CSV during the next high state."
global: True
helpLink: security-onion-console-customization#reverse-dns
forcedType: bool
modules:
elastalertengine:
aiRepoUrl:
@@ -202,6 +205,7 @@ soc:
showAiSummaries:
description: Show AI summaries for ElastAlert rules.
global: True
forcedType: bool
additionalAlerters:
title: "Notifications: Sev 0/Default Alerters"
description: "Specify default alerters to enable for outbound notifications. These alerters will be used unless overridden by higher severity alerter settings. Specify one alerter name (Ex: 'email') per line. Alerters refers to ElastAlert 2 alerters, as documented at https://elastalert2.readthedocs.io. A full update of the ElastAlert rule engine, via the Detections screen, is required in order to apply these changes. Requires a valid Security Onion license key."
@@ -338,6 +342,7 @@ soc:
description: 'Automatically update Sigma rules on a regular basis. This will update the rules based on the configured frequency.'
global: True
advanced: True
forcedType: bool
communityRulesImportFrequencySeconds:
description: 'How often to check for new Sigma rules (in seconds). This applies to both Community Rule Packages and any configured Git repos.'
global: True
@@ -395,6 +400,7 @@ soc:
description: Set to true if the SOC case management module, natively integrated with Elasticsearch, should be enabled.
global: True
advanced: True
forcedType: bool
extractCommonObservables:
description: List of indexed fields to automatically extract into a case observable, when attaching related events to a case.
global: True
@@ -421,6 +427,7 @@ soc:
lookupTunnelParent:
description: When true, if a pivoted event appears to be encapsulated, such as in a VXLAN packet, then SOC will pivot to the VXLAN packet stream. When false, SOC will attempt to pivot to the encapsulated packet stream itself, but at the risk that it may be unable to locate it in the stored PCAP data.
global: True
forcedType: bool
maxScrollSize:
description: The maximum number of documents to request in a single Elasticsearch scroll request.
bulkIndexWorkerCount:
@@ -477,10 +484,12 @@ soc:
showAiSummaries:
description: Show AI summaries for Strelka rules.
global: True
forcedType: bool
autoUpdateEnabled:
description: 'Automatically update YARA rules on a regular basis. This will update the rules based on the configured frequency.'
global: True
advanced: True
forcedType: bool
autoEnabledYaraRules:
description: 'YARA rules to automatically enable on initial import. Format is $Ruleset - for example, for the default shipped ruleset: securityonion-yara'
global: True
@@ -536,10 +545,12 @@ soc:
showAiSummaries:
description: Show AI summaries for Suricata rules.
global: True
forcedType: bool
autoUpdateEnabled:
description: 'Automatically update Suricata rules on a regular basis. This will update the rules based on the configured frequency.'
global: True
advanced: True
forcedType: bool
communityRulesImportFrequencySeconds:
description: 'How often to check for new Suricata rules (in seconds).'
global: True
@@ -669,7 +680,7 @@ soc:
adapters:
description: Configuration for AI adapters used by the Onion AI assistant. Please see documentation for help on which fields are required for which protocols.
global: True
advanced: True
advanced: False
forcedType: "[]{}"
helpLink: onion-ai
syntax: json
@@ -709,6 +720,7 @@ soc:
enabled:
description: Set to true to enable the Onion AI assistant in SOC.
global: True
forcedType: bool
investigationPrompt:
description: Prompt given to Onion AI when beginning an investigation.
global: True
@@ -734,7 +746,7 @@ soc:
availableModels:
description: List of AI models available for use in SOC as well as model specific warning thresholds.
global: True
advanced: True
advanced: False
forcedType: "[]{}"
helpLink: onion-ai
syntax: json
@@ -789,9 +801,11 @@ soc:
casesEnabled:
description: Set to true to enable case management in SOC.
global: True
forcedType: bool
detectionsEnabled:
description: Set to true to enable the Detections module in SOC.
global: True
forcedType: bool
inactiveTools:
description: List of external tools to remove from the SOC UI.
global: True
@@ -867,6 +881,7 @@ soc:
showUnreviewedAiSummaries:
description: Show AI summaries in detections even if they have not yet been reviewed by a human.
global: True
forcedType: bool
templateDetections:
suricata:
description: The template used when creating a new Suricata detection. [publicId] will be replaced with an unused Public Id.
@@ -904,6 +919,7 @@ soc:
customEnabled:
description: Set to true to allow users add their own artifact types directly in the SOC UI.
global: True
forcedType: bool
category:
labels:
description: List of available case categories.
@@ -911,6 +927,7 @@ soc:
customEnabled:
description: Set to true to allow users add their own categories directly in the SOC UI.
global: True
forcedType: bool
pap:
labels:
description: List of available PAP (Permissible Actions Protocol) values.
@@ -918,6 +935,7 @@ soc:
customEnabled:
description: Set to true to allow users add their own PAP values directly in the SOC UI.
global: True
forcedType: bool
severity:
labels:
description: List of available case severities.
@@ -925,6 +943,7 @@ soc:
customEnabled:
description: Set to true to allow users add their own severities directly in the SOC UI.
global: True
forcedType: bool
status:
labels:
description: List of available case statuses. Note that some default statuses have special characteristics and related functionality built into SOC.
@@ -932,6 +951,7 @@ soc:
customEnabled:
description: Set to true to allow users add their own case statuses directly in the SOC UI.
global: True
forcedType: bool
tags:
labels:
description: List of available tags.
@@ -939,6 +959,7 @@ soc:
customEnabled:
description: Set to true to allow users add their own tags directly in the SOC UI.
global: True
forcedType: bool
tlp:
labels:
description: List of available TLP (Traffic Light Protocol) values.
@@ -946,3 +967,4 @@ soc:
customEnabled:
description: Set to true to allow users add their own TLP values directly in the SOC UI.
global: True
forcedType: bool

File diff suppressed because it is too large Load Diff

View File

@@ -161,7 +161,7 @@ suricata:
address-groups:
HOME_NET:
description: Assign a list of hosts, or networks, using CIDR notation, to this Suricata variable. The variable can then be re-used within Suricata rules. This allows for a single adjustment to the variable that will then affect all rules referencing the variable.
regex: ^(((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(\/([0-9]|[1-2][0-9]|3[0-2]))?$|^((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)\.){3}(25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?))|:))|(([0-9A-Fa-f]{1,4}:){5}((:[0-9A-Fa-f]{1,4}){1,2}|:((25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)\.){3}(25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)|:))|(([0-9A-Fa-f]{1,4}:){4}((:[0-9A-Fa-f]{1,4}){1,3}|:((25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)\.){3}(25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)|:))|(([0-9A-Fa-f]{1,4}:){3}((:[0-9A-Fa-f]{1,4}){1,4}|:((25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)\.){3}(25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)|:))|(([0-9A-Fa-f]{1,4}:){2}((:[0-9A-Fa-f]{1,4}){1,5}|:((25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)\.){3}(25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)|:))|(([0-9A-Fa-f]{1,4}:){1}((:[0-9A-Fa-f]{1,4}){1,6}|:((25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)\.){3}(25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)|:))|(:((:[0-9A-Fa-f]{1,4}){1,7}|:)))(\/([0-9]|[1-9][0-9]|1[0-1][0-9]|12[0-8]))?$
regex: ^!?((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(\/([0-9]|[1-2][0-9]|3[0-2]))?$|^!?((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)\.){3}(25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)|:))|(([0-9A-Fa-f]{1,4}:){5}((:[0-9A-Fa-f]{1,4}){1,2}|:((25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)\.){3}(25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)|:))|(([0-9A-Fa-f]{1,4}:){4}((:[0-9A-Fa-f]{1,4}){1,3}|:((25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)\.){3}(25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)|:))|(([0-9A-Fa-f]{1,4}:){3}((:[0-9A-Fa-f]{1,4}){1,4}|:((25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)\.){3}(25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)|:))|(([0-9A-Fa-f]{1,4}:){2}((:[0-9A-Fa-f]{1,4}){1,5}|:((25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)\.){3}(25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)|:))|(([0-9A-Fa-f]{1,4}:){1}((:[0-9A-Fa-f]{1,4}){1,6}|:((25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)\.){3}(25[0-5]|(2[0-4]|1[0-9])[0-9]|0?[0-9][0-9]?)|:))|(:((:[0-9A-Fa-f]{1,4}){1,7}|:)))(\/([0-9]|[1-9][0-9]|1[0-1][0-9]|12[0-8]))?$
regexFailureMessage: You must enter a valid IP address or CIDR.
forcedType: "[]string"
duplicates: True

View File

@@ -27,7 +27,7 @@ echo ""
sleep 3
rm -rf /tmp/nids-testing/output
mkdir -p /tmp/nids-testing/output
mkdir -p /tmp/nids-testing/output/suripcap
chown suricata:socore /tmp/nids-testing/output
mkdir -p /tmp/nids-testing/rules
@@ -45,7 +45,7 @@ echo "==== Begin Suricata Output ==="
-v /opt/so/conf/suricata/bpf:/etc/suricata/bpf:ro \
-v /tmp/nids-testing/output/:/nsm/:rw \
{{ MANAGER }}:5000/{{ IMAGEREPO }}/so-suricata:{{ VERSION }} \
--runmode single -v -k none -r /input.pcap -l /tmp --init-errors-fatal --set outputs.6.pcap-log.enabled=no
--runmode single -v -k none -r /input.pcap -l /tmp --init-errors-fatal
echo "==== End Suricata Output ==="
echo ""

View File

@@ -38,6 +38,7 @@ zeekzkgsync:
- source: salt://zeek/zkg
- user: 937
- group: 939
- clean: True
- makedirs: True
- exclude_pat: README

Binary file not shown.