Josh Patterson
43f72c1f9f
Parallelize so-elasticsearch-templates-load template PUTs
...
Load component and index templates as throttled background jobs (max 10
concurrent) instead of sequential curl PUTs, matching the bounded-concurrency
+ flock-serialized-output pattern used by the fleet/ILM load scripts. Keeps a
wait barrier between the component phase and the index phase so index
templates never load before their referenced component templates. Failures are
tracked via per-job marker files since counter increments can't escape
background subshells.
2026-06-12 15:11:34 -04:00
Josh Patterson
6c42c419e2
Serialize ILM policy-load output with flock to stop interleaving
...
A single printf per block was not actually one write() call, so
concurrent jobs still occasionally interleaved their label and response
lines. Hold an flock around just the printf (curl still runs in
parallel) so each policy's block prints intact, keeping live
completion-order streaming.
2026-06-11 15:42:41 -04:00
Josh Patterson
07d3b148b5
fix output
2026-06-11 13:37:26 -04:00
Josh Patterson
780d9faf0d
Parallelize so-elasticsearch-ilm-policy-load PUTs
...
Run the ~300 ILM policy PUTs concurrently (bounded to 10 in flight via a
throttle gate) instead of one serial curl per policy. Adds a put_policy
helper and waits for all background jobs before exiting. Preserves policy
parity; only the scheduling changes. Drops the dead empty sid cookie arg
(falls back to basic auth from curl.config as before).
2026-06-11 12:08:32 -04:00
reyesj2
ac907ba45f
fix elasticsearch template generation issue
2026-06-05 16:42:08 -05:00
Jorge Reyes
638aca97c8
Merge pull request #15877 from Security-Onion-Solutions/reyesj2-patch-1
...
update redis index template
2026-05-13 13:44:04 -05:00
reyesj2
d56bf01823
add zeek.ja4d ingest pipeline
2026-05-13 12:32:54 -05:00
reyesj2
492ae80da7
add ingest latency metrics
2026-05-11 16:51:38 -05:00
Jorge Reyes
4a2177c827
update redis index template
...
missing redis integration component templates
2026-05-11 16:15:56 -05:00
Josh Brower
499f7102bd
cleanup status code
2026-05-07 11:27:49 -04:00
reyesj2
dceed421ae
update grok type conversion to convert processor
2026-05-05 13:41:00 -05:00
reyesj2
b6acf3b522
typo
2026-04-24 09:24:58 -05:00
reyesj2
fdfca469cc
prevent non-manager nodes from running elasticsearch.cluster state manually
2026-04-23 09:53:07 -05:00
reyesj2
22f869734e
add check for files before attempting to use file pattern to load templates
2026-04-22 23:11:31 -05:00
reyesj2
72dbb69a1c
fix searchnodes running elasticsearch/cluster state
2026-04-22 20:37:48 -05:00
reyesj2
ebb93b4fa7
add wait_for_so-elasticsearch state and split elasticsearch cluster configuration out of enabled.sls
2026-04-17 14:43:07 -05:00
reyesj2
ba00ae8a7b
supress noisy warning from ES 9.3.3
2026-04-16 14:41:25 -05:00
reyesj2
16a4a42faf
check for addon-index templates dir before attempting to load addon index templates
2026-04-14 19:26:37 -05:00
reyesj2
a232cd89cc
ES 9.3.3
2026-04-13 13:36:51 -05:00
reyesj2
dd40e44530
show when addon integrations are already loaded
2026-04-13 12:36:42 -05:00
reyesj2
29e13b2c0b
elasticsearch ilm policy load script
2026-04-13 10:00:17 -05:00
reyesj2
abcad9fde0
addon statefile
2026-04-12 00:36:30 -05:00
reyesj2
a43947cca5
elasticsearch template load script -- for addon index templates
2026-04-12 00:23:26 -05:00
reyesj2
b0584a4dc5
only append "-mappings" to component template names as needed
2026-04-11 15:22:50 -05:00
reyesj2
6298397534
rework elasticsearch template load script -- for core templates
2026-04-11 04:40:47 -05:00
reyesj2
378d1ec81b
initialize vars
2026-04-09 18:41:40 -05:00
reyesj2
89e49d0bf3
rework elasticsearch index template generation
2026-04-09 16:44:51 -05:00
reyesj2
8101bc4941
ES 9.3.2
2026-04-06 15:08:30 -05:00
Josh Brower
1f9bf45b66
Lowercase network transport
2026-03-24 11:24:59 -04:00
Josh Brower
165e69cd11
Add support for websockets
2026-03-23 07:52:36 -04:00
Josh Patterson
20bf88b338
ensure bool sliders for elasticsearch
2026-03-19 13:52:40 -04:00
Josh Patterson
c2c5aea244
ensure bool sliders for each state:enabled annotation
2026-03-19 12:35:38 -04:00
Josh Patterson
a982056363
Merge remote-tracking branch 'origin/3/dev' into delta
2026-03-18 15:45:15 -04:00
reyesj2
c16ff2bd99
so-idh and so-redis datastream config
2026-03-18 14:31:23 -05:00
Josh Patterson
74ad2990a7
Merge remote-tracking branch 'origin/3/dev' into delta
2026-03-18 13:05:02 -04:00
Jorge Reyes
20c4da50b1
Merge pull request #15632 from Security-Onion-Solutions/reyesj2-15601
...
fix global override settings affecting non-data stream indices
2026-03-18 10:51:17 -05:00
Josh Patterson
e19e83bebb
allow user defined ulimits
2026-03-18 10:38:15 -04:00
Doug Burks
930985b770
update helpLink references for new documentation
2026-03-18 09:46:45 -04:00
Jorge Reyes
346dc446de
Merge pull request #15630 from Security-Onion-Solutions/reyesj2-449
...
use elasticsearch recommended vm.max_map_count
2026-03-17 15:36:06 -05:00
reyesj2
7e7b8dc8a8
vm.max_map_count allow for minion specific values
2026-03-17 15:23:46 -05:00
Josh Patterson
2349750e13
DOCKER to DOCKERMERGED
2026-03-17 16:19:02 -04:00
reyesj2
2c6c502067
use elasticsearch recommended vm.max_map_count
2026-03-17 15:12:29 -05:00
Josh Patterson
00986dc2fd
Merge remote-tracking branch 'origin/delta' into customulimit
2026-03-17 16:04:09 -04:00
Mike Reeves
2d97dfc8a1
Add customizable ulimit settings for all Docker containers
...
Add ulimits as a configurable advanced setting for every container,
allowing customization through the web UI. Move hardcoded ulimits
from elasticsearch and zeek into defaults.yaml and fix elasticsearch
ulimits that were incorrectly nested under the environment key.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-17 15:10:42 -04:00
Josh Patterson
4dc377c99f
DOCKER to DOCKERMERGED
2026-03-17 15:06:06 -04:00
reyesj2
a52e5d0474
update index template priorities + explicity add datastream config options
2026-03-17 13:50:15 -05:00
Josh Patterson
1a7d72c630
ensure empty directory tracked by git
2026-03-17 11:11:02 -04:00
reyesj2
eaf3f10adc
remove unused close/delete configs on datastream index templates
2026-03-16 17:26:45 -05:00
reyesj2
84f4e460f6
update index patterns
2026-03-16 16:53:22 -05:00
reyesj2
88841c9814
remove ilm configs from non-datastream indices
2026-03-16 16:52:42 -05:00