Commit Graph
29 Commits
Author SHA1 Message Date
Josh Patterson f566a8965d Merge branch '3/dev' into feature/logstash-log-level 2026-08-21 11:45:00 -04:00
Josh Patterson 12744353fb Allow ten custom logstash pipelines instead of five 2026-08-21 10:21:12 -04:00
Josh Patterson 52fc0cb828 Expose Logstash log.level and log.format in SOC
The Logstash log level was hardcoded to info in log4j2.properties, and
logstash.yml carried no log.level key, so the only way to raise verbosity
for troubleshooting was to hand-edit a file that the next highstate
overwrites. Add log_x_level and log_x_format to logstash:config so both
render into logstash.yml, annotated as advanced per-node settings with
the value sets Logstash 9.3.7 accepts.

log4j2.properties gains jinja, so it moves to log4j2.properties.jinja and
is rendered by a discrete lslog4j2 state rather than the lsetcsync
recurse, which cannot rename. The recurse exclude_pat now matches both
names so it neither copies the template verbatim nor lets clean: True
delete the rendered file, matching how pipelines.yml is already handled.

The appender layout is selected at render time so log.format actually
changes the log output instead of being a dead setting, keeping the
existing file name so nothing downstream moves. rootLogger.level now
follows ls.log.level rather than claiming info regardless of the
configured level.
2026-08-21 09:47:09 -04:00
Josh Patterson dff3d76efd Expose Logstash 9.3.7 pipeline settings per pipeline in SOC
Add logstash:pipeline_settings carrying the 27 pipeline-scoped settings
Logstash 9.3.7 accepts, annotated individually per pipeline and rendered
into pipelines.yml. A blank setting inherits from logstash.yml. Restart
logstash when pipelines.yml changes, and add the missing managerhype
annotation.

Fixes #15090
2026-08-19 16:21:28 -04:00
reyesj2 492ae80da7 add ingest latency metrics 2026-05-11 16:51:38 -05:00
reyesj2 7977a020ac elasticsearch 9.0.8 2025-12-16 16:03:47 -06:00
Josh Patterson 44a5b3b1e5 MANAGERHYPE setup is now complete! 2025-03-12 21:05:04 -04:00
reyesj2 d5faf535c3 Only interact with logstash configuration when Kafka pipeline is enabled otherwise leave it default
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
2024-07-10 11:36:44 -04:00
m0duspwnens 50f0c43212 merge dev 2024-06-26 12:33:32 -04:00
m0duspwnens 55f8303dc2 remove manager and search pipelines from heavynode 2024-06-17 10:06:43 -04:00
reyesj2 3efdb4e532 Reconfigure logstash Kafka input
- TODO: Configure what topics are pulled to searchnodes via the SOC UI

Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
2024-05-01 13:01:29 -04:00
reyesj2 af53dcda1b Remove references to kafkanode
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
2024-04-11 15:32:00 -04:00
reyesj2 446f1ffdf5 merge 2.4/dev
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
2024-03-25 13:55:48 -04:00
Wes 1818e134ca Change numbers for Logstash 2024-02-01 14:01:55 +00:00
Wes 1853dc398b Custom pipeline configuration 2024-01-31 20:17:33 +00:00
reyesj2 86dc7cc804 Kafka init
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
2023-11-29 13:34:25 -05:00
Josh Brower 78db64a419 Auto-managed Fleet Server URLs 2023-07-14 08:40:26 -04:00
Josh Brower c99e7da5a7 Remove Comments 2023-07-11 10:26:18 -04:00
Josh Brower 31edf2e8ea Tighten & Document Pipelines 2023-07-10 14:17:42 -04:00
Josh Brower 8c16feb772 Rename Fleet pipelines 2023-07-09 12:22:55 -04:00
m0duspwnens 2c4eccd7e0 2.4 heavynode changes 2023-06-14 10:40:05 -04:00
Josh Brower 24445cf36a Rename Fleet pipelines 2023-05-16 16:43:21 -04:00
m0duspwnens 7a4fea7a12 fix nginx merge conflicts 2023-05-15 11:40:12 -04:00
m0duspwnens a0ce46e702 enable/disable logstash in ui 2023-05-10 11:16:03 -04:00
Mike Reeves f1c91e91b1 Remove port bindings from logstash config section 2023-05-05 15:10:21 -04:00
m0duspwnens 082704ce1f logstash jinja for ui 2023-05-04 13:07:07 -04:00
Mike Reeves c7604e893e Fix annotations and defaults for logstash 2023-05-03 09:17:37 -04:00
Mike Reeves a38495ce39 Fix pillar top for firewall 2023-05-01 10:40:36 -04:00
Mike Reeves 38629a7676 fix defaults for logstash 2023-05-01 09:55:16 -04:00