CSEC_PUBLIC/securityonion
1
0
Fork 0
mirror of https://github.com/Security-Onion-Solutions/securityonion.git synced 2026-01-09 09:44:17 +01:00
Code Issues Packages Projects Releases Wiki Activity
15,278 Commits 22 Branches 120 Tags
4e50dabc56b23276904bb10cd85e9f5cf5f5d109
Commit Graph

513 Commits

Author SHA1 Message Date
reyesj2
8080e05444 on fresh install kafka nodes pillar may not have populated. Avoiding this by only generating kafka input pipeline when kafka nodes pillar is not empty 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-06-14 14:17:26 -04:00
reyesj2
f372b0907b Use kafka:password for kafka certs 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-06-12 15:41:10 -04:00
reyesj2
e8106befe9 Append '-securityonion' to all Security Onion related Kafka topics. Adjust logstash to ingest all topics ending in '-securityonion' to avoid having to manually list topic names 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-06-12 12:05:16 -04:00
reyesj2
b1beb617b3 Logstash should be disabled when Kafka is enabled except when a minion override exists OR node is a standalone 
- Standalone subscribes to Kafka topics via logstash for ingest

Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-22 13:38:09 -04:00
reyesj2
2ad87bf1fe merge 2.4/dev 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-08 16:30:45 -04:00
reyesj2
eca2a4a9c8 Logstash consumer threads should match topic partition count 
- Default is set to 3. If there are too many consumer threads it may lead to idle logstash worker threads and could require decreasing this value to saturate workers

Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-08 16:17:09 -04:00
m0duspwnens
dcc1f656ee predownload logstash and elastic for new searchnode and heavynode 2024-05-07 10:13:51 -04:00
reyesj2
e960ae66a3 Merge remote-tracking branch 'remotes/origin/2.4/dev' into reyesj2/kafka 2024-05-02 15:12:27 -04:00
reyesj2
3efdb4e532 Reconfigure logstash Kafka input 
- TODO: Configure what topics are pulled to searchnodes via the SOC UI

Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-01 13:01:29 -04:00
Jason Ertel
84db82852c annotation updates for custom settings 2024-04-30 15:14:56 -04:00
reyesj2
af53dcda1b Remove references to kafkanode 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-04-11 15:32:00 -04:00
m0duspwnens
d3bd56b131 disable logstash and redis if kafka enabled 2024-04-10 14:13:27 -04:00
reyesj2
d67ebabc95 Remove logstash output to kafka pipeline. Add additional topics for searchnodes to ingest and add partition/offset info to event 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-04-08 16:38:03 -04:00
reyesj2
65274e89d7 Add client_id to logstash pipeline. To identify which searchnode is pulling messages 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-04-05 15:38:00 -04:00
reyesj2
721e04f793 initial logstash input from kafka over ssl 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-04-05 13:37:14 -04:00
reyesj2
82830c8173 Fix typos and fix error related to elasticsearch saltstate being called from logstash state. Logstash will be removed from kafkanodes in future 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-04-02 16:37:39 -04:00
reyesj2
446f1ffdf5 merge 2.4/dev 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-03-25 13:55:48 -04:00
Wes
1818e134ca Change numbers for Logstash 2024-02-01 14:01:55 +00:00
Wes
136097f981 Custom Logstash pipeline annotations 2024-01-31 21:47:09 +00:00
Wes
4672a5b8eb Custom pipeline configuration in UI 2024-01-31 20:18:17 +00:00
Wes
1853dc398b Custom pipeline configuration 2024-01-31 20:17:33 +00:00
m0duspwnens
33a9ac5701 use logstash nodes for logstash extra_hosts 2023-12-15 15:42:49 -05:00
m0duspwnens
03b2a7d2de change 9805 pipeline to send to self. fix extra_hosts for logstash 2023-12-14 10:01:03 -05:00
reyesj2
8cf29682bb Update to merge in 2.4/dev 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2023-11-29 13:41:23 -05:00
reyesj2
86dc7cc804 Kafka init 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2023-11-29 13:34:25 -05:00
Josh Brower
8c7767b381 Dont overwrite metadata 2023-11-03 08:41:33 -04:00
Wes
4dc64400c5 Support document_id 2023-11-01 13:36:32 +00:00
Jason Ertel
8a751e097d cert path refactor 2023-08-23 14:32:05 -04:00
Josh Brower
2472d6a727 Don't watch certs on search nodes 2023-08-03 18:52:29 -04:00
Josh Brower
1c8a8c460c Restart logstash when certs change 2023-08-02 17:53:29 -04:00
Josh Brower
b6dd347eb8 Heavy Node add manager 2023-07-31 15:22:29 -04:00
Josh Brower
78db64a419 Auto-managed Fleet Server URLs 2023-07-14 08:40:26 -04:00
Josh Brower
c99e7da5a7 Remove Comments 2023-07-11 10:26:18 -04:00
Josh Brower
31edf2e8ea Tighten & Document Pipelines 2023-07-10 14:17:42 -04:00
Josh Brower
7805ca8beb Add Failover Support 2023-07-10 10:38:14 -04:00
Josh Brower
8c16feb772 Rename Fleet pipelines 2023-07-09 12:22:55 -04:00
Josh Brower
e8860a7d2c Fix perms 2023-07-08 09:04:55 -04:00
Josh Brower
beb26596fd Merge remote-tracking branch 'origin/2.4/dev' into 2.4/fleetautogen 2023-07-07 19:12:47 -04:00
Josh Brower
ff3bb11fbb Elastic Fleet Certs Refactor 2023-07-07 16:44:16 -04:00
m0duspwnens
55bed0771b remove so-logstash-get-unparsed, use so-redis-count instead 2023-07-07 09:52:21 -04:00
Mike Reeves
cb8faf7c5f Fix the rest of the analyst entries 2023-06-26 16:14:04 -04:00
m0duspwnens
36272efda7 create ES_LOGSTASH_NODES which removes heavynodes 2023-06-22 09:46:42 -04:00
m0duspwnens
469390696e 2.4 receiver changes 2023-06-15 11:04:16 -04:00
m0duspwnens
2c4eccd7e0 2.4 heavynode changes 2023-06-14 10:40:05 -04:00
m0duspwnens
66dc6274e6 exclude elasticsearch.ca state from fleet and receiver nodes 2023-05-31 15:59:36 -04:00
Mike Reeves
5315c51197 Allow additional docker parameters 2023-05-18 16:52:38 -04:00
Mike Reeves
7ab31e36af Merge branch '2.4/dev' of https://github.com/Security-Onion-Solutions/securityonion into airgaps 2023-05-18 15:19:15 -04:00
Mike Reeves
0fd9fb9294 Allow additional docker parameters 2023-05-18 15:19:09 -04:00
Josh Brower
7d0251952c Filter out uneeded Logstash metadata 2023-05-17 11:06:16 -04:00
Josh Brower
24445cf36a Rename Fleet pipelines 2023-05-16 16:43:21 -04:00