CSEC_PUBLIC/securityonion
1
0
Fork 0
mirror of https://github.com/Security-Onion-Solutions/securityonion.git synced 2025-12-07 09:42:46 +01:00
Code Issues Packages Projects Releases Wiki Activity
15,002 Commits 24 Branches 119 Tags
949cea95f4428f259fbaac39f02b6f22d1b14ef5
Commit Graph

15002 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
reyesj2
949cea95f4 Update pillarWatch config for global.pipeline 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-29 23:19:44 -04:00
reyesj2
386be4e746 WIP: Manage Kafka nodes pillar role value 
This way when kafka_controllers is updated the pillar value gets updated and any non-controllers get updated to revert to 'broker' only role.
 Needs more testing when a new controller joins in this manner Kafka errors due to cluster metadata being out of sync. One solution is to remove /nsm/kafka/data/__cluster_metadata-0/quorum-state and restart cluster. Alternative is working with Kafka cli tools to inform cluster of new voter, likely best option but requires a wrapper script of some sort to be created for updating cluster in-place.
Easiest option is to have all receivers join grid and then configure Kafka with specific controllers via SOC UI prior to enabling Kafka. This way Kafka cluster comes up in the desired configuration with no need for immediately modifying cluster

Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-29 16:48:39 -04:00
reyesj2
d9ec556061 Update some annotations and defaults 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-29 16:41:02 -04:00
reyesj2
876d860488 elastic agent should be able to communicate over 9092 for sending logs to kafka brokers 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-29 16:40:15 -04:00
reyesj2
59097070ef Revert "Remove unneeded jolokia aggregate metrics to reduce data ingested to influx" 
This reverts commit 1c1a1a1d3f.
 2024-05-28 12:17:43 -04:00
reyesj2
77b5aa4369 Correct dashboard name 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-28 11:34:35 -04:00
reyesj2
0d7c331ff0 only show specific fields when hovering over Kafka influxdb panels 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-28 11:29:38 -04:00
reyesj2
1c1a1a1d3f Remove unneeded jolokia aggregate metrics to reduce data ingested to influx 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-28 11:14:19 -04:00
reyesj2
47efcfd6e2 Add basic Kafka metrics to 'Security Onion Performance' influxdb dashboard 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-28 10:55:11 -04:00
reyesj2
15a0b959aa Add jolokia metrics for influxdb dashboard 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-28 10:51:39 -04:00
reyesj2
fcb6a47e8c Remove redis.sh telegraf script when Kafka is global pipeline 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-26 21:10:41 -04:00
reyesj2
382cd24a57 Small changes needed for using new Kafka docker image + added Kafka logging output to /opt/so/log/kafka/ 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-22 13:39:21 -04:00
reyesj2
b1beb617b3 Logstash should be disabled when Kafka is enabled except when a minion override exists OR node is a standalone 
- Standalone subscribes to Kafka topics via logstash for ingest

Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-22 13:38:09 -04:00
reyesj2
91f8b1fef7 Set default replication factor back to Kafka default 
If replication factor is > 1 Kafka will fail to start until another broker is added
  - For internal automated testing purposes a Standalone will be utilized

Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-22 13:35:09 -04:00
reyesj2
2ad87bf1fe merge 2.4/dev 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-08 16:30:45 -04:00
reyesj2
eca2a4a9c8 Logstash consumer threads should match topic partition count 
- Default is set to 3. If there are too many consumer threads it may lead to idle logstash worker threads and could require decreasing this value to saturate workers

Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-08 16:17:09 -04:00
reyesj2
dff609d829 Add basic read-only metric collection from Kafka 
Signed-off-by: reyesj2 <94730068+reyesj2@users.noreply.github.com>
 2024-05-08 16:13:09 -04:00
weslambert
b916465b06 Merge pull request #12974 from Security-Onion-Solutions/fix/strelka_yara 
Account for 0 active rules and change watch
 2024-05-08 15:59:20 -04:00
weslambert
0567b93534 Remove mode 2024-05-08 15:39:59 -04:00
Wes
77e2117051 Account for 0 active rules and change watch 2024-05-08 18:47:52 +00:00
Doug Burks
c7845bdf56 Merge pull request #12970 from Security-Onion-Solutions/dougburks-patch-1 
FIX: Adjust so-import-pcap so that suricata works when it is pcapengine #12969
 2024-05-08 13:28:05 -04:00
Doug Burks
5a5a1e86ac FIX: Adjust so-import-pcap so that suricata works when it is pcapengine #12969 2024-05-08 13:26:36 -04:00
Josh Patterson
796eefc2f0 Merge pull request #12965 from Security-Onion-Solutions/orchit 
searchnode installation improvements
 2024-05-08 10:24:33 -04:00
m0duspwnens
1862deaf5e add copyright 2024-05-08 10:14:08 -04:00
m0duspwnens
0d2e5e0065 need repo and docker first 2024-05-08 09:50:01 -04:00
m0duspwnens
5dc098f0fc remove test file 2024-05-08 08:54:24 -04:00
Mike Reeves
af681881e6 Merge pull request #12963 from Security-Onion-Solutions/TOoSmOotH-patch-4 
Make the url list read only
 2024-05-08 08:45:34 -04:00
Josh Brower
47dc911b79 Merge pull request #12964 from Security-Onion-Solutions/2.4/agstrelka 
remove old yara airgap code
 2024-05-08 08:45:16 -04:00
DefensiveDepth
6d2ecce9b7 remove old yara airgap code 2024-05-08 08:43:37 -04:00
Mike Reeves
326c59bb26 Update soc_idstools.yaml 2024-05-08 08:42:38 -04:00
Mike Reeves
c1257f1c13 Merge pull request #12961 from Security-Onion-Solutions/TOoSmOotH-patch-3 
Change so soc writes urls as a list
 2024-05-07 17:23:12 -04:00
Mike Reeves
2eee617788 Update soc_idstools.yaml 2024-05-07 17:21:01 -04:00
Jason Ertel
70ef8092a7 Merge pull request #12959 from Security-Onion-Solutions/jertel/testcy 
update suri regex for testing
 2024-05-07 11:37:31 -07:00
Jason Ertel
8364b2a730 update for testing 2024-05-07 14:30:52 -04:00
coreyogburn
cb7dea1295 Merge pull request #12957 from Security-Onion-Solutions/cogburn/retry-import 
Specify Error Retry Wait and Error Limit for All Detection Engines
 2024-05-07 11:20:26 -06:00
Corey Ogburn
1da88b70ac Specify Error Retry Wait and Error Limit for All Detection Engines 
If a sync errors out, the engine will wait `communityRulesImportErrorSeconds` seconds instead of the usual `communityRulesImportFrequencySeconds` seconds wait.

If `failAfterConsecutiveErrorCount` errors happen in a row when syncing detections to ElasticSearch then the sync is considered a failure and will give up and try again later. This assumes ElasticSearch is the source of the errors and backs of in hopes it'll be able to fix itself.
 2024-05-07 10:34:50 -06:00
Jason Ertel
b4817fa062 Merge pull request #12956 from Security-Onion-Solutions/jertel/testcy 
test regexes for detections
 2024-05-07 08:45:38 -07:00
weslambert
bc24227732 Merge pull request #12955 from Security-Onion-Solutions/fix/cef 
Add CEF
 2024-05-07 11:23:53 -04:00
weslambert
2e70d157e2 Add ref 2024-05-07 11:13:51 -04:00
m0duspwnens
5e2e5b2724 Merge remote-tracking branch 'origin/2.4/dev' into orchit 2024-05-07 10:44:14 -04:00
m0duspwnens
dcc1f656ee predownload logstash and elastic for new searchnode and heavynode 2024-05-07 10:13:51 -04:00
weslambert
23da1f6ee9 Merge pull request #12951 from Security-Onion-Solutions/fix/remove_watch 
Remove watch
 2024-05-07 09:23:56 -04:00
Wes
bee8c2c1ce Remove watch 2024-05-07 13:21:59 +00:00
Jason Ertel
4ebe070cd8 test regexes for detections 2024-05-06 19:03:12 -04:00
weslambert
a5e89c0854 Merge pull request #12947 from Security-Onion-Solutions/fix/strelka_yara_distributed 
Fix YARA rules for distributed deployments
 2024-05-06 15:53:08 -04:00
weslambert
a25e43db8f Merge pull request #12948 from Security-Onion-Solutions/fix/strelka_yara_watch 
Restart Strelka backend when YARA rules change
 2024-05-06 15:52:57 -04:00
Josh Brower
b997e44715 Merge pull request #12939 from Security-Onion-Solutions/2.4/detections-airgap 
Initial airgap support for detections
 2024-05-06 15:46:29 -04:00
Wes
1e48955376 Restart when rules change 2024-05-06 19:39:03 +00:00
Wes
5056ec526b Add compiled directory 2024-05-06 19:27:38 +00:00
m0duspwnens
2431d7b028 Merge branch '2.4/detections-airgap' of https://github.com/Security-Onion-Solutions/securityonion into 2.4/detections-airgap 2024-05-06 15:27:27 -04:00