diff --git a/CHANGELOG-Japanese.md b/CHANGELOG-Japanese.md index 296b665b..ca7a6c4e 100644 --- a/CHANGELOG-Japanese.md +++ b/CHANGELOG-Japanese.md @@ -6,6 +6,8 @@ **改善:** +- EventID解析のオプションをmetricsオプションに変更した。(旧: -s -> 新: -M) (#706) (@hitenkoku) + **バグ修正:** ## v1.6.0 [2022/09/16] diff --git a/CHANGELOG.md b/CHANGELOG.md index 52eb2549..abb4731a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,9 @@ **Enhancements:** +- Changed Event ID Statistics option to Event ID Metrics option. (`-s, --statistics` -> `-M, --metrics`) (#706) (@hitenkoku) + (Note: `statistics_event_info.txt` was changed to `event_id_info.txt`.) + **Bug Fixes:** ## v1.6.0 [2022/09/16] diff --git a/README-Japanese.md b/README-Japanese.md index afef4813..bbd1ab02 100644 --- a/README-Japanese.md +++ b/README-Japanese.md @@ -370,7 +370,7 @@ macOSの環境設定から「セキュリティとプライバシー」を開き * `--level-tuning`: アラート`level`のカスタムチューニング * `-L, --logon-summary`: ログオンイベントのサマリを出力する。 * `-P, --pivot-keywords-list`: ピボットする不審なキーワードのリスト作成。 -* `-s, --statistics`: イベントIDに基づくイベントの合計と割合の集計を出力する。 +* `-M, --metrics`: イベントIDに基づくイベントの合計と割合の集計を出力する。 * `--set-default-profile`: デフォルトプロファイルを変更する。 * `-u, --update`: GitHubの[hayabusa-rules](https://github.com/Yamato-Security/hayabusa-rules)リポジトリにある最新のルールに同期させる。 @@ -418,8 +418,8 @@ OTHER-ACTIONS: --contributors コントリビュータの一覧表示 -L, --logon-summary 成功と失敗したログオン情報の要約を出力する --level-tuning [] ルールlevelのチューニング (デフォルト: ./rules/config/level_tuning.txt) + -M, --metrics イベントIDの統計情報を表示する -p, --pivot-keywords-list ピボットキーワードの一覧作成 - -s, --statistics イベントIDの統計情報を表示する --set-default-profile デフォルトの出力コンフィグを設定する -u, --update-rules rulesフォルダをhayabusa-rulesのgithubリポジトリの最新版に更新する @@ -509,12 +509,13 @@ hayabusa-1.6.0-win-x64.exe -l -m critical -p -o keywords * イベントIDの統計情報を出力する: ```bash -hayabusa-1.6.0-win-x64.exe -f Security.evtx -s +hayabusa-1.6.0-win-x64.exe -f Security.evtx -M ``` + * ログオンサマリを出力する: ```bash -hayabusa-1.6.0-win-x64.exe -L -f Security.evtx -s +hayabusa-1.6.0-win-x64.exe -L -f Security.evtx -M ``` * 詳細なメッセージを出力する(処理に時間がかかるファイル、パースエラー等を特定するのに便利): diff --git a/README.md b/README.md index 0bc30352..d0174604 100644 --- a/README.md +++ b/README.md @@ -170,7 +170,7 @@ You can learn how to import CSV files into Timesketch [here](doc/TimesketchImpor * Threat hunting based on IoC signatures written in easy to read/create/edit YML based hayabusa rules. * Sigma rule support to convert sigma rules to hayabusa rules. * Currently it supports the most sigma rules compared to other similar tools and even supports count rules and new aggregators such as `|equalsfield`. -* Event log statistics. (Useful for getting a picture of what types of events there are and for tuning your log settings.) +* Event ID metrics. (Useful for getting a picture of what types of events there are and for tuning your log settings.) * Rule tuning configuration by excluding unneeded or noisy rules. * MITRE ATT&CK mapping of tactics. * Rule level tuning. @@ -361,7 +361,7 @@ You should now be able to run hayabusa. * `--level-tuning`: Custom tune the alerts' `level`. * `-L, --logon-summary`: Print a summary of logon events. * `-P, --pivot-keywords-list`: Print a list of suspicious keywords to pivot on. -* `-s, --statistics`: Print metrics of the count and percentage of events based on Event ID. +* `-M, --metrics`: Print metrics of the number and percentage of events based on Event ID. * `--set-default-profile`: Change the default profile. * `-u, --update`: Sync the rules to the latest rules in the [hayabusa-rules](https://github.com/Yamato-Security/hayabusa-rules) GitHub repository. @@ -409,8 +409,8 @@ OTHER-ACTIONS: --contributors Print the list of contributors -L, --logon-summary Print a summary of successful and failed logons --level-tuning [] Tune alert levels (default: ./rules/config/level_tuning.txt) + -M, --metrics Print event ID metrics -p, --pivot-keywords-list Create a list of pivot keywords - -s, --statistics Print statistics of event IDs --set-default-profile Set default output profile -u, --update-rules Update to the latest rules in the hayabusa-rules github repository @@ -497,16 +497,16 @@ hayabusa-1.6.0-win-x64.exe -l -m low hayabusa-1.6.0-win-x64.exe -l -m critical -p -o keywords ``` -* Print Event ID statistics: +* Print Event ID metrics: ```bash -hayabusa-1.6.0-win-x64.exe -f Security.evtx -s +hayabusa-1.6.0-win-x64.exe -f Security.evtx -M ``` * Print logon summary: ```bash -hayabusa-1.6.0-win-x64.exe -L -f Security.evtx -s +hayabusa-1.6.0-win-x64.exe -L -f Security.evtx -M ``` * Print verbose information (useful for determining which files take long to process, parsing errors, etc...): diff --git a/contributors.txt b/contributors.txt index dd3e8a57..e61924a1 100644 --- a/contributors.txt +++ b/contributors.txt @@ -2,7 +2,7 @@ Hayabusa was possible thanks to the following people (in alphabetical order): Akira Nishikawa (@nishikawaakira): Previous lead developer, core hayabusa rule support, etc... Fukusuke Takahashi (fukuseket): Static compiling for Windows, race condition and other bug fixes. -Garigariganzy (@garigariganzy31): Developer, event ID statistics implementation, etc... +Garigariganzy (@garigariganzy31): Developer, event ID metrics implementation, etc... ItiB (@itiB_S144) : Core developer, sigmac hayabusa backend, rule creation, etc... James Takai / hachiyone(@hach1yon): Current lead developer, tokio multi-threading, sigma aggregation logic, sigmac backend, rule creation, sigma count implementation etc… Kazuminn (@k47_um1n): Core Developer diff --git a/doc/ElasticStackImport/ElasticStackImport-English.md b/doc/ElasticStackImport/ElasticStackImport-English.md index 4a8c96a3..7a3219e9 100644 --- a/doc/ElasticStackImport/ElasticStackImport-English.md +++ b/doc/ElasticStackImport/ElasticStackImport-English.md @@ -51,7 +51,7 @@ As shown below, click on `Advanced` and perform the following settings before cl 1. Title the `Index name` as `evtxlogs-hayabusa`. 2. Under `Index settings`, add `, "number_of_replicas": 0` so that the index health status does not turn yellow. -3. Under `Mappings`, change the `RuleTitle` type of `text` to `keyword` so that we can do statistics on the rule titles and change the `EventID` type of `long` to `keyword` in order to import without errors. +3. Under `Mappings`, change the `RuleTitle` type of `text` to `keyword` so that we can calculate metrics on the rule titles and change the `EventID` type of `long` to `keyword` in order to import without errors. 4. Under `Ingest pipeline`, add `, "field": "Timestamp"` under the `remove` section. Timestamps will be displayed as `@timestamp` so this duplicate field is not needed. Also, delete the following in order to import without errors: ``` { diff --git a/rules b/rules index fe99c87c..2b0f88d1 160000 --- a/rules +++ b/rules @@ -1 +1 @@ -Subproject commit fe99c87c886ca5b66b5e67242eeddfacc469d420 +Subproject commit 2b0f88d1c09b5b9979b99686a29a244993508210 diff --git a/src/detections/configs.rs b/src/detections/configs.rs index f1849a65..0bd52d60 100644 --- a/src/detections/configs.rs +++ b/src/detections/configs.rs @@ -188,9 +188,9 @@ pub struct Config { #[clap(help_heading = Some("ADVANCED"), short, long = "thread-number", value_name = "NUMBER")] pub thread_number: Option, - /// Print statistics of event IDs - #[clap(help_heading = Some("OTHER-ACTIONS"), short, long)] - pub statistics: bool, + /// Print event ID metrics + #[clap(help_heading = Some("OTHER-ACTIONS"), short='M', long)] + pub metrics: bool, /// Print a summary of successful and failed logons #[clap(help_heading = Some("OTHER-ACTIONS"), short = 'L', long = "logon-summary")] @@ -266,11 +266,11 @@ impl ConfigReader<'_> { args: parse.clone(), headless_help: String::default(), event_timeline_config: load_eventcode_info( - utils::check_setting_path(&parse.config, "statistics_event_info.txt", false) + utils::check_setting_path(&parse.config, "event_id_info.txt", false) .unwrap_or_else(|| { utils::check_setting_path( &CURRENT_EXE_PATH.to_path_buf(), - "rules/config/statistics_event_info.txt", + "rules/config/event_id_info.txt", true, ) .unwrap() @@ -581,7 +581,7 @@ fn load_eventcode_info(path: &str) -> EventInfoConfig { return config; } - // statistics_event_infoが読み込めなかったらエラーで終了とする。 + // event_id_info.txtが読み込めなかったらエラーで終了とする。 read_result.unwrap().into_iter().for_each(|line| { if line.len() != 2 { return; diff --git a/src/detections/detection.rs b/src/detections/detection.rs index 507a3ec8..ef428b6f 100644 --- a/src/detections/detection.rs +++ b/src/detections/detection.rs @@ -14,7 +14,7 @@ use crate::detections::message::DetectInfo; use crate::detections::message::ERROR_LOG_STACK; use crate::detections::message::{CH_CONFIG, DEFAULT_DETAILS, TAGS_CONFIG}; use crate::detections::message::{ - LOGONSUMMARY_FLAG, PIVOT_KEYWORD_LIST_FLAG, QUIET_ERRORS_FLAG, STATISTICS_FLAG, + LOGONSUMMARY_FLAG, METRICS_FLAG, PIVOT_KEYWORD_LIST_FLAG, QUIET_ERRORS_FLAG, }; use crate::detections::pivot::insert_pivot_keyword; use crate::detections::rule; @@ -599,7 +599,7 @@ impl Detection { st_rc: &HashMap, err_rc: &u128, ) { - if *STATISTICS_FLAG { + if *METRICS_FLAG { return; } let mut sorted_ld_rc: Vec<(&String, &u128)> = ld_rc.iter().collect(); diff --git a/src/detections/message.rs b/src/detections/message.rs index 74a48783..9f46e0bf 100644 --- a/src/detections/message.rs +++ b/src/detections/message.rs @@ -46,7 +46,7 @@ lazy_static! { ); pub static ref QUIET_ERRORS_FLAG: bool = configs::CONFIG.read().unwrap().args.quiet_errors; pub static ref ERROR_LOG_STACK: Mutex> = Mutex::new(Vec::new()); - pub static ref STATISTICS_FLAG: bool = configs::CONFIG.read().unwrap().args.statistics; + pub static ref METRICS_FLAG: bool = configs::CONFIG.read().unwrap().args.metrics; pub static ref LOGONSUMMARY_FLAG: bool = configs::CONFIG.read().unwrap().args.logon_summary; pub static ref TAGS_CONFIG: HashMap = create_output_filter_config( utils::check_setting_path(&CURRENT_EXE_PATH.to_path_buf(), "config/mitre_tactics.txt", true) diff --git a/src/detections/utils.rs b/src/detections/utils.rs index 837da55d..259dc32b 100644 --- a/src/detections/utils.rs +++ b/src/detections/utils.rs @@ -410,7 +410,7 @@ pub fn check_rule_config() -> Result<(), String> { "target_event_IDs.txt", "default_details.txt", "level_tuning.txt", - "statistics_event_info.txt", + "event_id_info.txt", "eventkey_alias.txt", ]; let mut not_exist_file = vec![]; diff --git a/src/main.rs b/src/main.rs index 5da1b162..c4454714 100644 --- a/src/main.rs +++ b/src/main.rs @@ -11,8 +11,8 @@ use hayabusa::detections::configs::{load_pivot_keywords, TargetEventTime, TARGET use hayabusa::detections::configs::{CONFIG, CURRENT_EXE_PATH}; use hayabusa::detections::detection::{self, EvtxRecordInfo}; use hayabusa::detections::message::{ - AlertMessage, ERROR_LOG_PATH, ERROR_LOG_STACK, LOGONSUMMARY_FLAG, PIVOT_KEYWORD_LIST_FLAG, - QUIET_ERRORS_FLAG, STATISTICS_FLAG, + AlertMessage, ERROR_LOG_PATH, ERROR_LOG_STACK, LOGONSUMMARY_FLAG, METRICS_FLAG, + PIVOT_KEYWORD_LIST_FLAG, QUIET_ERRORS_FLAG, }; use hayabusa::detections::pivot::PivotKeyword; use hayabusa::detections::pivot::PIVOT_KEYWORD; @@ -193,11 +193,11 @@ impl App { return; } - if *STATISTICS_FLAG { + if *METRICS_FLAG { write_color_buffer( &BufferWriter::stdout(ColorChoice::Always), None, - "Generating Event ID Statistics", + "Generating Event ID Metrics", true, ) .ok(); @@ -565,7 +565,7 @@ impl App { } println!(); detection.add_aggcondition_msges(&self.rt); - if !(*STATISTICS_FLAG || *LOGONSUMMARY_FLAG || *PIVOT_KEYWORD_LIST_FLAG) { + if !(*METRICS_FLAG || *LOGONSUMMARY_FLAG || *PIVOT_KEYWORD_LIST_FLAG) { after_fact(total_records); } } @@ -647,7 +647,7 @@ impl App { // timeline機能の実行 tl.start(&records_per_detect); - if !(*STATISTICS_FLAG || *LOGONSUMMARY_FLAG) { + if !(*METRICS_FLAG || *LOGONSUMMARY_FLAG) { // ruleファイルの検知 detection = detection.start(&self.rt, records_per_detect); } diff --git a/src/timeline/statistics.rs b/src/timeline/metrics.rs similarity index 93% rename from src/timeline/statistics.rs rename to src/timeline/metrics.rs index 6e6982e1..d04e2ddd 100644 --- a/src/timeline/statistics.rs +++ b/src/timeline/metrics.rs @@ -1,9 +1,9 @@ -use crate::detections::message::{LOGONSUMMARY_FLAG, STATISTICS_FLAG}; +use crate::detections::message::{LOGONSUMMARY_FLAG, METRICS_FLAG}; use crate::detections::{detection::EvtxRecordInfo, utils}; use hashbrown::HashMap; #[derive(Debug)] -pub struct EventStatistics { +pub struct EventMetrics { pub total: usize, pub filepath: String, pub start_time: String, @@ -14,7 +14,7 @@ pub struct EventStatistics { /** * Windows Event Logの統計情報を出力する */ -impl EventStatistics { +impl EventMetrics { pub fn new( total: usize, filepath: String, @@ -22,8 +22,8 @@ impl EventStatistics { end_time: String, stats_list: HashMap, stats_login_list: HashMap, - ) -> EventStatistics { - EventStatistics { + ) -> EventMetrics { + EventMetrics { total, filepath, start_time, @@ -34,8 +34,8 @@ impl EventStatistics { } pub fn evt_stats_start(&mut self, records: &[EvtxRecordInfo]) { - // 引数でstatisticsオプションが指定されている時だけ、統計情報を出力する。 - if !*STATISTICS_FLAG { + // 引数でmetricsオプションが指定されている時だけ、統計情報を出力する。 + if !*METRICS_FLAG { return; } diff --git a/src/timeline/mod.rs b/src/timeline/mod.rs index c6200b52..7557f9f8 100644 --- a/src/timeline/mod.rs +++ b/src/timeline/mod.rs @@ -1,2 +1,2 @@ -pub mod statistics; +pub mod metrics; pub mod timelines; diff --git a/src/timeline/timelines.rs b/src/timeline/timelines.rs index a0cad83a..751643cd 100644 --- a/src/timeline/timelines.rs +++ b/src/timeline/timelines.rs @@ -1,13 +1,13 @@ -use crate::detections::message::{LOGONSUMMARY_FLAG, STATISTICS_FLAG}; +use crate::detections::message::{LOGONSUMMARY_FLAG, METRICS_FLAG}; use crate::detections::{configs::CONFIG, detection::EvtxRecordInfo}; use prettytable::{Cell, Row, Table}; -use super::statistics::EventStatistics; +use super::metrics::EventMetrics; use hashbrown::HashMap; #[derive(Debug)] pub struct Timeline { - pub stats: EventStatistics, + pub stats: EventMetrics, } impl Default for Timeline { @@ -26,7 +26,7 @@ impl Timeline { let statsloginlst = HashMap::new(); let statistic = - EventStatistics::new(totalcnt, filepath, starttm, endtm, statslst, statsloginlst); + EventMetrics::new(totalcnt, filepath, starttm, endtm, statslst, statsloginlst); Timeline { stats: statistic } } @@ -36,7 +36,7 @@ impl Timeline { } pub fn tm_stats_dsp_msg(&mut self) { - if !*STATISTICS_FLAG { + if !*METRICS_FLAG { return; } // 出力メッセージ作成 @@ -98,7 +98,7 @@ impl Timeline { .event_timeline_config .get_event_id(*event_id) .is_some(); - // statistics_event_info.txtに登録あるものは情報設定 + // event_id_info.txtに登録あるものは情報設定 if conf { // 出力メッセージ1行作成 msges.push(format!(