Merge pull request #708 from Yamato-Security/706-change-event-id-statistics-wording-to-metrics

Changed event id statistics wording to metrics

CHANGELOG-Japanese.md

@@ -6,6 +6,8 @@
 
 **改善:**
 
+- EventID解析のオプションをmetricsオプションに変更した。(旧: -s -> 新: -M) (#706) (@hitenkoku)
+
 **バグ修正:**
 
 ## v1.6.0 [2022/09/16]

CHANGELOG.md

@@ -6,6 +6,9 @@
 
 **Enhancements:**
 
+- Changed Event ID Statistics option to Event ID Metrics option. (`-s, --statistics`  -> `-M, --metrics`) (#706) (@hitenkoku)
+  (Note: `statistics_event_info.txt` was changed to `event_id_info.txt`.)
+
 **Bug Fixes:**
 
 ## v1.6.0 [2022/09/16]

README-Japanese.md

@@ -370,7 +370,7 @@ macOSの環境設定から「セキュリティとプライバシー」を開き
 * `--level-tuning`: アラート`level`のカスタムチューニング
 * `-L, --logon-summary`: ログオンイベントのサマリを出力する。
 * `-P, --pivot-keywords-list`: ピボットする不審なキーワードのリスト作成。 
-* `-s, --statistics`: イベントIDに基づくイベントの合計と割合の集計を出力する。
+* `-M, --metrics`: イベントIDに基づくイベントの合計と割合の集計を出力する。
 * `--set-default-profile`: デフォルトプロファイルを変更する。
 * `-u, --update`: GitHubの[hayabusa-rules](https://github.com/Yamato-Security/hayabusa-rules)リポジトリにある最新のルールに同期させる。
 
@@ -418,8 +418,8 @@ OTHER-ACTIONS:
         --contributors                          コントリビュータの一覧表示
     -L, --logon-summary                         成功と失敗したログオン情報の要約を出力する
         --level-tuning [<FILE>]                 ルールlevelのチューニング (デフォルト: ./rules/config/level_tuning.txt)
+    -M, --metrics                               イベントIDの統計情報を表示する
     -p, --pivot-keywords-list                   ピボットキーワードの一覧作成
-    -s, --statistics                            イベントIDの統計情報を表示する
         --set-default-profile <PROFILE>         デフォルトの出力コンフィグを設定する
     -u, --update-rules                          rulesフォルダをhayabusa-rulesのgithubリポジトリの最新版に更新する
 
@@ -509,12 +509,13 @@ hayabusa-1.6.0-win-x64.exe -l -m critical -p -o keywords
 * イベントIDの統計情報を出力する:
 
 ```bash
-hayabusa-1.6.0-win-x64.exe -f Security.evtx -s
+hayabusa-1.6.0-win-x64.exe -f Security.evtx -M
 ```
+
 * ログオンサマリを出力する:
 
 ```bash
-hayabusa-1.6.0-win-x64.exe -L -f Security.evtx -s
+hayabusa-1.6.0-win-x64.exe -L -f Security.evtx -M
 ```
 
 * 詳細なメッセージを出力する(処理に時間がかかるファイル、パースエラー等を特定するのに便利):

README.md

@@ -170,7 +170,7 @@ You can learn how to import CSV files into Timesketch [here](doc/TimesketchImpor
 * Threat hunting based on IoC signatures written in easy to read/create/edit YML based hayabusa rules.
 * Sigma rule support to convert sigma rules to hayabusa rules.
 * Currently it supports the most sigma rules compared to other similar tools and even supports count rules and new aggregators such as `|equalsfield`.
-* Event log statistics. (Useful for getting a picture of what types of events there are and for tuning your log settings.)
+* Event ID metrics. (Useful for getting a picture of what types of events there are and for tuning your log settings.)
 * Rule tuning configuration by excluding unneeded or noisy rules.
 * MITRE ATT&CK mapping of tactics.
 * Rule level tuning.
@@ -361,7 +361,7 @@ You should now be able to run hayabusa.
 * `--level-tuning`: Custom tune the alerts' `level`.
 * `-L, --logon-summary`: Print a summary of logon events.
 * `-P, --pivot-keywords-list`: Print a list of suspicious keywords to pivot on.
-* `-s, --statistics`: Print metrics of the count and percentage of events based on Event ID.
+* `-M, --metrics`: Print metrics of the number and percentage of events based on Event ID.
 * `--set-default-profile`: Change the default profile.
 * `-u, --update`: Sync the rules to the latest rules in the [hayabusa-rules](https://github.com/Yamato-Security/hayabusa-rules) GitHub repository.
 
@@ -409,8 +409,8 @@ OTHER-ACTIONS:
         --contributors                     Print the list of contributors
     -L, --logon-summary                    Print a summary of successful and failed logons
         --level-tuning [<FILE>]            Tune alert levels (default: ./rules/config/level_tuning.txt)
+    -M, --metrics                          Print event ID metrics
     -p, --pivot-keywords-list              Create a list of pivot keywords
-    -s, --statistics                       Print statistics of event IDs
         --set-default-profile <PROFILE>    Set default output profile
     -u, --update-rules                     Update to the latest rules in the hayabusa-rules github repository
 
@@ -497,16 +497,16 @@ hayabusa-1.6.0-win-x64.exe -l -m low
 hayabusa-1.6.0-win-x64.exe -l -m critical -p -o keywords
 ```
 
-* Print Event ID statistics:
+* Print Event ID metrics:
 
 ```bash
-hayabusa-1.6.0-win-x64.exe -f Security.evtx -s
+hayabusa-1.6.0-win-x64.exe -f Security.evtx -M
 ```
 
 * Print logon summary:
 
 ```bash
-hayabusa-1.6.0-win-x64.exe -L -f Security.evtx -s
+hayabusa-1.6.0-win-x64.exe -L -f Security.evtx -M
 ```
 
 * Print verbose information (useful for determining which files take long to process, parsing errors, etc...):

contributors.txt

@@ -2,7 +2,7 @@ Hayabusa was possible thanks to the following people (in alphabetical order):
 
 Akira Nishikawa (@nishikawaakira): Previous lead developer, core hayabusa rule support, etc...
 Fukusuke Takahashi (fukuseket): Static compiling for Windows, race condition and other bug fixes.
-Garigariganzy (@garigariganzy31): Developer, event ID statistics implementation, etc... 
+Garigariganzy (@garigariganzy31): Developer, event ID metrics implementation, etc... 
 ItiB (@itiB_S144) :  Core developer, sigmac hayabusa backend, rule creation, etc...
 James Takai / hachiyone(@hach1yon): Current lead developer, tokio multi-threading, sigma aggregation logic, sigmac backend, rule creation, sigma count implementation etc…
 Kazuminn (@k47_um1n): Core Developer

doc/ElasticStackImport/ElasticStackImport-English.md

@@ -51,7 +51,7 @@ As shown below, click on `Advanced` and perform the following settings before cl
 
 1. Title the `Index name` as `evtxlogs-hayabusa`.
 2. Under `Index settings`, add `, "number_of_replicas": 0` so that the index health status does not turn yellow.
-3. Under `Mappings`, change the `RuleTitle` type of `text` to `keyword` so that we can do statistics on the rule titles and change the `EventID` type of `long` to `keyword` in order to import without errors.
+3. Under `Mappings`, change the `RuleTitle` type of `text` to `keyword` so that we can calculate metrics on the rule titles and change the `EventID` type of `long` to `keyword` in order to import without errors.
 4. Under `Ingest pipeline`, add `, "field": "Timestamp"` under the `remove` section. Timestamps will be displayed as `@timestamp` so this duplicate field is not needed. Also, delete the following in order to import without errors:
    ```
     {

src/detections/configs.rs

@@ -188,9 +188,9 @@ pub struct Config {
     #[clap(help_heading = Some("ADVANCED"), short, long = "thread-number", value_name = "NUMBER")]
     pub thread_number: Option<usize>,
 
-    /// Print statistics of event IDs
-    #[clap(help_heading = Some("OTHER-ACTIONS"), short, long)]
-    pub statistics: bool,
+    /// Print event ID metrics
+    #[clap(help_heading = Some("OTHER-ACTIONS"), short='M', long)]
+    pub metrics: bool,
 
     /// Print a summary of successful and failed logons
     #[clap(help_heading = Some("OTHER-ACTIONS"), short = 'L', long = "logon-summary")]
@@ -266,11 +266,11 @@ impl ConfigReader<'_> {
             args: parse.clone(),
             headless_help: String::default(),
             event_timeline_config: load_eventcode_info(
-                utils::check_setting_path(&parse.config, "statistics_event_info.txt", false)
+                utils::check_setting_path(&parse.config, "event_id_info.txt", false)
                     .unwrap_or_else(|| {
                         utils::check_setting_path(
                             &CURRENT_EXE_PATH.to_path_buf(),
-                            "rules/config/statistics_event_info.txt",
+                            "rules/config/event_id_info.txt",
                             true,
                         )
                         .unwrap()
@@ -581,7 +581,7 @@ fn load_eventcode_info(path: &str) -> EventInfoConfig {
         return config;
     }
 
-    // statistics_event_infoが読み込めなかったらエラーで終了とする。
+    // event_id_info.txtが読み込めなかったらエラーで終了とする。
     read_result.unwrap().into_iter().for_each(|line| {
         if line.len() != 2 {
             return;

src/detections/detection.rs

@@ -14,7 +14,7 @@ use crate::detections::message::DetectInfo;
 use crate::detections::message::ERROR_LOG_STACK;
 use crate::detections::message::{CH_CONFIG, DEFAULT_DETAILS, TAGS_CONFIG};
 use crate::detections::message::{
-    LOGONSUMMARY_FLAG, PIVOT_KEYWORD_LIST_FLAG, QUIET_ERRORS_FLAG, STATISTICS_FLAG,
+    LOGONSUMMARY_FLAG, METRICS_FLAG, PIVOT_KEYWORD_LIST_FLAG, QUIET_ERRORS_FLAG,
 };
 use crate::detections::pivot::insert_pivot_keyword;
 use crate::detections::rule;
@@ -599,7 +599,7 @@ impl Detection {
         st_rc: &HashMap<String, u128>,
         err_rc: &u128,
     ) {
-        if *STATISTICS_FLAG {
+        if *METRICS_FLAG {
             return;
         }
         let mut sorted_ld_rc: Vec<(&String, &u128)> = ld_rc.iter().collect();

src/detections/message.rs

@@ -46,7 +46,7 @@ lazy_static! {
     );
     pub static ref QUIET_ERRORS_FLAG: bool = configs::CONFIG.read().unwrap().args.quiet_errors;
     pub static ref ERROR_LOG_STACK: Mutex<Vec<String>> = Mutex::new(Vec::new());
-    pub static ref STATISTICS_FLAG: bool = configs::CONFIG.read().unwrap().args.statistics;
+    pub static ref METRICS_FLAG: bool = configs::CONFIG.read().unwrap().args.metrics;
     pub static ref LOGONSUMMARY_FLAG: bool = configs::CONFIG.read().unwrap().args.logon_summary;
     pub static ref TAGS_CONFIG: HashMap<String, String> = create_output_filter_config(
         utils::check_setting_path(&CURRENT_EXE_PATH.to_path_buf(), "config/mitre_tactics.txt", true)

src/detections/utils.rs

@@ -410,7 +410,7 @@ pub fn check_rule_config() -> Result<(), String> {
         "target_event_IDs.txt",
         "default_details.txt",
         "level_tuning.txt",
-        "statistics_event_info.txt",
+        "event_id_info.txt",
         "eventkey_alias.txt",
     ];
     let mut not_exist_file = vec![];

src/main.rs

@@ -11,8 +11,8 @@ use hayabusa::detections::configs::{load_pivot_keywords, TargetEventTime, TARGET
 use hayabusa::detections::configs::{CONFIG, CURRENT_EXE_PATH};
 use hayabusa::detections::detection::{self, EvtxRecordInfo};
 use hayabusa::detections::message::{
-    AlertMessage, ERROR_LOG_PATH, ERROR_LOG_STACK, LOGONSUMMARY_FLAG, PIVOT_KEYWORD_LIST_FLAG,
-    QUIET_ERRORS_FLAG, STATISTICS_FLAG,
+    AlertMessage, ERROR_LOG_PATH, ERROR_LOG_STACK, LOGONSUMMARY_FLAG, METRICS_FLAG,
+    PIVOT_KEYWORD_LIST_FLAG, QUIET_ERRORS_FLAG,
 };
 use hayabusa::detections::pivot::PivotKeyword;
 use hayabusa::detections::pivot::PIVOT_KEYWORD;
@@ -193,11 +193,11 @@ impl App {
             return;
         }
 
-        if *STATISTICS_FLAG {
+        if *METRICS_FLAG {
             write_color_buffer(
                 &BufferWriter::stdout(ColorChoice::Always),
                 None,
-                "Generating Event ID Statistics",
+                "Generating Event ID Metrics",
                 true,
             )
             .ok();
@@ -565,7 +565,7 @@ impl App {
         }
         println!();
         detection.add_aggcondition_msges(&self.rt);
-        if !(*STATISTICS_FLAG || *LOGONSUMMARY_FLAG || *PIVOT_KEYWORD_LIST_FLAG) {
+        if !(*METRICS_FLAG || *LOGONSUMMARY_FLAG || *PIVOT_KEYWORD_LIST_FLAG) {
             after_fact(total_records);
         }
     }
@@ -647,7 +647,7 @@ impl App {
             // timeline機能の実行
             tl.start(&records_per_detect);
 
-            if !(*STATISTICS_FLAG || *LOGONSUMMARY_FLAG) {
+            if !(*METRICS_FLAG || *LOGONSUMMARY_FLAG) {
                 // ruleファイルの検知
                 detection = detection.start(&self.rt, records_per_detect);
             }

src/timeline/metrics.rs

@@ -1,9 +1,9 @@
-use crate::detections::message::{LOGONSUMMARY_FLAG, STATISTICS_FLAG};
+use crate::detections::message::{LOGONSUMMARY_FLAG, METRICS_FLAG};
 use crate::detections::{detection::EvtxRecordInfo, utils};
 use hashbrown::HashMap;
 
 #[derive(Debug)]
-pub struct EventStatistics {
+pub struct EventMetrics {
     pub total: usize,
     pub filepath: String,
     pub start_time: String,
@@ -14,7 +14,7 @@ pub struct EventStatistics {
 /**
 * Windows Event Logの統計情報を出力する
 */
-impl EventStatistics {
+impl EventMetrics {
     pub fn new(
         total: usize,
         filepath: String,
@@ -22,8 +22,8 @@ impl EventStatistics {
         end_time: String,
         stats_list: HashMap<String, usize>,
         stats_login_list: HashMap<String, [usize; 2]>,
-    ) -> EventStatistics {
-        EventStatistics {
+    ) -> EventMetrics {
+        EventMetrics {
             total,
             filepath,
             start_time,
@@ -34,8 +34,8 @@ impl EventStatistics {
     }
 
     pub fn evt_stats_start(&mut self, records: &[EvtxRecordInfo]) {
-        // 引数でstatisticsオプションが指定されている時だけ、統計情報を出力する。
-        if !*STATISTICS_FLAG {
+        // 引数でmetricsオプションが指定されている時だけ、統計情報を出力する。
+        if !*METRICS_FLAG {
             return;
         }
 

src/timeline/mod.rs

@@ -1,2 +1,2 @@
-pub mod statistics;
+pub mod metrics;
 pub mod timelines;

src/timeline/timelines.rs

@@ -1,13 +1,13 @@
-use crate::detections::message::{LOGONSUMMARY_FLAG, STATISTICS_FLAG};
+use crate::detections::message::{LOGONSUMMARY_FLAG, METRICS_FLAG};
 use crate::detections::{configs::CONFIG, detection::EvtxRecordInfo};
 use prettytable::{Cell, Row, Table};
 
-use super::statistics::EventStatistics;
+use super::metrics::EventMetrics;
 use hashbrown::HashMap;
 
 #[derive(Debug)]
 pub struct Timeline {
-    pub stats: EventStatistics,
+    pub stats: EventMetrics,
 }
 
 impl Default for Timeline {
@@ -26,7 +26,7 @@ impl Timeline {
         let statsloginlst = HashMap::new();
 
         let statistic =
-            EventStatistics::new(totalcnt, filepath, starttm, endtm, statslst, statsloginlst);
+            EventMetrics::new(totalcnt, filepath, starttm, endtm, statslst, statsloginlst);
         Timeline { stats: statistic }
     }
 
@@ -36,7 +36,7 @@ impl Timeline {
     }
 
     pub fn tm_stats_dsp_msg(&mut self) {
-        if !*STATISTICS_FLAG {
+        if !*METRICS_FLAG {
             return;
         }
         // 出力メッセージ作成
@@ -98,7 +98,7 @@ impl Timeline {
                 .event_timeline_config
                 .get_event_id(*event_id)
                 .is_some();
-            // statistics_event_info.txtに登録あるものは情報設定
+            // event_id_info.txtに登録あるものは情報設定
             if conf {
                 // 出力メッセージ1行作成
                 msges.push(format!(