most detections summary by date (#551)

* added Date with most detections by level #550

* cargo fmt

* updated changelog #550

* updated readme #550

* removed  most undefined detections date in summary #550

* cargo fmt

* add space after level tuning

* changed undefined rule detection count to no show #550

* cargo fmt

* readme update

* channel abb update

* channel abb update

* readme update

Co-authored-by: Tanaka Zakku <71482215+YamatoSecurity@users.noreply.github.com>

CHANGELOG-Japanese.md

@@ -7,6 +7,7 @@
 - 検知されたイベントが5つ以上の時、イベント頻度のタイムラインを作成するようにした。 (#533) (@hitenkoku)
 - `--all-tags`オプションでルールにある全てのtagsを、outputで指定したcsvのMitreAttackの列に出力するようにした。 (#525) (@hitenkoku)
 - `-R` / `--display-record-id` オプションの追加。evtx file内のレコードを特定するレコードID`<Event><System><EventRecordID>`が出力できるようになった。 (#548) (@hitenkoku)
+- レベルごとの検知数が最も多い日を表示するようにした。 (#550) (@hitenkoku)
 
 **改善:**
 

CHANGELOG.md

@@ -7,6 +7,7 @@
 - Added Event Frequency Timeline feature to visualize the number of events. (Note: There needs to be more than 5 events.) (#533)(@hitenkoku)
 - Display all the `tags` defined in a rule to the `MitreAttack` column when saving to CSV file with the `--all-tags` option. (#525) (@hitenkoku)
 - Added the `-R / --display-record-id` option: Display the event record ID (`<Event><System><EventRecordID>`). (#548) (@hitenkoku)
+- Display date with most detections by level. (#550) (@hitenkoku)
 
 **Enhancements:**
 

README-Japanese.md

@@ -66,6 +66,7 @@ Hayabusaは、日本の[Yamato Security](https://yamatosecurity.connpass.com/)
   - [プログレスバー](#プログレスバー)
   - [標準出力へのカラー設定](#標準出力へのカラー設定)
   - [イベント頻度タイムライン](#イベント頻度タイムライン)
+  - [各レベルの最多検知日の出力](#各レベルの最多検知日の出力)
 - [Hayabusaルール](#hayabusaルール)
   - [Hayabusa v.s. 変換されたSigmaルール](#hayabusa-vs-変換されたsigmaルール)
   - [検知ルールのチューニング](#検知ルールのチューニング)
@@ -543,6 +544,8 @@ CSVファイルとして保存する場合、以下の列が追加されます:
 * `Microsoft-Windows-DHCP-Server/Operational` : DHCP-Svr
 * `Microsoft-Windows-DriverFrameworks-UserMode/Operational` : DvrFmwk
 * `Microsoft-Windows-NTLM/Operational` : NTLM
+* `Microsoft-Windows-Security-Mitigations/KernelMode` : SecMitigations
+* `Microsoft-Windows-Security-Mitigations/UserMode` : SecMitigations
 * `Microsoft-Windows-SmbClient/Security` : SmbCliSec
 * `Microsoft-Windows-Sysmon/Operational` : Sysmon
 * `Microsoft-Windows-TaskScheduler/Operational` : TaskSch
@@ -574,6 +577,10 @@ Hayabusaの結果は`level`毎に文字色が変わります。
 検知したイベントの数が5以上の時、頻度のタイムライン(スパークライン)を画面に出力します。
 マーカーの数は最大10個です。
 
+## 各レベルの最多検知日の出力
+
+各レベルで最も検知された日付を画面に出力します。
+
 # Hayabusaルール
 
 Hayabusa検知ルールはSigmaのようなYML形式で記述されています。`rules`ディレクトリに入っていますが、将来的には[https://github.com/Yamato-Security/hayabusa-rules](https://github.com/Yamato-Security/hayabusa-rules)のレポジトリで管理する予定なので、ルールのissueとpull requestはhayabusaのレポジトリではなく、ルールレポジトリへお願いします。

README.md

@@ -65,6 +65,7 @@ Hayabusa is a **Windows event log fast forensics timeline generator** and **thre
   - [Progress Bar](#progress-bar)
   - [Color Output](#color-output)
   - [Event Fequency Timeline](#event-fequency-timeline)
+  - [Dates with most detections categorized by level](#dates-with-most-detections-categorized-by-level)
 - [Hayabusa Rules](#hayabusa-rules)
   - [Hayabusa v.s. Converted Sigma Rules](#hayabusa-vs-converted-sigma-rules)
   - [Detection Rule Tuning](#detection-rule-tuning)
@@ -527,7 +528,7 @@ If you want to output all the tags defined in a rule, please specify the `--all-
 ## Channel Abbreviations
 
 In order to save space, we use the following abbreviations when displaying Channel.
-You can freely edit these abbreviations in the `config/config/channel_abbreviations.txt` configuration file.
+You can freely edit these abbreviations in the `config/channel_abbreviations.txt` configuration file.
 
 * `Application` : App
 * `DNS Server` : DNS-Svr
@@ -542,6 +543,8 @@ You can freely edit these abbreviations in the `config/config/channel_abbreviati
 * `Microsoft-Windows-DHCP-Server/Operational` : DHCP-Svr
 * `Microsoft-Windows-DriverFrameworks-UserMode/Operational` : DvrFmwk
 * `Microsoft-Windows-NTLM/Operational` : NTLM
+* `Microsoft-Windows-Security-Mitigations/KernelMode` : SecMitigations
+* `Microsoft-Windows-Security-Mitigations/UserMode` : SecMitigations
 * `Microsoft-Windows-SmbClient/Security` : SmbCliSec
 * `Microsoft-Windows-Sysmon/Operational` : Sysmon
 * `Microsoft-Windows-TaskScheduler/Operational` : TaskSch
@@ -572,6 +575,10 @@ If you want to disable color output, you can use `--no-color` option.
 The Event Frequency Timeline feature displays a sparkline frequency timeline of detected events.
 Note: There needs to be more than 5 events.
 
+## Dates with most detections categorized by level
+
+A summary of the dates with the most detections categorized by level (`critical`, `high`, etc...).
+
 # Hayabusa Rules
 
 Hayabusa detection rules are written in a sigma-like YML format and are located in the `rules` folder. In the future, we plan to host the rules at [https://github.com/Yamato-Security/hayabusa-rules](https://github.com/Yamato-Security/hayabusa-rules) so please send any issues and pull requests for rules there instead of the main hayabusa repository.

config/channel_abbreviations.txt

@@ -12,6 +12,8 @@ Microsoft-Windows-Bits-Client/Operational,BitsCli
 Microsoft-Windows-DHCP-Server/Operational,DHCP-Svr
 Microsoft-Windows-DriverFrameworks-UserMode/Operational,DvrFmwk
 Microsoft-Windows-NTLM/Operational,NTLM
+Microsoft-Windows-Security-Mitigations/KernelMode,SecMitigations
+Microsoft-Windows-Security-Mitigations/UserMode,SecMitigations
 Microsoft-Windows-SmbClient/Security,SmbCliSec
 Microsoft-Windows-Sysmon/Operational,Sysmon
 Microsoft-Windows-TaskScheduler/Operational,TaskSch

src/afterfact.rs

@@ -203,6 +203,21 @@ fn emit_csv<W: std::io::Write>(
     let mut total_detect_counts_by_level: Vec<u128> = vec![0; 6];
     let mut unique_detect_counts_by_level: Vec<u128> = vec![0; 6];
     let mut detected_rule_files: Vec<String> = Vec::new();
+    let mut detect_counts_by_date_and_level: HashMap<String, HashMap<String, u128>> =
+        HashMap::new();
+
+    let levels = Vec::from([
+        "critical",
+        "high",
+        "medium",
+        "low",
+        "informational",
+        "undefined",
+    ]);
+    // レベル別、日ごとの集計用変数の初期化
+    for level_init in levels {
+        detect_counts_by_date_and_level.insert(level_init.to_string(), HashMap::new());
+    }
 
     println!();
     let mut timestamps: Vec<i64> = Vec::new();
@@ -216,6 +231,7 @@ fn emit_csv<W: std::io::Write>(
             if level == "informational" {
                 level = "info".to_string();
             }
+            let time_str = format_time(time);
             if displayflag {
                 let record_id = detect_info
                     .record_id
@@ -232,7 +248,7 @@ fn emit_csv<W: std::io::Write>(
                     .collect::<String>();
 
                 let dispformat = DisplayFormat {
-                    timestamp: &_format_cellpos(&format_time(time), ColPos::First),
+                    timestamp: &_format_cellpos(&time_str, ColPos::First),
                     level: &_format_cellpos(&level, ColPos::Other),
                     computer: &_format_cellpos(&detect_info.computername, ColPos::Other),
                     event_i_d: &_format_cellpos(&detect_info.eventid, ColPos::Other),
@@ -258,7 +274,7 @@ fn emit_csv<W: std::io::Write>(
             } else {
                 // csv output format
                 wtr.serialize(CsvFormat {
-                    timestamp: &format_time(time),
+                    timestamp: &time_str,
                     level: &level,
                     computer: &detect_info.computername,
                     event_i_d: &detect_info.eventid,
@@ -275,11 +291,21 @@ fn emit_csv<W: std::io::Write>(
             let level_suffix = *configs::LEVELMAP
                 .get(&detect_info.level.to_uppercase())
                 .unwrap_or(&0) as usize;
+            let time_str_date = &time_str[0..10];
+            let mut detect_counts_by_date = detect_counts_by_date_and_level
+                .get(&detect_info.level.to_lowercase())
+                .unwrap()
+                .clone();
+            *detect_counts_by_date
+                .entry(time_str_date.to_string())
+                .or_insert(0) += 1;
             if !detected_rule_files.contains(&detect_info.rulepath) {
                 detected_rule_files.push(detect_info.rulepath.clone());
                 unique_detect_counts_by_level[level_suffix] += 1;
             }
             total_detect_counts_by_level[level_suffix] += 1;
+            detect_counts_by_date_and_level
+                .insert(detect_info.level.to_lowercase(), detect_counts_by_date);
         }
     }
     if displayflag {
@@ -308,6 +334,10 @@ fn emit_csv<W: std::io::Write>(
         "Data reduction: {} events ({:.2}%)",
         reducted_record_cnt, reducted_percent
     );
+    println!();
+
+    _print_detection_summary_by_date(detect_counts_by_date_and_level, &color_map);
+
     println!();
     _print_unique_results(
         total_detect_counts_by_level,
@@ -315,6 +345,8 @@ fn emit_csv<W: std::io::Write>(
         "detections".to_string(),
         &color_map,
     );
+    println!();
+
     _print_unique_results(
         unique_detect_counts_by_level,
         "Unique".to_string(),
@@ -389,6 +421,9 @@ fn _print_unique_results(
     .ok();
 
     for (i, level_name) in levels.iter().enumerate() {
+        if "undefined" == *level_name {
+            continue;
+        }
         let output_raw_str = format!(
             "{} {} {}: {}",
             head_word, level_name, tail_word, counts_by_level[i]
@@ -402,6 +437,45 @@ fn _print_unique_results(
     }
 }
 
+/// 各レベル毎で最も高い検知数を出した日付を出力する
+fn _print_detection_summary_by_date(
+    detect_counts_by_date: HashMap<String, HashMap<String, u128>>,
+    color_map: &HashMap<String, Color>,
+) {
+    let buf_wtr = BufferWriter::stdout(ColorChoice::Always);
+    let mut wtr = buf_wtr.buffer();
+    wtr.set_color(ColorSpec::new().set_fg(None)).ok();
+
+    let output_levels = Vec::from(["critical", "high", "medium", "low", "informational"]);
+
+    for level in output_levels {
+        // output_levelsはlevelsからundefinedを除外した配列であり、各要素は必ず初期化されているのでSomeであることが保証されているのでunwrapをそのまま実施
+        let detections_by_day = detect_counts_by_date.get(level).unwrap();
+        let mut max_detect_str = String::default();
+        let mut tmp_cnt: u128 = 0;
+        let mut date_str = String::default();
+        for (date, cnt) in detections_by_day {
+            if cnt > &tmp_cnt {
+                date_str = date.clone();
+                max_detect_str = format!("{} (Count: {})", date, cnt);
+                tmp_cnt = *cnt;
+            }
+        }
+        wtr.set_color(ColorSpec::new().set_fg(_get_output_color(color_map, level)))
+            .ok();
+        if date_str == String::default() {
+            max_detect_str = "-".to_string();
+        }
+        writeln!(
+            wtr,
+            "Date with most {} detections: {}",
+            level, &max_detect_str
+        )
+        .ok();
+    }
+    buf_wtr.print(&wtr).ok();
+}
+
 fn format_time(time: &DateTime<Utc>) -> String {
     if configs::CONFIG.read().unwrap().args.is_present("utc") {
         format_rfc(time)

src/options/level_tuning.rs

@@ -105,6 +105,7 @@ impl LevelTuning {
                 .ok();
             }
         }
+        println!();
         Result::Ok(())
     }
 }