Merge pull request #671 from Yamato-Security/667-enhancement-add-top-alerts-to-results-summary

Added top alerts to results summary
2022-08-21 11:14:11 +09:00
parent e498007ea6 f50668fe36
commit c41b73836e
10 changed files with 113 additions and 10 deletions
--- a/CHANGELOG-Japanese.md
+++ b/CHANGELOG-Japanese.md
@@ -1,5 +1,19 @@
 # 変更点

+## v1.6.0 [2022/XX/XX]
+
+**新機能:**
+
+- XXX
+
+**改善:**
+
+- 結果概要に各レベルで検知した上位5つのルールを表示するようにした。 (#667) (@hitenkoku)
+
+**バグ修正:**
+
+- XXX
+
 ## v1.5.1 [2022/08/20]

 **改善:**
@@ -161,7 +175,7 @@

 **新機能:**

- `-C / --config` オプションの追加。検知ルールのコンフィグを指定することが可能。(Windowsでのライブ調査に便利) (@hitenkoku) 
+- `-C / --config` オプションの追加。検知ルールのコンフィグを指定することが可能。(Windowsでのライブ調査に便利) (@hitenkoku)
 - `|equalsfield` と記載することでルール内で二つのフィールドの値が一致するかを記載に対応。 (@hach1yon)
 - `-p / --pivot-keywords-list` オプションの追加。攻撃されたマシン名や疑わしいユーザ名などの情報をピボットキーワードリストとして出力する。 (@kazuminn)
 - `-F / --full-data`オプションの追加。ルールの`details`で指定されたフィールドだけではなく、全フィールド情報を出力する。(@hach1yon)
@@ -192,7 +206,7 @@

 - `-r / --rules`オプションで一つのルール指定が可能。(ルールをテストする際に便利！) (@kazuminn)
 - ルール更新オプション (`-u / --update-rules`): [hayabusa-rules](https://github.com/Yamato-Security/hayabusa-rules)レポジトリにある最新のルールに更新できる。 (@hitenkoku)
- ライブ調査オプション (`-l / --live-analysis`): Windowsイベントログディレクトリを指定しないで、楽にWindows端末でライブ調査ができる。(@hitenkoku) 
+- ライブ調査オプション (`-l / --live-analysis`): Windowsイベントログディレクトリを指定しないで、楽にWindows端末でライブ調査ができる。(@hitenkoku)

 **改善:**

--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,19 @@
 # Changes

+## v1.6.0 [2022/XX/XX]
+
+**New Features:**
+
+- XXX
+
+**Enhancements:**
+
+- Added top alert rules to results summary. (#667) (@hitenkoku)
+
+**Bug Fixes:**
+
+- XXX
+
 ## v1.5.1 [2022/08/20]

 **Enhancements:**
@@ -162,7 +176,7 @@

 **New Features:**

- Specify config directory (`-C / --config`): When specifying a different rules directory, the rules config directory will still be the default `rules/config`, so this option is useful when you want to test rules and their config files in a different directory. (@hitenkoku) 
+- Specify config directory (`-C / --config`): When specifying a different rules directory, the rules config directory will still be the default `rules/config`, so this option is useful when you want to test rules and their config files in a different directory. (@hitenkoku)
 - `|equalsfield` aggregator: In order to write rules that compare if two fields are equal or not. (@hach1yon)
 - Pivot keyword list generator feature (`-p / --pivot-keywords-list`): Will generate a list of keywords to grep for to quickly identify compromised machines, suspicious usernames, files, etc... (@kazuminn)
 - `-F / --full-data` option: Will output all field information in addition to the fields defined in the rule’s `details`. (@hach1yon)
@@ -193,7 +207,7 @@

 - Can specify a single rule with the `-r / --rules` option. (Great for testing rules!) (@kazuminn)
 - Rule update option (`-u / --update-rules`): Update to the latest rules in the [hayabusa-rules](https://github.com/Yamato-Security/hayabusa-rules) repository. (@hitenkoku)
- Live analysis option (`-l / --live-analysis`): Can easily perform live analysis on Windows machines without specifying the Windows event log directory. (@hitenkoku) 
+- Live analysis option (`-l / --live-analysis`): Can easily perform live analysis on Windows machines without specifying the Windows event log directory. (@hitenkoku)

 **Enhancements:**

--- a/Cargo.lock
+++ b/Cargo.lock
@@ -707,7 +707,7 @@ dependencies = [

 [[package]]
 name = "hayabusa"
-version = "1.5.1"
+version = "1.6.0-dev"
 dependencies = [
 "base64",
 "bytesize",
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "hayabusa"
-version = "1.5.1"
+version = "1.6.0-dev"
 authors = ["Yamato Security @SecurityYamato"]
 edition = "2021"

--- a/README-Japanese.md
+++ b/README-Japanese.md
@@ -10,7 +10,7 @@
 [tag-1]: https://img.shields.io/github/downloads/Yamato-Security/hayabusa/total?style=plastic&label=GitHub%F0%9F%A6%85Downloads
 [tag-2]: https://img.shields.io/github/stars/Yamato-Security/hayabusa?style=plastic&label=GitHub%F0%9F%A6%85Stars
 [tag-3]: https://img.shields.io/github/v/release/Yamato-Security/hayabusa?display_name=tag&label=latest-version&style=plastic
-[tag-4]: https://img.shields.io/badge/Black%20Hat%20Arsenal-Asia%202022-blue
+[tag-4]: https://github.com/toolswatch/badges/blob/master/arsenal/asia/2022.svg
 [tag-5]: https://rust-reportcard.xuri.me/badge/github.com/Yamato-Security/hayabusa
 [tag-6]: https://img.shields.io/badge/Maintenance%20Level-Actively%20Developed-brightgreen.svg
 [tag-7]: https://img.shields.io/badge/Twitter-00acee?logo=twitter&logoColor=white
--- a/README.md
+++ b/README.md
@@ -10,7 +10,7 @@
 [tag-1]: https://img.shields.io/github/downloads/Yamato-Security/hayabusa/total?style=plastic&label=GitHub%F0%9F%A6%85Downloads
 [tag-2]: https://img.shields.io/github/stars/Yamato-Security/hayabusa?style=plastic&label=GitHub%F0%9F%A6%85Stars
 [tag-3]: https://img.shields.io/github/v/release/Yamato-Security/hayabusa?display_name=tag&label=latest-version&style=plastic
-[tag-4]: https://img.shields.io/badge/Black%20Hat%20Arsenal-Asia%202022-blue
+[tag-4]: https://github.com/toolswatch/badges/blob/master/arsenal/asia/2022.svg
 [tag-5]: https://rust-reportcard.xuri.me/badge/github.com/Yamato-Security/hayabusa
 [tag-6]: https://img.shields.io/badge/Maintenance%20Level-Actively%20Developed-brightgreen.svg
 [tag-7]: https://img.shields.io/badge/Twitter-00acee?logo=twitter&logoColor=white
--- a/src/afterfact.rs
+++ b/src/afterfact.rs
@@ -184,12 +184,15 @@ fn emit_csv<W: std::io::Write>(
        HashMap::new();
    let mut detect_counts_by_computer_and_level: HashMap<String, HashMap<String, i128>> =
        HashMap::new();
+    let mut detect_counts_by_rule_and_level: HashMap<String, HashMap<String, i128>> =
+        HashMap::new();

    let levels = Vec::from(["crit", "high", "med ", "low ", "info", "undefined"]);
    // レベル別、日ごとの集計用変数の初期化
    for level_init in levels {
        detect_counts_by_date_and_level.insert(level_init.to_string(), HashMap::new());
        detect_counts_by_computer_and_level.insert(level_init.to_string(), HashMap::new());
+        detect_counts_by_rule_and_level.insert(level_init.to_string(), HashMap::new());
    }
    if displayflag {
        println!();
@@ -247,6 +250,7 @@ fn emit_csv<W: std::io::Write>(
                )
                .unwrap_or(&0) as usize;
            let time_str_date = format_time(time, true);
+
            let mut detect_counts_by_date = detect_counts_by_date_and_level
                .get(&detect_info.level.to_lowercase())
                .unwrap_or_else(|| detect_counts_by_date_and_level.get("undefined").unwrap())
@@ -258,6 +262,7 @@ fn emit_csv<W: std::io::Write>(
                detected_rule_files.insert(detect_info.rulepath.clone());
                unique_detect_counts_by_level[level_suffix] += 1;
            }
+
            let computer_rule_check_key =
                format!("{}|{}", &detect_info.computername, &detect_info.rulepath);
            if !detected_computer_and_rule_names.contains(&computer_rule_check_key) {
@@ -277,6 +282,20 @@ fn emit_csv<W: std::io::Write>(
                    .insert(detect_info.level.to_lowercase(), detect_counts_by_computer);
            }

+            let mut detect_counts_by_rules = detect_counts_by_rule_and_level
+                .get(&detect_info.level.to_lowercase())
+                .unwrap_or_else(|| {
+                    detect_counts_by_computer_and_level
+                        .get("undefined")
+                        .unwrap()
+                })
+                .clone();
+            *detect_counts_by_rules
+                .entry(Clone::clone(&detect_info.ruletitle))
+                .or_insert(0) += 1;
+            detect_counts_by_rule_and_level
+                .insert(detect_info.level.to_lowercase(), detect_counts_by_rules);
+
            total_detect_counts_by_level[level_suffix] += 1;
            detect_counts_by_date_and_level
                .insert(detect_info.level.to_lowercase(), detect_counts_by_date);
@@ -375,6 +394,9 @@ fn emit_csv<W: std::io::Write>(
    println!();

    _print_detection_summary_by_computer(detect_counts_by_computer_and_level, &color_map);
+    println!();
+
+    _print_detection_summary_by_rule(detect_counts_by_rule_and_level, &color_map);

    Ok(())
 }
@@ -570,6 +592,55 @@ fn _print_detection_summary_by_computer(
    buf_wtr.print(&wtr).ok();
 }

+/// 各レベルごとで検出数が多かったルールのタイトルを出力する関数
+fn _print_detection_summary_by_rule(
+    detect_counts_by_rule_and_level: HashMap<String, HashMap<String, i128>>,
+    color_map: &HashMap<String, Color>,
+) {
+    let buf_wtr = BufferWriter::stdout(ColorChoice::Always);
+    let mut wtr = buf_wtr.buffer();
+    wtr.set_color(ColorSpec::new().set_fg(None)).ok();
+    let level_cnt = detect_counts_by_rule_and_level.len();
+    for (idx, level) in LEVEL_ABBR.values().enumerate() {
+        // output_levelsはlevelsからundefinedを除外した配列であり、各要素は必ず初期化されているのでSomeであることが保証されているのでunwrapをそのまま実施
+        let detections_by_computer = detect_counts_by_rule_and_level.get(level).unwrap();
+        let mut result_vec: Vec<String> = Vec::new();
+        let mut sorted_detections: Vec<(&String, &i128)> = detections_by_computer.iter().collect();
+
+        sorted_detections.sort_by(|a, b| (-a.1).cmp(&(-b.1)));
+
+        for x in sorted_detections.iter().take(5) {
+            result_vec.push(format!(
+                "{} ({})",
+                x.0,
+                x.1.to_formatted_string(&Locale::en)
+            ));
+        }
+        let result_str = if result_vec.is_empty() {
+            "None".to_string()
+        } else {
+            result_vec.join("\n")
+        };
+
+        wtr.set_color(ColorSpec::new().set_fg(_get_output_color(
+            color_map,
+            LEVEL_FULL.get(level.as_str()).unwrap(),
+        )))
+        .ok();
+        writeln!(
+            wtr,
+            "Top {} alerts:\n{}",
+            LEVEL_FULL.get(level.as_str()).unwrap(),
+            &result_str
+        )
+        .ok();
+        if idx != level_cnt - 1 {
+            writeln!(wtr).ok();
+        }
+    }
+    buf_wtr.print(&wtr).ok();
+}
+
 /// get timestamp to input datetime.
 fn _get_timestamp(time: &DateTime<Utc>) -> i64 {
    if configs::CONFIG.read().unwrap().args.utc {
@@ -663,6 +734,7 @@ mod tests {
                output.to_string(),
                DetectInfo {
                    rulepath: test_rulepath.to_string(),
+                    ruletitle: test_title.to_string(),
                    level: test_level.to_string(),
                    computername: test_computername.to_string(),
                    eventid: test_eventid.to_string(),
--- a/src/detections/detection.rs
+++ b/src/detections/detection.rs
@@ -37,7 +37,7 @@ use super::message::LEVEL_ABBR;
 // イベントファイルの1レコード分の情報を保持する構造体
 #[derive(Clone, Debug)]
 pub struct EvtxRecordInfo {
-    pub evtx_filepath: String, // イベントファイルのファイルパス　ログで出力するときに使う
+    pub evtx_filepath: String, // イベントファイルのファイルパス ログで出力するときに使う
    pub record: Value,         // 1レコード分のデータをJSON形式にシリアライズしたもの
    pub data_string: String,
    pub key_2_value: HashMap<String, String>,
@@ -362,6 +362,7 @@ impl Detection {

        let detect_info = DetectInfo {
            rulepath: (&rule.rulepath).to_owned(),
+            ruletitle: rule.yaml["title"].as_str().unwrap_or("-").to_string(),
            level: LEVEL_ABBR.get(&level).unwrap_or(&level).to_string(),
            computername: record_info.record["Event"]["System"]["Computer"]
                .to_string()
@@ -492,6 +493,7 @@ impl Detection {

        let detect_info = DetectInfo {
            rulepath: (&rule.rulepath).to_owned(),
+            ruletitle: rule.yaml["title"].as_str().unwrap_or("-").to_string(),
            level: LEVEL_ABBR.get(&level).unwrap_or(&level).to_string(),
            computername: "-".to_owned(),
            eventid: "-".to_owned(),
--- a/src/detections/message.rs
+++ b/src/detections/message.rs
@@ -24,6 +24,7 @@ use termcolor::{BufferWriter, ColorChoice};
 #[derive(Debug, Clone)]
 pub struct DetectInfo {
    pub rulepath: String,
+    pub ruletitle: String,
    pub level: String,
    pub computername: String,
    pub eventid: String,
@@ -634,6 +635,7 @@ mod tests {
        for i in 1..2001 {
            let detect_info = DetectInfo {
                rulepath: "".to_string(),
+                ruletitle: "".to_string(),
                level: "".to_string(),
                computername: "".to_string(),
                eventid: i.to_string(),
--- a/src/main.rs
+++ b/src/main.rs
@@ -305,7 +305,6 @@ impl App {

        let analysis_end_time: DateTime<Local> = Local::now();
        let analysis_duration = analysis_end_time.signed_duration_since(analysis_start_time);
-        println!();
        write_color_buffer(
            &BufferWriter::stdout(ColorChoice::Always),
            None,