CSEC_PUBLIC/hayabusa
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

v1.2 pre-release marge (#495)

Browse Source 
* Fix/fix clippy warn (#434)

- Fixed following Clippy Warnings(previous warning count: 671 -> after: 4)
  - clippy::needless_return
  - clippy::println_empty_string
  - clippy::redundant_field_names
  - clippy::single_char_pattern
  - clippy::len_zero
  - clippy::iter_nth_zero
  - clippy::bool_comparison
  - clippy::question_mark
  - clippy::needless_collect
  - clippy::unnecessary_unwrap
  - clippy::ptr_arg
  - clippy::needless_collect
  - clippy::needless_borrow
  - clippy::new_without_default
  - clippy::assign_op_pattern
  - clippy::bool_assert_comparison
  - clippy::into_iter_on_ref
  - clippy::deref_addrof
  - clippy::while_let_on_iterator
  - clippy::match_like_matches_macro
  - clippy::or_fun_call
  - clippy::useless_conversion
  - clippy::let_and_return
  - clippy::redundant_clone
  - clippy::redundant_closure
  - clippy::cmp_owned
  - clippy::upper_case_acronyms
  - clippy::map_identity
  - clippy::unused_io_amount
  - clippy::assertions_on_constants
  - clippy::op_ref
  - clippy::useless_vec
  - clippy::vec_init_then_push
  - clippy::useless_format
  - clippy::bind_instead_of_map
  - clippy::bool_comparison
  - clippy::clone_on_copy
  - clippy::too_many_arguments
  - clippy::module_inception
  - fixed clippy::needless_lifetimes
  - fixed clippy::borrowed_box (Thanks for helping by hach1yon!)

* Merge main and output fix#443#444 (#445)

* removed tools/sigmac (#441)

* removed tools/sigmac

- moved tools/sigmac to hayabusa-rules repo

* fixed doc link tools/sigmac

* fixed submodule track

* fixed submodule track from latest to v1.1.0 tag

* fixed link

* erased enter #444

* erased enter #444

* reverted logo enter

* fixed rules submodule target commit #444

Co-authored-by: Yamato Security <71482215+YamatoSecurity@users.noreply.github.com>

* readme update screenshots etc (#448)

* Opensslを静的にコンパイルするためにCargo.tomlの設定変更 (#437)

* cargo update - openssl static

* updated cargo

* macos2apple

* cargo update

* cargo update

* aliasキーがない場合もEvent.EventDataを自動で走査する (#442)

* add no event key

* support not-register-alias search

* added checking EventData when key do not match in alias #290

- added checking key in Event.EventData, if key is not exist in eventkey_alias.txt.

* cargo fmt

* fixed panic when filter files does not exists

* fixed errorlog format when filter config files does not exist

Co-authored-by: DustInDark <nextsasasa@gmail.com>

* changed downcast library from mopa to downcast_rs #447 (#450)

* Fixed Clippy Warnings (#451)

* fixed clippy warn

* fixed cargo clippy warnging

* fixed clippy warngings in clippy ver 0.1.59

* fixed clippy warnings clippy::unnecessary_to_owned

* added temporary blackhat arsenal badge

* added rust report card badges #453

* added repository maintenance levels badge #453

* documentation update macOS usage etc

* update

* added clippy workflow #428 (#429)

* added clippy workflow #428

* fixed action yaml to run clippy #428

* fixed indent

* fixed workflow

* fixed workflow error

* fixed indent

* changed no annotation #428

* adujusted annotation version

* fixed clippy::needless_match

* remove if let exception

* removed unnecessary permission check #428

* statistics event id update (#457)

* Feature/#440 refactoring #395 (#464)

* updated submodule

* fix degrade for pull req #464 (#468)

* fix degrade for pull req #464

* add trim

* Fearture/ added output update result#410 (#452)

* add git2 crate #391

* added Update option #391

* updated readme #391

* fixed cargo.lock

* fixed option if-statement #391

* changed utc short option and rule-update short option #391

* updated readme

* updated readme

* fixed -u long option & version number update #391

* added fast-forwarding rules repository #391

* updated command line option #391

* moved output logo prev update rule

* fixed readme #391

* removed recursive option in readme

* changed rules update from clone and pull to submodule update #391

* fixed document

* changed unnecessary clone recursively to clone only

* English message update.

* cargo fmt

* English message update. ( 4657c35e5c cherry-pick)

* added create rules folder when rules folder is not exist

* fixed gitmodules github-rules url from ssh to https

* added output of updated file #420

* fixed error #410

* changed update rule list seq

* added test

* fixed output #410

* fixed output and fixed output date field  when  modified field is lacked #410

* fixed compile error

* fixed output

- added enter after Latest rule update output
- added output when no exist new rule
- fixed Latest rule update date format
- changed output from 'Latest rule update' to 'Latest rules update'

* fixed compile error

* changed modified date source from rules folder to each yml rule file

* formatting use chrono in main.rs

* merge develop clippy ci

* fixed output when no update rule #410

- removed Latest rule update

- no output "Rules update successfully" when No rule changed

* Change English

Co-authored-by: Tanaka Zakku <71482215+YamatoSecurity@users.noreply.github.com>

* Remove unnecessary code from timeline_event_info and rename files for… (#470)

* Remove unnecessary code from timeline_event_info and rename files for issue462

* Remove unnecessary code #462

* add equalsfield pipe (#467)

* Enhancement: add config config #456 (#471)

* added config option #456

* added process of option to speicifed config folder #456

following files adjust config option.

* noisy_rules.txt

* exclude_rules.txt

* fixed usage in readme

* updated rules submodule:

* fixed process when yml file exist in .git folder

* ignore when yml file exist in .git folder

* Add: --level-tuning option's outline

* Add: read Rule files

* Add: input rule_level.txt files & read rules

* cargo fmt

* Add: level-tuning function

* Reface: split to options file

* WIP: Text overwrite failed...

* Fix: Text overwrite was failed

* Add: Error handlings

* Add: id, level validation

* mv: IDS_REGEX to configs file

* fix: level tuning's file name

* Cargo fmt

* Pivot Keyword List機能の追加 (#412)

* add get_pivot_keyword() func

* change function name and call it's function

* [WIP] support config file

* compilete output

* cargo fmt

* [WIP] add test

* add test

* support -o option in pivot

* add pivot mod

* fix miss

* pass test in pivot.rs

* add comment

* pass all test

* add fast return

* fix output

* add test config file

* review

* rebase

* cargo fmt

* test pass

* fix clippy in my commit

* cargo fmt

* little refactor

* change file input logic and config format

* [WIP] change output

* [wip] change deta structure

* change output & change data structure

* pass test

* add config

* cargo fmt & clippy & rebase

* fix cllipy

* delete /rules/ in .gitignore

* clean comment

* clean

* clean

* fix rebase miss

* fix rebase miss

* fix clippy

* file name output on -o to stdout

* add pivot_keywords.txt to ./config

* updated english

* Documentation update

* cargo fmt and clean

* updated translate japanese

* readme update

* readme update

Co-authored-by: DustInDark <nextsasasa@gmail.com>
Co-authored-by: Tanaka Zakku <71482215+YamatoSecurity@users.noreply.github.com>

* Add: test

* Add: README.md

* Cargo fmt

* Use
#[cfg(test)]

* Fixed output stop when  control char exist in windows terminal (#485)

* added control character filter in details #382

* fixed document

- removed fixed windows teminal caution in readme

* fixed level tuning test and added test files #390

* changed level_tuning.txt header from next_level to new_level

* fixed convert miss change to low level

* added run args rules path to check test easy #390

* fixed comment out processing in level_tuning.txt

* fixed config to show level-tuning option

* fixed level-tuning option usage from required to option

* reduce output mitre attack detail tachnique No. by config file (#483)

* reduced mitre attck tag output by config file #477

* prepared 1.2.0 version toml

* added test files and mitre attck strategy tag file #477

* fixed cargo.toml version

* updated cargo.lock

* output tag english update

* cargo fmt

Co-authored-by: Tanaka Zakku <71482215+YamatoSecurity@users.noreply.github.com>

* Fix: test file's path was incorrect

* Add: add test_files/config/level_tuning.txt

* Add: Flush method.

* inserted debug data

* reverted config usage

* fixed test yaml file path

* Feature/#216 output allfields csvnewcolumn (#469)

* refactoring

* refactoring

* under constructing

* underconstructing

* under construction

* underconstructing

* fix existing testcase

* finish implement

* fmt

* add option

* change name

* fix control code bug

* fix disp

* change format and fix testcase

* fix help

* Fix: show usage when hayabusa has no args

* rm: debug line

* Enhance/warning architecture#478 (#482)

* added  enhance of architecture check #478

* changed check architecture process after output logo #478

* English msg update

* fixed detect method of os-bit to windows and linux

* removed mac and unix architecture and binary and updated its process of windows

* fix clippy

* added check on Wow64 env #478

* Update contributors.txt

Co-authored-by: Tanaka Zakku <71482215+YamatoSecurity@users.noreply.github.com>

* added --level-tuning option to usage

* Revert "added --level-tuning option to usage"

This reverts commit e6a74090a3.

* readme update

* Update README-Japanese.md

* readme, version, cargo update

* typo fix

* typo fix

* rm: duplicated test & fix test name

* Add: show logo, and some infos

* small english fix

* twitter link fix (#486)

* added feature of tag output reducing to agg condition #477 (#488)

* changed level output from informational to info #491

* updated rules submodule

* v1.2 changelog update (#473)

* changelog update

* Update CHANGELOG.md

added contributor in "Fields that are not defined in eventkey_alias.txt will automatically be searched in Event.EventData."

ref #442

* Update CHANGELOG-Japanese.md

Fields that are not defined in eventkey_alias.txt will automatically be searched in Event.EventData.

added contributor in "Fields that are not defined in eventkey_alias.txt will automatically be searched in Event.EventData."

ref #442

* Update CHANGELOG.md

added bug fixes (#444) and `Performance and. accuracy`  add contributor ref(#395)

* Update CHANGELOG-Japanese.md

* Translated v1.2 change log to Japanese

v1.2の内容を日本語に修正

* fixed typo

added lacked back quote.

* added description

added following issue and pr description to readme

- #216 / #469 L8
- #390 / #459 L9
- #478 / #482 L19
- #477/ #483 L20

* added description README.md

added following issue and pr description to readme

- #216 / #469 L8
- #390 / #459 L9
- #478 / #482 L19
- #477/ #483 L20

* changelog update

* changelog update

* update

Co-authored-by: DustInDark <nextsasasa@gmail.com>

* updated rules #493 (#494)

* Resolve conflict develop (#496)

* removed tools/sigmac (#441)

* removed tools/sigmac

- moved tools/sigmac to hayabusa-rules repo

* fixed doc link tools/sigmac

* fixed submodule track

* fixed submodule track from latest to v1.1.0 tag

* fixed link

* fixed rules submodule targe #444

* updated submodule

* updated rules submodule

Co-authored-by: Yamato Security <71482215+YamatoSecurity@users.noreply.github.com>

Co-authored-by: Yamato Security <71482215+YamatoSecurity@users.noreply.github.com>
Co-authored-by: kazuminn <warugaki.k.k@gmail.com>
Co-authored-by: James / hach1yon <32596618+hach1yon@users.noreply.github.com>
Co-authored-by: garigariganzy <tosada31@hotmail.co.jp>
Co-authored-by: itiB <is0312vx@ed.ritsumei.ac.jp>
This commit is contained in:
DustInDark
2022-04-15 12:13:00 +09:00
committed by GitHub
parent 2b5837dfc8
commit bcf8a33e8c
50 changed files with 3726 additions and 1982 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/afterfact.rs
+123 -116
View File
@@ -21,9 +21,11 @@ pub struct CsvFormat<'a> {
computer: &'a str,
event_i_d: &'a str,
level: &'a str,
mitre_attack: &'a str,
rule_title: &'a str,
details: &'a str,
mitre_attack: &'a str,
#[serde(skip_serializing_if = "Option::is_none")]
record_information: Option<&'a str>,
rule_path: &'a str,
file_path: &'a str,
}
@@ -37,6 +39,8 @@ pub struct DisplayFormat<'a> {
pub level: &'a str,
pub rule_title: &'a str,
pub details: &'a str,
#[serde(skip_serializing_if = "Option::is_none")]
pub record_information: Option<&'a str>,
}
/// level_color.txtファイルを読み込み対応する文字色のマッピングを返却する関数
@@ -49,7 +53,7 @@ pub fn set_output_color() -> Option<HashMap<String, Vec<u8>>> {
// color情報がない場合は通常の白色の出力が出てくるのみで動作への影響を与えない為warnとして処理する
AlertMessage::warn(
&mut BufWriter::new(std::io::stderr().lock()),
&read_result.as_ref().unwrap_err(),
read_result.as_ref().unwrap_err(),
)
.ok();
return None;
@@ -71,12 +75,12 @@ pub fn set_output_color() -> Option<HashMap<String, Vec<u8>>> {
return;
}
let color_code = convert_color_result.unwrap();
if level.len() == 0 || color_code.len() < 3 {
if level.is_empty() || color_code.len() < 3 {
return;
}
color_map.insert(level.to_string(), color_code);
});
return Some(color_map);
Some(color_map)
}
pub fn after_fact() {
@@ -120,16 +124,15 @@ fn emit_csv<W: std::io::Write>(
displayflag: bool,
color_map: Option<HashMap<String, Vec<u8>>>,
) -> io::Result<()> {
let mut wtr;
if displayflag {
wtr = csv::WriterBuilder::new()
let mut wtr = if displayflag {
csv::WriterBuilder::new()
.double_quote(false)
.quote_style(QuoteStyle::Never)
.delimiter(b'|')
.from_writer(writer);
.from_writer(writer)
} else {
wtr = csv::WriterBuilder::new().from_writer(writer);
}
csv::WriterBuilder::new().from_writer(writer)
};
let messages = print::MESSAGES.lock().unwrap();
// levelの区分が"Critical","High","Medium","Low","Informational","Undefined"の6つであるため
@@ -139,82 +142,49 @@ fn emit_csv<W: std::io::Write>(
for (time, detect_infos) in messages.iter() {
for detect_info in detect_infos {
let mut level = detect_info.level.to_string();
if level == "informational" {
level = "info".to_string();
}
if displayflag {
if color_map.is_some() {
let output_color =
_get_output_color(&color_map.as_ref().unwrap(), &detect_info.level);
wtr.serialize(DisplayFormat {
timestamp: &format!(
"{} ",
&format_time(time).truecolor(
output_color[0],
output_color[1],
output_color[2]
)
),
level: &format!(
" {} ",
&detect_info.level.truecolor(
output_color[0],
output_color[1],
output_color[2]
)
),
computer: &format!(
" {} ",
&detect_info.computername.truecolor(
output_color[0],
output_color[1],
output_color[2]
)
),
event_i_d: &format!(
" {} ",
&detect_info.eventid.truecolor(
output_color[0],
output_color[1],
output_color[2]
)
),
rule_title: &format!(
" {} ",
&detect_info.alert.truecolor(
output_color[0],
output_color[1],
output_color[2]
)
),
details: &format!(
" {}",
&detect_info.detail.truecolor(
output_color[0],
output_color[1],
output_color[2]
)
),
})?;
} else {
wtr.serialize(DisplayFormat {
timestamp: &format!("{} ", &format_time(time)),
level: &format!(" {} ", &detect_info.level),
computer: &format!(" {} ", &detect_info.computername),
event_i_d: &format!(" {} ", &detect_info.eventid),
rule_title: &format!(" {} ", &detect_info.alert),
details: &format!(" {}", &detect_info.detail),
})?;
}
let colors = color_map
.as_ref()
.map(|cl_mp| _get_output_color(cl_mp, &detect_info.level));
let colors = colors.as_ref();
let recinfo = detect_info
.record_information
.as_ref()
.map(|recinfo| _format_cell(recinfo, ColPos::Last, colors));
let details = detect_info
.detail
.chars()
.filter(|&c| !c.is_control())
.collect::<String>();
let dispformat = DisplayFormat {
timestamp: &_format_cell(&format_time(time), ColPos::First, colors),
level: &_format_cell(&level, ColPos::Other, colors),
computer: &_format_cell(&detect_info.computername, ColPos::Other, colors),
event_i_d: &_format_cell(&detect_info.eventid, ColPos::Other, colors),
rule_title: &_format_cell(&detect_info.alert, ColPos::Other, colors),
details: &_format_cell(&details, ColPos::Other, colors),
record_information: recinfo.as_deref(),
};
wtr.serialize(dispformat)?;
} else {
// csv出力時フォーマット
wtr.serialize(CsvFormat {
timestamp: &format_time(time),
file_path: &detect_info.filepath,
rule_path: &detect_info.rulepath,
level: &detect_info.level,
level: &level,
computer: &detect_info.computername,
event_i_d: &detect_info.eventid,
mitre_attack: &detect_info.tag_info,
rule_title: &detect_info.alert,
details: &detect_info.detail,
mitre_attack: &detect_info.tag_info,
record_information: detect_info.record_information.as_deref(),
file_path: &detect_info.filepath,
rule_path: &detect_info.rulepath,
})?;
}
let level_suffix = *configs::LEVELMAP
@@ -227,10 +197,10 @@ fn emit_csv<W: std::io::Write>(
total_detect_counts_by_level[level_suffix] += 1;
}
}
println!("");
println!();
wtr.flush()?;
println!("");
println!();
_print_unique_results(
total_detect_counts_by_level,
"Total".to_string(),
@@ -246,6 +216,29 @@ fn emit_csv<W: std::io::Write>(
Ok(())
}
enum ColPos {
First, // 先頭
Last, // 最後
Other, // それ以外
}
fn _format_cellpos(column: ColPos, colval: &str) -> String {
return match column {
ColPos::First => format!("{} ", colval),
ColPos::Last => format!(" {}", colval),
ColPos::Other => format!(" {} ", colval),
};
}
fn _format_cell(word: &str, column: ColPos, output_color: Option<&Vec<u8>>) -> String {
if let Some(color) = output_color {
let colval = format!("{}", word.truecolor(color[0], color[1], color[2]));
_format_cellpos(column, &colval)
} else {
_format_cellpos(column, word)
}
}
/// 与えられたユニークな検知数と全体の検知数の情報(レベル別と総計)を元に結果文を標準出力に表示する関数
fn _print_unique_results(
mut counts_by_level: Vec<u128>,
@@ -276,34 +269,32 @@ fn _print_unique_results(
)
.ok();
for (i, level_name) in levels.iter().enumerate() {
let output_str;
let output_raw_str = format!(
"{} {} {}: {}",
head_word, level_name, tail_word, counts_by_level[i]
);
if color_map.is_none() {
output_str = output_raw_str;
let output_str = if color_map.is_none() {
output_raw_str
} else {
let output_color =
_get_output_color(&color_map.as_ref().unwrap(), &level_name.to_string());
output_str = output_raw_str
let output_color = _get_output_color(color_map.as_ref().unwrap(), level_name);
output_raw_str
.truecolor(output_color[0], output_color[1], output_color[2])
.to_string();
}
.to_string()
};
writeln!(wtr, "{}", output_str).ok();
}
wtr.flush().ok();
}
/// levelに対応したtruecolorの値の配列を返す関数
fn _get_output_color(color_map: &HashMap<String, Vec<u8>>, level: &String) -> Vec<u8> {
fn _get_output_color(color_map: &HashMap<String, Vec<u8>>, level: &str) -> Vec<u8> {
// カラーをつけない場合は255,255,255で出力する
let mut output_color: Vec<u8> = vec![255, 255, 255];
let target_color = color_map.get(level);
if target_color.is_some() {
output_color = target_color.unwrap().to_vec();
if let Some(color) = target_color {
output_color = color.to_vec();
}
return output_color;
output_color
}
fn format_time(time: &DateTime<Utc>) -> String {
@@ -319,11 +310,11 @@ where
Tz::Offset: std::fmt::Display,
{
if configs::CONFIG.read().unwrap().args.is_present("rfc-2822") {
return time.to_rfc2822();
time.to_rfc2822()
} else if configs::CONFIG.read().unwrap().args.is_present("rfc-3339") {
return time.to_rfc3339();
time.to_rfc3339()
} else {
return time.format("%Y-%m-%d %H:%M:%S%.3f %:z").to_string();
time.format("%Y-%m-%d %H:%M:%S%.3f %:z").to_string()
}
}
@@ -331,6 +322,7 @@ where
mod tests {
use crate::afterfact::emit_csv;
use crate::detections::print;
use crate::detections::print::DetectInfo;
use chrono::{Local, TimeZone, Utc};
use serde_json::Value;
use std::fs::File;
@@ -353,6 +345,7 @@ mod tests {
let test_eventid = "1111";
let output = "pokepoke";
let test_attack = "execution/txxxx.yyy";
let test_recinfo = "record_infoinfo11";
{
let mut messages = print::MESSAGES.lock().unwrap();
messages.clear();
@@ -372,15 +365,19 @@ mod tests {
"##;
let event: Value = serde_json::from_str(val).unwrap();
messages.insert(
testfilepath.to_string(),
testrulepath.to_string(),
&event,
test_level.to_string(),
test_computername.to_string(),
test_eventid.to_string(),
test_title.to_string(),
output.to_string(),
test_attack.to_string(),
DetectInfo {
filepath: testfilepath.to_string(),
rulepath: testrulepath.to_string(),
level: test_level.to_string(),
computername: test_computername.to_string(),
eventid: test_eventid.to_string(),
alert: test_title.to_string(),
detail: String::default(),
tag_info: test_attack.to_string(),
record_information: Option::Some(test_recinfo.to_string()),
},
);
}
let expect_time = Utc
@@ -388,7 +385,7 @@ mod tests {
.unwrap();
let expect_tz = expect_time.with_timezone(&Local);
let expect =
"Timestamp,Computer,EventID,Level,RuleTitle,Details,MitreAttack,RulePath,FilePath\n"
"Timestamp,Computer,EventID,Level,MitreAttack,RuleTitle,Details,RecordInformation,RulePath,FilePath\n"
.to_string()
+ &expect_tz
.clone()
@@ -401,18 +398,19 @@ mod tests {
+ ","
+ test_level
+ ","
+ test_attack
+ ","
+ test_title
+ ","
+ output
+ ","
+ test_attack
+ test_recinfo
+ ","
+ testrulepath
+ ","
+ &testfilepath.to_string()
+ testfilepath
+ "\n";
let mut file: Box<dyn io::Write> =
Box::new(File::create("./test_emit_csv.csv".to_string()).unwrap());
let mut file: Box<dyn io::Write> = Box::new(File::create("./test_emit_csv.csv").unwrap());
assert!(emit_csv(&mut file, false, None).is_ok());
match read_to_string("./test_emit_csv.csv") {
Err(_) => panic!("Failed to open file."),
@@ -452,15 +450,19 @@ mod tests {
"##;
let event: Value = serde_json::from_str(val).unwrap();
messages.insert(
testfilepath.to_string(),
testrulepath.to_string(),
&event,
test_level.to_string(),
test_computername.to_string(),
test_eventid.to_string(),
test_title.to_string(),
output.to_string(),
test_attack.to_string(),
DetectInfo {
filepath: testfilepath.to_string(),
rulepath: testrulepath.to_string(),
level: test_level.to_string(),
computername: test_computername.to_string(),
eventid: test_eventid.to_string(),
alert: test_title.to_string(),
detail: String::default(),
tag_info: test_attack.to_string(),
record_information: Option::Some(String::default()),
},
);
messages.debug();
}
@@ -468,7 +470,8 @@ mod tests {
.datetime_from_str("1996-02-27T01:05:01Z", "%Y-%m-%dT%H:%M:%SZ")
.unwrap();
let expect_tz = expect_time.with_timezone(&Local);
let expect_header = "Timestamp|Computer|EventID|Level|RuleTitle|Details\n";
let expect_header =
"Timestamp|Computer|EventID|Level|RuleTitle|Details|RecordInformation\n";
let expect_colored = expect_header.to_string()
+ &get_white_color_string(
&expect_tz
@@ -486,6 +489,8 @@ mod tests {
+ &get_white_color_string(test_title)
+ " | "
+ &get_white_color_string(output)
+ " | "
+ &get_white_color_string("")
+ "\n";
let expect_nocoloed = expect_header.to_string()
+ &expect_tz
@@ -502,10 +507,12 @@ mod tests {
+ test_title
+ " | "
+ output
+ " | "
+ ""
+ "\n";
let mut file: Box<dyn io::Write> =
Box::new(File::create("./test_emit_csv_display.txt".to_string()).unwrap());
Box::new(File::create("./test_emit_csv_display.txt").unwrap());
assert!(emit_csv(&mut file, true, None).is_ok());
match read_to_string("./test_emit_csv_display.txt") {
Err(_) => panic!("Failed to open file."),
@@ -520,6 +527,6 @@ mod tests {
let white_color_header = "\u{1b}[38;2;255;255;255m";
let white_color_footer = "\u{1b}[0m";
return white_color_header.to_owned() + target + white_color_footer;
white_color_header.to_owned() + target + white_color_footer
}
}
src/detections/configs.rs
+140 -62
View File
@@ -1,10 +1,13 @@
use crate::detections::pivot::PivotKeyword;
use crate::detections::pivot::PIVOT_KEYWORD;
use crate::detections::print::AlertMessage;
use crate::detections::utils;
use chrono::{DateTime, Utc};
use clap::{App, AppSettings, ArgMatches};
use clap::{App, AppSettings, Arg, ArgMatches};
use hashbrown::HashMap;
use hashbrown::HashSet;
use lazy_static::lazy_static;
use regex::Regex;
use std::io::BufWriter;
use std::sync::RwLock;
lazy_static! {
@@ -16,24 +19,38 @@ lazy_static! {
levelmap.insert("MEDIUM".to_owned(), 3);
levelmap.insert("HIGH".to_owned(), 4);
levelmap.insert("CRITICAL".to_owned(), 5);
return levelmap;
levelmap
};
pub static ref EVENTKEY_ALIAS: EventKeyAliasConfig =
load_eventkey_alias("./rules/config/eventkey_alias.txt");
pub static ref EVENTKEY_ALIAS: EventKeyAliasConfig = load_eventkey_alias(&format!(
"{}/eventkey_alias.txt",
CONFIG.read().unwrap().folder_path
));
pub static ref IDS_REGEX: Regex =
Regex::new(r"^[0-9a-z]{8}-[0-9a-z]{4}-[0-9a-z]{4}-[0-9a-z]{4}-[0-9a-z]{12}$").unwrap();
}
#[derive(Clone)]
pub struct ConfigReader {
pub args: ArgMatches<'static>,
pub folder_path: String,
pub event_timeline_config: EventInfoConfig,
pub target_eventids: TargetEventIds,
}
impl Default for ConfigReader {
fn default() -> Self {
Self::new()
}
}
impl ConfigReader {
pub fn new() -> Self {
let arg = build_app();
let folder_path_str = arg.value_of("config").unwrap_or("rules/config").to_string();
ConfigReader {
args: build_app(),
event_timeline_config: load_eventcode_info("config/timeline_event_info.txt"),
args: arg,
folder_path: folder_path_str,
event_timeline_config: load_eventcode_info("config/statistics_event_info.txt"),
target_eventids: load_target_ids("config/target_eventids.txt"),
}
}
@@ -41,7 +58,7 @@ impl ConfigReader {
fn build_app<'a>() -> ArgMatches<'a> {
let program = std::env::args()
.nth(0)
.next()
.and_then(|s| {
std::path::PathBuf::from(s)
.file_stem()
@@ -55,8 +72,10 @@ fn build_app<'a>() -> ArgMatches<'a> {
let usages = "-d --directory=[DIRECTORY] 'Directory of multiple .evtx files.'
-f --filepath=[FILEPATH] 'File path to one .evtx file.'
-r --rules=[RULEFILE/RULEDIRECTORY] 'Rule file or directory. (Default: ./rules)'
-F --full-data 'Print all field information.'
-r --rules=[RULEDIRECTORY/RULEFILE] 'Rule file or directory (default: ./rules)'
-c --color 'Output with color. (Terminal needs to support True Color.)'
-C --config=[RULECONFIGDIRECTORY] 'Rule config folder. (Default: ./rules/config)'
-o --output=[CSV_TIMELINE] 'Save the timeline in CSV format. (Example: results.csv)'
-v --verbose 'Output verbose information.'
-D --enable-deprecated-rules 'Enable rules marked as deprecated.'
@@ -73,12 +92,18 @@ fn build_app<'a>() -> ArgMatches<'a> {
-s --statistics 'Prints statistics of event IDs.'
-q --quiet 'Quiet mode. Do not display the launch banner.'
-Q --quiet-errors 'Quiet errors mode. Do not save error logs.'
-p --pivot-keywords-list 'Create a list of pivot keywords.'
--contributors 'Prints the list of contributors.'";
App::new(&program)
.about("Hayabusa: Aiming to be the world's greatest Windows event log analysis tool!")
.version("1.1.0")
.author("Yamato Security (https://github.com/Yamato-Security/hayabusa)")
.version("1.2.0")
.author("Yamato Security (https://github.com/Yamato-Security/hayabusa) @SecurityYamato")
.setting(AppSettings::VersionlessSubcommands)
.arg(
// TODO: When update claps to 3.x, these can write in usage texts...
Arg::from_usage("--level-tuning=[LEVEL_TUNING_FILE] 'Adjust rule level.'")
.default_value("./config/level_tuning.txt"),
)
.usage(usages)
.args_from_usage(usages)
.get_matches()
@@ -91,7 +116,7 @@ fn is_test_mode() -> bool {
}
}
return false;
false
}
#[derive(Debug, Clone)]
@@ -99,19 +124,25 @@ pub struct TargetEventIds {
ids: HashSet<String>,
}
impl Default for TargetEventIds {
fn default() -> Self {
Self::new()
}
}
impl TargetEventIds {
pub fn new() -> TargetEventIds {
return TargetEventIds {
TargetEventIds {
ids: HashSet::new(),
};
}
}
pub fn is_target(&self, id: &String) -> bool {
pub fn is_target(&self, id: &str) -> bool {
// 中身が空の場合は全EventIdを対象とする。
if self.ids.is_empty() {
return true;
}
return self.ids.contains(id);
self.ids.contains(id)
}
}
@@ -121,7 +152,7 @@ fn load_target_ids(path: &str) -> TargetEventIds {
if lines.is_err() {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
&lines.as_ref().unwrap_err(),
lines.as_ref().unwrap_err(),
)
.ok();
return ret;
@@ -134,7 +165,7 @@ fn load_target_ids(path: &str) -> TargetEventIds {
ret.ids.insert(line);
}
return ret;
ret
}
#[derive(Debug, Clone)]
@@ -143,6 +174,12 @@ pub struct TargetEventTime {
end_time: Option<DateTime<Utc>>,
}
impl Default for TargetEventTime {
fn default() -> Self {
Self::new()
}
}
impl TargetEventTime {
pub fn new() -> Self {
let start_time =
@@ -180,17 +217,17 @@ impl TargetEventTime {
} else {
None
};
return Self::set(start_time, end_time);
Self::set(start_time, end_time)
}
pub fn set(
start_time: Option<chrono::DateTime<chrono::Utc>>,
end_time: Option<chrono::DateTime<chrono::Utc>>,
input_start_time: Option<chrono::DateTime<chrono::Utc>>,
input_end_time: Option<chrono::DateTime<chrono::Utc>>,
) -> Self {
return Self {
start_time: start_time,
end_time: end_time,
};
Self {
start_time: input_start_time,
end_time: input_end_time,
}
}
pub fn is_target(&self, eventtime: &Option<DateTime<Utc>>) -> bool {
@@ -207,7 +244,7 @@ impl TargetEventTime {
return false;
}
}
return true;
true
}
}
@@ -219,34 +256,41 @@ pub struct EventKeyAliasConfig {
impl EventKeyAliasConfig {
pub fn new() -> EventKeyAliasConfig {
return EventKeyAliasConfig {
EventKeyAliasConfig {
key_to_eventkey: HashMap::new(),
key_to_split_eventkey: HashMap::new(),
};
}
}
pub fn get_event_key(&self, alias: &String) -> Option<&String> {
return self.key_to_eventkey.get(alias);
pub fn get_event_key(&self, alias: &str) -> Option<&String> {
self.key_to_eventkey.get(alias)
}
pub fn get_event_key_split(&self, alias: &String) -> Option<&Vec<usize>> {
return self.key_to_split_eventkey.get(alias);
pub fn get_event_key_split(&self, alias: &str) -> Option<&Vec<usize>> {
self.key_to_split_eventkey.get(alias)
}
}
impl Default for EventKeyAliasConfig {
fn default() -> Self {
Self::new()
}
}
fn load_eventkey_alias(path: &str) -> EventKeyAliasConfig {
let mut config = EventKeyAliasConfig::new();
// eventkey_aliasが読み込めなかったらエラーで終了とする。
let read_result = utils::read_csv(path);
if read_result.is_err() {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
&read_result.as_ref().unwrap_err(),
read_result.as_ref().unwrap_err(),
)
.ok();
return config;
}
// eventkey_aliasが読み込めなかったらエラーで終了とする。
read_result.unwrap().into_iter().for_each(|line| {
if line.len() != 2 {
return;
@@ -255,39 +299,71 @@ fn load_eventkey_alias(path: &str) -> EventKeyAliasConfig {
let empty = &"".to_string();
let alias = line.get(0).unwrap_or(empty);
let event_key = line.get(1).unwrap_or(empty);
if alias.len() == 0 || event_key.len() == 0 {
if alias.is_empty() || event_key.is_empty() {
return;
}
config
.key_to_eventkey
.insert(alias.to_owned(), event_key.to_owned());
let splits = event_key.split(".").map(|s| s.len()).collect();
let splits = event_key.split('.').map(|s| s.len()).collect();
config
.key_to_split_eventkey
.insert(alias.to_owned(), splits);
});
config.key_to_eventkey.shrink_to_fit();
return config;
config
}
///設定ファイルを読み込み、keyとfieldsのマップをPIVOT_KEYWORD大域変数にロードする。
pub fn load_pivot_keywords(path: &str) {
let read_result = utils::read_txt(path);
if read_result.is_err() {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
read_result.as_ref().unwrap_err(),
)
.ok();
}
read_result.unwrap().into_iter().for_each(|line| {
let map: Vec<&str> = line.split('.').collect();
if map.len() != 2 {
return;
}
//存在しなければ、keyを作成
PIVOT_KEYWORD
.write()
.unwrap()
.entry(map[0].to_string())
.or_insert(PivotKeyword::new());
PIVOT_KEYWORD
.write()
.unwrap()
.get_mut(&map[0].to_string())
.unwrap()
.fields
.insert(map[1].to_string());
});
}
#[derive(Debug, Clone)]
pub struct EventInfo {
pub evttitle: String,
pub detectflg: String,
pub comment: String,
}
impl Default for EventInfo {
fn default() -> Self {
Self::new()
}
}
impl EventInfo {
pub fn new() -> EventInfo {
let evttitle = "Unknown".to_string();
let detectflg = "".to_string();
let comment = "".to_string();
return EventInfo {
evttitle,
detectflg,
comment,
};
EventInfo { evttitle }
}
}
#[derive(Debug, Clone)]
@@ -295,14 +371,20 @@ pub struct EventInfoConfig {
eventinfo: HashMap<String, EventInfo>,
}
impl Default for EventInfoConfig {
fn default() -> Self {
Self::new()
}
}
impl EventInfoConfig {
pub fn new() -> EventInfoConfig {
return EventInfoConfig {
EventInfoConfig {
eventinfo: HashMap::new(),
};
}
}
pub fn get_event_id(&self, eventid: &String) -> Option<&EventInfo> {
return self.eventinfo.get(eventid);
pub fn get_event_id(&self, eventid: &str) -> Option<&EventInfo> {
self.eventinfo.get(eventid)
}
}
@@ -313,33 +395,29 @@ fn load_eventcode_info(path: &str) -> EventInfoConfig {
if read_result.is_err() {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
&read_result.as_ref().unwrap_err(),
read_result.as_ref().unwrap_err(),
)
.ok();
return config;
}
// timeline_event_infoが読み込めなかったらエラーで終了とする。
// statistics_event_infoが読み込めなかったらエラーで終了とする。
read_result.unwrap().into_iter().for_each(|line| {
if line.len() != 4 {
if line.len() != 2 {
return;
}
let empty = &"".to_string();
let eventcode = line.get(0).unwrap_or(empty);
let event_title = line.get(1).unwrap_or(empty);
let detect_flg = line.get(2).unwrap_or(empty);
let comment = line.get(3).unwrap_or(empty);
infodata = EventInfo {
evttitle: event_title.to_string(),
detectflg: detect_flg.to_string(),
comment: comment.to_string(),
};
config
.eventinfo
.insert(eventcode.to_owned(), infodata.to_owned());
});
return config;
config
}
#[cfg(test)]
@@ -375,9 +453,9 @@ mod tests {
let within_range = Some("2019-02-27T01:05:01Z".parse::<DateTime<Utc>>().unwrap());
let out_of_range2 = Some("2021-02-27T01:05:01Z".parse::<DateTime<Utc>>().unwrap());
assert_eq!(time_filter.is_target(&out_of_range1), false);
assert_eq!(time_filter.is_target(&within_range), true);
assert_eq!(time_filter.is_target(&out_of_range2), false);
assert!(!time_filter.is_target(&out_of_range1));
assert!(time_filter.is_target(&within_range));
assert!(!time_filter.is_target(&out_of_range2));
}
#[test]
@@ -386,7 +464,7 @@ mod tests {
let end_time = Some("2020-03-30T12:00:09Z".parse::<DateTime<Utc>>().unwrap());
let time_filter = configs::TargetEventTime::set(start_time, end_time);
assert_eq!(time_filter.is_target(&start_time), true);
assert_eq!(time_filter.is_target(&end_time), true);
assert!(time_filter.is_target(&start_time));
assert!(time_filter.is_target(&end_time));
}
}
src/detections/detection.rs
+80 -51
View File
@@ -1,11 +1,15 @@
extern crate csv;
use crate::detections::configs;
use crate::detections::pivot::insert_pivot_keyword;
use crate::detections::print::AlertMessage;
use crate::detections::print::DetectInfo;
use crate::detections::print::ERROR_LOG_STACK;
use crate::detections::print::MESSAGES;
use crate::detections::print::PIVOT_KEYWORD_LIST_FLAG;
use crate::detections::print::QUIET_ERRORS_FLAG;
use crate::detections::print::STATISTICS_FLAG;
use crate::detections::print::TAGS_CONFIG;
use crate::detections::rule;
use crate::detections::rule::AggResult;
use crate::detections::rule::RuleNode;
@@ -28,11 +32,12 @@ pub struct EvtxRecordInfo {
pub record: Value, // 1レコード分のデータをJSON形式にシリアライズしたもの
pub data_string: String,
pub key_2_value: hashbrown::HashMap<String, String>,
pub record_information: Option<String>,
}
impl EvtxRecordInfo {
pub fn get_value(&self, key: &String) -> Option<&String> {
return self.key_2_value.get(key);
pub fn get_value(&self, key: &str) -> Option<&String> {
self.key_2_value.get(key)
}
}
@@ -42,12 +47,12 @@ pub struct Detection {
}
impl Detection {
pub fn new(rules: Vec<RuleNode>) -> Detection {
return Detection { rules: rules };
pub fn new(rule_nodes: Vec<RuleNode>) -> Detection {
Detection { rules: rule_nodes }
}
pub fn start(self, rt: &Runtime, records: Vec<EvtxRecordInfo>) -> Self {
return rt.block_on(self.execute_rules(records));
rt.block_on(self.execute_rules(records))
}
// ルールファイルをパースします。
@@ -104,9 +109,9 @@ impl Detection {
});
}
parseerror_count += 1;
println!(""); // 一行開けるためのprintln
println!(); // 一行開けるためのprintln
});
return Option::None;
Option::None
};
// parse rule files
let ret = rulefile_loader
@@ -120,7 +125,7 @@ impl Detection {
&parseerror_count,
&rulefile_loader.ignorerule_count,
);
return ret;
ret
}
// 複数のイベントレコードに対して、複数のルールを1個実行します。
@@ -132,10 +137,7 @@ impl Detection {
.into_iter()
.map(|rule| {
let records_cloned = Arc::clone(&records_arc);
return spawn(async move {
let moved_rule = Detection::execute_rule(rule, records_cloned);
return moved_rule;
});
spawn(async move { Detection::execute_rule(rule, records_cloned) })
})
.collect();
@@ -151,7 +153,7 @@ impl Detection {
// self.rulesが再度所有権を取り戻せるように、Detection::execute_ruleで引数に渡したruleを戻り値として返すようにしている。
self.rules = rules;
return self;
self
}
pub fn add_aggcondition_msges(self, rt: &Runtime) {
@@ -175,17 +177,23 @@ impl Detection {
fn execute_rule(mut rule: RuleNode, records: Arc<Vec<EvtxRecordInfo>>) -> RuleNode {
let agg_condition = rule.has_agg_condition();
for record_info in records.as_ref() {
let result = rule.select(&record_info);
let result = rule.select(record_info);
if !result {
continue;
}
if *PIVOT_KEYWORD_LIST_FLAG {
insert_pivot_keyword(&record_info.record);
continue;
}
// aggregation conditionが存在しない場合はそのまま出力対応を行う
if !agg_condition {
Detection::insert_message(&rule, &record_info);
Detection::insert_message(&rule, record_info);
}
}
return rule;
rule
}
/// 条件に合致したレコードを表示するための関数
@@ -193,23 +201,33 @@ impl Detection {
let tag_info: Vec<String> = rule.yaml["tags"]
.as_vec()
.unwrap_or(&Vec::default())
.into_iter()
.map(|info| info.as_str().unwrap_or("").replace("attack.", ""))
.iter()
.filter_map(|info| TAGS_CONFIG.get(info.as_str().unwrap_or(&String::default())))
.map(|str| str.to_owned())
.collect();
MESSAGES.lock().unwrap().insert(
record_info.evtx_filepath.to_string(),
rule.rulepath.to_string(),
&record_info.record,
rule.yaml["level"].as_str().unwrap_or("-").to_string(),
record_info.record["Event"]["System"]["Computer"]
let recinfo = record_info
.record_information
.as_ref()
.map(|recinfo| recinfo.to_string());
let detect_info = DetectInfo {
filepath: record_info.evtx_filepath.to_string(),
rulepath: rule.rulepath.to_string(),
level: rule.yaml["level"].as_str().unwrap_or("-").to_string(),
computername: record_info.record["Event"]["System"]["Computer"]
.to_string()
.replace("\"", ""),
get_serde_number_to_string(&record_info.record["Event"]["System"]["EventID"])
.unwrap_or("-".to_owned())
.to_string(),
rule.yaml["title"].as_str().unwrap_or("").to_string(),
.replace('\"', ""),
eventid: get_serde_number_to_string(&record_info.record["Event"]["System"]["EventID"])
.unwrap_or_else(|| "-".to_owned()),
alert: rule.yaml["title"].as_str().unwrap_or("").to_string(),
detail: String::default(),
tag_info: tag_info.join(" | "),
record_information: recinfo,
};
MESSAGES.lock().unwrap().insert(
&record_info.record,
rule.yaml["details"].as_str().unwrap_or("").to_string(),
tag_info.join(" : "),
detect_info,
);
}
@@ -218,21 +236,32 @@ impl Detection {
let tag_info: Vec<String> = rule.yaml["tags"]
.as_vec()
.unwrap_or(&Vec::default())
.into_iter()
.map(|info| info.as_str().unwrap_or("").replace("attack.", ""))
.iter()
.filter_map(|info| TAGS_CONFIG.get(info.as_str().unwrap_or(&String::default())))
.map(|str| str.to_owned())
.collect();
let output = Detection::create_count_output(rule, &agg_result);
MESSAGES.lock().unwrap().insert_message(
"-".to_owned(),
rule.rulepath.to_owned(),
agg_result.start_timedate,
rule.yaml["level"].as_str().unwrap_or("").to_owned(),
"-".to_owned(),
"-".to_owned(),
rule.yaml["title"].as_str().unwrap_or("").to_owned(),
output.to_owned(),
tag_info.join(" : "),
)
let rec_info = if configs::CONFIG.read().unwrap().args.is_present("full-data") {
Option::Some(String::default())
} else {
Option::None
};
let detect_info = DetectInfo {
filepath: "-".to_owned(),
rulepath: rule.rulepath.to_owned(),
level: rule.yaml["level"].as_str().unwrap_or("").to_owned(),
computername: "-".to_owned(),
eventid: "-".to_owned(),
alert: rule.yaml["title"].as_str().unwrap_or("").to_owned(),
detail: output,
record_information: rec_info,
tag_info: tag_info.join(" : "),
};
MESSAGES
.lock()
.unwrap()
.insert_message(detect_info, agg_result.start_timedate)
}
///aggregation conditionのcount部分の検知出力文の文字列を返す関数
@@ -242,15 +271,11 @@ impl Detection {
let agg_condition_raw_str: Vec<&str> = rule.yaml["detection"]["condition"]
.as_str()
.unwrap()
.split("|")
.split('|')
.collect();
// この関数が呼び出されている段階で既にaggregation conditionは存在する前提なのでunwrap前の確認は行わない
let agg_condition = rule.get_agg_condition().unwrap();
let exist_timeframe = rule.yaml["detection"]["timeframe"]
.as_str()
.unwrap_or("")
.to_string()
!= "";
let exist_timeframe = rule.yaml["detection"]["timeframe"].as_str().unwrap_or("") != "";
// この関数が呼び出されている段階で既にaggregation conditionは存在する前提なのでagg_conditionの配列の長さは2となる
ret.push_str(agg_condition_raw_str[1].trim());
if exist_timeframe {
@@ -281,8 +306,9 @@ impl Detection {
));
}
return ret;
ret
}
pub fn print_rule_load_info(
rc: &HashMap<String, u128>,
parseerror_count: &u128,
@@ -302,7 +328,7 @@ impl Detection {
"Total enabled detection rules: {}",
total - ignore_count - parseerror_count
);
println!("");
println!();
}
}
@@ -498,4 +524,7 @@ mod tests {
expected_output
);
}
#[test]
fn test_create_fields_value() {}
}
src/detections/mod.rs
+1
View File
@@ -1,5 +1,6 @@
pub mod configs;
pub mod detection;
pub mod pivot;
pub mod print;
pub mod rule;
pub mod utils;
src/detections/pivot.rs
+270
View File
@@ -0,0 +1,270 @@
use hashbrown::HashMap;
use hashbrown::HashSet;
use lazy_static::lazy_static;
use serde_json::Value;
use std::sync::RwLock;
use crate::detections::configs;
use crate::detections::utils::get_serde_number_to_string;
#[derive(Debug)]
pub struct PivotKeyword {
pub keywords: HashSet<String>,
pub fields: HashSet<String>,
}
lazy_static! {
pub static ref PIVOT_KEYWORD: RwLock<HashMap<String, PivotKeyword>> =
RwLock::new(HashMap::new());
}
impl Default for PivotKeyword {
fn default() -> Self {
Self::new()
}
}
impl PivotKeyword {
pub fn new() -> PivotKeyword {
PivotKeyword {
keywords: HashSet::new(),
fields: HashSet::new(),
}
}
}
///levelがlowより大きいレコードの場合、keywordがrecord内にみつかれば、
///それをPIVOT_KEYWORD.keywordsに入れる。
pub fn insert_pivot_keyword(event_record: &Value) {
//levelがlow異常なら続ける
let mut is_exist_event_key = false;
let mut tmp_event_record: &Value = event_record;
for s in ["Event", "System", "Level"] {
if let Some(record) = tmp_event_record.get(s) {
is_exist_event_key = true;
tmp_event_record = record;
}
}
if is_exist_event_key {
let hash_value = get_serde_number_to_string(tmp_event_record);
if hash_value.is_some() && hash_value.as_ref().unwrap() == "infomational"
|| hash_value.as_ref().unwrap() == "undefined"
|| hash_value.as_ref().unwrap() == "-"
{
return;
}
} else {
return;
}
for (_, pivot) in PIVOT_KEYWORD.write().unwrap().iter_mut() {
for field in &pivot.fields {
if let Some(array_str) = configs::EVENTKEY_ALIAS.get_event_key(&String::from(field)) {
let split: Vec<&str> = array_str.split('.').collect();
let mut is_exist_event_key = false;
let mut tmp_event_record: &Value = event_record;
for s in split {
if let Some(record) = tmp_event_record.get(s) {
is_exist_event_key = true;
tmp_event_record = record;
}
}
if is_exist_event_key {
let hash_value = get_serde_number_to_string(tmp_event_record);
if let Some(value) = hash_value {
if value == "-" || value == "127.0.0.1" || value == "::1" {
continue;
}
pivot.keywords.insert(value);
};
}
}
}
}
}
#[cfg(test)]
mod tests {
use crate::detections::configs::load_pivot_keywords;
use crate::detections::pivot::insert_pivot_keyword;
use crate::detections::pivot::PIVOT_KEYWORD;
use serde_json;
//PIVOT_KEYWORDはグローバルなので、他の関数の影響も考慮する必要がある。
#[test]
fn insert_pivot_keyword_local_ip4() {
load_pivot_keywords("test_files/config/pivot_keywords.txt");
let record_json_str = r#"
{
"Event": {
"System": {
"Level": "high"
},
"EventData": {
"IpAddress": "127.0.0.1"
}
}
}"#;
insert_pivot_keyword(&serde_json::from_str(record_json_str).unwrap());
assert!(!PIVOT_KEYWORD
.write()
.unwrap()
.get_mut("Ip Addresses")
.unwrap()
.keywords
.contains("127.0.0.1"));
}
#[test]
fn insert_pivot_keyword_ip4() {
load_pivot_keywords("test_files/config/pivot_keywords.txt");
let record_json_str = r#"
{
"Event": {
"System": {
"Level": "high"
},
"EventData": {
"IpAddress": "10.0.0.1"
}
}
}"#;
insert_pivot_keyword(&serde_json::from_str(record_json_str).unwrap());
assert!(PIVOT_KEYWORD
.write()
.unwrap()
.get_mut("Ip Addresses")
.unwrap()
.keywords
.contains("10.0.0.1"));
}
#[test]
fn insert_pivot_keyword_ip_empty() {
load_pivot_keywords("test_files/config/pivot_keywords.txt");
let record_json_str = r#"
{
"Event": {
"System": {
"Level": "high"
},
"EventData": {
"IpAddress": "-"
}
}
}"#;
insert_pivot_keyword(&serde_json::from_str(record_json_str).unwrap());
assert!(!PIVOT_KEYWORD
.write()
.unwrap()
.get_mut("Ip Addresses")
.unwrap()
.keywords
.contains("-"));
}
#[test]
fn insert_pivot_keyword_local_ip6() {
load_pivot_keywords("test_files/config/pivot_keywords.txt");
let record_json_str = r#"
{
"Event": {
"System": {
"Level": "high"
},
"EventData": {
"IpAddress": "::1"
}
}
}"#;
insert_pivot_keyword(&serde_json::from_str(record_json_str).unwrap());
assert!(!PIVOT_KEYWORD
.write()
.unwrap()
.get_mut("Ip Addresses")
.unwrap()
.keywords
.contains("::1"));
}
#[test]
fn insert_pivot_keyword_level_infomational() {
load_pivot_keywords("test_files/config/pivot_keywords.txt");
let record_json_str = r#"
{
"Event": {
"System": {
"Level": "infomational"
},
"EventData": {
"IpAddress": "10.0.0.2"
}
}
}"#;
insert_pivot_keyword(&serde_json::from_str(record_json_str).unwrap());
assert!(!PIVOT_KEYWORD
.write()
.unwrap()
.get_mut("Ip Addresses")
.unwrap()
.keywords
.contains("10.0.0.2"));
}
#[test]
fn insert_pivot_keyword_level_low() {
load_pivot_keywords("test_files/config/pivot_keywords.txt");
let record_json_str = r#"
{
"Event": {
"System": {
"Level": "low"
},
"EventData": {
"IpAddress": "10.0.0.1"
}
}
}"#;
insert_pivot_keyword(&serde_json::from_str(record_json_str).unwrap());
assert!(PIVOT_KEYWORD
.write()
.unwrap()
.get_mut("Ip Addresses")
.unwrap()
.keywords
.contains("10.0.0.1"));
}
#[test]
fn insert_pivot_keyword_level_none() {
load_pivot_keywords("test_files/config/pivot_keywords.txt");
let record_json_str = r#"
{
"Event": {
"System": {
"Level": "-"
},
"EventData": {
"IpAddress": "10.0.0.3"
}
}
}"#;
insert_pivot_keyword(&serde_json::from_str(record_json_str).unwrap());
assert!(!PIVOT_KEYWORD
.write()
.unwrap()
.get_mut("Ip Addresses")
.unwrap()
.keywords
.contains("10.0.0.3"));
}
}
src/detections/print.rs
+168 -132
View File
@@ -2,8 +2,6 @@ extern crate lazy_static;
use crate::detections::configs;
use crate::detections::utils;
use crate::detections::utils::get_serde_number_to_string;
use crate::filter::DataFilterRule;
use crate::filter::FILTER_REGEX;
use chrono::{DateTime, Local, TimeZone, Utc};
use hashbrown::HashMap;
use lazy_static::lazy_static;
@@ -33,6 +31,7 @@ pub struct DetectInfo {
pub alert: String,
pub detail: String,
pub tag_info: String,
pub record_information: Option<String>,
}
pub struct AlertMessage {}
@@ -55,6 +54,19 @@ lazy_static! {
.unwrap()
.args
.is_present("statistics");
pub static ref TAGS_CONFIG: HashMap<String, String> =
Message::create_tags_config("config/output_tag.txt");
pub static ref PIVOT_KEYWORD_LIST_FLAG: bool = configs::CONFIG
.read()
.unwrap()
.args
.is_present("pivot-keywords-list");
}
impl Default for Message {
fn default() -> Self {
Self::new()
}
}
impl Message {
@@ -63,74 +75,54 @@ impl Message {
Message { map: messages }
}
/// メッセージの設定を行う関数。aggcondition対応のためrecordではなく出力をする対象時間がDatetime形式での入力としている
pub fn insert_message(
&mut self,
target_file: String,
rule_path: String,
event_time: DateTime<Utc>,
level: String,
computername: String,
eventid: String,
event_title: String,
event_detail: String,
tag_info: String,
) {
let detect_info = DetectInfo {
filepath: target_file,
rulepath: rule_path,
level: level,
computername: computername,
eventid: eventid,
alert: event_title,
detail: event_detail,
tag_info: tag_info,
};
/// ファイルパスで記載されたtagでのフル名、表示の際に置き換えられる文字列のHashMapを作成する関数。tagではこのHashMapのキーに対応しない出力は出力しないものとする
/// ex. attack.impact,Impact
pub fn create_tags_config(path: &str) -> HashMap<String, String> {
let read_result = utils::read_csv(path);
if read_result.is_err() {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
read_result.as_ref().unwrap_err(),
)
.ok();
return HashMap::default();
}
let mut ret: HashMap<String, String> = HashMap::new();
read_result.unwrap().into_iter().for_each(|line| {
if line.len() != 2 {
return;
}
match self.map.get_mut(&event_time) {
Some(v) => {
v.push(detect_info);
}
None => {
let m = vec![detect_info; 1];
self.map.insert(event_time, m);
}
let empty = &"".to_string();
let tag_full_str = line.get(0).unwrap_or(empty).trim();
let tag_replace_str = line.get(1).unwrap_or(empty).trim();
ret.insert(tag_full_str.to_owned(), tag_replace_str.to_owned());
});
ret
}
/// メッセージの設定を行う関数。aggcondition対応のためrecordではなく出力をする対象時間がDatetime形式での入力としている
pub fn insert_message(&mut self, detect_info: DetectInfo, event_time: DateTime<Utc>) {
if let Some(v) = self.map.get_mut(&event_time) {
v.push(detect_info);
} else {
let m = vec![detect_info; 1];
self.map.insert(event_time, m);
}
}
/// メッセージを設定
pub fn insert(
&mut self,
target_file: String,
rule_path: String,
event_record: &Value,
level: String,
computername: String,
eventid: String,
event_title: String,
output: String,
tag_info: String,
) {
let message = &self.parse_message(event_record, output);
pub fn insert(&mut self, event_record: &Value, output: String, mut detect_info: DetectInfo) {
detect_info.detail = self.parse_message(event_record, output);
let default_time = Utc.ymd(1970, 1, 1).and_hms(0, 0, 0);
let time = Message::get_event_time(event_record).unwrap_or(default_time);
self.insert_message(
target_file,
rule_path,
time,
level,
computername,
eventid,
event_title,
message.to_string(),
tag_info,
)
self.insert_message(detect_info, time)
}
fn parse_message(&mut self, event_record: &Value, output: String) -> String {
let mut return_message: String = output;
let mut hash_map: HashMap<String, String> = HashMap::new();
let mut output_filter: Option<&DataFilterRule> = None;
for caps in ALIASREGEX.captures_iter(&return_message) {
let full_target_str = &caps[0];
let target_length = full_target_str.chars().count() - 2; // The meaning of 2 is two percent
@@ -140,26 +132,29 @@ impl Message {
.take(target_length)
.collect::<String>();
if let Some(array_str) = configs::EVENTKEY_ALIAS.get_event_key(&target_str) {
let split: Vec<&str> = array_str.split(".").collect();
let mut is_exist_event_key = false;
let mut tmp_event_record: &Value = event_record.into();
for s in &split {
if let Some(record) = tmp_event_record.get(s) {
is_exist_event_key = true;
tmp_event_record = record;
output_filter = FILTER_REGEX.get(&s.to_string());
}
let array_str =
if let Some(_array_str) = configs::EVENTKEY_ALIAS.get_event_key(&target_str) {
_array_str.to_string()
} else {
"Event.EventData.".to_owned() + &target_str
};
let split: Vec<&str> = array_str.split('.').collect();
let mut is_exist_event_key = false;
let mut tmp_event_record: &Value = event_record;
for s in &split {
if let Some(record) = tmp_event_record.get(s) {
is_exist_event_key = true;
tmp_event_record = record;
}
if is_exist_event_key {
let mut hash_value = get_serde_number_to_string(tmp_event_record);
if hash_value.is_some() {
if output_filter.is_some() {
hash_value =
utils::replace_target_character(hash_value.as_ref(), output_filter);
}
hash_map.insert(full_target_str.to_string(), hash_value.unwrap());
}
}
if is_exist_event_key {
let hash_value = get_serde_number_to_string(tmp_event_record);
if let Some(hash_value) = hash_value {
// UnicodeのWhitespace characterをそのままCSVに出力すると見難いので、スペースに変換する。なお、先頭と最後のWhitespace characterは単に削除される。
let hash_value: Vec<&str> = hash_value.split_whitespace().collect();
let hash_value = hash_value.join(" ");
hash_map.insert(full_target_str.to_string(), hash_value);
}
}
}
@@ -193,7 +188,7 @@ impl Message {
}
detect_count += detect_infos.len();
}
println!("");
println!();
println!("Total events:{:?}", detect_count);
}
@@ -224,44 +219,40 @@ impl AlertMessage {
}
let mut error_log_writer = BufWriter::new(File::create(path).unwrap());
error_log_writer
.write(
.write_all(
format!(
"user input: {:?}\n",
format_args!(
"{}",
env::args()
.map(|arg| arg)
.collect::<Vec<String>>()
.join(" ")
)
format_args!("{}", env::args().collect::<Vec<String>>().join(" "))
)
.as_bytes(),
)
.unwrap();
.ok();
for error_log in ERROR_LOG_STACK.lock().unwrap().iter() {
writeln!(error_log_writer, "{}", error_log).ok();
}
println!(
"Errors were generated. Please check {} for details.",
ERROR_LOG_PATH.to_string()
*ERROR_LOG_PATH
);
println!("");
println!();
}
/// ERRORメッセージを表示する関数
pub fn alert<W: Write>(w: &mut W, contents: &String) -> io::Result<()> {
pub fn alert<W: Write>(w: &mut W, contents: &str) -> io::Result<()> {
writeln!(w, "[ERROR] {}", contents)
}
/// WARNメッセージを表示する関数
pub fn warn<W: Write>(w: &mut W, contents: &String) -> io::Result<()> {
pub fn warn<W: Write>(w: &mut W, contents: &str) -> io::Result<()> {
writeln!(w, "[WARN] {}", contents)
}
}
#[cfg(test)]
mod tests {
use crate::detections::print::DetectInfo;
use crate::detections::print::{AlertMessage, Message};
use hashbrown::HashMap;
use serde_json::Value;
use std::io::BufWriter;
@@ -284,15 +275,19 @@ mod tests {
"##;
let event_record_1: Value = serde_json::from_str(json_str_1).unwrap();
message.insert(
"a".to_string(),
"test_rule".to_string(),
&event_record_1,
"high".to_string(),
"testcomputer1".to_string(),
"1".to_string(),
"test1".to_string(),
"CommandLine1: %CommandLine%".to_string(),
"txxx.001".to_string(),
DetectInfo {
filepath: "a".to_string(),
rulepath: "test_rule".to_string(),
level: "high".to_string(),
computername: "testcomputer1".to_string(),
eventid: "1".to_string(),
alert: "test1".to_string(),
detail: String::default(),
tag_info: "txxx.001".to_string(),
record_information: Option::Some("record_information1".to_string()),
},
);
let json_str_2 = r##"
@@ -311,15 +306,19 @@ mod tests {
"##;
let event_record_2: Value = serde_json::from_str(json_str_2).unwrap();
message.insert(
"a".to_string(),
"test_rule2".to_string(),
&event_record_2,
"high".to_string(),
"testcomputer2".to_string(),
"2".to_string(),
"test2".to_string(),
"CommandLine2: %CommandLine%".to_string(),
"txxx.002".to_string(),
DetectInfo {
filepath: "a".to_string(),
rulepath: "test_rule2".to_string(),
level: "high".to_string(),
computername: "testcomputer2".to_string(),
eventid: "2".to_string(),
alert: "test2".to_string(),
detail: String::default(),
tag_info: "txxx.002".to_string(),
record_information: Option::Some("record_information2".to_string()),
},
);
let json_str_3 = r##"
@@ -338,15 +337,19 @@ mod tests {
"##;
let event_record_3: Value = serde_json::from_str(json_str_3).unwrap();
message.insert(
"a".to_string(),
"test_rule3".to_string(),
&event_record_3,
"high".to_string(),
"testcomputer3".to_string(),
"3".to_string(),
"test3".to_string(),
"CommandLine3: %CommandLine%".to_string(),
"txxx.003".to_string(),
DetectInfo {
filepath: "a".to_string(),
rulepath: "test_rule3".to_string(),
level: "high".to_string(),
computername: "testcomputer3".to_string(),
eventid: "3".to_string(),
alert: "test3".to_string(),
detail: String::default(),
tag_info: "txxx.003".to_string(),
record_information: Option::Some("record_information3".to_string()),
},
);
let json_str_4 = r##"
@@ -360,41 +363,39 @@ mod tests {
"##;
let event_record_4: Value = serde_json::from_str(json_str_4).unwrap();
message.insert(
"a".to_string(),
"test_rule4".to_string(),
&event_record_4,
"medium".to_string(),
"testcomputer4".to_string(),
"4".to_string(),
"test4".to_string(),
"CommandLine4: %CommandLine%".to_string(),
"txxx.004".to_string(),
DetectInfo {
filepath: "a".to_string(),
rulepath: "test_rule4".to_string(),
level: "medium".to_string(),
computername: "testcomputer4".to_string(),
eventid: "4".to_string(),
alert: "test4".to_string(),
detail: String::default(),
tag_info: "txxx.004".to_string(),
record_information: Option::Some("record_information4".to_string()),
},
);
let display = format!("{}", format_args!("{:?}", message));
println!("display::::{}", display);
let expect = "Message { map: {1970-01-01T00:00:00Z: [DetectInfo { filepath: \"a\", rulepath: \"test_rule4\", level: \"medium\", computername: \"testcomputer4\", eventid: \"4\", alert: \"test4\", detail: \"CommandLine4: hoge\", tag_info: \"txxx.004\" }], 1996-02-27T01:05:01Z: [DetectInfo { filepath: \"a\", rulepath: \"test_rule\", level: \"high\", computername: \"testcomputer1\", eventid: \"1\", alert: \"test1\", detail: \"CommandLine1: hoge\", tag_info: \"txxx.001\" }, DetectInfo { filepath: \"a\", rulepath: \"test_rule2\", level: \"high\", computername: \"testcomputer2\", eventid: \"2\", alert: \"test2\", detail: \"CommandLine2: hoge\", tag_info: \"txxx.002\" }], 2000-01-21T09:06:01Z: [DetectInfo { filepath: \"a\", rulepath: \"test_rule3\", level: \"high\", computername: \"testcomputer3\", eventid: \"3\", alert: \"test3\", detail: \"CommandLine3: hoge\", tag_info: \"txxx.003\" }]} }";
let expect = "Message { map: {1970-01-01T00:00:00Z: [DetectInfo { filepath: \"a\", rulepath: \"test_rule4\", level: \"medium\", computername: \"testcomputer4\", eventid: \"4\", alert: \"test4\", detail: \"CommandLine4: hoge\", tag_info: \"txxx.004\", record_information: Some(\"record_information4\") }], 1996-02-27T01:05:01Z: [DetectInfo { filepath: \"a\", rulepath: \"test_rule\", level: \"high\", computername: \"testcomputer1\", eventid: \"1\", alert: \"test1\", detail: \"CommandLine1: hoge\", tag_info: \"txxx.001\", record_information: Some(\"record_information1\") }, DetectInfo { filepath: \"a\", rulepath: \"test_rule2\", level: \"high\", computername: \"testcomputer2\", eventid: \"2\", alert: \"test2\", detail: \"CommandLine2: hoge\", tag_info: \"txxx.002\", record_information: Some(\"record_information2\") }], 2000-01-21T09:06:01Z: [DetectInfo { filepath: \"a\", rulepath: \"test_rule3\", level: \"high\", computername: \"testcomputer3\", eventid: \"3\", alert: \"test3\", detail: \"CommandLine3: hoge\", tag_info: \"txxx.003\", record_information: Some(\"record_information3\") }]} }";
assert_eq!(display, expect);
}
#[test]
fn test_error_message() {
let input = "TEST!";
AlertMessage::alert(
&mut BufWriter::new(std::io::stdout().lock()),
&input.to_string(),
)
.expect("[ERROR] TEST!");
AlertMessage::alert(&mut BufWriter::new(std::io::stdout().lock()), input)
.expect("[ERROR] TEST!");
}
#[test]
fn test_warn_message() {
let input = "TESTWarn!";
AlertMessage::warn(
&mut BufWriter::new(std::io::stdout().lock()),
&input.to_string(),
)
.expect("[WARN] TESTWarn!");
AlertMessage::warn(&mut BufWriter::new(std::io::stdout().lock()), input)
.expect("[WARN] TESTWarn!");
}
#[test]
@@ -426,6 +427,27 @@ mod tests {
expected,
);
}
#[test]
fn test_parse_message_auto_search() {
let mut message = Message::new();
let json_str = r##"
{
"Event": {
"EventData": {
"NoAlias": "no_alias"
}
}
}
"##;
let event_record: Value = serde_json::from_str(json_str).unwrap();
let expected = "alias:no_alias";
assert_eq!(
message.parse_message(&event_record, "alias:%NoAlias%".to_owned()),
expected,
);
}
#[test]
/// outputで指定されているキーが、eventkey_alias.txt内で設定されていない場合の出力テスト
fn test_parse_message_not_exist_key_in_output() {
@@ -445,9 +467,9 @@ mod tests {
}
"##;
let event_record: Value = serde_json::from_str(json_str).unwrap();
let expected = "NoExistKey:%TESTNoExistKey%";
let expected = "NoExistAlias:%NoAliasNoHit%";
assert_eq!(
message.parse_message(&event_record, "NoExistKey:%TESTNoExistKey%".to_owned()),
message.parse_message(&event_record, "NoExistAlias:%NoAliasNoHit%".to_owned()),
expected,
);
}
@@ -479,4 +501,18 @@ mod tests {
expected,
);
}
#[test]
/// output_tag.txtの読み込みテスト
fn test_load_output_tag() {
let actual = Message::create_tags_config("test_files/config/output_tag.txt");
let expected: HashMap<String, String> = HashMap::from([
("attack.impact".to_string(), "Impact".to_string()),
("xxx".to_string(), "yyy".to_string()),
]);
assert_eq!(actual.len(), expected.len());
for (k, v) in expected.iter() {
assert!(actual.get(k).unwrap_or(&String::default()) == v);
}
}
}
src/detections/rule/aggregation_parser.rs
+69 -101
View File
@@ -28,15 +28,15 @@ pub struct AggregationParseInfo {
#[derive(Debug)]
pub enum AggregationConditionToken {
COUNT(String), // count
SPACE, // 空白
Count(String), // count
Space, // 空白
BY, // by
EQ, // ..と等しい
LE, // ..以下
LT, // ..未満
GE, // ..以上
GT, // .よりおおきい
KEYWORD(String), // BYのフィールド名
Keyword(String), // BYのフィールド名
}
/// SIGMAルールでいうAggregationConditionを解析する。
@@ -52,12 +52,12 @@ impl AggegationConditionCompiler {
pub fn compile(&self, condition_str: String) -> Result<Option<AggregationParseInfo>, String> {
let result = self.compile_body(condition_str);
if let Result::Err(msg) = result {
return Result::Err(format!(
Result::Err(format!(
"An aggregation condition parse error has occurred. {}",
msg
));
))
} else {
return result;
result
}
}
@@ -78,11 +78,11 @@ impl AggegationConditionCompiler {
.unwrap()
.as_str()
.to_string()
.replacen("|", "", 1);
.replacen('|', "", 1);
let tokens = self.tokenize(aggregation_str)?;
return self.parse(tokens);
self.parse(tokens)
}
/// 字句解析します。
@@ -90,10 +90,10 @@ impl AggegationConditionCompiler {
&self,
condition_str: String,
) -> Result<Vec<AggregationConditionToken>, String> {
let mut cur_condition_str = condition_str.clone();
let mut cur_condition_str = condition_str;
let mut tokens = Vec::new();
while cur_condition_str.len() != 0 {
while !cur_condition_str.is_empty() {
let captured = self::AGGREGATION_REGEXMAP.iter().find_map(|regex| {
return regex.captures(cur_condition_str.as_str());
});
@@ -105,7 +105,7 @@ impl AggegationConditionCompiler {
let mached_str = captured.unwrap().get(0).unwrap().as_str();
let token = self.to_enum(mached_str.to_string());
if let AggregationConditionToken::SPACE = token {
if let AggregationConditionToken::Space = token {
// 空白は特に意味ないので、読み飛ばす。
cur_condition_str = cur_condition_str.replacen(mached_str, "", 1);
continue;
@@ -115,19 +115,19 @@ impl AggegationConditionCompiler {
cur_condition_str = cur_condition_str.replacen(mached_str, "", 1);
}
return Result::Ok(tokens);
Result::Ok(tokens)
}
/// 比較演算子かどうか判定します。
fn is_cmp_op(&self, token: &AggregationConditionToken) -> bool {
return match token {
AggregationConditionToken::EQ => true,
AggregationConditionToken::LE => true,
AggregationConditionToken::LT => true,
AggregationConditionToken::GE => true,
AggregationConditionToken::GT => true,
_ => false,
};
matches!(
token,
AggregationConditionToken::EQ
| AggregationConditionToken::LE
| AggregationConditionToken::LT
| AggregationConditionToken::GE
| AggregationConditionToken::GT
)
}
/// 構文解析します。
@@ -144,7 +144,7 @@ impl AggegationConditionCompiler {
let token = token_ite.next().unwrap();
let mut count_field_name: Option<String> = Option::None;
if let AggregationConditionToken::COUNT(field_name) = token {
if let AggregationConditionToken::Count(field_name) = token {
if !field_name.is_empty() {
count_field_name = Option::Some(field_name);
}
@@ -173,7 +173,7 @@ impl AggegationConditionCompiler {
);
}
if let AggregationConditionToken::KEYWORD(keyword) = after_by.unwrap() {
if let AggregationConditionToken::Keyword(keyword) = after_by.unwrap() {
by_field_name = Option::Some(keyword);
token_ite.next()
} else {
@@ -200,14 +200,14 @@ impl AggegationConditionCompiler {
);
}
let token = token_ite.next().unwrap_or(AggregationConditionToken::SPACE);
let cmp_number = if let AggregationConditionToken::KEYWORD(number) = token {
let token = token_ite.next().unwrap_or(AggregationConditionToken::Space);
let cmp_number = if let AggregationConditionToken::Keyword(number) = token {
let number: Result<i64, _> = number.parse();
if number.is_err() {
if let Ok(num) = number {
num
} else {
// 比較演算子の後に数値が無い。
return Result::Err("The compare operator needs a number like '> 3'.".to_string());
} else {
number.unwrap()
}
} else {
// 比較演算子の後に数値が無い。
@@ -224,7 +224,7 @@ impl AggegationConditionCompiler {
_cmp_op: cmp_token,
_cmp_num: cmp_number,
};
return Result::Ok(Option::Some(info));
Result::Ok(Option::Some(info))
}
/// 文字列をConditionTokenに変換する。
@@ -232,25 +232,25 @@ impl AggegationConditionCompiler {
if token.starts_with("count(") {
let count_field = token
.replacen("count(", "", 1)
.replacen(")", "", 1)
.replace(" ", "");
return AggregationConditionToken::COUNT(count_field);
.replacen(')', "", 1)
.replace(' ', "");
AggregationConditionToken::Count(count_field)
} else if token == " " {
return AggregationConditionToken::SPACE;
AggregationConditionToken::Space
} else if token == "by" {
return AggregationConditionToken::BY;
AggregationConditionToken::BY
} else if token == "==" {
return AggregationConditionToken::EQ;
AggregationConditionToken::EQ
} else if token == "<=" {
return AggregationConditionToken::LE;
AggregationConditionToken::LE
} else if token == ">=" {
return AggregationConditionToken::GE;
AggregationConditionToken::GE
} else if token == "<" {
return AggregationConditionToken::LT;
AggregationConditionToken::LT
} else if token == ">" {
return AggregationConditionToken::GT;
AggregationConditionToken::GT
} else {
return AggregationConditionToken::KEYWORD(token);
AggregationConditionToken::Keyword(token)
}
}
}
@@ -266,9 +266,9 @@ mod tests {
// countが無いパターン
let compiler = AggegationConditionCompiler::new();
let result = compiler.compile("select1 and select2".to_string());
assert_eq!(true, result.is_ok());
assert!(result.is_ok());
let result = result.unwrap();
assert_eq!(true, result.is_none());
assert!(result.is_none());
}
#[test]
@@ -276,43 +276,23 @@ mod tests {
// 正常系 countの中身にフィールドが無い 各種演算子を試す
let token =
check_aggregation_condition_ope("select1 and select2|count() > 32".to_string(), 32);
let is_gt = match token {
AggregationConditionToken::GT => true,
_ => false,
};
assert_eq!(is_gt, true);
assert!(matches!(token, AggregationConditionToken::GT));
let token =
check_aggregation_condition_ope("select1 and select2|count() >= 43".to_string(), 43);
let is_gt = match token {
AggregationConditionToken::GE => true,
_ => false,
};
assert_eq!(is_gt, true);
assert!(matches!(token, AggregationConditionToken::GE));
let token =
check_aggregation_condition_ope("select1 and select2|count() < 59".to_string(), 59);
let is_gt = match token {
AggregationConditionToken::LT => true,
_ => false,
};
assert_eq!(is_gt, true);
assert!(matches!(token, AggregationConditionToken::LT));
let token =
check_aggregation_condition_ope("select1 and select2|count() <= 12".to_string(), 12);
let is_gt = match token {
AggregationConditionToken::LE => true,
_ => false,
};
assert_eq!(is_gt, true);
assert!(matches!(token, AggregationConditionToken::LE));
let token =
check_aggregation_condition_ope("select1 and select2|count() == 28".to_string(), 28);
let is_gt = match token {
AggregationConditionToken::EQ => true,
_ => false,
};
assert_eq!(is_gt, true);
assert!(matches!(token, AggregationConditionToken::EQ));
}
#[test]
@@ -320,19 +300,15 @@ mod tests {
let compiler = AggegationConditionCompiler::new();
let result = compiler.compile("select1 or select2 | count() by iiibbb > 27".to_string());
assert_eq!(true, result.is_ok());
assert!(result.is_ok());
let result = result.unwrap();
assert_eq!(true, result.is_some());
assert!(result.is_some());
let result = result.unwrap();
assert_eq!("iiibbb".to_string(), result._by_field_name.unwrap());
assert_eq!(true, result._field_name.is_none());
assert!(result._field_name.is_none());
assert_eq!(27, result._cmp_num);
let is_ok = match result._cmp_op {
AggregationConditionToken::GT => true,
_ => false,
};
assert_eq!(true, is_ok);
assert!(matches!(result._cmp_op, AggregationConditionToken::GT));
}
#[test]
@@ -340,19 +316,15 @@ mod tests {
let compiler = AggegationConditionCompiler::new();
let result = compiler.compile("select1 or select2 | count( hogehoge ) > 3".to_string());
assert_eq!(true, result.is_ok());
assert!(result.is_ok());
let result = result.unwrap();
assert_eq!(true, result.is_some());
assert!(result.is_some());
let result = result.unwrap();
assert_eq!(true, result._by_field_name.is_none());
assert!(result._by_field_name.is_none());
assert_eq!("hogehoge", result._field_name.unwrap());
assert_eq!(3, result._cmp_num);
let is_ok = match result._cmp_op {
AggregationConditionToken::GT => true,
_ => false,
};
assert_eq!(true, is_ok);
assert!(matches!(result._cmp_op, AggregationConditionToken::GT));
}
#[test]
@@ -361,19 +333,15 @@ mod tests {
let result =
compiler.compile("select1 or select2 | count( hogehoge) by snsn > 3".to_string());
assert_eq!(true, result.is_ok());
assert!(result.is_ok());
let result = result.unwrap();
assert_eq!(true, result.is_some());
assert!(result.is_some());
let result = result.unwrap();
assert_eq!("snsn".to_string(), result._by_field_name.unwrap());
assert_eq!("hogehoge", result._field_name.unwrap());
assert_eq!(3, result._cmp_num);
let is_ok = match result._cmp_op {
AggregationConditionToken::GT => true,
_ => false,
};
assert_eq!(true, is_ok);
assert!(matches!(result._cmp_op, AggregationConditionToken::GT));
}
#[test]
@@ -381,7 +349,7 @@ mod tests {
let compiler = AggegationConditionCompiler::new();
let result = compiler.compile("select1 or select2 |".to_string());
assert_eq!(true, result.is_err());
assert!(result.is_err());
assert_eq!(
"An aggregation condition parse error has occurred. There are no strings after the pipe(|)."
.to_string(),
@@ -395,7 +363,7 @@ mod tests {
let result =
compiler.compile("select1 or select2 | count( hogeess ) by ii-i > 33".to_string());
assert_eq!(true, result.is_err());
assert!(result.is_err());
assert_eq!(
"An aggregation condition parse error has occurred. An unusable character was found."
.to_string(),
@@ -410,7 +378,7 @@ mod tests {
let result =
compiler.compile("select1 or select2 | by count( hogehoge) by snsn > 3".to_string());
assert_eq!(true, result.is_err());
assert!(result.is_err());
assert_eq!("An aggregation condition parse error has occurred. The aggregation condition can only use count.".to_string(),result.unwrap_err());
}
@@ -420,7 +388,7 @@ mod tests {
let compiler = AggegationConditionCompiler::new();
let result = compiler.compile("select1 or select2 | count( hogehoge) 3".to_string());
assert_eq!(true, result.is_err());
assert!(result.is_err());
assert_eq!("An aggregation condition parse error has occurred. The count keyword needs a compare operator and number like '> 3'".to_string(),result.unwrap_err());
}
@@ -430,7 +398,7 @@ mod tests {
let compiler = AggegationConditionCompiler::new();
let result = compiler.compile("select1 or select2 | count( hogehoge) by".to_string());
assert_eq!(true, result.is_err());
assert!(result.is_err());
assert_eq!("An aggregation condition parse error has occurred. The by keyword needs a field name like 'by EventID'".to_string(),result.unwrap_err());
}
@@ -441,7 +409,7 @@ mod tests {
let result =
compiler.compile("select1 or select2 | count( hogehoge ) by hoe >".to_string());
assert_eq!(true, result.is_err());
assert!(result.is_err());
assert_eq!("An aggregation condition parse error has occurred. The compare operator needs a number like '> 3'.".to_string(),result.unwrap_err());
}
@@ -452,7 +420,7 @@ mod tests {
let result =
compiler.compile("select1 or select2 | count( hogehoge ) by hoe > 3 33".to_string());
assert_eq!(true, result.is_err());
assert!(result.is_err());
assert_eq!(
"An aggregation condition parse error has occurred. An unnecessary word was found."
.to_string(),
@@ -464,14 +432,14 @@ mod tests {
let compiler = AggegationConditionCompiler::new();
let result = compiler.compile(expr);
assert_eq!(true, result.is_ok());
assert!(result.is_ok());
let result = result.unwrap();
assert_eq!(true, result.is_some());
assert!(result.is_some());
let result = result.unwrap();
assert_eq!(true, result._by_field_name.is_none());
assert_eq!(true, result._field_name.is_none());
assert!(result._by_field_name.is_none());
assert!(result._field_name.is_none());
assert_eq!(cmp_num, result._cmp_num);
return result._cmp_op;
result._cmp_op
}
}
src/detections/rule/condition_parser.rs
+58 -69
View File
@@ -57,7 +57,7 @@ impl IntoIterator for ConditionToken {
impl ConditionToken {
fn replace_subtoken(&self, sub_tokens: Vec<ConditionToken>) -> ConditionToken {
return match self {
match self {
ConditionToken::ParenthesisContainer(_) => {
ConditionToken::ParenthesisContainer(sub_tokens)
}
@@ -74,12 +74,12 @@ impl ConditionToken {
ConditionToken::SelectionReference(name) => {
ConditionToken::SelectionReference(name.clone())
}
};
}
}
pub fn sub_tokens<'a>(&'a self) -> Vec<ConditionToken> {
pub fn sub_tokens(&self) -> Vec<ConditionToken> {
// TODO ここでcloneを使わずに実装できるようにしたい。
return match self {
match self {
ConditionToken::ParenthesisContainer(sub_tokens) => sub_tokens.clone(),
ConditionToken::AndContainer(sub_tokens) => sub_tokens.clone(),
ConditionToken::OrContainer(sub_tokens) => sub_tokens.clone(),
@@ -92,14 +92,14 @@ impl ConditionToken {
ConditionToken::And => vec![],
ConditionToken::Or => vec![],
ConditionToken::SelectionReference(_) => vec![],
};
}
}
pub fn sub_tokens_without_parenthesis<'a>(&'a self) -> Vec<ConditionToken> {
return match self {
pub fn sub_tokens_without_parenthesis(&self) -> Vec<ConditionToken> {
match self {
ConditionToken::ParenthesisContainer(_) => vec![],
_ => self.sub_tokens(),
};
}
}
}
@@ -119,8 +119,8 @@ impl ConditionCompiler {
) -> Result<Box<dyn SelectionNode>, String> {
// パイプはここでは処理しない
let captured = self::RE_PIPE.captures(&condition_str);
let condition_str = if captured.is_some() {
let captured = captured.unwrap().get(0).unwrap().as_str().to_string();
let condition_str = if let Some(cap) = captured {
let captured = cap.get(0).unwrap().as_str().to_string();
condition_str.replacen(&captured, "", 1)
} else {
condition_str
@@ -128,9 +128,9 @@ impl ConditionCompiler {
let result = self.compile_condition_body(condition_str, name_2_node);
if let Result::Err(msg) = result {
return Result::Err(format!("A condition parse error has occured. {}", msg));
Result::Err(format!("A condition parse error has occured. {}", msg))
} else {
return result;
result
}
}
@@ -144,7 +144,7 @@ impl ConditionCompiler {
let parsed = self.parse(tokens)?;
return self.to_selectnode(parsed, name_2_node);
self.to_selectnode(parsed, name_2_node)
}
/// 構文解析を実行する。
@@ -161,7 +161,7 @@ impl ConditionCompiler {
let token = self.parse_operand_container(tokens)?;
// 括弧で囲まれている部分を探して、もしあればその部分を再帰的に構文解析します。
return self.parse_rest_parenthesis(token);
self.parse_rest_parenthesis(token)
}
/// 括弧で囲まれている部分を探して、もしあればその部分を再帰的に構文解析します。
@@ -172,7 +172,7 @@ impl ConditionCompiler {
}
let sub_tokens = token.sub_tokens();
if sub_tokens.len() == 0 {
if sub_tokens.is_empty() {
return Result::Ok(token);
}
@@ -181,15 +181,15 @@ impl ConditionCompiler {
let new_token = self.parse_rest_parenthesis(sub_token)?;
new_sub_tokens.push(new_token);
}
return Result::Ok(token.replace_subtoken(new_sub_tokens));
Result::Ok(token.replace_subtoken(new_sub_tokens))
}
/// 字句解析を行う
fn tokenize(&self, condition_str: &String) -> Result<Vec<ConditionToken>, String> {
let mut cur_condition_str = condition_str.clone();
fn tokenize(&self, condition_str: &str) -> Result<Vec<ConditionToken>, String> {
let mut cur_condition_str = condition_str.to_string();
let mut tokens = Vec::new();
while cur_condition_str.len() != 0 {
while !cur_condition_str.is_empty() {
let captured = self::CONDITION_REGEXMAP.iter().find_map(|regex| {
return regex.captures(cur_condition_str.as_str());
});
@@ -210,25 +210,25 @@ impl ConditionCompiler {
cur_condition_str = cur_condition_str.replacen(mached_str, "", 1);
}
return Result::Ok(tokens);
Result::Ok(tokens)
}
/// 文字列をConditionTokenに変換する。
fn to_enum(&self, token: String) -> ConditionToken {
if token == "(" {
return ConditionToken::LeftParenthesis;
ConditionToken::LeftParenthesis
} else if token == ")" {
return ConditionToken::RightParenthesis;
ConditionToken::RightParenthesis
} else if token == " " {
return ConditionToken::Space;
ConditionToken::Space
} else if token == "not" {
return ConditionToken::Not;
ConditionToken::Not
} else if token == "and" {
return ConditionToken::And;
ConditionToken::And
} else if token == "or" {
return ConditionToken::Or;
ConditionToken::Or
} else {
return ConditionToken::SelectionReference(token.clone());
ConditionToken::SelectionReference(token)
}
}
@@ -241,10 +241,7 @@ impl ConditionCompiler {
let mut token_ite = tokens.into_iter();
while let Some(token) = token_ite.next() {
// まず、左括弧を探す。
let is_left = match token {
ConditionToken::LeftParenthesis => true,
_ => false,
};
let is_left = matches!(token, ConditionToken::LeftParenthesis);
if !is_left {
ret.push(token);
continue;
@@ -254,7 +251,7 @@ impl ConditionCompiler {
let mut left_cnt = 1;
let mut right_cnt = 0;
let mut sub_tokens = vec![];
while let Some(token) = token_ite.next() {
for token in token_ite.by_ref() {
if let ConditionToken::LeftParenthesis = token {
left_cnt += 1;
} else if let ConditionToken::RightParenthesis = token {
@@ -275,22 +272,19 @@ impl ConditionCompiler {
}
// この時点で右括弧が残っている場合は右括弧の数が左括弧よりも多いことを表している。
let is_right_left = ret.iter().any(|token| {
return match token {
ConditionToken::RightParenthesis => true,
_ => false,
};
});
let is_right_left = ret
.iter()
.any(|token| matches!(token, ConditionToken::RightParenthesis));
if is_right_left {
return Result::Err("'(' was expected but not found.".to_string());
}
return Result::Ok(ret);
Result::Ok(ret)
}
/// AND, ORをパースする。
fn parse_and_or_operator(&self, tokens: Vec<ConditionToken>) -> Result<ConditionToken, String> {
if tokens.len() == 0 {
if tokens.is_empty() {
// 長さ0は呼び出してはいけない
return Result::Err("Unknown error.".to_string());
}
@@ -339,7 +333,7 @@ impl ConditionCompiler {
// 次にOrでつながっている部分をまとめる
let or_contaienr = ConditionToken::OrContainer(operands);
return Result::Ok(or_contaienr);
Result::Ok(or_contaienr)
}
/// OperandContainerの中身をパースする。現状はNotをパースするためだけに存在している。
@@ -360,7 +354,7 @@ impl ConditionCompiler {
}
// 0はありえないはず
if sub_tokens.len() == 0 {
if sub_tokens.is_empty() {
return Result::Err("Unknown error.".to_string());
}
@@ -380,20 +374,20 @@ impl ConditionCompiler {
let second_token = sub_tokens_ite.next().unwrap();
if let ConditionToken::Not = first_token {
if let ConditionToken::Not = second_token {
return Result::Err("Not is continuous.".to_string());
Result::Err("Not is continuous.".to_string())
} else {
let not_container = ConditionToken::NotContainer(vec![second_token]);
return Result::Ok(not_container);
Result::Ok(not_container)
}
} else {
return Result::Err(
Result::Err(
"Unknown error. Maybe it is because there are multiple names of selection nodes."
.to_string(),
);
)
}
} else {
let sub_tokens = parent_token.sub_tokens_without_parenthesis();
if sub_tokens.len() == 0 {
if sub_tokens.is_empty() {
return Result::Ok(parent_token);
}
@@ -403,7 +397,7 @@ impl ConditionCompiler {
new_sub_tokens.push(new_sub_token);
}
return Result::Ok(parent_token.replace_subtoken(new_sub_tokens));
Result::Ok(parent_token.replace_subtoken(new_sub_tokens))
}
}
@@ -416,14 +410,14 @@ impl ConditionCompiler {
// RefSelectionNodeに変換
if let ConditionToken::SelectionReference(selection_name) = token {
let selection_node = name_2_node.get(&selection_name);
if selection_node.is_none() {
let err_msg = format!("{} is not defined.", selection_name);
return Result::Err(err_msg);
} else {
let selection_node = selection_node.unwrap();
if let Some(select_node) = selection_node {
let selection_node = select_node;
let selection_node = Arc::clone(selection_node);
let ref_node = RefSelectionNode::new(selection_node);
return Result::Ok(Box::new(ref_node));
} else {
let err_msg = format!("{} is not defined.", selection_name);
return Result::Err(err_msg);
}
}
@@ -459,16 +453,12 @@ impl ConditionCompiler {
return Result::Ok(Box::new(select_not_node));
}
return Result::Err("Unknown error".to_string());
Result::Err("Unknown error".to_string())
}
/// ConditionTokenがAndまたはOrTokenならばTrue
fn is_logical(&self, token: &ConditionToken) -> bool {
return match token {
ConditionToken::And => true,
ConditionToken::Or => true,
_ => false,
};
matches!(token, ConditionToken::And | ConditionToken::Or)
}
/// ConditionToken::OperandContainerに変換できる部分があれば変換する。
@@ -478,8 +468,7 @@ impl ConditionCompiler {
) -> Result<Vec<ConditionToken>, String> {
let mut ret = vec![];
let mut grouped_operands = vec![]; // ANDとORの間にあるトークンを表す。ANDとORをOperatorとしたときのOperand
let mut token_ite = tokens.into_iter();
while let Some(token) = token_ite.next() {
for token in tokens.into_iter() {
if self.is_logical(&token) {
// ここに来るのはエラーのはずだが、後でエラー出力するので、ここではエラー出さない。
if grouped_operands.is_empty() {
@@ -498,7 +487,7 @@ impl ConditionCompiler {
ret.push(ConditionToken::OperandContainer(grouped_operands));
}
return Result::Ok(ret);
Result::Ok(ret)
}
}
@@ -542,7 +531,7 @@ mod tests {
assert_eq!(rule_node.select(&recinfo), expect_select);
}
Err(_rec) => {
assert!(false, "Failed to parse json record.");
panic!("Failed to parse json record.");
}
}
}
@@ -582,10 +571,10 @@ mod tests {
Ok(record) => {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
assert_eq!(rule_node.select(&recinfo), true);
assert!(rule_node.select(&recinfo));
}
Err(_rec) => {
assert!(false, "Failed to parse json record.");
Err(_) => {
panic!("Failed to parse json record.");
}
}
}
@@ -626,10 +615,10 @@ mod tests {
Ok(record) => {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
assert_eq!(rule_node.select(&recinfo), false);
assert!(!rule_node.select(&recinfo));
}
Err(_rec) => {
assert!(false, "Failed to parse json record.");
Err(_) => {
panic!("Failed to parse json record.");
}
}
}
src/detections/rule/count.rs
+228 -276
View File
File diff suppressed because it is too large Load Diff
src/detections/rule/matchers.rs
+339 -187
View File
File diff suppressed because it is too large Load Diff
src/detections/rule/mod.rs
+89 -91
View File
@@ -21,7 +21,7 @@ use self::count::{AggRecordTimeInfo, TimeFrameInfo};
use super::detection::EvtxRecordInfo;
pub fn create_rule(rulepath: String, yaml: Yaml) -> RuleNode {
return RuleNode::new(rulepath, yaml);
RuleNode::new(rulepath, yaml)
}
/// Ruleファイルを表すノード
@@ -34,7 +34,7 @@ pub struct RuleNode {
impl Debug for RuleNode {
fn fmt(&self, _f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
return Result::Ok(());
Result::Ok(())
}
}
@@ -42,13 +42,13 @@ unsafe impl Sync for RuleNode {}
unsafe impl Send for RuleNode {}
impl RuleNode {
pub fn new(rulepath: String, yaml: Yaml) -> RuleNode {
return RuleNode {
rulepath: rulepath,
yaml: yaml,
pub fn new(rule_path: String, yaml_data: Yaml) -> RuleNode {
RuleNode {
rulepath: rule_path,
yaml: yaml_data,
detection: DetectionNode::new(),
countdata: HashMap::new(),
};
}
}
pub fn init(&mut self) -> Result<(), Vec<String>> {
@@ -56,14 +56,14 @@ impl RuleNode {
// detection node initialization
let detection_result = self.detection.init(&self.yaml["detection"]);
if detection_result.is_err() {
errmsgs.extend(detection_result.unwrap_err());
if let Err(err_detail) = detection_result {
errmsgs.extend(err_detail);
}
if errmsgs.is_empty() {
return Result::Ok(());
Result::Ok(())
} else {
return Result::Err(errmsgs);
Result::Err(errmsgs)
}
}
@@ -72,11 +72,11 @@ impl RuleNode {
if result && self.has_agg_condition() {
count::count(self, &event_record.record);
}
return result;
result
}
/// aggregation conditionが存在するかを返す関数
pub fn has_agg_condition(&self) -> bool {
return self.detection.aggregation_condition.is_some();
self.detection.aggregation_condition.is_some()
}
/// Aggregation Conditionの結果を配列で返却する関数
pub fn judge_satisfy_aggcondition(&self) -> Vec<AggResult> {
@@ -84,22 +84,18 @@ impl RuleNode {
if !self.has_agg_condition() {
return ret;
}
ret.append(&mut count::aggregation_condition_select(&self));
return ret;
ret.append(&mut count::aggregation_condition_select(self));
ret
}
pub fn check_exist_countdata(&self) -> bool {
self.countdata.len() > 0
!self.countdata.is_empty()
}
/// ルール内のAggregationParseInfo(Aggregation Condition)を取得する関数
pub fn get_agg_condition(&self) -> Option<&AggregationParseInfo> {
match self.detection.aggregation_condition.as_ref() {
None => {
return None;
}
Some(agg_parse_info) => {
return Some(agg_parse_info);
}
if self.detection.aggregation_condition.as_ref().is_some() {
return self.detection.aggregation_condition.as_ref();
}
None
}
}
@@ -110,22 +106,24 @@ pub fn get_detection_keys(node: &RuleNode) -> Vec<String> {
for key in detection.name_to_selection.keys() {
let selection = &detection.name_to_selection[key];
let desc = selection.get_descendants();
let keys = desc.iter().filter_map(|node| {
desc.iter().for_each(|node| {
if !node.is::<LeafSelectionNode>() {
return Option::None;
return;
}
let node = node.downcast_ref::<LeafSelectionNode>().unwrap();
let key = node.get_key();
if key.is_empty() {
return Option::None;
}
return Option::Some(key.to_string());
let keys = node.get_keys();
let keys = keys.iter().filter_map(|key| {
if key.is_empty() {
return None;
}
Some(key.to_string())
});
ret.extend(keys);
});
ret.extend(keys);
}
return ret;
ret
}
/// Ruleファイルのdetectionを表すノード
@@ -138,12 +136,12 @@ struct DetectionNode {
impl DetectionNode {
fn new() -> DetectionNode {
return DetectionNode {
DetectionNode {
name_to_selection: HashMap::new(),
condition: Option::None,
aggregation_condition: Option::None,
timeframe: Option::None,
};
}
}
fn init(&mut self, detection_yaml: &Yaml) -> Result<(), Vec<String>> {
@@ -169,7 +167,7 @@ impl DetectionNode {
]);
}
keys.nth(0).unwrap().to_string()
keys.next().unwrap().to_string()
};
// conditionをパースして、SelectionNodeに変換する
@@ -193,9 +191,9 @@ impl DetectionNode {
}
if err_msgs.is_empty() {
return Result::Ok(());
Result::Ok(())
} else {
return Result::Err(err_msgs);
Result::Err(err_msgs)
}
}
@@ -205,7 +203,7 @@ impl DetectionNode {
}
let condition = &self.condition.as_ref().unwrap();
return condition.select(event_record);
condition.select(event_record)
}
/// selectionノードをパースします。
@@ -221,7 +219,7 @@ impl DetectionNode {
let mut err_msgs = vec![];
for key in keys {
let name = key.as_str().unwrap_or("");
if name.len() == 0 {
if name.is_empty() {
continue;
}
// condition等、特殊なキーワードを無視する。
@@ -231,11 +229,11 @@ impl DetectionNode {
// パースして、エラーメッセージがあれば配列にためて、戻り値で返す。
let selection_node = self.parse_selection(&detection_hash[key]);
if selection_node.is_some() {
let mut selection_node = selection_node.unwrap();
if let Some(node) = selection_node {
let mut selection_node = node;
let init_result = selection_node.init();
if init_result.is_err() {
err_msgs.extend(init_result.unwrap_err());
if let Err(err_detail) = init_result {
err_msgs.extend(err_detail);
} else {
let rc_selection = Arc::new(selection_node);
self.name_to_selection
@@ -248,18 +246,18 @@ impl DetectionNode {
}
// selectionノードが無いのはエラー
if self.name_to_selection.len() == 0 {
if self.name_to_selection.is_empty() {
return Result::Err(vec![
"There is no selection node under detection.".to_string()
]);
}
return Result::Ok(());
Result::Ok(())
}
/// selectionをパースします。
fn parse_selection(&self, selection_yaml: &Yaml) -> Option<Box<dyn SelectionNode>> {
return Option::Some(self.parse_selection_recursively(vec![], selection_yaml));
Option::Some(self.parse_selection_recursively(vec![], selection_yaml))
}
/// selectionをパースします。
@@ -280,7 +278,7 @@ impl DetectionNode {
let child_node = self.parse_selection_recursively(child_key_list, child_yaml);
and_node.child_nodes.push(child_node);
});
return Box::new(and_node);
Box::new(and_node)
} else if yaml.as_vec().is_some() {
// 配列はOR条件と解釈する。
let mut or_node = selectionnodes::OrSelectionNode::new();
@@ -289,13 +287,13 @@ impl DetectionNode {
or_node.child_nodes.push(child_node);
});
return Box::new(or_node);
Box::new(or_node)
} else {
// 連想配列と配列以外は末端ノード
return Box::new(selectionnodes::LeafSelectionNode::new(
Box::new(selectionnodes::LeafSelectionNode::new(
key_list,
yaml.clone(),
));
))
}
}
}
@@ -317,19 +315,19 @@ pub struct AggResult {
impl AggResult {
pub fn new(
data: i64,
key: String,
field_values: Vec<String>,
start_timedate: DateTime<Utc>,
condition_op_num: String,
count_data: i64,
key_name: String,
field_value: Vec<String>,
event_start_timedate: DateTime<Utc>,
condition_op_number: String,
) -> AggResult {
return AggResult {
data: data,
key: key,
field_values: field_values,
start_timedate: start_timedate,
condition_op_num: condition_op_num,
};
AggResult {
data: count_data,
key: key_name,
field_values: field_value,
start_timedate: event_start_timedate,
condition_op_num: condition_op_number,
}
}
}
@@ -341,12 +339,12 @@ mod tests {
pub fn parse_rule_from_str(rule_str: &str) -> RuleNode {
let rule_yaml = YamlLoader::load_from_str(rule_str);
assert_eq!(rule_yaml.is_ok(), true);
assert!(rule_yaml.is_ok());
let rule_yamls = rule_yaml.unwrap();
let mut rule_yaml = rule_yamls.into_iter();
let mut rule_node = create_rule("testpath".to_string(), rule_yaml.next().unwrap());
assert_eq!(rule_node.init().is_ok(), true);
return rule_node;
assert!(rule_node.init().is_ok());
rule_node
}
#[test]
@@ -371,10 +369,10 @@ mod tests {
Ok(record) => {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
assert_eq!(rule_node.select(&recinfo), true);
assert!(rule_node.select(&recinfo));
}
Err(_) => {
assert!(false, "Failed to parse json record.");
panic!("Failed to parse json record.");
}
}
}
@@ -401,10 +399,10 @@ mod tests {
Ok(record) => {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
assert_eq!(rule_node.select(&recinfo), false);
assert!(!rule_node.select(&recinfo));
}
Err(_) => {
assert!(false, "Failed to parse json record.");
panic!("Failed to parse json record.");
}
}
}
@@ -431,10 +429,10 @@ mod tests {
Ok(record) => {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
assert_eq!(rule_node.select(&recinfo), false);
assert!(!rule_node.select(&recinfo));
}
Err(_) => {
assert!(false, "Failed to parse json record.");
panic!("Failed to parse json record.");
}
}
}
@@ -514,10 +512,10 @@ mod tests {
Ok(record) => {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
assert_eq!(rule_node.select(&recinfo), true);
assert!(rule_node.select(&recinfo));
}
Err(_) => {
assert!(false, "Failed to parse json record.");
panic!("Failed to parse json record.");
}
}
}
@@ -573,10 +571,10 @@ mod tests {
Ok(record) => {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
assert_eq!(rule_node.select(&recinfo), false);
assert!(!rule_node.select(&recinfo));
}
Err(_) => {
assert!(false, "Failed to parse json record.");
panic!("Failed to parse json record.");
}
}
}
@@ -639,10 +637,10 @@ mod tests {
Ok(record) => {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
assert_eq!(rule_node.select(&recinfo), true);
assert!(rule_node.select(&recinfo));
}
Err(_) => {
assert!(false, "Failed to parse json record.");
panic!("Failed to parse json record.");
}
}
}
@@ -683,10 +681,10 @@ mod tests {
Ok(record) => {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
assert_eq!(rule_node.select(&recinfo), true);
assert!(rule_node.select(&recinfo));
}
Err(_) => {
assert!(false, "Failed to parse json record.");
panic!("Failed to parse json record.");
}
}
}
@@ -728,10 +726,10 @@ mod tests {
Ok(record) => {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
assert_eq!(rule_node.select(&recinfo), false);
assert!(!rule_node.select(&recinfo));
}
Err(_) => {
assert!(false, "Failed to parse json record.");
panic!("Failed to parse json record.");
}
}
}
@@ -792,10 +790,10 @@ mod tests {
Ok(record) => {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
assert_eq!(rule_node.select(&recinfo), true);
assert!(rule_node.select(&recinfo));
}
Err(_) => {
assert!(false, "Failed to parse json record.");
panic!("Failed to parse json record.");
}
}
}
@@ -856,10 +854,10 @@ mod tests {
Ok(record) => {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
assert_eq!(rule_node.select(&recinfo), false);
assert!(!rule_node.select(&recinfo));
}
Err(_) => {
assert!(false, "Failed to parse json record.");
panic!("Failed to parse json record.");
}
}
}
@@ -902,10 +900,10 @@ mod tests {
Ok(record) => {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
assert_eq!(rule_node.select(&recinfo), true);
assert!(rule_node.select(&recinfo));
}
Err(_rec) => {
assert!(false, "Failed to parse json record.");
panic!("Failed to parse json record.");
}
}
}
@@ -961,15 +959,15 @@ mod tests {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
let result = rule_node.select(&recinfo);
assert_eq!(rule_node.detection.aggregation_condition.is_some(), true);
assert_eq!(result, true);
assert!(rule_node.detection.aggregation_condition.is_some());
assert!(result);
assert_eq!(
*&rule_node.countdata.get(key).unwrap().len() as i32,
rule_node.countdata.get(key).unwrap().len() as i32,
expect_count
);
}
Err(_rec) => {
assert!(false, "Failed to parse json record.");
panic!("Failed to parse json record.");
}
}
}
src/detections/rule/selectionnodes.rs
+107 -108
View File
@@ -1,13 +1,12 @@
use crate::detections::{detection::EvtxRecordInfo, utils};
use crate::filter::FILTER_REGEX;
use mopa::mopafy;
use downcast_rs::Downcast;
use std::{sync::Arc, vec};
use yaml_rust::Yaml;
use super::matchers;
use super::matchers::{self, DefaultMatcher};
// Ruleファイルの detection- selection配下のノードはこのtraitを実装する。
pub trait SelectionNode: mopa::Any {
pub trait SelectionNode: Downcast {
// 引数で指定されるイベントログのレコードが、条件に一致するかどうかを判定する
// このトレイトを実装する構造体毎に適切な判定処理を書く必要がある。
fn select(&self, event_record: &EvtxRecordInfo) -> bool;
@@ -19,12 +18,12 @@ pub trait SelectionNode: mopa::Any {
fn init(&mut self) -> Result<(), Vec<String>>;
// 子ノードを取得する(グラフ理論のchildと同じ意味)
fn get_childs(&self) -> Vec<&Box<dyn SelectionNode>>;
fn get_childs(&self) -> Vec<&dyn SelectionNode>;
// 子孫ノードを取得する(グラフ理論のdescendantと同じ意味)
fn get_descendants(&self) -> Vec<&Box<dyn SelectionNode>>;
fn get_descendants(&self) -> Vec<&dyn SelectionNode>;
}
mopafy!(SelectionNode);
downcast_rs::impl_downcast!(SelectionNode);
/// detection - selection配下でAND条件を表すノード
pub struct AndSelectionNode {
@@ -33,17 +32,17 @@ pub struct AndSelectionNode {
impl AndSelectionNode {
pub fn new() -> AndSelectionNode {
return AndSelectionNode {
AndSelectionNode {
child_nodes: vec![],
};
}
}
}
impl SelectionNode for AndSelectionNode {
fn select(&self, event_record: &EvtxRecordInfo) -> bool {
return self.child_nodes.iter().all(|child_node| {
return child_node.select(event_record);
});
self.child_nodes
.iter()
.all(|child_node| child_node.select(event_record))
}
fn init(&mut self) -> Result<(), Vec<String>> {
@@ -52,50 +51,47 @@ impl SelectionNode for AndSelectionNode {
.iter_mut()
.map(|child_node| {
let res = child_node.init();
if res.is_err() {
return res.unwrap_err();
if let Err(err) = res {
err
} else {
return vec![];
vec![]
}
})
.fold(
vec![],
|mut acc: Vec<String>, cur: Vec<String>| -> Vec<String> {
acc.extend(cur.into_iter());
return acc;
acc
},
);
if err_msgs.is_empty() {
return Result::Ok(());
Result::Ok(())
} else {
return Result::Err(err_msgs);
Result::Err(err_msgs)
}
}
fn get_childs(&self) -> Vec<&Box<dyn SelectionNode>> {
fn get_childs(&self) -> Vec<&dyn SelectionNode> {
let mut ret = vec![];
self.child_nodes.iter().for_each(|child_node| {
ret.push(child_node);
ret.push(child_node.as_ref());
});
return ret;
ret
}
fn get_descendants(&self) -> Vec<&Box<dyn SelectionNode>> {
fn get_descendants(&self) -> Vec<&dyn SelectionNode> {
let mut ret = self.get_childs();
self.child_nodes
.iter()
.map(|child_node| {
return child_node.get_descendants();
})
.flatten()
.flat_map(|child_node| child_node.get_descendants())
.for_each(|descendant_node| {
ret.push(descendant_node);
});
return ret;
ret
}
}
@@ -106,17 +102,17 @@ pub struct OrSelectionNode {
impl OrSelectionNode {
pub fn new() -> OrSelectionNode {
return OrSelectionNode {
OrSelectionNode {
child_nodes: vec![],
};
}
}
}
impl SelectionNode for OrSelectionNode {
fn select(&self, event_record: &EvtxRecordInfo) -> bool {
return self.child_nodes.iter().any(|child_node| {
return child_node.select(event_record);
});
self.child_nodes
.iter()
.any(|child_node| child_node.select(event_record))
}
fn init(&mut self) -> Result<(), Vec<String>> {
@@ -125,50 +121,47 @@ impl SelectionNode for OrSelectionNode {
.iter_mut()
.map(|child_node| {
let res = child_node.init();
if res.is_err() {
return res.unwrap_err();
if let Err(err) = res {
err
} else {
return vec![];
vec![]
}
})
.fold(
vec![],
|mut acc: Vec<String>, cur: Vec<String>| -> Vec<String> {
acc.extend(cur.into_iter());
return acc;
acc
},
);
if err_msgs.is_empty() {
return Result::Ok(());
Result::Ok(())
} else {
return Result::Err(err_msgs);
Result::Err(err_msgs)
}
}
fn get_childs(&self) -> Vec<&Box<dyn SelectionNode>> {
fn get_childs(&self) -> Vec<&dyn SelectionNode> {
let mut ret = vec![];
self.child_nodes.iter().for_each(|child_node| {
ret.push(child_node);
ret.push(child_node.as_ref());
});
return ret;
ret
}
fn get_descendants(&self) -> Vec<&Box<dyn SelectionNode>> {
fn get_descendants(&self) -> Vec<&dyn SelectionNode> {
let mut ret = self.get_childs();
self.child_nodes
.iter()
.map(|child_node| {
return child_node.get_descendants();
})
.flatten()
.flat_map(|child_node| child_node.get_descendants())
.for_each(|descendant_node| {
ret.push(descendant_node);
});
return ret;
ret
}
}
@@ -178,26 +171,26 @@ pub struct NotSelectionNode {
}
impl NotSelectionNode {
pub fn new(node: Box<dyn SelectionNode>) -> NotSelectionNode {
return NotSelectionNode { node: node };
pub fn new(select_node: Box<dyn SelectionNode>) -> NotSelectionNode {
NotSelectionNode { node: select_node }
}
}
impl SelectionNode for NotSelectionNode {
fn select(&self, event_record: &EvtxRecordInfo) -> bool {
return !self.node.select(event_record);
!self.node.select(event_record)
}
fn init(&mut self) -> Result<(), Vec<String>> {
return Result::Ok(());
Result::Ok(())
}
fn get_childs(&self) -> Vec<&Box<dyn SelectionNode>> {
return vec![];
fn get_childs(&self) -> Vec<&dyn SelectionNode> {
vec![]
}
fn get_descendants(&self) -> Vec<&Box<dyn SelectionNode>> {
return self.get_childs();
fn get_descendants(&self) -> Vec<&dyn SelectionNode> {
self.get_childs()
}
}
@@ -210,28 +203,28 @@ pub struct RefSelectionNode {
}
impl RefSelectionNode {
pub fn new(selection_node: Arc<Box<dyn SelectionNode>>) -> RefSelectionNode {
return RefSelectionNode {
selection_node: selection_node,
};
pub fn new(select_node: Arc<Box<dyn SelectionNode>>) -> RefSelectionNode {
RefSelectionNode {
selection_node: select_node,
}
}
}
impl SelectionNode for RefSelectionNode {
fn select(&self, event_record: &EvtxRecordInfo) -> bool {
return self.selection_node.select(event_record);
self.selection_node.select(event_record)
}
fn init(&mut self) -> Result<(), Vec<String>> {
return Result::Ok(());
Result::Ok(())
}
fn get_childs(&self) -> Vec<&Box<dyn SelectionNode>> {
return vec![&self.selection_node];
fn get_childs(&self) -> Vec<&dyn SelectionNode> {
vec![self.selection_node.as_ref().as_ref()]
}
fn get_descendants(&self) -> Vec<&Box<dyn SelectionNode>> {
return self.get_childs();
fn get_descendants(&self) -> Vec<&dyn SelectionNode> {
self.get_childs()
}
}
@@ -244,17 +237,35 @@ pub struct LeafSelectionNode {
}
impl LeafSelectionNode {
pub fn new(key_list: Vec<String>, value_yaml: Yaml) -> LeafSelectionNode {
return LeafSelectionNode {
pub fn new(keys: Vec<String>, value_yaml: Yaml) -> LeafSelectionNode {
LeafSelectionNode {
key: String::default(),
key_list: key_list,
key_list: keys,
select_value: value_yaml,
matcher: Option::None,
};
}
}
pub fn get_key(&self) -> &String {
return &self.key;
&self.key
}
pub fn get_keys(&self) -> Vec<&String> {
let mut keys = vec![];
if !self.key.is_empty() {
keys.push(&self.key);
}
if let Some(matcher) = &self.matcher {
let matcher = matcher.downcast_ref::<DefaultMatcher>();
if let Some(matcher) = matcher {
if let Some(eq_key) = matcher.get_eqfield_key() {
keys.push(eq_key);
}
}
}
keys
}
fn _create_key(&self) -> String {
@@ -263,8 +274,8 @@ impl LeafSelectionNode {
}
let topkey = self.key_list[0].to_string();
let values: Vec<&str> = topkey.split("|").collect();
return values[0].to_string();
let values: Vec<&str> = topkey.split('|').collect();
values[0].to_string()
}
/// JSON形式のEventJSONから値を取得する関数 aliasも考慮されている。
@@ -274,18 +285,18 @@ impl LeafSelectionNode {
return Option::Some(&record.data_string);
}
return record.get_value(self.get_key());
record.get_value(self.get_key())
}
/// matchers::LeafMatcherの一覧を取得する。
/// 上から順番に調べて、一番始めに一致したMatcherが適用される
fn get_matchers(&self) -> Vec<Box<dyn matchers::LeafMatcher>> {
return vec![
vec![
Box::new(matchers::MinlengthMatcher::new()),
Box::new(matchers::RegexesFileMatcher::new()),
Box::new(matchers::AllowlistFileMatcher::new()),
Box::new(matchers::DefaultMatcher::new()),
];
]
}
}
@@ -315,12 +326,8 @@ impl SelectionNode for LeafSelectionNode {
]
}
*/
let filter_rule = FILTER_REGEX.get(self.get_key());
if self.get_key() == "EventData" {
let values =
utils::get_event_value(&"Event.EventData.Data".to_string(), &event_record.record);
let values = utils::get_event_value("Event.EventData.Data", &event_record.record);
if values.is_none() {
return self
.matcher
@@ -333,15 +340,12 @@ impl SelectionNode for LeafSelectionNode {
let eventdata_data = values.unwrap();
if eventdata_data.is_boolean() || eventdata_data.is_i64() || eventdata_data.is_string()
{
let replaced_str = utils::replace_target_character(
event_record.get_value(self.get_key()),
filter_rule,
);
let event_value = event_record.get_value(self.get_key());
return self
.matcher
.as_ref()
.unwrap()
.is_match(replaced_str.as_ref(), event_record);
.is_match(event_value, event_record);
}
// 配列の場合は配列の要素のどれか一つでもルールに合致すれば条件に一致したことにする。
if eventdata_data.is_array() {
@@ -350,15 +354,12 @@ impl SelectionNode for LeafSelectionNode {
.unwrap()
.iter()
.any(|ary_element| {
let replaced_str = utils::replace_target_character(
utils::value_to_string(ary_element).as_ref(),
filter_rule,
);
let event_value = utils::value_to_string(ary_element);
return self
.matcher
.as_ref()
.unwrap()
.is_match(replaced_str.as_ref(), event_record);
.is_match(event_value.as_ref(), event_record);
});
} else {
return self
@@ -369,14 +370,12 @@ impl SelectionNode for LeafSelectionNode {
}
}
let replaced_str =
utils::replace_target_character(self.get_event_value(&event_record), filter_rule);
let event_value = self.get_event_value(event_record);
return self
.matcher
.as_ref()
.unwrap()
.is_match(replaced_str.as_ref(), event_record);
.is_match(event_value, event_record);
}
fn init(&mut self) -> Result<(), Vec<String>> {
@@ -409,12 +408,12 @@ impl SelectionNode for LeafSelectionNode {
.init(&match_key_list, &self.select_value);
}
fn get_childs(&self) -> Vec<&Box<dyn SelectionNode>> {
return vec![];
fn get_childs(&self) -> Vec<&dyn SelectionNode> {
vec![]
}
fn get_descendants(&self) -> Vec<&Box<dyn SelectionNode>> {
return vec![];
fn get_descendants(&self) -> Vec<&dyn SelectionNode> {
vec![]
}
}
@@ -445,10 +444,10 @@ mod tests {
Ok(record) => {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
assert_eq!(rule_node.select(&recinfo), true);
assert!(rule_node.select(&recinfo));
}
Err(_) => {
assert!(false, "Failed to parse json record.");
panic!("Failed to parse json record.");
}
}
}
@@ -478,10 +477,10 @@ mod tests {
Ok(record) => {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
assert_eq!(rule_node.select(&recinfo), false);
assert!(!rule_node.select(&recinfo));
}
Err(_) => {
assert!(false, "Failed to parse json record.");
panic!("Failed to parse json record.");
}
}
}
@@ -510,10 +509,10 @@ mod tests {
Ok(record) => {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
assert_eq!(rule_node.select(&recinfo), true);
assert!(rule_node.select(&recinfo));
}
Err(_) => {
assert!(false, "Failed to parse json record.");
panic!("Failed to parse json record.");
}
}
}
@@ -542,10 +541,10 @@ mod tests {
Ok(record) => {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
assert_eq!(rule_node.select(&recinfo), true);
assert!(rule_node.select(&recinfo));
}
Err(_) => {
assert!(false, "Failed to parse json record.");
panic!("Failed to parse json record.");
}
}
}
@@ -574,10 +573,10 @@ mod tests {
Ok(record) => {
let keys = detections::rule::get_detection_keys(&rule_node);
let recinfo = utils::create_rec_info(record, "testpath".to_owned(), &keys);
assert_eq!(rule_node.select(&recinfo), false);
assert!(!rule_node.select(&recinfo));
}
Err(_) => {
assert!(false, "Failed to parse json record.");
panic!("Failed to parse json record.");
}
}
}
src/detections/utils.rs
+226 -124
View File
@@ -3,7 +3,6 @@ extern crate csv;
extern crate regex;
use crate::detections::configs;
use crate::filter::DataFilterRule;
use tokio::runtime::Builder;
use tokio::runtime::Runtime;
@@ -11,76 +10,56 @@ use tokio::runtime::Runtime;
use chrono::{DateTime, TimeZone, Utc};
use regex::Regex;
use serde_json::Value;
use std::cmp::Ordering;
use std::fs::File;
use std::io::prelude::*;
use std::io::{BufRead, BufReader};
use std::str;
use std::string::String;
use std::vec;
use super::detection::EvtxRecordInfo;
pub fn concat_selection_key(key_list: &Vec<String>) -> String {
pub fn concat_selection_key(key_list: &[String]) -> String {
return key_list
.iter()
.fold("detection -> selection".to_string(), |mut acc, cur| {
acc = acc + " -> " + cur;
return acc;
acc
});
}
pub fn check_regex(string: &str, regex_list: &Vec<Regex>) -> bool {
pub fn check_regex(string: &str, regex_list: &[Regex]) -> bool {
for regex in regex_list {
if regex.is_match(string) == false {
if !regex.is_match(string) {
continue;
}
return true;
}
return false;
false
}
/// replace string from all defined regex in input to replace_str
pub fn replace_target_character<'a>(
input_str: Option<&'a String>,
replace_rule: Option<&'a DataFilterRule>,
) -> Option<String> {
if input_str.is_none() {
return None;
}
if replace_rule.is_none() {
return Some(input_str.unwrap().to_string());
}
let replace_regex_rule = &replace_rule.unwrap().regex_rule;
let replace_str = &replace_rule.unwrap().replace_str;
return Some(
replace_regex_rule
.replace_all(input_str.unwrap(), replace_str)
.to_string(),
);
}
pub fn check_allowlist(target: &str, regexes: &Vec<Regex>) -> bool {
pub fn check_allowlist(target: &str, regexes: &[Regex]) -> bool {
for regex in regexes {
if regex.is_match(target) {
return true;
}
}
return false;
false
}
pub fn value_to_string(value: &Value) -> Option<String> {
return match value {
match value {
Value::Null => Option::None,
Value::Bool(b) => Option::Some(b.to_string()),
Value::Number(n) => Option::Some(n.to_string()),
Value::String(s) => Option::Some(s.to_string()),
Value::String(s) => Option::Some(s.trim().to_string()),
Value::Array(_) => Option::None,
Value::Object(_) => Option::None,
};
}
}
pub fn read_txt(filename: &str) -> Result<Vec<String>, String> {
@@ -90,12 +69,12 @@ pub fn read_txt(filename: &str) -> Result<Vec<String>, String> {
return Result::Err(errmsg);
}
let reader = BufReader::new(f.unwrap());
return Result::Ok(
Result::Ok(
reader
.lines()
.map(|line| line.unwrap_or(String::default()))
.map(|line| line.unwrap_or_default())
.collect(),
);
)
}
pub fn read_csv(filename: &str) -> Result<Vec<Vec<String>>, String> {
@@ -106,11 +85,11 @@ pub fn read_csv(filename: &str) -> Result<Vec<Vec<String>>, String> {
let mut contents: String = String::new();
let mut ret = vec![];
let read_res = f.unwrap().read_to_string(&mut contents);
if read_res.is_err() {
return Result::Err(read_res.unwrap_err().to_string());
if let Err(e) = read_res {
return Result::Err(e.to_string());
}
let mut rdr = csv::Reader::from_reader(contents.as_bytes());
let mut rdr = csv::ReaderBuilder::new().from_reader(contents.as_bytes());
rdr.records().for_each(|r| {
if r.is_err() {
return;
@@ -122,19 +101,19 @@ pub fn read_csv(filename: &str) -> Result<Vec<Vec<String>>, String> {
ret.push(v);
});
return Result::Ok(ret);
Result::Ok(ret)
}
pub fn is_target_event_id(s: &String) -> bool {
return configs::CONFIG.read().unwrap().target_eventids.is_target(s);
pub fn is_target_event_id(s: &str) -> bool {
configs::CONFIG.read().unwrap().target_eventids.is_target(s)
}
pub fn get_event_id_key() -> String {
return "Event.System.EventID".to_string();
"Event.System.EventID".to_string()
}
pub fn get_event_time() -> String {
return "Event.System.TimeCreated_attributes.SystemTime".to_string();
"Event.System.TimeCreated_attributes.SystemTime".to_string()
}
pub fn str_time_to_datetime(system_time_str: &str) -> Option<DateTime<Utc>> {
@@ -146,62 +125,59 @@ pub fn str_time_to_datetime(system_time_str: &str) -> Option<DateTime<Utc>> {
if rfc3339_time.is_err() {
return Option::None;
}
let datetime = Utc
.from_local_datetime(&rfc3339_time.unwrap().naive_utc())
.single();
if datetime.is_none() {
return Option::None;
} else {
return Option::Some(datetime.unwrap());
}
Utc.from_local_datetime(&rfc3339_time.unwrap().naive_utc())
.single()
}
/// serde:Valueの型を確認し、文字列を返します。
pub fn get_serde_number_to_string(value: &serde_json::Value) -> Option<String> {
if value.is_string() {
return Option::Some(value.as_str().unwrap_or("").to_string());
Option::Some(value.as_str().unwrap_or("").to_string())
} else if value.is_object() {
// Object type is not specified record value.
return Option::None;
Option::None
} else {
return Option::Some(value.to_string());
Option::Some(value.to_string())
}
}
pub fn get_event_value<'a>(key: &String, event_value: &'a Value) -> Option<&'a Value> {
if key.len() == 0 {
pub fn get_event_value<'a>(key: &str, event_value: &'a Value) -> Option<&'a Value> {
if key.is_empty() {
return Option::None;
}
let event_key = configs::EVENTKEY_ALIAS.get_event_key(key);
let mut ret: &Value = event_value;
if let Some(event_key) = event_key {
let mut ret: &Value = event_value;
// get_event_keyが取得できてget_event_key_splitが取得できないことはない
let splits = configs::EVENTKEY_ALIAS.get_event_key_split(key);
let mut start_idx = 0;
for key in splits.unwrap() {
if ret.is_object() == false {
if !ret.is_object() {
return Option::None;
}
let val = &event_key[start_idx..(*key + start_idx)];
ret = &ret[val];
start_idx = *key + start_idx;
start_idx += *key;
start_idx += 1;
}
return Option::Some(ret);
Option::Some(ret)
} else {
let mut ret: &Value = event_value;
let event_key = key;
for key in event_key.split(".") {
if ret.is_object() == false {
let event_key = if !key.contains('.') {
"Event.EventData.".to_string() + key
} else {
key.to_string()
};
for key in event_key.split('.') {
if !ret.is_object() {
return Option::None;
}
ret = &ret[key];
}
return Option::Some(ret);
Option::Some(ret)
}
}
@@ -212,28 +188,19 @@ pub fn get_thread_num() -> usize {
.args
.value_of("thread-number")
.unwrap_or(def_thread_num_str.as_str());
return threadnum.parse::<usize>().unwrap().clone();
threadnum.parse::<usize>().unwrap()
}
pub fn create_tokio_runtime() -> Runtime {
return Builder::new_multi_thread()
Builder::new_multi_thread()
.worker_threads(get_thread_num())
.thread_name("yea-thread")
.build()
.unwrap();
.unwrap()
}
// EvtxRecordInfoを作成します。
pub fn create_rec_info(data: Value, path: String, keys: &Vec<String>) -> EvtxRecordInfo {
// EvtxRecordInfoを作る
let data_str = data.to_string();
let mut rec = EvtxRecordInfo {
evtx_filepath: path,
record: data,
data_string: data_str,
key_2_value: hashbrown::HashMap::new(),
};
pub fn create_rec_info(data: Value, path: String, keys: &[String]) -> EvtxRecordInfo {
// 高速化のための処理
// 例えば、Value型から"Event.System.EventID"の値を取得しようとすると、value["Event"]["System"]["EventID"]のように3回アクセスする必要がある。
@@ -241,8 +208,9 @@ pub fn create_rec_info(data: Value, path: String, keys: &Vec<String>) -> EvtxRec
// これなら、"Event.System.EventID"というキーを1回指定するだけで値を取得できるようになるので、高速化されるはず。
// あと、serde_jsonのValueからvalue["Event"]みたいな感じで値を取得する処理がなんか遅いので、そういう意味でも早くなるかも
// それと、serde_jsonでは内部的に標準ライブラリのhashmapを使用しているが、hashbrownを使った方が早くなるらしい。
let mut key_2_values = hashbrown::HashMap::new();
for key in keys {
let val = get_event_value(key, &rec.record);
let val = get_event_value(key, &data);
if val.is_none() {
continue;
}
@@ -252,45 +220,206 @@ pub fn create_rec_info(data: Value, path: String, keys: &Vec<String>) -> EvtxRec
continue;
}
rec.key_2_value.insert(key.to_string(), val.unwrap());
key_2_values.insert(key.to_string(), val.unwrap());
}
return rec;
// EvtxRecordInfoを作る
let data_str = data.to_string();
let rec_info = if configs::CONFIG.read().unwrap().args.is_present("full-data") {
Option::Some(create_recordinfos(&data))
} else {
Option::None
};
EvtxRecordInfo {
evtx_filepath: path,
record: data,
data_string: data_str,
key_2_value: key_2_values,
record_information: rec_info,
}
}
/**
* CSVのrecord infoカラムに出力する文字列を作る
*/
fn create_recordinfos(record: &Value) -> String {
let mut output = vec![];
_collect_recordinfo(&mut vec![], "", record, &mut output);
// 同じレコードなら毎回同じ出力になるようにソートしておく
output.sort_by(|(left, left_data), (right, right_data)| {
let ord = left.cmp(right);
if ord == Ordering::Equal {
left_data.cmp(right_data)
} else {
ord
}
});
let summary: Vec<String> = output
.iter()
.map(|(key, value)| {
return format!("{}:{}", key, value);
})
.collect();
// 標準出力する時はセルがハイプ区切りになるので、パイプ区切りにしない
if configs::CONFIG.read().unwrap().args.is_present("output") {
summary.join(" | ")
} else {
summary.join(" ")
}
}
/**
* CSVのfieldsカラムに出力する要素を全て収集する
*/
fn _collect_recordinfo<'a>(
keys: &mut Vec<&'a str>,
parent_key: &'a str,
value: &'a Value,
output: &mut Vec<(String, String)>,
) {
match value {
Value::Array(ary) => {
for sub_value in ary {
_collect_recordinfo(keys, parent_key, sub_value, output);
}
}
Value::Object(obj) => {
// lifetimeの関係でちょっと変な実装になっている
if !parent_key.is_empty() {
keys.push(parent_key);
}
for (key, value) in obj {
// 属性は出力しない
if key.ends_with("_attributes") {
continue;
}
// Event.Systemは出力しない
if key.eq("System") && keys.get(0).unwrap_or(&"").eq(&"Event") {
continue;
}
_collect_recordinfo(keys, key, value, output);
}
if !parent_key.is_empty() {
keys.pop();
}
}
Value::Null => (),
_ => {
// 一番子の要素の値しか収集しない
let strval = value_to_string(value);
if let Some(strval) = strval {
let strval = strval.trim().chars().fold(String::default(), |mut acc, c| {
if c.is_control() || c.is_ascii_whitespace() {
acc.push(' ');
} else {
acc.push(c);
};
acc
});
output.push((parent_key.to_string(), strval));
}
}
}
}
#[cfg(test)]
mod tests {
use crate::detections::utils;
use crate::filter::DataFilterRule;
use regex::Regex;
use serde_json::Value;
#[test]
fn test_create_recordinfos() {
let record_json_str = r#"
{
"Event": {
"System": {"EventID": 4103, "Channel": "PowerShell", "Computer":"DESKTOP-ICHIICHI"},
"UserData": {"User": "u1", "AccessMask": "%%1369", "Process":"lsass.exe"},
"UserData_attributes": {"xmlns": "http://schemas.microsoft.com/win/2004/08/events/event"}
},
"Event_attributes": {"xmlns": "http://schemas.microsoft.com/win/2004/08/events/event"}
}"#;
match serde_json::from_str(record_json_str) {
Ok(record) => {
let ret = utils::create_recordinfos(&record);
// Systemは除外される/属性(_attributesも除外される)/key順に並ぶ
let expected = "AccessMask:%%1369 Process:lsass.exe User:u1".to_string();
assert_eq!(ret, expected);
}
Err(_) => {
panic!("Failed to parse json record.");
}
}
}
#[test]
fn test_create_recordinfos2() {
// EventDataの特殊ケース
let record_json_str = r#"
{
"Event": {
"System": {"EventID": 4103, "Channel": "PowerShell", "Computer":"DESKTOP-ICHIICHI"},
"EventData": {
"Binary": "hogehoge",
"Data":[
"Data1",
"DataData2",
"",
"DataDataData3"
]
},
"EventData_attributes": {"xmlns": "http://schemas.microsoft.com/win/2004/08/events/event"}
},
"Event_attributes": {"xmlns": "http://schemas.microsoft.com/win/2004/08/events/event"}
}"#;
match serde_json::from_str(record_json_str) {
Ok(record) => {
let ret = utils::create_recordinfos(&record);
// Systemは除外される/属性(_attributesも除外される)/key順に並ぶ
let expected = "Binary:hogehoge Data: Data:Data1 Data:DataData2 Data:DataDataData3"
.to_string();
assert_eq!(ret, expected);
}
Err(_) => {
panic!("Failed to parse json record.");
}
}
}
#[test]
fn test_check_regex() {
let regexes = utils::read_txt("./rules/config/regex/detectlist_suspicous_services.txt")
.unwrap()
.into_iter()
.map(|regex_str| Regex::new(&regex_str).unwrap())
.collect();
let regexes: Vec<Regex> =
utils::read_txt("./rules/config/regex/detectlist_suspicous_services.txt")
.unwrap()
.into_iter()
.map(|regex_str| Regex::new(&regex_str).unwrap())
.collect();
let regextext = utils::check_regex("\\cvtres.exe", &regexes);
assert!(regextext == true);
assert!(regextext);
let regextext = utils::check_regex("\\hogehoge.exe", &regexes);
assert!(regextext == false);
assert!(!regextext);
}
#[test]
fn test_check_allowlist() {
let commandline = "\"C:\\Program Files\\Google\\Update\\GoogleUpdate.exe\"";
let allowlist = utils::read_txt("./rules/config/regex/allowlist_legitimate_services.txt")
.unwrap()
.into_iter()
.map(|allow_str| Regex::new(&allow_str).unwrap())
.collect();
assert!(true == utils::check_allowlist(commandline, &allowlist));
let allowlist: Vec<Regex> =
utils::read_txt("./rules/config/regex/allowlist_legitimate_services.txt")
.unwrap()
.into_iter()
.map(|allow_str| Regex::new(&allow_str).unwrap())
.collect();
assert!(utils::check_allowlist(commandline, &allowlist));
let commandline = "\"C:\\Program Files\\Google\\Update\\GoogleUpdate2.exe\"";
assert!(false == utils::check_allowlist(commandline, &allowlist));
assert!(!utils::check_allowlist(commandline, &allowlist));
}
#[test]
@@ -350,31 +479,4 @@ mod tests {
assert!(utils::get_serde_number_to_string(&event_record["Event"]["EventData"]).is_none());
}
#[test]
/// 指定された文字から指定されたregexぉ実行する関数が動作するかのテスト
fn test_remove_space_control() {
let test_filter_rule = DataFilterRule {
regex_rule: Regex::new(r"[\r\n\t]+").unwrap(),
replace_str: "".to_string(),
};
let none_test_str: Option<&String> = None;
assert_eq!(
utils::replace_target_character(none_test_str, None).is_none(),
true
);
assert_eq!(
utils::replace_target_character(none_test_str, Some(&test_filter_rule)).is_none(),
true
);
let tmp = "h\ra\ny\ta\tb\nu\r\nsa".to_string();
let test_str: Option<&String> = Some(&tmp);
assert_eq!(
utils::replace_target_character(test_str, Some(&test_filter_rule)).unwrap(),
"hayabusa"
);
}
}
src/filter.rs
+12 -80
View File
@@ -2,92 +2,18 @@ use crate::detections::configs;
use crate::detections::print::AlertMessage;
use crate::detections::print::ERROR_LOG_STACK;
use crate::detections::print::QUIET_ERRORS_FLAG;
use crate::detections::utils;
use hashbrown::HashMap;
use hashbrown::HashSet;
use lazy_static::lazy_static;
use regex::Regex;
use std::fs::File;
use std::io::BufWriter;
use std::io::{BufRead, BufReader};
lazy_static! {
static ref IDS_REGEX: Regex =
Regex::new(r"^[0-9a-z]{8}-[0-9a-z]{4}-[0-9a-z]{4}-[0-9a-z]{4}-[0-9a-z]{12}$").unwrap();
pub static ref FILTER_REGEX: HashMap<String, DataFilterRule> = load_record_filters();
}
#[derive(Debug)]
pub struct DataFilterRule {
pub regex_rule: Regex,
pub replace_str: String,
}
fn load_record_filters() -> HashMap<String, DataFilterRule> {
let file_path = "./rules/config/regex/record_data_filter.txt";
let read_result = utils::read_csv(file_path);
let mut ret = HashMap::new();
if read_result.is_err() {
if configs::CONFIG.read().unwrap().args.is_present("verbose") {
AlertMessage::warn(
&mut BufWriter::new(std::io::stderr().lock()),
&format!("{} does not exist", file_path),
)
.ok();
}
if !*QUIET_ERRORS_FLAG {
ERROR_LOG_STACK
.lock()
.unwrap()
.push(format!("{} does not exist", file_path));
}
return HashMap::default();
}
read_result.unwrap().into_iter().for_each(|line| {
if line.len() != 3 {
return;
}
let empty = &"".to_string();
let key = line.get(0).unwrap_or(empty).trim();
let regex_str = line.get(1).unwrap_or(empty).trim();
let replaced_str = line.get(2).unwrap_or(empty).trim();
if key.len() == 0 || regex_str.len() == 0 {
return;
}
let regex_rule: Option<Regex> = match Regex::new(regex_str) {
Ok(regex) => Some(regex),
Err(_err) => {
let errmsg = format!("failed to read regex filter in record_data_filter.txt");
if configs::CONFIG.read().unwrap().args.is_present("verbose") {
AlertMessage::alert(&mut BufWriter::new(std::io::stderr().lock()), &errmsg)
.ok();
}
if !*QUIET_ERRORS_FLAG {
ERROR_LOG_STACK
.lock()
.unwrap()
.push(format!("[ERROR] {}", errmsg));
}
None
}
};
if regex_rule.is_none() {
return;
}
ret.insert(
key.to_string(),
DataFilterRule {
regex_rule: regex_rule.unwrap(),
replace_str: replaced_str.to_string(),
},
);
});
return ret;
}
#[derive(Clone, Debug)]
pub struct RuleExclude {
pub no_use_rule: HashSet<String>,
@@ -104,12 +30,18 @@ pub fn exclude_ids() -> RuleExclude {
.args
.is_present("enable-noisy-rules")
{
exclude_ids.insert_ids("./rules/config/noisy_rules.txt");
exclude_ids.insert_ids(&format!(
"{}/noisy_rules.txt",
configs::CONFIG.read().unwrap().folder_path
));
};
exclude_ids.insert_ids("./rules/config/exclude_rules.txt");
exclude_ids.insert_ids(&format!(
"{}/exclude_rules.txt",
configs::CONFIG.read().unwrap().folder_path
));
return exclude_ids;
exclude_ids
}
impl RuleExclude {
@@ -129,14 +61,14 @@ impl RuleExclude {
.unwrap()
.push(format!("{} does not exist", filename));
}
return ();
return;
}
let reader = BufReader::new(f.unwrap());
for v in reader.lines() {
let v = v.unwrap().split("#").collect::<Vec<&str>>()[0]
let v = v.unwrap().split('#').collect::<Vec<&str>>()[0]
.trim()
.to_string();
if v.is_empty() || !IDS_REGEX.is_match(&v) {
if v.is_empty() || !configs::IDS_REGEX.is_match(&v) {
// 空行は無視する。IDの検証
continue;
}
src/lib.rs
+1
View File
@@ -3,5 +3,6 @@ pub mod detections;
pub mod filter;
pub mod notify;
pub mod omikuji;
pub mod options;
pub mod timeline;
pub mod yaml;
src/main.rs
+357 -94
View File
@@ -1,33 +1,41 @@
extern crate downcast_rs;
extern crate serde;
extern crate serde_derive;
#[cfg(target_os = "windows")]
extern crate static_vcruntime;
use chrono::Datelike;
use chrono::{DateTime, Local};
use chrono::{DateTime, Datelike, Local, TimeZone};
use evtx::{EvtxParser, ParserSettings};
use git2::Repository;
use hashbrown::{HashMap, HashSet};
use hayabusa::detections::configs::load_pivot_keywords;
use hayabusa::detections::detection::{self, EvtxRecordInfo};
use hayabusa::detections::pivot::PIVOT_KEYWORD;
use hayabusa::detections::print::{
AlertMessage, ERROR_LOG_PATH, ERROR_LOG_STACK, QUIET_ERRORS_FLAG, STATISTICS_FLAG,
AlertMessage, ERROR_LOG_PATH, ERROR_LOG_STACK, PIVOT_KEYWORD_LIST_FLAG, QUIET_ERRORS_FLAG,
STATISTICS_FLAG,
};
use hayabusa::detections::rule::{get_detection_keys, RuleNode};
use hayabusa::filter;
use hayabusa::omikuji::Omikuji;
use hayabusa::options::level_tuning::LevelTuning;
use hayabusa::yaml::ParseYaml;
use hayabusa::{afterfact::after_fact, detections::utils};
use hayabusa::{detections::configs, timeline::timeline::Timeline};
use hayabusa::{detections::configs, timeline::timelines::Timeline};
use hhmmss::Hhmmss;
use pbr::ProgressBar;
use serde_json::Value;
use std::collections::{HashMap, HashSet};
use std::ffi::OsStr;
use std::cmp::Ordering;
use std::ffi::{OsStr, OsString};
use std::fmt::Display;
use std::fs::create_dir;
use std::io::BufWriter;
use std::io::{BufWriter, Write};
use std::path::Path;
use std::sync::Arc;
use std::time::SystemTime;
use std::{
env,
fs::{self, File},
path::PathBuf,
vec,
@@ -37,7 +45,7 @@ use tokio::spawn;
use tokio::task::JoinHandle;
#[cfg(target_os = "windows")]
use {is_elevated::is_elevated, std::env};
use is_elevated::is_elevated;
// 一度にtimelineやdetectionを実行する行数
const MAX_DETECT_RECORDS: usize = 5000;
@@ -53,25 +61,56 @@ pub struct App {
rule_keys: Vec<String>,
}
impl Default for App {
fn default() -> Self {
Self::new()
}
}
impl App {
pub fn new() -> App {
return App {
App {
rt: utils::create_tokio_runtime(),
rule_keys: Vec::new(),
};
}
}
fn exec(&mut self) {
if *PIVOT_KEYWORD_LIST_FLAG {
load_pivot_keywords("config/pivot_keywords.txt");
}
let analysis_start_time: DateTime<Local> = Local::now();
// Show usage when no arguments.
if std::env::args().len() == 1 {
self.output_logo();
println!();
println!("{}", configs::CONFIG.read().unwrap().args.usage());
println!();
return;
}
if !configs::CONFIG.read().unwrap().args.is_present("quiet") {
self.output_logo();
println!("");
println!();
self.output_eggs(&format!(
"{:02}/{:02}",
&analysis_start_time.month().to_owned(),
&analysis_start_time.day().to_owned()
));
}
if !self.is_matched_architecture_and_binary() {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
"The hayabusa version you ran does not match your PC architecture.\nPlease use the correct architecture. (Binary ending in -x64.exe for 64-bit and -x86.exe for 32-bit.)",
)
.ok();
println!();
return;
}
if configs::CONFIG
.read()
.unwrap()
@@ -79,7 +118,11 @@ impl App {
.is_present("update-rules")
{
match self.update_rules() {
Ok(_ok) => println!("Rules updated successfully."),
Ok(output) => {
if output != "You currently have the latest rules." {
println!("Rules updated successfully.");
}
}
Err(e) => {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
@@ -88,25 +131,34 @@ impl App {
.ok();
}
}
println!();
return;
}
if !Path::new("./config").exists() {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
&"Hayabusa could not find the config directory.\nPlease run it from the Hayabusa root directory.\nExample: ./hayabusa-1.0.0-windows-x64.exe".to_string()
"Hayabusa could not find the config directory.\nPlease run it from the Hayabusa root directory.\nExample: ./hayabusa-1.0.0-windows-x64.exe"
)
.ok();
return;
}
if configs::CONFIG.read().unwrap().args.args.len() == 0 {
println!(
"{}",
configs::CONFIG.read().unwrap().args.usage().to_string()
);
println!("");
return;
}
if let Some(csv_path) = configs::CONFIG.read().unwrap().args.value_of("output") {
for (key, _) in PIVOT_KEYWORD.read().unwrap().iter() {
let keywords_file_name = csv_path.to_owned() + "-" + key + ".txt";
if Path::new(&keywords_file_name).exists() {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
&format!(
" The file {} already exists. Please specify a different filename.",
&keywords_file_name
),
)
.ok();
return;
}
}
if Path::new(csv_path).exists() {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
@@ -119,9 +171,10 @@ impl App {
return;
}
}
if *STATISTICS_FLAG {
println!("Generating Event ID Statistics");
println!("");
println!();
}
if configs::CONFIG
.read()
@@ -138,26 +191,26 @@ impl App {
if !filepath.ends_with(".evtx")
|| Path::new(filepath)
.file_stem()
.unwrap_or(OsStr::new("."))
.unwrap_or_else(|| OsStr::new("."))
.to_str()
.unwrap()
.trim()
.starts_with(".")
.starts_with('.')
{
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
&"--filepath only accepts .evtx files. Hidden files are ignored.".to_string(),
"--filepath only accepts .evtx files. Hidden files are ignored.",
)
.ok();
return;
}
self.analysis_files(vec![PathBuf::from(filepath)]);
} else if let Some(directory) = configs::CONFIG.read().unwrap().args.value_of("directory") {
let evtx_files = self.collect_evtxfiles(&directory);
if evtx_files.len() == 0 {
let evtx_files = self.collect_evtxfiles(directory);
if evtx_files.is_empty() {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
&"No .evtx files were found.".to_string(),
"No .evtx files were found.",
)
.ok();
return;
@@ -171,27 +224,116 @@ impl App {
{
self.print_contributors();
return;
} else if configs::CONFIG
.read()
.unwrap()
.args
.is_present("level-tuning")
{
let level_tuning_config_path = configs::CONFIG
.read()
.unwrap()
.args
.value_of("level-tuning")
.unwrap_or("./config/level_tuning.txt")
.to_string();
if Path::new(&level_tuning_config_path).exists() {
if let Err(err) = LevelTuning::run(
&level_tuning_config_path,
configs::CONFIG
.read()
.unwrap()
.args
.value_of("rules")
.unwrap_or("rules"),
) {
AlertMessage::alert(&mut BufWriter::new(std::io::stderr().lock()), &err).ok();
}
} else {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
"Need rule_levels.txt file to use --level-tuning option [default: ./config/level_tuning.txt]",
)
.ok();
}
return;
}
let analysis_end_time: DateTime<Local> = Local::now();
let analysis_duration = analysis_end_time.signed_duration_since(analysis_start_time);
println!("");
println!();
println!("Elapsed Time: {}", &analysis_duration.hhmmssxxx());
println!("");
println!();
// Qオプションを付けた場合もしくはパースのエラーがない場合はerrorのstackが9となるのでエラーログファイル自体が生成されない。
if ERROR_LOG_STACK.lock().unwrap().len() > 0 {
AlertMessage::create_error_log(ERROR_LOG_PATH.to_string());
}
if *PIVOT_KEYWORD_LIST_FLAG {
//ファイル出力の場合
if let Some(pivot_file) = configs::CONFIG.read().unwrap().args.value_of("output") {
for (key, pivot_keyword) in PIVOT_KEYWORD.read().unwrap().iter() {
let mut f = BufWriter::new(
fs::File::create(pivot_file.to_owned() + "-" + key + ".txt").unwrap(),
);
let mut output = "".to_string();
output += &format!("{}: ", key).to_string();
output += "( ";
for i in pivot_keyword.fields.iter() {
output += &format!("%{}% ", i).to_string();
}
output += "):";
output += "\n";
for i in pivot_keyword.keywords.iter() {
output += &format!("{}\n", i).to_string();
}
f.write_all(output.as_bytes()).unwrap();
}
//output to stdout
let mut output =
"Pivot keyword results saved to the following files:\n".to_string();
for (key, _) in PIVOT_KEYWORD.read().unwrap().iter() {
output += &(pivot_file.to_owned() + "-" + key + ".txt" + "\n");
}
println!("{}", output);
} else {
//標準出力の場合
let mut output = "The following pivot keywords were found:\n".to_string();
for (key, pivot_keyword) in PIVOT_KEYWORD.read().unwrap().iter() {
output += &format!("{}: ", key).to_string();
output += "( ";
for i in pivot_keyword.fields.iter() {
output += &format!("%{}% ", i).to_string();
}
output += "):";
output += "\n";
for i in pivot_keyword.keywords.iter() {
output += &format!("{}\n", i).to_string();
}
output += "\n";
}
print!("{}", output);
}
}
}
#[cfg(not(target_os = "windows"))]
fn collect_liveanalysis_files(&self) -> Option<Vec<PathBuf>> {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
&"-l / --liveanalysis needs to be run as Administrator on Windows.\r\n".to_string(),
"-l / --liveanalysis needs to be run as Administrator on Windows.\r\n",
)
.ok();
return None;
None
}
#[cfg(target_os = "windows")]
@@ -200,22 +342,22 @@ impl App {
let log_dir = env::var("windir").expect("windir is not found");
let evtx_files =
self.collect_evtxfiles(&[log_dir, "System32\\winevt\\Logs".to_string()].join("/"));
if evtx_files.len() == 0 {
if evtx_files.is_empty() {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
&"No .evtx files were found.".to_string(),
"No .evtx files were found.",
)
.ok();
return None;
}
return Some(evtx_files);
Some(evtx_files)
} else {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
&"-l / --liveanalysis needs to be run as Administrator on Windows.\r\n".to_string(),
"-l / --liveanalysis needs to be run as Administrator on Windows.\r\n",
)
.ok();
return None;
None
}
}
@@ -243,27 +385,27 @@ impl App {
let path = e.unwrap().path();
if path.is_dir() {
path.to_str().and_then(|path_str| {
path.to_str().map(|path_str| {
let subdir_ret = self.collect_evtxfiles(path_str);
ret.extend(subdir_ret);
return Option::Some(());
Option::Some(())
});
} else {
let path_str = path.to_str().unwrap_or("");
if path_str.ends_with(".evtx")
&& !Path::new(path_str)
.file_stem()
.unwrap_or(OsStr::new("."))
.unwrap_or_else(|| OsStr::new("."))
.to_str()
.unwrap()
.starts_with(".")
.starts_with('.')
{
ret.push(path);
}
}
}
return ret;
ret
}
fn print_contributors(&self) {
@@ -295,10 +437,10 @@ impl App {
&filter::exclude_ids(),
);
if rule_files.len() == 0 {
if rule_files.is_empty() {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
&"No rules were loaded. Please download the latest rules with the --update-rules option.\r\n".to_string(),
"No rules were loaded. Please download the latest rules with the --update-rules option.\r\n",
)
.ok();
return;
@@ -316,7 +458,7 @@ impl App {
pb.inc();
}
detection.add_aggcondition_msges(&self.rt);
if !*STATISTICS_FLAG {
if !*STATISTICS_FLAG && !*PIVOT_KEYWORD_LIST_FLAG {
after_fact();
}
}
@@ -369,14 +511,14 @@ impl App {
// target_eventids.txtでフィルタする。
let data = record_result.unwrap().data;
if self._is_target_event_id(&data) == false {
if !self._is_target_event_id(&data) {
continue;
}
// EvtxRecordInfo構造体に変更
records_per_detect.push(data);
}
if records_per_detect.len() == 0 {
if records_per_detect.is_empty() {
break;
}
@@ -397,7 +539,7 @@ impl App {
tl.tm_stats_dsp_msg();
return detection;
detection
}
async fn create_rec_infos(
@@ -407,28 +549,28 @@ impl App {
) -> Vec<EvtxRecordInfo> {
let path = Arc::new(path.to_string());
let rule_keys = Arc::new(rule_keys);
let threads: Vec<JoinHandle<EvtxRecordInfo>> = records_per_detect
.into_iter()
.map(|rec| {
let arc_rule_keys = Arc::clone(&rule_keys);
let arc_path = Arc::clone(&path);
return spawn(async move {
let rec_info =
utils::create_rec_info(rec, arc_path.to_string(), &arc_rule_keys);
return rec_info;
let threads: Vec<JoinHandle<EvtxRecordInfo>> = {
let this = records_per_detect
.into_iter()
.map(|rec| -> JoinHandle<EvtxRecordInfo> {
let arc_rule_keys = Arc::clone(&rule_keys);
let arc_path = Arc::clone(&path);
spawn(async move {
utils::create_rec_info(rec, arc_path.to_string(), &arc_rule_keys)
})
});
})
.collect();
FromIterator::from_iter(this)
};
let mut ret = vec![];
for thread in threads.into_iter() {
ret.push(thread.await.unwrap());
}
return ret;
ret
}
fn get_all_keys(&self, rules: &Vec<RuleNode>) -> Vec<String> {
fn get_all_keys(&self, rules: &[RuleNode]) -> Vec<String> {
let mut key_set = HashSet::new();
for rule in rules {
let keys = get_detection_keys(rule);
@@ -436,7 +578,7 @@ impl App {
}
let ret: Vec<String> = key_set.into_iter().collect();
return ret;
ret
}
// target_eventids.txtの設定を元にフィルタする。
@@ -446,11 +588,11 @@ impl App {
return true;
}
return match eventid.unwrap() {
match eventid.unwrap() {
Value::String(s) => utils::is_target_event_id(s),
Value::Number(n) => utils::is_target_event_id(&n.to_string()),
_ => true, // レコードからEventIdが取得できない場合は、特にフィルタしない
};
}
}
fn evtx_to_jsons(&self, evtx_filepath: PathBuf) -> Option<EvtxParser<File>> {
@@ -462,11 +604,11 @@ impl App {
parse_config = parse_config.num_threads(0); // 設定しないと遅かったので、設定しておく。
let evtx_parser = evtx_parser.with_configuration(parse_config);
return Option::Some(evtx_parser);
Option::Some(evtx_parser)
}
Err(e) => {
eprintln!("{}", e);
return Option::None;
Option::None
}
}
}
@@ -479,8 +621,8 @@ impl App {
/// output logo
fn output_logo(&self) {
let fp = &format!("art/logo.txt");
let content = fs::read_to_string(fp).unwrap_or("".to_owned());
let fp = &"art/logo.txt".to_string();
let content = fs::read_to_string(fp).unwrap_or_default();
println!("{}", content);
}
@@ -495,29 +637,36 @@ impl App {
match eggs.get(exec_datestr) {
None => {}
Some(path) => {
let content = fs::read_to_string(path).unwrap_or("".to_owned());
let content = fs::read_to_string(path).unwrap_or_default();
println!("{}", content);
}
}
}
/// update rules(hayabusa-rules subrepository)
fn update_rules(&self) -> Result<(), git2::Error> {
fn update_rules(&self) -> Result<String, git2::Error> {
let mut result;
let mut prev_modified_time: SystemTime = SystemTime::UNIX_EPOCH;
let mut prev_modified_rules: HashSet<String> = HashSet::default();
let hayabusa_repo = Repository::open(Path::new("."));
let hayabusa_rule_repo = Repository::open(Path::new("./rules"));
let hayabusa_rule_repo = Repository::open(Path::new("rules"));
if hayabusa_repo.is_err() && hayabusa_rule_repo.is_err() {
println!(
"Attempting to git clone the hayabusa-rules repository into the rules folder."
);
// レポジトリが開けなかった段階でhayabusa rulesのgit cloneを実施する
self.clone_rules()
result = self.clone_rules();
} else if hayabusa_rule_repo.is_ok() {
// rulesのrepositoryが確認できる場合
// origin/mainのfetchができなくなるケースはネットワークなどのケースが考えられるため、git cloneは実施しない
self.pull_repository(hayabusa_rule_repo.unwrap())
prev_modified_rules = self.get_updated_rules("rules", &prev_modified_time);
prev_modified_time = fs::metadata("rules").unwrap().modified().unwrap();
result = self.pull_repository(hayabusa_rule_repo.unwrap());
} else {
//hayabusa repositoryがあればsubmodule情報もあると思われるのでupdate
let rules_path = Path::new("./rules");
// hayabusa-rulesのrepositoryがrulesに存在しない場合
// hayabusa repositoryがあればsubmodule情報もあると思われるのでupdate
prev_modified_time = fs::metadata("rules").unwrap().modified().unwrap();
let rules_path = Path::new("rules");
if !rules_path.exists() {
create_dir(rules_path).ok();
}
@@ -529,28 +678,31 @@ impl App {
for mut submodule in submodules {
submodule.update(true, None)?;
let submodule_repo = submodule.open()?;
match self.pull_repository(submodule_repo) {
Ok(it) => it,
Err(e) => {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
&format!("Failed submodule update. {}", e),
)
.ok();
is_success_submodule_update = false;
}
if let Err(e) = self.pull_repository(submodule_repo) {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
&format!("Failed submodule update. {}", e),
)
.ok();
is_success_submodule_update = false;
}
}
if is_success_submodule_update {
Ok(())
result = Ok("Successed submodule update".to_string());
} else {
Err(git2::Error::from_str(&String::default()))
result = Err(git2::Error::from_str(&String::default()));
}
}
if result.is_ok() {
let updated_modified_rules = self.get_updated_rules("rules", &prev_modified_time);
result =
self.print_diff_modified_rule_dates(prev_modified_rules, updated_modified_rules);
}
result
}
/// Pull(fetch and fast-forward merge) repositoryto input_repo.
fn pull_repository(&self, input_repo: Repository) -> Result<(), git2::Error> {
fn pull_repository(&self, input_repo: Repository) -> Result<String, git2::Error> {
match input_repo
.find_remote("origin")?
.fetch(&["main"], None, None)
@@ -568,18 +720,18 @@ impl App {
let fetch_commit = input_repo.reference_to_annotated_commit(&fetch_head)?;
let analysis = input_repo.merge_analysis(&[&fetch_commit])?;
if analysis.0.is_up_to_date() {
Ok(())
Ok("Already up to date".to_string())
} else if analysis.0.is_fast_forward() {
let mut reference = input_repo.find_reference("refs/heads/main")?;
reference.set_target(fetch_commit.id(), "Fast-Forward")?;
input_repo.set_head("refs/heads/main")?;
input_repo.checkout_head(Some(git2::build::CheckoutBuilder::default().force()))?;
Ok(())
Ok("Finished fast forward merge.".to_string())
} else if analysis.0.is_normal() {
AlertMessage::alert(
&mut BufWriter::new(std::io::stderr().lock()),
&"update-rules option is git Fast-Forward merge only. please check your rules folder."
.to_string(),
"update-rules option is git Fast-Forward merge only. please check your rules folder."
,
).ok();
Err(git2::Error::from_str(&String::default()))
} else {
@@ -588,14 +740,14 @@ impl App {
}
/// git clone でhauyabusa-rules レポジトリをrulesフォルダにgit cloneする関数
fn clone_rules(&self) -> Result<(), git2::Error> {
fn clone_rules(&self) -> Result<String, git2::Error> {
match Repository::clone(
"https://github.com/Yamato-Security/hayabusa-rules.git",
"rules",
) {
Ok(_repo) => {
println!("Finished cloning the hayabusa-rules repository.");
Ok(())
Ok("Finished clone".to_string())
}
Err(e) => {
AlertMessage::alert(
@@ -610,11 +762,106 @@ impl App {
}
}
}
/// Create rules folder files Hashset. Format is "[rule title in yaml]|[filepath]|[filemodified date]|[rule type in yaml]"
fn get_updated_rules(
&self,
rule_folder_path: &str,
target_date: &SystemTime,
) -> HashSet<String> {
let mut rulefile_loader = ParseYaml::new();
// level in read_dir is hard code to check all rules.
rulefile_loader
.read_dir(
rule_folder_path,
"INFORMATIONAL",
&filter::RuleExclude {
no_use_rule: HashSet::new(),
},
)
.ok();
let hash_set_keys: HashSet<String> = rulefile_loader
.files
.into_iter()
.filter_map(|(filepath, yaml)| {
let file_modified_date = fs::metadata(&filepath).unwrap().modified().unwrap();
if file_modified_date.cmp(target_date).is_gt() {
let yaml_date = yaml["date"].as_str().unwrap_or("-");
return Option::Some(format!(
"{}|{}|{}|{}",
yaml["title"].as_str().unwrap_or(&String::default()),
yaml["modified"].as_str().unwrap_or(yaml_date),
&filepath,
yaml["ruletype"].as_str().unwrap_or("Other")
));
}
Option::None
})
.collect();
hash_set_keys
}
/// print updated rule files.
fn print_diff_modified_rule_dates(
&self,
prev_sets: HashSet<String>,
updated_sets: HashSet<String>,
) -> Result<String, git2::Error> {
let diff = updated_sets.difference(&prev_sets);
let mut update_count_by_rule_type: HashMap<String, u128> = HashMap::new();
let mut latest_update_date = Local.timestamp(0, 0);
for diff_key in diff {
let tmp: Vec<&str> = diff_key.split('|').collect();
let file_modified_date = fs::metadata(&tmp[2]).unwrap().modified().unwrap();
let dt_local: DateTime<Local> = file_modified_date.into();
if latest_update_date.cmp(&dt_local) == Ordering::Less {
latest_update_date = dt_local;
}
*update_count_by_rule_type
.entry(tmp[3].to_string())
.or_insert(0b0) += 1;
println!(
"[Updated] {} (Modified: {} | Path: {})",
tmp[0], tmp[1], tmp[2]
);
}
println!();
for (key, value) in &update_count_by_rule_type {
println!("Updated {} rules: {}", key, value);
}
if !&update_count_by_rule_type.is_empty() {
Ok("Rule updated".to_string())
} else {
println!("You currently have the latest rules.");
Ok("You currently have the latest rules.".to_string())
}
}
/// check architecture
fn is_matched_architecture_and_binary(&self) -> bool {
if cfg!(target_os = "windows") {
let is_processor_arch_32bit = env::var_os("PROCESSOR_ARCHITECTURE")
.unwrap_or_default()
.eq("x86");
// PROCESSOR_ARCHITEW6432は32bit環境には存在しないため、環境変数存在しなかった場合は32bit環境であると判断する
let not_wow_flag = env::var_os("PROCESSOR_ARCHITEW6432")
.unwrap_or_else(|| OsString::from("x86"))
.eq("x86");
return (cfg!(target_pointer_width = "64") && !is_processor_arch_32bit)
|| (cfg!(target_pointer_width = "32") && is_processor_arch_32bit && not_wow_flag);
}
true
}
}
#[cfg(test)]
mod tests {
use crate::App;
use std::time::SystemTime;
#[test]
fn test_collect_evtxfiles() {
@@ -631,4 +878,20 @@ mod tests {
assert_eq!(is_contains, &true);
})
}
#[test]
fn test_get_updated_rules() {
let app = App::new();
let prev_modified_time: SystemTime = SystemTime::UNIX_EPOCH;
let prev_modified_rules =
app.get_updated_rules("test_files/rules/level_yaml", &prev_modified_time);
assert_eq!(prev_modified_rules.len(), 5);
let target_time: SystemTime = SystemTime::now();
let prev_modified_rules2 =
app.get_updated_rules("test_files/rules/level_yaml", &target_time);
assert_eq!(prev_modified_rules2.len(), 0);
}
}
src/notify/slack.rs
+1 -1
View File
@@ -18,7 +18,7 @@ impl SlackNotify {
eprintln!("WEBHOOK_URL not found");
return false;
}
return true;
true
}
// send message to slack.
src/options/level_tuning.rs
+161
View File
@@ -0,0 +1,161 @@
use crate::detections::{configs, utils};
use crate::filter;
use crate::yaml::ParseYaml;
use std::collections::HashMap;
use std::fs::{self, File};
use std::io::Write;
pub struct LevelTuning {}
impl LevelTuning {
pub fn run(level_tuning_config_path: &str, rules_path: &str) -> Result<(), String> {
let read_result = utils::read_csv(level_tuning_config_path);
if read_result.is_err() {
return Result::Err(read_result.as_ref().unwrap_err().to_string());
}
// Read Tuning files
let mut tuning_map: HashMap<String, String> = HashMap::new();
read_result.unwrap().into_iter().try_for_each(|line| -> Result<(), String> {
let id = match line.get(0) {
Some(_id) => {
if !configs::IDS_REGEX.is_match(_id) {
return Result::Err(format!("Failed to read level tuning file. {} is not correct id format, fix it.", _id));
}
_id
}
_ => return Result::Err("Failed to read id...".to_string())
};
let level = match line.get(1) {
Some(_level) => {
if _level.starts_with("informational")
|| _level.starts_with("low")
|| _level.starts_with("medium")
|| _level.starts_with("high")
|| _level.starts_with("critical") {
_level.split('#').collect::<Vec<&str>>()[0]
} else {
return Result::Err("level tuning file's level must in informational, low, medium, high, critical".to_string())
}
}
_ => return Result::Err("Failed to read level...".to_string())
};
tuning_map.insert(id.to_string(), level.to_string());
Ok(())
})?;
// Read Rule files
let mut rulefile_loader = ParseYaml::new();
let result_readdir =
rulefile_loader.read_dir(rules_path, "informational", &filter::exclude_ids());
if result_readdir.is_err() {
return Result::Err(format!("{}", result_readdir.unwrap_err()));
}
// Convert rule files
for (path, rule) in rulefile_loader.files {
if let Some(new_level) = tuning_map.get(rule["id"].as_str().unwrap()) {
println!("path: {}", path);
let mut content = match fs::read_to_string(&path) {
Ok(_content) => _content,
Err(e) => return Result::Err(e.to_string()),
};
let past_level = "level: ".to_string() + rule["level"].as_str().unwrap();
if new_level.starts_with("informational") {
content = content.replace(&past_level, "level: informational");
}
if new_level.starts_with("low") {
content = content.replace(&past_level, "level: low");
}
if new_level.starts_with("medium") {
content = content.replace(&past_level, "level: medium");
}
if new_level.starts_with("high") {
content = content.replace(&past_level, "level: high");
}
if new_level.starts_with("critical") {
content = content.replace(&past_level, "level: critical");
}
let mut file = match File::options().write(true).truncate(true).open(&path) {
Ok(file) => file,
Err(e) => return Result::Err(e.to_string()),
};
file.write_all(content.as_bytes()).unwrap();
file.flush().unwrap();
println!(
"level: {} -> {}",
rule["level"].as_str().unwrap(),
new_level
);
}
}
Result::Ok(())
}
}
#[cfg(test)]
mod tests {
// use crate::{filter::RuleExclude, yaml};
// use hashbrown::HashSet;
use super::*;
#[test]
fn rule_level_failed_to_open_file() -> Result<(), String> {
let level_tuning_config_path = "./none.txt";
let res = LevelTuning::run(level_tuning_config_path, "");
let expected = Result::Err("Cannot open file. [file:./none.txt]".to_string());
assert_eq!(res, expected);
Ok(())
}
#[test]
fn rule_level_id_error_file() -> Result<(), String> {
let level_tuning_config_path = "./test_files/config/level_tuning_error1.txt";
let res = LevelTuning::run(level_tuning_config_path, "");
let expected = Result::Err("Failed to read level tuning file. 12345678-1234-1234-1234-12 is not correct id format, fix it.".to_string());
assert_eq!(res, expected);
Ok(())
}
#[test]
fn rule_level_level_error_file() -> Result<(), String> {
let level_tuning_config_path = "./test_files/config/level_tuning_error2.txt";
let res = LevelTuning::run(level_tuning_config_path, "");
let expected = Result::Err(
"level tuning file's level must in informational, low, medium, high, critical"
.to_string(),
);
assert_eq!(res, expected);
Ok(())
}
#[test]
fn test_level_tuning_update_rule_files() {
let level_tuning_config_path = "./test_files/config/level_tuning.txt";
let rule_str = r#"
id: 12345678-1234-1234-1234-123456789012
level: informational
"#;
let expected_rule = r#"
id: 12345678-1234-1234-1234-123456789012
level: high
"#;
let path = "test_files/rules/level_tuning_test.yml";
let mut file = File::create(path).unwrap();
let buf = rule_str.as_bytes();
file.write_all(buf).unwrap();
file.flush().unwrap();
let res = LevelTuning::run(level_tuning_config_path, path);
assert_eq!(res, Ok(()));
assert_eq!(fs::read_to_string(path).unwrap(), expected_rule);
fs::remove_file(path).unwrap();
}
}
src/options/mod.rs
+1
View File
@@ -0,0 +1 @@
pub mod level_tuning;
src/timeline/mod.rs
+1 -1
View File
@@ -1,2 +1,2 @@
pub mod statistics;
pub mod timeline;
pub mod timelines;
src/timeline/statistics.rs
+11 -13
View File
@@ -20,16 +20,16 @@ impl EventStatistics {
end_time: String,
stats_list: HashMap<String, usize>,
) -> EventStatistics {
return EventStatistics {
EventStatistics {
total,
filepath,
start_time,
end_time,
stats_list,
};
}
}
pub fn start(&mut self, records: &Vec<EvtxRecordInfo>) {
pub fn start(&mut self, records: &[EvtxRecordInfo]) {
// 引数でstatisticsオプションが指定されている時だけ、統計情報を出力する。
if !configs::CONFIG
.read()
@@ -49,8 +49,8 @@ impl EventStatistics {
self.stats_eventid(records);
}
fn stats_time_cnt(&mut self, records: &Vec<EvtxRecordInfo>) {
if records.len() == 0 {
fn stats_time_cnt(&mut self, records: &[EvtxRecordInfo]) {
if records.is_empty() {
return;
}
self.filepath = records[0].evtx_filepath.as_str().to_owned();
@@ -59,21 +59,19 @@ impl EventStatistics {
// もうちょっと感じに書けるといえば書けます。
for record in records.iter() {
let evttime = utils::get_event_value(
&"Event.System.TimeCreated_attributes.SystemTime".to_string(),
"Event.System.TimeCreated_attributes.SystemTime",
&record.record,
)
.and_then(|evt_value| {
return Option::Some(evt_value.to_string());
});
.map(|evt_value| evt_value.to_string());
if evttime.is_none() {
continue;
}
let evttime = evttime.unwrap();
if self.start_time.len() == 0 || evttime < self.start_time {
if self.start_time.is_empty() || evttime < self.start_time {
self.start_time = evttime.to_string();
}
if self.end_time.len() == 0 || evttime > self.end_time {
if self.end_time.is_empty() || evttime > self.end_time {
self.end_time = evttime;
}
}
@@ -81,10 +79,10 @@ impl EventStatistics {
}
// EventIDで集計
fn stats_eventid(&mut self, records: &Vec<EvtxRecordInfo>) {
fn stats_eventid(&mut self, records: &[EvtxRecordInfo]) {
// let mut evtstat_map = HashMap::new();
for record in records.iter() {
let evtid = utils::get_event_value(&"EventID".to_string(), &record.record);
let evtid = utils::get_event_value("EventID", &record.record);
if evtid.is_none() {
continue;
}
src/timeline/timeline.rs → src/timeline/timelines.rs
+16 -12
View File
@@ -8,6 +8,12 @@ pub struct Timeline {
pub stats: EventStatistics,
}
impl Default for Timeline {
fn default() -> Self {
Self::new()
}
}
impl Timeline {
pub fn new() -> Timeline {
let totalcnt = 0;
@@ -17,10 +23,10 @@ impl Timeline {
let statslst = HashMap::new();
let statistic = EventStatistics::new(totalcnt, filepath, starttm, endtm, statslst);
return Timeline { stats: statistic };
Timeline { stats: statistic }
}
pub fn start(&mut self, records: &Vec<EvtxRecordInfo>) {
pub fn start(&mut self, records: &[EvtxRecordInfo]) {
self.stats.start(records);
}
@@ -41,12 +47,12 @@ impl Timeline {
sammsges.push(format!("Total Event Records: {}\n", self.stats.total));
sammsges.push(format!("First Timestamp: {}", self.stats.start_time));
sammsges.push(format!("Last Timestamp: {}\n", self.stats.end_time));
sammsges.push("Count (Percent)\tID\tEvent\t\tTimeline".to_string());
sammsges.push("--------------- ------- --------------- -------".to_string());
sammsges.push("Count (Percent)\tID\tEvent\t".to_string());
sammsges.push("--------------- ------- ---------------".to_string());
// 集計件数でソート
let mut mapsorted: Vec<_> = self.stats.stats_list.iter().collect();
mapsorted.sort_by(|x, y| y.1.cmp(&x.1));
mapsorted.sort_by(|x, y| y.1.cmp(x.1));
// イベントID毎の出力メッセージ生成
let stats_msges: Vec<String> = self.tm_stats_set_msg(mapsorted);
@@ -68,33 +74,31 @@ impl Timeline {
// イベント情報取得(eventtitleなど)
let conf = configs::CONFIG.read().unwrap();
// timeline_event_info.txtに登録あるものは情報設定
// statistics_event_info.txtに登録あるものは情報設定
match conf.event_timeline_config.get_event_id(*event_id) {
Some(e) => {
// 出力メッセージ1行作成
msges.push(format!(
"{0} ({1:.1}%)\t{2}\t{3}\t{4}",
"{0} ({1:.1}%)\t{2}\t{3}",
event_cnt,
(rate * 1000.0).round() / 10.0,
event_id,
e.evttitle,
e.detectflg
));
}
None => {
// 出力メッセージ1行作成
msges.push(format!(
"{0} ({1:.1}%)\t{2}\t{3}\t{4}",
"{0} ({1:.1}%)\t{2}\t{3}",
event_cnt,
(rate * 1000.0).round() / 10.0,
event_id,
"Unknown".to_string(),
"".to_string()
"Unknown",
));
}
}
}
msges.push("---------------------------------------".to_string());
return msges;
msges
}
}
src/yaml.rs
+31 -30
View File
@@ -23,6 +23,12 @@ pub struct ParseYaml {
pub errorrule_count: u128,
}
impl Default for ParseYaml {
fn default() -> Self {
Self::new()
}
}
impl ParseYaml {
pub fn new() -> ParseYaml {
ParseYaml {
@@ -37,7 +43,7 @@ impl ParseYaml {
let mut file_content = String::new();
let mut fr = fs::File::open(path)
.map(|f| BufReader::new(f))
.map(BufReader::new)
.map_err(|e| e.to_string())?;
fr.read_to_string(&mut file_content)
@@ -76,7 +82,7 @@ impl ParseYaml {
.as_ref()
.to_path_buf()
.extension()
.unwrap_or(OsStr::new(""))
.unwrap_or_else(|| OsStr::new(""))
!= "yml"
{
return io::Result::Ok(String::default());
@@ -126,7 +132,7 @@ impl ParseYaml {
yaml_docs.extend(yaml_contents.unwrap().into_iter().map(|yaml_content| {
let filepath = format!("{}", path.as_ref().to_path_buf().display());
return (filepath, yaml_content);
(filepath, yaml_content)
}));
} else {
let mut entries = fs::read_dir(path)?;
@@ -144,7 +150,12 @@ impl ParseYaml {
// 拡張子がymlでないファイルは無視
let path = entry.path();
if path.extension().unwrap_or(OsStr::new("")) != "yml" {
if path.extension().unwrap_or_else(|| OsStr::new("")) != "yml" {
return io::Result::Ok(ret);
}
// ignore if yml file in .git folder.
if path.to_str().unwrap().contains("/.git/") {
return io::Result::Ok(ret);
}
@@ -192,10 +203,10 @@ impl ParseYaml {
let yaml_contents = yaml_contents.unwrap().into_iter().map(|yaml_content| {
let filepath = format!("{}", entry.path().display());
return (filepath, yaml_content);
(filepath, yaml_content)
});
ret.extend(yaml_contents);
return io::Result::Ok(ret);
io::Result::Ok(ret)
})?;
}
@@ -254,11 +265,11 @@ impl ParseYaml {
}
}
return Option::Some((filepath, yaml_doc));
Option::Some((filepath, yaml_doc))
})
.collect();
self.files.extend(files);
return io::Result::Ok(String::default());
io::Result::Ok(String::default())
}
}
@@ -283,7 +294,7 @@ mod tests {
no_use_rule: HashSet::new(),
};
let _ = &yaml.read_dir(
"test_files/rules/yaml/1.yml".to_string(),
"test_files/rules/yaml/1.yml",
&String::default(),
&exclude_ids,
);
@@ -298,11 +309,7 @@ mod tests {
let exclude_ids = RuleExclude {
no_use_rule: HashSet::new(),
};
let _ = &yaml.read_dir(
"test_files/rules/yaml/".to_string(),
&String::default(),
&exclude_ids,
);
let _ = &yaml.read_dir("test_files/rules/yaml/", &String::default(), &exclude_ids);
assert_ne!(yaml.files.len(), 0);
}
@@ -329,7 +336,7 @@ mod tests {
let path = Path::new("test_files/rules/yaml/error.yml");
let ret = yaml.read_file(path.to_path_buf()).unwrap();
let rule = YamlLoader::load_from_str(&ret);
assert_eq!(rule.is_err(), true);
assert!(rule.is_err());
}
#[test]
@@ -337,8 +344,7 @@ mod tests {
fn test_default_level_read_yaml() {
let mut yaml = yaml::ParseYaml::new();
let path = Path::new("test_files/rules/level_yaml");
yaml.read_dir(path.to_path_buf(), &"", &filter::exclude_ids())
.unwrap();
yaml.read_dir(path, "", &filter::exclude_ids()).unwrap();
assert_eq!(yaml.files.len(), 5);
}
@@ -346,7 +352,7 @@ mod tests {
fn test_info_level_read_yaml() {
let mut yaml = yaml::ParseYaml::new();
let path = Path::new("test_files/rules/level_yaml");
yaml.read_dir(path.to_path_buf(), &"informational", &filter::exclude_ids())
yaml.read_dir(path, "informational", &filter::exclude_ids())
.unwrap();
assert_eq!(yaml.files.len(), 5);
}
@@ -354,15 +360,14 @@ mod tests {
fn test_low_level_read_yaml() {
let mut yaml = yaml::ParseYaml::new();
let path = Path::new("test_files/rules/level_yaml");
yaml.read_dir(path.to_path_buf(), &"LOW", &filter::exclude_ids())
.unwrap();
yaml.read_dir(path, "LOW", &filter::exclude_ids()).unwrap();
assert_eq!(yaml.files.len(), 4);
}
#[test]
fn test_medium_level_read_yaml() {
let mut yaml = yaml::ParseYaml::new();
let path = Path::new("test_files/rules/level_yaml");
yaml.read_dir(path.to_path_buf(), &"MEDIUM", &filter::exclude_ids())
yaml.read_dir(path, "MEDIUM", &filter::exclude_ids())
.unwrap();
assert_eq!(yaml.files.len(), 3);
}
@@ -370,15 +375,14 @@ mod tests {
fn test_high_level_read_yaml() {
let mut yaml = yaml::ParseYaml::new();
let path = Path::new("test_files/rules/level_yaml");
yaml.read_dir(path.to_path_buf(), &"HIGH", &filter::exclude_ids())
.unwrap();
yaml.read_dir(path, "HIGH", &filter::exclude_ids()).unwrap();
assert_eq!(yaml.files.len(), 2);
}
#[test]
fn test_critical_level_read_yaml() {
let mut yaml = yaml::ParseYaml::new();
let path = Path::new("test_files/rules/level_yaml");
yaml.read_dir(path.to_path_buf(), &"CRITICAL", &filter::exclude_ids())
yaml.read_dir(path, "CRITICAL", &filter::exclude_ids())
.unwrap();
assert_eq!(yaml.files.len(), 1);
}
@@ -388,8 +392,7 @@ mod tests {
let mut yaml = yaml::ParseYaml::new();
let path = Path::new("test_files/rules/yaml");
yaml.read_dir(path.to_path_buf(), &"", &filter::exclude_ids())
.unwrap();
yaml.read_dir(path, "", &filter::exclude_ids()).unwrap();
assert_eq!(yaml.ignorerule_count, 10);
}
#[test]
@@ -401,8 +404,7 @@ mod tests {
let exclude_ids = RuleExclude {
no_use_rule: HashSet::new(),
};
yaml.read_dir(path.to_path_buf(), &"", &exclude_ids)
.unwrap();
yaml.read_dir(path, "", &exclude_ids).unwrap();
assert_eq!(yaml.ignorerule_count, 0);
}
#[test]
@@ -412,8 +414,7 @@ mod tests {
let exclude_ids = RuleExclude {
no_use_rule: HashSet::new(),
};
yaml.read_dir(path.to_path_buf(), &"", &exclude_ids)
.unwrap();
yaml.read_dir(path, "", &exclude_ids).unwrap();
assert_eq!(yaml.ignorerule_count, 1);
}
}