v1.2 pre-release marge (#495)

* Fix/fix clippy warn (#434)

- Fixed following Clippy Warnings(previous warning count: 671 -> after: 4)
  - clippy::needless_return
  - clippy::println_empty_string
  - clippy::redundant_field_names
  - clippy::single_char_pattern
  - clippy::len_zero
  - clippy::iter_nth_zero
  - clippy::bool_comparison
  - clippy::question_mark
  - clippy::needless_collect
  - clippy::unnecessary_unwrap
  - clippy::ptr_arg
  - clippy::needless_collect
  - clippy::needless_borrow
  - clippy::new_without_default
  - clippy::assign_op_pattern
  - clippy::bool_assert_comparison
  - clippy::into_iter_on_ref
  - clippy::deref_addrof
  - clippy::while_let_on_iterator
  - clippy::match_like_matches_macro
  - clippy::or_fun_call
  - clippy::useless_conversion
  - clippy::let_and_return
  - clippy::redundant_clone
  - clippy::redundant_closure
  - clippy::cmp_owned
  - clippy::upper_case_acronyms
  - clippy::map_identity
  - clippy::unused_io_amount
  - clippy::assertions_on_constants
  - clippy::op_ref
  - clippy::useless_vec
  - clippy::vec_init_then_push
  - clippy::useless_format
  - clippy::bind_instead_of_map
  - clippy::bool_comparison
  - clippy::clone_on_copy
  - clippy::too_many_arguments
  - clippy::module_inception
  - fixed clippy::needless_lifetimes
  - fixed clippy::borrowed_box (Thanks for helping by hach1yon!)

* Merge main and output fix#443#444 (#445)

* removed tools/sigmac (#441)

* removed tools/sigmac

- moved tools/sigmac to hayabusa-rules repo

* fixed doc link tools/sigmac

* fixed submodule track

* fixed submodule track from latest to v1.1.0 tag

* fixed link

* erased enter #444

* erased enter #444

* reverted logo enter

* fixed rules submodule target commit #444

Co-authored-by: Yamato Security <71482215+YamatoSecurity@users.noreply.github.com>

* readme update screenshots etc (#448)

* Opensslを静的にコンパイルするためにCargo.tomlの設定変更 (#437)

* cargo update - openssl static

* updated cargo

* macos2apple

* cargo update

* cargo update

* aliasキーがない場合もEvent.EventDataを自動で走査する (#442)

* add no event key

* support not-register-alias search

* added checking EventData when key do not match in alias #290

- added checking key in Event.EventData, if key is not exist in eventkey_alias.txt.

* cargo fmt

* fixed panic when filter files does not exists

* fixed errorlog format when filter config files does not exist

Co-authored-by: DustInDark <nextsasasa@gmail.com>

* changed downcast library from mopa to downcast_rs #447 (#450)

* Fixed Clippy Warnings (#451)

* fixed clippy warn

* fixed cargo clippy warnging

* fixed clippy warngings in clippy ver 0.1.59

* fixed clippy warnings clippy::unnecessary_to_owned

* added temporary blackhat arsenal badge

* added rust report card badges #453

* added repository maintenance levels badge #453

* documentation update macOS usage etc

* update

* added clippy workflow #428 (#429)

* added clippy workflow #428

* fixed action yaml to run clippy #428

* fixed indent

* fixed workflow

* fixed workflow error

* fixed indent

* changed no annotation #428

* adujusted annotation version

* fixed clippy::needless_match

* remove if let exception

* removed unnecessary permission check #428

* statistics event id update (#457)

* Feature/#440 refactoring #395 (#464)

* updated submodule

* fix degrade for pull req #464 (#468)

* fix degrade for pull req #464

* add trim

* Fearture/ added output update result#410 (#452)

* add git2 crate #391

* added Update option #391

* updated readme #391

* fixed cargo.lock

* fixed option if-statement #391

* changed utc short option and rule-update short option #391

* updated readme

* updated readme

* fixed -u long option & version number update #391

* added fast-forwarding rules repository #391

* updated command line option #391

* moved output logo prev update rule

* fixed readme #391

* removed recursive option in readme

* changed rules update from clone and pull to submodule update #391

* fixed document

* changed unnecessary clone recursively to clone only

* English message update.

* cargo fmt

* English message update. ( 4657c35e5c cherry-pick)

* added create rules folder when rules folder is not exist

* fixed gitmodules github-rules url from ssh to https

* added output of updated file #420

* fixed error #410

* changed update rule list seq

* added test

* fixed output #410

* fixed output and fixed output date field  when  modified field is lacked #410

* fixed compile error

* fixed output

- added enter after Latest rule update output
- added output when no exist new rule
- fixed Latest rule update date format
- changed output from 'Latest rule update' to 'Latest rules update'

* fixed compile error

* changed modified date source from rules folder to each yml rule file

* formatting use chrono in main.rs

* merge develop clippy ci

* fixed output when no update rule #410

- removed Latest rule update

- no output "Rules update successfully" when No rule changed

* Change English

Co-authored-by: Tanaka Zakku <71482215+YamatoSecurity@users.noreply.github.com>

* Remove unnecessary code from timeline_event_info and rename files for… (#470)

* Remove unnecessary code from timeline_event_info and rename files for issue462

* Remove unnecessary code #462

* add equalsfield pipe (#467)

* Enhancement: add config config #456 (#471)

* added config option #456

* added process of option to speicifed config folder #456

following files adjust config option.

* noisy_rules.txt

* exclude_rules.txt

* fixed usage in readme

* updated rules submodule:

* fixed process when yml file exist in .git folder

* ignore when yml file exist in .git folder

* Add: --level-tuning option's outline

* Add: read Rule files

* Add: input rule_level.txt files & read rules

* cargo fmt

* Add: level-tuning function

* Reface: split to options file

* WIP: Text overwrite failed...

* Fix: Text overwrite was failed

* Add: Error handlings

* Add: id, level validation

* mv: IDS_REGEX to configs file

* fix: level tuning's file name

* Cargo fmt

* Pivot Keyword List機能の追加 (#412)

* add get_pivot_keyword() func

* change function name and call it's function

* [WIP] support config file

* compilete output

* cargo fmt

* [WIP] add test

* add test

* support -o option in pivot

* add pivot mod

* fix miss

* pass test in pivot.rs

* add comment

* pass all test

* add fast return

* fix output

* add test config file

* review

* rebase

* cargo fmt

* test pass

* fix clippy in my commit

* cargo fmt

* little refactor

* change file input logic and config format

* [WIP] change output

* [wip] change deta structure

* change output & change data structure

* pass test

* add config

* cargo fmt & clippy & rebase

* fix cllipy

* delete /rules/ in .gitignore

* clean comment

* clean

* clean

* fix rebase miss

* fix rebase miss

* fix clippy

* file name output on -o to stdout

* add pivot_keywords.txt to ./config

* updated english

* Documentation update

* cargo fmt and clean

* updated translate japanese

* readme update

* readme update

Co-authored-by: DustInDark <nextsasasa@gmail.com>
Co-authored-by: Tanaka Zakku <71482215+YamatoSecurity@users.noreply.github.com>

* Add: test

* Add: README.md

* Cargo fmt

* Use
#[cfg(test)]

* Fixed output stop when  control char exist in windows terminal (#485)

* added control character filter in details #382

* fixed document

- removed fixed windows teminal caution in readme

* fixed level tuning test and added test files #390

* changed level_tuning.txt header from next_level to new_level

* fixed convert miss change to low level

* added run args rules path to check test easy #390

* fixed comment out processing in level_tuning.txt

* fixed config to show level-tuning option

* fixed level-tuning option usage from required to option

* reduce output mitre attack detail tachnique No. by config file (#483)

* reduced mitre attck tag output by config file #477

* prepared 1.2.0 version toml

* added test files and mitre attck strategy tag file #477

* fixed cargo.toml version

* updated cargo.lock

* output tag english update

* cargo fmt

Co-authored-by: Tanaka Zakku <71482215+YamatoSecurity@users.noreply.github.com>

* Fix: test file's path was incorrect

* Add: add test_files/config/level_tuning.txt

* Add: Flush method.

* inserted debug data

* reverted config usage

* fixed test yaml file path

* Feature/#216 output allfields csvnewcolumn (#469)

* refactoring

* refactoring

* under constructing

* underconstructing

* under construction

* underconstructing

* fix existing testcase

* finish implement

* fmt

* add option

* change name

* fix control code bug

* fix disp

* change format and fix testcase

* fix help

* Fix: show usage when hayabusa has no args

* rm: debug line

* Enhance/warning architecture#478 (#482)

* added  enhance of architecture check #478

* changed check architecture process after output logo #478

* English msg update

* fixed detect method of os-bit to windows and linux

* removed mac and unix architecture and binary and updated its process of windows

* fix clippy

* added check on Wow64 env #478

* Update contributors.txt

Co-authored-by: Tanaka Zakku <71482215+YamatoSecurity@users.noreply.github.com>

* added --level-tuning option to usage

* Revert "added --level-tuning option to usage"

This reverts commit e6a74090a3.

* readme update

* Update README-Japanese.md

* readme, version, cargo update

* typo fix

* typo fix

* rm: duplicated test & fix test name

* Add: show logo, and some infos

* small english fix

* twitter link fix (#486)

* added feature of tag output reducing to agg condition #477 (#488)

* changed level output from informational to info #491

* updated rules submodule

* v1.2 changelog update (#473)

* changelog update

* Update CHANGELOG.md

added contributor in "Fields that are not defined in eventkey_alias.txt will automatically be searched in Event.EventData."

ref #442

* Update CHANGELOG-Japanese.md

Fields that are not defined in eventkey_alias.txt will automatically be searched in Event.EventData.

added contributor in "Fields that are not defined in eventkey_alias.txt will automatically be searched in Event.EventData."

ref #442

* Update CHANGELOG.md

added bug fixes (#444) and `Performance and. accuracy`  add contributor ref(#395)

* Update CHANGELOG-Japanese.md

* Translated v1.2 change log to Japanese

v1.2の内容を日本語に修正

* fixed typo

added lacked back quote.

* added description

added following issue and pr description to readme

- #216 / #469 L8
- #390 / #459 L9
- #478 / #482 L19
- #477/ #483 L20

* added description README.md

added following issue and pr description to readme

- #216 / #469 L8
- #390 / #459 L9
- #478 / #482 L19
- #477/ #483 L20

* changelog update

* changelog update

* update

Co-authored-by: DustInDark <nextsasasa@gmail.com>

* updated rules #493 (#494)

* Resolve conflict develop (#496)

* removed tools/sigmac (#441)

* removed tools/sigmac

- moved tools/sigmac to hayabusa-rules repo

* fixed doc link tools/sigmac

* fixed submodule track

* fixed submodule track from latest to v1.1.0 tag

* fixed link

* fixed rules submodule targe #444

* updated submodule

* updated rules submodule

Co-authored-by: Yamato Security <71482215+YamatoSecurity@users.noreply.github.com>

Co-authored-by: Yamato Security <71482215+YamatoSecurity@users.noreply.github.com>
Co-authored-by: kazuminn <warugaki.k.k@gmail.com>
Co-authored-by: James / hach1yon <32596618+hach1yon@users.noreply.github.com>
Co-authored-by: garigariganzy <tosada31@hotmail.co.jp>
Co-authored-by: itiB <is0312vx@ed.ritsumei.ac.jp>

src/timeline/mod.rs

@@ -1,2 +1,2 @@
 pub mod statistics;
-pub mod timeline;
+pub mod timelines;

src/timeline/statistics.rs

@@ -20,16 +20,16 @@ impl EventStatistics {
         end_time: String,
         stats_list: HashMap<String, usize>,
     ) -> EventStatistics {
-        return EventStatistics {
+        EventStatistics {
             total,
             filepath,
             start_time,
             end_time,
             stats_list,
-        };
+        }
     }
 
-    pub fn start(&mut self, records: &Vec<EvtxRecordInfo>) {
+    pub fn start(&mut self, records: &[EvtxRecordInfo]) {
         // 引数でstatisticsオプションが指定されている時だけ、統計情報を出力する。
         if !configs::CONFIG
             .read()
@@ -49,8 +49,8 @@ impl EventStatistics {
         self.stats_eventid(records);
     }
 
-    fn stats_time_cnt(&mut self, records: &Vec<EvtxRecordInfo>) {
-        if records.len() == 0 {
+    fn stats_time_cnt(&mut self, records: &[EvtxRecordInfo]) {
+        if records.is_empty() {
             return;
         }
         self.filepath = records[0].evtx_filepath.as_str().to_owned();
@@ -59,21 +59,19 @@ impl EventStatistics {
         // もうちょっと感じに書けるといえば書けます。
         for record in records.iter() {
             let evttime = utils::get_event_value(
-                &"Event.System.TimeCreated_attributes.SystemTime".to_string(),
+                "Event.System.TimeCreated_attributes.SystemTime",
                 &record.record,
             )
-            .and_then(|evt_value| {
-                return Option::Some(evt_value.to_string());
-            });
+            .map(|evt_value| evt_value.to_string());
             if evttime.is_none() {
                 continue;
             }
 
             let evttime = evttime.unwrap();
-            if self.start_time.len() == 0 || evttime < self.start_time {
+            if self.start_time.is_empty() || evttime < self.start_time {
                 self.start_time = evttime.to_string();
             }
-            if self.end_time.len() == 0 || evttime > self.end_time {
+            if self.end_time.is_empty() || evttime > self.end_time {
                 self.end_time = evttime;
             }
         }
@@ -81,10 +79,10 @@ impl EventStatistics {
     }
 
     // EventIDで集計
-    fn stats_eventid(&mut self, records: &Vec<EvtxRecordInfo>) {
+    fn stats_eventid(&mut self, records: &[EvtxRecordInfo]) {
         //        let mut evtstat_map = HashMap::new();
         for record in records.iter() {
-            let evtid = utils::get_event_value(&"EventID".to_string(), &record.record);
+            let evtid = utils::get_event_value("EventID", &record.record);
             if evtid.is_none() {
                 continue;
             }

src/timeline/timelines.rs

@@ -8,6 +8,12 @@ pub struct Timeline {
     pub stats: EventStatistics,
 }
 
+impl Default for Timeline {
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
 impl Timeline {
     pub fn new() -> Timeline {
         let totalcnt = 0;
@@ -17,10 +23,10 @@ impl Timeline {
         let statslst = HashMap::new();
 
         let statistic = EventStatistics::new(totalcnt, filepath, starttm, endtm, statslst);
-        return Timeline { stats: statistic };
+        Timeline { stats: statistic }
     }
 
-    pub fn start(&mut self, records: &Vec<EvtxRecordInfo>) {
+    pub fn start(&mut self, records: &[EvtxRecordInfo]) {
         self.stats.start(records);
     }
 
@@ -41,12 +47,12 @@ impl Timeline {
         sammsges.push(format!("Total Event Records: {}\n", self.stats.total));
         sammsges.push(format!("First Timestamp: {}", self.stats.start_time));
         sammsges.push(format!("Last Timestamp: {}\n", self.stats.end_time));
-        sammsges.push("Count (Percent)\tID\tEvent\t\tTimeline".to_string());
-        sammsges.push("--------------- ------- --------------- -------".to_string());
+        sammsges.push("Count (Percent)\tID\tEvent\t".to_string());
+        sammsges.push("--------------- ------- ---------------".to_string());
 
         // 集計件数でソート
         let mut mapsorted: Vec<_> = self.stats.stats_list.iter().collect();
-        mapsorted.sort_by(|x, y| y.1.cmp(&x.1));
+        mapsorted.sort_by(|x, y| y.1.cmp(x.1));
 
         // イベントID毎の出力メッセージ生成
         let stats_msges: Vec<String> = self.tm_stats_set_msg(mapsorted);
@@ -68,33 +74,31 @@ impl Timeline {
 
             // イベント情報取得(eventtitleなど)
             let conf = configs::CONFIG.read().unwrap();
-            // timeline_event_info.txtに登録あるものは情報設定
+            // statistics_event_info.txtに登録あるものは情報設定
             match conf.event_timeline_config.get_event_id(*event_id) {
                 Some(e) => {
                     // 出力メッセージ1行作成
                     msges.push(format!(
-                        "{0} ({1:.1}%)\t{2}\t{3}\t{4}",
+                        "{0} ({1:.1}%)\t{2}\t{3}",
                         event_cnt,
                         (rate * 1000.0).round() / 10.0,
                         event_id,
                         e.evttitle,
-                        e.detectflg
                     ));
                 }
                 None => {
                     // 出力メッセージ1行作成
                     msges.push(format!(
-                        "{0} ({1:.1}%)\t{2}\t{3}\t{4}",
+                        "{0} ({1:.1}%)\t{2}\t{3}",
                         event_cnt,
                         (rate * 1000.0).round() / 10.0,
                         event_id,
-                        "Unknown".to_string(),
-                        "".to_string()
+                        "Unknown",
                     ));
                 }
             }
         }
         msges.push("---------------------------------------".to_string());
-        return msges;
+        msges
     }
 }