Commit Graph

43 Commits

Author SHA1 Message Date
DustInDark
bcf8a33e8c v1.2 pre-release marge (#495)
* Fix/fix clippy warn (#434)

- Fixed following Clippy Warnings(previous warning count: 671 -> after: 4)
  - clippy::needless_return
  - clippy::println_empty_string
  - clippy::redundant_field_names
  - clippy::single_char_pattern
  - clippy::len_zero
  - clippy::iter_nth_zero
  - clippy::bool_comparison
  - clippy::question_mark
  - clippy::needless_collect
  - clippy::unnecessary_unwrap
  - clippy::ptr_arg
  - clippy::needless_collect
  - clippy::needless_borrow
  - clippy::new_without_default
  - clippy::assign_op_pattern
  - clippy::bool_assert_comparison
  - clippy::into_iter_on_ref
  - clippy::deref_addrof
  - clippy::while_let_on_iterator
  - clippy::match_like_matches_macro
  - clippy::or_fun_call
  - clippy::useless_conversion
  - clippy::let_and_return
  - clippy::redundant_clone
  - clippy::redundant_closure
  - clippy::cmp_owned
  - clippy::upper_case_acronyms
  - clippy::map_identity
  - clippy::unused_io_amount
  - clippy::assertions_on_constants
  - clippy::op_ref
  - clippy::useless_vec
  - clippy::vec_init_then_push
  - clippy::useless_format
  - clippy::bind_instead_of_map
  - clippy::bool_comparison
  - clippy::clone_on_copy
  - clippy::too_many_arguments
  - clippy::module_inception
  - fixed clippy::needless_lifetimes
  - fixed clippy::borrowed_box (Thanks for helping by hach1yon!)

* Merge main and output fix#443#444 (#445)

* removed tools/sigmac (#441)

* removed tools/sigmac

- moved tools/sigmac to hayabusa-rules repo

* fixed doc link tools/sigmac

* fixed submodule track

* fixed submodule track from latest to v1.1.0 tag

* fixed link

* erased enter #444

* erased enter #444

* reverted logo enter

* fixed rules submodule target commit #444

Co-authored-by: Yamato Security <71482215+YamatoSecurity@users.noreply.github.com>

* readme update screenshots etc (#448)

* Opensslを静的にコンパイルするためにCargo.tomlの設定変更 (#437)

* cargo update - openssl static

* updated cargo

* macos2apple

* cargo update

* cargo update

* aliasキーがない場合もEvent.EventDataを自動で走査する (#442)

* add no event key

* support not-register-alias search

* added checking EventData when key do not match in alias #290

- added checking key in Event.EventData, if key is not exist in eventkey_alias.txt.

* cargo fmt

* fixed panic when filter files does not exists

* fixed errorlog format when filter config files does not exist

Co-authored-by: DustInDark <nextsasasa@gmail.com>

* changed downcast library from mopa to downcast_rs #447 (#450)

* Fixed Clippy Warnings (#451)

* fixed clippy warn

* fixed cargo clippy warnging

* fixed clippy warngings in clippy ver 0.1.59

* fixed clippy warnings clippy::unnecessary_to_owned

* added temporary blackhat arsenal badge

* added rust report card badges #453

* added repository maintenance levels badge #453

* documentation update macOS usage etc

* update

* added clippy workflow #428 (#429)

* added clippy workflow #428

* fixed action yaml to run clippy #428

* fixed indent

* fixed workflow

* fixed workflow error

* fixed indent

* changed no annotation #428

* adujusted annotation version

* fixed clippy::needless_match

* remove if let exception

* removed unnecessary permission check #428

* statistics event id update (#457)

* Feature/#440 refactoring #395 (#464)

* updated submodule

* fix degrade for pull req #464 (#468)

* fix degrade for pull req #464

* add trim

* Fearture/ added output update result#410 (#452)

* add git2 crate #391

* added Update option #391

* updated readme #391

* fixed cargo.lock

* fixed option if-statement #391

* changed utc short option and rule-update short option #391

* updated readme

* updated readme

* fixed -u long option & version number update #391

* added fast-forwarding rules repository #391

* updated command line option #391

* moved output logo prev update rule

* fixed readme #391

* removed recursive option in readme

* changed rules update from clone and pull to submodule update #391

* fixed document

* changed unnecessary clone recursively to clone only

* English message update.

* cargo fmt

* English message update. ( 4657c35e5c cherry-pick)

* added create rules folder when rules folder is not exist

* fixed gitmodules github-rules url from ssh to https

* added output of updated file #420

* fixed error #410

* changed update rule list seq

* added test

* fixed output #410

* fixed output and fixed output date field  when  modified field is lacked #410

* fixed compile error

* fixed output

- added enter after Latest rule update output
- added output when no exist new rule
- fixed Latest rule update date format
- changed output from 'Latest rule update' to 'Latest rules update'

* fixed compile error

* changed modified date source from rules folder to each yml rule file

* formatting use chrono in main.rs

* merge develop clippy ci

* fixed output when no update rule #410

- removed Latest rule update

- no output "Rules update successfully" when No rule changed

* Change English

Co-authored-by: Tanaka Zakku <71482215+YamatoSecurity@users.noreply.github.com>

* Remove unnecessary code from timeline_event_info and rename files for… (#470)

* Remove unnecessary code from timeline_event_info and rename files for issue462

* Remove unnecessary code #462

* add equalsfield pipe (#467)

* Enhancement: add config config #456 (#471)

* added config option #456

* added process of option to speicifed config folder #456

following files adjust config option.

* noisy_rules.txt

* exclude_rules.txt

* fixed usage in readme

* updated rules submodule:

* fixed process when yml file exist in .git folder

* ignore when yml file exist in .git folder

* Add: --level-tuning option's outline

* Add: read Rule files

* Add: input rule_level.txt files & read rules

* cargo fmt

* Add: level-tuning function

* Reface: split to options file

* WIP: Text overwrite failed...

* Fix: Text overwrite was failed

* Add: Error handlings

* Add: id, level validation

* mv: IDS_REGEX to configs file

* fix: level tuning's file name

* Cargo fmt

* Pivot Keyword List機能の追加 (#412)

* add get_pivot_keyword() func

* change function name and call it's function

* [WIP] support config file

* compilete output

* cargo fmt

* [WIP] add test

* add test

* support -o option in pivot

* add pivot mod

* fix miss

* pass test in pivot.rs

* add comment

* pass all test

* add fast return

* fix output

* add test config file

* review

* rebase

* cargo fmt

* test pass

* fix clippy in my commit

* cargo fmt

* little refactor

* change file input logic and config format

* [WIP] change output

* [wip] change deta structure

* change output & change data structure

* pass test

* add config

* cargo fmt & clippy & rebase

* fix cllipy

* delete /rules/ in .gitignore

* clean comment

* clean

* clean

* fix rebase miss

* fix rebase miss

* fix clippy

* file name output on -o to stdout

* add pivot_keywords.txt to ./config

* updated english

* Documentation update

* cargo fmt and clean

* updated translate japanese

* readme update

* readme update

Co-authored-by: DustInDark <nextsasasa@gmail.com>
Co-authored-by: Tanaka Zakku <71482215+YamatoSecurity@users.noreply.github.com>

* Add: test

* Add: README.md

* Cargo fmt

* Use
#[cfg(test)]

* Fixed output stop when  control char exist in windows terminal (#485)

* added control character filter in details #382

* fixed document

- removed fixed windows teminal caution in readme

* fixed level tuning test and added test files #390

* changed level_tuning.txt header from next_level to new_level

* fixed convert miss change to low level

* added run args rules path to check test easy #390

* fixed comment out processing in level_tuning.txt

* fixed config to show level-tuning option

* fixed level-tuning option usage from required to option

* reduce output mitre attack detail tachnique No. by config file (#483)

* reduced mitre attck tag output by config file #477

* prepared 1.2.0 version toml

* added test files and mitre attck strategy tag file #477

* fixed cargo.toml version

* updated cargo.lock

* output tag english update

* cargo fmt

Co-authored-by: Tanaka Zakku <71482215+YamatoSecurity@users.noreply.github.com>

* Fix: test file's path was incorrect

* Add: add test_files/config/level_tuning.txt

* Add: Flush method.

* inserted debug data

* reverted config usage

* fixed test yaml file path

* Feature/#216 output allfields csvnewcolumn (#469)

* refactoring

* refactoring

* under constructing

* underconstructing

* under construction

* underconstructing

* fix existing testcase

* finish implement

* fmt

* add option

* change name

* fix control code bug

* fix disp

* change format and fix testcase

* fix help

* Fix: show usage when hayabusa has no args

* rm: debug line

* Enhance/warning architecture#478 (#482)

* added  enhance of architecture check #478

* changed check architecture process after output logo #478

* English msg update

* fixed detect method of os-bit to windows and linux

* removed mac and unix architecture and binary and updated its process of windows

* fix clippy

* added check on Wow64 env #478

* Update contributors.txt

Co-authored-by: Tanaka Zakku <71482215+YamatoSecurity@users.noreply.github.com>

* added --level-tuning option to usage

* Revert "added --level-tuning option to usage"

This reverts commit e6a74090a3.

* readme update

* Update README-Japanese.md

* readme, version, cargo update

* typo fix

* typo fix

* rm: duplicated test & fix test name

* Add: show logo, and some infos

* small english fix

* twitter link fix (#486)

* added feature of tag output reducing to agg condition #477 (#488)

* changed level output from informational to info #491

* updated rules submodule

* v1.2 changelog update (#473)

* changelog update

* Update CHANGELOG.md

added contributor in "Fields that are not defined in eventkey_alias.txt will automatically be searched in Event.EventData."

ref #442

* Update CHANGELOG-Japanese.md

Fields that are not defined in eventkey_alias.txt will automatically be searched in Event.EventData.

added contributor in "Fields that are not defined in eventkey_alias.txt will automatically be searched in Event.EventData."

ref #442

* Update CHANGELOG.md

added bug fixes (#444) and `Performance and. accuracy`  add contributor ref(#395)

* Update CHANGELOG-Japanese.md

* Translated v1.2 change log to Japanese

v1.2の内容を日本語に修正

* fixed typo

added lacked back quote.

* added description

added following issue and pr description to readme

- #216 / #469 L8
- #390 / #459 L9
- #478 / #482 L19
- #477/ #483 L20

* added description README.md

added following issue and pr description to readme

- #216 / #469 L8
- #390 / #459 L9
- #478 / #482 L19
- #477/ #483 L20

* changelog update

* changelog update

* update

Co-authored-by: DustInDark <nextsasasa@gmail.com>

* updated rules #493 (#494)

* Resolve conflict develop (#496)

* removed tools/sigmac (#441)

* removed tools/sigmac

- moved tools/sigmac to hayabusa-rules repo

* fixed doc link tools/sigmac

* fixed submodule track

* fixed submodule track from latest to v1.1.0 tag

* fixed link

* fixed rules submodule targe #444

* updated submodule

* updated rules submodule

Co-authored-by: Yamato Security <71482215+YamatoSecurity@users.noreply.github.com>

Co-authored-by: Yamato Security <71482215+YamatoSecurity@users.noreply.github.com>
Co-authored-by: kazuminn <warugaki.k.k@gmail.com>
Co-authored-by: James / hach1yon <32596618+hach1yon@users.noreply.github.com>
Co-authored-by: garigariganzy <tosada31@hotmail.co.jp>
Co-authored-by: itiB <is0312vx@ed.ritsumei.ac.jp>
2022-04-15 12:13:00 +09:00
DustInDark
92c472d451 Hotfix/moved rule configs to hayabusa rules repo#409 (#414)
* fixed target config path #409

* fixed target config file path in test #409

* fixed rules target #409

* Documentation fix, deleted unneeded config files

* added workflow

* changed submodule option

* fixed worksflow to ref submodule

* fixed gitmodules

* fixed workflow

* check code insert

* added update submodules command

* test rules update

* removed test runs

* fixed error

Co-authored-by: Tanaka Zakku <71482215+YamatoSecurity@users.noreply.github.com>
2022-02-26 18:19:19 +09:00
DustInDark
df30adfdef changed hashmap library to tuneup #368 (#369)
* added color code emit_csv test

* replaced HashMap and HashSet to hashbrown #368

* removed debug output in test #368

* fixed colored test
2022-02-09 01:59:39 +09:00
DustInDark
84de8d01af remove yaml ignore check#271 (#385)
* removed yaml ignore label check #271

* moved exclude rule filter check #271

* fixed colored test
2022-02-09 01:59:12 +09:00
kazuminn
d1597b2322 ルール場所指定オプションでファイルを扱えるようにする (#364)
* add only rule file path in --rules

* add error handling for metadata

* refactor

* add test

* rename test function
2022-01-31 12:09:25 +09:00
DustInDark
3412434d99 fixed error 2021-12-22 14:56:10 +09:00
DustInDark
bccdd8fef9 fixed error
- changed writer from stderr to bufwriter

- changed alert,warn function arg fro String to borrow-String
2021-12-21 14:44:26 +09:00
DustInDark
13494ec609 fixed tests
errored no defined error file in alert function call
2021-12-21 02:53:46 +09:00
DustInDark
46211711d6 fixed #301 #303 #309
Squashed commit of the following:

commit 617f12177fbf5066e141b5c1adf969b25c03fa3c
Author: DustInDark <nextsasasa@gmail.com>
Date:   Tue Dec 21 00:57:13 2021 +0900

    fix test typo and merge #301

commit 78926ebf55ae48566152c4097990ca1b1b536b53
Merge: c492ba1 83d891b
Author: DustInDark <nextsasasa@gmail.com>
Date:   Tue Dec 21 00:22:55 2021 +0900

    Merge branch 'main' into feature/output_errorlog_file#301

commit c492ba120a0d977d909b714c2506bd198200853b
Author: DustInDark <nextsasasa@gmail.com>
Date:   Tue Dec 21 00:18:52 2021 +0900

    renamed hayabusa-logs to logs

commit ac018917300e535c2bfc62b6a9df081d4beb1568
Author: DustInDark <nextsasasa@gmail.com>
Date:   Mon Dec 20 23:48:48 2021 +0900

    changed output file path deprecated #303

commit dcef677117555f2fac929b6d3b24ac18b5fb08fc
Author: DustInDark <nextsasasa@gmail.com>
Date:   Mon Dec 20 23:47:42 2021 +0900

    removed error file delete logic

commit b09dec2e4a5c679c3b3c242a655f01cb3b49d490
Author: DustInDark <nextsasasa@gmail.com>
Date:   Mon Dec 20 23:46:49 2021 +0900

    fixed -Q option flag #309
2021-12-21 01:03:33 +09:00
DustInDark
1aebdca160 Revert "Feature/output errorlog#301" (#314) 2021-12-20 20:59:30 +09:00
DustInDark
3c1753109a fixed compile error #301 2021-12-20 15:28:00 +09:00
DustInDark
7d5f10e6cb changed rule read warn output from std to errorlog write #301 2021-12-20 11:47:49 +09:00
DustInDark
dbba49b815 Hotfix/not work count#278 (#281)
* fixed countup structure #278

* fixed countup structure and count up field logic #278

* fixed tests #278

* added  no output aggregation detect message  when output exist in rule yaml #232

* moved get_agg_condtion to rulenode function #278

* added field_values to output count fields data #232 #278

- fixed count logic #278
- fixed count test to adjust field_values add
- added count test

* fixed count output format #232

* fixed compile error

* fixed count output #232

- moved output check to create_count_output
- fixed yaml condition reference
- adjust top and tail multi space

* added create count output test #232

* removed count by file #278

- commented by @YamatoSecurity

* changed sort function to sort_unstable_by

* fixed typo

* adjust to comment #281

ref: https://github.com/Yamato-Security/hayabusa/pull/281#discussion_r767283508

* adjust comment #281

refs
-
https://github.com/Yamato-Security/hayabusa/pull/281#discussion_r767285993
-
https://github.com/Yamato-Security/hayabusa/pull/281#discussion_r767286713

* adjust coment #281

ref:
https://github.com/Yamato-Security/hayabusa/pull/281#discussion_r767287831

* omitted code #281

* adjust comment #281

ref:
https://github.com/Yamato-Security/hayabusa/pull/281#discussion_r767302595

* adjust comment #281

ref:
https://github.com/Yamato-Security/hayabusa/pull/281#discussion_r767303168

* adjust comment

ref:
https://github.com/Yamato-Security/hayabusa/pull/281#discussion_r767307535

* omitted unnecessary code #281

* adjust comment #281

ref:
https://github.com/Yamato-Security/hayabusa/pull/281#discussion_r767288428

* adjust commnet #281

ref:
https://github.com/Yamato-Security/hayabusa/pull/281#discussion_r767286731

* adjust comment #281

ref:
https://github.com/Yamato-Security/hayabusa/pull/281#discussion_r767285716

* adjust comment #281

ref:
159191ec36 (r767288428)

* adjust  test result  #281

* removed debug print statement in testfunction

* adjust comment #281

ref

https://github.com/Yamato-Security/hayabusa/pull/281#discussion_r767286731

* fixed output by level  #278 #284

- fixed result counting process when rule has no aggregation condition #278

- added total output by level #284

* removed unnecessary crate

* fixed output #284

* removed unnecessary total/unique sum process #284

* add testcase and fix testcase bug

* add testcase, add check to check_cout()

* fixed count logic #278

* fixed test parameter

* add testcase

* fmt

* fixed count field check process #278

* fix testcase #281

* fixed comment typo

* removed one time used variable in test case #281

* fixed count field check process #278

* changed insert position #278

* changed contributor list

* fixed contributors list`

* passed with timeframe case #278

* passed all count test #278

* removed debug print

* removed debug print

* removed debug print

* cargo fmt

* changed by0level output format #284

* reduce clone() #278 #281

* changed for loop to map #278 #281

* fixed compile error

* changed priority from output in yml to  aggregation output case aggregation condition exist in rule. #232

* fixed testcase #232

* changed if-let to generics #278 #281

* fixed error when test to sample_evtx#278 #281

* changed if-let to generic #278 #281

* adjust unwrap none error #278 #281

* fixed compile error and test case failed #278

Co-authored-by: ichiichi11 <takai.wa.hajime@gmail.com>
2021-12-19 20:48:29 +09:00
Yamato Security
a023ba46a6 Usage menu update (#302)
* Usage menu update

* usage menuの微調整

* fixed options #302

- changed show-deprecated to enable-deprecated-rules
- changed csv-timeline to output
- change show-noisyalerts to enable-noisy-rules

* fixed option #302

- changed starttimeline to start-timeline

* fixed option #302

- changed q to quiet option

* fixed options #302

- changed endtimeline to end-timeline option
- changed threadnum to thread-number option

Co-authored-by: DustInDark <nextsasasa@gmail.com>
2021-12-19 20:03:39 +09:00
kazuminn
7a6d264be0 feature : statusがdeprecatedなルールを読み込まない (#272)
* feature status deprecated exclude

* clean

* change logic and option name

* fix option description
2021-12-14 18:42:23 +09:00
kazuminn
a00a114101 refactor : rename variables and fix typo and add test (#270) 2021-12-10 23:01:47 +09:00
kazuminn
a2495b6b50 fix miss 2021-12-09 01:35:53 +09:00
kazuminn
db3616b56d add test rule files 2021-12-09 01:29:23 +09:00
kazuminn
360d80b578 clear 2021-12-09 01:15:01 +09:00
kazuminn
b9831ca38a add test for exclude rules 2021-12-09 00:57:40 +09:00
ichiichi11
191d1df9f0 add exclude files and fix bugs. 2021-12-04 19:23:50 +09:00
ichiichi11
9169214553 fix bug. 2021-12-04 19:09:41 +09:00
ichiichi11
c961c3768c change from hashmap to hashset and remove unnecessary copy. 2021-12-04 18:46:11 +09:00
kazuminn
446e540d6f merge main into feature/fill_no_use_rules 2021-12-02 00:49:54 +09:00
kazuminn
b9c415eab5 add 2021-12-02 00:43:31 +09:00
kazuminn
838a935d34 pass test 2021-12-02 00:33:19 +09:00
kazuminn
341a5e4f86 feature fillter no use rules 2021-11-30 22:54:36 +09:00
DustInDark
84f17323da Hotfix/load rule level changed info to informational#237#238 (#240)
* changed INFO to informational #237

- INFO in rule level is changed  to informational

* changed level load default rule from LOW to INFORMATIONAL #238

* fixed level description in doc and help menu #238

* removed test files

* removed test check file
2021-11-28 18:27:58 +09:00
Yamato Security
bc230f7cd5 英語修正 (#236)
* 英語修正

* cargo fmt

* fixed test assertion string data

Co-authored-by: DustInDark <nextsasasa@gmail.com>
2021-11-27 11:21:55 +09:00
DustInDark
b48f774b93 Feature/output unique detection#209 (#225)
* checked contributors #141

- because RustyBlue code contributor(not hayabusa contributor) was mixed in hayabusa contributor

* changed yaml count name

* changed ruletype string #157

* fixed output of parse error #157

* fixed output

* added level unique detection output #209
2021-11-24 21:15:43 +09:00
DustInDark
b53342218c Feature/output logo#206 (#222)
* add output logo #206

* added newline and orgnization name #206

* add output rule count #200

* Changed yml summarize the totals for each folder hierarchy. #157

* added analyzing evtx file count output #157

* added loaded rule count output #157

* added quiet option #206
2021-11-21 15:16:44 +09:00
DustInDark
199a8231c1 v1.0でリリースしない機能の削除、contributorsの表示、levelオプションのデフォルト値修正 #141 #211 (#218)
* changed default level to Low #211

* fixed usage #211

* erased Lang option #195

* changed output credit to contributors #141

* Removed contributor information for uncreated features and features that will not be introduced in v1.0. #141

* removed slack notification feature #202

- removed config option
- removed artifact slack notification call

* removed description of slack notification #202

* fixed default level to Low #211

* removed description about slack notification #202
2021-11-20 09:56:59 +09:00
DustInDark
e2ac686c3f Feature/verbose output rule and file#188 (#219)
* added verbose output rule and evtx path #188

* fixed typo

* changed yaml read error to warn message #188

- added AlertMessage::warn
- yaml read error changed from error to warn
2021-11-20 09:10:17 +09:00
James
22c8302c4c change from stdout to stderr. (#190) 2021-11-12 13:21:14 +09:00
DustInDark
66b8f2de9e Feature/risk level condition#45 (#186)
* add risk level filter arguments #45

* fix default level in help #45

* add test yaml files #45

* refactoring and fix level argument usage.

* cargo fmt --all

Co-authored-by: ichiichi11 <takai.wa.hajime@gmail.com>
2021-11-11 23:47:29 +09:00
DustInDark
be04a0410e Hotfix/hidden file read159 (#180)
* added error output of no evtx extension in  filepath and directory args #159

* fixed error of  hidden file read #159

- file extension is limited to yml  when load of rule

* fix for no extension rule file.

Co-authored-by: ichiichi11 <takai.wa.hajime@gmail.com>
2021-11-10 22:55:20 +09:00
James
e77a193c5c Feature/#158 add rulefilepath column (#168)
* add level csv column

* update

* Feature/output detect count151 (#167)

* add output process count of detects events #151

* add output process count of detects event when output stdio #151

* add format enter

* update

Co-authored-by: DustInDark <nextsasasa@gmail.com>
2021-11-09 00:35:28 +09:00
James
403844ae45 finish (#136) 2021-09-13 23:26:15 +09:00
Alan Smithee
a68a59417d Feature/add eventfilepath to csv #76 (#89)
* Feature/call error message struct#66 (#69)

* change  way to use write trait #66

* change call error message struct #66

* erase finished TODO #66

* erase comment in error message format test #66

* resolve conflict #66

* Feature/call error message struct#66 (#71)

* change ERROR writeln struct #66

* add evtx file path export to csv #76

* fixed test case #76

* fix for #76

* forget cargo fmt -all

* fix testcase

Co-authored-by: ichiichi11 <takai.wa.hajime@gmail.com>
2021-05-01 09:49:48 +09:00
akiranishikawa
f58d5f316b resolved #40 2020-12-07 12:18:48 +09:00
ichiichi11
1abdbafb5a under constructing 2020-11-21 15:04:28 +09:00
akiranishikawa
fefbd01615 Changed to load only when enabled is true 2020-11-09 10:20:32 +09:00
akiranishikawa
d26fccbcda Change from toml to yaml 2020-11-09 09:04:10 +09:00