diff --git a/README-English.md b/README-English.md index 1779d279..28fc7940 100644 --- a/README-English.md +++ b/README-English.md @@ -6,75 +6,58 @@

-# Hayabusa -Hayabusa is a very fast Windows event analyzer used for creating forensic timelines and performing threat hunting based on IoCs written in either hayabusa or SIGMA rules. It can be run live, offline, pushed out as agents to be run on endpoints in an enterprise after an incident. - # About Hayabusa -Hayabusa ("falcon" in Japanese) was written by the Yamato Security group in Japan. First inspired by the DeepblueCLI Windows event log analyzer, we started in 2020 porting it over to Rust for the RustyBlue project, then created SIGMA-like flexible signatures based in YAML, and then added a backend to SIGMA to support converting SIGMA rules into hayabusa rules. Supporting multi-threading, (to our knowledge) it is currently the fastest forensics timeline generator and threat hunting tool as well supports the most features in SIGMA. It can analyze multiple Windows event logs and consolidate the results into one timeline for easy analysis. It will output in CSV to be imported into tools like Timeline Explorer and Excel for analysis. +Hayabusa ("falcon" in Japanese) is a **Windows event log fast forensics timeline generator** and **threat hunting tool** created by the [Yamato Security](https://yamatosecurity.connpass.com/) group in Japan. It is written in [Rust](https://www.rust-lang.org/) and supports multi-threading in order to be as fast as possible. It supports converted [sigma](https://github.com/SigmaHQ/sigma) and hayabusa detection rules written in YAML in order to be as easily customizable and extensible as possible. It can be run either on a live system or by gathering logs from multiple systems. The output will be consolidated into a single CSV timeline for easy analysis in Excel or [timeline explorer](https://ericzimmerman.github.io/#!index.md). + +## Fast forensics timeline generation +Windows event log analysis has traditionally been a very long and tedious process because Windows event logs are 1) in a data format that is hard to analyze and 2) the majority of data is noise and not useful for investigations. Hayabusa's main goal is to extract out only useful data and present it in an easy-to-read format that is usable not only by professionally trained analysts but any Windows system administrator. +Hayabusa is not intended to be a replacement for tools like [Evtx Explorer](https://ericzimmerman.github.io/#!index.md) or [Event Log Explorer](https://eventlogxp.com/) for slower deep-dive analysis but is intended for letting analysts get 80% of their work done in 20% of the time. + +## Threat hunting +Hayabusa currently has over 1000 detection rules and the ultimate goal is to be able to push out hayabusa agents to all Windows endpoints after an incident or for periodic threat hunting and have them alert back to a central server. + +# About the development + First inspired by the [DeepBlueCLI](https://github.com/sans-blue-team/DeepBlueCLI) Windows event log analyzer, we started in 2020 porting it over to Rust for the [RustyBlue](https://github.com/Yamato-Security/RustyBlue) project, then created sigma-like flexible detection signatures written in YAML, and then added a backend to sigma to support converting sigma rules into our hayabusa rule format. # Screenshots -Add screenshots here. +Startup: + +![Hayabusa Startup](/screenshots/hayabusa-start.png) + + +Terminal output: + +![Hayabusa terminal output](/screenshots/hayabusa-results.png) + + +Results summary: + +![Hayabusa results summary](/screenshots/hayabusa-results-summary.png) + # Features -* Cross-platform support: Windows, Linux, macOS (Intel + ARM) -* Faster than a hayabusa falcon! -* English and Japanese support +* Cross-platform support: Windows, Linux, macOS +* Developed in Rust to be memory safe and faster than a hayabusa falcon! * Multi-thread support -* Creating event timelines for forensic investigations and incident response +* Creates a single easy-to-analyze CSV timeline for forensic investigations and incident response * Threat hunting based on IoC signatures written in easy to read/create/edit YAML based hayabusa rules -* SIGMA support to convert SIGMA rules to hayabusa rules +* Sigma rule support to convert sigma rules to hayabusa rules +* Currently it supports the most sigma rules compared to other similar tools and even supports count rules * Event log statistics (Useful for getting a picture of what types of events there are and for tuning your log settings) +* Rule tuning configuration by excluding bad rules or noisy rules + +# Planned Features +* Enterprise-wide hunting on all endpoints +* Japanese language support +* MITRE ATT&CK mapping +* MITRE ATT&CK heatmap generation +* User logon and failed logon summary +* Input from JSON logs +* Output to JSON -> import to Elastic Stack/Splunk # Downloads You can download pre-compiled binaries for the Windows, Linux and macOS at [Releases.](https://github.com/Yamato-Security/hayabusa/releases) -# Usage -## Command line options -```` -USAGE: - hayabusa.exe [FLAGS] [OPTIONS] - -FLAGS: - --credits Prints a list of contributors - -h, --help Prints help information - --rfc-2822 Output date and time in RFC 2822 format. Example: Mon, 07 Aug 2006 12:34:56 -0600 - -s, --statistics Prints statistics for event logs - -u, --utc Output time in UTC format (default: local time) - -V, --version Prints version information - -OPTIONS: - --csv-timeline Save timeline to CSV file - -d, --directory Event log files directory - -f, --filepath Event file path - --human-readable-timeline Human readable timeline - -l, --lang Output language - -t, --threadnum Number of threads (Default is the number of CPU cores) -```` - -## Usage examples -* Run hayabusa against one Windows event log file: -```` -hayabusa.exe --filepath=eventlog.evtx -```` - -* Run hayabusa against a directory with multiple Windows event log files: -```` -hayabusa.exe --directory=.\evtx -```` - -* Export to a CSV file: -```` -hayabusa.exe --directory=.\evtx --csv-timeline results.csv -```` - -# Hayabusa rules -Hayabusa attack detection rules are written in a SIGMA-like YAML format. - -Please read [AboutRuleCreation-English.md](./doc/AboutRuleCreation-English.md) to understand about how to create rules. - -All of the rules are in the `rules` folder. -You can check out the current rules to use as a template in creating new ones. - # Compiling from source If you have rust installed, you can compile from source with the following command. @@ -82,8 +65,113 @@ If you have rust installed, you can compile from source with the following comma cargo build --release ```` +# Usage +## Command line options +```` +USAGE: + -f --filepath=[FILEPATH] 'File path to one .evtx file' + --csv-timeline=[CSV_TIMELINE] 'Save the timeline in CSV format' + --rfc-2822 'Output date and time in RFC 2822 format. Example: Mon, 07 Aug 2006 12:34:56 -0600' + --rfc-3339 'Output date and time in RFC 3339 format. Example: 2006-08-07T12:34:56.485214 -06:00' + --verbose 'Output verbose information to target event file path and rule file' + -q 'Quiet mode. Do not display the launch banner' + -r --rules=[RULEDIRECTORY] 'Rule file directory (default: ./rules)' + -m --min-level=[LEVEL] 'Minimum level for rules (default: informational)' (Possiblities are: informational, low, medium, high, critical) + -u --utc 'Output time in UTC format (default: local time)' + -d --directory=[DIRECTORY] 'Directory of multiple .evtx files' + -s --statistics 'Prints statistics of event IDs' + -n --show-noisyalerts 'do not exclude noisy rules' + -t --threadnum=[NUM] 'Thread number (default: optimal number for performance)' (Usually there is no performance benefit in increasing the number of threads but you may want to lower to a smaller number to reduce CPU load.) + --contributors 'Prints the list of contributors' +```` + +## Usage examples +* Run hayabusa against one Windows event log file: +```` +hayabusa.exe -f eventlog.evtx +```` + +* Run hayabusa against the sample-evtx directory with multiple Windows event log files: +```` +hayabusa.exe -d .\sample-evtx +```` + +* Export to a single CSV file for further analysis with excel or timeline explorer: +```` +hayabusa.exe -d .\sample-evtx --csv-timeline results.csv +```` + +* Only run hayabusa rules: +```` +hayabusa.exe -d .\sample-evtx --csv-timeline results.csv -r ./rules/hayabusa +```` + +* Only run sigma rules and show noisy alerts (disabled by default): +```` +hayabusa.exe -d .\sample-evtx --csv-timeline results.csv -r ./rules/sigma --show-noisyalerts +```` + +* Only run rules to analyze logons and output in the UTC timezone: +```` +hayabusa.exe -d .\sample-evtx --csv-timeline results.csv -r ./rules/hayabusa/default/events/Security/Logons -u +```` + +* Run on a live Windows machine (requires Administrator privileges) and only detect alerts (potentially malicious behavior): +```` +hayabusa.exe -d C:\Windows\System32\winevt\Logs -m low +```` + +* Get event ID statistics: +```` +hayabusa.exe -d C:\Windows\System32\winevt\Logs -s +```` + +## Testing hayabusa out on sample evtx files +We have provided some sample evtx files for you to test hayabusa and/or create new rules at [https://github.com/Yamato-Security/hayabusa-sample-evtx](https://github.com/Yamato-Security/hayabusa-sample-evtx) + +# Hayabusa rules +Hayabusa detection rules are written in a sigma-like YAML format and are located at [https://github.com/Yamato-Security/hayabusa-rules](https://github.com/Yamato-Security/hayabusa-rules). + +Please read [AboutRuleCreation-English.md](./doc/AboutRuleCreation-English.md) to understand about the rule format how to create rules. + +All of the rules are in the `rules` folder. +`informational` level rules are considered `events`, while anything rated `low` and higher are considered `alerts`. + +The hayabusa rule directory structure is separated into 3 directories: + * `default`: logs that are turned on by default + * `non-default`: logs that need to be turned on through group policy + * `sysmon`: logs that are generated by [sysmon](https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon). + * `testing`: a temporary directory to put rules that you are currently testing + +Rules are further seperated into directories by log type (Example: Security, System, etc...) and are named in the following format: + * Alert format: `__.yml` + * Alert example: `1102_IndicatorRemovalOnHost-ClearWindowsEventLogs_SecurityLogCleared.yml` + * Event format: `_.yml` + * Event example: `4776_NTLM-LogonToLocalAccount.yml` + +Please check out the current rules to use as a template in creating new ones or for checking the detection logic. + +## Hayabusa v.s. converted Sigma rules +Sigma rules need to first be converted to hayabusa rule format explained [here](https://github.com/Yamato-Security/hayabusa/blob/main/tools/sigmac/README-English.md). Hayabusa rules are designed solely for Windows event log analysis and have the following benefits: +1. An extra `output` field to display additional information taken from only the useful fields in the log. +2. They are all tested against sample logs and are known to work. + > Some sigma rules may not work as intended due to bugs in the conversion process, unsupported features, or differences in implementation (such as in regular expressions). +3. Japanese output in the `title_jp` and `output_jp` field. + +**Limitations**: To our knowledge, hayabusa provides the greatest support for sigma rules out of any open source Windows event log analysis tool, however, there are still rules that are not supported: +1. Rules that use regular expressions that do not work with the [Rust regex crate](https://docs.rs/regex/1.5.4/regex/) +2. Rules that use `1 of them` or `all of them` +3. Rules that use the following modifiers: `base64`, `base64offset`, `utf16le`, `utf16be`, `wide`, `utf16`. + +## Detection rule tuning +Like firewalls and IDSes, any signature-based tool will require some tuning to fit your environment so you may need to permanently or temporarily exclude certain rules. + +You can add a rule ID (Example: 4fe151c2-ecf9-4fae-95ae-b88ec9c2fca6) to `config/exclude-rules.txt` in order to ignore any rule you do not need. + +You can also add a rule ID to `config/noisy-rules.txt` in order to ignore the rule by default but still be able to use the rule with the `-n` or `--show-noisyalerts` option. + # Other Windows event log analyzers and related projects -There is no "one tool to rule them all" and we have found that each have their own merits so we recommend checking out these other great tools and projects and see which ones you like. +There is no "one tool to rule them all" and we have found that each has its own merits so we recommend checking out these other great tools and projects and seeing which ones you like. - [APT-Hunter](https://github.com/ahmedkhlief/APT-Hunter) - Attack detection tool written in Python. - [Chainsaw](https://github.com/countercept/chainsaw) - A similar SIGMA based attack detection tool written in Rust. @@ -94,17 +182,31 @@ There is no "one tool to rule them all" and we have found that each have their o - [EVTX parser](https://github.com/omerbenamram/evtx) - the Rust library we used written by [@OBenamram](https://twitter.com/obenamram). - [LogonTracer](https://github.com/JPCERTCC/LogonTracer) - A graphical interface to visualize logons to detect lateral movement by [JPCERTCC](https://twitter.com/jpcert_en). - [RustyBlue](https://github.com/Yamato-Security/RustyBlue) - Rust port of DeepBlueCLI by [Eric Conrad](https://twitter.com/eric_conrad). -- [SIGMA](https://github.com/SigmaHQ/sigma) - Generic SIEM rules. +- [Sigma](https://github.com/SigmaHQ/sigma) - Community based generic SIEM rules. - [so-import-evtx](https://docs.securityonion.net/en/2.3/so-import-evtx.html) - Import evtx files into Security Onion. - [Timeline Explorer](https://ericzimmerman.github.io/#!index.md) - The best CSV timeline analyzer by [Eric Zimmerman](https://twitter.com/ericrzimmerman). +- [Windows Event Log Analysis - Analyst Reference](https://www.forwarddefense.com/media/attachments/2021/05/15/windows-event-log-analyst-reference.pdf) - by Forward Defense's Steve Anson. - [Zircolite](https://github.com/wagga40/Zircolite) - SIGMA based attack detection tool written in Python. -## License +## Comparison to other similar tools that support sigma +It is not possible to do a perfect comparison as these tools support a different number of sigma rules. +Hayabusa supports the largest number of sigma rules as well as will run additional hayabusa rules so will may take more time than other tools that do not do as much analysis. +Also, time and memory usage will differ dramatically depending on what sample event log files are used, command-line options, rule tuning, etc... so please understand that results will vary. -Hayabusa is released under GPLv3 and all rules are release under the Detection Rule License (DRL) 1.1 +The following were taken based on approximately 500 logs (130MB) from our sample-evtx repository at 2021/12/09. -## Contributing +| | Elapsed Time | Memory Usage | Total Sigma Events Detected | Unique Sigma Events Detected | +| :---: | :---: | :---: | :---: | :---: | +| Chainsaw | 10 seconds | 75 MB | 552 | 170 | +| Hayabusa | xx | xx | 9783 | 265 | +| Zircolite | 55 seconds | 400 MB | 1954 | 237 | -We would love any form of contributing. Pull requests and rule creation are the best but feature requests, notifying us of bugs, etc... are also very welcome. +# License + +Hayabusa is released under GPLv3 and all rules are released under the Detection Rule License (DRL) 1.1 + +# Contribution + +We would love any form of contribution. Pull requests, rule creation and sample evtx logs are the best but feature requests, notifying us of bugs, etc... are also very welcome. At the least, if you like our tool then please give us a star on Github and show your support! diff --git a/art/christmas.txt b/art/christmas.txt new file mode 100644 index 00000000..48a94f93 --- /dev/null +++ b/art/christmas.txt @@ -0,0 +1,13 @@ + ／⌒＼、__／⌒￣} + ＼＿＿（__）＿＿/ + 〃〓／￣ > < ￣＼〓〃 + ミ☆／： （:: ::）：： >☆彡 + ★≡〃＼／〉:: ::〈＼／ ≡〃★ + ●※○ ^^^^^^^^ ○※● +〃≡★ Merry Christmas ★≡〃 + ☆〓 〓☆ + 〃≡★ （;) ★≡〃 + ●※○- ,＿】【＿, ,-○※● + ★〃≡〓 ○ 〓≡〃★ + ミ☆-★※★-☆彡 + ● \ No newline at end of file diff --git a/art/happynewyear.txt b/art/happynewyear.txt new file mode 100644 index 00000000..14c2bc91 --- /dev/null +++ b/art/happynewyear.txt @@ -0,0 +1,10 @@ + _〆 + (∴) + (￣￣￣) + ＜(￣￣￣￣)＞ + [二◆二二◆二] + |◇ ● ◇| + |◆ ◆| + |＿＿＿＿| + + A Happy New Year!! diff --git a/art/logo.txt b/art/logo.txt index 9e74bef3..3e2f102e 100644 --- a/art/logo.txt +++ b/art/logo.txt @@ -1,4 +1,4 @@ -██╗ ██╗ █████╗ ██╗ ██╗ █████╗ ██████╗ ██╗ ██╗███████╗ █████╗ +██╗ ██╗ █████╗ ██╗ ██╗ █████╗ ██████╗ ██╗ ██╗███████╗ █████╗ ██║ ██║██╔══██╗╚██╗ ██╔╝██╔══██╗██╔══██╗██║ ██║██╔════╝██╔══██╗ ███████║███████║ ╚████╔╝ ███████║██████╔╝██║ ██║███████╗███████║ ██╔══██║██╔══██║ ╚██╔╝ ██╔══██║██╔══██╗██║ ██║╚════██║██╔══██║ diff --git a/art/ninja.txt b/art/ninja.txt new file mode 100644 index 00000000..505ddce6 --- /dev/null +++ b/art/ninja.txt @@ -0,0 +1,37 @@ + + Today is Ninja Day (2/22)! + + .`,I>>+<;"' + .,}u#zcccccz*#W&jI. + `\&zccccccccccccccz&B~ + `u8zcccccccccccccccccc*%v. + "BMcccccccccccccccccccczWBn + . '$Wccccccccccccccccccccc*&1 + 't$@x. !@zccccMccccccccccccccccccM/ + ^|$%M8i. +Wcccc&McccccccccccccMccccc$[ + .1&v&$M^ `cccc*$#WWMMM##***zzc%*ccccB$ + ^c$#8$} ,*ccc8@xn@x]}{11{f$cn#%cccz8< + ;B$B@z' IBccz$%(z$x}[[[]])$#/-$ccccW( + .u%M$$< '$zcW$t .`^",""`'. c*ccz@^ + 'c$$*%c' }Wz%$i ..'`'. 1Mcz%; + -$@B$%, cz%$Mr1|v&&&&88#j[;,tWzB) + ,%Bzz@{ 'W8$#cccccz&WMcccccc88@z..^,l~+?-+!,' + .f$&c#*'.z$zcccccc$:c#ccccc8$%tM&WWMMMMMW%$$#' + l@8W$B,.tWzccccc@,?&ccczWc_{tzBB%%8%B@$$@r]` + 'c$$$${~$@Wzcccc']8czWB8@, .'^+v$$$$$$$%j~` + `?*$$$$$$$MM%8M*c j88%MW$$%uz@$%f!``""^`. + ^f$$$$$8W#z8%zc*W$r *Wz*z*M%zcccz#M&%x` + .ItB$&WM*z$&cccz@$%&W$; vMM#vW@*cccccccccz\:`. + ;@$$B*cccccM$*cccW$$~c$' ~n}?(@*ccccccccccccc#8#{,. + '"_zWM#ccccccccB%ccczB$$$B1+`,$$$#cccccccccccccccczM%B[. + `/8zcccccccccccccM$WccczB$z#Mz@`z$WcccczWcccccccccccccc#@B: + [B#cccccccccccccccc%$*ccc&MnnBz@`%$8cccc8&cccccccccccccccz8$! + .r%zcccccccccccccccccz$%cc8@nnnun%#$$WcccM@zccccccccccccccccc&$i + .u@zcccccccccccccccccccW$W&$Wnnnn8/Wu%Mcc*$Mccccccccccccccccccc8$] + z$WcccccccccccccccccccccB$$BvnnnM")vnu@&z%Wccccccccccccccccccccz%$j. + l$@zcccccccccccccccccccz&$&znnnn#/ tznnu*B@*cccccccccccccccccccccz@$#' + c$Wcccccccccc*Wzccccc#8B#unnnnnc$$cWBunnnn#$BW*ccccccccccccccz*ccc#$$W' + '$@zccccccccccz$BzczW%Wvnnnnnnv%$$$*M$@nnnnnuW$$@&zcccccccccc#@*cccc%$$# + [$#ccccccccccccB$%WMcnnnnnnnnz$$$B&cc#@8nnnnnnu#@$$&*cccccccMB*ccccc#$$$, + @%ccccccccccccz$#cxcnnnnnnnnM$$$@zcccc*$8nnnnnnnnW8$$%MMMM*#&zccccccc@$$| + "$*cccccccccccc#$cnx@WnnnnnnW$$$$Wccccc#@$8unnnn*@Wu&@$$$$$$@#cccccccc&$$W diff --git a/art/takoyaki.txt b/art/takoyaki.txt new file mode 100644 index 00000000..29320d36 --- /dev/null +++ b/art/takoyaki.txt @@ -0,0 +1,43 @@ +@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ +@@#------@@@@@@@@--------@@@@@@@@--------@@@@@@@@--------@@@@@@@@--------@@@@@@@@------#@@ +@@* @@@@@@@% @@@@@@@@ @@@@@@@@ @@@@@@@@ @@@@@@@@ *@@ +@@* @@@@@@@% @@@@@@@@ @@@@@@@@ @@@@@@@@ @@@@@@@@ *@@ +@@#------********--------********--------********--------********--------********=-----#@@ +@@@@@@@@@ @@@@@@@# @@@@@@@@ @@@@@@@@ @@@@@@@% @@@@@@@@@ +@@@@@@@@@ @@@@@@@# @@@@@@@@ @@@@@@@@ @@@@@@@% @@@@@@@@@ +@@@@@@@@@ @@@@@@@# @@@@@@@@ @@@@@@@@ @@@@@@@% @@@@@@@@@ +@@@@@@@@@-------=@@@@@@@%-------=@@@@@@@@-------=@@@@@@@@-------=@@@@@@@@-------=@@@@@@@@@ +@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ +@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@=:@@@@@@@ +@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@*.+@@@@@@@@ +@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@= *@@@@@@@@@ +@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@%:-@@@@@@@@@@@ +@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@*.*@@@@@@@@@@@@ +@@@@@@@@@@@%*=-:::-=*@@@@@@@@@%*=-:::-+#@@@@@@@@@#+=-::-=+#@@@@@@@@@#+--::: =%@@@@@@@@@@@@ +@@@@@@@@@*: = :#@@@@+. -%@@@@= =@@@@%- .+@@@@@@@@@@ +@@@@@@@@: :: #* -@%. - *= =@* -. =#. *@+ :: -#: .#@@@@@@@@ +@@@@@@@. : -. . : .= : : = . %@@@@@@@ +@@@@@@+ =*. -+=. .++ =+= :*= =+- -*- .++- . -@@@@@@@ +@@@@@@= *@@@@=-#@@@@+:+@% .*@@@%==#@@@@=:*@# :#@@@#==%@@@%--#@+ -%@@@*-=@@@@#--%@-:@@@@@@@ +@@@@-== #@@@@@@@@@@@@@@@+ %@@@@@@@@@@@@@@@- .@@@@@@@@@@@@@@@@. :@@@@@@@@@@@@@@@@ :=:@@@@@ +@@@@#. =*#@@@@@@@@@@@* *-.%@@@@@@@@@@@@@= #::@@@@@@@@@@@@@@:.# -@@@@@@@@@@@%*=. .*@@@@@ +@@@@@@#- .:-=+*#%*::%@@*.-#@@@@@@@@@+.-%@@=.=%@@@@@@@@%=.=@@@=.+%#*+=-:. :*@@@@@@@ +@@@@@@@@@*- ..::- :====-. .=++++++: .-====-. :::.. :*@@@@@@@@@@ +@@@@@@@@@@@@%+-. .-+#@@@@@@@@@@@@@ +@@@@@@@@@@@@@@@@@%#*+-::. ..:-=*#%@@@@@@@@@@@@@@@@@@ +@@@@@@@@@@@@@@@@@@@@@@@@@@@@%##***++++=============++++***##%@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ +@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ +@@======@@#--%@@#=+@@+=#@#=--=#@#==@@==#@@+-=@@@+=*@@==%==#@@@*====+%@@@#--%@@*=*@%==%==@@ +@@*+ ##@@. .@@* .@= :@- .+*: -# %% +@# -@@- =@: =@ *@@@- -#+ #@@. .@@= .@* % %@ +@@@# @@@: =- -@* .@@ #@@% @# *@% *. *@- =@@ #@@@= =@@= -@- -= -@@+ .#@:.@@ +@@@# @@+ :: ** .@: :@+ -- +@@# #@@: .:: %- =%. =@ *@@@- :+- .%* :: +@@+ @@@=-@@ +@@@%++@@*+%@@#+*%+*@@*+#@@*++*@@@@@**@@@++@@@*+##+#@%++%++%@@@#+++*#@@*+%@@%+*@@%**@@@*+@@ +@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ +@@@@@@@@@:.......@@@@@@@@:.......@@@@@@@@:.......@@@@@@@@:.......%@@@@@@@:.......@@@@@@@@@ +@@@@@@@@@ @@@@@@@@ @@@@@@@@ @@@@@@@@ %@@@@@@@ @@@@@@@@@ +@@@@@@@@@ @@@@@@@@ @@@@@@@@ @@@@@@@@ %@@@@@@@ @@@@@@@@@ +@@@@@@@@@ @@@@@@@@ @@@@@@@@ @@@@@@@@ %@@@@@@@ @@@@@@@@@ +@@#:::::-########::::::::########::::::::########::::::::########::::::::########::::::#@@ +@@* .@@@@@@@@ @@@@@@@@ @@@@@@@@ @@@@@@@@ @@@@@@@@ *@@ +@@* .@@@@@@@@ @@@@@@@@ @@@@@@@@ @@@@@@@@ @@@@@@@@ *@@ +@@%+++++*@@@@@@@@++++++++@@@@@@@@++++++++@@@@@@@@++++++++@@@@@@@@++++++++@@@@@@@@++++++%@@ \ No newline at end of file diff --git a/config/exclude-rules.txt b/config/exclude-rules.txt index 47044f72..201932cc 100644 --- a/config/exclude-rules.txt +++ b/config/exclude-rules.txt @@ -1,5 +1,5 @@ -c92f1896-d1d2-43c3-92d5-7a5b35c217bb # rules/sigma/other/win_exchange_cve_2021_42321.yml (rule parse error) -83809e84-4475-4b69-bc3e-4aad8568612f # rules/sigma/builtin/win_exchange_transportagent.yml (rule parse error) -7b449a5e-1db5-4dd0-a2dc-4e3a67282538 # replaced by hayabusa rule -c265cf08-3f99-46c1-8d59-328247057d57 # replaced by hayabusa rule -66b6be3d-55d0-4f47-9855-d69df21740ea # replaced by hayabusa rule \ No newline at end of file +4fe151c2-ecf9-4fae-95ae-b88ec9c2fca6 +c92f1896-d1d2-43c3-92d5-7a5b35c217bb +7b449a5e-1db5-4dd0-a2dc-4e3a67282538 +c265cf08-3f99-46c1-8d59-328247057d57 +66b6be3d-55d0-4f47-9855-d69df21740ea \ No newline at end of file diff --git a/config/noisy-rules.txt b/config/noisy-rules.txt index 6e03bcf7..1fa83b45 100644 --- a/config/noisy-rules.txt +++ b/config/noisy-rules.txt @@ -1,5 +1,6 @@ -0f06a3a5-6a09-413f-8743-e6cf35561297 # sysmon_wmi_event_subscription.yml -b0d77106-7bb0-41fe-bd94-d1752164d066 # win_rare_schtasks_creations.yml -66bfef30-22a5-4fcd-ad44-8d81e60922ae # win_rare_service_installs.yml -e98374a6-e2d9-4076-9b5c-11bdb2569995 # win_susp_failed_logons_single_source.yml -6309ffc4-8fa2-47cf-96b8-a2f72e58e538 # win_susp_failed_logons_single_source2.yml \ No newline at end of file +0f06a3a5-6a09-413f-8743-e6cf35561297 +b0d77106-7bb0-41fe-bd94-d1752164d066 +66bfef30-22a5-4fcd-ad44-8d81e60922ae +e98374a6-e2d9-4076-9b5c-11bdb2569995 +6309ffc4-8fa2-47cf-96b8-a2f72e58e538 +b20f6158-9438-41be-83da-a5a16ac90c2b \ No newline at end of file diff --git a/doc/AboutRuleCreation-English.md b/doc/AboutRuleCreation-English.md index e391c336..a4964187 100644 --- a/doc/AboutRuleCreation-English.md +++ b/doc/AboutRuleCreation-English.md @@ -1,45 +1,99 @@ # Rule files -Hayabusa uses detection rules for Windows Event logs in a YAML format file. -It can express complex detection rules by combining not only simple string matching but also regular expressions, AND, OR, and other conditions. -In this section, we will explain how to write the detection rules. +Hayabusa detection rules are written in [YAML](https://en.wikipedia.org/wiki/YAML) format. +It can express complex detection rules by combining not only simple string matching but also regular expressions, `AND`, `OR`, and other conditions. +In this section, we will explain how to write hayabusa detection rules. # Rule file format -The rule file format is described below. +Example: `````` -title: PowerShell Execution Pipeline -description: This rule detect powershell execution pipeline. -author: Zach Mathis +#Author section +author: Eric Conrad, Zach Mathis +creation_date: 2020/11/08 +updated_date: 2021/11/26 + +#Alert section +title: User added to local Administrators group +title_jp: ユーザがローカル管理者グループに追加された +output: 'User: %MemberName% : SID: %MemberSid% : Group: %TargetUserName%' +output_jp: 'ユーザ: %MemberName% : SID: %MemberSid% : グループ名: %TargetUserName%' +description: A user was added to the local Administrators group. +description_jp: ユーザがローカル管理者グループに追加された。 + +#Rule section +id: 611e2e76-a28f-4255-812c-eb8836b2f5bb +level: high +status: stable detection: selection: - Event.System.EventID: 7040 - Event.System.Channel: System + Channel: Security + EventID: 4732 + TargetUserName: Administrators + condition: selection falsepositives: - - unknown -level: medium -output: 'command=%CommandLine%' -creation_date: 2020/11/8 -updated_date: 2020/11/8 + - system administrator +tags: + - attack.persistence + - attack.t1098 +references: + - https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/event.aspx?eventid=4732 +sample-evtx: ./sample-evtx/EVTX-to-MITRE-Attack/TA0003-Persistence/T1098.xxx-Account manipulation/ID4732-User added to local admin groups.evtx +logsource: default +ruletype: Hayabusa `````` +> #Author section +* **author [required]**: Name of the author(s). +* **contributor** [optional]: Name of any contributor(s) (anyone who made any minor corrections). +* **creation_date [required]**: Date the rule was made. +* **updated_date** [optional]: Date the rule was updated. -* title [required]: Rule file title. This will also be the name of the alert that gets displayed so the briefer the better. -* description [optional]: A description of the rule. This does not get displayed so you can make this long. -* author [optional]: The name of the person or persons who created the logic for the rule. -* detection [required]: The detection logic goes here. -* falsepositives [optional]: The possibilities for false positives. For example: `system administrator`, `normal user usage`, `normal system usage`, `legacy application`, `security team`. If it is unknown, write `unknown`. -* level [optional]: Risk level. Please write one of the following: `informational`,`low`,`medium`,`high`,`critical` -* output [required]: The details of the alert. (Please output any and only useful fields in the Windows event log for easy analysis.) -* creation_date [optional]: The creation date. -* updated_date [optional]: The date of the last revision. +> #Alert section +* **title [required]**: Rule file title. This will also be the name of the alert that gets displayed so the briefer the better. (Should not be longer than 85 characters.) +* **title_jp** [optional]: The title in Japanese. +* output [optional]: The details of the alert that gets displayed. Please output any fields in the Windows event log that are useful for analysis. Fields are seperated by `" : "` (two spaces on both sides). Field placeholders are enclosed with a `%` (Example: `%MemberName%`) and need to be defined in `config\eventkey_alias.txt`. (Explained below.) +* **output_jp** [optional]: The output message in Japanese. +* **description** [optional]: A description of the rule. This does not get displayed so you can make this long. +* **description_jp** [optional]: The description in Japanese. -# About detection notation +> #Rule section +* **id [required]**: A randomly generated version 4 UUID used to uniquely identify the rule. You can generate one [here](https://www.uuidgenerator.net/version4). +* **level [required]**: Severity level based on [sigma's definition](https://github.com/SigmaHQ/sigma/wiki/Specification). Please write one of the following: `informational`,`low`,`medium`,`high`,`critical` +* **status[required]**: `stable` for tested rules and `testing` for rules that need to be tested. +* **detection [required]**: The detection logic goes here. (Explained below.) +* **falsepositives [required]**: The possibilities for false positives. For example: `system administrator`, `normal user usage`, `normal system usage`, `legacy application`, `security team`, `none`. If it is unknown, please write `unknown`. +* **tags** [optional]: If the technique is a [LOLBINS/LOLBAS](https://lolbas-project.github.io/) technique, please add the `lolbas` tag. If the alert can be mapped to a technique in the [MITRE ATT&CK](https://attack.mitre.org/) framework, please add the tactic ID (Example: `attack.t1098`) and any applicable tactics below: + * attack.impact -> Impact + * attack.initial_access -> Initial Access + * attack.execution -> Execution + * attack.lateral_movement -> Lateral Movement + * attack.persistence -> Persistence + * attack.privilege_escalation -> Privilege Escalation + * attack.reconnaissance -> Reconnaissance + * attack.collection -> Collection + * attack.command_and_control -> Command and Control + * attack.credential_access -> Credential Access + * attack.defense_evasion -> Defense Evasion + * attack.discovery -> Discovery + * attack.exfiltration -> Exfiltration + * attack.resource_development -> Resource Development +* **references** [optional]: Any links to references. +* **sample-evtx [required]**: File path or URL to an event log file that this rule will detect. +* **logsource [required]**: The source of where the log comes from. Please specify one of the following: + * `default`: For logs that are turned on in Windows by default. + * `non-default`: For logs that need to be turned on through group policy, etc... + * `sysmon`: Logs that require sysmon to be installed. +* **non-default-setting** [optional]: Explanation of how to turn on the log setting for `non-default` log sources. +* **ruletype [required]**: `Hayabusa` for hayabusa rules. Rules automatically converted from sigma will be `Sigma`. + +# Detection field ## Detection fundamentals First, the fundamentals of how to create a detection rule will be explained. ### How to write AND and OR logic -To write AND logic, we use the YAML hash. -The detection rule below defines that *both conditions* have to be true in order for the rule to match. +To write AND logic, we use nested dictionaries. +The detection rule below defines that **both conditions** have to be true in order for the rule to match. * EventID has to exactly be `7040`. +* **AND** * Channel has to exactly be `System`. `````` @@ -47,11 +101,13 @@ detection: selection: Event.System.EventID: 7040 Event.System.Channel: System + condition: selection `````` -To write OR logic, we use arrays. -In the detection rule below, *either one* of the conditions will result in the rule being triggered. -* EventID has to exactly be `7040`. +To write OR logic, we use lists (Dictionaries that start with `- `). +In the detection rule below, **either one** of the conditions will result in the rule being triggered. +* EventID has to exactly be `7040`. +* **OR** * Channel has to exactly be `System`. `````` @@ -59,11 +115,13 @@ detection: selection: - Event.System.EventID: 7040 - Event.System.Channel: System + condition: selection `````` -We can also combine AND and OR logic as shown below. +We can also combine `AND` and `OR` logic as shown below. In this case, the rule matches when the following two conditions are both true. -* EventID is either exactly `7040` or `7041`. +* EventID is either exactly `7040` **OR** `7041`. +* **AND** * Channel is exactly `System`. `````` @@ -73,10 +131,11 @@ detection: - 7040 - 7041 Event.System.Channel: System + condition: selection `````` -### Eventkey -The following is an excerpt of a Windows event log, formatted in the original XML. The `Event.System.Channel` in the rule file example refers to the XML `System`. For Windows event logs in XML format, as in this example, if you want to specify the values contained in the nested XML tags as the condition of the rule, you can specify them by connecting them with a dot (`. `). In the rule file, these strings connected together with dots are called `eventkeys`. +### Eventkeys +The following is an excerpt of a Windows event log, formatted in the original XML. The `Event.System.Channel` field in the rule file example above refers to the original XML tag: `System`. Nested XML tags are replaced by tag names seperated by dots (`.`). In hayabusa rules, these field strings connected together with dots are refered to as `eventkeys`. `````` @@ -91,18 +150,22 @@ The following is an excerpt of a Windows event log, formatted in the original XM `````` -### Eventkey Aliases +#### Eventkey Aliases Long eventkeys with many `.` seperations are common, so hayabusa will use aliases to make them easier to work with. Aliases are defined in the `config\eventkey_alias.txt` file. This file is a CSV file made up of `alias` and `event_key` mappings. You can rewrite the rule above as shown below with aliases making the rule easier to read. `````` detection: selection: - EventID: 7040 Channel: System + EventID: 7040 + condition: selection `````` +#### Caution: Undefined Eventkey Aliases +Not all eventkey aliases are defined in `config\eventkey_alias.txt`. If you are not getting the correct data in the `output`(Alert details) message, and instead are getting results like `%EventID%`, then you need to update `config\eventkey_alias.txt` with a new alias. + ### How to use XML attributes in conditions -When Windows event logs are output in XML format, the XML attributes may have values set in them. In the example below, `Name` in `Provider Name` is an XML attribute. +XML elements may have attributes set by adding a space to the element. For example, `Name` in `Provider Name` below is an XML attribute of the `Provider` element. ```````````` @@ -115,7 +178,7 @@ When Windows event logs are output in XML format, the XML attributes may have va ```````````` -To specify XML attributes in an eventkey, use the format `{eventkey}_attributes.{attribute_name}`. For example, to specify the `Name` attribute of the `Provider` tag in the rule file, it would look like this: +To specify XML attributes in an eventkey, use the format `{eventkey}_attributes.{attribute_name}`. For example, to specify the `Name` attribute of the `Provider` element in a rule file, it would look like this: `````` detection: @@ -123,10 +186,11 @@ detection: Channel: Security EventID: 4672 Event.System.Provider_attributes.Name: 'Microsoft-Windows-Security-Auditing' + condition: selection `````` ### EventData -Windows event logs are divided into two parts: the `System` part where the fundamental data (Event ID, Timestamp, Record ID, Log name (aka Channel)) is written, and the `EventData` part where arbitrary data is written depending on the Event ID. The problem is that the names of the tags nested in EventData are all `Data` so the eventkeys described so far cannot distinguish between `SubjectUserSid` and `SubjectUserName`. +Windows event logs are divided into two parts: the `System` part where the fundamental data (Event ID, Timestamp, Record ID, Log name (Channel)) is written, and the `EventData` part where arbitrary data is written depending on the Event ID. The problem is that the names of the tags nested in EventData are all called `Data` so the eventkeys described so far cannot distinguish between `SubjectUserSid` and `SubjectUserName`. ```````````` @@ -146,19 +210,20 @@ Windows event logs are divided into two parts: the `System` part where the funda ```````````` -To deal with this problem, you can specify the value specified for `Name` in the eventkey instead of `Data`. For example, if you want to use `SubjectUserName` and `SubjectDomainName` in the EventData as a condition of the rule, you can describe it as follows: +To deal with this problem, you can specify the value assigned in `Data Name`. For example, if you want to use `SubjectUserName` and `SubjectDomainName` in the EventData as a condition of a rule, you can describe it as follows: `````` detection: selection: - EventID: 7040 Channel: System + EventID: 7040 Event.EventData.SubjectUserName: hayabusa Event.EventData.SubjectDomainName: DESKTOP-HAYBUSA + condition: selection `````` -### Special patterns in EventData -Some of the tags nested in EventData do not have a Name attribute. +### Abnormal patterns in EventData +Some of the tags nested in `EventData` do not have a `Name` attribute. `````` @@ -175,7 +240,7 @@ Some of the tags nested in EventData do not have a Name attribute. `````` -To detect an event log like the one above, you can specify an eventkey named `EventData`. In this case, the condition will matach as long as any one of the nested tags without a Name attribute matches. +To detect an event log like the one above, you can specify an eventkey named `EventData`. In this case, the condition will match as long as any one of the nested tags without a `Name` attribute matches. `````` detection: @@ -183,145 +248,234 @@ detection: Channel: Security EventID: 5379 EventData: None + condition: selection `````` ## Pipes -A pipe can be used with eventkeys as shown below. All of the conditions we have described so far use exact matches, but by using pipes, you can describe more flexible detection rules. In the following example, if the value of `CommandLine` matches the regular expression `yamato.*hayabusa`, it will match the condition. +A pipe can be used with eventkeys as shown below for matching strings. All of the conditions we have described so far use exact matches, but by using pipes, you can describe more flexible detection rules. In the following example, if the value of `EventData` matches the regular expression `[\s\S]*EngineVersion=2\.0[\s\S]*`, it will match the condition. `````` detection: selection: - EventID: 7040 - Channel: System - CommandLine|re: yamato.*hayabusa + Channel: Microsoft-Windows-PowerShell/Operational + EventID: 400 + EventData|re: '[\s\S]*EngineVersion=2\.0[\s\S]*' + condition: selection `````` This is a list of what you can specify after the pipe. At the moment, hayabusa does not support chaining multiple pipes together. * startswith: Checks the string from the beginning * endswith: Checks the end of the string * contains: Checks if a word is contained in the data -* re: Use regular expressions. (We are using the regex crate so please out the documentation at https://docs.rs/regex/1.5.4/regex/ on how to write regular expressions.) +* re: Use regular expressions. (We are using the regex crate so please out the documentation at https://docs.rs/regex/1.5.4/regex/ to know how to write correct regular expressions.) + > Caution: Some sigma rules that use regular expressions may fail to detect due to differences in how rust uses regular expressions. ## Wildcards -Wildcards can be specified for the value corresponding to eventkey. In the example below, if the CommandLine starts with the string "hayabusa", it will be processed as if it matches the condition. -The specification is fundamentally the same as SIGMA rule wildcards. +Wildcards can be used in eventkeys. In the example below, if `ProcessCommandLine` starts with the string "malware", the rule will match. +The specification is fundamentally the same as sigma rule wildcards. `````` detection: selection: - EventID: 7040 - Channel: System - CommandLine: hayabusa* + Channel: Security + EventID: 4688 + ProcessCommandLine: malware* + condition: selection `````` -The following wildcards can be used. +The following two wildcards can be used. * `*`: Matches any string of zero or more characters. (Internally it is converted to the regular expression `. *`.) -* `? `: Matches any single character. (Internally converted to the regular expression `. `.) +* `?`: Matches any single character. (Internally converted to the regular expression `. `.) -When wildcards are used, they will be interpreted according to the following rules. -* Wildcards (`*` and `?`) can be escaped by using a backslash (`/`). +About escaping wildcards: +* Wildcards (`*` and `?`) can be escaped by using a backslash: `\*`, `\?`. * If you want to use a backslash right before a wildcard then write `\\*` or `\\?`. -* No escaping is required if you are using backslashes by themselves. +* Escaping is not required if you are using backslashes by themselves. -## Nesting keywords in the eventkey -The eventkey can be nested with specific keywords. -In the example below, if the value of CommandLine matches the wildcard `aa*bb` and the length of the string is 10 or more, the condition will match. +## Nesting keywords inside eventkeys +Eventkeys can be nested with specific keywords. +In the example below, the rule will match if the following are true: +* `ServiceName` is called `malicious-service` or contains a regular expression in `./config/regex/regexes_suspicous_service.txt`. +* `ImagePath` has a minimum of 1000 characters. +* `ImagePath` does not have any matches in the `allowlist`. `````` detection: selection: - EventID: 7040 Channel: System - CommandLine: - value: aa*bb - min_length: 10 + EventID: 7045 + ServiceName: + - value: malicious-service + - regexes: ./config/regex/regexes_suspicous_services.txt + ImagePath: + min_length: 1000 + allowlist: ./config/regex/allowlist_legitimate_serviceimage.txt + condition: selection `````` -Currently, the following keywords can be specified. -* value: matches by string (wildcards and pipes can also be specified). -* min_length: matches when the number of characters is greater than or equal to the specified number. -* regexes: matches if one of the regular expressions in the file that you specify in this field matches. -* allowlist: rule will be skipped if there is any match found in the list of regular expressions in the file that you specify in this field. +Currently, the following keywords can be specified: +* `value`: matches by string (wildcards and pipes can also be specified). +* `min_length`: matches when the number of characters is greater than or equal to the specified number. +* `regexes`: matches if one of the regular expressions in the file that you specify in this field matches. +* `allowlist`: rule will be skipped if there is any match found in the list of regular expressions in the file that you specify in this field. -### regexes and allowlist -Hayabusa provides built-in rules using regular expressions (defined in `config/regex/regexes_suspicous_service.txt`) and allowlisting (defined in `config/regex/allowlist_legimate_serviceimage.txt`). -regexes and allowlist can be edited to change the behavior of all referenced rules at once. +### regexes and allowlist keywords +Hayabusa has two built-in regular expression files: +* `./config/regex/regexes_suspicous_services.txt`: to detect suspicious service names +* `./config/regex/allowlist_legitimate_serviceimage.txt`: to allow legitimate services + +Files defined in `regexes` and `allowlist` can be edited to change the behavior of all rules that reference them without having to change any rule file itself. -You can also specify to use different regexes and allowlist textfiles. -Please refer to the default regexes_suspicous_service.txt and allowlist_legimate_serviceimage.txt when creating your own. +You can also use different regexes and allowlist textfiles that you create. +Please refer to the default `./config/regexes_suspicous_services.txt` and `./config/allowlist_legitimate_serviceimage.txt` when creating your own. ## condition With the notation we explained above, you can express AND and OR logic but it will be confusing if you are trying to define complex logic. -When you want to make more complex rules, you should use `condition` as shown below. +When you want to make more complex rules, you should use the `condition` keyword as shown below. `````` detection: - selection_1: - EventID: 7040 - selection_2: - EventID: 7041 - selection_3: - Channel: System - selection_4: - CommandLine|contains: lsass.exe - selection_5: - CommandLine|contains: services.exe - selection_6: - ParentProcessName|contains: wininit.exe - - condition: ( selection_1 or selection_2 ) and selection_3 and ( selection_4 or selection_5 ) and ( not selection_6 ) + SELECTION_1: + EventID: 3 + SELECTION_2: + Initiated: 'true' + SELECTION_3: + DestinationPort: + - '4444' + - '666' + SELECTION_4: + Image: '*\Program Files*' + SELECTION_5: + DestinationIp: + - 10.* + - 192.168.* + - 172.16.* + - 127.* + SELECTION_6: + DestinationIsIpv6: 'false' + condition: (SELECTION_1 and (SELECTION_2 and SELECTION_3) and not ((SELECTION_4 or (SELECTION_5 and SELECTION_6)))) `````` -The following keywords can be used for `condition`. -* {expression1} and {expression2}: Require both {expression1} AND {expression2} -* {expression1} or {expression2}: Require either {expression1} OR {expression2} -* not {expression}: Reverse the logic of {expression} -* ( {expression} ) : Set precedance of {expression}. It follows the same precedance logic as in mathematics. +The following expressions can be used for `condition`. +* `{expression1} and {expression2}`: Require both {expression1} AND {expression2} +* `{expression1} or {expression2}`: Require either {expression1} OR {expression2} +* `not {expression}`: Reverse the logic of {expression} +* `( {expression} )`: Set precedance of {expression}. It follows the same precedance logic as in mathematics. -In the above example, names such as selection_1 and selection_2 are used for grouping conditional expressions, but there is no need to add the prefix "selection", and the user can define any name. However, the only characters that can be used are those that match the regular expression `\w`. +In the above example, selection names such as `SELECTION_1`, `SELECTION_2`, etc... are used but they can be named anything as long as they only contain the following characters: `a-z A-Z 0-9 _` +> However, please use the standard convention of `selection_1`, `selection_2`, `filter_1`, `filter_2`, etc... to make things easy to read whenever possible. ## aggregation condition -The condition field described above implements not only the AND and OR conditions, but also a function to aggregate the detected event logs. -This function is called the aggregation condition, and is specified by connecting it with a pipe after the condition. -In the example below, a conditional expression is used to determine if there are three or more `DestinationIp` values for each `SubjectUserName`. +### Basics +The `condition` keyword described above implements not only `AND` and `OR` logic, but is also able to count or "aggregate" events. +This function is called the "aggregation condition" and is specified by connecting a condition with a pipe. +In the example below, a conditional expression is used to determine if there are ten or more `AccountName` values for any given `ComputerName` within a timeframe of 24 hours. `````` detection: - selection: - EventID: 7040 - Channel: System - condition: selection | count(DestinationIp) by SubjectUserName >= 3 + selection: + Channel: Security + EventID: 4648 + condition: selection | count(AccountName) by ComputerName >= 10 + timeframe: 24h `````` -The aggregation condition can be defined in the following format. Note that {number} must be a number. -* `count() {operator} {number}`: For log events that match the first condition before the pipe, the condition will match if the number of matched logs satisfies the condition expression specified by {operator} and {number}. +The aggregation condition can be defined in the following format: +* `count() {operator} {number}`: For log events that match the first condition before the pipe, the condition will match if the number of matched logs satisfies the condition expression specified by `{operator}` and `{number}`. -![](count1.png) - -* `count() by {eventkey_2} {operator} {number}`: Log events that match the first condition before pipe are grouped by {eventkey_2}, and if the number of matched events for each grouping satisfies the condition expression specified by {operator} and {number}, the condition will match. - -![](count2.png) - -* `count({eventkey}) {operator} {number}`: Counts how many different values of {eventkey} exist in the log event that match the condition before the condition pipe. If the number satisfies the conditional expression specified in {operator} and {number}, the condition is considered to have been met. - -![](count3.png) - -* `count({eventkey_1}) by {eventkey_2} {operator} {number}`: The logs that match the condition before the condition pipe are grouped by {eventkey_2}, and the number of values of {eventkey_1} in each group is counted. If the values counted for each grouping satisfy the conditional expression specified by {operator} and {number}, the condition is considered to have been met. - -![](count4.png) - -In addition, the following can be specified for the above operator: +`{operator}` can be one of the following: * `==`: If the value is equal to the specified value, it is treated as matching the condition. * `>=`: If the value is greater than or equal to the specified value, the condition is considered to have been met. * `>`: If the value is greater than the specified value, the condition is considered to have been met. * `<=`: If the value is less than or equal to the specified value, the condition is considered to have been met. -* `<`: If the value is less than the specified value, it will be treated as if the condition is met. +* `<`: If the value is less than the specified value, it will be treated as if the condition is met. -# Alert details -You can specify the message (alert details) that will be outputted when the detection condition is met. -In addition to outputting a fixed string, it is also possible to display the value in the event log by enclosing the eventkey in a `%`. -In the example below, the eventkey value ScriptBlockText is used in the message when a detection is made. +`{number}` must be a number. -`````` -output: 'command=%ScriptBlockText%' -`````` \ No newline at end of file +`timeframe` can be defined in the following: +* `15s`: 15 seconds +* `30m`: 30 minutes +* `12h`: 12 hours +* `7d`: 7 days +* `3M`: 3 months +> `timeframe` is not required by highly encouraged to define when possible for performance and memory efficiency. + + +### Four patterns for aggregation conditions: +1. No count argument or `by` keyword. Example: `selection | count() > 10` + > If `selection` is matches more than 10 times within the timeframe, the condition will match. +2. No count argument but there is a `by` keyword. Example: `selection | count() by Date > 10` + > `selection` will have to be true more than 10 times for the **same** `Date`. +3. There is a count argument but no `by` keyword. Example: `selection | count(Users) > 10` + > If `selection` matches and `Users` is **different** more than 10 times within the timeframe, the condition will match. +4. There is both a count argument and `by` keyword. Example: `selection | count(Users) by Date > 10` + > For the **same** `Date`, there will need to be more than 10 **different** `Users` in order for the condition to match. + +### Pattern 1 example: +This is the most basic pattern: `count() {operator} {number}`. The rule below will match if `selection` happens 3 or more times. + +![](count1_EN.png) + +### Pattern 2 example: +`count() by {eventkey} {operator} {number}`: Log events that match the `condition` before the pipe are grouped by the **same** `{eventkey}`. If the number of matched events for each grouping satisfies the condition specified by `{operator}` and `{number}`, then the condition will match. + +![](count2_EN.png) + +### Pattern 3 example: +`count({eventkey}) {operator} {number}`: Counts how many **different** values of `{eventkey}` exist in the log event that match the condition before the condition pipe. If the number satisfies the conditional expression specified in `{operator}` and `{number}`, the condition is considered to have been met. + +![](count3_EN.png) + +### Pattern 4 example: +`count({eventkey_1}) by {eventkey_2} {operator} {number}`: The logs that match the condition before the condition pipe are grouped by the **same** `{eventkey_2}`, and the number of **different** values of `{eventkey_1}` in each group is counted. If the values counted for each grouping satisfy the conditional expression specified by `{operator}` and `{number}`, the condition will match. + +![](count4_EN.png) + +# Rule creation advice +1. **When possible, always specify the `Channel` name.** In the future, we may filter on channel names so your rule may be ignored if the proper `Channel` is not set. +2. **When possible, always specify the `EventID`.** In the future, we may filter on channel names so your rule may be ignored if the proper `EventID` is not set. +3. **Do not use multiple `selection` fields when it is not needed.** For example: + +Bad example: +``` +detection: + SELECTION_1: + EventID: 4625 + SELECTION_2: + LogonType: 3 + condition: SELECTION_1 and SELECTION_2 +``` + +Good example: +``` +detection: + selection: + EventID: 4625 + LogonType: 3 + condition: selection +``` + +4. **Name any of your filtering selections `filter`, `filter_1`, `filter_2`, etc... and do not use excessive grouping.** + +Bad example: +``` +detection: + SELECTION_1: + EventID: 4625 + SELECTION_2: + LogonType: 3 + SELECTION_3: + IpAddress: '-' + condition: ((SELECTION_1 and SELECTION_2) and not (SELECTION_3)) +``` + +Good example: +``` +detection: + selection: + EventID: 4625 + LogonType: 3 + filter: + IpAddress: '-' + condition: selection and not filter +``` \ No newline at end of file diff --git a/doc/count1_EN.png b/doc/count1_EN.png new file mode 100644 index 00000000..f046b707 Binary files /dev/null and b/doc/count1_EN.png differ diff --git a/doc/count2_EN.png b/doc/count2_EN.png new file mode 100644 index 00000000..91dda4e1 Binary files /dev/null and b/doc/count2_EN.png differ diff --git a/doc/count3_EN.png b/doc/count3_EN.png new file mode 100644 index 00000000..80026fc5 Binary files /dev/null and b/doc/count3_EN.png differ diff --git a/doc/count4_EN.png b/doc/count4_EN.png new file mode 100644 index 00000000..a1324718 Binary files /dev/null and b/doc/count4_EN.png differ diff --git a/rules/sigma/builtin/win_hidden_user_creation.yml b/rules/sigma/builtin/win_hidden_user_creation.yml index 526c96dd..45f43c4a 100644 --- a/rules/sigma/builtin/win_hidden_user_creation.yml +++ b/rules/sigma/builtin/win_hidden_user_creation.yml @@ -1,4 +1,3 @@ - title: Hidden Local User Creation author: Christian Burkard date: 2021/05/03 diff --git a/rules/sigma/builtin/win_user_added_to_local_administrators.yml b/rules/sigma/builtin/win_user_added_to_local_administrators.yml index 4fe138b6..06b76c48 100644 --- a/rules/sigma/builtin/win_user_added_to_local_administrators.yml +++ b/rules/sigma/builtin/win_user_added_to_local_administrators.yml @@ -1,4 +1,3 @@ - title: User Added to Local Administrators author: Florian Roth date: 2017/03/14 diff --git a/rules/sigma/builtin/win_user_creation.yml b/rules/sigma/builtin/win_user_creation.yml index aaa45500..27ec53cc 100644 --- a/rules/sigma/builtin/win_user_creation.yml +++ b/rules/sigma/builtin/win_user_creation.yml @@ -1,4 +1,3 @@ - title: Local User Creation author: Patrick Bareiss date: 2019/04/18 diff --git a/rules/sigma/other/win_exchange_cve_2021_42321.yml b/rules/sigma/other/win_exchange_cve_2021_42321.yml index 77e2a949..e17e37cf 100644 --- a/rules/sigma/other/win_exchange_cve_2021_42321.yml +++ b/rules/sigma/other/win_exchange_cve_2021_42321.yml @@ -1,4 +1,3 @@ - title: Possible Exploitation of Exchange RCE CVE-2021-42321 author: Florian Roth, @testanull date: 2021/11/18 diff --git a/screenshots/hayabusa-results-summary.png b/screenshots/hayabusa-results-summary.png new file mode 100644 index 00000000..00b3e73d Binary files /dev/null and b/screenshots/hayabusa-results-summary.png differ diff --git a/screenshots/hayabusa-results.png b/screenshots/hayabusa-results.png new file mode 100644 index 00000000..29299386 Binary files /dev/null and b/screenshots/hayabusa-results.png differ diff --git a/screenshots/hayabusa-start.png b/screenshots/hayabusa-start.png new file mode 100644 index 00000000..d5162e72 Binary files /dev/null and b/screenshots/hayabusa-start.png differ diff --git a/src/detections/configs.rs b/src/detections/configs.rs index da65ad8e..c19079f5 100644 --- a/src/detections/configs.rs +++ b/src/detections/configs.rs @@ -64,6 +64,7 @@ fn build_app<'a>() -> ArgMatches<'a> { -u --utc 'Output time in UTC format (default: local time)' -d --directory=[DIRECTORY] 'Directory of multiple .evtx files' -s --statistics 'Prints statistics of event IDs' + -n --show-noisyalerts 'do not exclude noisy rules' -t --threadnum=[NUM] 'Thread number (default: optimal number for performance)' --contributors 'Prints the list of contributors'"; App::new(&program) diff --git a/src/detections/detection.rs b/src/detections/detection.rs index b7173ffd..10a1542a 100644 --- a/src/detections/detection.rs +++ b/src/detections/detection.rs @@ -11,6 +11,7 @@ use crate::detections::print::MESSAGES; use crate::detections::rule; use crate::detections::rule::RuleNode; use crate::detections::utils::get_serde_number_to_string; +use crate::filter; use crate::yaml::ParseYaml; use std::sync::Arc; @@ -51,10 +52,15 @@ impl Detection { } // ルールファイルをパースします。 - pub fn parse_rule_files(level: String, rulespath: Option<&str>) -> Vec { + pub fn parse_rule_files( + level: String, + rulespath: Option<&str>, + exclude_ids: &filter::RuleExclude, + ) -> Vec { // ルールファイルのパースを実行 let mut rulefile_loader = ParseYaml::new(); - let result_readdir = rulefile_loader.read_dir(rulespath.unwrap_or(DIRPATH_RULES), &level); + let result_readdir = + rulefile_loader.read_dir(rulespath.unwrap_or(DIRPATH_RULES), &level, exclude_ids); if result_readdir.is_err() { AlertMessage::alert( &mut std::io::stderr().lock(), @@ -269,6 +275,6 @@ impl Detection { fn test_parse_rule_files() { let level = "informational"; let opt_rule_path = Some("./test_files/rules/level_yaml"); - let cole = Detection::parse_rule_files(level.to_owned(), opt_rule_path); + let cole = Detection::parse_rule_files(level.to_owned(), opt_rule_path, &filter::exclude_ids()); assert_eq!(5, cole.len()); } diff --git a/src/filter.rs b/src/filter.rs new file mode 100644 index 00000000..113283cf --- /dev/null +++ b/src/filter.rs @@ -0,0 +1,44 @@ +use crate::detections::configs; +use std::collections::HashSet; +use std::fs; + +#[derive(Clone, Debug)] +pub struct RuleExclude { + pub no_use_rule: HashSet, +} + +pub fn exclude_ids() -> RuleExclude { + let mut ids; + match fs::read("config/exclude-rules.txt") { + Ok(file) => ids = String::from_utf8(file).unwrap(), + Err(_) => panic!("config/exclude-rules.txt does not exist"), + }; + + if !configs::CONFIG + .read() + .unwrap() + .args + .is_present("show-noisyalerts") + { + ids += "\n"; // 改行を入れないとexclude-rulesの一番最後の行とnoisy-rules.txtの一番最初の行が一行にまとめられてしまう。 + match fs::read("config/noisy-rules.txt") { + Ok(file) => ids += &String::from_utf8(file).unwrap(), + Err(_) => panic!("config/noisy-rules.txt does not exist"), + }; + } + + let mut exclude_ids = RuleExclude { + no_use_rule: HashSet::new(), + }; + + for v in ids.split_whitespace() { + let v = v.to_string(); + if v.is_empty() { + // 空行は無視する。 + continue; + } + exclude_ids.no_use_rule.insert(v); + } + + return exclude_ids; +} diff --git a/src/lib.rs b/src/lib.rs index 044dac24..9bd8f144 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -1,5 +1,6 @@ pub mod afterfact; pub mod detections; +pub mod filter; pub mod notify; pub mod omikuji; pub mod timeline; diff --git a/src/main.rs b/src/main.rs index d9da8b1d..db64be91 100644 --- a/src/main.rs +++ b/src/main.rs @@ -1,17 +1,20 @@ extern crate serde; extern crate serde_derive; -use chrono::{DateTime, Utc}; +use chrono::Datelike; +use chrono::{DateTime, Local}; use evtx::{EvtxParser, ParserSettings}; use hayabusa::detections::detection; use hayabusa::detections::detection::EvtxRecordInfo; use hayabusa::detections::print::AlertMessage; +use hayabusa::filter; use hayabusa::omikuji::Omikuji; use hayabusa::{afterfact::after_fact, detections::utils}; use hayabusa::{detections::configs, timeline::timeline::Timeline}; use hhmmss::Hhmmss; use pbr::ProgressBar; use serde_json::Value; +use std::collections::HashMap; use std::{ fs::{self, File}, path::PathBuf, @@ -19,12 +22,18 @@ use std::{ }; // 一度にtimelineやdetectionを実行する行数 -const MAX_DETECT_RECORDS: usize = 40000; +const MAX_DETECT_RECORDS: usize = 5000; fn main() { + let analysis_start_time: DateTime = Local::now(); if !configs::CONFIG.read().unwrap().args.is_present("q") { output_logo(); println!(""); + output_eggs(&format!( + "{:02}/{:02}", + &analysis_start_time.month().to_owned(), + &analysis_start_time.day().to_owned() + )); } if configs::CONFIG.read().unwrap().args.args.len() == 0 { println!( @@ -33,7 +42,6 @@ fn main() { ); return; } - let analysis_start_time: DateTime = Utc::now(); if let Some(filepath) = configs::CONFIG.read().unwrap().args.value_of("filepath") { if !filepath.ends_with(".evtx") { AlertMessage::alert( @@ -64,7 +72,7 @@ fn main() { print_contributors(); return; } - let analysis_end_time: DateTime = Utc::now(); + let analysis_end_time: DateTime = Local::now(); let analysis_duration = analysis_end_time.signed_duration_since(analysis_start_time); println!("Elapsed Time: {}", &analysis_duration.hhmmssxxx()); println!(""); @@ -120,11 +128,12 @@ fn analysis_files(evtx_files: Vec) { .value_of("level") .unwrap_or("informational") .to_uppercase(); + println!("Analyzing event files: {:?}", evtx_files.len()); - println!("Analyzing Event Files: {:?}", evtx_files.len()); let rule_files = detection::Detection::parse_rule_files( level, configs::CONFIG.read().unwrap().args.value_of("rules"), + &filter::exclude_ids(), ); let mut pb = ProgressBar::new(evtx_files.len() as u64); let mut detection = detection::Detection::new(rule_files); @@ -248,12 +257,30 @@ fn _output_with_omikuji(omikuji: Omikuji) { println!("{}", content); } +/// output logo fn output_logo() { let fp = &format!("art/logo.txt"); - let content = fs::read_to_string(fp).unwrap(); + let content = fs::read_to_string(fp).unwrap_or("".to_owned()); println!("{}", content); } +/// output easter egg arts +fn output_eggs(exec_datestr: &str) { + let mut eggs: HashMap<&str, &str> = HashMap::new(); + eggs.insert("01/01", "art/happynewyear.txt"); + eggs.insert("02/22", "art/ninja.txt"); + eggs.insert("08/08", "art/takoyaki.txt"); + eggs.insert("12/25", "art/christmas.txt"); + + match eggs.get(exec_datestr) { + None => {} + Some(path) => { + let content = fs::read_to_string(path).unwrap_or("".to_owned()); + println!("{}", content); + } + } +} + #[cfg(test)] mod tests { use crate::collect_evtxfiles; diff --git a/src/yaml.rs b/src/yaml.rs index 075cafee..33b50a6a 100644 --- a/src/yaml.rs +++ b/src/yaml.rs @@ -3,6 +3,7 @@ extern crate yaml_rust; use crate::detections::configs; use crate::detections::print::AlertMessage; +use crate::filter::RuleExclude; use std::collections::HashMap; use std::ffi::OsStr; use std::fs; @@ -42,13 +43,18 @@ impl ParseYaml { Ok(file_content) } - pub fn read_dir>(&mut self, path: P, level: &str) -> io::Result { + pub fn read_dir>( + &mut self, + path: P, + level: &str, + exclude_ids: &RuleExclude, + ) -> io::Result { let mut entries = fs::read_dir(path)?; let yaml_docs = entries.try_fold(vec![], |mut ret, entry| { let entry = entry?; // フォルダは再帰的に呼び出す。 if entry.file_type()?.is_dir() { - self.read_dir(entry.path(), level)?; + self.read_dir(entry.path(), level, exclude_ids)?; return io::Result::Ok(ret); } // ファイル以外は無視 @@ -115,6 +121,7 @@ impl ParseYaml { .unwrap_or(&0) + 1, ); + if configs::CONFIG.read().unwrap().args.is_present("verbose") { println!("Loaded yml file path: {}", filepath); } @@ -130,6 +137,21 @@ impl ParseYaml { return Option::None; } + //除外されたルールは無視する + let rule_id = &yaml_doc["id"].as_str(); + if rule_id.is_some() { + match exclude_ids + .no_use_rule + .get(&rule_id.unwrap_or("").to_string()) + { + None => (), + Some(_) => { + self.ignorerule_count += 1; + return Option::None; + } + } + } + return Option::Some((filepath, yaml_doc)); }) .collect(); @@ -141,14 +163,24 @@ impl ParseYaml { #[cfg(test)] mod tests { + use crate::filter; use crate::yaml; + use crate::yaml::RuleExclude; + use std::collections::HashSet; use std::path::Path; use yaml_rust::YamlLoader; #[test] fn test_read_dir_yaml() { let mut yaml = yaml::ParseYaml::new(); - &yaml.read_dir("test_files/rules/yaml/".to_string(), &"".to_owned()); + let exclude_ids = RuleExclude { + no_use_rule: HashSet::new(), + }; + let _ = &yaml.read_dir( + "test_files/rules/yaml/".to_string(), + &"".to_owned(), + &exclude_ids, + ); assert_ne!(yaml.files.len(), 0); } @@ -183,7 +215,8 @@ mod tests { fn test_default_level_read_yaml() { let mut yaml = yaml::ParseYaml::new(); let path = Path::new("test_files/rules/level_yaml"); - yaml.read_dir(path.to_path_buf(), &"").unwrap(); + yaml.read_dir(path.to_path_buf(), &"", &filter::exclude_ids()) + .unwrap(); assert_eq!(yaml.files.len(), 5); } @@ -191,36 +224,59 @@ mod tests { fn test_info_level_read_yaml() { let mut yaml = yaml::ParseYaml::new(); let path = Path::new("test_files/rules/level_yaml"); - yaml.read_dir(path.to_path_buf(), &"informational").unwrap(); + yaml.read_dir(path.to_path_buf(), &"informational", &filter::exclude_ids()) + .unwrap(); assert_eq!(yaml.files.len(), 5); } #[test] fn test_low_level_read_yaml() { let mut yaml = yaml::ParseYaml::new(); let path = Path::new("test_files/rules/level_yaml"); - - yaml.read_dir(path.to_path_buf(), &"LOW").unwrap(); + yaml.read_dir(path.to_path_buf(), &"LOW", &filter::exclude_ids()) + .unwrap(); assert_eq!(yaml.files.len(), 4); } #[test] fn test_medium_level_read_yaml() { let mut yaml = yaml::ParseYaml::new(); let path = Path::new("test_files/rules/level_yaml"); - yaml.read_dir(path.to_path_buf(), &"MEDIUM").unwrap(); + yaml.read_dir(path.to_path_buf(), &"MEDIUM", &filter::exclude_ids()) + .unwrap(); assert_eq!(yaml.files.len(), 3); } #[test] fn test_high_level_read_yaml() { let mut yaml = yaml::ParseYaml::new(); let path = Path::new("test_files/rules/level_yaml"); - yaml.read_dir(path.to_path_buf(), &"HIGH").unwrap(); + yaml.read_dir(path.to_path_buf(), &"HIGH", &filter::exclude_ids()) + .unwrap(); assert_eq!(yaml.files.len(), 2); } #[test] fn test_critical_level_read_yaml() { let mut yaml = yaml::ParseYaml::new(); let path = Path::new("test_files/rules/level_yaml"); - yaml.read_dir(path.to_path_buf(), &"CRITICAL").unwrap(); + yaml.read_dir(path.to_path_buf(), &"CRITICAL", &filter::exclude_ids()) + .unwrap(); assert_eq!(yaml.files.len(), 1); } + #[test] + fn test_all_exclude_rules_file() { + let mut yaml = yaml::ParseYaml::new(); + let path = Path::new("test_files/rules/yaml"); + yaml.read_dir(path.to_path_buf(), &"", &filter::exclude_ids()) + .unwrap(); + assert_eq!(yaml.ignorerule_count, 10); + } + #[test] + fn test_none_exclude_rules_file() { + let mut yaml = yaml::ParseYaml::new(); + let path = Path::new("test_files/rules/yaml"); + let exclude_ids = RuleExclude { + no_use_rule: HashSet::new(), + }; + yaml.read_dir(path.to_path_buf(), &"", &exclude_ids) + .unwrap(); + assert_eq!(yaml.ignorerule_count, 0); + } } diff --git a/test_files/rules/yaml/1.yml b/test_files/rules/yaml/1.yml index 5f844d26..23a32d6a 100644 --- a/test_files/rules/yaml/1.yml +++ b/test_files/rules/yaml/1.yml @@ -15,5 +15,4 @@ falsepositives: level: medium output: 'CommandLine=%CommandLine%¥nParentImage=%ParentImage%' creation_date: 2020/11/8 -updated_date: 2020/11/8 - +updated_date: 2020/11/8 \ No newline at end of file diff --git a/test_files/rules/yaml/exclude1.yml b/test_files/rules/yaml/exclude1.yml new file mode 100644 index 00000000..76e3e73d --- /dev/null +++ b/test_files/rules/yaml/exclude1.yml @@ -0,0 +1,19 @@ +title: Sysmon Check command lines +id : 4fe151c2-ecf9-4fae-95ae-b88ec9c2fca6 +description: hogehoge +enabled: true +author: Yea +logsource: + product: windows +detection: + selection: + EventLog: Sysmon + EventID: 1 + CommandLine: '*' + condition: selection +falsepositives: + - unknown +level: medium +output: 'CommandLine=%CommandLine%¥nParentImage=%ParentImage%' +creation_date: 2020/11/8 +updated_date: 2020/11/8 \ No newline at end of file diff --git a/test_files/rules/yaml/exclude2.yml b/test_files/rules/yaml/exclude2.yml new file mode 100644 index 00000000..e17e37cf --- /dev/null +++ b/test_files/rules/yaml/exclude2.yml @@ -0,0 +1,21 @@ +title: Possible Exploitation of Exchange RCE CVE-2021-42321 +author: Florian Roth, @testanull +date: 2021/11/18 +description: Detects log entries that appear in exploitation attempts against MS Exchange + RCE CVE-2021-42321 +detection: + condition: 'Cmdlet failed. Cmdlet Get-App, ' +falsepositives: +- Unknown, please report false positives via https://github.com/SigmaHQ/sigma/issues +id: c92f1896-d1d2-43c3-92d5-7a5b35c217bb +level: critical +logsource: + product: windows + service: msexchange-management +references: +- https://msrc.microsoft.com/update-guide/vulnerability/CVE-2021-42321 +status: experimental +tags: +- attack.lateral_movement +- attack.t1210 +ruletype: SIGMA diff --git a/test_files/rules/yaml/exclude3.yml b/test_files/rules/yaml/exclude3.yml new file mode 100644 index 00000000..45f43c4a --- /dev/null +++ b/test_files/rules/yaml/exclude3.yml @@ -0,0 +1,28 @@ +title: Hidden Local User Creation +author: Christian Burkard +date: 2021/05/03 +description: Detects the creation of a local hidden user account which should not + happen for event ID 4720. +detection: + SELECTION_1: + EventID: 4720 + SELECTION_2: + TargetUserName: '*$' + condition: (SELECTION_1 and SELECTION_2) +falsepositives: +- unknown +fields: +- EventCode +- AccountName +id: 7b449a5e-1db5-4dd0-a2dc-4e3a67282538 +level: high +logsource: + product: windows + service: security +references: +- https://twitter.com/SBousseaden/status/1387743867663958021 +status: experimental +tags: +- attack.persistence +- attack.t1136.001 +ruletype: SIGMA diff --git a/test_files/rules/yaml/exclude4.yml b/test_files/rules/yaml/exclude4.yml new file mode 100644 index 00000000..06b76c48 --- /dev/null +++ b/test_files/rules/yaml/exclude4.yml @@ -0,0 +1,30 @@ +title: User Added to Local Administrators +author: Florian Roth +date: 2017/03/14 +description: This rule triggers on user accounts that are added to the local Administrators + group, which could be legitimate activity or a sign of privilege escalation activity +detection: + SELECTION_1: + EventID: 4732 + SELECTION_2: + TargetUserName: Administr* + SELECTION_3: + TargetSid: S-1-5-32-544 + SELECTION_4: + SubjectUserName: '*$' + condition: ((SELECTION_1 and (SELECTION_2 or SELECTION_3)) and not (SELECTION_4)) +falsepositives: +- Legitimate administrative activity +id: c265cf08-3f99-46c1-8d59-328247057d57 +level: medium +logsource: + product: windows + service: security +modified: 2021/07/07 +status: stable +tags: +- attack.privilege_escalation +- attack.t1078 +- attack.persistence +- attack.t1098 +ruletype: SIGMA diff --git a/test_files/rules/yaml/exclude5.yml b/test_files/rules/yaml/exclude5.yml new file mode 100644 index 00000000..27ec53cc --- /dev/null +++ b/test_files/rules/yaml/exclude5.yml @@ -0,0 +1,31 @@ +title: Local User Creation +author: Patrick Bareiss +date: 2019/04/18 +description: Detects local user creation on windows servers, which shouldn't happen + in an Active Directory environment. Apply this Sigma Use Case on your windows server + logs and not on your DC logs. +detection: + SELECTION_1: + EventID: 4720 + condition: SELECTION_1 +falsepositives: +- Domain Controller Logs +- Local accounts managed by privileged account management tools +fields: +- EventCode +- AccountName +- AccountDomain +id: 66b6be3d-55d0-4f47-9855-d69df21740ea +level: low +logsource: + product: windows + service: security +modified: 2020/08/23 +references: +- https://patrick-bareiss.com/detecting-local-user-creation-in-ad-with-sigma/ +status: experimental +tags: +- attack.persistence +- attack.t1136 +- attack.t1136.001 +ruletype: SIGMA diff --git a/test_files/rules/yaml/noisy1.yml b/test_files/rules/yaml/noisy1.yml new file mode 100644 index 00000000..6ea217b6 --- /dev/null +++ b/test_files/rules/yaml/noisy1.yml @@ -0,0 +1,25 @@ +title: WMI Event Subscription +author: Tom Ueltschi (@c_APT_ure) +date: 2019/01/12 +description: Detects creation of WMI event subscription persistence method +detection: + SELECTION_1: + EventID: 19 + SELECTION_2: + EventID: 20 + SELECTION_3: + EventID: 21 + condition: (SELECTION_1 or SELECTION_2 or SELECTION_3) +falsepositives: +- exclude legitimate (vetted) use of WMI event subscription in your network +id: 0f06a3a5-6a09-413f-8743-e6cf35561297 +level: high +logsource: + category: wmi_event + product: windows +status: experimental +tags: +- attack.t1084 +- attack.persistence +- attack.t1546.003 +ruletype: SIGMA \ No newline at end of file diff --git a/test_files/rules/yaml/noisy2.yml b/test_files/rules/yaml/noisy2.yml new file mode 100644 index 00000000..2296fba4 --- /dev/null +++ b/test_files/rules/yaml/noisy2.yml @@ -0,0 +1,31 @@ +title: Rare Schtasks Creations +author: Florian Roth +date: 2017/03/23 +description: Detects rare scheduled tasks creations that only appear a few times per + time frame and could reveal password dumpers, backdoor installs or other types of + malicious code +detection: + SELECTION_1: + EventID: 4698 + condition: SELECTION_1 | count() by TaskName < 5 +falsepositives: +- Software installation +- Software updates +id: b0d77106-7bb0-41fe-bd94-d1752164d066 +level: low +logsource: + definition: The Advanced Audit Policy setting Object Access > Audit Other Object + Access Events has to be configured to allow this detection (not in the baseline + recommendations by Microsoft). We also recommend extracting the Command field + from the embedded XML in the event data. + product: windows + service: security +status: experimental +tags: +- attack.execution +- attack.privilege_escalation +- attack.persistence +- attack.t1053 +- car.2013-08-001 +- attack.t1053.005 +ruletype: SIGMA diff --git a/test_files/rules/yaml/noisy3.yml b/test_files/rules/yaml/noisy3.yml new file mode 100644 index 00000000..7e2071a0 --- /dev/null +++ b/test_files/rules/yaml/noisy3.yml @@ -0,0 +1,26 @@ +title: Rare Service Installs +author: Florian Roth +date: 2017/03/08 +description: Detects rare service installs that only appear a few times per time frame + and could reveal password dumpers, backdoor installs or other types of malicious + services +detection: + SELECTION_1: + EventID: 7045 + condition: SELECTION_1 | count() by ServiceFileName < 5 +falsepositives: +- Software installation +- Software updates +id: 66bfef30-22a5-4fcd-ad44-8d81e60922ae +level: low +logsource: + product: windows + service: system +status: experimental +tags: +- attack.persistence +- attack.privilege_escalation +- attack.t1050 +- car.2013-09-005 +- attack.t1543.003 +ruletype: SIGMA diff --git a/test_files/rules/yaml/noisy4.yml b/test_files/rules/yaml/noisy4.yml new file mode 100644 index 00000000..39bbd1a3 --- /dev/null +++ b/test_files/rules/yaml/noisy4.yml @@ -0,0 +1,33 @@ +title: Failed Logins with Different Accounts from Single Source System +author: Florian Roth +date: 2017/01/10 +description: Detects suspicious failed logins with different user accounts from a + single source system +detection: + SELECTION_1: + EventID: 529 + SELECTION_2: + EventID: 4625 + SELECTION_3: + TargetUserName: '*' + SELECTION_4: + WorkstationName: '*' + condition: ((SELECTION_1 or SELECTION_2) and SELECTION_3 and SELECTION_4) | count(TargetUserName) + by WorkstationName > 3 +falsepositives: +- Terminal servers +- Jump servers +- Other multiuser systems like Citrix server farms +- Workstations with frequently changing users +id: e98374a6-e2d9-4076-9b5c-11bdb2569995 +level: medium +logsource: + product: windows + service: security +modified: 2021/09/21 +status: experimental +tags: +- attack.persistence +- attack.privilege_escalation +- attack.t1078 +ruletype: SIGMA diff --git a/test_files/rules/yaml/noisy5.yml b/test_files/rules/yaml/noisy5.yml new file mode 100644 index 00000000..ddfc134a --- /dev/null +++ b/test_files/rules/yaml/noisy5.yml @@ -0,0 +1,34 @@ +title: Failed Logins with Different Accounts from Single Source System +author: Florian Roth +date: 2017/01/10 +description: Detects suspicious failed logins with different user accounts from a + single source system +detection: + SELECTION_1: + EventID: 4776 + SELECTION_2: + TargetUserName: '*' + SELECTION_3: + Workstation: '*' + condition: (SELECTION_1 and SELECTION_2 and SELECTION_3) | count(TargetUserName) + by Workstation > 3 +falsepositives: +- Terminal servers +- Jump servers +- Other multiuser systems like Citrix server farms +- Workstations with frequently changing users +id: 6309ffc4-8fa2-47cf-96b8-a2f72e58e538 +level: medium +logsource: + product: windows + service: security +modified: 2021/09/21 +related: +- id: e98374a6-e2d9-4076-9b5c-11bdb2569995 + type: derived +status: experimental +tags: +- attack.persistence +- attack.privilege_escalation +- attack.t1078 +ruletype: SIGMA diff --git a/tools/sigmac/README-English.md b/tools/sigmac/README-English.md index 458dc898..869601bd 100644 --- a/tools/sigmac/README-English.md +++ b/tools/sigmac/README-English.md @@ -47,8 +47,7 @@ cp splitter.py $sigma_path ### Convert Rule -Conversion rules can be created by executing `convert.sh`. -The rules will be created to hayabusa_rules folder. +`convert.sh` will convert sigma rules to hayabusa rules and save them in a new `hayabusa_rules` folder. ```sh export sigma_path=/path/to/sigma_repository @@ -71,4 +70,4 @@ sigma/rules/windows/process_creation/process_creation_apt_turla_commands_medium. ## Sigma rule parsing errors -Some rules will have been able to be converted but will cause parsing errors. We will continue to fix these bugs but for the meantime the majority of Sigma rules do work so please ignore the errors for now. +Some rules will have been able to be converted but will cause parsing errors or will not be usable due to various bugs. We will continue to fix these bugs but for the meantime the majority of Sigma rules do work so please ignore the errors for now. diff --git a/tools/sigmac/README-Japanese.md b/tools/sigmac/README-Japanese.md index 8baca41b..bfb89ee9 100644 --- a/tools/sigmac/README-Japanese.md +++ b/tools/sigmac/README-Japanese.md @@ -48,7 +48,7 @@ cp splitter.py $sigma_path * 注意：`/path/to/sigma_repository`そのままではなくて、自分のSigmaレポジトリのパスを指定してください。 ### ルールの変換 -convert.shを実行することでルールの変換が実行されます。変換されたルールはhayabusa_rulesフォルダに作成されます。 +`convert.sh`を実行することでルールの変換が実行されます。変換されたルールは`hayabusa_rules`フォルダに作成されます。 ```sh export sigma_path=/path/to/sigma_repository diff --git a/tools/sigmac/hayabusa.py b/tools/sigmac/hayabusa.py index ae53c407..3d9684bf 100644 --- a/tools/sigmac/hayabusa.py +++ b/tools/sigmac/hayabusa.py @@ -268,18 +268,18 @@ class HayabusaBackend(SingleTextQueryBackend): # parsed.sigmaParser.parsedyamlがOrderedDictならこんなことしなくていい、後で別のやり方があるか調べる # 順番固定してもいいかも bs.write("title: " + parsed_yaml["title"]+"\n") - bs.write("ruletype: SIGMA\n") + bs.write("ruletype: Sigma\n") del parsed_yaml["title"] - # detectionの部分をクリアする前にtimeflameだけ確保しておく。 - timeflame = None - if "timeflame" in parsed_yaml["detection"]: - timeflame = parsed_yaml["detection"]["timeflame"] + # detectionの部分をクリアする前にtimeframeだけ確保しておく。 + timeframe = None + if "timeframe" in parsed_yaml["detection"]: + timeframe = parsed_yaml["detection"]["timeframe"] # detectionの部分だけ変更して出力する。 parsed_yaml["detection"] = {} - if timeflame is not None and len(timeflame) != 0: - parsed_yaml["detection"]["timeflame"] = timeflame + if timeframe is not None and len(timeframe) != 0: + parsed_yaml["detection"]["timeframe"] = timeframe parsed_yaml["detection"]["condition"] = result for key, values in self.name_2_selection.items(): # fieldnameの有無を確認している