diff --git a/README-Japanese.md b/README-Japanese.md index c3197250..135e367c 100644 --- a/README-Japanese.md +++ b/README-Japanese.md @@ -27,7 +27,6 @@ Hayabusaは、日本の[Yamato Security](https://yamatosecurity.connpass.com/) - [主な目的](#主な目的) - [スレット(脅威)ハンティング](#スレット脅威ハンティング) - [フォレンジックタイムラインの高速生成](#フォレンジックタイムラインの高速生成) -- [開発について](#開発について) - [スクリーンショット](#スクリーンショット) - [起動画面:](#起動画面) - [ターミナル出力画面:](#ターミナル出力画面) @@ -45,7 +44,7 @@ Hayabusaは、日本の[Yamato Security](https://yamatosecurity.connpass.com/) - [Linuxでのコンパイルの注意点](#linuxでのコンパイルの注意点) - [アドバンス: Rustパッケージの更新](#アドバンス-rustパッケージの更新) - [Hayabusaの実行](#hayabusaの実行) - - [注意: Anti-Virus/EDR Warnings](#注意-anti-virusedr-warnings) + - [注意: アンチウィルス/EDRの誤検知](#注意-アンチウィルスedrの誤検知) - [Windows](#windows) - [Windows Terminalで利用する際の注意事項](#windows-terminalで利用する際の注意事項) - [Linux](#linux) @@ -57,7 +56,7 @@ Hayabusaは、日本の[Yamato Security](https://yamatosecurity.connpass.com/) - [Hayabusaの出力](#hayabusaの出力) - [プログレスバー](#プログレスバー) - [標準出力へのカラー設定](#標準出力へのカラー設定) -- [Hayabusa ルール](#hayabusa-ルール) +- [Hayabusaルール](#hayabusaルール) - [Hayabusa v.s. 変換されたSigmaルール](#hayabusa-vs-変換されたsigmaルール) - [検知ルールのチューニング](#検知ルールのチューニング) - [イベントIDフィルタリング](#イベントidフィルタリング) @@ -84,10 +83,6 @@ Windowsのイベントログは、 から、従来は非常に長い時間と手間がかかる解析作業となっていました。 Hayabusa は、有用なデータのみを抽出し、専門的なトレーニングを受けた分析者だけでなく、Windowsのシステム管理者であれば誰でも利用できる読みやすい形式で提示することを主な目的としています。 [Evtx Explorer](https://ericzimmerman.github.io/#!index.md)や[Event Log Explorer](https://eventlogxp.com/)のような深掘り分析を行うツールの代替ではなく、分析者が20%の時間で80%の作業を行えるようにすることを目的としています。 -# 開発について - -[DeepBlueCLI](https://github.com/sans-blue-team/DeepBlueCLI)というWindowsイベントログ解析ツールに触発されて、2020年に[RustyBlue](https://github.com/Yamato-Security/RustyBlue)プロジェクト用にRustに移植することから始めました。その後、YMLで書かれたSigmaのような柔軟な検知シグネチャを作り、SigmaルールをHayabusaルール形式へ変換するツールも作成しました。 - # スクリーンショット ## 起動画面: @@ -225,9 +220,11 @@ cargo update # Hayabusaの実行 -## 注意: Anti-Virus/EDR Warnings +## 注意: アンチウィルス/EDRの誤検知 -You may receive warning from anti-virus or EDR when trying to run Hayabusa. These are false positives so you may need to configure your security products to allow running Hayabusa. If you are worried about malware, please check the Hayabusa source code and compile the binaries yourself. +Hayabusaを実行する際にアンチウィルスやEDRにブロックされる可能性があります。 +誤検知のため、セキュリティ対策の製品がHayabusaを許可するように設定する必要があります。 +マルウェア感染が心配のであれば、ソースコードを確認した上で、自分でバイナリをコンパイルして下さい。 ## Windows @@ -330,7 +327,7 @@ hayabusa.exe -f eventlog.evtx hayabusa.exe -d .\hayabusa-sample-evtx ``` -* １つのCSVファイルにエクスポートして、EXCELやTimeline Explorerでさらに分析することができます: +* １つのCSVファイルにエクスポートして、ExcelやTimeline Explorerでさらに分析することができます: ```bash hayabusa.exe -d .\hayabusa-sample-evtx -o results.csv @@ -369,7 +366,7 @@ hayabusa.exe -d .\hayabusa-sample-evtx --enable-deprecated-rules --enable-noisy- * ログオン情報を分析するルールのみを実行し、UTCタイムゾーンで出力します: ```bash -hayabusa.exe -d .\hayabusa-sample-evtx -r ./rules/Hayabusa/default/events/Security/Logons -U -o results.csv +hayabusa.exe -d .\hayabusa-sample-evtx -r .\rules\hayabusa\default\events\Security\Logons -U -o results.csv ``` * 起動中のWindows端末上で実行し（Administrator権限が必要）、アラート（悪意のある可能性のある動作）のみを検知します: @@ -450,7 +447,7 @@ CSVファイルとして保存する場合、以下の2つのフィールドが 注意: True Colorに対応しているターミナルが必要です。 例: [Windows Terminal](https://docs.microsoft.com/en-us/windows/terminal/install) またはmacOSの[iTerm2](https://iterm2.com/)。 -# Hayabusa ルール +# Hayabusaルール Hayabusa検知ルールはSigmaのようなYML形式で記述されています。`rules`ディレクトリに入っていますが、将来的には[https://github.com/Yamato-Security/hayabusa-rules](https://github.com/Yamato-Security/hayabusa-rules)のレポジトリで管理する予定なので、ルールのissueとpull requestはhayabusaのレポジトリではなく、ルールレポジトリへお願いします。 diff --git a/README.md b/README.md index 09d17372..efa980eb 100644 --- a/README.md +++ b/README.md @@ -24,45 +24,44 @@ Hayabusa is a **Windows event log fast forensics timeline generator** and **thre - [About Hayabusa](#about-hayabusa) - [Table of Contents](#table-of-contents) - - [Main goals](#main-goals) - - [Threat hunting](#threat-hunting) - - [Fast forensics timeline generation](#fast-forensics-timeline-generation) -- [About the development](#about-the-development) + - [Main Goals](#main-goals) + - [Threat Hunting](#threat-hunting) + - [Fast Forensics Timeline Generation](#fast-forensics-timeline-generation) - [Screenshots](#screenshots) - [Startup](#startup) - - [Terminal output](#terminal-output) - - [Results summary](#results-summary) + - [Terminal Output](#terminal-output) + - [Results Summary](#results-summary) - [Analysis in Excel](#analysis-in-excel) - [Analysis in Timeline Explorer](#analysis-in-timeline-explorer) - - [Critical alert filtering and computer grouping in Timeline Explorer](#critical-alert-filtering-and-computer-grouping-in-timeline-explorer) -- [Sample timeline results](#sample-timeline-results) + - [Critical Alert Filtering and Computer Grouping in Timeline Explorer](#critical-alert-filtering-and-computer-grouping-in-timeline-explorer) +- [Sample Timeline Results](#sample-timeline-results) - [Features](#features) - [Planned Features](#planned-features) - [Downloads](#downloads) -- [Compiling from source (Optional)](#compiling-from-source-optional) - - [Cross-compiling 32-bit Windows binaries](#cross-compiling-32-bit-windows-binaries) - - [Notes on compiling on macOS](#notes-on-compiling-on-macos) - - [Notes on compiling on Linux](#notes-on-compiling-on-linux) - - [Advanced: Updating Rust packages](#advanced-updating-rust-packages) +- [Compiling From Source (Optional)](#compiling-from-source-optional) + - [Cross-compiling 32-bit Windows Binaries](#cross-compiling-32-bit-windows-binaries) + - [macOS Compiling Notes](#macos-compiling-notes) + - [Linux Compiling Notes](#linux-compiling-notes) + - [Advanced: Updating Rust Packages](#advanced-updating-rust-packages) - [Running Hayabusa](#running-hayabusa) - [Caution: Anti-Virus/EDR Warnings](#caution-anti-virusedr-warnings) - [Windows](#windows) - - [Caution: Output printed to screen may stop in Windows Terminal](#caution-output-printed-to-screen-may-stop-in-windows-terminal) + - [Caution: Windows Terminal Bug](#caution-windows-terminal-bug) - [Linux](#linux) - [macOS](#macos) - [Usage](#usage) - - [Command line options](#command-line-options) - - [Usage examples](#usage-examples) -- [Testing hayabusa out on sample evtx files](#testing-hayabusa-out-on-sample-evtx-files) -- [Hayabusa output](#hayabusa-output) - - [Progress bar](#progress-bar) + - [Command Line Options](#command-line-options) + - [Usage Examples](#usage-examples) +- [Testing Hayabusa on Sample Evtx Files](#testing-hayabusa-on-sample-evtx-files) +- [Hayabusa Output](#hayabusa-output) + - [Progress Bar](#progress-bar) - [Color Output](#color-output) -- [Hayabusa rules](#hayabusa-rules) - - [Hayabusa v.s. converted Sigma rules](#hayabusa-vs-converted-sigma-rules) - - [Detection rule tuning](#detection-rule-tuning) - - [Event ID filtering](#event-id-filtering) -- [Other Windows event log analyzers and related projects](#other-windows-event-log-analyzers-and-related-projects) - - [Comparison to other similar tools that support sigma](#comparison-to-other-similar-tools-that-support-sigma) +- [Hayabusa Rules](#hayabusa-rules) + - [Hayabusa v.s. Converted Sigma Rules](#hayabusa-vs-converted-sigma-rules) + - [Detection Rule Tuning](#detection-rule-tuning) + - [Event ID Filtering](#event-id-filtering) +- [Other Windows Event Log Analyzers and Related Projects](#other-windows-event-log-analyzers-and-related-projects) + - [Comparison To Other Similar Tools](#comparison-to-other-similar-tools) - [Community Documentation](#community-documentation) - [English](#english) - [Japanese](#japanese) @@ -70,33 +69,29 @@ Hayabusa is a **Windows event log fast forensics timeline generator** and **thre - [Bug Submission](#bug-submission) - [License](#license) -## Main goals +## Main Goals -### Threat hunting +### Threat Hunting Hayabusa currently has over 1300 sigma rules and around 70 hayabusa rules with more rules being added regularly. The ultimate goal is to be able to push out hayabusa agents to all Windows endpoints after an incident or for periodic threat hunting and have them alert back to a central server. -### Fast forensics timeline generation +### Fast Forensics Timeline Generation Windows event log analysis has traditionally been a very long and tedious process because Windows event logs are 1) in a data format that is hard to analyze and 2) the majority of data is noise and not useful for investigations. Hayabusa's main goal is to extract out only useful data and present it in an easy-to-read format that is usable not only by professionally trained analysts but any Windows system administrator. Hayabusa is not intended to be a replacement for tools like [Evtx Explorer](https://ericzimmerman.github.io/#!index.md) or [Event Log Explorer](https://eventlogxp.com/) for more deep-dive analysis but is intended for letting analysts get 80% of their work done in 20% of the time. -# About the development - -First inspired by the [DeepBlueCLI](https://github.com/sans-blue-team/DeepBlueCLI) Windows event log analyzer, we started in 2020 porting it over to Rust for the [RustyBlue](https://github.com/Yamato-Security/RustyBlue) project, then created sigma-like flexible detection signatures written in YML, and then added a backend to sigma to support converting sigma rules into our hayabusa rule format. - # Screenshots ## Startup ![Hayabusa Startup](/screenshots/Hayabusa-Startup.png) -## Terminal output +## Terminal Output ![Hayabusa terminal output](/screenshots/Hayabusa-Results.png) -## Results summary +## Results Summary ![Hayabusa results summary](/screenshots/HayabusaResultsSummary.png) @@ -108,11 +103,11 @@ First inspired by the [DeepBlueCLI](https://github.com/sans-blue-team/DeepBlueCL ![Hayabusa analysis in Timeline Explorer](screenshots/TimelineExplorer-ColoredTimeline.png) -## Critical alert filtering and computer grouping in Timeline Explorer +## Critical Alert Filtering and Computer Grouping in Timeline Explorer ![Critical alert filtering and computer grouping in Timeline Explorer](screenshots/TimelineExplorer-CriticalAlerts-ComputerGrouping.png) -# Sample timeline results +# Sample Timeline Results You can check out sample CSV timelines [here](https://github.com/Yamato-Security/hayabusa/tree/main/sample-results). @@ -160,7 +155,7 @@ hayabusa.exe -u If the update fails, you may need to rename the `rules` folder and try again. -# Compiling from source (Optional) +# Compiling From Source (Optional) If you have Rust installed, you can compile from source with the following command: @@ -177,7 +172,7 @@ rustup update stable The compiled binary will be outputted in the `target/release` folder. -## Cross-compiling 32-bit Windows binaries +## Cross-compiling 32-bit Windows Binaries You can create 32-bit binaries on 64-bit Windows systems with the following: ```bash @@ -186,7 +181,7 @@ rustup target add i686-pc-windows-msvc rustup run stable-i686-pc-windows-msvc cargo build --release ``` -## Notes on compiling on macOS +## macOS Compiling Notes If you receive compile errors about openssl, you will need to install [Homebrew](https://brew.sh/) and then install the following packages: ```bash @@ -194,7 +189,7 @@ brew install pkg-config brew install openssl ``` -## Notes on compiling on Linux +## Linux Compiling Notes If you receive compile errors about openssl, you will need to install the following package. @@ -208,7 +203,7 @@ Fedora-based distros: sudo yum install openssl-devel ``` -## Advanced: Updating Rust packages +## Advanced: Updating Rust Packages You can update to the latest Rust crates before compiling to get the latest libraries: @@ -229,7 +224,7 @@ You may receive warning from anti-virus or EDR when trying to run hayabusa. Thes In Command Prompt or Windows Terminal, just run 32-bit or 64-bit Windoows binary from the hayabusa root directory. Example: `hayabusa-1.2.0-windows-x64.exe` -### Caution: Output printed to screen may stop in Windows Terminal +### Caution: Windows Terminal Bug As of Feb 1, 2022, Windows Terminal will freeze midway when displaying results to the screen when run against the sample evtx files. This is because there is a control code (0x9D) in the output. @@ -285,7 +280,7 @@ You should now be able to run hayabusa. # Usage -## Command line options +## Command Line Options ```bash USAGE: @@ -312,7 +307,7 @@ USAGE: --contributors 'Prints the list of contributors.' ``` -## Usage examples +## Usage Examples * Run hayabusa against one Windows event log file: @@ -405,7 +400,7 @@ Checking target evtx FilePath: "./hayabusa-sample-evtx/YamatoSecurity/T1218.004_ By default, hayabusa will save error messages to error log files. If you do not want to save error messages, please add `-Q`. -# Testing hayabusa out on sample evtx files +# Testing Hayabusa on Sample Evtx Files We have provided some sample evtx files for you to test hayabusa and/or create new rules at [https://github.com/Yamato-Security/hayabusa-sample-evtx](https://github.com/Yamato-Security/hayabusa-sample-evtx) @@ -417,7 +412,7 @@ git clone https://github.com/Yamato-Security/hayabusa-sample-evtx.git > Note: You need to run the binary from the Hayabusa root directory. -# Hayabusa output +# Hayabusa Output When hayabusa output is being displayed to the screen (the default), it will display the following information: @@ -433,7 +428,7 @@ When saving to a CSV file an additional two fields will be added: * `Rule Path`: The path to the detection rule that generated the alert or event. * `File Path`: The path to the evtx file that caused the alert or event. -## Progress bar +## Progress Bar The progress bar will only work with multiple evtx files. It will display in real time the number and percent of evtx files that it has finished analyzing. @@ -445,7 +440,7 @@ You can change the default colors in the config file at `./config/level_color.tx Note: Color can only be displayed in terminals that support [True Color](https://en.wikipedia.org/wiki/Color_depth#True_color_(24-bit)). Example: [Windows Terminal](https://docs.microsoft.com/en-us/windows/terminal/install) or [iTerm2](https://iterm2.com/) for macOS. -# Hayabusa rules +# Hayabusa Rules Hayabusa detection rules are written in a sigma-like YML format and are located in the `rules` folder. In the future, we plan to host the rules at [https://github.com/Yamato-Security/hayabusa-rules](https://github.com/Yamato-Security/hayabusa-rules) so please send any issues and pull requests for rules there instead of the main hayabusa repository. @@ -470,7 +465,7 @@ Rules are further seperated into directories by log type (Example: Security, Sys Please check out the current rules to use as a template in creating new ones or for checking the detection logic. -## Hayabusa v.s. converted Sigma rules +## Hayabusa v.s. Converted Sigma Rules Sigma rules need to first be converted to hayabusa rule format explained [here](https://github.com/Yamato-Security/hayabusa-rules/blob/main/tools/sigmac/README.md). Hayabusa rules are designed solely for Windows event log analysis and have the following benefits: @@ -485,7 +480,7 @@ Sigma rules need to first be converted to hayabusa rule format explained [here]( > Note: the limitation is in the sigma rule converter and not in hayabusa itself. -## Detection rule tuning +## Detection Rule Tuning Like firewalls and IDSes, any signature-based tool will require some tuning to fit your environment so you may need to permanently or temporarily exclude certain rules. @@ -493,7 +488,7 @@ You can add a rule ID (Example: `4fe151c2-ecf9-4fae-95ae-b88ec9c2fca6`) to `rule You can also add a rule ID to `rules/config/noisy_rules.txt` in order to ignore the rule by default but still be able to use the rule with the `-n` or `--enable-noisy-rules` option. -## Event ID filtering +## Event ID Filtering You can filter on event IDs by placing event ID numbers in `config/target_eventids.txt`. This will increase performance so it is recommended if you only need to search for certain IDs. @@ -502,7 +497,7 @@ We have provided a sample ID filter list at [`config/target_eventids_sample.txt` Please use this list if you want the best performance but be aware that there is a slight possibility for missing events (false negatives). -# Other Windows event log analyzers and related projects +# Other Windows Event Log Analyzers and Related Projects There is no "one tool to rule them all" and we have found that each has its own merits so we recommend checking out these other great tools and projects and seeing which ones you like. @@ -528,17 +523,17 @@ There is no "one tool to rule them all" and we have found that each has its own * [WELA (Windows Event Log Analyzer)](https://github.com/Yamato-Security/WELA) - The swiff-army knife for Windows event logs by [Yamato Security](https://github.com/Yamato-Security/) * [Zircolite](https://github.com/wagga40/Zircolite) - Sigma-based attack detection tool written in Python. -## Comparison to other similar tools that support sigma +## Comparison To Other Similar Tools Please understand that it is not possible to do a perfect comparison as results will differ based on the target sample data, command-line options, rule tuning, etc... -In our tests, we have found hayabusa to support the largest number of sigma rules out of all the tools while still maintaining very fast speeds and does not require a great amount of memory. +In our tests, we have found hayabusa to support the largest number of sigma rules compared to other similar tools while still maintaining very fast speeds and does not require a great amount of memory. The following benchmarks were taken on a Lenovo P51 based on approximately 500 evtx files (130MB) from our [sample-evtx repository](https://github.com/Yamato-Security/hayabusa-sample-evtx) at 2021/12/23 with hayabusa version 1.0.0. | | Elapsed Time | Memory Usage | Unique Sigma Rules With Detections | | :-------: | :----------: | :----------------------------------------------------------: | :--------------------------------: | | Chainsaw | 7.5 seconds | 75 MB | 170 | -| Hayabusa | 7.8 seconds | 340 MB | 267 | +| Hayabusa | 7.8 seconds | 340 MB (memory usage depends on the amount of alerts) | 267 | | Zircolite | 34 seconds | 380 MB (normally requires 3 times the size of the log files) | 237 | * With hayabusa rules enabled, it will detect around 300 unique alerts and events.