removed tools/sigmac (#441)

* removed tools/sigmac

- moved tools/sigmac to hayabusa-rules repo

* fixed doc link tools/sigmac

* fixed submodule track

* fixed submodule track from latest to v1.1.0 tag

* fixed link
This commit is contained in:
DustInDark
2022-03-05 22:26:22 +09:00
committed by GitHub
parent db857f81af
commit b3cfedf4a5
56 changed files with 5 additions and 1549 deletions

View File

@@ -15,7 +15,7 @@
# Hayabusa について
Hayabusaは、日本の[Yamato Security](https://yamatosecurity.connpass.com/)グループによって作られた**Windowsイベントログのファストフォレンジックタイムライン生成**および**スレットハンティングツール**です。 Hayabusaは日本語で[「ハヤブサ」](https://en.wikipedia.org/wiki/Peregrine_falcon)を意味し、ハヤブサが世界で最も速く、狩猟(hunting)に優れ、とても訓練しやすい動物であることから選ばれました。[Rust](https://www.rust-lang.org/) で開発され、マルチスレッドに対応し、可能な限り高速に動作するよう配慮されています。[Sigma](https://github.com/SigmaHQ/Sigma)ルールをHayabusaルール形式に変換する[ツール](https://github.com/Yamato-Security/hayabusa/tree/main/tools/Sigmac)も提供しています。Hayabusaの検知ルールもSigmaと同様にYML形式であり、カスタマイズ性や拡張性に優れます。稼働中のシステムで実行してライブ調査することも、複数のシステムからログを収集してオフライン調査することも可能です。(※現時点では、リアルタイムアラートや定期的なスキャンには対応していません。) 出力は一つのCSVタイムラインにまとめられ、Excelや[Timeline Explorer](https://ericzimmerman.github.io/#!index.md)で簡単に分析できるようになります。
Hayabusaは、日本の[Yamato Security](https://yamatosecurity.connpass.com/)グループによって作られた**Windowsイベントログのファストフォレンジックタイムライン生成**および**スレットハンティングツール**です。 Hayabusaは日本語で[「ハヤブサ」](https://en.wikipedia.org/wiki/Peregrine_falcon)を意味し、ハヤブサが世界で最も速く、狩猟(hunting)に優れ、とても訓練しやすい動物であることから選ばれました。[Rust](https://www.rust-lang.org/) で開発され、マルチスレッドに対応し、可能な限り高速に動作するよう配慮されています。[Sigma](https://github.com/SigmaHQ/Sigma)ルールをHayabusaルール形式に変換する[ツール](https://github.com/Yamato-Security/hayabusa-rules/tree/main/tools/sigmac)も提供しています。Hayabusaの検知ルールもSigmaと同様にYML形式であり、カスタマイズ性や拡張性に優れます。稼働中のシステムで実行してライブ調査することも、複数のシステムからログを収集してオフライン調査することも可能です。(※現時点では、リアルタイムアラートや定期的なスキャンには対応していません。) 出力は一つのCSVタイムラインにまとめられ、Excelや[Timeline Explorer](https://ericzimmerman.github.io/#!index.md)で簡単に分析できるようになります。
## 目次
@@ -411,7 +411,7 @@ Hayabusaルールのディレクトリ構造は、3つのディレクトリに
## Hayabusa v.s. 変換されたSigmaルール
Sigmaルールは、最初にHayabusaルール形式に変換する必要があります。変換のやり方は[ここ](https://github.com/Yamato-Security/Hayabusa/blob/main/tools/Sigmac/README-Japanese.md)で説明されています。Hayabusaルールは、Windowsのイベントログ解析専用に設計されており、以下のような利点があります:
Sigmaルールは、最初にHayabusaルール形式に変換する必要があります。変換のやり方は[ここ](https://github.com/Yamato-Security/hayabusa-rules/tree/main/tools/sigmac/README-Japanese.md)で説明されています。Hayabusaルールは、Windowsのイベントログ解析専用に設計されており、以下のような利点があります:
1. ログの有用なフィールドのみから抽出された追加情報を表示するための `details`フィールドを追加しています。
2. Hayabusaルールはすべてサンプルログに対してテストされ、検知することが確認されています。

View File

@@ -15,7 +15,7 @@
# About Hayabusa
Hayabusa is a **Windows event log fast forensics timeline generator** and **threat hunting tool** created by the [Yamato Security](https://yamatosecurity.connpass.com/) group in Japan. Hayabusa means ["peregrine falcon"](https://en.wikipedia.org/wiki/Peregrine_falcon") in Japanese and was chosen as peregrine falcons are the fastest animal in the world, great at hunting and highly trainable. It is written in [Rust](https://www.rust-lang.org/) and supports multi-threading in order to be as fast as possible. We have provided a [tool](https://github.com/Yamato-Security/hayabusa/tree/main/tools/sigmac) to convert [sigma](https://github.com/SigmaHQ/sigma) rules into hayabusa rule format. The hayabusa detection rules, like sigma, are also written in YML in order to be as easily customizable and extensible as possible. It can be run either on running systems for live analysis or by gathering logs from multiple systems for offline analysis. (At the moment, it does not support real-time alerting or periodic scans.) The output will be consolidated into a single CSV timeline for easy analysis in Excel or [Timeline Explorer](https://ericzimmerman.github.io/#!index.md).
Hayabusa is a **Windows event log fast forensics timeline generator** and **threat hunting tool** created by the [Yamato Security](https://yamatosecurity.connpass.com/) group in Japan. Hayabusa means ["peregrine falcon"](https://en.wikipedia.org/wiki/Peregrine_falcon") in Japanese and was chosen as peregrine falcons are the fastest animal in the world, great at hunting and highly trainable. It is written in [Rust](https://www.rust-lang.org/) and supports multi-threading in order to be as fast as possible. We have provided a [tool](https://github.com/Yamato-Security/hayabusa-rules/tree/main/tools/sigmac) to convert [sigma](https://github.com/SigmaHQ/sigma) rules into hayabusa rule format. The hayabusa detection rules, like sigma, are also written in YML in order to be as easily customizable and extensible as possible. It can be run either on running systems for live analysis or by gathering logs from multiple systems for offline analysis. (At the moment, it does not support real-time alerting or periodic scans.) The output will be consolidated into a single CSV timeline for easy analysis in Excel or [Timeline Explorer](https://ericzimmerman.github.io/#!index.md).
## Table of Contents
@@ -407,7 +407,7 @@ Please check out the current rules to use as a template in creating new ones or
## Hayabusa v.s. converted Sigma rules
Sigma rules need to first be converted to hayabusa rule format explained [here](https://github.com/Yamato-Security/hayabusa/blob/main/tools/sigmac/README-English.md). Hayabusa rules are designed solely for Windows event log analysis and have the following benefits:
Sigma rules need to first be converted to hayabusa rule format explained [here](https://github.com/Yamato-Security/hayabusa-rules/blob/main/tools/sigmac/README.md). Hayabusa rules are designed solely for Windows event log analysis and have the following benefits:
1. An extra `details` field to display additional information taken from only the useful fields in the log.
2. They are all tested against sample logs and are known to work.

2
rules

Submodule rules updated: 6d9781e349...abab0c4d41

View File

@@ -1,73 +0,0 @@
# Automatic conversion of Sigma to Hayabusa rules
[![python](https://img.shields.io/badge/python-3.8-blue)](https://www.python.org/)
![version](https://img.shields.io/badge/Platform-Win-green)
![version](https://img.shields.io/badge/Platform-Lin-green)
![version](https://img.shields.io/badge/Platform-Mac-green)
You can use `hayabusa.py`, a `sigmac` backend, to automatically convert Sigma rules to Hayabusa rules.
## Pre-converted Sigma rules
Sigma rules have already been pre-converted to hayabusa format and placed in the `./rules/Sigma` directory.
Please refer to this documentation to convert rules on your own for local testing, using the latest rules, etc...
## Python requirements
You need Python 3.8+ and the following modules: `pyyaml`, `ruamel.yaml`, `requests`.
```sh
pip3 install -r requirements.txt
```
## About Sigma
[https://github.com/SigmaHQ/sigma](https://github.com/SigmaHQ/sigma)
## Settings
hayabusa.py needs `sigmac` from the Sigma repository.
Before using hayabusa.py, please clone the Sigma repository.
```sh
git clone https://github.com/SigmaHQ/sigma.git
```
## Usage
Create an environmental variable `$sigma_path` that points to the Sigma repository and register haybausa as a backend for Sigma:
```sh
export sigma_path=/path/to/sigma_repository
cp hayabusa.py $sigma_path/tools/sigma/backends
cp convert.sh $sigma_path
cp splitter.py $sigma_path
```
* CautionBe sure to specify the path to your Sigma repository in place of `/path/to/sigma_repository`.
### Convert Rule
`convert.sh` will convert sigma rules to hayabusa rules and save them in a new `hayabusa_rules` folder.
```sh
export sigma_path=/path/to/sigma_repository
cd $sigma_path
sh convert.sh
```
`sigmac` which we use for convert rule files has many options.
If you want to use some option, edit `convert.sh`
## Currently unsupported rules
The following rules currently cannot be automatically converted because it contains an aggregation operator that has not been implemented yet.
```
sigma/rules/windows/builtin/win_susp_samr_pwset.yml
sigma/rules/windows/image_load/sysmon_mimikatz_inmemory_detection.yml
sigma/rules/windows/process_creation/process_creation_apt_turla_commands_medium.yml
```
## Sigma rule parsing errors
Some rules will have been able to be converted but will cause parsing errors or will not be usable due to various bugs. We will continue to fix these bugs but for the meantime the majority of Sigma rules do work so please ignore the errors for now.

View File

@@ -1,74 +0,0 @@
# SIGMAからHayabusaルールへの自動変換
[![python](https://img.shields.io/badge/python-3.8-blue)](https://www.python.org/)
![version](https://img.shields.io/badge/Platform-Win-green)
![version](https://img.shields.io/badge/Platform-Lin-green)
![version](https://img.shields.io/badge/Platform-Mac-green)
`hayabusa.py` はSigmaルールをHayabusaルールに変換する`sigmac`のバックエンドです。
Sigmaの持つ多くの検知ルールをHayabusaのルールセットに追加することでルールを作成する手間を省くことができます。
## 事前に変換されたSigmaルールについて
Sigmaからhayabusa形式に変換されたルールが`./rules/Sigma`ディレクトリに用意されています。
ローカル環境で新しいルールをテストしたり、Sigmaの最新のルールを変換したりしたい場合は、以下のドキュメンテーションをご参考下さい。
## Pythonの環境依存
Python 3.8以上と次のモジュールが必要です:`pyyaml``ruamel.yaml``requests`
以下のコマンドでインストール可能です。
```sh
pip3 install -r requirements.txt
```
## Sigmaについて
[https://github.com/SigmaHQ/sigma](https://github.com/SigmaHQ/sigma)
## 環境設定
hayabusa.pyはSigmaリポジトリの中にある`sigmac`を使います。
事前に任意のディレクトリにSigmaリポジトリをcloneしてください。
```sh
git clone https://github.com/SigmaHQ/sigma.git
```
## 使い方
Sigmaレポジトリのパスが書いてある`$sigma_path`という環境変数を設定して、hayabusaをSigmaのbackendとして登録します:
```sh
export sigma_path=/path/to/sigma_repository
cp hayabusa.py $sigma_path/tools/sigma/backends
cp convert.sh $sigma_path
cp splitter.py $sigma_path
```
* 注意:`/path/to/sigma_repository`そのままではなくて、自分のSigmaレポジトリのパスを指定してください。
### ルールの変換
`convert.sh`を実行することでルールの変換が実行されます。変換されたルールは`hayabusa_rules`フォルダに作成されます。
```sh
export sigma_path=/path/to/sigma_repository
cd $sigma_path
sh convert.sh
```
ルールの変換に利用しているsigmacには様々なオプションが用意されています。オプションを変更する場合はconvert.shを編集してください。
## 現在サポートされていないルール
以下のルールは、まだ実装されていないaggregation operatorが含まれているため、現在は自動変換できません。
```
sigma/rules/windows/builtin/win_susp_samr_pwset.yml
sigma/rules/windows/image_load/sysmon_mimikatz_inmemory_detection.yml
sigma/rules/windows/process_creation/process_creation_apt_turla_commands_medium.yml
```
## Sigmaルールのパースエラーについて
一部のルールは変換できたものの、パースエラーが発生しています。
これらのバグは引き続き修正していきますが、当面はSigmaのルールの大部分は動作しますので、今のところエラーは無視してください。

View File

@@ -1,5 +0,0 @@
Readme for the hayabusa backend for `sigmac`:
[English](README-English.md)
[Japanese](README-Japanese.md)

View File

@@ -1,3 +0,0 @@
rm -rf hayabusa_rules
python ./tools/sigmac -t hayabusa --config ./tools/config/generic/sysmon.yml --defer-abort -r rules/windows/ > sigma_to_hayabusa.yml
python splitter.py

View File

@@ -1,305 +0,0 @@
import copy
from collections import OrderedDict
from io import StringIO
import yaml
import re
from sigma.backends.base import SingleTextQueryBackend
from sigma.parser.condition import SigmaAggregationParser, ConditionOR, ConditionAND
from sigma.parser.modifiers.base import SigmaTypeModifier
from sigma.parser.modifiers.type import SigmaRegularExpressionModifier
SPECIAL_REGEX = re.compile("^\{(\d)+,?(\d+)?\}")
class HayabusaBackend(SingleTextQueryBackend):
"""Base class for backends that generate one text-based expression from a Sigma rule"""
# see tools.py
# use this value when sigmac parse argument of "-t"
identifier = "hayabusa"
active = True
# the following class variables define the generation and behavior of queries from a parse tree some are prefilled with default values that are quite usual
# Token used for linking expressions with logical AND
andToken = " and "
orToken = " or " # Same for OR
notToken = " not " # Same for NOT
# Syntax for subexpressions, usually parenthesis around it. %s is inner expression
subExpression = "(%s)"
valueExpression = "%s" # Expression of values, %s represents value
# Expression of typed values generated by type modifiers. modifier identifier -> expression dict, %s represents value
typedValueExpression = dict()
sort_condition_lists = False
mapListsSpecialHandling = True
name_idx = 1
selection_prefix = "SELECTION_{0}"
name_2_selection = OrderedDict()
def __init__(self, sigmaconfig, options):
super().__init__(sigmaconfig)
self.re_init()
def re_init(self):
self.name_idx = 1
self.name_2_selection = OrderedDict()
def cleanValue(self, val):
return val
def generateListNode(self, node):
return self.generateORNode(node)
def create_new_selection(self):
name = self.selection_prefix.format(self.name_idx)
self.name_idx += 1
return name
def generateMapItemNode(self, node):
fieldname, value = node
transformed_fieldname = self.fieldNameMapping(fieldname, value)
if self.mapListsSpecialHandling == False and type(value) in (str, int, list) or self.mapListsSpecialHandling == True and type(value) in (str, int):
name = self.create_new_selection()
self.name_2_selection[name] = [
(transformed_fieldname, self.generateNode(value))]
return name
elif type(value) == list:
return self.generateMapItemListNode(transformed_fieldname, value)
elif isinstance(value, SigmaTypeModifier):
return self.generateMapItemTypedNode(transformed_fieldname, value)
elif value is None:
# nullは正規表現で表す。これでいいのかちょっと不安
return self.generateNode((transformed_fieldname+"|re", "^$"))
else:
raise TypeError(
"Backend does not support map values of type " + str(type(value)))
def generateMapItemTypedNode(self, fieldname, value):
# `|re`オプションに対応
if type(value) == SigmaRegularExpressionModifier:
fieldname = fieldname + "|re"
# pythonとかの正規表現では/(スラッシュ)や"(ダブルクオート)をエスケープしてもエラーが出ないが、Rustの正規表現エンジンではスラッシュやダブルクオートをエスケープするとエラーが出てしまう
# そこでスラッシュやダブルクオートのエスケープは消しておく。
# あと、この実装は結構怪しいので、将来バージョンではこの実装を無くして、hayabusa側で使用する正規表現エンジンを普通のpythonとかで使われているやつに変えた方がいいと思う。
regex_value = value.value.replace('\/', '/')
regex_value = regex_value.replace("\\\"", "\"")
# 追加のケースとして、pythonとかの正規表現では{はエスケープ不要だが、Rustでは必要なので、それを修正するためのコード。めんどい
idx = 0
prev_regex = regex_value
regex_value = ""
while idx < len(prev_regex):
# 既にエスケープされているものはスキップする。
if prev_regex[idx:idx+2] == "\\{" or prev_regex[idx:idx+2] == "\\}":
regex_value += prev_regex[idx:idx+2]
idx += 2
continue
ch = prev_regex[idx]
# エスケープ不要な}はここに来ないように、以降の処理でidxを調整している。なのでここにくる}はエスケープが必要。
if ch == "}":
regex_value += "\\}"
idx += 1
continue
# {じゃない場合はそのまま足すだけ
if ch != "{":
regex_value += ch
idx += 1
continue
# {の場合の処理
reg_match = SPECIAL_REGEX.match(prev_regex[idx:])
if reg_match == None:
# 文字列としての{なので、エスケープ必要
regex_value += "\\{"
idx += 1
else:
# これは桁数を指定する{なので、エスケープ不要で}までidxをスキップ
regex_value += reg_match.group()
idx += len(reg_match.group())
return self.generateNode((fieldname, regex_value))
else:
raise NotImplementedError(
"Type modifier '{}' is not supported by backend".format(value.identifier))
def generateMapItemListNode(self, fieldname, value):
# 下記のようなケースに対応
# selection:
# EventID:
### - 1
### - 2
# 基本的にリストはORと良く、generateListNodeもORNodeを生成している。
# しかし、上記のケースでgenerateListNode()を実行すると、下記のようなYAMLになってしまう。
# selection:
### EventID: 1 or 2
# 上記のようにならないように、修正している。
# なお、generateMapItemListNode()を有効にするために、self.mapListsSpecialHandling = Trueとしている
if self._is_all_str(value):
name = self.create_new_selection()
self.name_2_selection[name] = [(fieldname, value)]
return name
list_values = list()
for sub_node in value:
list_values.append((fieldname, sub_node))
return self.subExpression % self.generateORNode(list_values)
def _is_all_str(self, values):
for value in values:
if type(value) != str:
return False
return True
def generateAggregation(self, agg):
# python3 tools/sigmac rules/windows/process_creation/win_dnscat2_powershell_implementation.yml --config tools/config/generic/sysmon.yml --target hayabusa
if agg == None:
return ""
if agg.aggfunc == SigmaAggregationParser.AGGFUNC_COUNT:
# condition の中に "|" は1つのみ
# | 以降をそのまま出力する
target = '|'
condition = agg.parser.parsedyaml["detection"]["condition"]
# conditionはなんと複数指定されることもあるらしい!!!!!
# If multiple conditions are given, they are logically linked with OR.と仕様書に書いてある。詳細はSigmaRuleの仕様を参照のこと。
# とりあえず、複数指定のconditionは未対応ということでエラーにするとして、(なお、デフォルトのbase.pyの実装で複数指定のconditionはexceptionがraiseされるので、そのような処理は追加で実装しなくてよい)
# 問題となるのはagg.parser.parsedyaml["detection"]["condition"]の型
###
# 下記のように指定すると、agg.parser.parsedyaml["detection"]["condition"]の型はstringになるが
### conditon: selection1
###
# 下記のように指定すると、agg.parser.parsedyaml["detection"]["condition"]の型はlistになる
# conditon:
### - selection1
###
# なのでlistのケースも想定して、下記のような実装とする。
if type(condition) == list:
condition = condition[0]
index = condition.find(target)
return condition[index:]
# count以外は対応していないので、エラーを返す
raise NotImplementedError(
"This rule contains aggregation operator not implemented for this backend")
def generateValueNode(self, node):
# このメソッドをオーバーライドしておかないとint型もstr型として扱われてしまうので、int型やint型として、str型はstr型として処理するために実装した。
# このメソッドは最悪無くてもいいような気もする。
if type(node) == int:
return node
else:
return self.valueExpression % (self.cleanValue(str(node)))
# 全部strかどうかを判定
def is_keyword_list(self, node):
if type(node) != ConditionOR:
return False
for item in node.items:
if type(item) != str:
return False
return True
def generateANDNode(self, node):
generated = list()
for val in node:
if type(val) == str or type(val) == int:
# 普通はtupleでkeyとvalueのペアであるが、これはkeyが指定されていないケース
# keyが指定されていない場合は、EventLog全体をgrep検索することになっている。(詳細はSigmaルールの仕様書を参照のこと)
# 具体的には"all of"とか使うとこの分岐に来る
name = self.create_new_selection()
self.name_2_selection[name] = [(None, val)]
generated_node = name
else:
# 普通はこっちにくる
generated_node = self.generateNode(val)
generated.append(generated_node)
filtered = [g for g in generated if g is not None]
if filtered:
if self.sort_condition_lists:
filtered = sorted(filtered)
return self.andToken.join(filtered)
else:
return None
def generateORNode(self, node):
if self.is_keyword_list(node) == True:
# 普通はtupleでkeyとvalueのペアであるが、これはkeyが指定されていないケース
# 全てkeyが指定されていない場合はここに来る。
name = self.create_new_selection()
self.name_2_selection[name] = [(None, val) for val in node]
return name
name = None
generated = list()
for val in node:
# 普通はtupleでkeyとvalueのペアであるが、これはkeyが指定されていないケース
if type(val) == str or type(val) == int:
if name is None:
name = self.create_new_selection()
self.name_2_selection[name] = list()
self.name_2_selection[name].append((None, val))
else:
generated.append(self.generateNode(val))
if name is not None:
generated.append(name)
filtered = [g for g in generated if g is not None]
if filtered:
if self.sort_condition_lists:
filtered = sorted(filtered)
return self.orToken.join(filtered)
else:
return None
def generateQuery(self, parsed):
# このクラスのインスタンスは再利用されるので、内部のメンバ変数をresetする。
self.re_init()
result = self.generateNode(parsed.parsedSearch)
if parsed.parsedAgg:
res = self.generateAggregation(parsed.parsedAgg)
result += " " + res
ret = ""
with StringIO() as bs:
# 元のyamlをいじるとこの後の処理に影響を与える可能性があるので、deepCopyする
parsed_yaml = copy.deepcopy(parsed.sigmaParser.parsedyaml)
# なんかタイトルは先頭に来てほしいので、そのための処理
# parsed.sigmaParser.parsedyamlがOrderedDictならこんなことしなくていい、後で別のやり方があるか調べる
# 順番固定してもいいかも
bs.write("title: " + parsed_yaml["title"]+"\n")
bs.write("ruletype: Sigma\n")
del parsed_yaml["title"]
# detectionの部分をクリアする前にtimeframeだけ確保しておく。
timeframe = None
if "timeframe" in parsed_yaml["detection"]:
timeframe = parsed_yaml["detection"]["timeframe"]
# detectionの部分だけ変更して出力する。
parsed_yaml["detection"] = {}
if timeframe is not None and len(timeframe) != 0:
parsed_yaml["detection"]["timeframe"] = timeframe
parsed_yaml["detection"]["condition"] = result
for key, values in self.name_2_selection.items():
# fieldnameの有無を確認している
if values[0][0]:
# 通常はfieldnameがあってその場合は連想配列で初期化
parsed_yaml["detection"][key] = {}
else:
# is_keyword_list() == Trueの場合だけ、ここにくる
parsed_yaml["detection"][key] = []
for fieldname, value in values:
if fieldname == None:
## is_keyword_list() == Trueの場合
parsed_yaml["detection"][key].append(value)
else:
## is_keyword_list() == Falseの場合
parsed_yaml["detection"][key][fieldname] = value
yaml.dump(parsed_yaml, bs, indent=4, default_flow_style=False)
ret = bs.getvalue()
ret += "---\n"
return ret

View File

@@ -1,3 +0,0 @@
pyyaml
ruamel.yaml
requests

View File

@@ -1,38 +0,0 @@
## pip install pyyaml
import os
import ruamel.yaml
yaml = ruamel.yaml.YAML()
def load_ymls( filepath ):
with open(filepath) as f:
return list(yaml.load_all(f))
def dump_yml( filepath, data ):
with open(filepath, "w") as stream:
yaml.dump(data, stream )
def main():
loaded_ymls = load_ymls("sigma_to_hayabusa.yml")
for loaded_yml in loaded_ymls:
if loaded_yml == None:
continue
if loaded_yml["yml_path"] == None or len(loaded_yml["yml_path"]) == 0:
continue
out_dir = "hayabusa_rules/" + loaded_yml["yml_path"]
out_path = out_dir + "/" + loaded_yml["yml_filename"]
if not os.path.exists(out_dir):
os.makedirs(out_dir)
loaded_yml.pop("yml_path")
loaded_yml.pop("yml_filename")
dump_yml(out_path,loaded_yml)
if __name__ == "__main__":
main()

View File

@@ -1,12 +0,0 @@
このフォルダにはテストに必要なファイルが格納されています。
テストを実行する際には、toos/sigmacにあるファイルに加え、このフォルダのファイルもsigmaディレクトリにコピーしてください。
このフォルダにあるファイルについて説明します。
* test_rules: テスト用のSIGMAルールが格納されたフォルダです。
* convert_test.sh: テストを実行するためのシェルスクリプトです。test_rulesフォルダ内のSIGMAルールをhayabusaルールに変換し、hayabusa_rules_testフォルダに出力します。
* expected_rules: test_rulesフォルダ内のルールを正しく変換すると、このフォルダに設置されているhayabusaルールと同じになるはずです。
テストは下記のように実行します。
* convert_test.shを実行する。
* WinMerge等のツールを利用して、expected_rulesフォルダとhayabusa_rules_testフォルダに差分を確認する。
* 差分が無ければテストOKです。差分があれば、内容を確認して適宜修正してください。

View File

@@ -1,3 +0,0 @@
rm -rf hayabusa_rules
python ./tools/sigmac -t hayabusa --config ./tools/config/generic/sysmon.yml --defer-abort -r test_rules/ > sigma_to_hayabusa.yml
python splitter.py hayabusa_rules_test

View File

@@ -1,17 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "simple test\n"
detection:
SELECTION_1:
EventID: 4100
condition: SELECTION_1
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,26 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "map test and escape str test and empty string test and null test\n"
detection:
SELECTION_1:
EventID: 4100
SELECTION_2:
ObjectType: Key
SELECTION_3:
ObjectKey: "aaaValu__-*|3'|e "
SELECTION_4:
Ojb: ''
SELECTION_5:
aaa|re: ^$
condition: (SELECTION_1 and SELECTION_2 and SELECTION_3 and SELECTION_4 and SELECTION_5)
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,26 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "list test\n"
detection:
SELECTION_1:
EventID: 4100
SELECTION_2:
EventID: 9000
SELECTION_3:
EventID: 8000
SELECTION_4:
EventID: aaaa
SELECTION_5:
ObjectType: Key
condition: ((SELECTION_1 or SELECTION_2 or SELECTION_3 or SELECTION_4) and SELECTION_5)
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,24 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "list test\n"
detection:
SELECTION_1:
- 2
- dee
- testtesttest
SELECTION_2:
EventID: 22
SELECTION_3:
EventID: 33
condition: ((SELECTION_1) and (SELECTION_2 or SELECTION_3))
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,26 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "all modifier\n"
detection:
SELECTION_1:
- 2
- dee
- testtesttest
SELECTION_2:
EventID: 22
SELECTION_3:
EventID: 33
SELECTION_4:
EventID: hoge
condition: ((SELECTION_1) and (SELECTION_2 and SELECTION_3 and SELECTION_4))
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,20 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "contains modifier\n"
detection:
SELECTION_1:
UserName: '*hogehoge*'
SELECTION_2:
TargetUserName: '*testest2*'
condition: (SELECTION_1 or SELECTION_2)
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,20 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "endswith pipe modifier and startswith pipe modifier\n"
detection:
SELECTION_1:
UserName: '*hogehoge_end'
SELECTION_2:
TargetUserName: test_start*
condition: (SELECTION_1 or SELECTION_2)
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,20 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "base64 encode modifier\n"
detection:
SELECTION_1:
UserName: YmFzZTY0X2VuY29kZWQ=
SELECTION_2:
TargetUserName: test_start
condition: (SELECTION_1 and SELECTION_2)
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,26 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "re modifier test\n"
detection:
SELECTION_1:
UserName|re: aaa
SELECTION_2:
UserName2|re: .*bbbb$
SELECTION_3:
UserName3|re: cccc/dd/dd
SELECTION_4:
UserName4|re: cccc"dddd
SELECTION_5:
UserName5|re: cccc"dddd
condition: (SELECTION_1 and SELECTION_2 and SELECTION_3 and SELECTION_4 and SELECTION_5)
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,35 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "all of test\n"
detection:
SELECTION_1:
EventID: 3
SELECTION_2:
EventID: 7
SELECTION_3:
EventID: a
SELECTION_4:
UserName: abc
SELECTION_5:
process: nnn
SELECTION_6:
parentprocess: 2
SELECTION_7:
uuu: zzzz
SELECTION_8:
xxxx: 3
SELECTION_9:
ppp: iiii
condition: (((SELECTION_1 or SELECTION_2 or SELECTION_3) and SELECTION_4 and SELECTION_5
and SELECTION_6 and SELECTION_7 and SELECTION_8) or SELECTION_9)
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,33 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "1 of\n"
detection:
SELECTION_1:
EventID: 3
SELECTION_2:
EventID: 7
SELECTION_3:
UserName: abc
SELECTION_4:
process: nnn
SELECTION_5:
parentprocess: sss
SELECTION_6:
uuu: zzzz
SELECTION_7:
xxxx: yyyyy
SELECTION_8:
ppp: iiii
condition: ((((SELECTION_1 or SELECTION_2) and SELECTION_3) or (SELECTION_4 and
SELECTION_5) or (SELECTION_6 and SELECTION_7)) and SELECTION_8)
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,31 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "all of them\n"
detection:
SELECTION_1:
EventID: 3
SELECTION_2:
EventID: 7
SELECTION_3:
UserName: abc
SELECTION_4:
process: nnn
SELECTION_5:
parentprocess: sss
SELECTION_6:
uuu: zzzz
SELECTION_7:
xxxx: yyyyy
condition: ((SELECTION_1 or SELECTION_2) and SELECTION_3 and SELECTION_4 and SELECTION_5
and SELECTION_6 and SELECTION_7)
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,31 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "1 of them\n"
detection:
SELECTION_1:
EventID: 3
SELECTION_2:
EventID: 7
SELECTION_3:
UserName: abc
SELECTION_4:
process: nnn
SELECTION_5:
parentprocess: sss
SELECTION_6:
uuu: zzzz
SELECTION_7:
xxxx: yyyyy
condition: (((SELECTION_1 or SELECTION_2) and SELECTION_3) or (SELECTION_4 and SELECTION_5)
or (SELECTION_6 and SELECTION_7))
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,19 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "timeflame \n"
detection:
SELECTION_1:
EventID: 3
condition: SELECTION_1
timeflame: 2d
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,25 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "condition and or\n"
detection:
SELECTION_1:
EventID: 3
SELECTION_2:
aaa: bbb
SELECTION_3:
ccc: ddd
SELECTION_4:
eee: fff
condition: (SELECTION_1 and SELECTION_2 and SELECTION_3 and SELECTION_4)
timeflame: 2d
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,25 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "condition and or\n"
detection:
SELECTION_1:
EventID: 3
SELECTION_2:
aaa: bbb
SELECTION_3:
ccc: ddd
SELECTION_4:
eee: fff
condition: (SELECTION_1 and SELECTION_2 and SELECTION_3 and SELECTION_4)
timeflame: 2d
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,25 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "condition or\n"
detection:
SELECTION_1:
EventID: 3
SELECTION_2:
aaa: bbb
SELECTION_3:
ccc: ddd
SELECTION_4:
eee: fff
condition: (SELECTION_1 or SELECTION_2 or SELECTION_3 or SELECTION_4)
timeflame: 2d
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,27 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "() \n"
detection:
SELECTION_1:
EventID: 3
SELECTION_2:
aaa: bbb
SELECTION_3:
ccc: ddd
SELECTION_4:
eee: fff
SELECTION_5:
ggg: hhh
condition: ((SELECTION_1 and (SELECTION_2 or (SELECTION_3 and SELECTION_4))) or
SELECTION_5)
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,18 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "condition not\n"
detection:
SELECTION_1:
EventID: 3
condition: ' not (SELECTION_1)'
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,27 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "condition not ()\n"
detection:
SELECTION_1:
EventID: 3
SELECTION_2:
aaa: bbb
SELECTION_3:
ccc: ddd
SELECTION_4:
eee: fff
SELECTION_5:
ggg: hhh
condition: ( not (SELECTION_1) and not ((SELECTION_2 and not ((SELECTION_3 or
SELECTION_4)))) and SELECTION_5)
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,20 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "condition count\n"
detection:
SELECTION_1:
EventID: 3
SELECTION_2:
aaa: bbb
condition: (SELECTION_1 and not (SELECTION_2)) | count() < 3
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,20 +0,0 @@
title: test
ruletype: SIGMA
author: test
date: 2021/12/4
description: "condition count\n"
detection:
SELECTION_1:
EventID: 3
SELECTION_2:
aaa: bbb
condition: (SELECTION_1 and not (SELECTION_2)) | count(TEAMNAME) by HOGE < 3
falsepositives:
- Unknown
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
level: medium
logsource:
product: windows
service: security
status: experimental

View File

@@ -1,17 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
simple test
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection:
EventID: 4100
condition: selection
falsepositives:
- Unknown
level: medium

View File

@@ -1,21 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
map test and escape str test and empty string test and null test
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection:
EventID: 4100
ObjectType: 'Key'
ObjectKey: 'aaaValu__-*|3''|e '
Ojb: ''
aaa: null
condition: selection
falsepositives:
- Unknown
level: medium

View File

@@ -1,22 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
list test
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection:
EventID:
- 4100
- 9000
- 8000
- "aaaa"
ObjectType: 'Key'
condition: selection
falsepositives:
- Unknown
level: medium

View File

@@ -1,23 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
list test
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection:
- 2
- dee
- testtesttest
SELECTION_2:
EventID:
- 22
- 33
condition: selection and SELECTION_2
falsepositives:
- Unknown
level: medium

View File

@@ -1,24 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
all modifier
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection:
- 2
- dee
- testtesttest
SELECTION_2:
EventID|all:
- 22
- 33
- hoge
condition: selection and SELECTION_2
falsepositives:
- Unknown
level: medium

View File

@@ -1,18 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
contains modifier
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection:
- UserName|contains: hogehoge
- TargetUserName|contains: testest2
condition: selection
falsepositives:
- Unknown
level: medium

View File

@@ -1,18 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
endswith pipe modifier and startswith pipe modifier
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection:
- UserName|endswith: hogehoge_end
- TargetUserName|startswith: test_start
condition: selection
falsepositives:
- Unknown
level: medium

View File

@@ -1,18 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
base64 encode modifier
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection:
UserName|base64: base64_encoded
TargetUserName: test_start
condition: selection
falsepositives:
- Unknown
level: medium

View File

@@ -1,22 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
re modifier test
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection:
UserName|re: aaa
UserName2|re: .*bbbb$
UserName3|re: cccc\/dd/\//dd # see hayabusa.py generateMapItemTypedNode()
UserName4|re: cccc\"dd"\""dd # see hayabusa.py generateMapItemTypedNode()
UserName5|re: cccc{{3}0dddd # see hayabusa.py generateMapItemTypedNode()
UserName6|re: cccc{{3}0d{32}dd # see hayabusa.py generateMapItemTypedNode()
condition: selection
falsepositives:
- Unknown
level: medium

View File

@@ -1,29 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
all of test
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection1:
EventID:
- 3
- 7
- a
UserName: abc
selection2:
process: nnn
parentprocess: 2
selection3:
uuu: zzzz
xxxx: 3
another:
ppp: iiii
condition: all of selection* or another
falsepositives:
- Unknown
level: medium

View File

@@ -1,28 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
1 of
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection1:
EventID:
- 3
- 7
UserName: abc
selection2:
process: nnn
parentprocess: sss
selection3:
uuu: zzzz
xxxx: yyyyy
another:
ppp: iiii
condition: 1 of selection* and another
falsepositives:
- Unknown
level: medium

View File

@@ -1,26 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
all of them
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection1:
EventID:
- 3
- 7
UserName: abc
selection2:
process: nnn
parentprocess: sss
selection3:
uuu: zzzz
xxxx: yyyyy
condition: all of them
falsepositives:
- Unknown
level: medium

View File

@@ -1,26 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
1 of them
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection1:
EventID:
- 3
- 7
UserName: abc
selection2:
process: nnn
parentprocess: sss
selection3:
uuu: zzzz
xxxx: yyyyy
condition: 1 of them
falsepositives:
- Unknown
level: medium

View File

@@ -1,18 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
timeflame
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection1:
EventID: 3
timeflame: 2d
condition: selection1
falsepositives:
- Unknown
level: medium

View File

@@ -1,24 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
condition and or
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection1:
EventID: 3
selection2:
aaa: bbb
selection3:
ccc: ddd
selection4:
eee: fff
timeflame: 2d
condition: selection1 and selection2 and selection3 and selection4
falsepositives:
- Unknown
level: medium

View File

@@ -1,24 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
condition and
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection1:
EventID: 3
selection2:
aaa: bbb
selection3:
ccc: ddd
selection4:
eee: fff
timeflame: 2d
condition: selection1 and selection2 and selection3 and selection4
falsepositives:
- Unknown
level: medium

View File

@@ -1,24 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
condition or
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection1:
EventID: 3
selection2:
aaa: bbb
selection3:
ccc: ddd
selection4:
eee: fff
timeflame: 2d
condition: selection1 or selection2 or selection3 or selection4
falsepositives:
- Unknown
level: medium

View File

@@ -1,25 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
()
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection1:
EventID: 3
selection2:
aaa: bbb
selection3:
ccc: ddd
selection4:
eee: fff
selection5:
ggg: hhh
condition: selection1 and ( selection2 or (selection3 and selection4) ) or selection5
falsepositives:
- Unknown
level: medium

View File

@@ -1,17 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
condition not
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection1:
EventID: 3
condition: not selection1
falsepositives:
- Unknown
level: medium

View File

@@ -1,25 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
condition not ()
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection1:
EventID: 3
selection2:
aaa: bbb
selection3:
ccc: ddd
selection4:
eee: fff
selection5:
ggg: hhh
condition: not selection1 and not( selection2 and not (selection3 or selection4)) and selection5
falsepositives:
- Unknown
level: medium

View File

@@ -1,19 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
condition count
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection1:
EventID: 3
selection2:
aaa: bbb
condition: selection1 and not selection2 | count() < 3
falsepositives:
- Unknown
level: medium

View File

@@ -1,19 +0,0 @@
title: test
id: ff151c33-45fa-475d-af4f-c2f93571f4fe
description: |
condition count
status: experimental
date: 2021/12/4
author: test
logsource:
product: windows
service: security
detection:
selection1:
EventID: 3
selection2:
aaa: bbb
condition: selection1 and not selection2 | count(TEAMNAME) by HOGE < 3
falsepositives:
- Unknown
level: medium