CSEC_PUBLIC/hayabusa
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #648 from Yamato-Security/637-separate-mitre-attck-tags-and-other-tags-when-outputting

Browse Source 
Separate mitre attck tags and other tags when outputting
This commit is contained in:
Yamato Security
2022-08-08 11:11:34 +09:00
committed by GitHub
parent e494b550fc 4411457663
commit 8712800c5b
13 changed files with 147 additions and 70 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
CHANGELOG-Japanese.md
View File

@@ -9,6 +9,7 @@
**改善:**
- ルールのアップデート機能のルールパスの出力から./を削除した。 (#642) (@hitenkoku)
- MITRE ATT&CK関連のタグとその他タグを出力するための出力用のエイリアスを追加した。 (#637) (@hitenkoku)
**バグ修正:**

1
CHANGELOG.md
View File

@@ -9,6 +9,7 @@
**Enhancements:**
- Removed ./ from rule path when updating. (#642) (@hitenkoku)
- Added new output alias for MITRE ATT&CK tags and other tags. (#637) (@hitenkoku)
**Bug Fixes:**

9
README-Japanese.md
View File

@@ -332,7 +332,6 @@ OPTIONS:
--RFC-3339 RFC 3339形式で日付と時刻を出力する (例: 2022-02-22 22:00:00.123456-06:00)
--US-military-time 24時間制(ミリタリータイム)のアメリカ形式で日付と時刻を出力する (例: 02-22-2022 22:00:00.123 -06:00)
--US-time アメリカ形式で日付と時刻を出力する (例: 02-22-2022 10:00:00.123 PM -06:00)
--all-tags 出力したCSVファイルにルール内のタグ情報を全て出力する
-c, --rules-config <RULE_CONFIG_DIRECTORY> ルールフォルダのコンフィグディレクトリ (デフォルト: ./rules/config)
--contributors コントリビュータの一覧表示
-d, --directory <DIRECTORY> .evtxファイルを持つディレクトリのパス
@@ -509,7 +508,9 @@ Hayabusaの結果を標準出力に表示しているとき（デフォルト）
* `Title`: YML検知ルールの`title`フィールドから来ています。
* `RecordID`: イベントレコードIDです。`<Event><System><EventRecordID>`フィールドから来ています。
* `Details`: YML検知ルールの`details`フィールドから来ていますが、このフィールドはHayabusaルールにしかありません。このフィールドはアラートとイベントに関する追加情報を提供し、ログのフィールドから有用なデータを抽出することができます。イベントキーのマッピングが間違っている場合、もしくはフィールドが存在しない場合で抽出ができなかった箇所は`n/a` (not available)と記載されます。YML検知ルールに`details`フィールドが存在しない時のdetailsのメッセージを`./rules/config/default_details.txt`で設定できます。`default_details.txt`では`Provider Name`、`EventID`、`details`の組み合わせで設定することができます。default_details.txt`やYML検知ルールに対応するルールが記載されていない場合はすべてのフィールド情報を出力します。
* `MitreAttack`: MITRE ATT&CKの戦術。
* `MitreTactics`: MITRE ATT&CKの戦術。
* `MitreTags`: MITRE ATT&CKの戦術以外の情報。attack.g(グループ)、attack.t(技術)、attack.s(ソフトウェア)の情報を出力します。
* `OtherTags`: YML検知ルールの`tags` フィールドから`MitreTactics`, `MitreTags` 以外の月情報を出力します。
* `RuleFile`: アラートまたはイベントを生成した検知ルールのファイル名。
* `EvtxFile`: アラートまたはイベントを起こしたevtxファイルへのパス。
* `RecordInformation`: すべてのフィールド情報。
@@ -527,7 +528,9 @@ default_profiles.txtをprofile.txtに書かれているプロファイルで上
|%Channel% | `Channel` |
|%Level% | `Level` |
|%EventID% | `EventID` |
|%MitreAttack% | `MitreAttack` |
|%MitreTactics% | `MitreTactics` |
|%MitreTags% | `MitreTags` |
|%OtherTags% | `OtherTags` |
|%RecordID% | `RecordID` |
|%RuleTitle% | `Title` |
|%Details% | `Details` |

9
README.md
View File

@@ -328,7 +328,6 @@ OPTIONS:
--RFC-3339 Output timestamp in RFC 3339 format (ex: 2022-02-22 22:00:00.123456-06:00)
--US-military-time Output timestamp in US military time format (ex: 02-22-2022 22:00:00.123 -06:00)
--US-time Output timestamp in US time format (ex: 02-22-2022 10:00:00.123 PM -06:00)
--all-tags Output all tags when saving to a CSV file
-c, --rules-config <RULE_CONFIG_DIRECTORY> Specify custom rule config folder (default: ./rules/config)
--contributors Print the list of contributors
-d, --directory <DIRECTORY> Directory of multiple .evtx files
@@ -506,7 +505,9 @@ When hayabusa output is being displayed to the screen (the default), it can disp
* `RecordID`: This comes from the `<Event><System><EventRecordID>` field in the event log.
* `Title`: This comes from the `title` field in the YML detection rule.
* `Details`: This comes from the `details` field in the YML detection rule, however, only hayabusa rules have this field. This field gives extra information about the alert or event and can extract useful data from the fields in event logs. For example, usernames, command line information, process information, etc... When a placeholder points to a field that does not exist or there is an incorrect alias mapping, it will be outputted as `n/a` (not available). If the `details` field is not specified (i.e. sigma rules), default `details` messages to extract fields defined in `./rules/config/default_details.txt` will be outputted. You can add more default `details` messages by adding the `Provider　Name`, `EventID` and `details` message you want to output in `default_details.txt`. When no `details` field is defined in a rule nor in `default_details.txt`, all fields will be outputted to the `details` column.
* `MitreAttack`: MITRE ATT&CK tactics.
* `MitreTactics`: MITRE ATT&CK tactics.
* `MitreTags`: MITRE ATT&CK group, technique, software.
* `OtherTags`: This comes from the `tags` field in YML detection rule which is excluded `MitreTactics` and `MitreTags`.
* `RuleFile`: The filename of the detection rule that generated the alert or event.
* `EvtxFile`: The evtx filename that caused the alert or event.
* `RecordInformation`: All field information.
@@ -525,7 +526,9 @@ Please use `--set-default-profile` option when you want to overwrite `default_p
|%Channel% | `Channel` |
|%Level% | `Level` |
|%EventID% | `EventID` |
|%MitreAttack% | `MitreAttack` |
|%MitreTactics% | `MitreTactics` |
|%MitreTags% | `MitreTags` |
|%OtherTags% | `OtherTags` |
|%RecordID% | `RecordID` |
|%RuleTitle% | `Title` |
|%Details% | `Details` |

9
config/default_profile.yaml
View File

@@ -2,12 +2,9 @@
Timestamp: "%Timestamp%"
Computer: "%Computer%"
Channel: "%Channel%"
Level: "%Level%"
EventID: "%EventID%"
MitreAttack: "%MitreAttack%"
Level: "%Level%"
MitreTactics: "%MitreTactics%"
RecordID: "%RecordID%"
RuleTitle: "%RuleTitle%"
Details: "%Details%"
RecordInformation: "%RecordInformation%"
RuleFile: "%RuleFile%"
EvtxFile: "%EvtxFile%"
Details: "%Details%"

0
config/output_tag.txt → config/mitre_tactics.txt
View File

35
config/profiles.yaml
View File

@@ -1,3 +1,4 @@
#Standard profile minus MITRE ATT&CK Tactics and Record ID.
minimal:
Timestamp: "%Timestamp%"
Computer: "%Computer%"
@@ -13,32 +14,56 @@ standard:
Channel: "%Channel%"
EventID: "%EventID%"
Level: "%Level%"
Tags: "%MitreAttack%"
MitreTactics: "%MitreTactics%"
RecordID: "%RecordID%"
RuleTitle: "%RuleTitle%"
Details: "%Details%"
verbose-1:
#Standard profile plus MitreTags(MITRE techniques, software and groups), rule filename and EVTX filename.
verbose:
Timestamp: "%Timestamp%"
Computer: "%Computer%"
Channel: "%Channel%"
EventID: "%EventID%"
Level: "%Level%"
Tags: "%MitreAttack%"
MitreTactics: "%MitreTactics%"
MitreTags: "%MitreTags%"
OtherTags: "%OtherTags%"
RecordID: "%RecordID%"
RuleTitle: "%RuleTitle%"
Details: "%Details%"
RuleFile: "%RuleFile%"
EvtxFile: "%EvtxFile%"
verbose-2:
#Verbose profile with all field information instead of the minimal fields defined in Details.
verbose-all-field-info:
Timestamp: "%Timestamp%"
Computer: "%Computer%"
Channel: "%Channel%"
EventID: "%EventID%"
Level: "%Level%"
Tags: "%MitreAttack%"
MitreTactics: "%MitreTactics%"
MitreTags: "%MitreTags%"
OtherTags: "%OtherTags%"
RecordID: "%RecordID%"
RuleTitle: "%RuleTitle%"
AllFieldInfo: "%RecordInformation%"
RuleFile: "%RuleFile%"
EvtxFile: "%EvtxFile%"
#Verbose profile plus all field information. (Warning: this will more than double the output file size!)
verbose-details-and-all-field-info:
Timestamp: "%Timestamp%"
Computer: "%Computer%"
Channel: "%Channel%"
EventID: "%EventID%"
Level: "%Level%"
MitreTactics: "%MitreTactics%"
MitreTags: "%MitreTags%"
OtherTags: "%OtherTags%"
RecordID: "%RecordID%"
RuleTitle: "%RuleTitle%"
Details: "%Details%"
RuleFile: "%RuleFile%"
EvtxFile: "%EvtxFile%"
AllFieldInfo: "%RecordInformation%"

7
src/afterfact.rs
View File

@@ -579,11 +579,8 @@ mod tests {
#[test]
fn test_emit_csv_output() {
let mock_ch_filter = message::create_output_filter_config(
"rules/config/channel_abbreviations.txt",
true,
false,
);
let mock_ch_filter =
message::create_output_filter_config("test_files/config/channel_abbreviations.txt");
let test_filepath: &str = "test.evtx";
let test_rulepath: &str = "test-rule.yml";
let test_title = "test_title";

4
src/detections/configs.rs
View File

@@ -93,10 +93,6 @@ pub struct Config {
#[clap(short = 'o', long, value_name = "CSV_TIMELINE")]
pub output: Option<PathBuf>,
/// Output all tags when saving to a CSV file
#[clap(long = "all-tags")]
pub all_tags: bool,
/// Output verbose information
#[clap(short = 'v', long)]
pub verbose: bool,

98
src/detections/detection.rs
View File

@@ -6,6 +6,7 @@ use crate::options::profile::{
LOAEDED_PROFILE_ALIAS, PRELOAD_PROFILE, PRELOAD_PROFILE_REGEX, PROFILES,
};
use chrono::{TimeZone, Utc};
use itertools::Itertools;
use termcolor::{BufferWriter, Color, ColorChoice};
use crate::detections::message::AlertMessage;
@@ -208,7 +209,7 @@ impl Detection {
/// 条件に合致したレコードを格納するための関数
fn insert_message(rule: &RuleNode, record_info: &EvtxRecordInfo) {
let tag_info: Vec<String> = Detection::get_tag_info(rule);
let tag_info: &Vec<String> = &Detection::get_tag_info(rule);
let recinfo = record_info
.record_information
.as_ref()
@@ -275,9 +276,6 @@ impl Detection {
"%EventID%" => {
profile_converter.insert("%EventID%".to_string(), eid.to_owned());
}
"%MitreAttack%" => {
profile_converter.insert("%MitreAttack%".to_string(), tag_info.join(" : "));
}
"%RecordID%" => {
profile_converter.insert(
"%RecordID%".to_string(),
@@ -319,6 +317,44 @@ impl Detection {
.to_string(),
);
}
"%MitreTactics%" => {
let tactics: &Vec<String> = &tag_info
.iter()
.filter(|x| TAGS_CONFIG.values().contains(x))
.map(|y| y.to_owned())
.collect();
profile_converter.insert("%MitreTactics%".to_string(), tactics.join(" : "));
}
"%MitreTags%" => {
let techniques: &Vec<String> = &tag_info
.iter()
.filter(|x| {
!TAGS_CONFIG.values().contains(x)
&& (x.starts_with("attack.t")
|| x.starts_with("attack.g")
|| x.starts_with("attack.s"))
})
.map(|y| {
let mut replaced_tag = y.replace("attack.", "");
make_ascii_titlecase(&mut replaced_tag)
})
.collect();
profile_converter.insert("%MitreTags%".to_string(), techniques.join(" : "));
}
"%OtherTags%" => {
let tags: &Vec<String> = &tag_info
.iter()
.filter(|x| {
!(TAGS_CONFIG.values().contains(x)
|| x.starts_with("attack.t")
|| x.starts_with("attack.g")
|| x.starts_with("attack.s"))
})
.map(|y| y.to_owned())
.collect();
profile_converter.insert("%OtherTags%".to_string(), tags.join(" : "));
}
_ => {}
}
}
@@ -350,7 +386,7 @@ impl Detection {
/// insert aggregation condition detection message to output stack
fn insert_agg_message(rule: &RuleNode, agg_result: AggResult) {
let tag_info: Vec<String> = Detection::get_tag_info(rule);
let tag_info: &Vec<String> = &Detection::get_tag_info(rule);
let output = Detection::create_count_output(rule, &agg_result);
let rec_info = if LOAEDED_PROFILE_ALIAS.contains("%RecordInformation%") {
Option::Some(String::default())
@@ -386,9 +422,6 @@ impl Detection {
"%EventID%" => {
profile_converter.insert("%EventID%".to_string(), "-".to_owned());
}
"%MitreAttack%" => {
profile_converter.insert("%MitreAttack%".to_owned(), tag_info.join(" : "));
}
"%RecordID%" => {
profile_converter.insert("%RecordID%".to_string(), "".to_owned());
}
@@ -415,6 +448,43 @@ impl Detection {
"%EvtxFile%" => {
profile_converter.insert("%EvtxFile%".to_string(), "-".to_owned());
}
"%MitreTactics%" => {
let tactics: &Vec<String> = &tag_info
.iter()
.filter(|x| TAGS_CONFIG.values().contains(x))
.map(|y| y.to_owned())
.collect();
profile_converter.insert("%MitreTactics%".to_string(), tactics.join(" : "));
}
"%MitreTags%" => {
let techniques: &Vec<String> = &tag_info
.iter()
.filter(|x| {
!TAGS_CONFIG.values().contains(x)
&& (x.starts_with("attack.t")
|| x.starts_with("attack.g")
|| x.starts_with("attack.s"))
})
.map(|y| {
let mut replaced_tag = y.replace("attack.", "");
make_ascii_titlecase(&mut replaced_tag)
})
.collect();
profile_converter.insert("%MitreTags%".to_string(), techniques.join(" : "));
}
"%OtherTags%" => {
let tags: &Vec<String> = &tag_info
.iter()
.filter(|x| {
!(TAGS_CONFIG.values().contains(x)
|| x.starts_with("attack.t")
|| x.starts_with("attack.g")
|| x.starts_with("attack.s"))
})
.map(|y| y.to_owned())
.collect();
profile_converter.insert("%OtherTags%".to_string(), tags.join(" : "));
}
_ => {}
}
}
@@ -447,8 +517,14 @@ impl Detection {
.as_vec()
.unwrap_or(&Vec::default())
.iter()
.filter_map(|info| TAGS_CONFIG.get(info.as_str().unwrap_or(&String::default())))
.map(|str| str.to_owned())
.map(|info| {
if let Some(tag) = TAGS_CONFIG.get(info.as_str().unwrap_or(&String::default()))
{
tag.to_owned()
} else {
info.as_str().unwrap_or(&String::default()).to_owned()
}
})
.collect(),
true => rule.yaml["tags"]
.as_vec()
@@ -457,7 +533,7 @@ impl Detection {
.map(
|info| match TAGS_CONFIG.get(info.as_str().unwrap_or(&String::default())) {
Some(s) => s.to_owned(),
_ => info.as_str().unwrap_or("").replace("attack.", ""),
_ => info.as_str().unwrap_or("").to_string(),
},
)
.collect(),

38
src/detections/message.rs
View File

@@ -48,11 +48,9 @@ lazy_static! {
pub static ref STATISTICS_FLAG: bool = configs::CONFIG.read().unwrap().args.statistics;
pub static ref LOGONSUMMARY_FLAG: bool = configs::CONFIG.read().unwrap().args.logon_summary;
pub static ref TAGS_CONFIG: HashMap<String, String> = create_output_filter_config(
utils::check_setting_path(&CURRENT_EXE_PATH.to_path_buf(), "config/output_tag.txt")
utils::check_setting_path(&CURRENT_EXE_PATH.to_path_buf(), "config/mitre_tactics.txt")
.to_str()
.unwrap(),
true,
configs::CONFIG.read().unwrap().args.all_tags
);
pub static ref CH_CONFIG: HashMap<String, String> = create_output_filter_config(
utils::check_setting_path(
@@ -61,8 +59,6 @@ lazy_static! {
)
.to_str()
.unwrap(),
false,
configs::CONFIG.read().unwrap().args.all_tags
);
pub static ref PIVOT_KEYWORD_LIST_FLAG: bool =
configs::CONFIG.read().unwrap().args.pivot_keywords_list;
@@ -94,15 +90,8 @@ lazy_static! {
/// ファイルパスで記載されたtagでのフル名、表示の際に置き換えられる文字列のHashMapを作成する関数。
/// ex. attack.impact,Impact
pub fn create_output_filter_config(
path: &str,
read_tags: bool,
pass_flag: bool,
) -> HashMap<String, String> {
pub fn create_output_filter_config(path: &str) -> HashMap<String, String> {
let mut ret: HashMap<String, String> = HashMap::new();
if read_tags && pass_flag {
return ret;
}
let read_result = utils::read_csv(path);
if read_result.is_err() {
AlertMessage::alert(read_result.as_ref().unwrap_err()).ok();
@@ -591,9 +580,9 @@ mod tests {
);
}
#[test]
/// test of loading output filter config by output_tag.txt
fn test_load_output_tag() {
let actual = create_output_filter_config("test_files/config/output_tag.txt", true, false);
/// test of loading output filter config by mitre_tactics.txt
fn test_load_mitre_tactics_log() {
let actual = create_output_filter_config("test_files/config/mitre_tactics.txt");
let expected: HashMap<String, String> = HashMap::from([
("attack.impact".to_string(), "Impact".to_string()),
("xxx".to_string(), "yyy".to_string()),
@@ -601,24 +590,11 @@ mod tests {
_check_hashmap_element(&expected, actual);
}
#[test]
/// test of loading pass by output_tag.txt
fn test_no_load_output_tag() {
let actual = create_output_filter_config("test_files/config/output_tag.txt", true, true);
let expected: HashMap<String, String> = HashMap::new();
_check_hashmap_element(&expected, actual);
}
#[test]
/// loading test to channel_abbrevations.txt
fn test_load_abbrevations() {
let actual =
create_output_filter_config("test_files/config/channel_abbreviations.txt", false, true);
let actual2 = create_output_filter_config(
"test_files/config/channel_abbreviations.txt",
false,
false,
);
let actual = create_output_filter_config("test_files/config/channel_abbreviations.txt");
let actual2 = create_output_filter_config("test_files/config/channel_abbreviations.txt");
let expected: HashMap<String, String> = HashMap::from([
("Security".to_string(), "Sec".to_string()),
("xxx".to_string(), "yyy".to_string()),

6
src/options/profile.rs
View File

@@ -36,12 +36,14 @@ lazy_static! {
"%Channel%",
"%Level%",
"%EventID%",
"%MitreAttack%",
"%RecordID%",
"%RuleTitle%",
"%RecordInformation%",
"%RuleFile%",
"%EvtxFile%"
"%EvtxFile%",
"%MitreTactics%",
"%MitreTags%",
"%OtherTags%"
];
pub static ref PRELOAD_PROFILE_REGEX: RegexSet = RegexSet::new(&*PRELOAD_PROFILE).unwrap();
}

0
test_files/config/output_tag.txt → test_files/config/mitre_tactics.txt
View File