From 571f9ff51f2679fd6c846c530ece4e3cb814371a Mon Sep 17 00:00:00 2001 From: Shirofune-Security <43838376+Shirofune-Security@users.noreply.github.com> Date: Fri, 18 Sep 2026 22:53:31 +0900 Subject: [PATCH 1/2] Document domain NTLM audit correction in changelogs --- CHANGELOG-Japanese.md | 3 ++- CHANGELOG.md | 3 ++- website/docs/resources/changelog.ja.md | 2 ++ website/docs/resources/changelog.md | 4 ++++ 4 files changed, 10 insertions(+), 2 deletions(-) diff --git a/CHANGELOG-Japanese.md b/CHANGELOG-Japanese.md index d4b501a2..1f2bc0d2 100644 --- a/CHANGELOG-Japanese.md +++ b/CHANGELOG-Japanese.md @@ -10,6 +10,7 @@ **バグ修正:** +- `configure`で全てのホストに`AuditNTLMInDomain=2`を設定していた問題を修正し、ドメインコントローラと確認できたホストにのみ`7` (Enable all)を設定するようにした。その他のホストや役割を判定できないホストでは、この設定を変更しない。ドメインNTLM監査設定を明示的に表示し、設定後の値の確認とレジストリエラーの報告にも対応した。 (#389) (@Shirofune-Security) - ルールのフィルタ条件が全て適用されず最後の条件のみが適用されていたため、ルール数が正確ではなかった。 (#358) (@fukusuket) - 依存するログが無効になっているルールも使用可能として報告されていた。 (#358) (@fukusuket) - 複数のカテゴリに属するルールが重複してカウントされ、CSVファイルにも重複して出力されていた。 (#358) (@fukusuket) @@ -53,4 +54,4 @@ - `audit-settings`: Windows Event Log audit policy settingsをチェックする - `audit-filesize`: Windows Event Logファイルサイズをチェックする -- `update-rules`: WELAのSigmaルール設定ファイルを更新する \ No newline at end of file +- `update-rules`: WELAのSigmaルール設定ファイルを更新する diff --git a/CHANGELOG.md b/CHANGELOG.md index 96f9ad77..6df0f36e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,7 @@ **Bug Fixes:** +- Fixed domain NTLM auditing: `configure` now sets `AuditNTLMInDomain=7` (Enable all) only on confirmed domain controllers, instead of writing `2` on every host. This setting is left unchanged on other hosts and hosts whose role cannot be determined. Audit output reports the domain NTLM setting, and configuration verifies registry writes and reports failures. (#389) (@Shirofune-Security) - Rule filtering applied only the last criterion instead of all of them, so rule counts were inaccurate. (#358) (@fukusuket) - Rules were reported as usable even when the logs they depend on were disabled. (#358) (@fukusuket) - Rules that belong to multiple categories were counted and written to the CSV files multiple times. (#358) (@fukusuket) @@ -55,4 +56,4 @@ - `audit-settings`: Check Windows Event Log audit policy settings. - `audit-filesize`: Check Windows Event Log file size. -- `update-rules`: Update WELA's Sigma rules config files. \ No newline at end of file +- `update-rules`: Update WELA's Sigma rules config files. diff --git a/website/docs/resources/changelog.ja.md b/website/docs/resources/changelog.ja.md index 161dfb31..15df5727 100644 --- a/website/docs/resources/changelog.ja.md +++ b/website/docs/resources/changelog.ja.md @@ -8,10 +8,12 @@ **改善:** - ベースライン定義を`WELA.ps1`から`config/baselines.json`に外部化し、ベースラインの追加・変更をJSONの編集のみで行えるようにした。 (#358) (@fukusuket) +- `Microsoft-Windows-DFSN-Server/Admin`チャネルを`audit-settings`と`audit-filesize`の確認対象に追加した。 (#358) (@fukusuket) - MITRE ATT&CK Navigatorのヒートマップを ATT&CK v19 に対応させ、ATT&CK側でrevokedとなった技術IDを置換先に書き換えるようにした(例: v19で`T1685`に統合された`T1562`と`T1562.001`)。Navigatorはrevokedのエントリを黙って破棄するため、従来はその分のカバレッジがヒートマップから欠落していた。 (@fukusuket) **バグ修正:** +- `configure`で全てのホストに`AuditNTLMInDomain=2`を設定していた問題を修正し、ドメインコントローラと確認できたホストにのみ`7` (Enable all)を設定するようにした。その他のホストや役割を判定できないホストでは、この設定を変更しない。ドメインNTLM監査設定を明示的に表示し、設定後の値の確認とレジストリエラーの報告にも対応した。 (#389) (@Shirofune-Security) - ルールのフィルタ条件が全て適用されず最後の条件のみが適用されていたため、ルール数が正確ではなかった。 (#358) (@fukusuket) - 依存するログが無効になっているルールも使用可能として報告されていた。 (#358) (@fukusuket) - 複数のカテゴリに属するルールが重複してカウントされ、CSVファイルにも重複して出力されていた。 (#358) (@fukusuket) diff --git a/website/docs/resources/changelog.md b/website/docs/resources/changelog.md index f6da9eca..747d2958 100644 --- a/website/docs/resources/changelog.md +++ b/website/docs/resources/changelog.md @@ -7,11 +7,15 @@ **Improvements:** +- Added a `configure-sacl` command that sets targeted audit SACLs on the autostart/persistence registry keys and sensitive files the detection rules watch, so File System (4663), Registry (4657) and Handle Manipulation (4656) auditing produce useful events without enabling global object auditing. It covers machine-wide objects plus per-user HKCU keys and profile AppData across all user profiles and the Default profile (so future users inherit the SACL). Targets live in `config/audit_sacl_targets.json`. (#361) (@YamatoSecurity) +- `configure` now also enables Detailed Tracking > Process Termination (4689), Object Access > Detailed File Share (5145), and (on domain controllers) LDAP query logging (Directory Service 1644 via NTDS `15 Field Engineering`), so a full detection baseline is applied without any manual `auditpol`/registry steps. (#361) (@YamatoSecurity) - Baseline definitions were moved out of `WELA.ps1` into a `config/baselines.json` config file, so adding or changing a baseline is now a JSON-only edit. (#358) (@fukusuket) +- The `Microsoft-Windows-DFSN-Server/Admin` channel is now checked by `audit-settings` and `audit-filesize`. (#358) (@fukusuket) - MITRE ATT&CK Navigator heatmaps are now generated for ATT&CK v19, and technique IDs that ATT&CK has revoked are rewritten to their replacements (for example `T1562` and `T1562.001`, which v19 folded into `T1685`). Navigator silently discards revoked entries, so that coverage used to disappear from the heatmap. (@fukusuket) **Bug Fixes:** +- Fixed domain NTLM auditing: `configure` now sets `AuditNTLMInDomain=7` (Enable all) only on confirmed domain controllers, instead of writing `2` on every host. This setting is left unchanged on other hosts and hosts whose role cannot be determined. Audit output reports the domain NTLM setting, and configuration verifies registry writes and reports failures. (#389) (@Shirofune-Security) - Rule filtering applied only the last criterion instead of all of them, so rule counts were inaccurate. (#358) (@fukusuket) - Rules were reported as usable even when the logs they depend on were disabled. (#358) (@fukusuket) - Rules that belong to multiple categories were counted and written to the CSV files multiple times. (#358) (@fukusuket) From bf69494bd9025bf4e922004711855eebac4970a4 Mon Sep 17 00:00:00 2001 From: Shirofune-Security <43838376+Shirofune-Security@users.noreply.github.com> Date: Fri, 18 Sep 2026 22:59:25 +0900 Subject: [PATCH 2/2] Report configuration-only audit states without rule coverage inference --- .github/workflows/test-domain-ntlm.yml | 6 +++ WELA.ps1 | 9 +++- tests/DomainNtlm.Tests.ps1 | 11 +++++ tests/DomainNtlmAuditOutput.Tests.ps1 | 65 ++++++++++++++++++++++++++ website/docs/commands/usage.md | 8 ++++ 5 files changed, 98 insertions(+), 1 deletion(-) create mode 100644 tests/DomainNtlmAuditOutput.Tests.ps1 diff --git a/.github/workflows/test-domain-ntlm.yml b/.github/workflows/test-domain-ntlm.yml index facc0e50..d94dfec0 100644 --- a/.github/workflows/test-domain-ntlm.yml +++ b/.github/workflows/test-domain-ntlm.yml @@ -17,3 +17,9 @@ jobs: - name: Test domain NTLM policy in PowerShell 7 shell: pwsh run: ./tests/DomainNtlm.Tests.ps1 + - name: Test domain NTLM audit output in Windows PowerShell 5.1 + shell: powershell + run: ./tests/DomainNtlmAuditOutput.Tests.ps1 + - name: Test domain NTLM audit output in PowerShell 7 + shell: pwsh + run: ./tests/DomainNtlmAuditOutput.Tests.ps1 diff --git a/WELA.ps1 b/WELA.ps1 index 9244b2c7..0dd71dff 100644 --- a/WELA.ps1 +++ b/WELA.ps1 @@ -485,7 +485,14 @@ function AuditLogSetting { $disabledCount = ($_.Group | Where-Object { $notEnabled -contains $_.CurrentSetting } | ForEach-Object { $_.Rules.Count } | Measure-Object -Sum).Sum $out = "" $color = "" - if (@($_.Group | Where-Object { $_.CurrentSetting -ne "Unknown" }).Count -eq 0) { + if (@($_.Group | Where-Object { $_.Rules.Count -gt 0 }).Count -eq 0) { + # Configuration-only rows have no rule coverage to aggregate. + # Preserve their observed state, including applicability and errors. + $out = ($_.Group | Select-Object -ExpandProperty CurrentSetting -Unique) -join '; ' + if (-not $out) { $out = 'Unknown' } + $color = 'DarkYellow' + } + elseif (@($_.Group | Where-Object { $_.CurrentSetting -ne "Unknown" }).Count -eq 0) { # 設定を確認できないカテゴリ。無効と断定はできない $out = "Unknown" $color = "DarkYellow" diff --git a/tests/DomainNtlm.Tests.ps1 b/tests/DomainNtlm.Tests.ps1 index fb57c41a..719d0278 100644 --- a/tests/DomainNtlm.Tests.ps1 +++ b/tests/DomainNtlm.Tests.ps1 @@ -102,6 +102,17 @@ Assert-Equal $script:writes 0 'Declining preserves policy' $script:response = '' Set-WelaDomainNtlmAudit Assert-Equal $script:value 7 'Confirming applies policy' +foreach ($confirmation in @('y', 'Y')) { + Reset-Policy 2 + $script:response = $confirmation + Set-WelaDomainNtlmAudit + Assert-Equal $script:value 7 "Confirmation '$confirmation' applies policy" + Assert-Equal $script:writes 1 "Confirmation '$confirmation' writes once" +} +Reset-Policy 2 +$script:response = 'N' +Set-WelaDomainNtlmAudit +Assert-Equal $script:writes 0 'Uppercase refusal preserves policy' Reset-Policy 2 $script:writeFails = $true Assert-Throws { Set-WelaDomainNtlmAudit -Auto } 'Write failure propagates' diff --git a/tests/DomainNtlmAuditOutput.Tests.ps1 b/tests/DomainNtlmAuditOutput.Tests.ps1 new file mode 100644 index 00000000..0b352fd5 --- /dev/null +++ b/tests/DomainNtlmAuditOutput.Tests.ps1 @@ -0,0 +1,65 @@ +# Exercise the real audit renderer/CSV exports with injected observations and rules. +# Only a temporary directory is written; no Windows policy is read or changed. +$ErrorActionPreference = 'Stop' +$tokens = $null; $parseErrors = $null +$ast = [System.Management.Automation.Language.Parser]::ParseFile((Join-Path $PSScriptRoot '../WELA.ps1'), [ref]$tokens, [ref]$parseErrors) +if ($parseErrors.Count) { throw ($parseErrors | Out-String) } +$class = $ast.Find({ param($node) $node -is [System.Management.Automation.Language.TypeDefinitionAst] -and $node.Name -eq 'WELA' }, $true) +. ([scriptblock]::Create($class.Extent.Text)) +$definition = $ast.Find({ param($node) $node -is [System.Management.Automation.Language.FunctionDefinitionAst] -and $node.Name -eq 'AuditLogSetting' }, $true) +. ([scriptblock]::Create($definition.Extent.Text)) + +$script:assertions = 0 +function Assert-Equal($Actual, $Expected, [string]$Message) { + if ($Actual -cne $Expected) { throw "$Message. Expected '$Expected', got '$Actual'." } + $script:assertions++ +} +function TestAdministrator { return $true } +function CollectAuditpol { param([switch]$UseCached) return $true } +function GetAuditpol { return @{} } +function Get-WelaDomainNtlmState { return [pscustomobject]@{ Description = $script:description } } +function Export-MitreHeatmap { param($sigmaRules, $OutputPath, $UseIdealCount) } +function BuildAuditResult { + param($all_rules, $Baseline, $enabledguid) + $all_rules[0].applicable = $true + @( + [WELA]::new('Fixture rules', 'Available', 'Success', @($all_rules[0])) + [WELA]::new('Fixture rules', 'Unavailable', 'No Auditing', @($all_rules[1])) + ) +} + +$script:ScriptRoot = Join-Path ([IO.Path]::GetTempPath()) ('wela-domain-output-' + [guid]::NewGuid().ToString('N')) +$null = New-Item -ItemType Directory -Path $script:ScriptRoot +$script:SecurityRulesPath = Join-Path $script:ScriptRoot 'rules.json' +try { + @( + @{ id = 'available-rule'; title = 'Available fixture'; level = 'high'; subcategory_guids = @() } + @{ id = 'unavailable-rule'; title = 'Unavailable fixture'; level = 'medium'; subcategory_guids = @() } + ) | ConvertTo-Json -Depth 4 | Set-Content -LiteralPath $script:SecurityRulesPath -Encoding UTF8 + foreach ($observed in @( + 'Enable all (7)', + 'Disabled (0)', + 'Not configured', + 'Value 2 (not interpreted as Enable all)', + 'Not applicable (Windows client)', + 'Not applicable (member or standalone server, including non-DC AD CS)', + 'Unknown (computer role could not be determined)', + 'Unknown (domain NTLM registry read failed: Access denied)' + )) { + $script:description = $observed + $output = (AuditLogSetting -outType std -Baseline YamatoSecurity 6>&1 | Out-String) + $expectedHeading = 'NTLM Authentication: ' + $observed + Assert-Equal ($output -match ('(?m)^' + [regex]::Escape($expectedHeading) + '\r?$')) $true "Console heading retains '$observed'" + Assert-Equal ($output -match 'NTLM Authentication: Partially Enabled') $false 'An empty rule array does not imply partial enablement' + Assert-Equal ($output -match 'Fixture rules: Partially Enabled') $true 'Ordinary rule coverage aggregation is preserved' + $row = @(Import-Csv -LiteralPath (Join-Path $script:ScriptRoot 'WELA-Audit-Result.csv') | Where-Object SubCategory -eq 'Domain NTLM auditing') + Assert-Equal $row.Count 1 'CSV contains one domain NTLM setting row' + Assert-Equal $row[0].CurrentSetting $observed 'CSV retains the observed configuration state' + Assert-Equal $row[0].RuleCount '0' 'Configuration row claims no detection rules' + Assert-Equal @(Import-Csv -LiteralPath (Join-Path $script:ScriptRoot 'UsableRules.csv')).Count 1 'Configuration row does not change usable rule counts' + Assert-Equal @(Import-Csv -LiteralPath (Join-Path $script:ScriptRoot 'UnusableRules.csv')).Count 1 'Configuration row does not change unusable rule counts' + } + Write-Host "PASS: $script:assertions domain NTLM output assertions (mocked observations; temporary CSV files only)." +} finally { + Remove-Item -LiteralPath $script:ScriptRoot -Recurse -Force +} diff --git a/website/docs/commands/usage.md b/website/docs/commands/usage.md index 5e273ab1..353b6545 100644 --- a/website/docs/commands/usage.md +++ b/website/docs/commands/usage.md @@ -55,6 +55,14 @@ an isolated DC before deployment. `tests/DomainNtlm.Tests.ps1` uses mocked OS an registry access; the associated Windows workflow runs Windows PowerShell 5.1 and PowerShell 7 without changing host policy. +Live-DC validation for [issue #363](https://github.com/Yamato-Security/WELA/issues/363) +remains pending. Before closing that issue, record the Windows build and confirmed +DC role, the previous registry value/type, the verified `AuditNTLMInDomain=7` +DWORD, and representative NTLM event XML from benign test authentication. Also +check that a second run is idempotent and that clients, member servers and non-DC CAs leave +this domain-only setting unchanged. Mocked policy tests and console/CSV regression +tests do not provide this event-generation evidence. + #### `configure` command examples Apply Yamato Security's recommended settings (with confirmation prompt before changing settings): ```