Merge main and preserve NTLM output regression coverage

.github/workflows/test-domain-ntlm.yml

@@ -17,3 +17,9 @@ jobs:
       - name: Test domain NTLM policy in PowerShell 7
         shell: pwsh
         run: ./tests/DomainNtlm.Tests.ps1
+      - name: Test domain NTLM audit output in Windows PowerShell 5.1
+        shell: powershell
+        run: ./tests/DomainNtlmAuditOutput.Tests.ps1
+      - name: Test domain NTLM audit output in PowerShell 7
+        shell: pwsh
+        run: ./tests/DomainNtlmAuditOutput.Tests.ps1

CHANGELOG-Japanese.md

@@ -13,6 +13,7 @@
 **バグ修正:**
 
 - 設定時に外部コマンドの終了コードと変更後の設定値を確認し、処理の終了前にも再確認するようにした。書き込み失敗、設定の未反映、CAサービスの再起動失敗、最終確認時の設定の不一致を明示的に報告し、一律に成功とせず、0以外の終了コードを返すようにした。 (#392) (@Shirofune-Security)
+- `configure`で全てのホストに`AuditNTLMInDomain=2`を設定していた問題を修正し、ドメインコントローラと確認できたホストにのみ`7` (Enable all)を設定するようにした。その他のホストや役割を判定できないホストでは、この設定を変更しない。ドメインNTLM監査設定を明示的に表示し、設定後の値の確認とレジストリエラーの報告にも対応した。 (#389) (@Shirofune-Security)
 - ルールのフィルタ条件が全て適用されず最後の条件のみが適用されていたため、ルール数が正確ではなかった。 (#358) (@fukusuket)
 - 依存するログが無効になっているルールも使用可能として報告されていた。 (#358) (@fukusuket)
 - 複数のカテゴリに属するルールが重複してカウントされ、CSVファイルにも重複して出力されていた。 (#358) (@fukusuket)

CHANGELOG.md

@@ -15,6 +15,7 @@
 **Bug Fixes:**
 
 - Configuration now checks native command exit codes, verifies settings after applying changes, and checks them again before finishing. Failed writes, ineffective changes, CA restart failures and settings that no longer match at the final check produce explicit results and a nonzero exit code instead of unconditional success. (#392) (@Shirofune-Security)
+- Fixed domain NTLM auditing: `configure` now sets `AuditNTLMInDomain=7` (Enable all) only on confirmed domain controllers, instead of writing `2` on every host. This setting is left unchanged on other hosts and hosts whose role cannot be determined. Audit output reports the domain NTLM setting, and configuration verifies registry writes and reports failures. (#389) (@Shirofune-Security)
 - Rule filtering applied only the last criterion instead of all of them, so rule counts were inaccurate. (#358) (@fukusuket)
 - Rules were reported as usable even when the logs they depend on were disabled. (#358) (@fukusuket)
 - Rules that belong to multiple categories were counted and written to the CSV files multiple times. (#358) (@fukusuket)

WELA.ps1

@@ -585,7 +585,14 @@ function AuditLogSetting {
             $disabledCount = ($_.Group |  Where-Object { $notEnabled -contains $_.CurrentSetting } | ForEach-Object { $_.Rules.Count } | Measure-Object -Sum).Sum
             $out = ""
             $color = ""
-            if (@($_.Group | Where-Object { $_.CurrentSetting -ne "Unknown" }).Count -eq 0) {
+            if (@($_.Group | Where-Object { $_.Rules.Count -gt 0 }).Count -eq 0) {
+                # Configuration-only rows have no rule coverage to aggregate.
+                # Preserve their observed state, including applicability and errors.
+                $out = ($_.Group | Select-Object -ExpandProperty CurrentSetting -Unique) -join '; '
+                if (-not $out) { $out = 'Unknown' }
+                $color = 'DarkYellow'
+            }
+            elseif (@($_.Group | Where-Object { $_.CurrentSetting -ne "Unknown" }).Count -eq 0) {
                 # 設定を確認できないカテゴリ。無効と断定はできない
                 $out = "Unknown"
                 $color = "DarkYellow"

tests/DomainNtlm.Tests.ps1

@@ -102,6 +102,17 @@ Assert-Equal $script:writes 0 'Declining preserves policy'
 $script:response = ''
 Set-WelaDomainNtlmAudit
 Assert-Equal $script:value 7 'Confirming applies policy'
+foreach ($confirmation in @('y', 'Y')) {
+    Reset-Policy 2
+    $script:response = $confirmation
+    Set-WelaDomainNtlmAudit
+    Assert-Equal $script:value 7 "Confirmation '$confirmation' applies policy"
+    Assert-Equal $script:writes 1 "Confirmation '$confirmation' writes once"
+}
+Reset-Policy 2
+$script:response = 'N'
+Set-WelaDomainNtlmAudit
+Assert-Equal $script:writes 0 'Uppercase refusal preserves policy'
 Reset-Policy 2
 $script:writeFails = $true
 Assert-Throws { Set-WelaDomainNtlmAudit -Auto } 'Write failure propagates'

tests/DomainNtlmAuditOutput.Tests.ps1

@@ -0,0 +1,72 @@
+# Exercise the real audit renderer/CSV exports with injected observations and rules.
+# Only a temporary directory is written; no Windows policy is read or changed.
+$ErrorActionPreference = 'Stop'
+$tokens = $null; $parseErrors = $null
+$ast = [System.Management.Automation.Language.Parser]::ParseFile((Join-Path $PSScriptRoot '../WELA.ps1'), [ref]$tokens, [ref]$parseErrors)
+if ($parseErrors.Count) { throw ($parseErrors | Out-String) }
+$class = $ast.Find({ param($node) $node -is [System.Management.Automation.Language.TypeDefinitionAst] -and $node.Name -eq 'WELA' }, $true)
+. ([scriptblock]::Create($class.Extent.Text))
+$definition = $ast.Find({ param($node) $node -is [System.Management.Automation.Language.FunctionDefinitionAst] -and $node.Name -eq 'AuditLogSetting' }, $true)
+. ([scriptblock]::Create($definition.Extent.Text))
+
+$script:assertions = 0
+function Assert-Equal($Actual, $Expected, [string]$Message) {
+    if ($Actual -cne $Expected) { throw "$Message. Expected '$Expected', got '$Actual'." }
+    $script:assertions++
+}
+function TestAdministrator { return $true }
+function CollectAuditpol { param([switch]$UseCached) return $true }
+function GetAuditpol { return @{} }
+function Get-WelaOutgoingNtlmState {
+    return [pscustomobject]@{ Description = 'Audit all (1)'; PolicySource = 'Mocked policy source' }
+}
+function Get-WelaDomainNtlmState { return [pscustomobject]@{ Description = $script:description } }
+function Export-MitreHeatmap { param($sigmaRules, $OutputPath, $UseIdealCount) }
+function BuildAuditResult {
+    param($all_rules, $Baseline, $enabledguid)
+    $all_rules[0].applicable = $true
+    @(
+        [WELA]::new('Fixture rules', 'Available', 'Success', @($all_rules[0]))
+        [WELA]::new('Fixture rules', 'Unavailable', 'No Auditing', @($all_rules[1]))
+    )
+}
+
+$script:ScriptRoot = Join-Path ([IO.Path]::GetTempPath()) ('wela-domain-output-' + [guid]::NewGuid().ToString('N'))
+$null = New-Item -ItemType Directory -Path $script:ScriptRoot
+$script:SecurityRulesPath = Join-Path $script:ScriptRoot 'rules.json'
+try {
+    @(
+        @{ id = 'available-rule'; title = 'Available fixture'; level = 'high'; subcategory_guids = @() }
+        @{ id = 'unavailable-rule'; title = 'Unavailable fixture'; level = 'medium'; subcategory_guids = @() }
+    ) | ConvertTo-Json -Depth 4 | Set-Content -LiteralPath $script:SecurityRulesPath -Encoding UTF8
+    foreach ($observed in @(
+        'Enable all (7)',
+        'Disabled (0)',
+        'Not configured',
+        'Value 2 (not interpreted as Enable all)',
+        'Not applicable (Windows client)',
+        'Not applicable (member or standalone server, including non-DC AD CS)',
+        'Unknown (computer role could not be determined)',
+        'Unknown (domain NTLM registry read failed: Access denied)'
+    )) {
+        $script:description = $observed
+        $output = (AuditLogSetting -outType std -Baseline YamatoSecurity 6>&1 | Out-String)
+        $expectedHeading = 'NTLM Authentication: Audit all (1); ' + $observed
+        Assert-Equal ($output -match ('(?m)^' + [regex]::Escape($expectedHeading) + '\r?$')) $true "Console heading retains '$observed'"
+        Assert-Equal ($output -match 'NTLM Authentication: Partially Enabled') $false 'An empty rule array does not imply partial enablement'
+        Assert-Equal ($output -match 'Fixture rules: Partially Enabled') $true 'Ordinary rule coverage aggregation is preserved'
+        $row = @(Import-Csv -LiteralPath (Join-Path $script:ScriptRoot 'WELA-Audit-Result.csv') | Where-Object SubCategory -eq 'Domain NTLM auditing')
+        Assert-Equal $row.Count 1 'CSV contains one domain NTLM setting row'
+        Assert-Equal $row[0].CurrentSetting $observed 'CSV retains the observed configuration state'
+        Assert-Equal $row[0].RuleCount '0' 'Configuration row claims no detection rules'
+        $outgoingRow = @(Import-Csv -LiteralPath (Join-Path $script:ScriptRoot 'WELA-Audit-Result.csv') | Where-Object SubCategory -eq 'Outgoing NTLM policy')
+        Assert-Equal $outgoingRow.Count 1 'CSV contains one outgoing NTLM setting row'
+        Assert-Equal $outgoingRow[0].CurrentSetting 'Audit all (1)' 'CSV retains the independent outgoing NTLM state'
+        Assert-Equal $outgoingRow[0].RuleCount '0' 'Outgoing configuration row claims no detection rules'
+        Assert-Equal @(Import-Csv -LiteralPath (Join-Path $script:ScriptRoot 'UsableRules.csv')).Count 1 'Configuration row does not change usable rule counts'
+        Assert-Equal @(Import-Csv -LiteralPath (Join-Path $script:ScriptRoot 'UnusableRules.csv')).Count 1 'Configuration row does not change unusable rule counts'
+    }
+    Write-Host "PASS: $script:assertions domain NTLM output assertions (mocked observations; temporary CSV files only)."
+} finally {
+    Remove-Item -LiteralPath $script:ScriptRoot -Recurse -Force
+}

website/docs/commands/usage.md

@@ -55,6 +55,14 @@ an isolated DC before deployment. `tests/DomainNtlm.Tests.ps1` uses mocked OS an
 registry access; the associated Windows workflow runs Windows PowerShell 5.1 and
 PowerShell 7 without changing host policy.
 
+Live-DC validation for [issue #363](https://github.com/Yamato-Security/WELA/issues/363)
+remains pending. Before closing that issue, record the Windows build and confirmed
+DC role, the previous registry value/type, the verified `AuditNTLMInDomain=7`
+DWORD, and representative NTLM event XML from benign test authentication. Also
+check that a second run is idempotent and that clients, member servers and non-DC CAs leave
+this domain-only setting unchanged. Mocked policy tests and console/CSV regression
+tests do not provide this event-generation evidence.
+
 #### `configure` command examples
 Apply Yamato Security's recommended settings (with confirmation prompt before changing settings):
 ```

website/docs/resources/changelog.ja.md

@@ -16,6 +16,7 @@
 **バグ修正:**
 
 - 設定時に外部コマンドの終了コードと変更後の設定値を確認し、処理の終了前にも再確認するようにした。書き込み失敗、設定の未反映、CAサービスの再起動失敗、最終確認時の設定の不一致を明示的に報告し、一律に成功とせず、0以外の終了コードを返すようにした。 (#392) (@Shirofune-Security)
+- `configure`で全てのホストに`AuditNTLMInDomain=2`を設定していた問題を修正し、ドメインコントローラと確認できたホストにのみ`7` (Enable all)を設定するようにした。その他のホストや役割を判定できないホストでは、この設定を変更しない。ドメインNTLM監査設定を明示的に表示し、設定後の値の確認とレジストリエラーの報告にも対応した。 (#389) (@Shirofune-Security)
 - ルールのフィルタ条件が全て適用されず最後の条件のみが適用されていたため、ルール数が正確ではなかった。 (#358) (@fukusuket)
 - 依存するログが無効になっているルールも使用可能として報告されていた。 (#358) (@fukusuket)
 - 複数のカテゴリに属するルールが重複してカウントされ、CSVファイルにも重複して出力されていた。 (#358) (@fukusuket)

website/docs/resources/changelog.md

@@ -18,6 +18,7 @@
 **Bug Fixes:**
 
 - Configuration now checks native command exit codes, verifies settings after applying changes, and checks them again before finishing. Failed writes, ineffective changes, CA restart failures and settings that no longer match at the final check produce explicit results and a nonzero exit code instead of unconditional success. (#392) (@Shirofune-Security)
+- Fixed domain NTLM auditing: `configure` now sets `AuditNTLMInDomain=7` (Enable all) only on confirmed domain controllers, instead of writing `2` on every host. This setting is left unchanged on other hosts and hosts whose role cannot be determined. Audit output reports the domain NTLM setting, and configuration verifies registry writes and reports failures. (#389) (@Shirofune-Security)
 - Rule filtering applied only the last criterion instead of all of them, so rule counts were inaccurate. (#358) (@fukusuket)
 - Rules were reported as usable even when the logs they depend on were disabled. (#358) (@fukusuket)
 - Rules that belong to multiple categories were counted and written to the CSV files multiple times. (#358) (@fukusuket)