<# .SYNOPSIS Continuous Secure Boot rollout orchestrator that runs until deployment is complete. .DESCRIPTION This script provides full end-to-end automation for Secure Boot certificate rollout: 1. Generates rollout waves based on aggregation data 2. Creates AD groups and GPO for each wave 3. Monitors for device updates (Event 1808) 4. Detects blocked buckets (unreachable devices) 5. Progresses to next wave automatically 6. Runs until ALL eligible devices are updated Completion Criteria: - No devices remaining in: Action Required, High Confidence, Observation, Temporarily Paused - Out of scope (by design): Not Supported, Secure Boot Disabled - Runs continuously until complete - no arbitrary wave limit Rollout Strategy: - HIGH CONFIDENCE: All devices in first wave (safe) - ACTION REQUIRED: Progressive doubles (1→2→4→8...) Blocking Logic: - After MaxWaitHours, orchestrator pings devices that haven't updated - If device is UNREACHABLE (ping fails) → bucket is BLOCKED for investigation - If device is REACHABLE but not updated → keep waiting (may need reboot) - Blocked buckets are excluded until admin unblocks them Auto-Unblocking: - If a device in a blocked bucket later shows as updated (Event 1808), the bucket is automatically unblocked and rollout proceeds - This handles devices that were temporarily offline but came back Device Tracking: - Tracks devices by hostname (assumes names don't change during rollout) - Note: JSON collection doesn't include a unique machine ID; add one for better tracking .PARAMETER AggregationInputPath Path to raw JSON device data (from Detect script) .PARAMETER ReportBasePath Base path for aggregation reports .PARAMETER TargetOU Distinguished Name of the OU to link GPOs. Optional - if not specified, GPO is linked to domain root for domain-wide coverage. Security group filtering ensures only targeted devices receive the policy. .PARAMETER MaxWaitHours Hours to wait for devices to update before checking reachability. After this time, devices that haven't updated are pinged. Unreachable devices cause the bucket to be blocked. Default: 72 (3 days) .PARAMETER PollIntervalMinutes Minutes between status checks. Default: 1440 (1 day) .PARAMETER AllowListPath Path to a file containing hostnames to ALLOW for rollout (targeted rollout). Supports .txt (one hostname per line) or .csv (with Hostname/ComputerName/Name column). When specified, ONLY these devices will be included in rollout. BlockList is still applied after AllowList. .PARAMETER AllowADGroup Name of an AD security group containing computer accounts to ALLOW. Example: "SecureBoot-Pilot-Computers" or "Wave1-Devices" When specified, ONLY devices in this group will be included in rollout. Combine with AllowListPath for both file and AD-based targeting. .PARAMETER ExclusionListPath Path to a file containing hostnames to EXCLUDE from rollout (VIP/executive devices). Supports .txt (one hostname per line) or .csv (with Hostname/ComputerName/Name column). These devices will never be included in any rollout wave. BlockList is applied AFTER AllowList filtering. .PARAMETER ExcludeADGroup Name of an AD security group containing computer accounts to exclude. Example: "VIP-Computers" or "Executive-Devices" Combine with ExclusionListPath for both file and AD-based exclusions. .PARAMETER UseWinCS Use WinCS (Windows Configuration System) instead of GPO/AvailableUpdatesPolicy. WinCS deploys Secure Boot enablement by running WinCsFlags.exe directly on each endpoint. WinCsFlags.exe runs under SYSTEM context via a scheduled task. This method is useful for: - Faster rollouts (immediate effect vs waiting for GPO processing) - Non-domain joined devices - Environments without AD/GPO infrastructure Reference: https://support.microsoft.com/en-us/topic/windows-configuration-system-wincs-apis-for-secure-boot-d3e64aa0-6095-4f8a-b8e4-fbfda254a8fe .PARAMETER WinCSKey The WinCS key to use for Secure Boot enablement. Default: F33E0C8E002 This key corresponds to the Secure Boot rollout configuration. .PARAMETER DryRun Show what would be done without making changes .PARAMETER ListBlockedBuckets Display all currently blocked buckets and exit .PARAMETER UnblockBucket Unblock a specific bucket by key and exit .PARAMETER UnblockAll Unblock all buckets and exit .PARAMETER EnableTaskOnDisabled Deploy Enable-SecureBootUpdateTask.ps1 to all devices with disabled scheduled task. Creates a GPO with a one-time scheduled task that runs the Enable script with -Quiet option. This is useful to fix devices that have the Secure-Boot-Update task disabled. .EXAMPLE .\Start-SecureBootRolloutOrchestrator.ps1 ` -AggregationInputPath "\\server\SecureBootLogs$\Json" ` -ReportBasePath "E:\SecureBootReports" ` -TargetOU "OU=Workstations,DC=contoso,DC=com" .EXAMPLE # List blocked buckets .\Start-SecureBootRolloutOrchestrator.ps1 -ReportBasePath "E:\SecureBootReports" -ListBlockedBuckets .EXAMPLE # Unblock a specific bucket .\Start-SecureBootRolloutOrchestrator.ps1 -ReportBasePath "E:\SecureBootReports" -UnblockBucket "Dell_Latitude5520_BIOS1.2.3" .EXAMPLE # Exclude VIP devices from rollout using a text file .\Start-SecureBootRolloutOrchestrator.ps1 ` -AggregationInputPath "\\server\SecureBootLogs$\Json" ` -ReportBasePath "E:\SecureBootReports" ` -ExclusionListPath "C:\Admin\VIP-Devices.txt" .EXAMPLE # Exclude devices in an AD security group (e.g., executive laptops) .\Start-SecureBootRolloutOrchestrator.ps1 ` -AggregationInputPath "\\server\SecureBootLogs$\Json" ` -ReportBasePath "E:\SecureBootReports" ` -ExcludeADGroup "VIP-Computers" .EXAMPLE # Use WinCS (Windows Configuration System) instead of GPO/AvailableUpdatesPolicy # WinCsFlags.exe runs under SYSTEM context on each endpoint via scheduled task .\Start-SecureBootRolloutOrchestrator.ps1 ` -AggregationInputPath "\\server\SecureBootLogs$\Json" ` -ReportBasePath "E:\SecureBootReports" ` -UseWinCS ` -WinCSKey "F33E0C8E002" #> [CmdletBinding()] param( [Parameter(Mandatory = $false)] [string]$AggregationInputPath, [Parameter(Mandatory = $false)] [string]$ReportBasePath, [Parameter(Mandatory = $false)] [string]$TargetOU, [Parameter(Mandatory = $false)] [string]$WavePrefix = "SecureBoot-Rollout", [Parameter(Mandatory = $false)] [int]$MaxWaitHours = 72, [Parameter(Mandatory = $false)] [int]$PollIntervalMinutes = 1440, [Parameter(Mandatory = $false)] [int]$ProcessingBatchSize = 5000, [Parameter(Mandatory = $false)] [int]$DeviceLogSampleSize = 25, [Parameter(Mandatory = $false)] [switch]$LargeScaleMode, # ============================================================================ # AllowList / BlockList Parameters # ============================================================================ # AllowList = Only include these devices (targeted rollout) # BlockList = Exclude these devices (they will never be rolled out) # Processing order: AllowList first (if specified), then BlockList [Parameter(Mandatory = $false)] [string]$AllowListPath, [Parameter(Mandatory = $false)] [string]$AllowADGroup, [Parameter(Mandatory = $false)] [string]$ExclusionListPath, [Parameter(Mandatory = $false)] [string]$ExcludeADGroup, # ============================================================================ # WinCS (Windows Configuration System) Parameters # ============================================================================ # WinCS is an alternative to AvailableUpdatesPolicy GPO deployment. # It uses WinCsFlags.exe on each endpoint to enable Secure Boot rollout. # WinCsFlags.exe runs under SYSTEM context on the endpoint. # Reference: https://support.microsoft.com/en-us/topic/windows-configuration-system-wincs-apis-for-secure-boot-d3e64aa0-6095-4f8a-b8e4-fbfda254a8fe [Parameter(Mandatory = $false)] [switch]$UseWinCS, [Parameter(Mandatory = $false)] [string]$WinCSKey = "F33E0C8E002", [Parameter(Mandatory = $false)] [switch]$DryRun, [Parameter(Mandatory = $false)] [switch]$ListBlockedBuckets, [Parameter(Mandatory = $false)] [string]$UnblockBucket, [Parameter(Mandatory = $false)] [switch]$UnblockAll, [Parameter(Mandatory = $false)] [switch]$EnableTaskOnDisabled ) $ErrorActionPreference = "Stop" $ScriptRoot = $PSScriptRoot $DownloadUrl = "https://aka.ms/getsecureboot" $DownloadSubPage = "Deployment and Monitoring Samples" # ============================================================================ # DEPENDENCY VALIDATION # ============================================================================ function Test-ScriptDependencies { param( [Parameter(Mandatory = $true)] [string]$ScriptDirectory, [Parameter(Mandatory = $true)] [string[]]$RequiredScripts ) $missingScripts = @() foreach ($script in $RequiredScripts) { $scriptPath = Join-Path $ScriptDirectory $script if (-not (Test-Path $scriptPath)) { $missingScripts += $script } } if ($missingScripts.Count -gt 0) { Write-Host "" Write-Host ("=" * 70) -ForegroundColor Red Write-Host " MISSING DEPENDENCIES" -ForegroundColor Red Write-Host ("=" * 70) -ForegroundColor Red Write-Host "" Write-Host "The following required scripts were not found:" -ForegroundColor Yellow foreach ($script in $missingScripts) { Write-Host " - $script" -ForegroundColor White } Write-Host "" Write-Host "Please download the latest scripts from:" -ForegroundColor Cyan Write-Host " URL: $DownloadUrl" -ForegroundColor White Write-Host " Navigate to: '$DownloadSubPage'" -ForegroundColor White Write-Host "" Write-Host "Extract all scripts to the same directory and run again." -ForegroundColor Yellow Write-Host "" return $false } return $true } # Required scripts for orchestrator $requiredScripts = @( "Aggregate-SecureBootData.ps1", "Enable-SecureBootUpdateTask.ps1", "Deploy-GPO-SecureBootCollection.ps1", "Detect-SecureBootCertUpdateStatus.ps1" ) if (-not (Test-ScriptDependencies -ScriptDirectory $PSScriptRoot -RequiredScripts $requiredScripts)) { exit 1 } # ============================================================================ # PARAMETER VALIDATION # ============================================================================ # Admin commands only need ReportBasePath $isAdminCommand = $ListBlockedBuckets -or $UnblockBucket -or $UnblockAll -or $EnableTaskOnDisabled if (-not $ReportBasePath) { Write-Host "ERROR: -ReportBasePath is required." -ForegroundColor Red exit 1 } if (-not $isAdminCommand -and -not $AggregationInputPath) { Write-Host "ERROR: -AggregationInputPath is required for rollout (not needed for -ListBlockedBuckets, -UnblockBucket, -UnblockAll)" -ForegroundColor Red exit 1 } # ============================================================================ # GPO DETECTION - CHECK FOR DETECTION GPO # ============================================================================ if (-not $isAdminCommand -and -not $DryRun) { $CollectionGPOName = "SecureBoot-EventCollection" # Check if GroupPolicy module is available if (Get-Module -ListAvailable -Name GroupPolicy) { Import-Module GroupPolicy -ErrorAction SilentlyContinue Write-Host "Checking for Detection GPO..." -ForegroundColor Yellow try { # Check if GPO exists $existingGpo = Get-GPO -Name $CollectionGPOName -ErrorAction SilentlyContinue if ($existingGpo) { Write-Host " Detection GPO found: $CollectionGPOName" -ForegroundColor Green } else { Write-Host "" Write-Host ("=" * 70) -ForegroundColor Yellow Write-Host " WARNING: DETECTION GPO NOT FOUND" -ForegroundColor Yellow Write-Host ("=" * 70) -ForegroundColor Yellow Write-Host "" Write-Host "The detection GPO '$CollectionGPOName' was not found." -ForegroundColor Yellow Write-Host "Without this GPO, no device data will be collected." -ForegroundColor Yellow Write-Host "" Write-Host "To deploy the Detection GPO, run:" -ForegroundColor Cyan Write-Host " .\Deploy-GPO-SecureBootCollection.ps1 -DomainName -AutoDetectOU" -ForegroundColor White Write-Host "" Write-Host "Continue anyway? (Y/N)" -ForegroundColor Yellow $response = Read-Host if ($response -notmatch '^[Yy]') { Write-Host "Aborting. Deploy the Detection GPO first." -ForegroundColor Red exit 1 } } } catch { Write-Host " Unable to check for GPO: $($_.Exception.Message)" -ForegroundColor Yellow } } else { Write-Host " GroupPolicy module not available - skipping GPO check" -ForegroundColor Gray } Write-Host "" } # ============================================================================ # STATE FILE PATHS # ============================================================================ $stateDir = Join-Path $ReportBasePath "RolloutState" if (-not (Test-Path $stateDir)) { New-Item -ItemType Directory -Path $stateDir -Force | Out-Null } $rolloutStatePath = Join-Path $stateDir "RolloutState.json" $blockedBucketsPath = Join-Path $stateDir "BlockedBuckets.json" $adminApprovedPath = Join-Path $stateDir "AdminApprovedBuckets.json" $deviceHistoryPath = Join-Path $stateDir "DeviceHistory.json" $processingCheckpointPath = Join-Path $stateDir "ProcessingCheckpoint.json" # ============================================================================ # PS 5.1 COMPATIBILITY: ConvertTo-Hashtable # ============================================================================ # ConvertFrom-Json -AsHashtable is PS7+ only. This provides compatibility. function ConvertTo-Hashtable { param( [Parameter(ValueFromPipeline = $true)] $InputObject ) process { if ($null -eq $InputObject) { return @{} } if ($InputObject -is [System.Collections.IDictionary]) { return $InputObject } if ($InputObject -is [PSCustomObject]) { # Use [ordered] for consistent key ordering and safe duplicate handling $hash = [ordered]@{} foreach ($prop in $InputObject.PSObject.Properties) { # Indexed assignment safely handles duplicates by overwriting $hash[$prop.Name] = ConvertTo-Hashtable $prop.Value } return $hash } if ($InputObject -is [System.Collections.IEnumerable] -and $InputObject -isnot [string]) { return @($InputObject | ForEach-Object { ConvertTo-Hashtable $_ }) } return $InputObject } } # ============================================================================ # ADMIN COMMANDS: List/Unblock Buckets # ============================================================================ if ($ListBlockedBuckets) { Write-Host "" Write-Host ("=" * 80) -ForegroundColor Yellow Write-Host " BLOCKED BUCKETS" -ForegroundColor Yellow Write-Host ("=" * 80) -ForegroundColor Yellow Write-Host "" if (Test-Path $blockedBucketsPath) { $blocked = Get-Content $blockedBucketsPath -Raw | ConvertFrom-Json | ConvertTo-Hashtable if ($blocked.Count -eq 0) { Write-Host "No blocked buckets." -ForegroundColor Green } else { Write-Host "Total blocked: $($blocked.Count)" -ForegroundColor Red Write-Host "" foreach ($key in $blocked.Keys) { $info = $blocked[$key] Write-Host "Bucket: $key" -ForegroundColor Red Write-Host " Blocked At: $($info.BlockedAt)" -ForegroundColor Gray Write-Host " Reason: $($info.Reason)" -ForegroundColor Gray Write-Host " Failed Device: $($info.FailedDevices)" -ForegroundColor Gray Write-Host " Last Reported: $($info.LastReported)" -ForegroundColor Gray Write-Host " Wave: $($info.WaveNumber)" -ForegroundColor Gray Write-Host " Devices in Bucket: $($info.DevicesInBucket)" -ForegroundColor Gray Write-Host "" } Write-Host "To unblock a bucket:" Write-Host " .\Start-SecureBootRolloutOrchestrator.ps1 -ReportBasePath '$ReportBasePath' -UnblockBucket 'BUCKET_KEY'" -ForegroundColor Cyan Write-Host "" Write-Host "To unblock all:" Write-Host " .\Start-SecureBootRolloutOrchestrator.ps1 -ReportBasePath '$ReportBasePath' -UnblockAll" -ForegroundColor Cyan } } else { Write-Host "No blocked buckets file found." -ForegroundColor Green } Write-Host "" exit 0 } if ($UnblockBucket) { Write-Host "" if (Test-Path $blockedBucketsPath) { $blocked = Get-Content $blockedBucketsPath -Raw | ConvertFrom-Json | ConvertTo-Hashtable if ($blocked.Contains($UnblockBucket)) { $blocked.Remove($UnblockBucket) $blocked | ConvertTo-Json -Depth 10 | Out-File $blockedBucketsPath -Encoding UTF8 -Force # Add to admin-approved list to prevent re-blocking $adminApproved = @{} if (Test-Path $adminApprovedPath) { $adminApproved = Get-Content $adminApprovedPath -Raw | ConvertFrom-Json | ConvertTo-Hashtable } $adminApproved[$UnblockBucket] = @{ ApprovedAt = Get-Date -Format "yyyy-MM-dd HH:mm:ss" ApprovedBy = $env:USERNAME } $adminApproved | ConvertTo-Json -Depth 10 | Out-File $adminApprovedPath -Encoding UTF8 -Force Write-Host "Unblocked bucket: $UnblockBucket" -ForegroundColor Green Write-Host "Added to admin-approved list (will not be re-blocked automatically)" -ForegroundColor Cyan } else { Write-Host "Bucket not found: $UnblockBucket" -ForegroundColor Yellow Write-Host "Available buckets:" -ForegroundColor Gray $blocked.Keys | ForEach-Object { Write-Host " $_" -ForegroundColor Gray } } } else { Write-Host "No blocked buckets file found." -ForegroundColor Yellow } Write-Host "" exit 0 } if ($UnblockAll) { Write-Host "" if (Test-Path $blockedBucketsPath) { $blocked = Get-Content $blockedBucketsPath -Raw | ConvertFrom-Json | ConvertTo-Hashtable $count = $blocked.Count @{} | ConvertTo-Json | Out-File $blockedBucketsPath -Encoding UTF8 -Force Write-Host "Unblocked all $count buckets." -ForegroundColor Green } else { Write-Host "No blocked buckets file found." -ForegroundColor Yellow } Write-Host "" exit 0 } # ============================================================================ # HELPER FUNCTIONS # ============================================================================ function Get-RolloutState { if (Test-Path $rolloutStatePath) { try { $loaded = Get-Content $rolloutStatePath -Raw | ConvertFrom-Json | ConvertTo-Hashtable # Validate required properties exist if ($null -eq $loaded.CurrentWave) { throw "Invalid state file - missing CurrentWave" } # Ensure WaveHistory is always an array (fixes PS5.1 JSON deserialization) if ($null -eq $loaded.WaveHistory) { $loaded.WaveHistory = @() } elseif ($loaded.WaveHistory -isnot [array]) { $loaded.WaveHistory = @($loaded.WaveHistory) } return $loaded } catch { Write-Log "Corrupted RolloutState.json detected: $($_.Exception.Message)" "WARN" Write-Log "Backing up corrupted file and starting fresh" "WARN" $backupPath = "$rolloutStatePath.corrupted.$(Get-Date -Format 'yyyyMMdd-HHmmss')" Move-Item $rolloutStatePath $backupPath -Force -ErrorAction SilentlyContinue } } return @{ CurrentWave = 0 StartedAt = $null LastAggregation = $null TotalDevicesTargeted = 0 TotalDevicesUpdated = 0 Status = "NotStarted" WaveHistory = @() } } function Save-RolloutState { param($State) $State | ConvertTo-Json -Depth 10 | Out-File $rolloutStatePath -Encoding UTF8 -Force } function Get-WeekdayProjection { <# .SYNOPSIS Calculate projected completion date accounting for weekends (no progress on Sat/Sun) #> param( [int]$RemainingDevices, [double]$DevicesPerDay, [datetime]$StartDate = (Get-Date) ) if ($DevicesPerDay -le 0 -or $RemainingDevices -le 0) { return @{ ProjectedDate = $null WorkingDaysNeeded = 0 CalendarDaysNeeded = 0 } } # Calculate working days needed (excluding weekends) $workingDaysNeeded = [math]::Ceiling($RemainingDevices / $DevicesPerDay) # Convert working days to calendar days (add weekends) $currentDate = $StartDate.Date $daysAdded = 0 $workingDaysAdded = 0 while ($workingDaysAdded -lt $workingDaysNeeded) { $currentDate = $currentDate.AddDays(1) $daysAdded++ # Only count weekdays if ($currentDate.DayOfWeek -ne [DayOfWeek]::Saturday -and $currentDate.DayOfWeek -ne [DayOfWeek]::Sunday) { $workingDaysAdded++ } } return @{ ProjectedDate = $currentDate.ToString("yyyy-MM-dd") WorkingDaysNeeded = $workingDaysNeeded CalendarDaysNeeded = $daysAdded } } function Save-RolloutSummary { <# .SYNOPSIS Save rollout summary with projection information for dashboard display #> param( [hashtable]$State, [int]$TotalDevices, [int]$UpdatedDevices, [int]$NotUpdatedDevices, [double]$DevicesPerDay ) $summaryPath = Join-Path $stateDir "SecureBootRolloutSummary.json" # Calculate weekend-aware projection $projection = Get-WeekdayProjection -RemainingDevices $NotUpdatedDevices -DevicesPerDay $DevicesPerDay $summary = @{ GeneratedAt = (Get-Date -Format "yyyy-MM-dd HH:mm:ss") RolloutStartDate = $State.StartedAt LastAggregation = $State.LastAggregation CurrentWave = $State.CurrentWave Status = $State.Status # Device counts TotalDevices = $TotalDevices UpdatedDevices = $UpdatedDevices NotUpdatedDevices = $NotUpdatedDevices PercentUpdated = if ($TotalDevices -gt 0) { [math]::Round(($UpdatedDevices / $TotalDevices) * 100, 1) } else { 0 } # Velocity metrics DevicesPerDay = [math]::Round($DevicesPerDay, 1) TotalDevicesTargeted = $State.TotalDevicesTargeted TotalWaves = $State.CurrentWave # Weekend-aware projection ProjectedCompletionDate = $projection.ProjectedDate WorkingDaysRemaining = $projection.WorkingDaysNeeded CalendarDaysRemaining = $projection.CalendarDaysNeeded # Note about weekend exclusion ProjectionNote = "Projected completion excludes weekends (Sat/Sun)" } $summary | ConvertTo-Json -Depth 5 | Out-File $summaryPath -Encoding UTF8 -Force Write-Log "Rollout summary saved: $summaryPath" "INFO" return $summary } function Get-BlockedBuckets { if (Test-Path $blockedBucketsPath) { return Get-Content $blockedBucketsPath -Raw | ConvertFrom-Json | ConvertTo-Hashtable } return @{} } function Save-BlockedBuckets { param($Blocked) $Blocked | ConvertTo-Json -Depth 10 | Out-File $blockedBucketsPath -Encoding UTF8 -Force } function Get-AdminApproved { if (Test-Path $adminApprovedPath) { return Get-Content $adminApprovedPath -Raw | ConvertFrom-Json | ConvertTo-Hashtable } return @{} } function Get-DeviceHistory { if (Test-Path $deviceHistoryPath) { return Get-Content $deviceHistoryPath -Raw | ConvertFrom-Json | ConvertTo-Hashtable } return @{} } function Save-DeviceHistory { param($History) $History | ConvertTo-Json -Depth 10 | Out-File $deviceHistoryPath -Encoding UTF8 -Force } function Save-ProcessingCheckpoint { param( [string]$Stage, [int]$Processed, [int]$Total, [hashtable]$Metrics = @{} ) $checkpoint = @{ Stage = $Stage UpdatedAt = Get-Date -Format "yyyy-MM-dd HH:mm:ss" Processed = $Processed Total = $Total Percent = if ($Total -gt 0) { [math]::Round(($Processed / $Total) * 100, 2) } else { 0 } Metrics = $Metrics } $checkpoint | ConvertTo-Json -Depth 6 | Out-File $processingCheckpointPath -Encoding UTF8 -Force } function Get-NotUpdatedIndexes { param([array]$Devices) $hostSet = [System.Collections.Generic.HashSet[string]]::new([StringComparer]::OrdinalIgnoreCase) $bucketCounts = @{} foreach ($device in $Devices) { $hostname = if ($device.Hostname) { $device.Hostname } elseif ($device.HostName) { $device.HostName } else { $null } if ($hostname) { [void]$hostSet.Add($hostname) } $bucketKey = Get-BucketKey $device if ($bucketKey) { if ($bucketCounts.ContainsKey($bucketKey)) { $bucketCounts[$bucketKey]++ } else { $bucketCounts[$bucketKey] = 1 } } } return @{ HostSet = $hostSet BucketCounts = $bucketCounts } } function Write-Log { param([string]$Message, [string]$Level = "INFO") $timestamp = Get-Date -Format "yyyy-MM-dd HH:mm:ss" $color = switch ($Level) { "OK" { "Green" } "WARN" { "Yellow" } "ERROR" { "Red" } "BLOCKED" { "DarkRed" } "WAVE" { "Cyan" } default { "White" } } Write-Host "[$timestamp] [$Level] $Message" -ForegroundColor $color # Also log to file $logFile = Join-Path $stateDir "Orchestrator_$(Get-Date -Format 'yyyyMMdd').log" "[$timestamp] [$Level] $Message" | Out-File $logFile -Append -Encoding UTF8 } function Get-BucketKey { param($Device) # Use BucketId from device JSON if available (SHA256 hash from detection script) if ($Device.BucketId -and "$($Device.BucketId)" -ne "") { return "$($Device.BucketId)" } # Fallback: construct from manufacturer|model|bios $mfr = if ($Device.WMI_Manufacturer) { $Device.WMI_Manufacturer } else { $Device.Manufacturer } $model = if ($Device.WMI_Model) { $Device.WMI_Model } else { $Device.Model } $bios = if ($Device.BIOSDescription) { $Device.BIOSDescription } else { $Device.BIOS } return "$mfr|$model|$bios" } # ============================================================================ # VIP/EXCLUSION LIST LOADING # ============================================================================ function Get-ExcludedHostnames { param( [string]$ExclusionFilePath, [string]$ADGroupName ) $excluded = [System.Collections.Generic.HashSet[string]]::new([StringComparer]::OrdinalIgnoreCase) # Load from file (supports .txt or .csv) if ($ExclusionFilePath -and (Test-Path $ExclusionFilePath)) { $extension = [System.IO.Path]::GetExtension($ExclusionFilePath).ToLower() if ($extension -eq ".csv") { # CSV format: expects a 'Hostname' or 'ComputerName' column $csvData = Import-Csv $ExclusionFilePath $hostCol = if ($csvData[0].PSObject.Properties.Name -contains 'Hostname') { 'Hostname' } elseif ($csvData[0].PSObject.Properties.Name -contains 'ComputerName') { 'ComputerName' } elseif ($csvData[0].PSObject.Properties.Name -contains 'Name') { 'Name' } else { $null } if ($hostCol) { foreach ($row in $csvData) { if (![string]::IsNullOrWhiteSpace($row.$hostCol)) { [void]$excluded.Add($row.$hostCol.Trim()) } } } } else { # Plain text: one hostname per line Get-Content $ExclusionFilePath | ForEach-Object { $line = $_.Trim() if ($line -and -not $line.StartsWith('#')) { [void]$excluded.Add($line) } } } Write-Log "Loaded $($excluded.Count) hostnames from exclusion file: $ExclusionFilePath" "INFO" } # Load from AD security group if ($ADGroupName) { try { $groupMembers = Get-ADGroupMember -Identity $ADGroupName -Recursive -ErrorAction Stop | Where-Object { $_.objectClass -eq 'computer' } foreach ($member in $groupMembers) { [void]$excluded.Add($member.Name) } Write-Log "Loaded $($groupMembers.Count) computers from AD group: $ADGroupName" "INFO" } catch { Write-Log "Could not load AD group '$ADGroupName': $_" "WARN" } } return @($excluded) } # ============================================================================ # ALLOW LIST LOADING (Targeted Rollout) # ============================================================================ function Get-AllowedHostnames { <# .SYNOPSIS Loads hostnames from an AllowList file and/or AD group for targeted rollout. When an AllowList is specified, ONLY these devices will be included in rollout. #> param( [string]$AllowFilePath, [string]$ADGroupName ) $allowed = [System.Collections.Generic.HashSet[string]]::new([StringComparer]::OrdinalIgnoreCase) # Load from file (supports .txt or .csv) if ($AllowFilePath -and (Test-Path $AllowFilePath)) { $extension = [System.IO.Path]::GetExtension($AllowFilePath).ToLower() if ($extension -eq ".csv") { # CSV format: expects a 'Hostname' or 'ComputerName' column $csvData = Import-Csv $AllowFilePath if ($csvData.Count -gt 0) { $hostCol = if ($csvData[0].PSObject.Properties.Name -contains 'Hostname') { 'Hostname' } elseif ($csvData[0].PSObject.Properties.Name -contains 'ComputerName') { 'ComputerName' } elseif ($csvData[0].PSObject.Properties.Name -contains 'Name') { 'Name' } else { $null } if ($hostCol) { foreach ($row in $csvData) { if (![string]::IsNullOrWhiteSpace($row.$hostCol)) { [void]$allowed.Add($row.$hostCol.Trim()) } } } } } else { # Plain text: one hostname per line Get-Content $AllowFilePath | ForEach-Object { $line = $_.Trim() if ($line -and -not $line.StartsWith('#')) { [void]$allowed.Add($line) } } } Write-Log "Loaded $($allowed.Count) hostnames from allow list file: $AllowFilePath" "INFO" } # Load from AD security group if ($ADGroupName) { try { $groupMembers = Get-ADGroupMember -Identity $ADGroupName -Recursive -ErrorAction Stop | Where-Object { $_.objectClass -eq 'computer' } foreach ($member in $groupMembers) { [void]$allowed.Add($member.Name) } Write-Log "Loaded $($groupMembers.Count) computers from AD allow group: $ADGroupName" "INFO" } catch { Write-Log "Could not load AD group '$ADGroupName': $_" "WARN" } } return @($allowed) } # ============================================================================ # DATA FRESHNESS AND MONITORING # ============================================================================ function Get-DataFreshness { <# .SYNOPSIS Checks how fresh the detection data is by examining JSON file timestamps. Returns statistics on when endpoints last reported. #> param([string]$JsonPath) $jsonFiles = Get-ChildItem -Path $JsonPath -Filter "*.json" -ErrorAction SilentlyContinue if ($jsonFiles.Count -eq 0) { return @{ TotalFiles = 0 FreshFiles = 0 StaleFiles = 0 NoDataFiles = 0 OldestFile = $null NewestFile = $null AvgAgeHours = 0 Warning = "No JSON files found - detection may not be deployed" } } $now = Get-Date $freshThresholdHours = 24 # Files updated in last 24 hours are "fresh" $staleThresholdHours = 72 # Files older than 72 hours are "stale" $fresh = 0 $stale = 0 $ages = @() foreach ($file in $jsonFiles) { $ageHours = ($now - $file.LastWriteTime).TotalHours $ages += $ageHours if ($ageHours -le $freshThresholdHours) { $fresh++ } elseif ($ageHours -ge $staleThresholdHours) { $stale++ } } $oldestFile = $jsonFiles | Sort-Object LastWriteTime | Select-Object -First 1 $newestFile = $jsonFiles | Sort-Object LastWriteTime -Descending | Select-Object -First 1 $warning = $null if ($stale -gt ($jsonFiles.Count * 0.5)) { $warning = "More than 50% of devices have stale data (>72 hours) - check detection GPO" } elseif ($fresh -lt ($jsonFiles.Count * 0.3)) { $warning = "Less than 30% of devices reported recently - detection may not be running" } return @{ TotalFiles = $jsonFiles.Count FreshFiles = $fresh StaleFiles = $stale MediumFiles = $jsonFiles.Count - $fresh - $stale OldestFile = $oldestFile.LastWriteTime NewestFile = $newestFile.LastWriteTime AvgAgeHours = [math]::Round(($ages | Measure-Object -Average).Average, 1) Warning = $warning } } function Test-DetectionGPODeployed { <# .SYNOPSIS Verifies that the detection/monitoring infrastructure is in place. #> param([string]$JsonPath) # Check 1: JSON path exists if (-not (Test-Path $JsonPath)) { return @{ IsDeployed = $false Message = "JSON input path does not exist: $JsonPath" } } # Check 2: At least some JSON files exist $jsonCount = (Get-ChildItem -Path $JsonPath -Filter "*.json" -ErrorAction SilentlyContinue).Count if ($jsonCount -eq 0) { return @{ IsDeployed = $false Message = "No JSON files in $JsonPath - Detection GPO may not be deployed or devices haven't reported yet" } } # Check 3: Files are reasonably recent (at least some in last week) $recentFiles = Get-ChildItem -Path $JsonPath -Filter "*.json" -ErrorAction SilentlyContinue | Where-Object { $_.LastWriteTime -gt (Get-Date).AddDays(-7) } if ($recentFiles.Count -eq 0) { return @{ IsDeployed = $false Message = "No JSON files updated in last 7 days - Detection GPO may be broken or devices offline" } } return @{ IsDeployed = $true Message = "Detection appears active: $jsonCount files, $($recentFiles.Count) updated recently" FileCount = $jsonCount RecentCount = $recentFiles.Count } } # ============================================================================ # DEVICE TRACKING (BY HOSTNAME) # ============================================================================ function Update-DeviceHistory { <# .SYNOPSIS Tracks devices by hostname since we don't have a unique machine identifier. Note: BucketId is one-to-many (same hardware config = same bucket). If a unique identifier is added to JSON collection, update this function. #> param( [array]$CurrentDevices, [hashtable]$DeviceHistory ) foreach ($device in $CurrentDevices) { $hostname = $device.Hostname if (-not $hostname) { continue } # Track device by hostname $DeviceHistory[$hostname] = @{ Hostname = $hostname BucketId = $device.BucketId Manufacturer = $device.WMI_Manufacturer Model = $device.WMI_Model LastSeen = Get-Date -Format "yyyy-MM-dd HH:mm:ss" Status = $device.UpdateStatus } } } # ============================================================================ # BLOCKED BUCKET DETECTION (Based on Device Reachability) # ============================================================================ <# .DESCRIPTION Blocking Logic: - A bucket is ONLY blocked if: 1. Device was targeted in a wave 2. MaxWaitHours has passed since wave started 3. Device is NOT REACHABLE (ping fails) - If device IS reachable but not yet updated, we keep waiting (update may be pending reboot - Event 1808 only fires after reboot) - Unreachable device indicates something went wrong and needs investigation Unblocking: - Use -ListBlockedBuckets to see blocked buckets - Use -UnblockBucket "BucketKey" to unblock specific bucket - Use -UnblockAll to unblock all buckets #> function Test-DeviceReachable { param( [string]$Hostname, [string]$DataPath # Path to device JSON files ) # Method 1: Check JSON file timestamp (fastest — no file parsing needed) # If the detection script ran recently, the file was written/updated, proving the device is alive if ($DataPath) { $deviceFile = Get-ChildItem -Path $DataPath -Filter "${Hostname}*" -File -ErrorAction SilentlyContinue | Select-Object -First 1 if ($deviceFile) { $hoursSinceWrite = ((Get-Date) - $deviceFile.LastWriteTime).TotalHours if ($hoursSinceWrite -lt 72) { return $true } } } # Method 2: Fallback to ping (only if JSON is stale or missing) try { $ping = Test-Connection -ComputerName $Hostname -Count 1 -Quiet -ErrorAction SilentlyContinue return $ping } catch { return $false } } function Update-BlockedBuckets { param( $RolloutState, $BlockedBuckets, $AdminApproved, [array]$NotUpdatedDevices, [hashtable]$NotUpdatedIndexes, [int]$MaxWaitHours, [bool]$DryRun = $false ) $now = Get-Date $newlyBlocked = @() $stillWaiting = @() $devicesToCheck = @() $hostSet = if ($NotUpdatedIndexes -and $NotUpdatedIndexes.HostSet) { $NotUpdatedIndexes.HostSet } else { (Get-NotUpdatedIndexes -Devices $NotUpdatedDevices).HostSet } $bucketCounts = if ($NotUpdatedIndexes -and $NotUpdatedIndexes.BucketCounts) { $NotUpdatedIndexes.BucketCounts } else { (Get-NotUpdatedIndexes -Devices $NotUpdatedDevices).BucketCounts } # Collect devices that are past the wait period and still not updated foreach ($wave in $RolloutState.WaveHistory) { if (-not $wave.StartedAt) { continue } $waveStart = [DateTime]::Parse($wave.StartedAt) $hoursSinceWave = ($now - $waveStart).TotalHours if ($hoursSinceWave -lt $MaxWaitHours) { # Still within wait period - don't check yet continue } # Check each device from this wave foreach ($deviceInfo in $wave.Devices) { $hostname = $deviceInfo.Hostname $bucketKey = $deviceInfo.BucketKey # Skip if bucket already blocked if ($BlockedBuckets.Contains($bucketKey)) { continue } # Skip if bucket is admin-approved AND wave started BEFORE approval # (only check devices targeted AFTER admin approval for re-blocking) if ($AdminApproved -and $AdminApproved.Contains($bucketKey)) { $approvalTime = [DateTime]::Parse($AdminApproved[$bucketKey].ApprovedAt) if ($waveStart -lt $approvalTime) { # This device was targeted before admin approval - skip continue } # Wave started after approval - this is fresh targeting, can check } # Is this device still in NotUpdated list? if ($hostSet.Contains($hostname)) { $devicesToCheck += @{ Hostname = $hostname BucketKey = $bucketKey WaveNumber = $wave.WaveNumber HoursSinceWave = [math]::Round($hoursSinceWave, 1) } } } } if ($devicesToCheck.Count -eq 0) { return $newlyBlocked } Write-Log "Checking reachability of $($devicesToCheck.Count) devices past wait period..." "INFO" # Track failures per bucket for decision-making $bucketFailures = @{} # BucketKey -> @{ Unreachable=@(); Alive=@() } # Check reachability of each device foreach ($device in $devicesToCheck) { $hostname = $device.Hostname $bucketKey = $device.BucketKey if ($DryRun) { Write-Log "[DRYRUN] Would check $hostname reachability" "INFO" continue } if (-not $bucketFailures.ContainsKey($bucketKey)) { $bucketFailures[$bucketKey] = @{ Unreachable = @(); AliveButFailed = @(); WaveNumber = $device.WaveNumber; HoursSinceWave = $device.HoursSinceWave } } $isReachable = Test-DeviceReachable -Hostname $hostname -DataPath $AggregationInputPath if (-not $isReachable) { $bucketFailures[$bucketKey].Unreachable += $hostname } else { # Device IS reachable but not yet updated - could be temporary failure or waiting for reboot $bucketFailures[$bucketKey].AliveButFailed += $hostname $stillWaiting += $hostname } } # Decision per bucket: only block if devices are truly UNREACHABLE # Alive devices with failures = temporary, continue rollout foreach ($bucketKey in $bucketFailures.Keys) { $bf = $bucketFailures[$bucketKey] $unreachableCount = $bf.Unreachable.Count $aliveFailedCount = $bf.AliveButFailed.Count # Check if this bucket has any successes (from updated devices data) $bucketHasSuccesses = $stSuccessBuckets -and $stSuccessBuckets.Contains($bucketKey) if ($unreachableCount -gt 0 -and $aliveFailedCount -eq 0) { # ALL failing devices are unreachable - block the bucket if ($newlyBlocked -notcontains $bucketKey) { $BlockedBuckets[$bucketKey] = @{ BlockedAt = Get-Date -Format "yyyy-MM-dd HH:mm:ss" Reason = "All $unreachableCount device(s) unreachable after $($bf.HoursSinceWave) hours" FailedDevices = ($bf.Unreachable -join ", ") WaveNumber = $bf.WaveNumber DevicesInBucket = if ($bucketCounts.ContainsKey($bucketKey)) { $bucketCounts[$bucketKey] } else { 0 } } $newlyBlocked += $bucketKey Write-Log "BUCKET BLOCKED: $bucketKey ($unreachableCount device(s) unreachable: $($bf.Unreachable -join ', '))" "BLOCKED" } } elseif ($aliveFailedCount -gt 0) { # Devices are alive but not updated - temporary failure, DO NOT block Write-Log "Bucket $($bucketKey.Substring(0, [Math]::Min(16, $bucketKey.Length)))...: $aliveFailedCount device(s) alive but pending, $unreachableCount unreachable - NOT blocking (temporary)" "INFO" if ($unreachableCount -gt 0) { Write-Log " Unreachable: $($bf.Unreachable -join ', ')" "WARN" } Write-Log " Alive but pending: $($bf.AliveButFailed -join ', ')" "INFO" # Track failure count in rollout state for monitoring if (-not $RolloutState.TemporaryFailures) { $RolloutState.TemporaryFailures = @{} } $RolloutState.TemporaryFailures[$bucketKey] = @{ AliveButFailed = $bf.AliveButFailed Unreachable = $bf.Unreachable LastChecked = Get-Date -Format "yyyy-MM-dd HH:mm:ss" } } } if ($stillWaiting.Count -gt 0) { Write-Log "Devices reachable but pending update (may need reboot): $($stillWaiting.Count)" "INFO" } return $newlyBlocked } # ============================================================================ # AUTO-UNBLOCK: Unblock buckets when devices update successfully # ============================================================================ function Update-AutoUnblockedBuckets { <# .DESCRIPTION Checks if devices in blocked buckets have updated (Event 1808). Auto-unblocks if ALL targeted devices in the bucket have updated. If only SOME devices updated, notifies admin who can manually unblock. Admin can manually unblock using: .\Start-SecureBootRolloutOrchestrator.ps1 -ReportBasePath "path" -UnblockBucket "BucketKey" #> param( $BlockedBuckets, $RolloutState, [array]$NotUpdatedDevices, [string]$ReportBasePath, [hashtable]$NotUpdatedIndexes, [int]$LogSampleSize = 25 ) $autoUnblocked = @() $bucketsToCheck = @($BlockedBuckets.Keys) $hostSet = if ($NotUpdatedIndexes -and $NotUpdatedIndexes.HostSet) { $NotUpdatedIndexes.HostSet } else { (Get-NotUpdatedIndexes -Devices $NotUpdatedDevices).HostSet } foreach ($bucketKey in $bucketsToCheck) { $bucketInfo = $BlockedBuckets[$bucketKey] # Get all devices we targeted from this bucket historically $targetedDevicesInBucket = @() foreach ($wave in $RolloutState.WaveHistory) { $targetedDevicesInBucket += @($wave.Devices | Where-Object { $_.BucketKey -eq $bucketKey }) } if ($targetedDevicesInBucket.Count -eq 0) { continue } # Check how many targeted devices are still in NotUpdated vs updated $updatedDevices = @() $stillPendingDevices = @() foreach ($targetedDevice in $targetedDevicesInBucket) { if ($hostSet.Contains($targetedDevice.Hostname)) { $stillPendingDevices += $targetedDevice.Hostname } else { $updatedDevices += $targetedDevice.Hostname } } if ($updatedDevices.Count -gt 0 -and $stillPendingDevices.Count -eq 0) { # ALL targeted devices have updated - auto-unblock! $BlockedBuckets.Remove($bucketKey) $autoUnblocked += @{ BucketKey = $bucketKey UpdatedDevices = $updatedDevices PreviouslyBlockedAt = $bucketInfo.BlockedAt Reason = "All $($updatedDevices.Count) targeted device(s) successfully updated" } Write-Log "AUTO-UNBLOCKED: $bucketKey (All $($updatedDevices.Count) targeted device(s) updated successfully)" "OK" # Increment OEM wave count for this bucket's OEM (per-OEM tracking) $bucketOEM = if ($bucketKey -match '\|') { ($bucketKey -split '\|')[0] } else { 'Unknown' } # Extract OEM from pipe-delimited key or default if (-not $RolloutState.OEMWaveCounts) { $RolloutState.OEMWaveCounts = @{} } $currentWave = if ($RolloutState.OEMWaveCounts[$bucketOEM]) { $RolloutState.OEMWaveCounts[$bucketOEM] } else { 0 } $RolloutState.OEMWaveCounts[$bucketOEM] = $currentWave + 1 Write-Log " OEM '$bucketOEM' wave count incremented to $($currentWave + 1) (next allocation: $([int][Math]::Pow(2, $currentWave + 1)) devices)" "INFO" } elseif ($updatedDevices.Count -gt 0 -and $stillPendingDevices.Count -gt 0) { # SOME devices updated but others are still pending - notify admin (only once) if (-not $bucketInfo.UnblockCandidate) { $bucketInfo.UnblockCandidate = $true $bucketInfo.UpdatedDevices = $updatedDevices $bucketInfo.PendingDevices = $stillPendingDevices $bucketInfo.NotifiedAt = (Get-Date).ToString("yyyy-MM-dd HH:mm:ss") Write-Log "" "INFO" Write-Log "========== PARTIAL UPDATE IN BLOCKED BUCKET ==========" "INFO" Write-Log "Bucket: $bucketKey" "INFO" $updatedSample = @($updatedDevices | Select-Object -First $LogSampleSize) $pendingSample = @($stillPendingDevices | Select-Object -First $LogSampleSize) $updatedSuffix = if ($updatedDevices.Count -gt $LogSampleSize) { " ... (+$($updatedDevices.Count - $LogSampleSize) more)" } else { "" } $pendingSuffix = if ($stillPendingDevices.Count -gt $LogSampleSize) { " ... (+$($stillPendingDevices.Count - $LogSampleSize) more)" } else { "" } Write-Log "Updated devices ($($updatedDevices.Count)): $($updatedSample -join ', ')$updatedSuffix" "OK" Write-Log "Still pending ($($stillPendingDevices.Count)): $($pendingSample -join ', ')$pendingSuffix" "WARN" Write-Log "" "INFO" Write-Log "To manually unblock this bucket after verification, run:" "INFO" Write-Log " .\Start-SecureBootRolloutOrchestrator.ps1 -ReportBasePath `"$ReportBasePath`" -UnblockBucket `"$bucketKey`"" "INFO" Write-Log "=======================================================" "INFO" Write-Log "" "INFO" } } } return $autoUnblocked } # ============================================================================ # WAVE GENERATION (INLINED - excludes blocked buckets) # ============================================================================ function New-RolloutWave { param( [string]$AggregationPath, $BlockedBuckets, $RolloutState, [int]$MaxDevicesPerWave = 50, [string[]]$AllowedHostnames = @(), [string[]]$ExcludedHostnames = @() ) # Load aggregation data $notUptodateCsv = Get-ChildItem -Path $AggregationPath -Filter "*NotUptodate*.csv" | Where-Object { $_.Name -notlike "*Buckets*" } | Sort-Object LastWriteTime -Descending | Select-Object -First 1 if (-not $notUptodateCsv) { Write-Log "No NotUptodate CSV found" "ERROR" return $null } $allNotUpdated = @(Import-Csv $notUptodateCsv.FullName) # Normalize HostName -> Hostname for consistency (CSV uses HostName, code uses Hostname) foreach ($device in $allNotUpdated) { if ($device.PSObject.Properties['HostName'] -and -not $device.PSObject.Properties['Hostname']) { $device | Add-Member -NotePropertyName 'Hostname' -NotePropertyValue $device.HostName -Force } } # Filter out blocked buckets $eligibleDevices = @($allNotUpdated | Where-Object { $bucketKey = Get-BucketKey $_ -not $BlockedBuckets.Contains($bucketKey) }) # Filter to ONLY allowed devices (if AllowList is specified) # AllowList = targeted rollout - only these devices will be considered if ($AllowedHostnames.Count -gt 0) { $beforeCount = $eligibleDevices.Count $eligibleDevices = @($eligibleDevices | Where-Object { $_.Hostname -in $AllowedHostnames }) $allowedCount = $eligibleDevices.Count Write-Log "AllowList applied: $allowedCount of $beforeCount devices are in allow list" "INFO" } # Filter out VIP/excluded devices (BlockList) # BlockList is applied AFTER AllowList if ($ExcludedHostnames.Count -gt 0) { $beforeCount = $eligibleDevices.Count $eligibleDevices = @($eligibleDevices | Where-Object { $_.Hostname -notin $ExcludedHostnames }) $excludedCount = $beforeCount - $eligibleDevices.Count if ($excludedCount -gt 0) { Write-Log "Excluded $excludedCount VIP/protected devices from rollout" "INFO" } } if ($eligibleDevices.Count -eq 0) { Write-Log "No eligible devices remaining (all updated or blocked)" "OK" return $null } # Get devices already in rollout (from previous waves) $devicesAlreadyInRollout = @() if ($RolloutState.WaveHistory -and $RolloutState.WaveHistory.Count -gt 0) { $devicesAlreadyInRollout = @($RolloutState.WaveHistory | ForEach-Object { $_.Devices | ForEach-Object { $_.Hostname } } | Where-Object { $_ }) } Write-Log "Devices already in rollout: $($devicesAlreadyInRollout.Count)" "INFO" # Separate by confidence level $highConfidenceDevices = @($eligibleDevices | Where-Object { $_.ConfidenceLevel -eq "High Confidence" -and $_.Hostname -notin $devicesAlreadyInRollout }) # Action Required includes: # - Explicit "Action Required" # - Empty/null ConfidenceLevel # - ANY unknown/unrecognized ConfidenceLevel value (treated as Action Required) $knownSafeCategories = @( "High Confidence", "Temporarily Paused", "Under Observation", "Under Observation - More Data Needed", "Not Supported", "Not Supported - Known Limitation" ) $actionRequiredDevices = @($eligibleDevices | Where-Object { $_.ConfidenceLevel -notin $knownSafeCategories -and $_.Hostname -notin $devicesAlreadyInRollout }) Write-Log "High Confidence (not in rollout): $($highConfidenceDevices.Count)" "INFO" Write-Log "Action Required (not in rollout): $($actionRequiredDevices.Count)" "INFO" # Build wave devices $waveDevices = @() # HIGH CONFIDENCE: Include ALL (safe for rollout) if ($highConfidenceDevices.Count -gt 0) { Write-Log "Adding all $($highConfidenceDevices.Count) High Confidence devices" "WAVE" $waveDevices += $highConfidenceDevices } # ACTION REQUIRED: Progressive rollout (bucket-based with OEM-spread for zero-success buckets) # Strategy: # - Buckets with 0 successes: Spread across OEMs (1 per OEM -> 2 per OEM -> 4 per OEM) # - Buckets with ≥1 success: Double freely without OEM restriction if ($actionRequiredDevices.Count -gt 0) { # Load bucket success counts from updated devices CSV (devices that successfully updated) $updatedCsv = Get-ChildItem -Path $AggregationPath -Filter "*updated_devices*.csv" | Sort-Object LastWriteTime -Descending | Select-Object -First 1 $bucketStats = @{} if ($updatedCsv) { $updatedDevices = Import-Csv $updatedCsv.FullName # Count successes per BucketId $updatedDevices | ForEach-Object { $key = Get-BucketKey $_ if ($key) { if (-not $bucketStats.ContainsKey($key)) { $bucketStats[$key] = @{ Successes = 0; Pending = 0; Total = 0 } } $bucketStats[$key].Successes++ $bucketStats[$key].Total++ } } Write-Log "Loaded $($updatedDevices.Count) updated devices across $($bucketStats.Count) buckets" "INFO" } else { # Fallback: try ActionRequired_Buckets CSV $bucketsCsv = Get-ChildItem -Path $AggregationPath -Filter "*ActionRequired_Buckets*.csv" | Sort-Object LastWriteTime -Descending | Select-Object -First 1 if ($bucketsCsv) { Import-Csv $bucketsCsv.FullName | ForEach-Object { $key = if ($_.BucketId) { $_.BucketId } else { "$($_.Manufacturer)|$($_.Model)|$($_.BIOS)" } $bucketStats[$key] = @{ Successes = [int]$_.Successes Pending = [int]$_.Pending Total = [int]$_.TotalDevices } } } } # Group NotUpdated devices by bucket (Manufacturer|Model|BIOS) $buckets = $actionRequiredDevices | Group-Object { Get-BucketKey $_ } # Separate buckets: zero-success vs has-success $zeroSuccessBuckets = @() $hasSuccessBuckets = @() foreach ($bucket in $buckets) { $bucketKey = $bucket.Name $bucketDevices = @($bucket.Group) $bucketHostnames = @($bucketDevices | ForEach-Object { $_.Hostname }) # Count successes in this bucket $stats = $bucketStats[$bucketKey] $successes = if ($stats) { $stats.Successes } else { 0 } # Find devices deployed to this bucket from wave history $deployedToBucket = @() foreach ($wave in $RolloutState.WaveHistory) { foreach ($device in $wave.Devices) { if ($device.BucketKey -eq $bucketKey -and $device.Hostname) { $deployedToBucket += $device.Hostname } } } $deployedToBucket = @($deployedToBucket | Sort-Object -Unique) # Check if ALL deployed devices reported success $stillPending = @($deployedToBucket | Where-Object { $_ -in $bucketHostnames }) $confirmedSuccess = $deployedToBucket.Count - $stillPending.Count # If pending, skip this bucket until all confirm if ($stillPending.Count -gt 0) { $parts = $bucketKey -split '\|' $displayName = "$($parts[0]) - $($parts[1])" Write-Log " Bucket: $displayName - Deployed=$($deployedToBucket.Count), Confirmed=$confirmedSuccess, Pending=$($stillPending.Count) (waiting)" "INFO" continue } # Remaining eligible = devices not yet deployed $devicesNotYetTargeted = @($bucketDevices | Where-Object { $_.Hostname -notin $deployedToBucket }) if ($devicesNotYetTargeted.Count -eq 0) { continue } # Categorize by success count $bucketInfo = @{ BucketKey = $bucketKey Devices = $devicesNotYetTargeted ConfirmedSuccess = $confirmedSuccess Successes = $successes OEM = if ($bucket.Group[0].WMI_Manufacturer) { $bucket.Group[0].WMI_Manufacturer } elseif ($bucketKey -match '\|') { ($bucketKey -split '\|')[0] } else { 'Unknown' } } if ($successes -eq 0) { $zeroSuccessBuckets += $bucketInfo } else { $hasSuccessBuckets += $bucketInfo } } # === PROCESS HAS-SUCCESS BUCKETS (≥1 success) === # Double the number of successes — if 14 succeeded, deploy 28 next foreach ($bucketInfo in $hasSuccessBuckets) { $nextBatchSize = $bucketInfo.Successes * 2 $nextBatchSize = [Math]::Min($nextBatchSize, $MaxDevicesPerWave) $nextBatchSize = [Math]::Min($nextBatchSize, $bucketInfo.Devices.Count) if ($nextBatchSize -gt 0) { $selectedDevices = @($bucketInfo.Devices | Select-Object -First $nextBatchSize) $waveDevices += $selectedDevices $parts = if ($bucketInfo.BucketKey -match '\|') { $bucketInfo.BucketKey -split '\|' } else { @($bucketInfo.OEM, $bucketInfo.BucketKey.Substring(0, [Math]::Min(12, $bucketInfo.BucketKey.Length))) } $displayName = "$($parts[0]) - $($parts[1])" Write-Log " [HAS-SUCCESS] $displayName - Successes=$($bucketInfo.Successes), Deploying=$nextBatchSize (2x confirmed)" "INFO" } } # === PROCESS ZERO-SUCCESS BUCKETS (spread across OEMs with per-OEM tracking) === # Goal: Spread risk across different OEMs, track progress per OEM independently # Each OEM progresses based on its own success history: # - OEM with successes: Gets more devices next wave (2^waveCount) # - OEM without successes: Stays at current level until success confirmed if ($zeroSuccessBuckets.Count -gt 0) { # Initialize per-OEM wave counts if not exists if (-not $RolloutState.OEMWaveCounts) { $RolloutState.OEMWaveCounts = @{} } # Group zero-success buckets by OEM $oemBuckets = $zeroSuccessBuckets | Group-Object { $_.OEM } $totalZeroSuccessAdded = 0 $oemsDeployedTo = @() foreach ($oemGroup in $oemBuckets) { $oemName = $oemGroup.Name # Get this OEM's wave count (starts at 0) $oemWaveCount = if ($RolloutState.OEMWaveCounts[$oemName]) { $RolloutState.OEMWaveCounts[$oemName] } else { 0 } # Calculate devices for THIS OEM: 2^waveCount (1, 2, 4, 8...) $devicesForThisOEM = [int][Math]::Pow(2, $oemWaveCount) $devicesForThisOEM = [Math]::Max(1, $devicesForThisOEM) $oemDevicesAdded = 0 # Pick from each bucket under this OEM foreach ($bucketInfo in $oemGroup.Group) { $remaining = $devicesForThisOEM - $oemDevicesAdded if ($remaining -le 0) { break } $toTake = [Math]::Min($remaining, $bucketInfo.Devices.Count) if ($toTake -gt 0) { $selectedDevices = @($bucketInfo.Devices | Select-Object -First $toTake) $waveDevices += $selectedDevices $oemDevicesAdded += $toTake $totalZeroSuccessAdded += $toTake $parts = if ($bucketInfo.BucketKey -match '\|') { $bucketInfo.BucketKey -split '\|' } else { @($bucketInfo.OEM, $bucketInfo.BucketKey.Substring(0, [Math]::Min(12, $bucketInfo.BucketKey.Length))) } $displayName = "$($parts[0]) - $($parts[1])" Write-Log " [ZERO-SUCCESS] $displayName - Deploying=$toTake (OEM wave $oemWaveCount = ${devicesForThisOEM}/OEM)" "WARN" } } if ($oemDevicesAdded -gt 0) { Write-Log " OEM: $oemName - Wave $oemWaveCount, Added $oemDevicesAdded devices" "INFO" $oemsDeployedTo += $oemName } } # Track which OEMs we deployed to (for incrementing on next success check) if ($oemsDeployedTo.Count -gt 0) { $RolloutState.PendingOEMWaveIncrement = $oemsDeployedTo Write-Log "Zero-success deployment: $totalZeroSuccessAdded devices across $($oemsDeployedTo.Count) OEMs" "INFO" } } } if (@($waveDevices).Count -eq 0) { return $null } return $waveDevices } # ============================================================================ # GPO DEPLOYMENT (INLINED - creates GPO, security group, links) # ============================================================================ function Deploy-GPOForWave { param( [string]$GPOName, [string]$TargetOU, [string]$SecurityGroupName, [array]$WaveHostnames, [bool]$DryRun = $false ) # ADMX Policy: SecureBoot.admx - SecureBoot_AvailableUpdatesPolicy # Registry Path: HKLM\SYSTEM\CurrentControlSet\Control\SecureBoot # Value Name: AvailableUpdatesPolicy # Enabled Value: 22852 (0x5944) - Update all Secure Boot keys + bootmgr # Disabled Value: 0 # # Using Group Policy Preferences (GPP) for reliable HKLM\SYSTEM path deployment # GPP creates settings under: Computer Configuration > Preferences > Windows Settings > Registry $RegistryKey = "HKLM\SYSTEM\CurrentControlSet\Control\SecureBoot" $RegistryValueName = "AvailableUpdatesPolicy" $RegistryValue = 22852 # 0x5944 - matches ADMX enabledValue Write-Log "Deploying GPO: $GPOName" "WAVE" Write-Log "Registry: $RegistryKey\$RegistryValueName = $RegistryValue (0x$($RegistryValue.ToString('X')))" "INFO" if ($DryRun) { Write-Log "[DRYRUN] Would create GPO: $GPOName" "INFO" Write-Log "[DRYRUN] Would create security group: $SecurityGroupName" "INFO" Write-Log "[DRYRUN] Would add $(@($WaveHostnames).Count) computers to group" "INFO" Write-Log "[DRYRUN] Would link GPO to: $TargetOU" "INFO" return $true } try { # Import required modules Import-Module GroupPolicy -ErrorAction Stop Import-Module ActiveDirectory -ErrorAction Stop } catch { Write-Log "Failed to import required modules (GroupPolicy, ActiveDirectory): $($_.Exception.Message)" "ERROR" return $false } # Step 1: Create or get GPO $existingGPO = Get-GPO -Name $GPOName -ErrorAction SilentlyContinue if ($existingGPO) { Write-Log "GPO already exists: $GPOName" "INFO" $gpo = $existingGPO } else { try { $gpo = New-GPO -Name $GPOName -Comment "Secure Boot Certificate Rollout - AvailableUpdatesPolicy=0x5944 - Created $(Get-Date -Format 'yyyy-MM-dd HH:mm')" Write-Log "Created GPO: $GPOName" "OK" } catch { Write-Log "Failed to create GPO: $($_.Exception.Message)" "ERROR" return $false } } # Step 2: Set registry value using Group Policy Preferences (GPP) # GPP is more reliable for HKLM\SYSTEM paths than Set-GPRegistryValue try { # First try to remove any existing preference for this value (to avoid duplicates) Remove-GPPrefRegistryValue -Name $GPOName -Context Computer -Key $RegistryKey -ValueName $RegistryValueName -ErrorAction SilentlyContinue # Create GPP registry preference with "Replace" action # Replace = Create if not exists, Update if exists (most reliable) # Update = Only update if exists (fails if value doesn't exist) Set-GPPrefRegistryValue -Name $GPOName ` -Context Computer ` -Action Replace ` -Key $RegistryKey ` -ValueName $RegistryValueName ` -Type DWord ` -Value $RegistryValue Write-Log "Configured GPP registry preference: $RegistryValueName = 0x5944 (Action=Replace)" "OK" } catch { Write-Log "GPP failed, trying Set-GPRegistryValue: $($_.Exception.Message)" "WARN" # Fallback to Set-GPRegistryValue (works if ADMX is deployed) try { Set-GPRegistryValue -Name $GPOName ` -Key $RegistryKey ` -ValueName $RegistryValueName ` -Type DWord ` -Value $RegistryValue Write-Log "Configured registry via Set-GPRegistryValue: $RegistryValueName = 0x5944" "OK" } catch { Write-Log "Failed to set registry value: $($_.Exception.Message)" "ERROR" return $false } } # Step 3: Create or get security group $existingGroup = Get-ADGroup -Filter "Name -eq '$SecurityGroupName'" -ErrorAction SilentlyContinue if (-not $existingGroup) { try { $group = New-ADGroup -Name $SecurityGroupName ` -GroupCategory Security ` -GroupScope DomainLocal ` -Description "Computers targeted for Secure Boot rollout - $GPOName" ` -PassThru Write-Log "Created security group: $SecurityGroupName" "OK" } catch { Write-Log "Failed to create security group: $($_.Exception.Message)" "ERROR" return $false } } else { Write-Log "Security group exists: $SecurityGroupName" "INFO" $group = $existingGroup } # Step 4: Add computers to security group $added = 0 $failed = 0 foreach ($hostname in $WaveHostnames) { try { $computer = Get-ADComputer -Identity $hostname -ErrorAction Stop Add-ADGroupMember -Identity $SecurityGroupName -Members $computer -ErrorAction SilentlyContinue $added++ } catch { $failed++ } } Write-Log "Added $added computers to security group ($failed not found in AD)" "OK" # Step 5: Configure security filtering on GPO try { # Remove default "Authenticated Users" Apply permission (keep Read) Set-GPPermission -Name $GPOName -TargetName "Authenticated Users" -TargetType Group -PermissionLevel GpoRead -Replace -ErrorAction SilentlyContinue # Add Apply permission for our security group Set-GPPermission -Name $GPOName -TargetName $SecurityGroupName -TargetType Group -PermissionLevel GpoApply -ErrorAction Stop Write-Log "Configured security filtering for: $SecurityGroupName" "OK" } catch { Write-Log "Failed to configure security filtering: $($_.Exception.Message)" "WARN" Write-Log "GPO may apply to all computers in the linked OU - verify manually" "WARN" } # Step 6: Link GPO to OU (CRITICAL for policy to apply) if ($TargetOU) { try { $existingLink = Get-GPInheritance -Target $TargetOU -ErrorAction SilentlyContinue | Select-Object -ExpandProperty GpoLinks | Where-Object { $_.DisplayName -eq $GPOName } if (-not $existingLink) { New-GPLink -Name $GPOName -Target $TargetOU -LinkEnabled Yes -ErrorAction Stop Write-Log "Linked GPO to: $TargetOU" "OK" Write-Log "GPO will apply at next gpupdate on target computers" "INFO" } else { Write-Log "GPO already linked to target OU" "INFO" } } catch { Write-Log "CRITICAL: Failed to link GPO to OU: $($_.Exception.Message)" "ERROR" Write-Log "GPO was created but NOT LINKED - it will NOT apply to any computers!" "ERROR" Write-Log "Manual fix required: New-GPLink -Name '$GPOName' -Target '$TargetOU' -LinkEnabled Yes" "ERROR" return $false } } else { Write-Log "WARNING: No TargetOU specified - GPO created but NOT LINKED!" "ERROR" Write-Log "Manual linking required for GPO to take effect" "ERROR" Write-Log "Run: New-GPLink -Name '$GPOName' -Target '' -LinkEnabled Yes" "ERROR" } # Step 7: Verify GPO configuration Write-Log "Verifying GPO configuration..." "INFO" try { $gpoReport = Get-GPO -Name $GPOName -ErrorAction Stop Write-Log "GPO Status: $($gpoReport.GpoStatus)" "INFO" # Check if registry setting is configured $regSettings = Get-GPRegistryValue -Name $GPOName -Key "HKLM\SYSTEM\CurrentControlSet\Control\SecureBoot" -ErrorAction SilentlyContinue if (-not $regSettings) { # Try GPP registry check (different path in GPO) Write-Log "Checking GPP registry preferences..." "INFO" } } catch { Write-Log "Could not verify GPO: $($_.Exception.Message)" "WARN" } return $true } # ============================================================================ # WINCS DEPLOYMENT (Alternative to AvailableUpdatesPolicy GPO) # ============================================================================ # Reference: https://support.microsoft.com/en-us/topic/windows-configuration-system-wincs-apis-for-secure-boot-d3e64aa0-6095-4f8a-b8e4-fbfda254a8fe # # WinCS Commands (run on endpoint under SYSTEM context): # Query: WinCsFlags.exe /query --key F33E0C8E002 # Apply: WinCsFlags.exe /apply --key "F33E0C8E002" # Reset: WinCsFlags.exe /reset --key "F33E0C8E002" # # This method deploys a GPO with a scheduled task that runs WinCsFlags.exe /apply # as SYSTEM on targeted endpoints. Similar to how detection script is deployed, # but runs once (at startup) instead of daily. function Deploy-WinCSGPOForWave { <# .SYNOPSIS Deploy WinCS Secure Boot enablement via GPO scheduled task. .DESCRIPTION Creates a GPO that deploys a scheduled task to run WinCsFlags.exe /apply under SYSTEM context at machine startup. Security group controls targeting. .PARAMETER GPOName Name for the GPO. .PARAMETER TargetOU OU to link the GPO to. .PARAMETER SecurityGroupName Security group for GPO filtering. .PARAMETER WaveHostnames Hostnames to add to the security group. .PARAMETER WinCSKey The WinCS key to apply (default: F33E0C8E002). .PARAMETER DryRun If true, only log what would be done. #> param( [Parameter(Mandatory = $true)] [string]$GPOName, [Parameter(Mandatory = $false)] [string]$TargetOU, [Parameter(Mandatory = $true)] [string]$SecurityGroupName, [Parameter(Mandatory = $true)] [array]$WaveHostnames, [Parameter(Mandatory = $false)] [string]$WinCSKey = "F33E0C8E002", [Parameter(Mandatory = $false)] [bool]$DryRun = $false ) # Scheduled Task configuration for WinCsFlags.exe $TaskName = "SecureBoot-WinCS-Apply" $TaskPath = "\Microsoft\Windows\SecureBoot\" $TaskDescription = "Applies Secure Boot configuration via WinCS - Key: $WinCSKey" Write-Log "Deploying WinCS GPO: $GPOName" "WAVE" Write-Log "Task will run: WinCsFlags.exe /apply --key `"$WinCSKey`"" "INFO" Write-Log "Trigger: At system startup (runs once as SYSTEM)" "INFO" if ($DryRun) { Write-Log "[DRYRUN] Would create GPO: $GPOName" "INFO" Write-Log "[DRYRUN] Would create security group: $SecurityGroupName" "INFO" Write-Log "[DRYRUN] Would add $(@($WaveHostnames).Count) computers to group" "INFO" Write-Log "[DRYRUN] Would deploy scheduled task: $TaskName" "INFO" Write-Log "[DRYRUN] Would link GPO to: $TargetOU" "INFO" return @{ Success = $true GPOCreated = $false GroupCreated = $false ComputersAdded = 0 } } try { # Import required modules Import-Module GroupPolicy -ErrorAction Stop Import-Module ActiveDirectory -ErrorAction Stop } catch { Write-Log "Failed to import required modules (GroupPolicy, ActiveDirectory): $($_.Exception.Message)" "ERROR" return @{ Success = $false; Error = $_.Exception.Message } } # Step 1: Create or get GPO $gpo = Get-GPO -Name $GPOName -ErrorAction SilentlyContinue if ($gpo) { Write-Log "GPO already exists: $GPOName" "INFO" } else { try { $gpo = New-GPO -Name $GPOName -Comment "Secure Boot WinCS Deployment - $WinCSKey - Created $(Get-Date -Format 'yyyy-MM-dd HH:mm')" Write-Log "Created GPO: $GPOName" "OK" } catch { Write-Log "Failed to create GPO: $($_.Exception.Message)" "ERROR" return @{ Success = $false; Error = $_.Exception.Message } } } # Step 2: Create scheduled task XML for GPO deployment # This creates a task that runs WinCsFlags.exe /apply at startup $taskXml = @" $TaskDescription SYSTEM true PT5M S-1-5-18 HighestAvailable IgnoreNew false false true true false false false true true false false false true false PT1H P30D 7 WinCsFlags.exe /apply --key "$WinCSKey" "@ # Step 3: Deploy scheduled task via GPO Preferences # Store task XML in SYSVOL for GPO Scheduled Tasks Immediate Task try { $gpoId = $gpo.Id.ToString() $sysvolPath = "\\$((Get-ADDomain).DNSRoot)\SYSVOL\$((Get-ADDomain).DNSRoot)\Policies\{$gpoId}\Machine\Preferences\ScheduledTasks" if (-not (Test-Path $sysvolPath)) { New-Item -ItemType Directory -Path $sysvolPath -Force | Out-Null } # Create ScheduledTasks.xml for GPP $gppTaskXml = @" $TaskDescription NT AUTHORITY\System S4U HighestAvailable PT5M PT1H false false IgnoreNew false false true true true true false PT1H 7 PT0S $(Get-Date -Format 'yyyy-MM-dd')T00:00:00 true WinCsFlags.exe /apply --key "$WinCSKey" "@ $gppTaskXml | Out-File -FilePath (Join-Path $sysvolPath "ScheduledTasks.xml") -Encoding UTF8 -Force Write-Log "Deployed scheduled task to GPO: $TaskName" "OK" } catch { Write-Log "Failed to deploy scheduled task XML: $($_.Exception.Message)" "WARN" Write-Log "Falling back to registry-based WinCS deployment" "INFO" # Fallback: Use WinCS registry approach if GPP scheduled task fails # WinCS can also be triggered via registry key # (Implementation depends on WinCS registry API if available) } # Step 4: Create or get security group $group = Get-ADGroup -Filter "Name -eq '$SecurityGroupName'" -ErrorAction SilentlyContinue if (-not $group) { try { $group = New-ADGroup -Name $SecurityGroupName ` -GroupCategory Security ` -GroupScope DomainLocal ` -Description "Computers targeted for Secure Boot WinCS rollout - $GPOName" ` -PassThru Write-Log "Created security group: $SecurityGroupName" "OK" } catch { Write-Log "Failed to create security group: $($_.Exception.Message)" "ERROR" return @{ Success = $false; Error = $_.Exception.Message } } } else { Write-Log "Security group exists: $SecurityGroupName" "INFO" } # Step 5: Add computers to security group $added = 0 $failed = 0 foreach ($hostname in $WaveHostnames) { try { $computer = Get-ADComputer -Identity $hostname -ErrorAction Stop Add-ADGroupMember -Identity $SecurityGroupName -Members $computer -ErrorAction SilentlyContinue $added++ } catch { $failed++ } } Write-Log "Added $added computers to security group ($failed not found in AD)" "OK" # Step 6: Configure security filtering on GPO try { Set-GPPermission -Name $GPOName -TargetName "Authenticated Users" -TargetType Group -PermissionLevel GpoRead -Replace -ErrorAction SilentlyContinue Set-GPPermission -Name $GPOName -TargetName $SecurityGroupName -TargetType Group -PermissionLevel GpoApply -ErrorAction Stop Write-Log "Configured security filtering for: $SecurityGroupName" "OK" } catch { Write-Log "Failed to configure security filtering: $($_.Exception.Message)" "WARN" } # Step 7: Link GPO to OU if ($TargetOU) { try { $existingLink = Get-GPInheritance -Target $TargetOU -ErrorAction SilentlyContinue | Select-Object -ExpandProperty GpoLinks | Where-Object { $_.DisplayName -eq $GPOName } if (-not $existingLink) { New-GPLink -Name $GPOName -Target $TargetOU -LinkEnabled Yes -ErrorAction Stop Write-Log "Linked GPO to: $TargetOU" "OK" } else { Write-Log "GPO already linked to target OU" "INFO" } } catch { Write-Log "CRITICAL: Failed to link GPO to OU: $($_.Exception.Message)" "ERROR" return @{ Success = $false; Error = "GPO link failed: $($_.Exception.Message)" } } } Write-Log "WinCS GPO deployment complete" "OK" Write-Log "Machines will run WinCsFlags.exe at next GPO refresh + reboot/startup" "INFO" return @{ Success = $true GPOCreated = $true GroupCreated = $true ComputersAdded = $added ComputersFailed = $failed } } # Wrapper function to maintain compatibility with main loop function Deploy-WinCSForWave { param( [Parameter(Mandatory = $true)] [array]$WaveHostnames, [Parameter(Mandatory = $false)] [string]$WinCSKey = "F33E0C8E002", [Parameter(Mandatory = $false)] [string]$WavePrefix = "SecureBoot-Rollout", [Parameter(Mandatory = $false)] [int]$WaveNumber = 1, [Parameter(Mandatory = $false)] [string]$TargetOU, [Parameter(Mandatory = $false)] [bool]$DryRun = $false ) $gpoName = "${WavePrefix}-WinCS-Wave${WaveNumber}" $securityGroup = "${WavePrefix}-WinCS-Wave${WaveNumber}" $result = Deploy-WinCSGPOForWave ` -GPOName $gpoName ` -TargetOU $TargetOU ` -SecurityGroupName $securityGroup ` -WaveHostnames $WaveHostnames ` -WinCSKey $WinCSKey ` -DryRun $DryRun # Convert to expected return format return @{ Success = $result.Success Applied = $result.ComputersAdded Skipped = 0 Failed = if ($result.ComputersFailed) { $result.ComputersFailed } else { 0 } Results = @() } } # ============================================================================ # ENABLE TASK DEPLOYMENT # ============================================================================ # Deploy Enable-SecureBootUpdateTask.ps1 to devices with disabled scheduled task. # Uses a GPO with an immediate scheduled task that runs once. function Deploy-EnableTaskGPO { <# .SYNOPSIS Deploy Enable-SecureBootUpdateTask.ps1 via GPO scheduled task. .DESCRIPTION Creates a GPO that deploys a one-time scheduled task to enable the Secure-Boot-Update scheduled task on target devices. .PARAMETER TargetOU OU to link the GPO to. .PARAMETER TargetHostnames Hostnames of devices with disabled task (from aggregation report). .PARAMETER DryRun If true, only log what would be done. #> param( [Parameter(Mandatory = $false)] [string]$TargetOU, [Parameter(Mandatory = $true)] [array]$TargetHostnames, [Parameter(Mandatory = $false)] [bool]$DryRun = $false ) $GPOName = "SecureBoot-EnableTask-Remediation" $SecurityGroupName = "SecureBoot-EnableTask-Devices" $TaskName = "SecureBoot-EnableTask-OneTime" $TaskDescription = "One-time task to enable Secure-Boot-Update scheduled task" Write-Log "=" * 70 "INFO" Write-Log "DEPLOYING ENABLE TASK REMEDIATION" "INFO" Write-Log "=" * 70 "INFO" Write-Log "Target devices: $($TargetHostnames.Count)" "INFO" Write-Log "GPO: $GPOName" "INFO" Write-Log "Security Group: $SecurityGroupName" "INFO" if ($DryRun) { Write-Log "[DRYRUN] Would create GPO: $GPOName" "INFO" Write-Log "[DRYRUN] Would create security group: $SecurityGroupName" "INFO" Write-Log "[DRYRUN] Would add $($TargetHostnames.Count) computers to group" "INFO" Write-Log "[DRYRUN] Would deploy one-time scheduled task to enable Secure-Boot-Update" "INFO" Write-Log "[DRYRUN] Would link GPO to: $TargetOU" "INFO" return @{ Success = $true ComputersAdded = 0 DryRun = $true } } try { # Import required modules Import-Module GroupPolicy -ErrorAction Stop Import-Module ActiveDirectory -ErrorAction Stop } catch { Write-Log "Failed to import required modules: $($_.Exception.Message)" "ERROR" return @{ Success = $false; Error = $_.Exception.Message } } # Step 1: Create or get GPO $gpo = Get-GPO -Name $GPOName -ErrorAction SilentlyContinue if ($gpo) { Write-Log "GPO already exists: $GPOName" "INFO" } else { try { $gpo = New-GPO -Name $GPOName -Comment "Secure Boot Task Enable Remediation - Created $(Get-Date -Format 'yyyy-MM-dd HH:mm')" Write-Log "Created GPO: $GPOName" "OK" } catch { Write-Log "Failed to create GPO: $($_.Exception.Message)" "ERROR" return @{ Success = $false; Error = $_.Exception.Message } } } # Step 2: Deploy scheduled task XML to GPO SYSVOL # The task runs a PowerShell command to enable the Secure-Boot-Update task try { $sysvolPath = "\\$($env:USERDNSDOMAIN)\SYSVOL\$($env:USERDNSDOMAIN)\Policies\{$($gpo.Id)}\Machine\Preferences\ScheduledTasks" if (-not (Test-Path $sysvolPath)) { New-Item -ItemType Directory -Path $sysvolPath -Force | Out-Null } # PowerShell command to enable the Secure-Boot-Update task $enableCommand = 'schtasks.exe /Change /TN "\Microsoft\Windows\PI\Secure-Boot-Update" /ENABLE 2>$null; if ($LASTEXITCODE -ne 0) { Get-ScheduledTask -TaskPath "\Microsoft\Windows\PI\" -TaskName "Secure-Boot-Update" -ErrorAction SilentlyContinue | Enable-ScheduledTask }' # Encode command for safe XML embedding $encodedCommand = [Convert]::ToBase64String([Text.Encoding]::Unicode.GetBytes($enableCommand)) $taskGuid = [guid]::NewGuid().ToString("B").ToUpper() # GPP Scheduled Task XML - Immediate task that runs once $gppTaskXml = @" $TaskDescription S-1-5-18 HighestAvailable PT5M PT1H false false IgnoreNew false false true true true true false PT1H 7 PT0S powershell.exe -NoProfile -NonInteractive -ExecutionPolicy Bypass -EncodedCommand $encodedCommand "@ $gppTaskXml | Out-File -FilePath (Join-Path $sysvolPath "ScheduledTasks.xml") -Encoding UTF8 -Force Write-Log "Deployed one-time scheduled task to GPO: $TaskName" "OK" } catch { Write-Log "Failed to deploy scheduled task XML: $($_.Exception.Message)" "ERROR" return @{ Success = $false; Error = $_.Exception.Message } } # Step 3: Create or get security group $group = Get-ADGroup -Filter "Name -eq '$SecurityGroupName'" -ErrorAction SilentlyContinue if (-not $group) { try { $group = New-ADGroup -Name $SecurityGroupName ` -GroupCategory Security ` -GroupScope DomainLocal ` -Description "Computers with disabled Secure-Boot-Update task - targeted for remediation" ` -PassThru Write-Log "Created security group: $SecurityGroupName" "OK" } catch { Write-Log "Failed to create security group: $($_.Exception.Message)" "ERROR" return @{ Success = $false; Error = $_.Exception.Message } } } else { Write-Log "Security group exists: $SecurityGroupName" "INFO" } # Step 4: Add computers to security group $added = 0 $failed = 0 foreach ($hostname in $TargetHostnames) { try { $computer = Get-ADComputer -Identity $hostname -ErrorAction Stop Add-ADGroupMember -Identity $SecurityGroupName -Members $computer -ErrorAction SilentlyContinue $added++ } catch { $failed++ Write-Log "Computer not found in AD: $hostname" "WARN" } } Write-Log "Added $added computers to security group ($failed not found in AD)" "OK" # Step 5: Configure security filtering on GPO try { Set-GPPermission -Name $GPOName -TargetName "Authenticated Users" -TargetType Group -PermissionLevel GpoRead -Replace -ErrorAction SilentlyContinue Set-GPPermission -Name $GPOName -TargetName $SecurityGroupName -TargetType Group -PermissionLevel GpoApply -ErrorAction Stop Write-Log "Configured security filtering for: $SecurityGroupName" "OK" } catch { Write-Log "Failed to configure security filtering: $($_.Exception.Message)" "WARN" } # Step 6: Link GPO to OU if ($TargetOU) { try { $existingLink = Get-GPInheritance -Target $TargetOU -ErrorAction SilentlyContinue | Select-Object -ExpandProperty GpoLinks | Where-Object { $_.DisplayName -eq $GPOName } if (-not $existingLink) { New-GPLink -Name $GPOName -Target $TargetOU -LinkEnabled Yes -ErrorAction Stop Write-Log "Linked GPO to: $TargetOU" "OK" } else { Write-Log "GPO already linked to target OU" "INFO" } } catch { Write-Log "Failed to link GPO to OU: $($_.Exception.Message)" "ERROR" return @{ Success = $false; Error = "GPO link failed: $($_.Exception.Message)" } } } else { Write-Log "No TargetOU specified - GPO will need to be manually linked" "WARN" } Write-Log "" "INFO" Write-Log "ENABLE TASK DEPLOYMENT COMPLETE" "OK" Write-Log "Devices will run the enable task at next GPO refresh (gpupdate)" "INFO" Write-Log "The task runs once as SYSTEM and enables Secure-Boot-Update" "INFO" Write-Log "" "INFO" return @{ Success = $true ComputersAdded = $added ComputersFailed = $failed GPOName = $GPOName SecurityGroup = $SecurityGroupName } } # ============================================================================ # ENABLE TASK ON DISABLED DEVICES # ============================================================================ if ($EnableTaskOnDisabled) { Write-Host "" Write-Host ("=" * 70) -ForegroundColor Yellow Write-Host " ENABLE TASK REMEDIATION - Fixing Disabled Scheduled Tasks" -ForegroundColor Yellow Write-Host ("=" * 70) -ForegroundColor Yellow Write-Host "" # Find devices with disabled task from aggregation data if (-not $AggregationInputPath) { Write-Host "ERROR: -AggregationInputPath is required to identify devices with disabled task" -ForegroundColor Red Write-Host "Usage: .\Start-SecureBootRolloutOrchestrator.ps1 -EnableTaskOnDisabled -AggregationInputPath -ReportBasePath " -ForegroundColor Gray exit 1 } Write-Host "Scanning for devices with disabled Secure-Boot-Update task..." -ForegroundColor Cyan # Load JSON files and find devices with disabled task $jsonFiles = Get-ChildItem -Path $AggregationInputPath -Filter "*.json" -Recurse -ErrorAction SilentlyContinue | Where-Object { $_.Name -notmatch "ScanHistory|RolloutState|RolloutPlan" } $disabledTaskDevices = @() foreach ($file in $jsonFiles) { try { $device = Get-Content $file.FullName -Raw | ConvertFrom-Json if ($device.SecureBootTaskEnabled -eq $false -or $device.SecureBootTaskStatus -eq 'Disabled' -or $device.SecureBootTaskStatus -eq 'NotFound') { # Only include devices that haven't already updated (no Event 1808) if ([int]$device.Event1808Count -eq 0) { $disabledTaskDevices += $device.HostName } } } catch { # Skip invalid files } } $disabledTaskDevices = $disabledTaskDevices | Select-Object -Unique if ($disabledTaskDevices.Count -eq 0) { Write-Host "" Write-Host "No devices found with disabled Secure-Boot-Update task." -ForegroundColor Green Write-Host "All devices either have the task enabled or have already updated." -ForegroundColor Gray exit 0 } Write-Host "" Write-Host "Found $($disabledTaskDevices.Count) devices with disabled task:" -ForegroundColor Yellow $disabledTaskDevices | Select-Object -First 20 | ForEach-Object { Write-Host " - $_" -ForegroundColor Gray } if ($disabledTaskDevices.Count -gt 20) { Write-Host " ... and $($disabledTaskDevices.Count - 20) more" -ForegroundColor Gray } Write-Host "" # Deploy the Enable Task GPO $result = Deploy-EnableTaskGPO -TargetHostnames $disabledTaskDevices -TargetOU $TargetOU -DryRun $DryRun if ($result.Success) { Write-Host "" Write-Host "SUCCESS: Enable Task GPO deployed" -ForegroundColor Green Write-Host " Computers added to security group: $($result.ComputersAdded)" -ForegroundColor Cyan if ($result.ComputersFailed -gt 0) { Write-Host " Computers not found in AD: $($result.ComputersFailed)" -ForegroundColor Yellow } Write-Host "" Write-Host "NEXT STEPS:" -ForegroundColor White Write-Host " 1. Devices will receive the GPO at next refresh (gpupdate /force)" -ForegroundColor Gray Write-Host " 2. The one-time task will enable Secure-Boot-Update" -ForegroundColor Gray Write-Host " 3. Re-run aggregation to verify task is now enabled" -ForegroundColor Gray } else { Write-Host "" Write-Host "FAILED: Could not deploy Enable Task GPO" -ForegroundColor Red Write-Host "Error: $($result.Error)" -ForegroundColor Red } exit 0 } # ============================================================================ # MAIN ORCHESTRATION LOOP # ============================================================================ Write-Host "" Write-Host ("=" * 80) -ForegroundColor Cyan Write-Host " SECURE BOOT ROLLOUT ORCHESTRATOR - CONTINUOUS DEPLOYMENT" -ForegroundColor Cyan Write-Host ("=" * 80) -ForegroundColor Cyan Write-Host "" if ($DryRun) { Write-Host "[DRY RUN MODE]" -ForegroundColor Magenta } if ($UseWinCS) { Write-Host "[WinCS MODE]" -ForegroundColor Yellow Write-Host "Using WinCsFlags.exe instead of GPO/AvailableUpdatesPolicy" -ForegroundColor Yellow Write-Host "WinCS Key: $WinCSKey" -ForegroundColor Gray Write-Host "" } Write-Log "Starting Secure Boot Rollout Orchestrator" "INFO" Write-Log "Input Path: $AggregationInputPath" "INFO" Write-Log "Report Path: $ReportBasePath" "INFO" if ($UseWinCS) { Write-Log "Deployment Method: WinCS (WinCsFlags.exe /apply --key `"$WinCSKey`")" "INFO" } else { Write-Log "Deployment Method: GPO (AvailableUpdatesPolicy)" "INFO" } # Resolve TargetOU - default to domain root for domain-wide coverage # Only needed for GPO deployment method (WinCS doesn't require AD/GPO) if (-not $UseWinCS -and -not $TargetOU) { try { # Try multiple methods to get domain DN $domainDN = $null # Method 1: Get-ADDomain (requires RSAT-AD-PowerShell) try { Import-Module ActiveDirectory -ErrorAction Stop $domainDN = (Get-ADDomain -ErrorAction Stop).DistinguishedName } catch { Write-Log "Get-ADDomain failed: $($_.Exception.Message)" "WARN" } # Method 2: Use RootDSE via ADSI if (-not $domainDN) { try { $rootDSE = [ADSI]"LDAP://RootDSE" $domainDN = $rootDSE.defaultNamingContext.ToString() } catch { Write-Log "ADSI RootDSE failed: $($_.Exception.Message)" "WARN" } } # Method 3: Parse from computer's domain membership if (-not $domainDN) { try { $domain = [System.DirectoryServices.ActiveDirectory.Domain]::GetComputerDomain() $domainDN = "DC=" + ($domain.Name -replace '\.', ',DC=') } catch { Write-Log "GetComputerDomain failed: $($_.Exception.Message)" "WARN" } } if ($domainDN) { $TargetOU = $domainDN Write-Log "Target: Domain Root ($domainDN) - GPO will apply domain-wide via security group filtering" "INFO" } else { Write-Log "Could not determine domain DN - GPO will be created but NOT LINKED!" "ERROR" Write-Log "Please specify -TargetOU parameter or link GPO manually after creation" "ERROR" $TargetOU = $null } } catch { Write-Log "Could not get domain DN - GPO will be created but not linked. Link manually if needed." "WARN" Write-Log "Error: $($_.Exception.Message)" "WARN" $TargetOU = $null } } else { Write-Log "Target OU: $TargetOU" "INFO" } Write-Log "Max Wait Hours: $MaxWaitHours" "INFO" Write-Log "Poll Interval: $PollIntervalMinutes minutes" "INFO" if ($LargeScaleMode) { Write-Log "LargeScaleMode enabled (batch size: $ProcessingBatchSize, log sample: $DeviceLogSampleSize)" "INFO" } # ============================================================================ # PREREQUISITE CHECK: Verify detection is deployed and working # ============================================================================ Write-Host "" Write-Log "Checking prerequisites..." "INFO" $detectionCheck = Test-DetectionGPODeployed -JsonPath $AggregationInputPath if (-not $detectionCheck.IsDeployed) { Write-Log $detectionCheck.Message "ERROR" Write-Host "" Write-Host "REQUIRED: Deploy detection infrastructure first:" -ForegroundColor Yellow Write-Host " 1. Run: Deploy-GPO-SecureBootCollection.ps1 -OUPath 'OU=...' -OutputPath '\\server\SecureBootLogs$'" -ForegroundColor Cyan Write-Host " 2. Wait for devices to report (12-24 hours)" -ForegroundColor Cyan Write-Host " 3. Re-run this orchestrator" -ForegroundColor Cyan Write-Host "" if (-not $DryRun) { return } } else { Write-Log $detectionCheck.Message "OK" } # Check data freshness $freshness = Get-DataFreshness -JsonPath $AggregationInputPath Write-Log "Data freshness: $($freshness.TotalFiles) files, $($freshness.FreshFiles) fresh (<24h), $($freshness.StaleFiles) stale (>72h)" "INFO" if ($freshness.Warning) { Write-Log $freshness.Warning "WARN" } # Load Allow List (targeted rollout - ONLY these devices will be rolled out) $allowedHostnames = @() if ($AllowListPath -or $AllowADGroup) { $allowedHostnames = Get-AllowedHostnames -AllowFilePath $AllowListPath -ADGroupName $AllowADGroup if ($allowedHostnames.Count -gt 0) { Write-Log "AllowList: ONLY $($allowedHostnames.Count) devices will be considered for rollout" "INFO" } else { Write-Log "AllowList specified but no devices found - this will block all rollouts!" "WARN" } } # Load VIP/exclusion list (BlockList) $excludedHostnames = @() if ($ExclusionListPath -or $ExcludeADGroup) { $excludedHostnames = Get-ExcludedHostnames -ExclusionFilePath $ExclusionListPath -ADGroupName $ExcludeADGroup if ($excludedHostnames.Count -gt 0) { Write-Log "VIP Exclusion: $($excludedHostnames.Count) devices will be skipped from rollout" "INFO" } } # Load state $rolloutState = Get-RolloutState $blockedBuckets = Get-BlockedBuckets $adminApproved = Get-AdminApproved $deviceHistory = Get-DeviceHistory if ($rolloutState.Status -eq "NotStarted") { $rolloutState.Status = "InProgress" $rolloutState.StartedAt = Get-Date -Format "yyyy-MM-dd HH:mm:ss" Write-Log "Starting new rollout" "WAVE" } Write-Log "Current Wave: $($rolloutState.CurrentWave)" "INFO" Write-Log "Blocked Buckets: $($blockedBuckets.Count)" "INFO" # Main loop - runs until all eligible devices are updated $iterationCount = 0 while ($true) { $iterationCount++ Write-Host "" Write-Host ("=" * 80) -ForegroundColor White Write-Log "=== ITERATION $iterationCount ===" "WAVE" Write-Host ("=" * 80) -ForegroundColor White # Step 1: Run aggregation Write-Log "Step 1: Running aggregation..." "INFO" # Orchestrator always reuses a single folder (LargeScaleMode) to avoid disk bloat # Admins running the aggregator manually get timestamped folders for point-in-time snapshots $aggregationPath = Join-Path $ReportBasePath "Aggregation_Current" # Check data freshness before aggregating $freshness = Get-DataFreshness -JsonPath $AggregationInputPath Write-Log "Data freshness: $($freshness.FreshFiles)/$($freshness.TotalFiles) devices reported in last 24h" "INFO" if ($freshness.Warning) { Write-Log $freshness.Warning "WARN" } $aggregateScript = Join-Path $ScriptRoot "Aggregate-SecureBootData.ps1" $scanHistoryPath = Join-Path $ReportBasePath "ScanHistory.json" $rolloutSummaryPath = Join-Path $stateDir "SecureBootRolloutSummary.json" if (Test-Path $aggregateScript) { if (-not $DryRun) { # Orchestrator always uses streaming + incremental for efficiency # Aggregator auto-elevates to PS7 if available for best performance $aggregateParams = @{ InputPath = $AggregationInputPath OutputPath = $aggregationPath StreamingMode = $true IncrementalMode = $true SkipReportIfUnchanged = $true ParallelThreads = 8 } # Pass rollout summary if it exists (for velocity/projection data) if (Test-Path $rolloutSummaryPath) { $aggregateParams['RolloutSummaryPath'] = $rolloutSummaryPath } & $aggregateScript @aggregateParams # Show command to generate full HTML dashboard with device tables Write-Host "" Write-Host "To generate full HTML dashboard with Manufacturer/Model tables, run:" -ForegroundColor Yellow Write-Host " $aggregateScript -InputPath `"$AggregationInputPath`" -OutputPath `"$aggregationPath`"" -ForegroundColor Yellow Write-Host "" } else { Write-Log "[DRYRUN] Would run aggregation" "INFO" # In DryRun, use existing aggregation data from ReportBasePath directly $aggregationPath = $ReportBasePath } } $rolloutState.LastAggregation = Get-Date -Format "yyyy-MM-dd HH:mm:ss" # Step 2: Load current device status Write-Log "Step 2: Loading device status..." "INFO" $notUptodateCsv = Get-ChildItem -Path $aggregationPath -Filter "*NotUptodate*.csv" -ErrorAction SilentlyContinue | Where-Object { $_.Name -notlike "*Buckets*" } | Sort-Object LastWriteTime -Descending | Select-Object -First 1 if (-not $notUptodateCsv -and -not $DryRun) { Write-Log "No aggregation data found. Waiting..." "WARN" Start-Sleep -Seconds ($PollIntervalMinutes * 60) continue } $notUpdatedDevices = if ($notUptodateCsv) { Import-Csv $notUptodateCsv.FullName } else { @() } Write-Log "Devices not updated: $($notUpdatedDevices.Count)" "INFO" $notUpdatedIndexes = Get-NotUpdatedIndexes -Devices $notUpdatedDevices # Step 3: Update device history (tracking by hostname) Write-Log "Step 3: Updating device history..." "INFO" Update-DeviceHistory -CurrentDevices $notUpdatedDevices -DeviceHistory $deviceHistory Save-DeviceHistory -History $deviceHistory # Step 4: Check for blocked buckets (unreachable devices) $existingBlockedCount = $blockedBuckets.Count Write-Log "Step 4: Checking for blocked buckets (pinging devices past wait period)..." "INFO" if ($existingBlockedCount -gt 0) { Write-Log "Currently blocked buckets from previous runs: $existingBlockedCount" "INFO" } if ($adminApproved.Count -gt 0) { Write-Log "Admin-approved buckets (will not be re-blocked): $($adminApproved.Count)" "INFO" } $newlyBlocked = Update-BlockedBuckets -RolloutState $rolloutState -BlockedBuckets $blockedBuckets -AdminApproved $adminApproved -NotUpdatedDevices $notUpdatedDevices -NotUpdatedIndexes $notUpdatedIndexes -MaxWaitHours $MaxWaitHours -DryRun:$DryRun if ($newlyBlocked.Count -gt 0) { Save-BlockedBuckets -Blocked $blockedBuckets Write-Log "Newly blocked buckets (this iteration): $($newlyBlocked.Count)" "BLOCKED" } # Step 4b: Auto-unblock buckets where devices have updated $autoUnblocked = Update-AutoUnblockedBuckets -BlockedBuckets $blockedBuckets -RolloutState $rolloutState -NotUpdatedDevices $notUpdatedDevices -ReportBasePath $ReportBasePath -NotUpdatedIndexes $notUpdatedIndexes -LogSampleSize $DeviceLogSampleSize if ($autoUnblocked.Count -gt 0) { Save-BlockedBuckets -Blocked $blockedBuckets Write-Log "Auto-unblocked buckets (devices updated): $($autoUnblocked.Count)" "OK" } # Step 5: Calculate remaining eligible devices $eligibleCount = 0 foreach ($device in $notUpdatedDevices) { $bucketKey = Get-BucketKey $device if (-not $blockedBuckets.Contains($bucketKey)) { $eligibleCount++ } } Write-Log "Eligible devices remaining: $eligibleCount" "INFO" Write-Log "Blocked buckets: $($blockedBuckets.Count)" "INFO" # Step 6: Check completion if ($eligibleCount -eq 0) { Write-Log "ROLLOUT COMPLETE - All eligible devices updated!" "OK" $rolloutState.Status = "Completed" $rolloutState.CompletedAt = Get-Date -Format "yyyy-MM-dd HH:mm:ss" Save-RolloutState -State $rolloutState break } # Step 6: Generate and deploy next wave Write-Log "Step 6: Generating rollout wave..." "INFO" $waveDevices = New-RolloutWave -AggregationPath $aggregationPath -BlockedBuckets $blockedBuckets -RolloutState $rolloutState -AllowedHostnames $allowedHostnames -ExcludedHostnames $excludedHostnames # Check if we have devices to deploy ($waveDevices could be $null, empty, or with actual devices) $hasDevices = $waveDevices -and @($waveDevices | Where-Object { $_ }).Count -gt 0 if ($hasDevices) { # Only increment wave number when we actually have devices to deploy $rolloutState.CurrentWave++ Write-Log "Wave $($rolloutState.CurrentWave): $(@($waveDevices).Count) devices" "WAVE" # Deploy GPO using inlined function $gpoName = "${WavePrefix}-Wave$($rolloutState.CurrentWave)" $securityGroup = "${WavePrefix}-Wave$($rolloutState.CurrentWave)" $hostnames = @($waveDevices | ForEach-Object { if ($_.Hostname) { $_.Hostname } elseif ($_.HostName) { $_.HostName } else { $null } } | Where-Object { $_ }) # Save hostnames file for reference/audit $hostnamesFile = Join-Path $stateDir "Wave$($rolloutState.CurrentWave)_Hostnames.txt" $hostnames | Out-File $hostnamesFile -Encoding UTF8 # Validate we have hostnames to deploy to if ($hostnames.Count -eq 0) { Write-Log "No valid hostnames found in wave $($rolloutState.CurrentWave) - devices may be missing Hostname property" "WARN" Write-Log "Skipping deployment for this wave - check device data" "WARN" # Still wait before next iteration if (-not $DryRun) { Write-Log "Sleeping for $PollIntervalMinutes minutes before retry..." "INFO" Start-Sleep -Seconds ($PollIntervalMinutes * 60) } continue } Write-Log "Deploying to $($hostnames.Count) hostnames in Wave $($rolloutState.CurrentWave)" "INFO" # Deploy using either WinCS or GPO method based on -UseWinCS parameter if ($UseWinCS) { # WinCS Method: Create GPO with scheduled task to run WinCsFlags.exe as SYSTEM on each endpoint Write-Log "Using WinCS deployment method (Key: $WinCSKey)" "WAVE" $wincsResult = Deploy-WinCSForWave -WaveHostnames $hostnames ` -WinCSKey $WinCSKey ` -WavePrefix $WavePrefix ` -WaveNumber $rolloutState.CurrentWave ` -TargetOU $TargetOU ` -DryRun:$DryRun if (-not $wincsResult.Success) { Write-Log "WinCS deployment had failures - Applied: $($wincsResult.Applied), Failed: $($wincsResult.Failed)" "WARN" } else { Write-Log "WinCS deployment successful - Applied: $($wincsResult.Applied), Skipped: $($wincsResult.Skipped)" "OK" } # Save WinCS results for audit $wincsResultFile = Join-Path $stateDir "Wave$($rolloutState.CurrentWave)_WinCS_Results.json" $wincsResult | ConvertTo-Json -Depth 5 | Out-File $wincsResultFile -Encoding UTF8 } else { # GPO Method: Create GPO with AvailableUpdatesPolicy registry setting $gpoResult = Deploy-GPOForWave -GPOName $gpoName -TargetOU $TargetOU -SecurityGroupName $securityGroup -WaveHostnames $hostnames -DryRun:$DryRun if (-not $gpoResult) { Write-Log "GPO deployment failed - will retry next iteration" "ERROR" } } # Record wave in state $waveRecord = @{ WaveNumber = $rolloutState.CurrentWave StartedAt = Get-Date -Format "yyyy-MM-dd HH:mm:ss" DeviceCount = @($waveDevices).Count Devices = @($waveDevices | ForEach-Object { @{ Hostname = if ($_.Hostname) { $_.Hostname } elseif ($_.HostName) { $_.HostName } else { $null } BucketKey = Get-BucketKey $_ } }) } # Ensure WaveHistory is always an array before appending (prevents hashtable merge issues) $rolloutState.WaveHistory = @($rolloutState.WaveHistory) + @($waveRecord) $rolloutState.TotalDevicesTargeted += @($waveDevices).Count Save-RolloutState -State $rolloutState Write-Log "Wave $($rolloutState.CurrentWave) deployed. Waiting $PollIntervalMinutes minutes..." "OK" } else { # Show status of deployed devices waiting for updates Write-Log "" "INFO" Write-Log "========== ALL DEVICES DEPLOYED - WAITING FOR STATUS ==========" "INFO" # Get all deployed devices from wave history $allDeployedLookup = @{} foreach ($wave in $rolloutState.WaveHistory) { foreach ($device in $wave.Devices) { if ($device.Hostname) { $allDeployedLookup[$device.Hostname] = @{ Hostname = $device.Hostname BucketKey = $device.BucketKey DeployedAt = $wave.StartedAt WaveNumber = $wave.WaveNumber } } } } $allDeployedDevices = @($allDeployedLookup.Values) if ($allDeployedDevices.Count -gt 0) { # Find which deployed devices are still pending (in NotUpdated list) $stillPendingCount = 0 $noLongerPendingCount = 0 $pendingSample = @() foreach ($deployed in $allDeployedDevices) { if ($notUpdatedIndexes.HostSet.Contains($deployed.Hostname)) { $stillPendingCount++ if ($pendingSample.Count -lt $DeviceLogSampleSize) { $pendingSample += $deployed.Hostname } } else { $noLongerPendingCount++ } } # Get actual Updated counts from aggregation - differentiate Event 1808 vs UEFICA2023Status $summaryCsv = Get-ChildItem -Path $aggregationPath -Filter "*Summary*.csv" | Sort-Object LastWriteTime -Descending | Select-Object -First 1 $actualUpdated = 0 $totalDevicesFromSummary = 0 $event1808Count = 0 $uefiStatusUpdated = 0 $needsRebootSample = @() if ($summaryCsv) { $summary = Import-Csv $summaryCsv.FullName | Select-Object -First 1 if ($summary.Updated) { $actualUpdated = [int]$summary.Updated } if ($summary.TotalDevices) { $totalDevicesFromSummary = [int]$summary.TotalDevices } } # Calculate velocity from wave history (devices updated per day) $devicesPerDay = 0 if ($rolloutState.StartedAt -and $actualUpdated -gt 0) { $startDate = [datetime]::Parse($rolloutState.StartedAt) $daysElapsed = ((Get-Date) - $startDate).TotalDays if ($daysElapsed -gt 0) { $devicesPerDay = $actualUpdated / $daysElapsed } } # Save rollout summary with weekend-aware projections # Use aggregator's NotUptodate count (excludes SB OFF devices) for consistency $notUpdatedCount = if ($summary -and $summary.NotUptodate) { [int]$summary.NotUptodate } else { $totalDevicesFromSummary - $actualUpdated } # Update rollout state with actual counts from aggregation $rolloutState.TotalDevicesUpdated = $actualUpdated if ($totalDevicesFromSummary -gt 0) { $rolloutState.TotalDevicesTargeted = $totalDevicesFromSummary } Save-RolloutState -State $rolloutState Save-RolloutSummary -State $rolloutState ` -TotalDevices $totalDevicesFromSummary ` -UpdatedDevices $actualUpdated ` -NotUpdatedDevices $notUpdatedCount ` -DevicesPerDay $devicesPerDay # Check raw data for devices with UEFICA2023Status=Updated but no Event 1808 (needs reboot) $dataFiles = Get-ChildItem -Path $AggregationInputPath -Filter "*.json" -ErrorAction SilentlyContinue $totalDataFiles = @($dataFiles).Count $batchSize = [Math]::Max(500, $ProcessingBatchSize) if ($LargeScaleMode) { $batchSize = [Math]::Max(2000, $ProcessingBatchSize) } if ($totalDataFiles -gt 0) { for ($idx = 0; $idx -lt $totalDataFiles; $idx += $batchSize) { $end = [Math]::Min($idx + $batchSize - 1, $totalDataFiles - 1) $batchFiles = $dataFiles[$idx..$end] foreach ($file in $batchFiles) { try { $deviceData = Get-Content $file.FullName -Raw | ConvertFrom-Json $hostname = $deviceData.Hostname if (-not $hostname) { continue } $has1808 = [int]$deviceData.Event1808Count -gt 0 $hasUefiUpdated = $deviceData.UEFICA2023Status -eq "Updated" if ($has1808) { $event1808Count++ } elseif ($hasUefiUpdated) { $uefiStatusUpdated++ if ($needsRebootSample.Count -lt $DeviceLogSampleSize) { $needsRebootSample += $hostname } } } catch { } } Save-ProcessingCheckpoint -Stage "RebootStatusScan" -Processed ($end + 1) -Total $totalDataFiles -Metrics @{ Event1808Count = $event1808Count UEFIUpdatedAwaitingReboot = $uefiStatusUpdated } } } Write-Log "Total deployed: $($allDeployedDevices.Count)" "INFO" Write-Log "Updated (Event 1808 confirmed): $event1808Count" "OK" if ($uefiStatusUpdated -gt 0) { Write-Log "Updated (UEFICA2023Status=Updated, awaiting reboot): $uefiStatusUpdated" "OK" $rebootSuffix = if ($uefiStatusUpdated -gt $DeviceLogSampleSize) { " ... (+$($uefiStatusUpdated - $DeviceLogSampleSize) more)" } else { "" } Write-Log " Devices needing reboot for Event 1808 (sample): $($needsRebootSample -join ', ')$rebootSuffix" "INFO" Write-Log " These devices will report Event 1808 after next reboot" "INFO" } Write-Log "No longer pending: $noLongerPendingCount (includes SecureBoot OFF, missing devices)" "INFO" Write-Log "Awaiting status: $stillPendingCount" "INFO" if ($stillPendingCount -gt 0) { $pendingSuffix = if ($stillPendingCount -gt $DeviceLogSampleSize) { " ... (+$($stillPendingCount - $DeviceLogSampleSize) more)" } else { "" } Write-Log "Pending devices (sample): $($pendingSample -join ', ')$pendingSuffix" "WARN" } } else { Write-Log "No devices have been deployed yet" "INFO" } Write-Log "================================================================" "INFO" Write-Log "" "INFO" } # Wait before next iteration if (-not $DryRun) { Write-Log "Sleeping for $PollIntervalMinutes minutes..." "INFO" Start-Sleep -Seconds ($PollIntervalMinutes * 60) } else { Write-Log "[DRYRUN] Would wait $PollIntervalMinutes minutes" "INFO" break # Exit after one iteration in dry run } } # ============================================================================ # FINAL SUMMARY # ============================================================================ Write-Host "" Write-Host ("=" * 80) -ForegroundColor Green Write-Host " ROLLOUT ORCHESTRATOR SUMMARY" -ForegroundColor Green Write-Host ("=" * 80) -ForegroundColor Green Write-Host "" $finalState = Get-RolloutState $finalBlocked = Get-BlockedBuckets Write-Host "Status: $($finalState.Status)" -ForegroundColor $(if ($finalState.Status -eq "Completed") { "Green" } else { "Yellow" }) Write-Host "Total Waves: $($finalState.CurrentWave)" Write-Host "Devices Targeted: $($finalState.TotalDevicesTargeted)" Write-Host "Blocked Buckets: $($finalBlocked.Count)" -ForegroundColor $(if ($finalBlocked.Count -gt 0) { "Red" } else { "Green" }) Write-Host "Devices Tracked: $($deviceHistory.Count)" -ForegroundColor Gray Write-Host "" if ($finalBlocked.Count -gt 0) { Write-Host "BLOCKED BUCKETS (require manual review):" -ForegroundColor Red foreach ($key in $finalBlocked.Keys) { $info = $finalBlocked[$key] Write-Host " - $key" -ForegroundColor Red Write-Host " Reason: $($info.Reason)" -ForegroundColor Gray } Write-Host "" Write-Host "Blocked buckets file: $blockedBucketsPath" -ForegroundColor Yellow } Write-Host "" Write-Host "State files:" -ForegroundColor Cyan Write-Host " Rollout State: $rolloutStatePath" Write-Host " Blocked Buckets: $blockedBucketsPath" Write-Host " Device History: $deviceHistoryPath" Write-Host "" # SIG # Begin signature block # MIIpzgYJKoZIhvcNAQcCoIIpvzCCKbsCAQExDzANBglghkgBZQMEAgEFADB5Bgor # BgEEAYI3AgEEoGswaTA0BgorBgEEAYI3AgEeMCYCAwEAAAQQH8w7YFlLCE63JNLG # KX7zUQIBAAIBAAIBAAIBAAIBADAxMA0GCWCGSAFlAwQCAQUABCBs8RFtNrqIl4Fo # YA56W9DeweNFN/NKTfim6LuT/S41waCCDdYwgga9MIIEpaADAgECAhMzAAAAHEif # gd+hsLd3AAAAAAAcMA0GCSqGSIb3DQEBDAUAMIGIMQswCQYDVQQGEwJVUzETMBEG # A1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWlj # cm9zb2Z0IENvcnBvcmF0aW9uMTIwMAYDVQQDEylNaWNyb3NvZnQgUm9vdCBDZXJ0 # aWZpY2F0ZSBBdXRob3JpdHkgMjAxMDAeFw0yNDA4MDgyMTM2MjNaFw0zNTA2MjMy # MjA0MDFaMF8xCzAJBgNVBAYTAlVTMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9y # YXRpb24xMDAuBgNVBAMTJ01pY3Jvc29mdCBXaW5kb3dzIENvZGUgU2lnbmluZyBQ # Q0EgMjAyNDCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAJp9a30nwXYq # Lq7j1TT/zCtt7vxU+CCj+7BkifS/B2gXKGU7OV9SXRJGP1yFs5p6jpsYi4cYzF56 # AV0AEmmEjV8wT2lvPU5BhN3wV30HqYPIYEj5P3WXf0kXD9fvjUf1GAtXEriJ8w7A # LNaVEm9Rs4ePA0ZsYHaCbU5kBUJQDXv76hafOcQgdFCA3I3zYtfzX2vOwx87uDOa # CuyKORZih9c3zTf+TLC5QYLyhVMBnDXEHDOrvaw92DSyIqpdgRWpufzqDFy1egVj # koXZhb+9pZ9heUzNXTXhOoXzexh6YzAL4flBWm+Bc1hQyESenEvBJznV+25u3h77 # jjgMUY44+WXQ4u9qddDe/U5SeAaKRvvibmi4z7QRpLvZsla0CPiOUGz00Do5sfkC # 0EwlsSzfM3+8A9rsyFVOgWDVPzt98OJP2EoaEOq8GE9GCoN2i7/4C2FCwff1BSCT # JWZO1Wcr2MteJE6UxGV+ihA8nN51YPKD2dYGoewrXvRzC/1HoUeSvlZf0mf9GHEt # vvkbJVRRo6PBf0md5t87Vb1mM/fIp1eypyaxmXkgpcBwuylsOq2kSVOJ5wBPoaEs # sJkeMcKnEuuu++UKdDHlS0DtsYjN0QnOucvTdSsdvhzKOSjJF3XVqr9f2C945LXT # 5rxKIHUIEDBcNYU6BKDDH6rfpKOOCSilAgMBAAGjggFGMIIBQjAOBgNVHQ8BAf8E # BAMCAYYwEAYJKwYBBAGCNxUBBAMCAQAwHQYDVR0OBBYEFB6C3w7XjLPXAjSDDtqr # rWW5r7jsMBkGCSsGAQQBgjcUAgQMHgoAUwB1AGIAQwBBMA8GA1UdEwEB/wQFMAMB # Af8wHwYDVR0jBBgwFoAU1fZWy4/oolxiaNE9lJBb186aGMQwVgYDVR0fBE8wTTBL # oEmgR4ZFaHR0cDovL2NybC5taWNyb3NvZnQuY29tL3BraS9jcmwvcHJvZHVjdHMv # TWljUm9vQ2VyQXV0XzIwMTAtMDYtMjMuY3JsMFoGCCsGAQUFBwEBBE4wTDBKBggr # BgEFBQcwAoY+aHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraS9jZXJ0cy9NaWNS # b29DZXJBdXRfMjAxMC0wNi0yMy5jcnQwDQYJKoZIhvcNAQEMBQADggIBAENf+N8/ # u+mUjDtc9btoA52RBc0XVDSBMQBMqxu56hXHBwuctUWs1XBqDDMIFCHu9c6Y/UF+ # TN8EIgjnujApKYmHP4f4EM3ARSmlzrpF5ozOJx0BA5FUv1jmpdf/2ZbqpvCxlxv/ # G1R4KjrSmmqPHzs6igw3b7RTbj7BxIS8fOIkwYWQhB2fLjlg+3HSrDGPFIhpIJWV # amMIR7a72OGonjdf45rspwqIHuynZU4avy9ruB/Rhhbwm+fMb8BMecIaTmkohx/E # ZZ5GNWcN6oTYW3G2BM3B3YznWkl9t4shP60fMue+2ksdHGWSE8EVTdSmGUdj0jrU # c46lGVFJISF3/MxcxnlNeP1Khyr+ZzT4Ets/I7mufpaLnLalzMR2zIuhGOAWWswe # sbjtFzkVUFgDR2SW903I0XKlbPEA6q8epHGJ9roxh85nsEKcBNUw4Scp68KCqSpF # BaKiyV1skd+l8U50WNePMb9Bzz0OfASal8v5sQG+DW01kN+I+RKUIbM5I50wJjiH # ymQFNDsbobFx9I95mCEEPU7fUZ3VT/HOUVbkmX7ltIC/eQAu5GO8fu+ceETMybvb # oxUM4dYNC+PzooUxfmC0DuKRwB21bX9+acuIBkxIm4Ed3O19w1VLoA7UNOUuJ7z6 # NQ2W/+q7cnfOPl2QVL4qlgCblUT2vmQpllV3MIIHETCCBPmgAwIBAgITMwAAASPV # jwJDBgPuLgAAAAABIzANBgkqhkiG9w0BAQwFADBfMQswCQYDVQQGEwJVUzEeMBwG # A1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMTAwLgYDVQQDEydNaWNyb3NvZnQg # V2luZG93cyBDb2RlIFNpZ25pbmcgUENBIDIwMjQwHhcNMjYwMzA1MTk1ODI2WhcN # MjcwMzAzMTk1ODI2WjB0MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3Rv # bjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0 # aW9uMR4wHAYDVQQDExVNaWNyb3NvZnQgQ29ycG9yYXRpb24wggIiMA0GCSqGSIb3 # DQEBAQUAA4ICDwAwggIKAoICAQDRofDhyqNp9ilU2DM6Ctcq1F931sSAyfMEmR3c # E/fpq4C15SQYoNRfyQ1YXmcjtfWjaAB4Dsqr+po0Q6IXSERF31/leCtiRtPrS8/1 # T9FA9AeOUTaFYyqHRz1M4MVeTwu89ZHo9xR26SVQmOHbFWLinZzlBV9WUJxfz0TN # MdWwAXzxnY9nln8Fn3Y0BnCXU8V3wjeCCAxoqM0v2gPG25eNn665gv7Zt3zh3vQY # hDkb48hXJPi4DvCrpBVwCd0Iv2Sgt4qftn5STyqhdu5WWqva+pk+Sa71Db9OAy6b # L640bGmzlUek1LAP97Ey8exl0c9azmLEQB97q4JRvhCSeejBM96a8Y6ccrrfhg/h # ZJEOka1eIDJxCuKdZaWCec0v+V7oI3RStJu1BEcF9d4FQaDSaWDAbDxqXUm1fHUJ # 9Xqky1yftpTfKFw3/gqsyXV44PE7EVHwwTCAGE5Z+skXRt4HEHuIhcOGxIFbH3nA # l5KvFxopRs2zExvh2WheW8Pu79sr3baamlQA2v4k9FqJl4k1RfwO5e0/9CKyDZ6c # ck/lV8zHM7PJtNtQUHc1FoyZ78GSlX6UFiJxDWzUzPbIHMcV4PSBV8/O7bs5h/bx # r3B4PxjIpWUztZhzXBhU9vhUjQgOX5ZLV7bmSJ+IrxtaN+5w2Nsewt9N+mfUUFyK # Av38uQIDAQABo4IBrzCCAaswDgYDVR0PAQH/BAQDAgeAMB8GA1UdJQQYMBYGCisG # AQQBgjc9BgEGCCsGAQUFBwMDMAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFNlj7H2k # ZtpiMmz55wigojWkWZZSMEUGA1UdEQQ+MDykOjA4MR4wHAYDVQQLExVNaWNyb3Nv # ZnQgQ29ycG9yYXRpb24xFjAUBgNVBAUTDTIzMDg2NSs1MDY5NTYwHwYDVR0jBBgw # FoAUHoLfDteMs9cCNIMO2qutZbmvuOwwagYDVR0fBGMwYTBfoF2gW4ZZaHR0cDov # L3d3dy5taWNyb3NvZnQuY29tL3BraW9wcy9jcmwvTWljcm9zb2Z0JTIwV2luZG93 # cyUyMENvZGUlMjBTaWduaW5nJTIwUENBJTIwMjAyNC5jcmwwdwYIKwYBBQUHAQEE # azBpMGcGCCsGAQUFBzAChltodHRwOi8vd3d3Lm1pY3Jvc29mdC5jb20vcGtpb3Bz # L2NlcnRzL01pY3Jvc29mdCUyMFdpbmRvd3MlMjBDb2RlJTIwU2lnbmluZyUyMFBD # QSUyMDIwMjQuY3J0MA0GCSqGSIb3DQEBDAUAA4ICAQAu6aVarz1Rb5/9eAD9oHOw # QS4jDknbBYnDwLcgMy/h995knuPL6/5K0XSUICUrjU3/bL56UD1tVwXsRehrHMkR # kgOMQk+7eH+Xvp6cyyTxwgIM7xy5YSf3VmJD3GI87lkGUqPFFOqGWjzZc3e/UQU7 # VbN4Elh4zJeKOdokXNuE3a5U1lAr1hnD6WpfIHO2nzGUHVSXDLMktWKMO89pMgLh # 6BXHdx1cAth+ugl86sdXC+VxCQJ28a2JvNq5RQcEBEuqSISe3/YRWYRTSLqhn8ea # v4lK4n+3fnAy2bmz76Z7UJ92X5KgTaHD+r8cvXmssFH3UbOj6EtE+c4SPNaFdIUW # GJyZBhLbThhlh1QlbTg0X47GhI89qZ/lxuYeEDYbPGaDry/KpMoUQxU+l49iQ6zd # kWSPGczB4ZDVRSrkDqz2K70tIvFaz8YfMhzTy5hhMNGvonCcguaFTgUsxr+OrvGC # WadOV/pZ3BcRafKvBDX+ney3HFcUByI3VPzIU7saoUbcujqTVA6GeNSybo8AI+71 # 9omBObyf2lriZb9a2/oVe5SiKhH/BDQVn+14Q26UxrUA/8zVOlzJS5eJQYhkHM9h # /Y2Nnmr2xLtFN0vW/aad7i104dVRaTLMBzbKIlCBOruBjM87frQc2Ncfb5Kme4u9 # o6GQVZf3kVy+p8uv8YgnQTGCG04wghtKAgEBMHYwXzELMAkGA1UEBhMCVVMxHjAc # BgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEwMC4GA1UEAxMnTWljcm9zb2Z0 # IFdpbmRvd3MgQ29kZSBTaWduaW5nIFBDQSAyMDI0AhMzAAABI9WPAkMGA+4uAAAA # AAEjMA0GCWCGSAFlAwQCAQUAoIH6MBkGCSqGSIb3DQEJAzEMBgorBgEEAYI3AgEE # MC8GCSqGSIb3DQEJBDEiBCBLa5hFp+fCvTrXFxPoZqEk9wnnDE6bvmevLKZGhqKX # 0zBQBgorBgEEAYI3CgMcMUIMQDg1NzY5QTAwNDI2N0M0RkE2NUY5OTc0NUE0RTFC # QkNGQzVFQTlBOEUyMkQ3RENDNjNBRTJCRDM2RkYxQUZGNzIwWgYKKwYBBAGCNwIB # DDFMMEqgJIAiAE0AaQBjAHIAbwBzAG8AZgB0ACAAVwBpAG4AZABvAHcAc6EigCBo # dHRwOi8vd3d3Lm1pY3Jvc29mdC5jb20vd2luZG93czANBgkqhkiG9w0BAQEFAASC # AgBcADrwv345T55HbxC/fexwGYKTq2M44xy6Ot48lPkA04qq7KhCm9NzTgi214mI # FrhFHNN5qTh6NQMEHgI4ZHaHPR0ETnlPaW9/L6s0f7MkacwG6/iQRzD3sjiMpm/A # y1fBCMyDTDsI51cCPut7A4n0lGiefajklUAcvSyFuK5Jp5BNGeoQpWJkpSccYNeo # cWUJhznvKI65b5A9iENZOD4cg+E/2Tqlnr6D2w+nL3Y7cA+zQABXXnjhz65DHsOv # Fsj73qfUmws6b6gk8KLpyE346oGxrgACO4ZTV+Z+rAeCXJFeaXy+VhG3PMV7TL4e # yI4gUIuIDMogUTRHk936DeN8VtpQA5tvo0k61n1cXrKZF8ebEWXzSC/h+03jmlV1 # zChXIHiz/CzC1G23Es4rbkXglidTJNdfQqWQicluu5caEw66+7Ph2efXpFXN5dsE # ioDAJTt/OpL1FW2JhQDNPANybWxQtN12N1lP5hbS3GPaEh0VYDHs+EX4Og9HWYAe # YoKQ45DFG1gO//lWGpx1bhxnZseKSLMg64I+GcEPs+mGV/hPvV+HoTuEe3lwzb+z # 6iQQPHjPI2Mo3NHnwPyfMpKfQViTVqCy7evmanQklYwgzF6Go8H8f2e0CPiIEkkw # VRPs7a+goU7ZF+151RGNTDNRS2TDNAMS+0Ek41HaIhRsJKGCF6wwgheoBgorBgEE # AYI3AwMBMYIXmDCCF5QGCSqGSIb3DQEHAqCCF4UwgheBAgEDMQ8wDQYJYIZIAWUD # BAIBBQAwggFZBgsqhkiG9w0BCRABBKCCAUgEggFEMIIBQAIBAQYKKwYBBAGEWQoD # ATAxMA0GCWCGSAFlAwQCAQUABCBIHjYwQD8UoY+KBjdYBcsQQoB4tsqUvP5XRFip # qXizQQIGaetgJC+VGBIyMDI2MDUwNjE1NDA1MC4zMVowBIACAfSggdmkgdYwgdMx # CzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRt # b25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xLTArBgNVBAsTJE1p # Y3Jvc29mdCBJcmVsYW5kIE9wZXJhdGlvbnMgTGltaXRlZDEnMCUGA1UECxMeblNo # aWVsZCBUU1MgRVNOOjM2MDUtMDVFMC1EOTQ3MSUwIwYDVQQDExxNaWNyb3NvZnQg # VGltZS1TdGFtcCBTZXJ2aWNloIIR+zCCBygwggUQoAMCAQICEzMAAAITsEM1Zs+v # legAAQAAAhMwDQYJKoZIhvcNAQELBQAwfDELMAkGA1UEBhMCVVMxEzARBgNVBAgT # Cldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29m # dCBDb3Jwb3JhdGlvbjEmMCQGA1UEAxMdTWljcm9zb2Z0IFRpbWUtU3RhbXAgUENB # IDIwMTAwHhcNMjUwODE0MTg0ODE3WhcNMjYxMTEzMTg0ODE3WjCB0zELMAkGA1UE # BhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAc # BgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEtMCsGA1UECxMkTWljcm9zb2Z0 # IElyZWxhbmQgT3BlcmF0aW9ucyBMaW1pdGVkMScwJQYDVQQLEx5uU2hpZWxkIFRT # UyBFU046MzYwNS0wNUUwLUQ5NDcxJTAjBgNVBAMTHE1pY3Jvc29mdCBUaW1lLVN0 # YW1wIFNlcnZpY2UwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQD0mXrg # uhnEMg1IWDP70pLk7O/mbnjx49XNz1FdZ7hPj8ymV+Brh6rXZEZ2nlxW+eN17m/F # +rZrH+Oe7u9Rbitk3iY5Sbm+H6RxixCVhDncXCAgHecSNxAeiasbeZl7+jOMVICv # oluCUq0h4DJI/MBwXPIB6vmUs1QcES9AwzwE6MzJqkK+HTGyDjEoVxUQlAsoR8IY # F98xkj9qa60cVvcJRNntpWkbYocQVQ2VnW/Awq/FdM9EOdvA8bPLKoknOd+ws0dD # i9e3a21LU94KgYjSE3U96rzIawhcz2ihzALToMY1Iz/gsDHa4q/CZSfo3AtzT62a # +fLrDbytkt6OyRF+dVah8S/WZZjSMdScevBIYFLyBU/2BwGzo/mDQ6kk8x/F1SQd # dGRww89bSEg/w1tbxblK6nwe7CdIpuOnICUYFR0z9XmtlvSxmaSfvXivpQsYr5ws # sA3pHcWFfo3SePrgXbstMrYFtLSkllpeOjR4M3PVBzF4gUtSAX5EGwtgOfwTxwKR # 7Erw2W3caL3Ml/nnDpR9Nn6TBMzEyoXGHv5N/Hv5oE5tn6fH3rUC2KoDLvNVXr2j # 8tZF0o9l29mf0RLIZtOc9+OQERG/bamtKUROVHDM/puYRU4pYtZXDG7CHttRZS5R # vVyP3fO+21BgZBq3kT0Assk2aW8soKyQHutouwIDAQABo4IBSTCCAUUwHQYDVR0O # BBYEFBOeEErH4WvKmFBYxGKkfj2wwUA6MB8GA1UdIwQYMBaAFJ+nFV0AXmJdg/Tl # 0mWnG1M1GelyMF8GA1UdHwRYMFYwVKBSoFCGTmh0dHA6Ly93d3cubWljcm9zb2Z0 # LmNvbS9wa2lvcHMvY3JsL01pY3Jvc29mdCUyMFRpbWUtU3RhbXAlMjBQQ0ElMjAy # MDEwKDEpLmNybDBsBggrBgEFBQcBAQRgMF4wXAYIKwYBBQUHMAKGUGh0dHA6Ly93 # d3cubWljcm9zb2Z0LmNvbS9wa2lvcHMvY2VydHMvTWljcm9zb2Z0JTIwVGltZS1T # dGFtcCUyMFBDQSUyMDIwMTAoMSkuY3J0MAwGA1UdEwEB/wQCMAAwFgYDVR0lAQH/ # BAwwCgYIKwYBBQUHAwgwDgYDVR0PAQH/BAQDAgeAMA0GCSqGSIb3DQEBCwUAA4IC # AQCCbFomsapDYPpQmFnpCXZJkU5o24ZtbcvMH4RL6XYEHUwm0FFIV2L+FVjfc2nG # wlCFDlMtWnQNdg6Qig9BzXusf4hWF6Y7yMK35TojVMjDpxHtz60Sj8mOnoSoRTVz # j+atoyOAeFD6toL85QCb3wDWvhsg8e2wGYtE4aZ4TlcsgVoEhlYe+HYI5chMo5td # V3nAa0nV1ll3BocAJcXnTqO1r66hR3LMB642VM8tOtnyfKHEbCT1WHp6INDsJAxZ # JJrwMlL09ReN6iL29N1Ltkxeq762/pDPfG2gEXn5gUri4T6aIaz3QXGbRUraVauY # WGORGXnPKgc53Abuyk1iQOiYI81Yi51RCZBgqm38eyyl9xv7GmdYgNB0zOATymPW # +nAuBYScfsu1Ph1kJ6gOj08rjRHEEPyQonvr2eCQTB/AIPYRf8xCTv14i86GmcfX # Ya5UHK9opmTldm+q08403Cvyr+oDfzvsi5bBaCdp5f6munDR1n9Au1sYZWuA/5NF # CO37Z1xkDk/dfgvAA2GI+zLQ6XhcJ2Ps7EEsW87OwI8M9pWeSn518MUb404GKvtq # pMnrzrbanKaDVX7qBz/VG/EL/CC9jIbTfd5wmq/Q6fRlE1iv6L86TCADcc/VosPR # oesSnDqW3TbreJGQK+tx1w5bzDeMLxMm5oZbILZL2MSPODCCB3EwggVZoAMCAQIC # EzMAAAAVxedrngKbSZkAAAAAABUwDQYJKoZIhvcNAQELBQAwgYgxCzAJBgNVBAYT # AlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYD # VQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xMjAwBgNVBAMTKU1pY3Jvc29mdCBS # b290IENlcnRpZmljYXRlIEF1dGhvcml0eSAyMDEwMB4XDTIxMDkzMDE4MjIyNVoX # DTMwMDkzMDE4MzIyNVowfDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0 # b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3Jh # dGlvbjEmMCQGA1UEAxMdTWljcm9zb2Z0IFRpbWUtU3RhbXAgUENBIDIwMTAwggIi # MA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDk4aZM57RyIQt5osvXJHm9DtWC # 0/3unAcH0qlsTnXIyjVX9gF/bErg4r25PhdgM/9cT8dm95VTcVrifkpa/rg2Z4VG # Iwy1jRPPdzLAEBjoYH1qUoNEt6aORmsHFPPFdvWGUNzBRMhxXFExN6AKOG6N7dcP # 2CZTfDlhAnrEqv1yaa8dq6z2Nr41JmTamDu6GnszrYBbfowQHJ1S/rboYiXcag/P # XfT+jlPP1uyFVk3v3byNpOORj7I5LFGc6XBpDco2LXCOMcg1KL3jtIckw+DJj361 # VI/c+gVVmG1oO5pGve2krnopN6zL64NF50ZuyjLVwIYwXE8s4mKyzbnijYjklqwB # Sru+cakXW2dg3viSkR4dPf0gz3N9QZpGdc3EXzTdEonW/aUgfX782Z5F37ZyL9t9 # X4C626p+Nuw2TPYrbqgSUei/BQOj0XOmTTd0lBw0gg/wEPK3Rxjtp+iZfD9M269e # wvPV2HM9Q07BMzlMjgK8QmguEOqEUUbi0b1qGFphAXPKZ6Je1yh2AuIzGHLXpyDw # wvoSCtdjbwzJNmSLW6CmgyFdXzB0kZSU2LlQ+QuJYfM2BjUYhEfb3BvR/bLUHMVr # 9lxSUV0S2yW6r1AFemzFER1y7435UsSFF5PAPBXbGjfHCBUYP3irRbb1Hode2o+e # FnJpxq57t7c+auIurQIDAQABo4IB3TCCAdkwEgYJKwYBBAGCNxUBBAUCAwEAATAj # BgkrBgEEAYI3FQIEFgQUKqdS/mTEmr6CkTxGNSnPEP8vBO4wHQYDVR0OBBYEFJ+n # FV0AXmJdg/Tl0mWnG1M1GelyMFwGA1UdIARVMFMwUQYMKwYBBAGCN0yDfQEBMEEw # PwYIKwYBBQUHAgEWM2h0dHA6Ly93d3cubWljcm9zb2Z0LmNvbS9wa2lvcHMvRG9j # cy9SZXBvc2l0b3J5Lmh0bTATBgNVHSUEDDAKBggrBgEFBQcDCDAZBgkrBgEEAYI3 # FAIEDB4KAFMAdQBiAEMAQTALBgNVHQ8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAf # BgNVHSMEGDAWgBTV9lbLj+iiXGJo0T2UkFvXzpoYxDBWBgNVHR8ETzBNMEugSaBH # hkVodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpL2NybC9wcm9kdWN0cy9NaWNS # b29DZXJBdXRfMjAxMC0wNi0yMy5jcmwwWgYIKwYBBQUHAQEETjBMMEoGCCsGAQUF # BzAChj5odHRwOi8vd3d3Lm1pY3Jvc29mdC5jb20vcGtpL2NlcnRzL01pY1Jvb0Nl # ckF1dF8yMDEwLTA2LTIzLmNydDANBgkqhkiG9w0BAQsFAAOCAgEAnVV9/Cqt4Swf # ZwExJFvhnnJL/Klv6lwUtj5OR2R4sQaTlz0xM7U518JxNj/aZGx80HU5bbsPMeTC # j/ts0aGUGCLu6WZnOlNN3Zi6th542DYunKmCVgADsAW+iehp4LoJ7nvfam++Kctu # 2D9IdQHZGN5tggz1bSNU5HhTdSRXud2f8449xvNo32X2pFaq95W2KFUn0CS9QKC/ # GbYSEhFdPSfgQJY4rPf5KYnDvBewVIVCs/wMnosZiefwC2qBwoEZQhlSdYo2wh3D # YXMuLGt7bj8sCXgU6ZGyqVvfSaN0DLzskYDSPeZKPmY7T7uG+jIa2Zb0j/aRAfbO # xnT99kxybxCrdTDFNLB62FD+CljdQDzHVG2dY3RILLFORy3BFARxv2T5JL5zbcqO # Cb2zAVdJVGTZc9d/HltEAY5aGZFrDZ+kKNxnGSgkujhLmm77IVRrakURR6nxt67I # 6IleT53S0Ex2tVdUCbFpAUR+fKFhbHP+CrvsQWY9af3LwUFJfn6Tvsv4O+S3Fb+0 # zj6lMVGEvL8CwYKiexcdFYmNcP7ntdAoGokLjzbaukz5m/8K6TT4JDVnK+ANuOaM # mdbhIurwJ0I9JZTmdHRbatGePu1+oDEzfbzL6Xu/OHBE0ZDxyKs6ijoIYn/ZcGNT # TY3ugm2lBRDBcQZqELQdVTNYs6FwZvKhggNWMIICPgIBATCCAQGhgdmkgdYwgdMx # CzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRt # b25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xLTArBgNVBAsTJE1p # Y3Jvc29mdCBJcmVsYW5kIE9wZXJhdGlvbnMgTGltaXRlZDEnMCUGA1UECxMeblNo # aWVsZCBUU1MgRVNOOjM2MDUtMDVFMC1EOTQ3MSUwIwYDVQQDExxNaWNyb3NvZnQg # VGltZS1TdGFtcCBTZXJ2aWNloiMKAQEwBwYFKw4DAhoDFQCYETxIKPGCNpybLz9U # R2Ts3GlHpqCBgzCBgKR+MHwxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5n # dG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9y # YXRpb24xJjAkBgNVBAMTHU1pY3Jvc29mdCBUaW1lLVN0YW1wIFBDQSAyMDEwMA0G # CSqGSIb3DQEBCwUAAgUA7aWvzTAiGA8yMDI2MDUwNjEyMTcxN1oYDzIwMjYwNTA3 # MTIxNzE3WjB0MDoGCisGAQQBhFkKBAExLDAqMAoCBQDtpa/NAgEAMAcCAQACAgMD # MAcCAQACAhPwMAoCBQDtpwFNAgEAMDYGCisGAQQBhFkKBAIxKDAmMAwGCisGAQQB # hFkKAwKgCjAIAgEAAgMHoSChCjAIAgEAAgMBhqAwDQYJKoZIhvcNAQELBQADggEB # ACmpzRrpCmZDSXkJOx3yN92Pkc/6TT4jeEKdIo3EwDv2wA2iYg0if+QL4gmXXbeK # dFjgzuZ2BPDSJNVSajjN4FwjDNkLWmPRxC4JQZhFc4qbezZHg3GTeHLtMDqehxIt # irnL97+kMGqEDVv7BCBIfzeIz4utdLz67WmvZxBxpzJsYBxRvb0CnNGQP95W1iaL # NE2QwJPEoUueS031/Tr3ChT2wtHsiJZxuG6FNhomyvwBUUifIIBGa0Wub7VV4wm8 # cyHnZrEOB6lBL3HYcQiBMaqruOosSBXHXlQlScBqpLkfjLZV+dZQfNrulYbgE+HV # hu2PaByTNA3q+MFoOnXWbp4xggQNMIIECQIBATCBkzB8MQswCQYDVQQGEwJVUzET # MBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMV # TWljcm9zb2Z0IENvcnBvcmF0aW9uMSYwJAYDVQQDEx1NaWNyb3NvZnQgVGltZS1T # dGFtcCBQQ0EgMjAxMAITMwAAAhOwQzVmz6+V6AABAAACEzANBglghkgBZQMEAgEF # AKCCAUowGgYJKoZIhvcNAQkDMQ0GCyqGSIb3DQEJEAEEMC8GCSqGSIb3DQEJBDEi # BCBYPuAl4VOACv/jfQ0FwN8DkrY7EjkB2cUMcBC74DfcpzCB+gYLKoZIhvcNAQkQ # Ai8xgeowgecwgeQwgb0EIMzhCW0UhTPwngOMDM/idWh1m9DFgaV5Qh+nzo5rnFho # MIGYMIGApH4wfDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAO # BgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEm # MCQGA1UEAxMdTWljcm9zb2Z0IFRpbWUtU3RhbXAgUENBIDIwMTACEzMAAAITsEM1 # Zs+vlegAAQAAAhMwIgQgiI37Eh/uYvB8HxK6mnk0lLVIFxvsZCUxoBkKsLvxnHIw # DQYJKoZIhvcNAQELBQAEggIAx6M416NPiNZ6SHdZ5nRy0Zh90w0mafRTximzPCYL # Y2AniQSkUlLP5OUyz+RoJJmV20aqL21M5n22iyLZjBSYmaHlTg0sTzWdrKLmM4tJ # nSGA3T3Nwq6iyMyNyMsXoNHnrR7+VKXlWxt87f8Jd3sr2avD0TNBZew/yQsuP68K # VKY3BfAiKxnk4ouKlTnbBAdUljoD59GOKCuNCDr6gYyqKNSbSgrVlZ5bgTuiwUvE # Gm+4s1q3xi6L367fnLBXrdq+UxdhvJ/Mukb2/iy9X8CeANhfj8kBZYhCBcqeQyqG # B2MJbPjPXlvs/lFRqGe/9uoSLOQPmLCorVejmGgyq/9aM7r7POOiTyyl+lFGJn1D # sq8NKoG6c1jBgibje6so0K8AU0uavRNVtDUE+2lUlZpLvolOxIbAK1VQzUyQtRWD # AEpMjZzVXEg7xtHFyxka32TucU91B9fbmVqMBXdIkTDz1aH34HnmviDtBdaxrKiG # 4RPw/OuAYiQN2g2PTHF8T0hLl0EvuNRrlBFJHxGhelMrI0lfeG6szaA/jMFl3Brp # LUDrnAzIOQ46U48h9JtpknfYh0f7BQfJ5ibgAc6dvDmzPsA1ikAKE9rasebA325O # l/fy2dru1QLbtRRA8Go0hpvK2sI0GPTQVk0RAu7oRGkyZ0h4aHa4urfz/fyBQraN # +7Q= # SIG # End signature block x

Windows NT KPTV 6.2 build 9200 (Windows Server 2012 Datacenter Edition) i586