Remoting, Background Jobs, and Parallelism
Run PowerShell work in the background with jobs, in parallel with thread jobs and ForEach-Object -Parallel, and on other machines over WinRM or SSH remoting.
By The Dark DebuggerUpdated 29 min read
Sooner or later an automation script runs into two problems. Some work takes too long when it's done one item at a time, and some work has to run on a machine other than the one running the script. This article covers both: background jobs (Start-Job), thread jobs (Start-ThreadJob), ForEach-Object -Parallel, and PowerShell remoting over WinRM and SSH. You should be comfortable writing functions and handling errors first. Everything here feeds into the capstone, Real-World PowerShell Scripts.
Why this matters
Picture a morning health check that visits 12 servers one after another. Each check spends about a second waiting on the network, so the script takes 12 seconds. At 200 servers it takes more than three minutes, and one host that hangs holds up everything behind it. When I ran the same 12 simulated checks four at a time, the run went from 12.0 seconds to 3.4 seconds. You'll see that measurement below.
The second problem is data that only exists on the server: free disk space, a service's log file, the state of a local process. If the server exposes an API you can call it (REST APIs). Often it doesn't, and you need to run PowerShell on the server itself. That's what remoting does.
Most examples below work on a folder of four application logs (api.log, auth.log, billing.log and worker.log, 9,500 lines in total). Each line has a timestamp, a level, a service name and a message:
Get-Content ./logs/api.log -TotalCount 22026-11-08T00:00:00Z INFO api GET /health 200 2ms
2026-11-08T00:00:01Z INFO api GET /api/orders 200 41msThree ways to run work concurrently
PowerShell has three built-in tools for running work concurrently. They differ in where the code runs, and that difference explains most of their behaviour:
Start-Job | Start-ThreadJob | ForEach-Object -Parallel | |
|---|---|---|---|
| Runs in | a new pwsh process per job | a thread in your process | threads in your process |
| Available in | 5.1 and 7 | 7 (built in) | 7.0 and later |
| Results come back as | deserialized copies | live objects | live objects |
| Default concurrency | unlimited | 5 (-ThrottleLimit) | 5 (-ThrottleLimit) |
| Best for | isolation, Windows PowerShell 5.1 | long-running background tasks | "do this for every item" |
Remoting adds a fourth option, Invoke-Command, which runs code on other machines and can fan out to many of them at once. We'll get to it after the local tools.
Background jobs with Start-Job
Start-Job runs a script block in a separate PowerShell process and returns immediately with a job object. Your prompt stays free while the job works. A job's life has three steps: wait for it, receive its output, remove it.
$job = Start-Job -Name ErrorScan -ScriptBlock {
Get-ChildItem ./logs/*.log | ForEach-Object {
[pscustomobject]@{
Log = $_.Name
Errors = (Select-String -Path $_.FullName -Pattern ' ERROR ').Count
}
}
}
$jobId Name PSJobTypeName State HasMoreData Location
-- ---- ------------- ----- ----------- --------
1 ErrorScan BackgroundJob Running True localhostWait-Job -Name ErrorScan blocks until the job finishes and prints the same row with State now Completed. Then you collect the output and clean up:
Receive-Job -Name ErrorScan | Select-Object Log, Errors
Remove-Job -Name ErrorScanLog Errors
--- ------
api.log 398
auth.log 104
billing.log 194
worker.log 257A few details matter here:
- Output is consumed when you receive it. When I called
Receive-Joba second time it returned nothing, andHasMoreDatahad flipped toFalse. UseReceive-Job -Keepif you need to read the output twice. - The
Select-Objectis deliberate. Received objects carry extra properties such asRunspaceId, and without it PowerShell switches to list view to show them all. - Finished jobs stay in
Get-Jobuntil you remove them, along with their output. A long-running session that never cleans up leaks memory.
For the common case of "start it, wait for it, give me the result, clean up", one pipeline covers all four steps:
$total = Start-Job { (Select-String -Path ./logs/*.log -Pattern ' ERROR ').Count } |
Receive-Job -Wait -AutoRemoveJob
"Total ERROR lines: $total"Total ERROR lines: 953A job is a whole new process
Each Start-Job launches a fresh pwsh process. That isolation is the point: a job that crashes or leaks memory can't take your session down with it. It also costs time, so I timed a job that does nothing:
"Caller PID: $PID"
Start-Job { "Job PID: $PID" } | Receive-Job -Wait -AutoRemoveJob
1..3 | ForEach-Object {
$t = Measure-Command { Start-Job { 1 } | Receive-Job -Wait -AutoRemoveJob }
'Start-Job round trip: {0:N0} ms' -f $t.TotalMilliseconds
}Caller PID: 2069
Job PID: 2117
Start-Job round trip: 502 ms
Start-Job round trip: 486 ms
Start-Job round trip: 532 msHalf a second of overhead per job is fine for a ten-minute export. It's a terrible fit for a thousand tiny tasks.
In PowerShell 7 a job starts in your current directory, which is why ./logs worked above, and -WorkingDirectory lets you choose another. Windows PowerShell 5.1 has no such parameter and starts jobs in your Documents folder, so use full paths in jobs that also have to run on 5.1.
Passing data in: $using: and -ArgumentList
The job's process has none of your variables. Referring to one silently gives you $null. Pass values in explicitly, either with the $using: scope modifier or with -ArgumentList and a param() block:
$pattern = ' WARN '
Start-Job { "Pattern inside the job: [$pattern]" } | Receive-Job -Wait -AutoRemoveJob
Start-Job { "Pattern inside the job: [$using:pattern]" } | Receive-Job -Wait -AutoRemoveJob
Start-Job -ArgumentList $pattern -ScriptBlock {
param($p)
"Pattern inside the job: [$p]"
} | Receive-Job -Wait -AutoRemoveJobPattern inside the job: []
Pattern inside the job: [ WARN ]
Pattern inside the job: [ WARN ]The first line is the bug you'll write at least once: no error, just an empty value. I use $using: for one or two values, and -ArgumentList with a named param() block when the script block is long enough to deserve a real signature.
Results come back as copies
Objects cross the process boundary by being serialized to XML and rebuilt on the other side, the same mechanism Export-Clixml uses (CSV, JSON, and XML). Properties survive, but methods don't:
$item = Start-Job { Get-Item ./logs/api.log } | Receive-Job -Wait -AutoRemoveJob
$item.pstypenames[0]
$item.Length
$item.OpenText()Deserialized.System.IO.FileInfo
241320
InvalidOperation:
Line |
4 | $item.OpenText()
| ~~~~~~~~~~~~~~~~
| Method invocation failed because [Deserialized.System.IO.FileInfo] does not contain a method named 'OpenText'.The Deserialized. prefix on the type name is the tell. Do the work that needs methods inside the job, and return plain data.
Job state and error handling
A job reports its outcome in State. The values you'll meet are NotStarted (queued behind a throttle limit), Running, Completed, Failed (a terminating error ended the script block) and Stopped (you called Stop-Job).
Completed doesn't mean "no errors". A non-terminating error is stored with the job and replayed when you receive it:
$job = Start-Job {
Get-Content ./logs/api.log, ./logs/payments.log | Measure-Object -Line
}
Wait-Job $job | Out-Null
$job.State
Receive-Job $job
Remove-Job $jobCompleted
Get-Content: Cannot find path '/home/sid/remoting/demo/logs/payments.log' because it does not exist.
Lines Words Characters Property
----- ----- ---------- --------
4200A terminating error (throw, or any error under -ErrorAction Stop) puts the job in Failed. The reason is stored on the child job, and Receive-Job -ErrorAction Stop turns it into something try/catch can handle:
$job = Start-Job {
$cfg = Get-Content ./deploy.json -ErrorAction Stop
"never reached"
}
Wait-Job $job | Out-Null
$job.State
$job.ChildJobs[0].JobStateInfo.Reason.Message
try {
Receive-Job $job -ErrorAction Stop
} catch {
"Caught: $($_.Exception.Message)"
}
Remove-Job $jobFailed
Cannot find path '/home/sid/remoting/demo/deploy.json' because it does not exist.
Caught: Cannot find path '/home/sid/remoting/demo/deploy.json' because it does not exist.In a script, check State for every job, or receive with -ErrorAction Stop inside try. Otherwise a failed job just produces no output, and your report quietly comes out short.
Timeouts and stuck jobs
Wait-Job -Timeout returns the job if it finished in time and nothing if it didn't, which lets you give up on a hung task:
$slow = Start-Job -Name SlowExport { Start-Sleep -Seconds 30; 'export done' }
$finished = Wait-Job $slow -Timeout 3
if (-not $finished) {
Write-Warning "SlowExport still $($slow.State) after 3 s; stopping it"
Stop-Job $slow
}
$slow.State
Remove-Job $slowWARNING: SlowExport still Running after 3 s; stopping it
StoppedRemove-Job refuses to delete a running job. It fails with "The command cannot remove the job with the job ID 11 because the job is not finished. To remove the job, first stop the job, or use the Force parameter." Remove-Job -Force stops and removes it in one step.
Thread jobs with Start-ThreadJob
Start-ThreadJob has the same interface (Get-Job, Wait-Job, Receive-Job, Remove-Job all work), but each job runs on a thread inside your current process. It ships with PowerShell 7 in the Microsoft.PowerShell.ThreadJob module. On Windows PowerShell 5.1 you'd install the ThreadJob module from the PowerShell Gallery; I didn't do that on the test machine.
"Caller PID: $PID"
Start-ThreadJob { "Thread job PID: $PID" } | Receive-Job -Wait -AutoRemoveJob
$fi = Start-ThreadJob { Get-Item ./logs/api.log } | Receive-Job -Wait -AutoRemoveJob
$fi.GetType().FullNameCaller PID: 2069
Thread job PID: 2069
System.IO.FileInfoThere's one process and no serialization, so you get a live FileInfo back with all its methods. The biggest difference I measured was memory. I started four sleeping jobs of each kind and measured them two seconds in:
$jobs = 1..4 | ForEach-Object { Start-Job { Start-Sleep -Seconds 4 } }
Start-Sleep -Seconds 2
$children = Get-Process pwsh | Where-Object { $_.Parent.Id -eq $PID }
'{0} child pwsh processes using {1:N0} MB' -f $children.Count, (($children | Measure-Object WorkingSet64 -Sum).Sum / 1MB)
$jobs | Remove-Job -Force
$before = (Get-Process -Id $PID).WorkingSet64
$jobs = 1..4 | ForEach-Object { Start-ThreadJob { Start-Sleep -Seconds 4 } }
Start-Sleep -Seconds 2
'4 thread jobs added {0:N0} MB to this process' -f (((Get-Process -Id $PID).WorkingSet64 - $before) / 1MB)
$jobs | Remove-Job -Force4 child pwsh processes using 428 MB
4 thread jobs added 9 MB to this processStartup time was less clear-cut than people often claim. A single thread job round trip took about 240 ms after the first one, against 500–650 ms for Start-Job. But when I started four of each at once, three times over, the four thread jobs were consistently slower: 868–1,088 ms, against 552–682 ms for four process jobs. That's probably because four runspaces initialising inside one process compete with each other, while four separate processes don't. Choose thread jobs for memory and live objects, not for raw startup speed.
Their throttle limit (default 5) works as a queue. With -ThrottleLimit 2 and six two-second jobs, grouping the jobs by State one second in showed two Running and four NotStarted. The rest started as slots freed up, and all six finished.
ForEach-Object -Parallel
For "do this to every item", PowerShell 7's ForEach-Object -Parallel is the tool you'll use most. It runs the script block for each pipeline item on a pool of threads, -ThrottleLimit at a time (default 5), and streams results back into the pipeline.
Here are the 12 health checks from the introduction. Start-Sleep -Seconds 1 stands in for a network call that spends a second waiting on a remote host. That's the I/O-bound kind of work where parallelism pays off most.
$servers = 'web01','web02','web03','web04','api01','api02',
'db01','db02','cache01','cache02','queue01','queue02'
$seq = Measure-Command {
$servers | ForEach-Object {
Start-Sleep -Seconds 1 # stands in for a 1-second network check
[pscustomobject]@{ Server = $_; Status = 'OK' }
}
}
$par = Measure-Command {
$servers | ForEach-Object -ThrottleLimit 4 -Parallel {
Start-Sleep -Seconds 1
[pscustomobject]@{ Server = $_; Status = 'OK' }
}
}
'Sequential: {0:N1} s' -f $seq.TotalSeconds
'Parallel (4 at a time): {0:N1} s' -f $par.TotalSecondsSequential: 12.0 s
Parallel (4 at a time): 3.4 sThat's 12 seconds of waiting split into three rounds of four, plus about 0.4 s of thread and runspace setup. The ceiling is items / ThrottleLimit rounds. For CPU-bound work the ceiling is also your core count, because threads beyond that just take turns. Choose the throttle limit for the system on the other end, not for your laptop: twenty parallel requests against a small internal API is a self-inflicted outage.
Order, $using:, -AsJob and -TimeoutSeconds
Output arrives in completion order, not input order. I piped in db01, web01, cache01 and api01 with simulated delays of 900, 100, 500 and 300 ms, and the answers came back as web01, api01, cache01, db01. If order matters, return objects that carry their own key and Sort-Object afterwards.
Each iteration runs in a fresh runspace, so the $using: rule from jobs applies unchanged. With $timeoutMs = 750 set in the caller, a plain $timeoutMs inside the block came out empty, and $using:timeoutMs came out as 750.
Two more switches are worth knowing:
-AsJobwraps the whole run in one job, so you can keep working and collect the results later. For the 12 servers I got a singlePSTaskJobwith 12 child jobs, andReceive-Job -Wait -AutoRemoveJobreturned all 12 results.-TimeoutSecondscaps the whole run, not each item. When I gave three items that sleep 1.5, 3 and 4.5 seconds a 2-second limit, the first finished and the other two were stopped, each writingInvalidOperation: The pipeline has been stopped.
Thread safety: sharing state between threads
In a process job, $using: hands the job a copy of the value. In thread jobs and -Parallel, $using: with a reference type (a hashtable, a list, any .NET object) hands every thread the same object. When I added keys to ($using:seen)[$_] from a -Parallel block, they appeared in the caller's hashtable. The same write inside a Start-Job changed only the job's copy, and the caller's hashtable stayed empty.
That shared object is where the trouble starts. Ordinary .NET collections aren't safe to modify from several threads at once. Here, four threads each add 50,000 numbers to a shared List[int]:
$list = [System.Collections.Generic.List[int]]::new()
1..4 | ForEach-Object -ThrottleLimit 4 -Parallel {
$shared = $using:list
foreach ($i in 1..50000) { $shared.Add($i) }
}
"Expected 200000, got $($list.Count)"Expected 200000, got 191246A second run gave 193929. There's no error and no warning, just missing data, and a different amount each run. Swapping in a collection built for concurrent writers, [System.Collections.Concurrent.ConcurrentBag[int]], and running the identical loop printed Expected 200000, got 200000.
A concurrent collection doesn't make your read-modify-write sequence safe, though. Counting log levels in a ConcurrentDictionary still loses updates, because two threads can read the same old value before either one writes:
$counts = [System.Collections.Concurrent.ConcurrentDictionary[string,int]]::new()
Get-ChildItem ./logs/*.log | ForEach-Object -ThrottleLimit 4 -Parallel {
$c = $using:counts
foreach ($line in [System.IO.File]::ReadLines($_.FullName)) {
$level = $line.Split(' ')[1]
$c[$level] = $c[$level] + 1
}
}
$counts.GetEnumerator() | Sort-Object Key | Format-Table Key, ValueThat printed ERROR 947, INFO 6866 and WARN 1523. The real totals are 953, 7014 and 1533. Replacing the increment with the atomic $null = $c.AddOrUpdate($level, 1, { param($key, $old) $old + 1 }) produced exactly those numbers.
My recommendation is to avoid shared state altogether. Have each iteration build its own local result, output it, and combine everything once all the threads are done. Nothing is shared, so nothing can race:
$perFile = Get-ChildItem ./logs/*.log | ForEach-Object -ThrottleLimit 4 -Parallel {
$local = @{}
foreach ($line in [System.IO.File]::ReadLines($_.FullName)) {
$level = $line.Split(' ')[1]
$local[$level] = $local[$level] + 1
}
foreach ($k in $local.Keys) {
[pscustomobject]@{ Log = $_.Name; Level = $k; Count = $local[$k] }
}
}
$perFile | Group-Object Level | ForEach-Object {
[pscustomobject]@{
Level = $_.Name
Count = [int]($_.Group | Measure-Object Count -Sum).Sum
}
} | Sort-Object LevelLevel Count
----- -----
ERROR 953
INFO 7014
WARN 1533PowerShell remoting
Remoting runs a script block in a PowerShell session on another machine and streams the results back as deserialized objects, just like jobs. There are two transports, and the parameter you pass decides which one you get:
| WinRM (WS-Management) | SSH | |
|---|---|---|
| Parameter | -ComputerName | -HostName |
| Port | 5985 (HTTP), 5986 (HTTPS) | 22 |
| Authentication | Kerberos in a domain, NTLM, certificates | SSH keys or password |
| Server side | Windows, with the WinRM service | any OS running sshd plus PowerShell 7 |
| Turned on by | Enable-PSRemoting (as admin) | a Subsystem powershell line in sshd_config |
| Credentials | -Credential | -UserName, -KeyFilePath |
| Fan-out throttle | -ThrottleLimit, default 32 | not exposed |
WinRM is the traditional choice inside a Windows domain, and Windows Server has it enabled by default. On a Windows client it usually isn't running: on the Windows 11 machine I wrote this on, Get-Service WinRM reported Stopped with start type Manual. Windows PowerShell 5.1 speaks only WinRM. I checked with Get-Command there, and it has no -HostName parameter, no ForEach-Object -Parallel and no Start-ThreadJob.
Turning remoting on
On a Windows server you want to manage over WinRM, run Enable-PSRemoting -Force once from an elevated prompt. I didn't run it: it reconfigures the service and the firewall, and the machine I wrote on was off limits for changes. For SSH remoting, the target needs sshd and PowerShell 7, plus one line in sshd_config that tells sshd how to start PowerShell as a subsystem. Restart sshd after adding it. On Linux:
Subsystem powershell /usr/bin/pwsh -sshs -NoLogoOn Windows, with OpenSSH Server installed, the path must not contain spaces, so Microsoft's documentation uses the 8.3 short name: Subsystem powershell c:/progra~1/powershell/7/pwsh.exe -sshs -NoLogo.
How I tested the SSH examples
I didn't have a second machine. Instead I ran a throwaway, unprivileged OpenSSH 9.6 sshd on 127.0.0.1 port 2222, with its own host key, a dedicated client key, and a Subsystem powershell line pointing at a two-line wrapper script that runs PowerShell 7.6.9 with -sshs. I deleted all of it afterwards. To keep the examples readable, I set the test-only connection details once with $PSDefaultParameterValues, so every remoting command below also received them:
$sshOpts = @{ UserKnownHostsFile = $env:TEST_KNOWN_HOSTS; StrictHostKeyChecking = 'accept-new' }
foreach ($cmd in 'Invoke-Command', 'New-PSSession', 'Enter-PSSession') {
$PSDefaultParameterValues["${cmd}:Port"] = 2222
$PSDefaultParameterValues["${cmd}:KeyFilePath"] = $env:TEST_SSH_KEY
$PSDefaultParameterValues["${cmd}:Options"] = $sshOpts
}Against a normal server on port 22 with your key in ~/.ssh, you can drop all three. Scope $PSDefaultParameterValues entries to specific cmdlets as shown. My first attempt used '*:Port', which also bound -Port to Get-PSSession and broke it with "Parameter set cannot be resolved".
Invoke-Command: run a script block remotely
Invoke-Command -HostName localhost -UserName sid -ScriptBlock {
"Running as $([Environment]::UserName) in PowerShell $($PSVersionTable.PSVersion), PID $PID"
}Warning: Permanently added '[localhost]:2222' (ED25519) to the list of known hosts.
Running as sid in PowerShell 7.6.9, PID 3712The warning comes from the ssh client the first time it sees a host key. With default settings you'd get an interactive "Are you sure you want to continue connecting?" prompt instead. Answer it once, interactively, before you put a host into an unattended script.
Local variables don't exist on the remote side; $using: and -ArgumentList work exactly as they did for jobs. Errors on the remote side come back as normal errors in your session, so -ErrorAction, -ErrorVariable and try/catch all work.
$logDir = '~/remoting/demo/logs'
$pattern = ' ERROR '
Invoke-Command -HostName localhost -ScriptBlock {
Select-String -Path (Join-Path $using:logDir '*.log') -Pattern $using:pattern |
Group-Object Filename -NoElement
} | Select-Object Name, Count
Invoke-Command -HostName localhost -ScriptBlock { Get-Item /etc/no-such-file.conf }Get-Item: Cannot find path '/etc/no-such-file.conf' because it does not exist.
Name Count
---- -----
api.log 398
auth.log 104
billing.log 194
worker.log 257The error appears first because PowerShell prints the error stream immediately but holds table output briefly to work out column widths. Notice also that Group-Object ran remotely, so only four small summary objects crossed the network. Filter and aggregate on the remote side. Shipping thousands of raw objects back to filter locally is the classic slow remoting script.
Enter-PSSession for interactive work
Enter-PSSession -HostName web01 -UserName deploy is the interactive version: your prompt changes to show the remote computer's name, every command you type runs there, and exit brings you back. It needs a real interactive terminal, and I couldn't drive one from my test harness, so that command wasn't executed for this article. Use it for poking around and diagnosing. Anything automated belongs in Invoke-Command.
Reusing a session with New-PSSession
Each Invoke-Command -HostName call opens a connection, starts a remote pwsh, runs the block and tears everything down. New-PSSession keeps one session open, so you pay the setup once, and variables persist between calls:
$s = New-PSSession -HostName localhost
$s | Format-Table Id, Name, Transport, ComputerName, State
Invoke-Command -Session $s -ScriptBlock { $startedAt = Get-Date -Format HH:mm:ss; $count = 0 }
Invoke-Command -Session $s -ScriptBlock { $count++; "count=$count, session started $startedAt" }
Invoke-Command -Session $s -ScriptBlock { $count++; "count=$count, session started $startedAt" }Id Name Transport ComputerName State
-- ---- --------- ------------ -----
2 Runspace1 SSH localhost Opened
count=1, session started 10:26:57
count=2, session started 10:26:57The difference in speed is large, even over loopback:
$fresh = Measure-Command {
1..3 | ForEach-Object { Invoke-Command -HostName localhost -ScriptBlock { 1 } }
}
$reused = Measure-Command {
1..3 | ForEach-Object { Invoke-Command -Session $s -ScriptBlock { 1 } }
}
'3 calls, new connection each: {0,5:N0} ms' -f $fresh.TotalMilliseconds
'3 calls, one reused session: {0,5:N0} ms' -f $reused.TotalMilliseconds
Remove-PSSession $s
"Open sessions: $(@(Get-PSSession).Count)"3 calls, new connection each: 1,219 ms
3 calls, one reused session: 33 ms
Open sessions: 0Sessions hold a process open on the server, so close them when you're done. In real scripts, put the Remove-PSSession in a finally block.
Fanning out to many machines
Pass several names and Invoke-Command connects to them concurrently, tagging every result with PSComputerName. When I passed -HostName localhost, 127.0.0.1, I got two results from two separate remote pwsh processes (PIDs 4182 and 4188), one per connection, even though both names point at the same machine. The fleet report in the next section is a complete fan-out with error handling. With WinRM the same call is Invoke-Command -ComputerName (Get-Content ./servers.txt) -ThrottleLimit 50 -ScriptBlock { … }, which scales to hundreds of servers. I couldn't run that one without a Windows domain.
One WinRM-specific trap is the "second hop". Inside a remote session, your credentials can't by default be passed on to a third machine, such as a file share. The symptom is access denied on a share that works fine from your own desktop. The usual fixes are resource-based Kerberos delegation, or passing an explicit -Credential to the inner command. SSH key authentication has a similar limit, because the remote session holds no password or ticket of yours that it could pass on.
Where you'll use this
These tools turn up wherever one machine has to drive many: patch and inventory reports across a server fleet, nightly log collection, pre-deployment checks ("is there enough disk on every web node?"), and bulk changes such as rotating a config value on fifty servers. Thread jobs and -Parallel turn up wherever a script spends its time waiting: checking endpoints, calling an API once per item, copying to several destinations.
Here's a complete fleet disk report. It runs one script block on every server, records which servers it couldn't reach instead of letting them vanish from the report, and writes a CSV. db99.invalid is deliberately unresolvable, to show the failure path.
$servers = 'localhost', '127.0.0.1', 'db99.invalid'
$results = Invoke-Command -HostName $servers -ErrorAction SilentlyContinue -ErrorVariable connectErrors -ScriptBlock {
$root = [System.IO.DriveInfo]::new('/')
[pscustomobject]@{
SizeGB = [math]::Round($root.TotalSize / 1GB, 1)
FreeGB = [math]::Round($root.AvailableFreeSpace / 1GB, 1)
FreePct = [math]::Round(100 * $root.AvailableFreeSpace / $root.TotalSize, 1)
UptimeHours = [math]::Round((Get-Uptime).TotalHours, 1)
}
}
$report = @(
foreach ($r in $results) {
[pscustomobject]@{
Server = $r.PSComputerName
Status = if ($r.FreePct -lt 15) { 'LOW DISK' } else { 'OK' }
FreeGB = $r.FreeGB
FreePct = $r.FreePct
UptimeHours = $r.UptimeHours
}
}
foreach ($e in $connectErrors) {
[pscustomobject]@{ Server = "$($e.TargetObject)"; Status = 'UNREACHABLE' }
}
)
$report | Sort-Object Server | Format-Table
$report | Export-Csv ./fleet-disk.csv -NoTypeInformation
Get-Content ./fleet-disk.csvssh: Could not resolve hostname db99.invalid: Name or service not known
Server Status FreeGB FreePct UptimeHours
------ ------ ------ ------- -----------
127.0.0.1 OK 951.50 94.50 0.10
db99.invalid UNREACHABLE
localhost OK 951.50 94.50 0.10
"Server","Status","FreeGB","FreePct","UptimeHours"
"127.0.0.1","OK","951.5","94.5","0.1"
"localhost","OK","951.5","94.5","0.1"
"db99.invalid","UNREACHABLE",,,The design choices are the important part:
- All the computation happens remotely. Only four numbers per server cross the network.
-ErrorVariablecollects connection failures while the healthy servers carry on. Each error'sTargetObjectis the host name, which is what makes theUNREACHABLErows possible. A report that silently drops broken servers looks like good news.- The output is plain objects, so the same
$reportfeedsFormat-Table,Export-Csv, orConvertTo-Jsonfor a dashboard.
In a scheduled version you'd finish with something like if ($report.Status -contains 'UNREACHABLE') { exit 1 }, so the scheduler marks the run as failed. The health monitor in Real-World PowerShell Scripts uses the same exit-code pattern.
Common mistakes
Assigning to a $using: variable
Collecting results by appending to an outer array looks natural, and it doesn't even parse:
$found = @()
Get-ChildItem ./logs/*.log | ForEach-Object -Parallel {
$using:found += $_.Name
}ParserError: /home/sid/remoting/demo/collect-names.ps1:3
Line |
3 | $using:found += $_.Name
| ~~~~~~~~~~~~
| The assignment expression is not valid. The input to an assignment
| operator must be an object that is able to accept assignments, such as a
| variable or a property.$using: values are read-only by design. Assign the pipeline's output instead: $found = Get-ChildItem ./logs/*.log | ForEach-Object -Parallel { $_.Name }. This is the same "output, don't share" rule from the thread-safety section.
Calling your own function inside -Parallel
function Test-Server {
param([string]$Name)
[pscustomobject]@{ Server = $Name; Status = 'OK' }
}
'web01' | ForEach-Object -Parallel { Test-Server -Name $_ }Test-Server: The term 'Test-Server' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.Each iteration's runspace starts empty. You have two fixes. One is to put the function in a module and Import-Module it inside the block (Scripts, Scope, and Modules). The other is to pass the function's definition in as a string and recreate it:
$funcDef = ${function:Test-Server}.ToString()
'web01','web02' | ForEach-Object -ThrottleLimit 4 -Parallel {
${function:Test-Server} = $using:funcDef
Test-Server -Name $_
}Server Status
------ ------
web01 OK
web02 OKThe same applies to Start-Job, Start-ThreadJob and remote sessions: none of them can see your session's functions.
Parallelising tiny work
-Parallel has a per-item cost for setting up the runspace and marshalling results. When the work per item is trivial, that cost is all you measure:
$plain = Measure-Command { $r1 = 1..1000 | ForEach-Object { $_ * 2 } }
$para = Measure-Command { $r2 = 1..1000 | ForEach-Object -ThrottleLimit 4 -Parallel { $_ * 2 } }
'Plain ForEach-Object: {0,7:N0} ms' -f $plain.TotalMilliseconds
'ForEach-Object -Parallel: {0,6:N0} ms' -f $para.TotalMillisecondsPlain ForEach-Object: 3 ms
ForEach-Object -Parallel: 12,102 msThat's about 12 ms of overhead per item on my test machine. Parallelise when each item spends a meaningful amount of time waiting (network calls, remote commands, slow disks) or doing heavy CPU work. If you have thousands of tiny items, split them into a few batches and process each batch with a plain loop inside one parallel iteration.
Summary
Start-Jobruns a script block in a new process: isolated, available on 5.1, and about half a second and over 100 MB per job in my tests. Its results come back as deserialized copies, with properties but without methods.- Every job follows the same lifecycle:
Wait-Job,Receive-Job(output is consumed unless you pass-Keep),Remove-Job.Receive-Job -Wait -AutoRemoveJobdoes all three. Completedcan still include non-terminating errors. Terminating errors giveFailed, with the reason inChildJobs[0].JobStateInfo.Reason. UseWait-Job -TimeoutplusStop-Jobfor hung work.Start-ThreadJobandForEach-Object -Parallelrun on threads in your process. They're cheap in memory and return live objects, but they share any reference-type object you pass with$using:.- Script blocks in jobs, threads and remote sessions can't see your variables or functions. Use
$using:or-ArgumentList, and never assign to$using:. - Don't mutate shared collections from parallel code. Output results from each iteration and aggregate afterwards, or use
System.Collections.Concurrenttypes with atomic methods such asAddOrUpdate. -ComputerNamemeans WinRM (Windows, Kerberos,Enable-PSRemoting);-HostNamemeans SSH (any OS, keys,Subsystem powershellinsshd_config). Filter remotely, and reuse sessions withNew-PSSessionwhen you make repeated calls.- When fanning out, capture failures with
-ErrorVariableand report unreachable hosts explicitly.
Exercises
Start one Start-Job per file in ./logs, named after the file's base name. Each job returns the file's line count. Wait for all of them, print a table of log name and line count, and remove every job so that Get-Job is empty afterwards.
Show Solution
Get-ChildItem ./logs/*.log | ForEach-Object {
Start-Job -Name $_.BaseName -ArgumentList $_.FullName -ScriptBlock {
param($Path)
(Get-Content $Path).Count
}
} | Out-Null
Get-Job | Wait-Job | ForEach-Object {
[pscustomobject]@{ Log = $_.Name; Lines = Receive-Job $_ }
}
Get-Job | Remove-Job
"Jobs left: $(@(Get-Job).Count)"Log Lines
--- -----
api 4200
auth 900
billing 1800
worker 2600
Jobs left: 0Passing the full path with -ArgumentList keeps the job independent of its working directory, which matters if the same script also has to run on Windows PowerShell 5.1.
Check the eight endpoints in this table with ForEach-Object -Parallel, four at a time. Each check sleeps for its endpoint's latency (in milliseconds) and returns Endpoint, LatencyMs and Status, which is Slow above 800 ms and OK otherwise. Sort the results slowest first, and report how long the run took compared with the sum of all the latencies.
$latency = @{
'web01' = 120; 'web02' = 950; 'api01' = 300; 'api02' = 1100
'db01' = 450; 'db02' = 200; 'cache01' = 60; 'queue01' = 700
}Show Solution
$elapsed = Measure-Command {
$report = $latency.Keys | ForEach-Object -ThrottleLimit 4 -Parallel {
$ms = ($using:latency)[$_]
Start-Sleep -Milliseconds $ms # simulated network round trip
[pscustomobject]@{
Endpoint = $_
LatencyMs = $ms
Status = if ($ms -gt 800) { 'Slow' } else { 'OK' }
}
}
}
$report | Sort-Object LatencyMs -Descending | Format-Table
'Finished in {0:N1} s; one at a time would take {1:N1} s' -f $elapsed.TotalSeconds,
(($latency.Values | Measure-Object -Sum).Sum / 1000)Endpoint LatencyMs Status
-------- --------- ------
api02 1100 Slow
web02 950 Slow
queue01 700 OK
[…]
cache01 60 OK
Finished in 1.6 s; one at a time would take 3.9 sThe lookup table only gets read, so sharing it through $using: is safe. The results are collected by assigning the pipeline's output, not by appending to a shared list. The run can't finish faster than its slowest endpoint (1.1 s) plus the setup cost.
Write a function Invoke-WithTimeout that takes -ScriptBlock, -TimeoutSeconds and an optional -ArgumentList, and runs the block as a thread job. If the job finishes in time, return its output and rethrow any terminating error. If it doesn't, stop it and throw a System.TimeoutException. Either way, no job may be left behind. Test it with a block that finishes in 1 s under a 3 s limit, and one that would take 10 s under a 2 s limit.
Show Solution
function Invoke-WithTimeout {
[CmdletBinding()]
param(
[Parameter(Mandatory)] [scriptblock] $ScriptBlock,
[Parameter(Mandatory)] [int] $TimeoutSeconds,
[object[]] $ArgumentList = @()
)
$job = Start-ThreadJob -ScriptBlock $ScriptBlock -ArgumentList $ArgumentList
try {
if (-not (Wait-Job $job -Timeout $TimeoutSeconds)) {
Stop-Job $job
throw [System.TimeoutException]::new("Gave up after $TimeoutSeconds s")
}
Receive-Job $job -ErrorAction Stop
}
finally {
Remove-Job $job -Force
}
}
Invoke-WithTimeout -TimeoutSeconds 3 -ArgumentList 'billing' -ScriptBlock {
param($Service)
Start-Sleep -Seconds 1
"$Service export finished"
}
$t = Measure-Command {
try {
Invoke-WithTimeout -TimeoutSeconds 2 -ScriptBlock { Start-Sleep -Seconds 10; 'never' }
} catch {
Write-Host "Caught $($_.Exception.GetType().Name): $($_.Exception.Message)"
}
}
'Returned after {0:N1} s' -f $t.TotalSeconds
"Jobs left: $(@(Get-Job).Count)"billing export finished
Caught TimeoutException: Gave up after 2 s
Returned after 2.0 s
Jobs left: 0The finally block is what guarantees cleanup on every path: success, timeout, and an error rethrown by Receive-Job -ErrorAction Stop. A thread job is the right vehicle here because starting one is cheap, and Stop-Job interrupted the Start-Sleep immediately.
Tested with: PowerShell 7.6.9 on Ubuntu 24.04 (WSL2), November 2026, with read-only checks in Windows PowerShell 5.1.26100 on Windows 11. Every job, thread-job, -Parallel and exercise example was run as shown, against the four sample logs. The 12-server and latency checks simulate network waits with Start-Sleep, and every timing is from my 8-core test machine, so yours will differ. The SSH remoting examples ran against a throwaway, unprivileged OpenSSH 9.6 sshd on 127.0.0.1:2222, so "localhost" and "127.0.0.1" are the same machine. Enter-PSSession wasn't executed because it needs an interactive terminal. The WinRM commands (Enable-PSRemoting, Invoke-Command -ComputerName) also weren't executed: the Windows machine was limited to read-only commands and there was no domain to test against. One surprise: four thread jobs started at once were slower to finish than four process jobs. Their advantage showed up in memory (9 MB against 428 MB), not in startup time. Paths from the test machine's scratch directories are shown shortened to /home/sid/….
Written by The Dark Debugger
Engineer documenting the things that took me too long to learn, so they take you less time. Every PowerShell tutorial here is tested end to end before publishing.