Regular Expressions
Match, extract, and replace text with .NET regular expressions in PowerShell: -match, -replace, Select-String and [regex], tested on real log files.
By The Dark DebuggerUpdated 28 min read
PowerShell's pipeline works best with objects, but a lot of what you need to automate
arrives as plain text: web server logs, sshd auth logs, the output of native tools,
files written by systems that predate JSON. Regular expressions are how you turn those
lines into objects. This article covers the .NET regex dialect and every place
PowerShell uses it: -match, -replace, -split, switch -Regex, Select-String and
the [regex] class. It assumes you know the comparison operators
and string operators, and it follows
CSV, JSON, and XML, which covers the text
that does have structure.
Why this matters
Picture an nginx access log on a busy API server. Someone asks: "Which endpoints are
returning 5xx errors, how slow are they, and who is scanning us for .env files?" Each
line looks like this:
198.51.100.23 - - [09/Nov/2026:14:02:20 +0000] "POST /api/payments HTTP/1.1" 502 166 "-" "python-requests/2.32.3" 30.001Splitting on spaces doesn't work: the user agent contains spaces, the timestamp
contains a space, and the URL may or may not have a query string. A regular expression
describes the shape of the line, pulls out each field by name, and hands you an
object. From there it's ordinary Group-Object and Where-Object. We'll build exactly
that parser in Where you'll use this. First, the building
blocks.
The sample files used throughout (app.log, access.log, auth.log, commits.txt)
are shown where they're first used. Here's app.log, the one you'll see most:
2026-11-09 14:01:58.120 INFO [orders] Started order sync batch=8812
2026-11-09 14:02:03.441 WARN [orders] Slow query took 2.4s table=order_items
2026-11-09 14:02:05.002 ERROR [payments] Payment gateway timeout after 30s order=A-10293
2026-11-09 14:02:05.003 ERROR [payments] System.TimeoutException: The operation has timed out.
at Payments.Gateway.Charge(Order order)
at Orders.Sync.Run()
2026-11-09 14:02:09.778 INFO [orders] Retrying order=A-10293 attempt=2
2026-11-09 14:02:11.010 INFO [orders] Completed order sync batch=8812 ok=41 failed=1
2026-11-09 14:03:00.000 ERROR [inventory] Connection refused host=10.0.2.21 port=5432Where PowerShell uses regex, and where it doesn't
PowerShell uses the .NET regular expression engine (System.Text.RegularExpressions),
the same one C# uses. It's a Perl-style dialect with named groups, lookbehind and
lazy quantifiers. Knowing which commands take a regex and which take a literal string
saves you from the most common bugs:
| Takes a regex | Takes a literal or wildcard |
|---|---|
-match, -notmatch, -cmatch | -like, -notlike (wildcards *, ?) |
-replace, -creplace | .Replace() method (literal text) |
-split (by default) | .Split() method (literal characters) |
switch -Regex | switch -Wildcard, plain switch |
Select-String -Pattern | Select-String -SimpleMatch |
[regex]::Match(), Matches(), Replace(), Split() | .Contains(), .StartsWith(), .IndexOf() |
[ValidatePattern()] on parameters | [ValidateSet()] |
The PowerShell operators are case-insensitive by default; prefix them with c
(-cmatch, -creplace, -csplit) for case-sensitive matching. The [regex] methods
are the opposite, and that difference catches people (we'll see it below).
Always write patterns in single quotes. In double quotes PowerShell expands $
variables before the regex engine ever sees the pattern, and $ is a regex anchor.
The .NET regex essentials
This is the subset you'll use 95% of the time:
| Token | Matches |
|---|---|
. | Any character except a newline |
\d, \w, \s | A digit, a word character (letter, digit, _), whitespace |
\D, \W, \S | The opposite of each of the above |
[abc], [a-z], [^"] | One character from a set; ^ inside brackets negates the set |
*, +, ? | Zero or more, one or more, zero or one of the previous item |
{3}, {1,3}, {2,} | Exactly 3, between 1 and 3, 2 or more |
*?, +? | Lazy versions: as few as possible |
^, $ | Start and end of the string (of each line with the m option) |
\b | A word boundary |
(...) | A numbered capture group |
(?<name>...) | A named capture group |
(?:...) | A group that doesn't capture |
a|b | Either a or b |
\., \[, \\ | A literal ., [, \; escape any metacharacter with \ |
One .NET-specific detail: \d matches any Unicode decimal digit, not only 0–9:
'٣' -match '\d'
'٣' -match '[0-9]'True
FalseThat's an Arabic-Indic three. If you're validating input that must be ASCII digits (an
employee number, a port), use [0-9].
-match and $Matches
-match returns True or False, and on success it fills the automatic variable
$Matches with what it found:
$line = '2026-11-09 14:02:05.002 ERROR [payments] Payment gateway timeout after 30s order=A-10293'
$line -match 'ERROR'
$line -match 'error'
$line -cmatch 'error'
$line -match 'order=(A-\d+)'
$MatchesTrue
True
False
True
Name Value
---- -----
1 A-10293
0 order=A-10293$Matches[0] is the whole match and $Matches[1] is the first capture group. Three
rules to remember:
-matchfinds the first match only. For all of them, useSelect-String -AllMatchesor[regex]::Matches().$Matchesis only updated when a match succeeds, so a failed match leaves the old value behind.- When the left side is an array,
-matchfilters the array and doesn't touch$Matchesat all.
The last two cause real bugs, and both are demonstrated in Common mistakes.
Named groups turn lines into objects
Numbered groups get confusing past two or three. Name them with (?<name>...) and read
them as $Matches.name:
$line = '2026-11-09 14:02:05.002 ERROR [payments] Payment gateway timeout after 30s order=A-10293'
$pattern = '^(?<date>\S+) (?<time>\S+) (?<level>[A-Z]+)\s+\[(?<component>[^\]]+)\] (?<message>.*)$'
if ($line -match $pattern) {
[pscustomobject]@{
Time = [datetime]"$($Matches.date) $($Matches.time)"
Level = $Matches.level
Component = $Matches.component
Message = $Matches.message
} | Format-List
}Time : 11/09/2026 14:02:05
Level : ERROR
Component : payments
Message : Payment gateway timeout after 30s order=A-10293Read the pattern left to right: ^ anchors at the start, \S+ grabs the date and time
(runs of non-spaces), [A-Z]+ the level, \s+ absorbs the padding after INFO ,
\[([^\]]+)\] captures everything between the brackets, and (?<message>.*)$ takes the
rest of the line. [^\]]+ ("one or more characters that aren't ]") is a pattern you'll
write constantly: it's precise and it can't run past the closing bracket.
switch -Regex for files with several line types
When a file mixes different kinds of lines, switch -Regex -File reads it line by line
and runs the block of every pattern that matches, with $Matches set for that pattern:
switch -Regex -File ./app.log {
'^\S+ \S+ ERROR\s+\[(?<component>[^\]]+)\]' { "error in $($Matches.component)" }
'^\s+at (?<frame>.+)$' { " stack: $($Matches.frame)" }
'batch=(?<batch>\d+).*ok=(?<ok>\d+) failed=(?<failed>\d+)' {
"batch $($Matches.batch): $($Matches.ok) ok, $($Matches.failed) failed"
}
}error in payments
error in payments
stack: Payments.Gateway.Charge(Order order)
stack: Orders.Sync.Run()
batch 8812: 41 ok, 1 failed
error in inventoryIt streams the file, so it handles multi-gigabyte logs without loading them into memory.
Add break inside a block if a line should stop at the first pattern that matches;
conditionals covers switch in full.
-replace and capture groups
-replace substitutes every match. In the replacement string, $1 refers to group 1,
${name} to a named group, $0 to the whole match, and $$ to a literal dollar sign:
'09/11/2026' -replace '(\d{2})/(\d{2})/(\d{4})', '$3-$2-$1'
'09/11/2026' -replace '(?<d>\d{2})/(?<m>\d{2})/(?<y>\d{4})', '${y}-${m}-${d}'
'Price: 40 EUR' -replace '(\d+) EUR', '$$$1 (approx.)'
'web01 web02 db01' -replace '\w+', '[$0]'2026-11-09
2026-11-09
Price: $40 (approx.)
[web01] [web02] [db01]The replacement string goes in single quotes for the same reason as the pattern:
"$1" in double quotes is PowerShell's (empty) variable $1, not a group reference.
It's the most common regex bug in PowerShell, and the output is shown in
Common mistakes.
-replace chains, which makes small cleanup pipelines readable:
$name -replace '\s+', ' ' -replace '^\s|\s$', '' collapses runs of whitespace and then
trims.
Computed replacements with a script block
Sometimes the replacement depends on the match: double a number, lowercase a value,
pad a date. In PowerShell 7 the right-hand side of -replace can be a script block,
with the current match in $_:
'retry after 30s, then 60s' -replace '\d+', { [int]$_.Value * 2 }
'user=ASMITH dept=FINANCE' -replace '(?<==)\w+', { $_.Value.ToLower() }retry after 60s, then 120s
user=asmith dept=financeBe careful if a script must also run on Windows PowerShell 5.1. It doesn't support this, and it doesn't fail either: it converts the script block to text and uses that as the replacement string. Here's the first line above, run in 5.1:
retry after [int]retry after 30s, then 60s.Value * 2 s, then [int]retry after 30s, then 60s.Value * 2 s(In a replacement string, $_ means "the entire input", which is why the input appears
twice.) The version that works everywhere is [regex]::Replace() with a
MatchEvaluator, which receives a Match object:
$evaluator = [System.Text.RegularExpressions.MatchEvaluator] { param($m) '{0:N0} MB' -f ([int]$m.Groups[1].Value / 1MB) }
[regex]::Replace('heap=268435456 cache=52428800', '(\d+)', $evaluator)heap=256 MB cache=50 MBSelect-String: grep that returns objects
Select-String searches files (or piped strings) for a pattern. It's PowerShell's
grep, except that it's case-insensitive by default, like the operators:
Select-String -Path ./logs/*.log -Pattern 'ERROR'logs/app-2026-11-08.log:1:2026-11-08 23:59:01.000 ERROR [orders] Deadlock detected table=order_items
logs/app-2026-11-09.log:3:2026-11-09 14:02:05.002 ERROR [payments] Payment gateway timeout after 30s order=A-10293
logs/app-2026-11-09.log:4:2026-11-09 14:02:05.003 ERROR [payments] System.TimeoutException: The operation has timed out.
logs/app-2026-11-09.log:9:2026-11-09 14:03:00.000 ERROR [inventory] Connection refused host=10.0.2.21 port=5432-Context before, after shows surrounding lines, which is how you see a stack trace
under an exception. The matching line is marked with >:
Select-String -Path ./app.log -Pattern 'TimeoutException' -Context 1, 2 app.log:3:2026-11-09 14:02:05.002 ERROR [payments] Payment gateway timeout after 30s order=A-10293
> app.log:4:2026-11-09 14:02:05.003 ERROR [payments] System.TimeoutException: The operation has timed out.
app.log:5: at Payments.Gateway.Charge(Order order)
app.log:6: at Orders.Sync.Run()The output looks like text, but each result is a MatchInfo object, and the groups
are in .Matches:
$hit = Select-String -Path ./app.log -Pattern 'order=(?<order>A-\d+)' | Select-Object -First 1
$hit | Format-List Filename, LineNumber, Line, Pattern
$hit.Matches[0].Groups['order'].ValueFilename : app.log
LineNumber : 3
Line : 2026-11-09 14:02:05.002 ERROR [payments] Payment gateway timeout after 30s order=A-10293
Pattern : order=(?<order>A-\d+)
A-10293By default Select-String records only the first match on each line. -AllMatches
records every one:
$line = Get-Content ./app.log | Select-Object -Last 2 | Select-Object -First 1
($line | Select-String '(\w+)=(\S+)').Matches.Value
'---'
($line | Select-String '(\w+)=(\S+)' -AllMatches).Matches.Valuebatch=8812
---
batch=8812
ok=41
failed=1A few more switches, shown together:
Select-String -Path ./logs/*.log -Pattern 'ERROR' -List | Select-Object Filename, LineNumber
Select-String -Path ./logs/*.log -Pattern 'Deadlock' -Quiet
Get-Content ./app.log | Select-String 'WARN' -RawFilename LineNumber
-------- ----------
app-2026-11-08.log 1
app-2026-11-09.log 3
True
2026-11-09 14:02:03.441 WARN [orders] Slow query took 2.4s table=order_items| Parameter | Effect |
|---|---|
-Pattern a, b | Match any of several patterns |
-CaseSensitive | Turn off the default case-insensitivity |
-NotMatch | Return lines that don't match |
-SimpleMatch | Treat the pattern as literal text, not a regex |
-Context 2, 3 | Include 2 lines before and 3 after each match |
-AllMatches | Record every match on a line, not just the first |
-List | Stop at the first match in each file ("which files mention X?") |
-Quiet | Return a single True/False |
-Raw (7 only) | Return the matching lines as plain strings |
For example, -NotMatch with two patterns gives you "everything that isn't an error or
a warning", stack frames included:
Select-String -Path ./app.log -Pattern 'ERROR', 'WARN' -NotMatch | ForEach-Object Line2026-11-09 14:01:58.120 INFO [orders] Started order sync batch=8812
at Payments.Gateway.Charge(Order order)
at Orders.Sync.Run()
2026-11-09 14:02:09.778 INFO [orders] Retrying order=A-10293 attempt=2
2026-11-09 14:02:11.010 INFO [orders] Completed order sync batch=8812 ok=41 failed=1To search a directory tree, combine it with Get-ChildItem -Recurse -Filter *.log
(see files and folders) and pipe the files in.
The [regex] class
The static methods on [regex] give you everything the operators don't: all matches
with their groups, compiled patterns you can reuse, and options. [regex]::Matches()
returns every match in a string. Here it pulls ticket IDs out of commits.txt:
a41f9c2 Fix rounding in invoice totals (INC-20931)
7be0d13 Bump postgres driver to 42.7.4 per CHG-1187
e09a7f1 Merge INC-20931 hotfix into release/2.3
5c11d02 Add retry to payment client, refs inc-21004 and CHG-1190
0f3e9aa Update README$text = Get-Content ./commits.txt -Raw
[regex]::Matches($text, '(INC|CHG)-\d+') | ForEach-Object Value
'---'
[regex]::Matches($text, '(INC|CHG)-\d+', 'IgnoreCase') | ForEach-Object ValueINC-20931
CHG-1187
INC-20931
CHG-1190
---
INC-20931
CHG-1187
INC-20931
inc-21004
CHG-1190The first call missed inc-21004. [regex] methods are case-sensitive by default,
unlike every PowerShell operator. The same pattern on the same text gives different
answers depending on which you use:
$text = 'Fix rounding (INC-20931), refs inc-21004'
$text -match 'inc-\d+'
$Matches[0]
[regex]::Match($text, 'inc-\d+').ValueTrue
INC-20931
inc-21004If you apply the same pattern to thousands of lines, build a [regex] object once and
reuse it. The Compiled option trades a slower start for faster matching:
$re = [regex]::new('^(?<ip>\S+) .*?" (?<status>\d{3}) ', 'Compiled')
$re.Options
(Get-Content ./access.log | Where-Object { $re.IsMatch($_) }).CountCompiled
13Options: IgnoreCase, Multiline, Singleline
Options can be passed as the last argument ('IgnoreCase, Multiline') or written
inline at the start of the pattern:
| Inline | Option | Effect |
|---|---|---|
(?i) | IgnoreCase | Case-insensitive matching |
(?m) | Multiline | ^ and $ match at the start and end of every line |
(?s) | Singleline | . also matches newlines |
(?x) | IgnorePatternWhitespace | Spaces and # comments in the pattern are ignored |
The names of Multiline and Singleline are confusing; they're not opposites. You need
them as soon as you read a whole file with Get-Content -Raw:
$log = Get-Content ./app.log -Raw
[regex]::Matches($log, '^\S+ \S+ ERROR.*$').Count
[regex]::Matches($log, '(?m)^\S+ \S+ ERROR.*$').Count0
3Without (?m), ^ means "start of the whole string", so only the first line could
match, and it's an INFO line. Singleline is for the opposite job: capturing a block
that spans lines, such as an exception and its stack trace:
$log = Get-Content ./app.log -Raw
$m = [regex]::Match($log, '(?s)System\.TimeoutException.*?(?=\n\d{4}-)')
$m.ValueSystem.TimeoutException: The operation has timed out.
at Payments.Gateway.Charge(Order order)
at Orders.Sync.Run()(?s) lets .*? cross newlines, the lazy *? stops as early as possible, and the
lookahead (?=\n\d{4}-) says "stop right before the next line that starts with a year".
Both of those get their own sections next.
Escaping: when the pattern comes from a variable
Any text you didn't write yourself (a user's search term, a file path, a version number) is full of metacharacters. Square brackets are a classic:
'warning: disk ok' -match '[ERROR]'
$Matches[0]
$needle = '[ERROR]'
'warning: disk ok' -match [regex]::Escape($needle)
[regex]::Escape('C:\Program Files (x86)\app.exe')
[regex]::Escape('1.2.3+build')True
r
False
C:\\Program\ Files\ \(x86\)\\app\.exe
1\.2\.3\+build[ERROR] is a character class meaning "one of E, R, O", and with case-insensitive
matching the r in warning qualifies. [regex]::Escape() backslash-escapes every
metacharacter, so the text is matched literally. The quieter version of the same bug
is a dot:
$version = '1.2.3'
'pkg 1.2.3 installed', 'pkg 1x2y3 installed' | Where-Object { $_ -match $version }
'---'
'pkg 1.2.3 installed', 'pkg 1x2y3 installed' | Where-Object { $_ -match [regex]::Escape($version) }pkg 1.2.3 installed
pkg 1x2y3 installed
---
pkg 1.2.3 installedRule of thumb: if a variable ends up inside a pattern, wrap it in [regex]::Escape()
unless you specifically want the caller to supply a regex. With Select-String, the
-SimpleMatch switch does the same job.
Greedy vs lazy
Quantifiers are greedy: they take as much as they can, then give back only what's needed for the rest of the pattern to match. With more than one quoted value on a line, that grabs too much:
$line = 'GET /api/orders?page=2 user="asmith" session="9f3c" took="112ms"'
$line -match 'user="(.*)"' | Out-Null; $Matches[1]
$line -match 'user="(.*?)"' | Out-Null; $Matches[1]
$line -match 'user="([^"]*)"' | Out-Null; $Matches[1]asmith" session="9f3c" took="112ms
asmith
asmithThe greedy .* ran to the end of the line and backed up to the last quote. The lazy
.*? stops at the first quote. The third version, a negated class [^"]*, says what you
actually mean ("anything except a quote"), can't overrun, and doesn't backtrack. I
prefer it whenever there's a clear delimiter.
Lookarounds
A lookaround checks what's before or after the current position without including it in the match. That makes them perfect for "replace the value but not the key":
| Syntax | Name | Means |
|---|---|---|
(?=...) | Lookahead | Followed by ... |
(?!...) | Negative lookahead | Not followed by ... |
(?<=...) | Lookbehind | Preceded by ... |
(?<!...) | Negative lookbehind | Not preceded by ... |
'/var/log/nginx/access.log' -match '(?<=/)[^/]+$' | Out-Null; $Matches[0]
'price: 40 USD, 35 EUR' -match '\d+(?= EUR)' | Out-Null; $Matches[0]
'Server=sql01;Database=orders;User Id=svc_orders;Password=S3cr3t!x;Encrypt=true' -replace '(?<=Password=)[^;]+', '********'access.log
35
Server=sql01;Database=orders;User Id=svc_orders;Password=********;Encrypt=trueThe connection-string example is one I'd use in real life before pasting a config into
a ticket: the lookbehind finds the position after Password=, and only the value is
replaced. (.NET allows variable-length lookbehind, which many other regex engines
don't.)
A negative lookahead expresses "ERROR lines, except timeouts":
Get-Content ./app.log | Where-Object { $_ -match 'ERROR(?!.*timeout)' }2026-11-09 14:03:00.000 ERROR [inventory] Connection refused host=10.0.2.21 port=5432Lookarounds can also match a pure position. This inserts thousands separators at every position that has a digit before it and a multiple of three digits after it:
'1234567', '42', '1000000' | ForEach-Object { $_ -replace '(?<=\d)(?=(\d{3})+$)', ',' }1,234,567
42
1,000,000And several lookaheads at the start of a pattern give you AND logic, which is how
password-policy checks are usually written (note -cmatch, since [a-z] and [A-Z]
mean nothing when case is ignored):
'Winter2026', 'winter2026!', 'Correct-Horse-7-Battery' | ForEach-Object {
[pscustomobject]@{
Password = $_
Valid = $_ -cmatch '^(?=.*\d)(?=.*[a-z])(?=.*[A-Z]).{12,}$'
}
}Password Valid
-------- -----
Winter2026 False
winter2026! False
Correct-Horse-7-Battery TrueCommon patterns, with honest caveats
Patterns for IP addresses, emails and dates are copied around the internet a lot. Here's what they actually do.
IPv4 addresses. The short pattern checks the shape, not the values:
$loose = '^(?:\d{1,3}\.){3}\d{1,3}$'
$octet = '(?:25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)'
$strict = "^(?:$octet\.){3}$octet$"
'10.0.1.11', '999.1.1.1', '10.0.1' | ForEach-Object {
[pscustomobject]@{ Value = $_; Loose = $_ -match $loose; Strict = $_ -match $strict }
}Value Loose Strict
----- ----- ------
10.0.1.11 True True
999.1.1.1 True False
10.0.1 False False($strict is built in double quotes on purpose, to insert $octet; the final $ is
followed by the closing quote, so PowerShell leaves it alone.) For finding IPs in a
log, the loose form with \b instead of ^/$ is usually fine. For validating,
you might reach for [ipaddress]::TryParse(), but it's more lenient than you'd expect:
$ip = $null
[ipaddress]::TryParse('10.1', [ref]$ip)
$ip.IPAddressToStringTrue
10.0.0.1It accepts the old shorthand forms, so 10.1 is "valid". Use the strict regex when you
need exactly four dotted octets.
Email addresses. No regex fully implements the email address standard, and the ones that try are unreadable. Pick a sanity check and know its limits:
$simple = '^[^@\s]+@[^@\s]+\.[^@\s]+$'
'asmith@example.com', 'asmith@localhost', 'Alice Smith <asmith@example.com>' | ForEach-Object {
$parsed = $null
[pscustomobject]@{
Value = $_
Regex = $_ -match $simple
MailAddress = [mailaddress]::TryCreate($_, [ref]$parsed)
Address = $parsed.Address
}
}Value Regex MailAddress Address
----- ----- ----------- -------
asmith@example.com True True asmith@example.com
asmith@localhost False True asmith@localhost
Alice Smith <asmith@example.com> False True asmith@example.com[mailaddress] accepts display-name forms and dotless domains, which is right for
parsing email headers and wrong for a sign-up form. The only real validation is sending
a message to the address.
Dates. A regex validates the shape of a date, not whether the day exists:
$d = [datetime]::MinValue
'2026-02-28', '2026-02-30' | ForEach-Object {
[pscustomobject]@{
Value = $_
Shape = $_ -match '^\d{4}-\d{2}-\d{2}$'
IsDate = [datetime]::TryParseExact($_, 'yyyy-MM-dd', [cultureinfo]::InvariantCulture, 'None', [ref]$d)
}
}Value Shape IsDate
----- ----- ------
2026-02-28 True True
2026-02-30 True FalseUse a regex to find dates in text, then a real parser to validate them.
Log lines. The general rule: anchor with ^, use \S+ for space-free fields,
[^"]* inside quotes and [^\]]+ inside brackets, and name every group. That's
exactly what the next section does.
Where you'll use this
Regex is the bridge between text and objects in admin work: summarizing web and auth
logs, redacting secrets before sharing a log, bulk-renaming files
(Get-ChildItem | Rename-Item -NewName { $_.Name -replace '^IMG_(\d+)', 'photo-$1' }),
validating script parameters with [ValidatePattern('^[A-Z]{3}-\d+$')], and scraping
the output of native commands that have no JSON mode.
Here's the access-log question from the start of the article, as a reusable function.
The sample access.log has thirteen lines in nginx's combined format plus the request
time; the last line is junk a TLS client sent to the plain-HTTP port, which is the kind
of line every real log has:
203.0.113.10 - - [09/Nov/2026:14:02:11 +0000] "GET /api/orders?page=2 HTTP/1.1" 200 5123 "-" "Mozilla/5.0 (X11; Linux x86_64)" 0.112
198.51.100.23 - - [09/Nov/2026:14:02:12 +0000] "POST /api/orders HTTP/1.1" 201 312 "-" "python-requests/2.32.3" 0.348
[…]
192.0.2.44 - - [09/Nov/2026:14:02:33 +0000] "GET /.env HTTP/1.1" 404 153 "-" "Mozilla/5.0 (compatible; scanner)" 0.001
[…]
- - - [09/Nov/2026:14:03:01 +0000] "\x16\x03\x01" 400 157 "-" "-" 0.000function ConvertFrom-AccessLog {
[CmdletBinding()]
param(
[Parameter(Mandatory, ValueFromPipeline)]
[AllowEmptyString()]
[string]$Line
)
begin {
# One named group per field of the nginx "combined" format plus $request_time.
$re = [regex]::new(
'^(?<ip>\S+) \S+ \S+ \[(?<time>[^\]]+)\] ' +
'"(?<method>[A-Z]+) (?<path>[^ "?]+)(?:\?(?<query>[^ "]*))? HTTP/[\d.]+" ' +
'(?<status>\d{3}) (?<bytes>\d+|-) "(?<referer>[^"]*)" "(?<agent>[^"]*)" (?<seconds>[\d.]+)$',
'Compiled')
$inv = [cultureinfo]::InvariantCulture
$skipped = 0
}
process {
$m = $re.Match($Line)
if (-not $m.Success) {
$skipped++
Write-Verbose "No match: $Line"
return
}
$g = $m.Groups
# nginx writes +0000; .NET's zzz wants +00:00, so insert the colon with a regex.
$time = $g['time'].Value -replace '(\d\d)(\d\d)$', '$1:$2'
[pscustomobject]@{
Client = $g['ip'].Value
Time = [datetimeoffset]::ParseExact($time, 'dd/MMM/yyyy:HH:mm:ss zzz', $inv)
Method = $g['method'].Value
Path = $g['path'].Value
Query = $g['query'].Value
Status = [int]$g['status'].Value
Bytes = if ($g['bytes'].Value -eq '-') { 0 } else { [int]$g['bytes'].Value }
Seconds = [double]$g['seconds'].Value
Agent = $g['agent'].Value
}
}
end {
if ($skipped) { Write-Warning "$skipped line(s) did not match the access log format" }
}
}
$requests = Get-Content ./access.log | ConvertFrom-AccessLog
$requests.Count
$requests[0] | Format-List
$requests | Group-Object Status -NoElement | Sort-Object Name | Format-Table Name, Count
$requests | Where-Object Status -ge 500 | Format-Table Time, Client, Method, Path, Status, Seconds
$requests | Where-Object Path -match '\.(php|env)$|^/wp-' | Group-Object Client -NoElement | Format-Table Name, CountWARNING: 1 line(s) did not match the access log format
12
Client : 203.0.113.10
Time : 11/09/2026 14:02:11 +00:00
Method : GET
Path : /api/orders
Query : page=2
Status : 200
Bytes : 5123
Seconds : 0.112
Agent : Mozilla/5.0 (X11; Linux x86_64)
Name Count
---- -----
200 6
201 1
404 2
500 1
502 2
Time Client Method Path Status Seconds
---- ------ ------ ---- ------ -------
11/09/2026 14:02:20 +00:00 198.51.100.23 POST /api/payments 502 30.00
11/09/2026 14:02:31 +00:00 198.51.100.23 POST /api/payments 502 30.00
11/09/2026 14:02:52 +00:00 203.0.113.77 GET /api/inventory 500 0.41
Name Count
---- -----
192.0.2.44 2The design decisions are the part worth copying:
- The regex runs once per line and everything after it is objects. Typed
properties (
[int]status,[double]seconds, a realDateTimeOffset) mean-ge 500and sorting by time just work, for the same reason as in the CSV article. - Lines that don't match are counted, not dropped silently. The warning tells you
the parser saw something unexpected;
-Verboseshows which lines. - The pattern is built once in
beginand reused inprocess, so piping a million-line log through it doesn't re-parse the pattern each time. - The fields are precise.
[^ "?]+stops the path at the query string,(?:\?(?<query>[^ "]*))?makes the query optional, and(?<bytes>\d+|-)handles nginx's-for "no body".
The two 502s took 30 seconds each, which looks like an upstream timeout on the payment
gateway, and 192.0.2.44 is probing for WordPress and .env files. That's the answer
to the original question, and the function is reusable for tomorrow's log. To run it
on a schedule and mail the summary, see
real-world PowerShell scripts.
Common mistakes
Using -split or -replace as if they were literal
('10.0.1.11' -split '.').Count
'10.0.1.11' -split '\.'
'v1.2.3' -replace '.', '-'
'v1.2.3'.Replace('.', '-')10
10
0
1
11
------
v1-2-3Splitting on . (any character) produced ten empty strings, and replacing . replaced
every character. Escape the dot ('\.'), or use the .Split() and .Replace() string
methods, which are literal.
Invalid patterns from paths and symbols
'C:\Temp\new\report.txt' -match 'C:\Temp\new'OperationStopped: Invalid pattern 'C:\Temp\new' at offset 4. Unrecognized escape sequence \T.'1+2+3' -split '+'OperationStopped: Invalid pattern '+' at offset 1. Quantifier '+' following nothing.Windows paths are full of backslashes, and +, *, ?, (, [ all mean something.
The fix is [regex]::Escape() for text you didn't write, or escaping by hand:
'C:\Temp\new\report.txt' -match [regex]::Escape('C:\Temp\new')
'1+2+3' -split '\+'True
1
2
3Worse than the error is the case where an unescaped path happens to be a valid
pattern (\n is a newline, \d a digit) and silently matches the wrong thing.
Double quotes around the replacement
'09/11/2026' -replace '(\d{2})/(\d{2})/(\d{4})', "$3-$2-$1"--PowerShell expanded $3, $2 and $1 as (undefined, empty) variables before
-replace ran, so the replacement was just --. No error. Use single quotes. If you
need a variable and a group reference in the same replacement, escape the group's
dollar with a backtick: "`$1-$suffix".
Trusting $Matches after a failed match, or after matching an array
'order=A-10293' -match 'order=(A-\d+)' | Out-Null
$Matches[1]
'no order id here' -match 'order=(A-\d+)'
$Matches[1]A-10293
False
A-10293The second match failed, but $Matches still holds the first result. In a loop, that
means a line without an order ID gets the previous line's ID. Only read $Matches
inside if ($x -match ...) { }.
And with an array on the left, -match is a filter that never sets $Matches:
$lines = Get-Content ./app.log
$lines -match '^\S+ \S+ ERROR'
'---'
$Matches.Count2026-11-09 14:02:05.002 ERROR [payments] Payment gateway timeout after 30s order=A-10293
2026-11-09 14:02:05.003 ERROR [payments] System.TimeoutException: The operation has timed out.
2026-11-09 14:03:00.000 ERROR [inventory] Connection refused host=10.0.2.21 port=5432
---
0To get groups from many lines, loop and match one line at a time, or use
Select-String and read .Matches on each result.
Summary
- PowerShell uses the .NET regex engine.
-match,-replace,-split,switch -RegexandSelect-Stringtake patterns and are case-insensitive by default;[regex]methods are case-sensitive by default;.Replace(),.Split()and-likeare not regex at all. - Write patterns and replacement strings in single quotes.
"$1"is an empty variable, not a group reference. $Matchesis set only by a successful-matchon a single string. Use named groups(?<name>...)and build[pscustomobject]s from them to turn text into objects.Select-StringreturnsMatchInfoobjects:-Contextfor surrounding lines,-AllMatchesfor every hit,-List,-Quiet,-NotMatchand-SimpleMatchfor the rest.(?m)makes^/$work per line onGet-Content -Rawtext;(?s)lets.cross newlines.- Prefer negated classes like
[^"]*over.*; use.*?when there's no single delimiter. Lookarounds match context without consuming it. [regex]::Escape()anything you didn't write. Script-block replacements need PowerShell 7;[regex]::Replace()with aMatchEvaluatorworks everywhere.- Regex checks shape, not meaning: validate IPs, emails and dates with a parser after matching them.
Exercises
From commits.txt, produce a sorted, de-duplicated list of every ticket ID (INC- or
CHG- followed by digits), normalized to upper case. inc-21004 must be included.
Show Solution
$text = Get-Content ./commits.txt -Raw
[regex]::Matches($text, '\b(?:INC|CHG)-\d+\b', 'IgnoreCase') |
ForEach-Object { $_.Value.ToUpper() } |
Sort-Object -UniqueCHG-1187
CHG-1190
INC-20931
INC-21004[regex]::Matches() finds every occurrence, not just the first; the IgnoreCase
option is needed because [regex] is case-sensitive by default. (?:...) groups the
alternation without creating a capture, and \b stops the pattern matching inside a
longer word.
Rewrite every M/D/YYYY date in this string as YYYY-MM-DD, with zero-padded month and
day:
Invoice 4471 issued 3/7/2026, due 11/30/2026, paid 12/2/2026Hint
A plain replacement string can reorder the groups, but it can't pad 3 to 03. A
script block replacement can.
Show Solution
$text = 'Invoice 4471 issued 3/7/2026, due 11/30/2026, paid 12/2/2026'
$text -replace '\b(?<m>\d{1,2})/(?<d>\d{1,2})/(?<y>\d{4})\b', '${y}-${m}-${d}'
$text -replace '\b(?<m>\d{1,2})/(?<d>\d{1,2})/(?<y>\d{4})\b', {
'{0}-{1:D2}-{2:D2}' -f $_.Groups['y'].Value, [int]$_.Groups['m'].Value, [int]$_.Groups['d'].Value
}Invoice 4471 issued 2026-3-7, due 2026-11-30, paid 2026-12-2
Invoice 4471 issued 2026-03-07, due 2026-11-30, paid 2026-12-02The first line shows why the string version isn't enough. In the script block, the
groups are converted to [int] so the D2 format specifier can pad them. On Windows
PowerShell 5.1, wrap the same script block in a MatchEvaluator and call
[regex]::Replace().
Before attaching these lines to a vendor ticket, mask the email address so only its
first character remains (a***@example.com) and replace the bearer token with
[redacted]:
2026-11-09 14:05:12 POST /api/login user=alice.smith@example.com status=200
2026-11-09 14:05:13 GET /api/orders Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhc21pdGgifQ.c2lnbmF0dXJl status=200Show Solution
$lines = @(
'2026-11-09 14:05:12 POST /api/login user=alice.smith@example.com status=200'
'2026-11-09 14:05:13 GET /api/orders Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhc21pdGgifQ.c2lnbmF0dXJl status=200'
)
$lines -replace '\b(\w)[\w.+-]*@', '$1***@' -replace '(?<=Bearer )\S+', '[redacted]'2026-11-09 14:05:12 POST /api/login user=a***@example.com status=200
2026-11-09 14:05:13 GET /api/orders Authorization: Bearer [redacted] status=200-replace works on every element of an array, so both lines are processed at once. The
first replacement keeps group 1 (the first character of the local part) and drops the
rest up to @. The second uses a lookbehind so Bearer itself stays in the output.
Given this auth.log, list each source IP with failed password attempts: the number of
attempts, the distinct user names tried, and the first and last attempt time, most
attempts first. Note that some lines say for invalid user admin and others
for root.
Nov 9 03:12:44 web01 sshd[2211]: Failed password for invalid user admin from 203.0.113.50 port 51234 ssh2
Nov 9 03:12:47 web01 sshd[2211]: Failed password for invalid user admin from 203.0.113.50 port 51240 ssh2
Nov 9 03:13:02 web01 sshd[2215]: Failed password for root from 198.51.100.7 port 40022 ssh2
Nov 9 03:13:05 web01 sshd[2215]: Failed password for root from 198.51.100.7 port 40026 ssh2
Nov 9 03:13:09 web01 sshd[2215]: Failed password for root from 198.51.100.7 port 40031 ssh2
Nov 9 07:45:10 web01 sshd[3120]: Accepted publickey for deploy from 192.0.2.15 port 50122 ssh2
Nov 9 08:01:33 web01 sshd[3301]: Failed password for deploy from 192.0.2.15 port 50190 ssh2
Nov 9 09:22:51 web01 sshd[3550]: Failed password for invalid user oracle from 203.0.113.50 port 33012 ssh2
Nov 9 09:22:55 web01 sshd[3550]: Connection closed by invalid user oracle 203.0.113.50 port 33012 [preauth]Hint
(?:invalid user )? makes that phrase optional. Syslog pads single-digit days with a
second space (Nov 9), so use + between month and day.
Show Solution
$failures = switch -Regex -File ./auth.log {
'^(?<time>\w{3} +\d+ [\d:]+) .*Failed password for (?:invalid user )?(?<user>\S+) from (?<ip>\S+)' {
[pscustomobject]@{ Time = $Matches.time; User = $Matches.user; IP = $Matches.ip }
}
}
$failures | Group-Object IP | ForEach-Object {
[pscustomobject]@{
IP = $_.Name
Attempts = $_.Count
Users = ($_.Group.User | Sort-Object -Unique) -join ', '
First = $_.Group[0].Time
Last = $_.Group[-1].Time
}
} | Sort-Object @{ Expression = 'Attempts'; Descending = $true }, IP | Format-TableIP Attempts Users First Last
-- -------- ----- ----- ----
198.51.100.7 3 root Nov 9 03:13:02 Nov 9 03:13:09
203.0.113.50 3 admin, oracle Nov 9 03:12:44 Nov 9 09:22:51
192.0.2.15 1 deploy Nov 9 08:01:33 Nov 9 08:01:33switch -Regex -File streams the log and emits an object only for lines that match,
so the Accepted publickey and Connection closed lines are ignored. The optional
non-capturing group means user is always the account name, whichever wording sshd
used. The secondary sort key on IP keeps the order stable when two addresses have
the same number of attempts.
Tested with: PowerShell 7.6.9 on Ubuntu 24.04 (WSL2) and Windows PowerShell
5.1.26100 on Windows 11, November 2026. Every PowerShell 7 example ran as shown against
the sample files above, in a 150-column terminal with the invariant culture (hence the
11/09/2026 date format). The 5.1 output of a script-block replacement came from a clean
powershell.exe -NoProfile session; I expected an error there and got silently wrong
text instead. IP addresses in the samples are from the documentation ranges
(192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24).
Written by The Dark Debugger
Engineer documenting the things that took me too long to learn, so they take you less time. Every PowerShell tutorial here is tested end to end before publishing.