Summary: in this tutorial, you will learn master the object-based pipeline — the heart of powershell. learn to filter, sort, select, group, measure, format, and export data with where-object, select-object, sort-object, and more.

The PowerShell Pipeline

The pipeline is the single most important concept in PowerShell. It's what makes PowerShell fundamentally different from traditional shells, and understanding it will transform how you think about command-line automation.

In traditional shells like Bash or CMD, the pipeline passes text from one command to another. You pipe raw text, and the receiving command has to parse that text, hoping the format doesn't change. PowerShell does something revolutionary: it passes objects through the pipeline. Complete, structured .NET objects with properties, methods, and type information.

Understanding Object-Based Pipelines

Let's start with a concrete example to understand why this matters.

The Traditional Text-Based Approach

In Bash, if you want to find all Firefox processes and stop them, you'd do something like this:

# Bash - text-based
ps aux | grep firefox | awk '{print $2}' | xargs kill

What's happening here?

ps aux outputs text in columns
grep firefox filters lines containing "firefox" (but might catch false matches like "firefox-extension-updater")
awk '{print $2}' extracts the second column (hoping it's the PID)
xargs kill passes those PIDs to kill

This is fragile. If the ps format changes, if column alignment shifts, if firefox appears in a different column, everything breaks. You're parsing unstructured text and hoping for the best.

The PowerShell Object-Based Approach

Here's the same task in PowerShell:

Get-Process -Name firefox | Stop-Process

That's it. Two commands. No parsing, no text manipulation, no fragility. What makes this work?

Get-Process returns actual Process objects—structured data with properties like Name, Id, CPU, Memory
The pipeline passes these complete objects to Stop-Process
Stop-Process receives the object and knows exactly how to stop it

You're not parsing text. You're working with structured data.

Seeing the Objects

Let's examine what's actually flowing through the pipeline:

# Get a process and see its structure
Get-Process -Name explorer | Get-Member
 
# This shows you everything:
# Properties: Name, Id, CPU, WorkingSet64, StartTime, etc.
# Methods: Kill(), WaitForExit(), Refresh(), etc.

Every object in PowerShell has:

Properties - data about the object (Name, Size, Date, etc.)
Methods - actions you can perform on the object (.ToString(), .Equals(), etc.)
Type - what kind of object it is (System.Diagnostics.Process, System.IO.FileInfo, etc.)

This is the foundation of PowerShell's power. Commands output objects, not text. The pipeline passes objects, not strings. Receiving commands work with objects, not parsed text.

How the Pipeline Works Internally

When you run:

Get-Process | Where-Object CPU -gt 10 | Select-Object Name, CPU

Here's what happens:

Get-Process executes and starts producing Process objects
As soon as the first object is ready, it's sent down the pipeline
Where-Object receives that first object, tests if CPU > 10
- If true, passes it to the next command
- If false, discards it and waits for the next object
Select-Object receives objects that passed the filter and selects just the Name and CPU properties
This continues streaming - one object at a time

This is called streaming. PowerShell doesn't wait for Get-Process to finish collecting all processes. It processes them one by one as they arrive. This is incredibly efficient for large datasets.

The streaming nature of the pipeline means you can work with millions of objects without running out of memory. Each object is processed and released, not accumulated.

Pipeline Variable: $_

Inside pipeline commands, $_ (or $PSItem in PowerShell 3+) represents the current object being processed. Think of it as "the object that just arrived from the previous command."

Get-Process | Where-Object { $_.CPU -gt 10 }
#                            ↑
#                            Current process object

$_ is only available inside pipeline script blocks. Outside the pipeline, it doesn't exist.

Filtering with Where-Object

Where-Object is how you filter the pipeline - keeping only objects that match certain criteria. This is like SQL's WHERE clause or array filter() methods in programming languages.

Why Filtering Matters

Imagine you have 200 running processes. You only care about ones using significant CPU. Where-Object lets you filter before spending time on further processing:

# Without filtering - processes all 200 processes
Get-Process | Sort-Object CPU -Descending | Select-Object -First 10
 
# With filtering - only processes high-CPU ones
Get-Process | Where-Object CPU -gt 50 | Sort-Object CPU -Descending

The second approach is more efficient and more explicit about your intent.

The Two Syntaxes

PowerShell offers two ways to use Where-Object:

1. Script Block Syntax (Original, most flexible):

Get-Process | Where-Object { $_.CPU -gt 10 }

2. Simplified Syntax (PowerShell 3+, cleaner for simple conditions):

Get-Process | Where-Object CPU -gt 10

Both do the same thing. The simplified syntax is cleaner for simple property comparisons. The script block syntax is more powerful for complex logic.

Understanding Comparison Operators

PowerShell has rich comparison operators specifically designed for filtering:

Equality Comparisons:

# -eq: equals
Get-Service | Where-Object Status -eq "Running"
 
# -ne: not equals
Get-Service | Where-Object Status -ne "Stopped"
 
# -in: value is in a set
Get-Process | Where-Object Name -in "chrome", "firefox", "edge"
 
# -notin: value is not in a set
Get-Service | Where-Object Status -notin "Stopped", "Disabled"

Why does PowerShell use -eq instead of ==? Because = is for assignment, and PowerShell wanted clarity. -eq can't be confused with assignment.

Numeric Comparisons:

# -gt: greater than
Get-Process | Where-Object CPU -gt 10
 
# -ge: greater than or equal
Get-Process | Where-Object WorkingSet64 -ge 100MB
 
# -lt: less than
Get-ChildItem | Where-Object Length -lt 1KB
 
# -le: less than or equal
Get-Process | Where-Object CPU -le 5

Note the MB and KB suffixes. PowerShell understands size units: KB, MB, GB, TB, PB. 100MB is 104,857,600 bytes.

Pattern Matching:

# -like: wildcard matching (* and ?)
Get-Service | Where-Object Name -like "*sql*"
Get-Process | Where-Object Name -like "chrome*"
 
# -notlike: doesn't match wildcard
Get-Service | Where-Object Name -notlike "svc*"
 
# -match: regular expression matching
Get-Service | Where-Object Name -match "^win"    # Starts with "win"
Get-Process | Where-Object Name -match "\d+$"    # Ends with digits
 
# -notmatch: doesn't match regex
Get-Process | Where-Object Name -notmatch "^svc"

Collection Checks:

# -contains: collection contains a value
$runningServices = Get-Service | Where-Object Status -eq "Running"
$runningServices | Where-Object { $_.DependentServices -contains "WinDefend" }
 
# -notcontains: collection doesn't contain a value
Get-Process | Where-Object { $_.Modules.ModuleName -notcontains "kernel32.dll" }

Complex Filtering with Logical Operators

Real-world filtering often requires multiple conditions. PowerShell provides logical operators:

# -and: both conditions must be true
Get-Process | Where-Object {
    $_.CPU -gt 10 -and $_.WorkingSet64 -gt 100MB
}
 
# -or: either condition can be true
Get-Service | Where-Object {
    $_.Status -eq "Running" -or $_.Status -eq "Paused"
}
 
# -not: negates the condition
Get-Process | Where-Object { -not $_.Responding }
 
# Combining logical operators with parentheses for clarity
Get-Service | Where-Object {
    ($_.Status -eq "Running" -or $_.Status -eq "Paused") -and
    $_.StartType -eq "Automatic"
}

Filtering for Null or Empty

A common scenario is checking if properties are null or empty:

# Find processes with a window title
Get-Process | Where-Object { $null -ne $_.MainWindowTitle -and $_.MainWindowTitle -ne "" }
 
# Find services with no dependencies
Get-Service | Where-Object { $_.DependentServices.Count -eq 0 }
 
# Find files modified in the last 7 days
Get-ChildItem | Where-Object { $_.LastWriteTime -gt (Get-Date).AddDays(-7) }

Performance Tip: When filtering, put the fastest/most restrictive conditions first. PowerShell evaluates left-to-right and stops as soon as a condition fails (-and) or succeeds (-or).

Selecting Properties with Select-Object

After filtering, you often want to work with just specific properties. Select-Object is your tool for this. Think of it like SQL's SELECT statement - choosing which columns to keep.

Why Select Properties?

By default, PowerShell displays many properties. Sometimes you only need a few:

# Get-Process shows: Name, Id, CPU, Memory, etc.
Get-Process
 
# But you might only care about Name and CPU
Get-Process | Select-Object Name, CPU

This makes output cleaner and more focused. It's also crucial when exporting data - you don't want 50 properties in your CSV when you only need 5.

Basic Property Selection

# Select specific properties
Get-Process | Select-Object Name, Id, CPU, WorkingSet64
 
# Select just one property (still returns objects with that property)
Get-Service | Select-Object Name
 
# Select all properties (normally PowerShell hides some)
Get-Process | Select-Object *

Limiting Results

Select-Object also controls how many objects pass through:

# Get the first 5 objects
Get-Process | Sort-Object CPU -Descending | Select-Object -First 5
 
# Get the last 3 objects
Get-Process | Sort-Object StartTime | Select-Object -Last 3
 
# Skip the first 10, then take 5
Get-Process | Select-Object -Skip 10 -First 5
 
# Get unique values (remove duplicates)
Get-Process | Select-Object ProcessName -Unique

These are incredibly useful for "top N" queries:

# Top 10 processes by memory
Get-Process | Sort-Object WorkingSet64 -Descending | Select-Object -First 10 Name, @{N='MemMB';E={[math]::Round($_.WorkingSet64/1MB,2)}}

Calculated Properties: Creating Custom Columns

This is where Select-Object becomes powerful. You can create calculated properties - custom properties derived from existing ones.

The syntax uses a hashtable with Name (or N) and Expression (or E):

# Basic calculated property
Get-Process | Select-Object Name, @{
    Name       = 'MemoryMB'
    Expression = { $_.WorkingSet64 / 1MB }
}
 
# Short form (N and E)
Get-Process | Select-Object Name, @{N='MemMB'; E={$_.WorkingSet64/1MB}}

Inside the Expression script block, $_ is the current object. You can do any calculation or transformation:

# Multiple calculated properties
Get-Process | Select-Object Name,
    @{N='MemMB'; E={[math]::Round($_.WorkingSet64/1MB, 2)}},
    @{N='CPUSeconds'; E={[math]::Round($_.CPU, 1)}},
    @{N='Running'; E={(New-TimeSpan -Start $_.StartTime).Days}}
 
# String manipulation
Get-ChildItem | Select-Object Name,
    @{N='Extension'; E={$_.Extension.ToUpper()}},
    @{N='SizeKB'; E={[math]::Round($_.Length/1KB, 1)}}
 
# Date calculations
Get-ChildItem | Select-Object Name,
    @{N='Age'; E={(New-TimeSpan -Start $_.LastWriteTime).Days}},
    @{N='Modified'; E={$_.LastWriteTime.ToString('yyyy-MM-dd')}}
 
# Conditional logic
Get-Service | Select-Object Name, Status,
    @{N='State'; E={
        if ($_.Status -eq 'Running') { '✓ Online' }
        else { '✗ Offline' }
    }}

This is extremely powerful. You're essentially adding computed columns to your data.

Expanding Properties

Some properties contain collections or complex objects. By default, PowerShell shows these as type names:

# This shows: System.Diagnostics.ProcessModuleCollection
Get-Process explorer | Select-Object Modules

To see the actual contents, use -ExpandProperty:

# This shows all the module objects inside
Get-Process explorer | Select-Object -ExpandProperty Modules

Common uses:

# Get an array of just the names
$names = Get-Service | Select-Object -ExpandProperty Name
# $names is now: @("AarSvc", "AdobeARMservice", ...)
 
# Get all modules used by Chrome
Get-Process chrome | Select-Object -ExpandProperty Modules | Select-Object ModuleName -Unique
 
# Get all dependent services
Get-Service W32Time | Select-Object -ExpandProperty DependentServices

-ExpandProperty unwraps the collection so you can work with its contents directly.

Sorting with Sort-Object

Data is rarely in the order you want. Sort-Object arranges objects by property values.

Basic Sorting

# Sort ascending (default)
Get-Process | Sort-Object CPU
 
# Sort descending
Get-Process | Sort-Object CPU -Descending
 
# Sort by property name
Get-ChildItem | Sort-Object Name
Get-ChildItem | Sort-Object Length -Descending

Sorting is essential before using -First or -Last:

# Wrong: gets first 5 in arbitrary order
Get-Process | Select-Object -First 5
 
# Right: gets 5 highest CPU users
Get-Process | Sort-Object CPU -Descending | Select-Object -First 5

Multi-Property Sorting

You can sort by multiple properties (like SQL ORDER BY with multiple columns):

# Sort by Status first, then by Name
Get-Service | Sort-Object Status, Name
 
# Mix ascending and descending
Get-Process | Sort-Object @{Expression='CPU'; Descending=$true}, Name

The syntax @{Expression='PropertyName'; Descending=$true} lets you control sort direction per property.

Removing Duplicates

Sort-Object -Unique removes duplicates:

# Remove duplicate values
1, 3, 2, 1, 3, 4 | Sort-Object -Unique
# Output: 1, 2, 3, 4
 
# Get unique process names
Get-Process | Select-Object ProcessName | Sort-Object ProcessName -Unique
 
# Unique file extensions in a directory
Get-ChildItem | Select-Object Extension | Sort-Object Extension -Unique

Case-Sensitive Sorting

By default, PowerShell sorts case-insensitively:

"banana", "Apple", "cherry" | Sort-Object
# Output: Apple, banana, cherry

For case-sensitive sorting:

"banana", "Apple", "cherry" | Sort-Object -CaseSensitive
# Output: Apple, banana, cherry (uppercase sorts before lowercase)

Grouping with Group-Object

Group-Object organizes objects into groups based on a property value. This is like SQL's GROUP BY or pivot tables.

Basic Grouping

# Group services by status
Get-Service | Group-Object Status
 
# Output shows:
# Count Name   Group
# ----- ----   -----
#    90 Stopped {AarSvc_..., AdobeARMservice, ...}
#    76 Running {AudioEndpointBuilder, Audiosrv, ...}

Each group has:

Count - number of objects in the group
Name - the property value that defines the group
Group - the actual objects in that group

Working with Groups

You can access the groups programmatically:

$groups = Get-Service | Group-Object Status
 
# Access individual groups
$groups[0].Name          # "Stopped"
$groups[0].Count         # 90
$groups[0].Group         # All stopped services
 
# Iterate through groups
foreach ($group in $groups) {
    Write-Host "$($group.Name): $($group.Count) services"
}

Grouping by Multiple Properties

# Group by multiple properties
Get-Process | Group-Object Name, Responding
 
# Group services by Status and StartType
Get-Service | Group-Object Status, StartType | Select-Object Count, Name

Using Grouping for Analysis

Grouping is powerful for data analysis:

# How many files of each type?
Get-ChildItem -File | Group-Object Extension | Sort-Object Count -Descending
 
# Which extensions use the most space?
Get-ChildItem -File | Group-Object Extension | ForEach-Object {
    [PSCustomObject]@{
        Extension  = $_.Name
        Count      = $_.Count
        TotalSizeMB = [math]::Round(($_.Group | Measure-Object Length -Sum).Sum / 1MB, 2)
    }
} | Sort-Object TotalSizeMB -Descending
 
# How many processes per company?
Get-Process | Where-Object Company | Group-Object Company | Sort-Object Count -Descending | Select-Object -First 10

Creating Hashtables with -AsHashTable

Sometimes you want quick lookup by group name:

# Create a hashtable of groups
$statusGroups = Get-Service | Group-Object Status -AsHashTable
 
# Now you can access by status name
$statusGroups['Running']     # All running services
$statusGroups['Stopped']     # All stopped services

This is incredibly useful for conditional logic based on groups.

Measuring with Measure-Object

Measure-Object calculates statistics on object properties - counts, sums, averages, min/max.

Counting Objects

# Count how many objects
Get-Process | Measure-Object
# Returns: Count: 142
 
# Count specific types
Get-Service | Where-Object Status -eq Running | Measure-Object
Get-ChildItem -File | Measure-Object

Calculating Statistics

For numeric properties, you can calculate sum, average, min, max:

# Sum, average, min, max of file sizes
Get-ChildItem -File | Measure-Object Length -Sum -Average -Minimum -Maximum
 
# Output:
# Count    : 47
# Average  : 15283.51...
# Sum      : 718325
# Maximum  : 125440
# Minimum  : 82

Practical Examples

# Total memory used by all Chrome processes
Get-Process chrome | Measure-Object WorkingSet64 -Sum | Select-Object -ExpandProperty Sum | ForEach-Object { "$([math]::Round($_/1GB, 2)) GB" }
 
# Average CPU time per process
Get-Process | Measure-Object CPU -Average
 
# Disk space used by logs
Get-ChildItem C:\Logs -Recurse -File | Measure-Object Length -Sum
 
# How many lines in all .ps1 files?
Get-ChildItem -Filter *.ps1 -Recurse | Get-Content | Measure-Object -Line

Counting Text Elements

Measure-Object can also count lines, words, and characters in text:

# Count lines, words, characters in a file
Get-Content script.ps1 | Measure-Object -Line -Word -Character
 
# Find the longest line in a file
Get-Content file.txt | Measure-Object -Maximum Length

Iterating with ForEach-Object

ForEach-Object (alias % or foreach) runs a script block for each object in the pipeline.

When to Use ForEach-Object

Use ForEach-Object when you need to perform an action on each object:

# Restart each service one by one
Get-Service -Name "MyApp*" | ForEach-Object {
    Write-Host "Restarting $($_.Name)..."
    $_ | Restart-Service
}
 
# Create a directory for each month
1..12 | ForEach-Object {
    $monthName = (Get-Date -Month $_).ToString("MMMM")
    New-Item -Name $monthName -ItemType Directory
}

The Syntax

# Basic syntax
Get-Process | ForEach-Object { $_.Name.ToUpper() }
 
# With named blocks (Begin, Process, End)
Get-ChildItem | ForEach-Object -Begin {
    $total = 0
} -Process {
    $total += $_.Length
} -End {
    "Total size: $total bytes"
}

The -Process block runs for each object. -Begin runs once before processing. -End runs once after all objects are processed.

Practical Examples

# Send email for each error in log
Get-Content error.log | Where-Object { $_ -match "CRITICAL" } | ForEach-Object {
    Send-MailMessage -To admin@example.com -Subject "Critical Error" -Body $_
}
 
# Download multiple files
$urls = @("https://example.com/file1.zip", "https://example.com/file2.zip")
$urls | ForEach-Object {
    $filename = $_.Split("/")[-1]
    Invoke-WebRequest -Uri $_ -OutFile $filename
}
 
# Process each CSV row
Import-Csv users.csv | ForEach-Object {
    New-ADUser -Name $_.Name -EmailAddress $_.Email -Department $_.Dept
}

Formatting Output

PowerShell has cmdlets to control how objects display.

Format-Table

# Default formatting (auto-selected properties)
Get-Process | Format-Table
 
# Choose properties
Get-Process | Format-Table Name, Id, CPU
 
# Auto-size columns
Get-Process | Format-Table -AutoSize
 
# Group by property
Get-Service | Format-Table -GroupBy Status

Format-List

Display as a list (property: value pairs):

# One property per line
Get-Process -Id $PID | Format-List *
 
# Choose properties
Get-Service w32time | Format-List Name, Status, StartType

Format-Wide

Display in columns:

# Show just names in columns
Get-Process | Format-Wide Name -Column 4

Important: Format cmdlets produce formatted output, not objects. Once you format, you can't pipe to other cmdlets. Always format last in your pipeline.

Exporting Data

After processing, you often want to save results.

Export-Csv

# Export to CSV
Get-Process | Select-Object Name, CPU, WorkingSet64 | Export-Csv processes.csv
 
# Without type information header
Get-Process | Export-Csv processes.csv -NoTypeInformation
 
# Append to existing CSV
Get-Process | Export-Csv processes.csv -Append

Export-Clixml

Preserves full object structure:

# Export complete objects
Get-Process | Export-Clixml processes.xml
 
# Import later
$processes = Import-Clixml processes.xml

ConvertTo-Json / ConvertTo-Html

# Convert to JSON
Get-Service | Select-Object Name, Status | ConvertTo-Json
 
# Create HTML report
Get-Process | Select-Object -First 10 Name, CPU | ConvertTo-Html | Out-File report.html

Out-File

Save text output:

Get-Service | Out-File services.txt
 
# Specific encoding
Get-Content file.txt | Out-File output.txt -Encoding UTF8

Exercises

🏋️ Exercise 1: Pipeline Filtering

Write a pipeline that:

Gets all running processes
Filters to those using more than 50MB of memory
Sorts by memory usage (highest first)
Shows only Name and memory in MB

Show Solution

Get-Process |
    Where-Object { $_.WorkingSet64 -gt 50MB } |
    Sort-Object WorkingSet64 -Descending |
    Select-Object Name, @{N='MemoryMB';E={[math]::Round($_.WorkingSet64/1MB,2)}}

🏋️ Exercise 2: Process Analysis

Create a report showing:

Top 10 processes by CPU time
Display: Process name, CPU seconds (rounded to 2 decimals), Memory in MB
Export to CSV

💡 Hint

Use WorkingSet64 for memory, divide by 1MB, and use [math]::Round() for clean numbers.

Show Solution

Get-Process |
    Sort-Object CPU -Descending |
    Select-Object -First 10 Name,
        @{N='CPUSeconds';E={[math]::Round($_.CPU,2)}},
        @{N='MemoryMB';E={[math]::Round($_.WorkingSet64/1MB,2)}} |
    Export-Csv top_processes.csv -NoTypeInformation
 
# View the CSV
Import-Csv top_processes.csv | Format-Table

🏋️ Exercise 3: Disk Space Report

Create a disk space report:

Get all drives
Show Name, Size in GB, Used in GB, Free in GB, % Used
Sort by % Used descending

Show Solution

Get-PSDrive -PSProvider FileSystem |
    Where-Object { $_.Used -gt 0 } |
    Select-Object Name,
        @{N='SizeGB';E={[math]::Round(($_.Used + $_.Free)/1GB,1)}},
        @{N='UsedGB';E={[math]::Round($_.Used/1GB,1)}},
        @{N='FreeGB';E={[math]::Round($_.Free/1GB,1)}},
        @{N='PercentUsed';E={[math]::Round($_.Used/($_.Used+$_.Free)*100,1)}} |
    Sort-Object PercentUsed -Descending |
    Format-Table -AutoSize

Summary

The pipeline is PowerShell's superpower:

Objects flow through - not text, but complete structured objects
Where-Object filters based on conditions
Select-Object chooses properties and creates calculated properties
Sort-Object orders objects by property values
Group-Object organizes objects into categories
Measure-Object calculates statistics
ForEach-Object performs actions on each object
Format cmdlets control display (always use last!)
Export cmdlets save results to files

Master the pipeline, and you master PowerShell.

The PowerShell Pipeline

Written by the ShellRAG Team