sed: Stream Editing and Find-and-Replace

sed (stream editor) reads text line by line, applies editing commands and prints the result. It is the tool for changing configuration files in scripts, rewriting URLs across a project, or trimming log output in a pipeline — anything you would otherwise open an editor for, but repeatable and automatable. This lesson covers substitution, addressing lines, deleting and inserting, and the in-place editing flag that makes sed a deployment workhorse.

How sed Works

sed takes a script of one or more commands and applies it to every line of input in turn. Each command has the form [address]command[options]. Without an address the command applies to every line; with one it applies only to matching lines. Input comes from files or standard input; output goes to standard output unless you use -i.

bash

sed 's/cat/dog/' pets.txt          # from a file
echo "one two" | sed 's/one/1/'    # from a pipe

The Substitute Command

s/pattern/replacement/flags is what most people use sed for. The pattern is a regular expression (basic by default, extended with -E).

bash

sed 's/error/ERROR/' app.log         # first match on each line
sed 's/error/ERROR/g' app.log        # every match on each line (g = global)
sed 's/error/ERROR/gi' app.log       # case-insensitive too
sed 's/error/ERROR/2' app.log        # only the second match per line
sed -E 's/([0-9]+)-([0-9]+)/\2-\1/' dates.txt   # swap two groups
sed 's/.*"user":"\([^"]*\)".*/\1/' events.json  # extract a value (BRE groups need \( \))

& in the replacement stands for the whole match, and \1–\9 refer to capture groups:

bash

echo "version 2.4" | sed -E 's/[0-9]+\.[0-9]+/v&/'     # version v2.4

The delimiter does not have to be /. When the pattern contains slashes — file paths, URLs — pick another character to avoid escaping:

bash

sed 's#/var/www/old#/var/www/new#g' site.conf
sed 's|http://|https://|g' links.txt

Addresses: Choosing Which Lines

| Address | Selects | |---|---| | 5 | line 5 | | 1,10 | lines 1 to 10 | | $ | the last line | | /regex/ | lines matching the pattern | | /start/,/end/ | from a line matching start to the next matching end | | 5,$ | from line 5 to the end | | 1~2 | every second line, starting at 1 (GNU) | | /regex/! | lines that do not match |

bash

sed '3s/foo/bar/' file.txt              # substitute only on line 3
sed '/^#/!s/localhost/0.0.0.0/' app.conf   # skip comment lines
sed -n '/BEGIN/,/END/p' report.txt      # print a block

Printing, Deleting, Inserting

-n suppresses automatic output so that only lines you explicitly print appear. d deletes lines, i inserts before, a appends after and c changes a whole line.

bash

sed -n '1,5p' file.txt                  # like head -5
sed -n '/ERROR/p' app.log               # like grep ERROR
sed '/^$/d' file.txt                    # delete blank lines
sed '1d' data.csv                       # drop the header row
sed '$d' file.txt                       # drop the last line
sed '/DEBUG/d' app.log                  # drop debug lines
sed '1i # Generated file - do not edit' config.ini
sed '/\[Service\]/a Restart=always' app.service
sed '/^Port /c Port 2222' sshd_config

Editing Files in Place

-i writes the result back to the file. Give it a suffix to keep a backup, which costs nothing and has rescued many administrators.

bash

sed -i 's/DEBUG=true/DEBUG=false/' .env
sed -i.bak 's/8080/80/g' docker-compose.yml     # creates docker-compose.yml.bak
sed -i '' 's/foo/bar/' file.txt                 # macOS/BSD sed requires an explicit (empty) suffix
find . -name "*.md" -exec sed -i 's/Master/Main/g' {} +

Preview first by running the same command without -i, or pipe it through diff:

bash

diff <(cat .env) <(sed 's/DEBUG=true/DEBUG=false/' .env)

Multiple Commands

Chain commands with -e, separate them with ;, or put them in a file with -f.

bash

sed -e 's/foo/bar/g' -e '/^$/d' file.txt
sed 's/foo/bar/g; /^$/d' file.txt
sed -f cleanup.sed file.txt

Commands run in order on each line before the next line is read, so a line deleted by an early command never reaches later ones.

Common Mistakes

Forgetting g, so only the first occurrence on each line changes.
Using +, ? or | without -E and getting a literal match.
Escaping / inside a URL instead of switching the delimiter.
Running sed -i on a symbolic link, which replaces the link with a regular file (add --follow-symlinks).
Expecting sed to edit across lines; it works one line at a time unless you use the advanced N command. For multi-line structures prefer awk or a proper parser.

Key Takeaways

s/pattern/replacement/g is the core command; use -E for extended regex and \1 for groups.
Any delimiter works — use # or | when the pattern contains slashes.
Addresses (5, 1,10, $, /regex/, ranges) restrict commands to specific lines.
-n with p prints selectively; d, i, a and c delete, insert, append and change lines.
-i.bak edits in place with a backup; preview without -i first.

Next lesson: awk: Column-Based Text Processing — treat every line as fields and compute sums, filters and reports.