sed has a memory. Here’s how to use it

Understand the cycle before adding commands

Run sed -n '/ERROR/p' service.log and the command seems simple enough: find a line, print it. Add a second command and the surprises begin. A substitution changes what the next address sees; d skips the remaining script; H adds a newline before it saves anything. Follow the two buffers through one cycle.

sed is a program that repeatedly reads input into a working buffer, runs a script, and normally prints the buffer. That working buffer is the pattern space. A second buffer, the hold space, survives across cycles. Once you understand that sequence, apparent surprises such as duplicated output become predictable rather than mysterious.

With -n, automatic printing stops; p becomes an explicit output operation. A d command discards the pattern space and starts the next cycle immediately, skipping the rest of the script. Commands later in a script see the text produced by earlier commands. Addresses therefore test the current pattern space, not an immutable copy of the original line.

printf 'red\nblue\n' | sed 's/red/green/; /green/s/$/ matched/'
# Without -n, p would print each matching line twice.
sed -n '/ERROR/p' service.log

Addresses are a selection language

A numeric address selects a line; $ selects the last line. A regex selects matching lines. A two-address range stays active from its first match until its closing condition. The ending regex is normally tested starting with a later line, so a start and end marker on one line do not necessarily make a one-line range. Repeated start markers inside an active range do not create nested ranges.

Use braces to apply several operations to the same selection. Inverting an address with ! is often clearer than constructing a complicated negative regex. For structured formats with nesting, a range is only a textual approximation: an INI section can be manageable, but arbitrary XML, YAML and JSON need their actual parsers.

sed -n '/^BEGIN$/,/^END$/p' report.txt
sed '/^[[:space:]]*#/d; /^[[:space:]]*$/d' settings.conf
sed -n '10,20{=;p;}' service.log

Replacement text is its own little language

The pattern and replacement halves of s are interpreted differently. In replacement text, & means the entire match, while backreferences insert captured groups. A literal ampersand needs escaping. Switching the delimiter reduces clutter for paths, but does not make arbitrary replacement data safe: the new delimiter, backslashes and ampersands still have meaning.

Use -E for extended regular expressions on modern GNU and BSD sed; avoid assuming GNU-only escapes work everywhere. Character classes such as [[:space:]] are more portable than \s. Locale affects classes and ranges. Set LC_ALL=C when processing deliberately byte-oriented ASCII records, not as a universal cure for text containing other languages.

printf 'host=old\n' | sed -E 's/^([^=]+)=(.*)$/key=\1 value=\2/'
printf 'red blue\n' | sed 's/[a-z][a-z]*/[&]/g'
sed 's|/srv/old/|/srv/new/|g' paths.txt

The hold space gives sed a memory

h replaces the hold space; H appends to it, adding a newline first. g replaces the pattern space with the hold space; G appends the hold space. x swaps the two buffers. These operations let you remember context, pair lines and build records without starting another process for every line.

The reversal example below stores an increasingly large prefix and prints only at the end. It is an interesting demonstration, but tac is the practical choice where available. Repeatedly copying growing buffers consumes memory and time; a clever one-liner can quietly turn a streaming task into a whole-file operation.

# Reverse a small input using a growing hold space.
printf 'one\ntwo\nthree\n' | sed -n '1!G;h;$p'
# Add a blank line after each input line: the hold space starts empty.
sed G notes.txt

The small reversal prints three, two, one on separate lines. G uses the previous accumulated buffer; h saves the new one.

N, P and D create a sliding window

N appends the next input line to the pattern space with an embedded newline. P prints only up to the first embedded newline. D removes that first segment and restarts the script without reading a new line when a newline remains. Together they can inspect overlapping pairs rather than disjoint pairs.

The guarded script below prints every adjacent pair, including pairs that cross the boundary between the pairs a naive N;p;d script would produce. For a one-line input it prints nothing. Guarding N on the last line avoids differences in end-of-input handling. Whole-file recipes using :a;N;$!ba trade away bounded memory and should be treated accordingly.

printf 'a\nb\nc\nd\n' | sed -n '
$!N
/\n/p
D
'
# Inspect hidden characters rather than guessing at whitespace.
sed -n 'l' suspect.txt

For a, b, c, d the pairs are a/b, b/c and c/d. The repeated middle lines show that the window overlaps.

Find an adjacent duplicate without remembering the file

Two-line windows can detect adjacent duplicates while storing only two records. This example emits one result for each equal neighboring pair. Three blue lines produce two matches, which is different from printing each duplicated value once. Use uniq when that simpler operation is what you need.

The unusual part is D: after comparing a pair, it discards only the first line and restarts on the second. The next N adds a new neighbor. That makes the buffer a sliding window rather than a collection of unrelated pairs.

printf 'red\nred\nblue\nblue\nblue\n' |
  sed -n '$!N;/^\(.*\)\n\1$/P;D'

Branch on a successful substitution

b branches unconditionally. t branches if a substitution succeeded since the most recent input read or conditional branch. A successful substitution can replace text with identical text; success does not necessarily mean progress. A loop using that flag must remove something or otherwise advance toward a terminating condition.

The example strips non-nested innermost parentheses repeatedly. This is useful as an illustration of iterative reduction, not a complete parser for expressions: quoting, escaping and malformed input change the problem. A loop that keeps matching without shrinking its input can consume CPU indefinitely. GNU timeout can put a time bound around experiments, but does not prove correctness.

printf 'a(b(c)d)e\n' | sed -E ':again; s/\([^()]*\)//g; t again'
# Expected result: ae

Keep a newline inside a filename

GNU sed -z changes the record delimiter to NUL. An embedded newline can then remain part of a record, which is useful when consuming NUL-delimited pathname streams. This is a GNU extension; do not put it into a portable sed script without checking the implementation.

od below makes the boundary visible. A newline appears inside the first name, while NUL separates the names. Feeding ordinary newline text to -z without NUL separators may turn the whole input into one large buffer.

printf 'first\nfile\0second file\0' | sed -z 's/^/prefix: /' | od -An -tc

In-place editing changes filesystem behavior

-i is convenient but is not a portable POSIX option. GNU and BSD argument conventions differ. The common attached backup suffix -i.bak is useful on many implementations, but test your target system and avoid assuming a script tested on Linux behaves identically on macOS.

GNU sed writes a temporary file and replaces the original. That can break hard-link relationships; symlink behavior deserves separate attention. Never redirect output onto the input pathname: the shell can truncate it before sed reads anything. Also, -n combined with -i can empty a file if no explicit print command runs. Preview the transformation and its diff before changing important configuration.

sed 's/old_endpoint/new_endpoint/g' settings.conf > settings.preview
diff -u settings.conf settings.preview
# After review, on a system whose -i syntax you have checked:
sed -i.bak 's/old_endpoint/new_endpoint/g' settings.conf

A sed script is executable policy

A GNU sed script can be a shell launcher. Its e command delegates to the shell with the permissions of the sed process. In an ordinary user session, the example below simply prints a message. Under unrestricted sudo, that same feature would inherit much greater authority. GTFOBins documents this privilege-policy abuse; it is not a sed CVE.

GNU --sandbox rejects scripts containing e, r or w commands. It is useful defense in depth but is not an operating-system sandbox: it does not remove access to designated input files, prevent excessive resource use, or make privileged in-place editing safe. Run untrusted transformations with minimal privileges, explicit input files and resource limits.

# GNU sed only; run as an ordinary user.
printf 'ignored\n' | sed -n '1e printf "sed ran a command\n"'
# The same script is rejected by GNU sandbox mode.
printf 'ignored\n' | sed --sandbox -n '1e printf "sed ran a command\n"'

The first command prints a message. The second exits with an error because e is forbidden. r and w are also forbidden in GNU sandbox mode.

A verified symlink race: CVE-2026-5958

CVE-2026-5958 concerns GNU sed invoked with both -i and --follow-symlinks. The affected implementation resolves the link target and opens the original link path in separate operations. An attacker able to replace the link during that window can influence which content is read and where the processed result is written. A privileged caller makes the consequence much more serious.

The Debian security tracker identifies an upstream fix in 4.10 and distribution backports, including Debian package versions still numbered 4.9. Check your distribution advisory and package revision rather than judging by the upstream number alone. Avoid privileged edits through links or directories controlled by untrusted users. Ordinary filtering of a trusted file is not the same exposure as this particular option combination.

The public disclosure includes a researcher reproduction. That establishes an exploitable flaw under the described conditions; the sources cited here do not establish use in a malicious intrusion.

Use sed where its model stays visible

A good sed script has a clear cycle, limited state and an obvious termination condition. Keep longer scripts in a reviewed -f file, with comments explaining why a branch or buffer operation exists. Test empty input, a single line, a missing final newline, a missing closing marker and unexpectedly long records. Those cases reveal assumptions that ordinary examples hide.

When the job becomes arithmetic, grouped statistics or a keyed lookup, reach for awk. When you need a grammar, reach for a parser. Tomorrow we will connect sed and awk through an explicit record format, using sed for local textual transformations and awk for decisions that require fields, counters and state.