Sort: decide what order—and equality—really mean

Start with lines, not columns

An incident report can contain all the right records and still tell the wrong story. Alphabetical ordering puts 100 before 20. A seemingly harmless uniqueness flag can discard an entire service. The useful question is not “how do I sort this?” but “what must remain distinguishable after sorting?”

`sort` reads records, compares them, and prints them in order. Normally a record is one line, and the comparison uses the whole line. It is a standalone utility, not a shell built-in. The simplest useful invocation is `sort names.txt`: read that file and print the ordered lines without changing the file.

This walkthrough uses Bash and GNU coreutils 9.4. Basic line sorting and options such as `-n`, `-k`, `-t`, and `-u` are widely portable; later GNU extensions are labeled. Run the examples in a disposable working directory. The commands below create `names.txt` there; they overwrite that name if it already exists. `printf` prints its quoted arguments, `%s` inserts each argument, and `\n` ends each line. Single quotes keep the shell from interpreting the text. The shell’s `>` writes output to a file rather than the terminal.

printf '%s\n' web api db > names.txt
LC_ALL=C sort names.txt

api
db
web

`LC_ALL=C` sets the C locale for this invocation only. A locale supplies comparison and character rules; C gives predictable byte ordering for these ASCII examples. Without the assignment, local language rules can change the ordering. Use the same locale throughout a processing pipeline. See the [GNU sort manual](https://www.gnu.org/software/coreutils/manual/html_node/sort-invocation.html).

Give the report an explicit format

We will rank service observations by latency, group them by service, and decide which duplicates to keep. Create `events.txt` below. Each record contains a service, latency in milliseconds, an observation label, and a sequence number. This overwrites `events.txt`; subsequent examples only read it unless stated otherwise.

The four fields are separated by a literal `|`. There is no header to accidentally sort into the data. The labels and sequence numbers make it possible to see whether ties move or records disappear.

printf '%s\n' \
  'web|20|first|1' \
  'api|100|first|2' \
  'web|3|second|3' \
  'api|20|second|4' \
  'db|20|first|5' > events.txt
cat events.txt

web|20|first|1
api|100|first|2
web|3|second|3
api|20|second|4
db|20|first|5

The backslash at the end of a command line continues it on the next line; it must be the final character. `cat` prints the file to confirm the fixture. This delimiter format is deliberately simple. `sort -t` is not a quoted-CSV parser: a delimiter inside a quoted CSV field is still a delimiter to sort.

Select the key and its comparison rule

Rank the report from lowest to highest latency. `-t` takes the field separator as its next argument, and `-k` selects a comparison key. In `-k 2,2n`, the key begins and ends in field 2; the attached `n` compares that key numerically. Fields are numbered from 1. The comma bounds the key: `-k 2` instead includes field 2 through the end of the line.

By default sort prints whole records, even when it compares only one field. It never extracts a column for us. Without the numeric modifier, a textual comparison of these latency strings puts 100 before 20 and 3.

LC_ALL=C sort -t '|' -k 2,2n events.txt

web|3|second|3
api|20|second|4
db|20|first|5
web|20|first|1
api|100|first|2

The three records at 20 milliseconds have equal numeric keys. GNU sort breaks that tie by comparing the whole lines, which puts api, db, and web in that order. A key defines the first comparison, not necessarily the only comparison.

Choose what a tie means

Suppose input order represents collection order, and equally slow observations should retain that order. GNU `-s` makes the sort stable: it disables the whole-line tie breaker. Records with equal selected keys remain in their original relative order.

This is an explicit data decision. Stability preserves the order supplied to sort; it does not establish that the input arrived chronologically. If multiple collectors interleaved records arbitrarily, a stable sort preserves that arbitrary tie order.

LC_ALL=C sort -s -t '|' -k 2,2n events.txt

web|3|second|3
web|20|first|1
api|20|second|4
db|20|first|5
api|100|first|2

Reverse one key, not the whole report

For triage, group observations alphabetically by service and put each service’s slowest observation first. Repeat `-k` to supply a secondary key. The first differing key decides the order.

`-k 1,1` compares just the service. In `-k 2,2nr`, `n` means numeric and the new `r` means reverse, so latency descends within each service. Keeping `r` attached to this key leaves the service names ascending. A global `-r` can reverse other comparisons too, so attach ordering modifiers to the keys whose meaning they change.

LC_ALL=C sort -t '|' -k 1,1 -k 2,2nr events.txt

api|100|first|2
api|20|second|4
db|20|first|5
web|20|first|1
web|3|second|3

Inspect the key instead of guessing

GNU `--debug` marks the portion of each output line used as a key and reports potentially questionable options on standard error, the separate channel for diagnostics. This is useful when a field boundary, leading space, or modifier gives an unexpected result.

Run the same numeric sort with debugging. For the first record, the first underline marks `3`; the second marks the entire record used by the default tie breaker. The output below shows only that first record. Remaining records have corresponding marks, and diagnostic wording can vary by version. Do not feed debug output into another data-processing command: it is an explanation of the sort, not the original report.

LC_ALL=C sort --debug -t '|' -k 2,2n events.txt

web|3|second|3
    _
______________

Expected output is abbreviated to the first record. GNU extensions in this article include `--debug`, stable sorting with `-s`, `-V`, `-z`, `-S`, and `--parallel`. Check a BSD or BusyBox implementation’s own manual before transferring these examples.

Uniqueness belongs to the key

GNU `-u` prints one representative of each group that compares equal and disables the whole-line tie breaker. Applied to whole lines, it removes repeated identical ASCII records in our C locale. Applied to a service key, it discards other observations for the same service—even if their latency and labels differ.

Here the goal is to retain the first supplied observation per service, then arrange those representatives by service. These are not the newest observations or the slowest observations. They are the first input representatives. The distinction matters when a deduplicated report becomes evidence.

LC_ALL=C sort -u -t '|' -k 1,1 events.txt

api|100|first|2
db|20|first|5
web|20|first|1

Adding latency as a second key would change equality to the service-and-latency pair. Likewise, `sort -n -u` can collapse different lines whose numeric prefixes compare equal. Decide whether equality means an entire record or selected fields before adding `-u`.

Use stability to compose different ordering rules

A stable sort also lets a second pass preserve a first pass’s work. First rank all observations by descending latency. Then group by service using a stable sort. Within each service, the second pass leaves the latency order intact.

The shell’s `|` connects the first command’s standard output to the second command’s standard input. No filename is needed on the second sort because it reads that incoming stream. This produces the same report as the two-key example above. Here one invocation is clearer and cheaper; separate passes become useful when a transformed key or a different delimiter makes one invocation awkward.

LC_ALL=C sort -s -t '|' -k 2,2nr events.txt |
  LC_ALL=C sort -s -t '|' -k 1,1

api|100|first|2
api|20|second|4
db|20|first|5
web|20|first|1
web|3|second|3

Both commands receive `LC_ALL=C`; placing that assignment before only the first command would not set the second command’s locale. For more elaborate transformations, the awk article explains field extraction and computed values. Sort compares keys; awk can calculate them first.

Turn inventories into a membership question

For an assessment of a supplied system inventory, sorting can reveal accounts introduced between snapshots. Create two synthetic username lists, sort each into a new file, and compare their membership. These commands overwrite the four named fixture files. They do not inspect or modify system accounts.

`sort -u` now operates on entire username lines. `comm` compares two already-sorted files: its first column is unique to the first file, its second to the second file, and its third shared. `-1` and `-3` suppress the first and third columns, leaving only entries in the later snapshot. Both input preparations and comm use the same locale.

printf '%s\n' deploy backup monitor > before.txt
printf '%s\n' monitor deploy backup support > after.txt
LC_ALL=C sort -u before.txt > before.sorted
LC_ALL=C sort -u after.txt > after.sorted
LC_ALL=C comm -1 -3 before.sorted after.sorted

support

This identifies a new name, not proof of an intrusion: an approved deployment could have created it, and an existing account could have changed privileges without changing its name. Keep the original artifacts. For positional text changes, use the diff article; for set membership, sorting deliberately removes source order. [GNU comm documentation](https://www.gnu.org/software/coreutils/manual/html_node/comm-invocation.html) describes the sorted-input requirement.

Numbers have more than one grammar

Ordinary `-n` compares decimal numeric prefixes exactly; it does not understand exponent notation or a leading plus sign. GNU `-g` parses a more general floating-point numeric prefix, including exponents. That extra grammar comes with possible precision loss and different treatment of nonnumbers.

This synthetic stream uses `printf` and the pipe already introduced. With `-g`, 2e3 is 2000, so it follows 1000. With `-n`, the recognized prefix of 2e3 is 2, so it sorts before 1000. Neither option validates that a whole field is a well-formed number. If invalid observations should fail the report, validate them before sorting rather than treating a numeric option as validation.

printf '%s\n' 2e3 1000 | LC_ALL=C sort -g

1000
2e3

Put numbered artifacts in human order

GNU `-V` supplies version ordering, which compares digit sequences numerically within text. That puts scan-2 before scan-10, unlike byte ordering. It is useful for numbered reports, rotation files, and similar artifacts.

It is not a promise to implement every project’s release policy or Semantic Versioning precedence. Package managers and release systems can have their own rules for prereleases, suffixes, and epochs. Use their comparison logic when the result determines an upgrade.

printf '%s\n' scan-10.txt scan-2.txt scan-1.txt | LC_ALL=C sort -V

scan-1.txt
scan-2.txt
scan-10.txt

See the [GNU version-sort overview](https://www.gnu.org/software/coreutils/manual/html_node/Version-sort-overview.html) for the intended text-and-number ordering model.

A filename is not necessarily a line

A filename may contain a newline. Sorting a newline-separated filename list then changes record boundaries: one name can become two apparent names. GNU `-z` instead reads and writes records terminated by NUL, the zero byte. Unix filenames cannot contain NUL, so this separator can preserve every filename byte.

The following demonstration creates no files. It sends two synthetic names through sort and displays the result as bytes. `printf` interprets `\0` as NUL here; the second synthetic name contains `\n` within the quoted format string, so that name includes a newline. `od` displays bytes instead of sending control characters directly to the terminal. Its `-A n` suppresses byte-offset labels, `-t x1` requests hexadecimal one-byte units, and `-v` shows all bytes without compressing repeated rows.

printf 'zebra\0odd\nname\0' | LC_ALL=C sort -z | od -A n -t x1 -v

 6f 64 64 0a 6e 61 6d 65 00 7a 65 62 72 61 00

`0a` is the newline inside the single name `odd\nname`; `00` separates that record from zebra. Every consumer after sort must also understand NUL boundaries. `-z` preserves boundaries, but it does not escape malicious terminal control sequences or stop a later command interpreting a filename as an option.

Check the contract and write a separate result

When another tool requires sorted input, check the exact ordering you promised. `-c` checks instead of rearranging records and reports the first disorder. This checks the original report against ascending numeric latency, which fails because the 3-millisecond record follows 100 milliseconds.

The shell variable `$?` is the preceding command’s exit status; double quotes let the shell expand it. Capture it immediately—another command would replace it. GNU sort returns 0 for a successful check, 1 for disorder, and 2 for an error. Do not collapse a missing input file and unsorted data into the same outcome.

LC_ALL=C sort -c -t '|' -k 2,2n events.txt
printf 'status=%s\n' "$?"

status=1

The status line is standard output; sort separately reports the disordered third record on standard error. To save a report, use a different output path, for example `LC_ALL=C sort -t '|' -k 2,2n events.txt > ranked.txt`. This overwrites ranked.txt. Never use `sort events.txt > events.txt`: the shell truncates the input before sort reads it. `-o` names an output file and supports ordinary in-place sorting, but preserving the original and writing a separate result is safer against interruption or I/O failure.

The large sort has a disk footprint

Small fixtures hide an important operational fact: a large sort can spill intermediate runs to temporary files. GNU `-S` requests an initial sort-buffer size, not a hard process-memory cap. `-T` chooses a temporary-file directory, and `--parallel` sets the number of sorting threads. More threads can increase memory consumption; a long input record can grow the buffer beyond the requested size.

Before running this small resource-policy example, create a private temporary directory. `mktemp -d` creates a directory with a unique name; `$(...)` runs that command and substitutes its output, which we assign to the shell variable `sort_tmp`. Double quotes preserve the directory name as one argument. `64M` means 64 MiB for this GNU option, and `--parallel=2` requests two threads. The command prints the same ascending-latency output shown earlier; this tiny fixture need not spill to disk.

sort_tmp=$(mktemp -d)
LC_ALL=C sort -T "$sort_tmp" -S 64M --parallel=2 -t '|' -k 2,2n events.txt

web|3|second|3
api|20|second|4
db|20|first|5
web|20|first|1
api|100|first|2

After sort finishes, remove the now-empty fixture directory with `rmdir "$sort_tmp"`; rmdir only removes an empty directory. For sensitive real input, choose approved storage and consider temporary data as part of the data-handling policy. Private directory permissions are not encryption or secure erasure. GNU `--compress-program` can invoke an external helper for temporary data; do not let untrusted configuration select that program. This is documented command execution by an option, not a demonstrated vulnerability in sort. Similarly, a broadly privileged permission to run sort can allow reading and writing files with that privilege. Limit the granted operation rather than assuming a text utility is inherently confined.