No matching examples. Try another query or clear the filters.
A visual reference · AWK
AWK, one change at a time
From one field to useful text transformations and reports.
Complete source. A small change. Its actual result.
The Org file is the canonical source. Read the HTML guide beside your work,
or download its PDF and source for offline reading.
Last changes: 2026-10-11. Unified tour–chapter–example IDs, numbered chapter anchors and updated direct links, downloads and comparisons. Reader pages now omit publishing instructions and highlight PDF downloads.
Inspect the complete local data, field separator and input-file reference.
AWK program
Read the whole Org source block, including its header and footer.
Captured result
Compare actual output with the input; each example starts a fresh AWK process.
The examples use portable AWK constructs and local input data. Org headers
choose input files, command-line arguments, result formatting and tangling;
they are separate from the AWK language. Explicit Noweb examples show their
expanded program. Input records are not edited on disk; file-writing examples
identify their generated output and show its read-back result.
Read from the beginning or jump to a stable example ID. Each example includes
its complete input and captured result. Copy link preserves the address of the
guide you are reading, including its current domain and path.
Read this first
This reference starts with one field and builds toward small reports. Every numbered example has a stable ID, Awk code (with explicit expansion for the two reuse examples), an input reference, a focused change, and captured standard output. Each example starts a new Awk process; state does not carry between blocks. The two explicitly marked Noweb examples share a function definition.
The examples use portable Awk constructs and ASCII sample data. They were executed with mawk 1.3.4 in LC_ALL=C. They were not executed inside Emacs or on macOS in this build. GNU-specific features are discussed separately, not silently required. This is a reference, not a set of exercises.
The execution model
An Awk program is a sequence of pattern/action rules. BEGIN runs once before input. For each record, matching rules run in source order. END runs once afterward. next skips to another record; exit finishes input processing but still runs END.
$0 is the whole record; $1 through $NF are fields. Ordinary variables have no dollar prefix. Input records are not edited on disk. print produces output; saving that output is a separate operation.
The main input has four rows and no header. Fields are name, group, score:
Ada labA 12
Ben labB 7
Cara labA 15
Dan labB 10
Boundaries and practical traps
FS="," is not a general CSV parser: quoted commas, embedded newlines and escaped quotes need a real CSV-aware parser. A recent GNU Awk offers --csv, but it is not used here.
Portable array iteration order is unspecified. The examples explicitly preserve order. GNU asort/asorti and PROCINFO["sorted_in"] are extensions.
Avoid GNU-only gensub, FPAT, arrays of arrays, and the third match argument in code intended for all Awk implementations. Check your actual interpreter, not just the command name.
The examples use ASCII, LF line endings, and the C locale. Character counting, case conversion, decimal punctuation, and regex behavior can depend on implementation and locale. CRLF may leave a trailing carriage return.
Numeric input must be validated for your data model. These examples do not provide a universal decimal/exponent parser. Large integers may lose precision in ordinary floating-point arithmetic.
Awk regexes use extended regular expressions. Do not assume PCRE escapes such as \d or lookbehind. Prefer [0-9] and POSIX classes such as [[:space:]].
print uses OFS and ORS; printf does not. Width is a minimum, not a truncation limit. Escape a literal percent as %% in a printf format.
Redirection with print value > "out.txt" opens a file and can truncate it when first opened. print value >> "out.txt" appends. close("out.txt") releases it. AWK-21-001 through AWK-21-003 demonstrate these effects on their named files under generated/. They reset their own outputs on each run; they do not edit the input fixtures.
NR==FNR is a common two-file idiom but misidentifies records when the first file is empty. The lookup example uses the filename explicitly; rename its test if you rename the file.
Awk has functions, not a portable preprocessor macro system. In Org, Noweb can assemble shared code with :noweb yes and . Most programs are complete. AWK-18-003 and AWK-18-004 explicitly demonstrate Noweb with the shared function below; their expanded standalone scripts are included.
The two REUSE examples reference this one named block. In Emacs, C-c C-v v (org-babel-expand-src-block) previews the complete expanded program. :noweb yes expands on execution, tangling and native export. The custom reference preserves the reference and also shows its expanded form. The Python builder supports only the simple named source inclusion used here, not all Org Babel or Noweb features.
function twice(x) { return 2 * x }
References and scope
The examples and explanations are original. Consult these primary references for the complete specifications and implementation-specific details:
Version: 1.2. Generated results are verified by executing every numbered block; this is not a claim of cross-platform or Emacs integration testing. No timings or random results are used.
Initialize from the first record. Initializing a maximum to zero would fail when all values are negative. || short-circuits when the first test succeeds.
Change: Remove duplicate lines while preserving first-seen order.
A missing array value acts as zero. Post-increment returns the old value; ! makes the first occurrence true. Later occurrences have a nonzero old value.
This deliberately narrow validation accepts only nonnegative integer strings. Adapt it for signed values or decimals. Do not silently interpret NA as zero.
Read the lookup file first. FILENAME selects it explicitly, avoiding the empty-first-file trap of NR==FNR. This exact filename must match the command-line argument.
Change: Add a fourth field containing twice the score.
$(NF + 1) addresses the first field beyond the record. Assigning there increases NF and rebuilds the record. Parentheses make the computed field index clear.
Reducing NF drops trailing fields and rebuilds $0 on the Awk implementations targeted here. Printing $1, $2 is a simpler alternative when no further record editing is needed.
Change: Expose conversion rather than relying on implicit types.
Literal strings compare lexically here: "10" precedes "2". Adding zero forces numeric conversion. Awk input fields can have numeric-string attributes, so do not generalize from quoted literals to every field comparison.
Change: Accept either of two complete group names.
Parentheses group alternatives. ^ and $ anchor the entire group, so only the complete alternatives match. ^labA|labB$ would anchor the alternatives differently.
Change: Keep the program fixed and change the pattern.
The header passes -v pattern=^A. A variable on the right of ~ is interpreted as a regular expression. Prefer /.../ for fixed patterns; dynamic strings introduce another escaping layer.
In sub/gsub replacement text, & inserts the complete matched text. Omitting the third argument modifies $0. Literal ampersands need additional escaping.
Portable multidimensional syntax combines subscripts using SUBSEP; it is not an array of arrays. Fixed output order is deliberate. Columns are group, pass count, review count.
Change: Produce a deterministic numeric ordering without GNU extensions.
Insertion sort is portable but quadratic in the worst case. This is suitable for tiny demonstrations; prefer an external sort or GNU asort for large collections. All values here are numeric.
Change: Bound memory while maintaining a rolling statistic.
A three-slot ring buffer discards the oldest value as each new one arrives. The first two windows are partial; thereafter the mean always uses three records.
Change: Extract section content without emitting delimiters.
A state variable selects records between marker lines. next keeps the marker lines out of the output. This simple format does not support nested sections.
Header records update section; later nonempty records inherit it. NF skips empty lines. The fixture begins with a header; add validation if that cannot be assumed.
This inner join emits only matching score records. Multiple lookup rows with the same key would overwrite earlier values; this fixture has unique lookup keys.
Change: Load a lookup without making it part of the main input sequence.
getline line < path reads a separate stream into line. It returns 1 for a record, 0 at EOF, -1 on error; it does not replace $0 or advance NR/FNR. close releases the file. The normal input remains scores.txt.
The incremental update maintains a mean using constant memory. It avoids a separate potentially large total but still uses finite-precision arithmetic.
Change: Extend the online mean with a measure of spread.
Welford-style updates accumulate squared deviations without subtracting two large sums. n-1 gives the sample variance; sqrt converts it to standard deviation.
Change: Express each score as a percentage of a known maximum.
The header supplies -v maximum=20. %% prints a literal percent sign in printf. A configurable denominator must be checked for zero in a general-purpose tool.
Change: Prevent a helper function from modifying global scratch state.
All function parameters are local. The unpassed extra parameter text acts as a local scratch variable; spacing before it is just a convention. The global text retains its value.
Change: Reuse an operation that updates a shared accumulator.
Arrays are passed by reference: changing a[key] changes the caller's array. Scalar arguments are passed by value. Do not pass a scalar where the function expects an array.
Change: Reuse source text while keeping the actual output visible.
The header sets :noweb yes. Org inserts the named awk-twice source block before execution or tangling. Its definition is printed in the shared-code appendix. The generated .awk script contains the expanded function.
Change: Use one shared implementation in a second context.
The same named function now participates in an aggregate. A source edit in awk-twice changes both examples at the next rebuild. This is text inclusion by Org, not an Awk macro feature.
Change: Inspect record shape as a small validation report.
An Awk data file is read as records, not executed as program text. Program text comes from the source block (or the script supplied with -f). Keep code and input paths distinct in automated pipelines.
Change: Create a small, documented handover format.
A tab character separates each column. This fixture contains no embedded tabs or newlines in values. The captured output can feed another tool; arbitrary text needs an explicit escaping or serialization policy.
The optional sign is followed by digits with an optional fractional part, or a leading dot plus digits. An optional exponent has its own optional sign. This grammar excludes NaN, infinity, hexadecimal and thousands separators.
Change: Do not turn absent measurements into zero.
With a literal semicolon separator, an empty third field remains distinguishable from a numeric zero. The fixture uses only numbers, empty values and NA; arbitrary other text would need numeric validation.
The denominator counts observed values, including zero. Missing values are counted separately. Reporting both counts makes the mean easier to interpret.
The first row maps header names to positions. Column order may change without changing the lookup expression. Duplicate header names would overwrite earlier positions; the next example detects them.
Change: Separate the delimiter from the decimal convention.
Under LC_ALL=C, numeric conversion expects a decimal point. This conversion assumes exactly one decimal comma, no thousands separator, and otherwise valid numeric text; it is not a general locale parser.
Each wide record produces two long records. The sample ID is repeated; each measurement name comes from the header. This version deliberately fixes the number of measurement columns.
The loop replaces two explicit print statements. This assumes every data row follows the same header and has the expected number of fields; use a field-count check for irregular input.
A composite key stores each sample/measurement pair. First-seen sample order is retained. The fixture has every pair exactly once; duplicate pairs would keep the last value.
Change: Avoid silent blanks when a pair is absent.
Parenthesized subscripts can test membership of a composite key. This distinguishes an absent cell from a present zero. Missing-value spelling is a deliberate output convention.
Rows become columns, including the original header. All cells are retained in memory. This example assumes a rectangular whitespace-separated table without spaces inside individual values.
Change: Use constant memory when the input is grouped.
Only the current group total is stored. Identical keys must be contiguous; otherwise the same key produces more than one summary row. END flushes the last group.
Change: Expand one list value into several records.
A semicolon separates the ID from a comma-separated list. split handles the inner list. There is no quoting or escaping in this deliberately simple format.
Change: Write selected columns and inspect the resulting file.
The first > opens and truncates this named output; subsequent writes keep using the open stream. END closes it and reads it back so the displayed output is the actual file content. The generated directory must already exist.
BEGIN deliberately resets this example file for repeatable runs. Each >> then reopens it in append mode. Using > after each close would truncate earlier output. Closing for every row is educational, not a performance recommendation.
Two fixed paths avoid turning input values directly into filenames. Non-labA rows go to the second file. Both files are initialized, closed after writing, and read back in a fixed order.
Change: Handle commas and quotes when emitting CSV.
Each field is enclosed in double quotes; internal double quotes are doubled. This writes valid CSV for the two fields in the fixture. The input is simple semicolon-delimited text, not a CSV parser.
Change: Contrast string truncation with numeric minimum width.
For %s, the precision .6 limits the displayed string length; the width 6 pads shorter strings. %03d zero-pads integers to at least three places but does not truncate longer numbers.
Change: Create output that can be reused in a report.
The first two lines form an Org table header and separator. The fixture contains no vertical bars or newlines in values. In this tour, output remains verbatim; :results output raw would insert the emitted table as Org markup.
Change: Report how much the transformation changed.
gsub returns the number of matched runs, not the number of individual whitespace characters. Because the target is a copy, $0 is preserved. This collapses whitespace but does not trim its ends.
After the first pass, close releases the input stream. A later getline can then reopen the file at the beginning. Without close, the second pass would remain at EOF. The fixture exists; production code should also check for -1 read errors.
Change: Combine text normalization, splitting and arrays.
Punctuation becomes whitespace and case is folded before counting. First-seen order makes the report deterministic. This C-locale tokenization is a simple ASCII policy; it is not linguistic word segmentation.
Change: Turn several record-level facts into one summary.
The fixture schema is path, numeric status, bytes. A status of 400 or above is counted as an error. Real logs may contain quoting, missing values or other fields and require a suitable parser.
[?] matches a literal question mark; .* consumes the suffix. This groups by the textual path before the query. It does not decode percent escapes or otherwise normalize URLs.
Change: Separate value selection from output ordering.
Each new value replaces the previous one for the same key, while the output order follows each key's first appearance. Last means last in file order, not latest by timestamp.
Change: Start a comparison with two explicit event types.
The first file is a lookup; the second drives output order. This reports additions and changes but deliberately omits deletions. Keys are unique in each snapshot.
Change: Assemble validation, transformation and evidence of the result.
Two early rejection rules keep invalid rows out of the calculation. The output separates accepted records with a data prefix from a small audit summary. The score grammar here is deliberately limited to nonnegative integers.