pipe-shell
A POSIX-ish command interpreter in C11.
A shell is the universal Swiss-army knife of Unix. A shell that
handles pipelines correctly in a few hundred lines is the
smallest version of bash worth understanding.
Once you can read it, the rest of Unix — daemons, init, job
control, container runtimes — falls into place.
This is that shell. It parses a command line, splits it on
|, applies redirections, forks a child for each
stage, and reaps the foreground pipeline in order. The whole
executor is about 30 lines of C.
What's in the box
- Simple commands:
ls -la - Pipelines:
ls | grep foo | wc -l - Input redirection:
wc -l < file.txt - Output redirection:
ls > out.txtorls >> out.txt - Stderr redirection:
ls /no 2> err.txtor2>>(append) - Multiple per stage:
cmd < in > out 2> err - Background: trailing
& - Built-ins:
exit [N],cd [DIR] - Interactive REPL and one-shot
--runCLI
What's deliberately missing: glob expansion, variable expansion,
quoting of any kind, ;/&&/
||, job control. Each of these is a 100 to 200
line feature on top of the existing structure. They are good
follow-up work, not part of the minimum.
Try it
The playground below runs the same parser in JavaScript. Type a command line; the AST updates as you type. The right panel shows the kernel call sequence that an executor would make for that AST: pipe, fork, dup2, exec, wait. Click an example to load it.
Playground
Parsed AST
Kernel call sequence
The parser
The parser does three passes. First, strip a trailing
& and record the background flag. Then split
on | into stages — we use strchr
instead of strtok_r because the latter silently
drops empty trailing fields, which is exactly the syntax
error we want to detect. Then tokenize each stage on
whitespace, treating <, >, and
>> as single-character metacharacters that
consume the next word as the redirection target.
The parser is non-allocating. It mutates a local copy of the input line in place and stores pointers into it. No garbage, no allocator pressure, no copies.
The pipeline recipe
The executor is 30 lines. The shape is:
int prev_fd = -1;
for (i = 0; i < n_stages; i++) {
int pipe_fd[2] = {-1, -1};
if (i + 1 < n_stages) pipe(pipe_fd);
pid_t pid = fork();
if (pid == 0) {
run_stage(&stages[i], prev_fd, pipe_fd[1]);
}
/* parent: drop both ends of the previous pipe */
if (prev_fd >= 0) close(prev_fd);
prev_fd = pipe_fd[0];
if (pipe_fd[1] >= 0) close(pipe_fd[1]);
}
if (!background) wait for every child;
The crucial detail: every pipe fd the parent
has a copy of must be closed, or the consumer's
read(2) will block forever. Get that wrong and
the user sees a shell that prints nothing and never returns.
I made this bug twice while writing the executor. It is the
most common shell bug.
The four rules
Pipeline programming has four rules. Get them right and the rest is bookkeeping.
-
The parent closes its copy of every pipe fd. If it does
not, the consumer's
read(2)blocks forever because the kernel will not generate EOF until every reference to the write end is gone. -
The child
dup2s the read end of its input pipe to stdin, and the write end of its output pipe to stdout. After the dup, the pipe fds can be closed. -
The child closes every fd it does not need before
exec. The fd table is preserved acrossexec, so unclosed fds leak into the new program. -
The parent
waitpids every child for a foreground pipeline, or none of them for a background pipeline.
Tests
62 assertions across 17 test cases, all passing. The suite
covers the parser (11 cases: simple commands, all five
redirect forms, multi-redirect, pipelines, background,
error cases) and the executor (6 cases: true,
false, input redirect, output redirect,
stderr redirect, multi-redirect, cd built-in).
62 assertions across 17 test cases, all passing. The suite
covers the parser (11 cases: simple commands, all five
redirect forms, multi-redirect, pipelines, background,
error cases) and the executor (6 cases: true,
false, input redirect, output redirect,
stderr redirect, multi-redirect, cd built-in).
Source
The repository lives at github.com/404Piyush/pipe-shell. The public header, the parser + executor, and the test suite are the three files worth reading. The header is the contract. The parser + executor is about 250 lines. The test suite is about 300 lines.
To build and test locally:
git clone https://github.com/404Piyush/pipe-shell
cd pipe-shell
make test
./pipe-shell
One of the gpu-engineering curriculum, a three-year systems and hardware roadmap. More projects incoming.