pipe-shell
A POSIX-ish command interpreter in C11.
A shell is the universal Swiss-army knife of Unix. A 500-line
shell that handles pipelines correctly is the smallest version
of bash worth understanding. Once you can read it,
the rest of Unix — daemons, init, job control, container
runtimes — falls into place.
This is that 500 lines. It parses a command line, splits it on
|, applies redirections, forks a child for each
stage, and reaps the foreground pipeline in order. The whole
executor is about 30 lines of C.
What's in the box
- Simple commands:
ls -la - Pipelines:
ls | grep foo | wc -l - Input redirection:
wc -l < file.txt - Output redirection:
ls > out.txtorls >> out.txt - Background: trailing
& - Built-ins:
exit [N],cd [DIR] - Interactive REPL and one-shot
--runCLI
What's deliberately missing: glob expansion, variable expansion,
quoting of any kind, ;/&&/
||, job control. Each of these is a 100 to 200
line feature on top of the existing structure. They are good
follow-up work, not part of the minimum.
Try it
The playground below runs the same parser in JavaScript. Type a command line; the AST updates as you type. The right panel shows the kernel call sequence that an executor would make for that AST: pipe, fork, dup2, exec, wait. Click an example to load it.
Playground
Parsed AST
Kernel call sequence
The parser
The parser does three passes. First, strip a trailing
& and record the background flag. Then split
on | into stages — we use strchr
instead of strtok_r because the latter silently
drops empty trailing fields, which is exactly the syntax
error we want to detect. Then tokenize each stage on
whitespace, treating <, >, and
>> as single-character metacharacters that
consume the next word as the redirection target.
The parser is non-allocating. It mutates a local copy of the input line in place and stores pointers into it. No garbage, no allocator pressure, no copies.
The pipeline recipe
The executor is 30 lines. The shape is:
int prev_fd = -1;
for (i = 0; i < n_stages; i++) {
int pipe_fd[2] = {-1, -1};
if (i + 1 < n_stages) pipe(pipe_fd);
pid_t pid = fork();
if (pid == 0) {
run_stage(&stages[i], prev_fd, pipe_fd[1]);
}
/* parent: drop both ends of the previous pipe */
if (prev_fd >= 0) close(prev_fd);
prev_fd = pipe_fd[0];
if (pipe_fd[1] >= 0) close(pipe_fd[1]);
}
if (!background) wait for every child;
The crucial detail: every pipe fd the parent
has a copy of must be closed, or the consumer's
read(2) will block forever. Get that wrong and
the user sees a shell that prints nothing and never returns.
I made this bug twice while writing the executor. It is the
most common shell bug.
The four rules
Pipeline programming has four rules. Get them right and the rest is bookkeeping.
-
The parent closes its copy of every pipe fd. If it does
not, the consumer's
read(2)blocks forever because the kernel will not generate EOF until every reference to the write end is gone. -
The child
dup2s the read end of its input pipe to stdin, and the write end of its output pipe to stdout. After the dup, the pipe fds can be closed. -
The child closes every fd it does not need before
exec. The fd table is preserved acrossexec, so unclosed fds leak into the new program. -
The parent
waitpids every child for a foreground pipeline, or none of them for a background pipeline.
Tests
56 assertions across 13 test cases, all passing. The suite
covers the parser (8 cases: simple commands, all three
redirections, pipelines, background, error cases) and the
executor (5 cases: true, false, input redirection, output
redirection, append redirection, pipeline-with-redirection,
the cd built-in).
The test framework is in the repo, no external dependency. Each
test is a plain C function that calls ASSERT_*
macros; the runner reports a pass/fail count and exits non-zero
on any failure.
Source
The repository lives at github.com/404Piyush/pipe-shell. The public header, the parser + executor, and the test suite are the three files worth reading. The header is the contract. The parser + executor is about 250 lines. The test suite is about 300 lines.
To build and test locally:
git clone https://github.com/404Piyush/pipe-shell
cd pipe-shell
make test
./pipe-shell
One of the gpu-engineering curriculum, a three-year systems and hardware roadmap. More projects incoming.