Linux kernel debate: moving beyond fork() + exec() with spawn templates

Since the earliest days of Unix, process creation has rested on two system calls: fork(), which creates a child process as a copy of the parent, and exec(), which runs a new program in its place. Linux implements them as clone() and execve(), but the model — and its costs — have not changed. A recent proposal from Li Chen to add "spawn templates" to the kernel was debated on the kernel mailing list; it will not be accepted in its current form, but it may point toward a new process-creation primitive.

The cost of fork() plus exec()

fork() is relatively expensive, because the kernel must copy the entire process state, memory included, for the child. Decades of optimization have not removed that fundamental cost, and the usual pattern makes it worse: an exec() immediately follows and discards the memory that was just copied. vfork() was an early attempt to optimize that case, but the sequence is still more expensive than it needs to be.

Spawn templates and cached setup

Chen's patch set targets applications that repeatedly launch the same executable — for instance a program that runs Git over and over to inspect a repository. Such an application can create a template with the new spawn_template_create() system call, which returns a file descriptor for the executable, identified either by descriptor (execfd) or by absolute path (filename), but not both. The kernel opens the file and caches information that speeds up later launches.

Each invocation is described by a spawn_template_spawn_args structure: argv points to the argument list, envp to the environment, and actions to an array of spawn_template_action entries covering file descriptors and signal handling. Closing descriptor four in the child, for example, is expressed as an action of type SPAWN_TEMPLATE_ACTION_CLOSE with fd set to four. The process is then started with spawn_template_spawn(), which internally follows close to the normal fork()/exec() path and keeps every check in place; the cached information is what makes it faster. Benchmarks in the cover letter show about a 2% improvement.

Reviewers: the fork() half is the problem

Mateusz Guzik posted the most detailed review, writing that "the entire fork + exec idiom is terrible and needs to be retired". He noted that the patch set leaves the fork() half untouched, even though that is where most of the cost lies, and argued that "creating a pristine process is the way to go". Christian Brauner was favorable — "The idea of having a builder api for exec isn't all that crazy" — but suggested building on the existing pidfd abstraction: an option to pidfd_open() would create an empty process, and a series of calls to a new pidfd_config() would configure its environment and the image to execute, in the manner of fsconfig().

A key objective for Brauner is supporting a user-space implementation of posix_spawn(), which fits the fork()/exec() use case well; developers would likely welcome a native version that does not hide fork() and exec() underneath. Chen agreed that the sketched API seemed better and said future work would go in that direction. Spawn templates will not land in the kernel as proposed — but Linux may finally gain a proper posix_spawn() implementation.