Unix Shell Patterns

An early version of this page was submitted to PLoP96, see http://www.cs.wustl.edu/~schmidt/PLoP-96/shell.ps.gz for the paper.

Introduction

Shell has emerged as a family of programming languages for the UNIX Operating System in the same sense that JobControlLanguage emerged as a programming language for mainframe job control in the 1960s. It is a family of languages because we find variant syntaxes and capabilities in the Bourne shell, the Korn shell, the C shell, and others. Most of the patterns described here apply equally well to all these variants.

These patterns do not form a full pattern language nor are they a comprehensive collection of patterns for the "architecture" of a Shell program. They cover frequent Shell programming problems that often are resolved only with expert insight.

Glossary

SetUID: A process mode that allows a process, started by one user, to run under the auspices and permissions of another user. SetUID permissions appear in the program file's mode.
Subshell: A Shell running in a process that is a child process of some parent process, also running a shell, often arranged so the two Shells can communicate through a pipe.
ShellVariables: Identifiers known to Shell that can be bound to values such as strings and integers. Their value can be referenced with a leading dollar sign; e.g. $DATE.
StandardOutput, StandardError?, StandardInput?, stdout, stderr, stdin: StandardOutput is an abstract resource used by programs to print results where they will most likely be read; for an interactive user, this is the user terminal; for a batch job, it would correspond to a line printer or file. It is also called stdout. Other resources include StandardError? (stderr), the standard resource for error output, usually tied to StandardError?, and StandardInput? (stdin), the standard resource for user input (usually the keyboard, but it can be a file).
Daemon: A program that is usually not interactive and that usually runs for a long time.

For example, gtty daemons are long-lived processes that wait for users to log in on a line.

The Pattern Intents

TestingAgainstTheNullString: How to test for the null string while avoiding aliasing problems caused by Shell command interpolation rules.
CallByReferenceSubshellVariables: How to effectively export values from a subshell to its parent shell.
UsingXargsForRepetition: How to efficiently unfold execution loops for many forms of Shell commands like grep, ls, etc.
UseIfsInsteadOfCut: An expressive and efficient way to extract delimited data fields from input text.
UsePasteInsteadOfSed: Avoiding paste limitations in sed that cause truncated lines when generating long lines.
UseExecToAvoidSubshell: The Shell equivalent of tail recursion of FORTRAN chaining.
SaveFileDescriptorsToAvoidSubshell: Using Shell input/output capabilities that reuse file descriptors and their bindings to avoid costly Shell spawns.
IsolateDaemonsFromTerminals: Avoiding naive problems with hidden input/output assignments in programs designed to run as detached background processes.
UniqueDelimitersShellPattern: Make sure that the contents of an archive file don't alias the EOI delimiter used in scripts.
CleanupTempFilesShellPattern: Making sure temp files do not get left behind when your shell program exits.
GrepForDevNull: Grep outputs differently when multiple files are given. This pattern is used when you only have one file to grep, but want the output to act the plural-file way.
CommentsAsDataShellPattern: To put "usage" info in a prominent place.
MoveToDirectory: Making sure the destination is unambiguously a directory reference
ShellScriptVsScriptingLanguageScript: When to use a shell script and when to use a scripting language.

Contributors: JimCoplien, AlanRobertson, GreggWonderly, MichaelLindner, PaulChisholm, StephenHill, RussellCorfman, BillTrost, other(s)

CategoryUnixShellPattern