The idea being that you give a process only the authority it needs to accomplish its job. Also known as the PrincipleOfLeastAuthority (especially in ObjectCapabilityModel circles).
For example, consider a program needing to write a log to disk. There are several possible privileges that could allow this, including:
Granted, completely secure systems only exist theoretically, but practically speaking, you can get as close as you'd like.
Micromanaging privileges is labor-intensive.
That's why you don't manage privileges, you let the operating system handle capabilities. When you supply a file to a program, you're doing the same micromanagement required by capabilities, except that you give the program the capability to a file, and not the filename + privileges to the file based on the program's identity. It's actually less work than managing a typical permissions system.
But keep in mind that POLA is a principle of security design, not a hard and fast rule that must be adhered to at all times, no matter what the cost. If you don't understand what that means then see ThreeLevelsOfAudience, because POLA is for a level 2 audience.
The antithesis of POLA is the unification of legitimately inseparable privileges. If two privileges can't be legitimately (in a meaningful and just manner) separated so that a person has only one or the other then they should be unified such that it's impossible to separate them.
Another example is the position of an object in the GUI's 3-space and its name. If arbitrary positioning isn't allowed then the objects' name decides its position, modulo a sortBlock of some kind. If arbitrary positioning is allowed then the object's position should be just as accessible as its name (preferably by encoding it in the name). The privileges of hiding an object in the 3-space and hiding it by mangling its name, can't be meaningfully separated.
There are many, many ways of structuring privileges. The forces of unification and minimalism pull in opposite directions. The designer's job is to find the sweet spot in between them. The place where they cancel out.
Contributors: WilliamUnderwood, RichardKulisz