HotSwap Hardware:
HotSwap Hardware is hardware where one doesn't need to "down" the machine to service components. A pretty typical example of some HotSwap parts is the humble hot swap disk. All one needs to do is yank it when they want to replace it (make sure your system is backed up and that the raid has been re-silvered, please.)
Also, HotSwap PCI cards are starting to become available for Intel as of 1999ish.
When is Intel going to get HotSwap memory and HotSwap CPUs?
Allegedly, IBM had hot-swappable memory and CPUs for their System/370 and subsequent architectures. I remember reading some material on this in the past, but I can no longer find relevant sources. However, a quick Google search for "hot-swappable CPU" yields a plethora of servers, some of them even based on x86 architecture, though mostly mainframes, which exhibit HotSwap processors.
HotSwap Software:
In software, the HotSwap feature enables you to deploy a new or changed version of the software without restarting the process or program.
The Sun JavaVirtualMachine supports limited HotSwaping of method body code in debug mode.
See also: RuntimeUpgradeableCore (hot-swapping the system that provides the hot-swap service?)