An "error provider" is simply a DotNet object that can be associated with an individual control. When you set its error message property, it suddenly become visible as a little red circle next to the control (you can set the icon to be whatever you choose), and it has a tool tip that displays the error message.
--MikeStallings? (on behalf of FormValidation)
BTW, these are typically called 'Server Validators' (or just 'Validators'), at least as of .NET 1.0 RTM. (I didn't participate in the original beta - maybe they were "error providers" back then?) At least, they're called so in the context of ASP.NET - I think validation exists in Windows Forms, but I haven't worked with it.
Server Validators validate the content of that object (the validator is itself a .NET server control with a ControlToValidate property, which you set to the control-you-wish-to-validate's ID property). Examples include a RequiredFieldValidator (value of control can't be empty), CompareValidator (compares the value to another control, e.g. comparing the 'password' and 'repeat password' fields), and RegularExpressionValidator (naturally, validates the content of a control via regular expression). There are several more, and you can inherit from BaseValidator to code your own. Typically these are used to validate TextBox controls, but any control marked with the ValidationPropertyAttribute (which specifies the public property of that control that should be validated) can be used in the ControlToValidate property of a validator.