An open set in a topology is a set belonging to that topology. An open set in any metric space (for instance the real numbers) is one such that every point in the set has a neighborhood contained in that set. (For all x in S, there exists epsilon > 0 s.t. |x - x_0| < epsilon => x_0 in S.}
In the plane you can get a pretty good visualization of what an open set is on the plane by the observation that open sets are regions of the plane that you would draw with dotted lines. That is, they do not include points on the boundary of the region. The same is true in any number of dimensions, but visualization is usually easiest in 2 dimensions.