The Nyquist stability criterion
The Nyquist stability criterion decides whether a feedback loop is stable by looking at the shape of the open-loop frequency response in the complex plane, rather than computing the closed-loop poles directly.
The −1 point and encirclements
Plot the open-loop transfer function H(jω) as ω runs from −∞ to +∞ — this is the Nyquist plot. The critical point is −1 + 0j. What matters is how many times, and in which direction, the curve encircles that point.
The counting rule Z = N + P
Let P be the number of open-loop poles in the right half-plane, N the number of clockwise encirclements of −1 by the Nyquist curve, and Z the number of closed-loop poles in the right half-plane. Then Z = N + P. The closed loop is stable only when Z = 0, so a stable open-loop system (P = 0) requires no encirclements of −1.
Relation to gain and phase margins
The distance from the curve to the −1 point reflects robustness: where the curve crosses the negative real axis gives the gain margin, and the angle at which it crosses the unit circle gives the phase margin. A curve that passes close to −1 indicates small margins.