In the measurement of the levitation force between a vertically magnetized permanent magnet (PM) and a bulk high-temperature superconductor (HTS), PM domains with horizontal components of magnetization are shown to produce a non-negligible contribution to the levitation force in most systems. Such domains are typically found in all PMs, even in those that exhibit zero net horizontal magnetic moment. Extension of this analysis leads to an HTS analogue of Earnshaw's theorem, in which the vertical stiffness is equal to the sum of the horizontal stiffness at the field-cooling position, independent of the angular distribution of magnetic moments within the PM.