We study the classical dynamics of an atom bouncing on a concave reflector, which is sometimes called a gravitational cavity. Experiments and simulations have previously shown that atoms released from particular heights exhibit anomalously long storage times but no explanations for this have been offered hitherto. Here we show that the stability is associated with orbits where the atom moves on a cyclic trajectory. These have a lower loss rate from the cavity because they explore only a small part of the mirror surface compared with generic trajectories in neighbouring parts of phase space. We identify and classify them.