We study orientational relaxation in a colloidal suspension of spherical particles whose dynamics is described by a generalized Smoluchowski equation in position and orientation space. We generalize the first cumulant relation for translational diffusion to the case of orientational diffusion. By specializing to a model of dipoles interacting with a Heisenberg pair interaction we obtain the spectrum of the memory function for orientational relaxation in a semi-dilute suspension. Four characteristic times are needed to describe the memory function accurately.