For pt.II see ibid., vol.17, no.7, p.1201 (1984). The solution of the Dirac equation for hydrogen-like atoms within the algebraic approximation, that is, by using finite basis sets, is considered. The model problem of a hydrogenic atom with nuclear charge Z perturbed by a potential-Z'/r is used to demonstrate the feasibility of including relativistic effects in diagrammatic perturbation theory calculations performed within the algebraic approximation. Both ground- and excited state calculations are reported. Particular attention is paid to the choice of basis sets and the handling of the negative-energy states. It is shown that the 'finite basis set disease' can be avoided in perturbation theory calculations by making an appropriate choice of basis sets. Negative-energy states must be included in the perturbation theory summations. The importance of this work to the development of a relativistic many-body perturbation theory of molecular electronic structure is discussed.