Anderson's chemical pseudopotential scheme is supposed to define a set of highly localized basis functions in terms of which the eigenstates of a solid can be expanded exactly. This sounds good in principle but little is known about the method in practice and it has not even been established that the basis functions always exist. This paper discusses some of the general properties of localized basis sets spanning a band of Bloch eigenstates and then looks at the Kronig-Penney model as an example. It is shown that the chemical pseudopotential equation has no solutions when the strength P of the attractive delta function potentials is less than about 1.4018, and that the basis functions generated are never unique when they do exist. Fortunately, it turns out that there is a generalized version of the chemical pseudopotential equation that can be solved no matter how weak the potentials, although the non-uniqueness of the basis functions remains. The solutions resemble atomic orbitals in the extreme tight-binding limit (although even then there are alternatives) but this is not true in general and so chemical pseudopotential theory should not be taken as a justification for using atomic orbitals as basis functions.