The magnetoresistance of mesoscopic GaAs/Al_xGa_1-xAs samples containing differently ordered antidot lattices with the same long-range periodicity of 0.3 mu m has been studied. In a periodic lattice, magnetic-field-induced two-level switching of several impurities has been observed. In contrast, in a disordered antidot lattice single-impurity switching dominates as the overlapping of the depletion regions around the antidots allows for only a few conducting channels through the sample. The impurity dwell time of one such level is seen to undergo large variations when sweeping an external magnetic field. It has been possible to observe the influence of other defects in the sample on the dominating impurity level, since fluctuations of the local electron density affect the Aharanov-Bohm oscillations, and therefore also the impurity double-well potential.