It has been shown, for fractionally charged anyonic quasi-particles in two-dimensional multiply connected manifolds, that there is a topological Z_n symmetry due to gauge invariance (Wu Y. S., Hatsugai Y. and Kohmoto M., Phys. Rev. Lett., 66 (1991) 659). We show that relativistic corrections may give rise to an interaction between anyonic excitations and the spin of electron, which may explicitly break the topological symmetry. Possible applications of the broken system are also discussed.