A systematic analysis of the data on swift heavy ion irradiation in semiconductors shows that the effects produced are determined not only by the electronic energy deposition but also by a number of other material characteristics. The study is carried out to highlight the role of film microstructure in determining the energy relaxation processes of heavy ions in semiconductors. To verify the idea, two sets of CdTe polycrystalline thin film samples, prepared using thermal evaporation and spray pyrolysis, which differ in their microstructure are irradiated with 100 MeV Ag⁷⁺ ions using a 15 UD Pelletron accelerator. Differences are noticed in the response of the two sets of films to the beam and are correlated with the differences in their microstructure. For the evaporated films defect annealing dominates at lower fluences, but at higher fluences the effects due to defect generation are dominant. An increase in the lattice constant with fluence due to an increase in the tensile strain is also observed in this case. For the spray deposited samples, however, defect generation always dominates and no change in the lattice parameter is observed. The optical band gap undergoes a red shift for both sets of films due to the introduction of band-tail states.