Static current-current correlation leads to a zero-frequency divergence (ZFD) in the definition of optical susceptibilities. Previous computations have shown non-equivalent results for two gauges ( p·A and E·r) for exactly the same unperturbed wavefunctions. We reveal that these problems are caused by the incorrect treatment of the time-dependent gauge phase factor in optical response theory. The gauge phase factor, which is conventionally ignored by the theory, is important in resolving the ZFD problem and obtaining equivalent results for these two gauges. The Hamiltonians with these two gauges are not necessarily equivalent unless the gauge phase factor is properly considered in the wavefunctions. Both Su-Shrieffer-Heeger (SSH) and Takayama-Lin-Liu-Maki (TLM) models of trans-polyacetylene serve as illustrative examples in studying the linear susceptibility ⁽¹⁾ through both current-current and dipole-dipole correlations. Previous improper results of ⁽¹⁾-calculations and for distribution functions obtained with both gauges are discussed. The importance of the gauge phase factor in solving the ZFD problem is emphasized on the basis of the SSH and TLM models. As a conclusion, the reason for dipole-dipole correlation being preferable to current-current correlation in practical computations is explained.