The problem of existence of non-analytic (Griffiths-like) contributions to the free energy of a weakly disordered Ising ferromagnet is studied from the point of view of replica theory. The consideration is undertaken in terms of the usual random temperature Ginzburg-Landau Hamiltonian in space dimensions D < 4 in the zero external magnetic field. It is shown that in the paramagnetic phase, at temperatures not too close to (where the behaviour of the pure system is correctly described by the Gaussian approximation), the free energy of the system has additional non-perturbative contributions of the form (where ), which has an essential singularity in the parameter which describes the strength of the disorder. It is demonstrated that this contribution appears due to nonlinear localized (instanton-like) solutions of the mean-field stationary equations which are characterized by the special type of the replica symmetry breaking. It is argued that physically these replica instantons describe the contribution from rare spatial `ferromagnetic islands' in which the local (random) temperature is below .