We derive effective actions for SU(2) Polyakov loops using inverse Monte Carlo techniques. In a first approach, we determine the effective couplings by requiring that the effective ensemble reproduces the single-site distribution of the Polyakov loops. The latter is flat below the critical temperature implying that the (untraced) Polyakov loop is distributed uniformly over its target space, the SU(2) group manifold. This allows for an analytic determination of the Binder cumulant and the distribution of the mean-field, which turns out to be approximately gaussian. In a second approach, we employ novel lattice Schwinger-Dyson equations which reflect the SU(2) × SU(2) invariance of the functional Haar measure. Expanding the effective action in terms of SU(2) group characters makes the numerics sufficiently stable so that we are able to extract a total number of 14 couplings. The resulting action is short-ranged and reproduces the Yang-Mills correlators very well.