A phenomenological model is proposed to describe the plasticity of soft magnetic elastomer composites under an external field. Along with high-elasticity, internal dry friction is introduced whose origin is attributed to the dipole-dipole interaction of the embedded particles. Numerical estimates of the model parameters are obtained from comparison with the measurements performed on iron carbonyl dispersions in soft silicon-rubber matrices.