We consider non-holonomic geodesic flows of left-invariant metrics and left-invariant non-integrable distributions on compact connected Lie groups. The equations of geodesic flows are reduced to the Euler-Poincaré-Suslov equations on the corresponding Lie algebras. The Poisson and symplectic structures give rise to various algebraic constructions of the integrable Hamiltonian systems. On the other hand, non-holonomic systems are not Hamiltonian and the integration methods for non-holonomic systems are much less developed. In this paper, using chains of subalgebras, we give constructions that lead to a large set of first integrals and to integrable cases of the Euler-Poincaré-Suslov equations. Furthermore, we give examples of non-holonomic geodesic flows that can be seen as a restriction of integrable sub-Riemannian geodesic flows.