We define an asymptotic symmetry algebra for three-dimensional Gödel spacetimes supported by a gauge field which turns out to be the semi-direct sum of the diffeomorphisms on the circle with two loop algebras. A class of fields admitting this asymptotic symmetry algebra and leading to well-defined conserved charges is found. The covariant Poisson bracket of the conserved charges is then shown to be centrally extended to the semi-direct sum of a Virasoro algebra and two affine algebras. The subsequent analysis of three-dimensional Gödel black holes indicates that the Virasoro central charge is negative.