The task of synthesizing optimal discrete stochastic systems to control devices and developing controller tuning parameters that have optimal rates of convergence is examined in the presence of unmodelled dynamics. Several standard kinds of optimality are discussed in the context of the adjustable models and the adaptation algorithms.