We extend our recent study (Robnik et al 1997 J. Phys. A: Math. Gen. 30 L803) of diffusion in strongly chaotic systems ( `the random model') to systems composed of several weakly coupled ergodic components. By this we mean that the system as a whole is ergodic, but the typical time for the transition from one to another component is very long, much longer than the ergodic time inside each individual component. Thus for short times the system behaves like a single component ergodic system and the random model applies (neglecting the coupling to other components). At times much longer than the transition time the system behaves like an ergodic system without internal structure (without decomposition into several components) and the random model applies again (with different parameters). At intermediate times there is the crossover regime which we describe in detail analytically for a two-component system and test it numerically in a double billiard system (butterfly billiard).