Non-perturbative features of the derivative expansion of the effective action of a single D3-brane are obtained by considering scattering amplitudes of open and closed strings. This motivates expressions for the coupling constant dependence of world-volume interactions of the form (F)⁴ (where F is the Born-Infeld field strength), (²φ)⁴ (where φ are the normal coordinates of the D3-brane) and other interactions related by N = 4 supersymmetry. These include terms that transform with non-trivial modular weight under Montonen-Olive duality. The leading D-instanton contributions that enter into these effective interactions are also shown to follow from an explicit stringy construction of the moduli space action for the D-instanton/D3-brane system in the presence of D3-brane open-string sources (but in the absence of a background antisymmetric tensor potential). Extending this action to include closed-string sources leads to a unified description of non-perturbative terms in the effective action of the form (embedding curvature)² together with open-string interactions that describe contributions of the second fundamental form.