In this paper the stability of inverse problems is discussed. It is taken into account that in inverse problems the structure of the solution space is usually completely different from the structure of the data space so that the definition of stability is not trivial. We solve this problem by assuming that under experimental circumstances both the model and the data can be characterized by a finite number of parameters. In the formal definition that we present, we first compare distances in the data space and distances in the model under variations of these parameters. Second, a normalization is introduced to ensure that quantities in the solution space can be compared directly with quantities in the data space. We note that it is impossible to obtain an objective solution of stability due to the freedom one has in the choice of the norm in the solution space and in the data space. This definition is used to examine the stability of linear inverse problems as well as for the Marchenko equation and inverse problems associated with transfer-matrix methods. For the Marchenko equation it is shown that the instability arises from the nonlinearity of the inverse problem.