We develop a general instability index theory for an eigenvalue problem of the type (Formula presented.) , for a class of self-adjoint operators (Formula presented.) on the line R1. More precisely, we construct an Evans-like function to show (a real eigenvalue) instability in terms of a Vakhitov–Kolokolov type condition on the wave. If this condition fails, we show by means of Lyapunov–Schmidt reduction arguments and the Kapitula–Kevrekidis–Sandstede index theory that spectral stability holds. Thus, we have a complete spectral picture, under fairly general assumptions on (Formula presented.). We apply the theory to a wide variety of examples. For the generalized Bullough–Dodd–Tzitzeica type models, we give instability results for travelling waves. For the generalized short pulse/Ostrovsky/Vakhnenko model, we construct (almost) explicit peakon solutions, which are found to be unstable, for all values of the parameters.
