From an image we construct an invertible orientation score, which provides an overview of local orientations in an image. This orientation score is a function on the group SE(2) of planar rotations and translations. It allows us to diffuse along multiple local line segments in an image. The transformation from image to orientation score amounts to convolutions with an oriented kernel rotated at multiple angles. Under conditions on the oriented kernel the transform between image and orientation score is a unitary transformation onto a reproducing kernel subspace of L₂(SE(2)) inducing well-posed image reconstruction by the adjoint. This allows us to relate operators on images to operators on orientation scores in a robust way such that we can deal with crossing lines and orientation uncertainty. To obtain reasonable Euclidean invariant image processing the operator on the orientation score must be both left invariant and non-linear. Therefore we consider non-linear operators on orientation scores which amount to direct products of linear left-invariant diffusions on orientation scores. These linear left-invariant convection-diffusions correspond to well-known stochastic processes on SE(2) for line completion and line enhancement and are given by group convolution with the corresponding Green's functions. We provide the exact Green's functions and approximations, which we use together with invertible orientation scores for automatic line enhancement and completion. Furthermore we apply non-linear diffusions on invertible orientation scores, where we can include adaptive curvature. Finally, we consider two other applications of the same theory applied to representations of different groups, such as smoothing of HARDI-images (defined on the group SE(3)) by left-invariant diffusion and signal processing via left-invariant convection (re-assignment), diffusion on Gabor transforms (defined on the group H(3)).