Gaussian processes are popular in spatial statistics, data mining and machine learning because of their versatility in quantifying spatial variability and in propagating uncertainty. Although there has been a prolific research activity about Gaussian processes over Euclidean domains, only recently this research has extended to non-Euclidean manifolds. This paper digs into vector-valued Gaussian processes defined over the product of a hypersphere and a Euclidean space of arbitrary dimension, which are of interest in various disciplines of the natural sciences and engineering. Under mild regularity conditions, we establish a surprising one-to-one correspondence between matrix-valued kernels associated with vector Gaussian processes over the product space, and what we term partial ultraspherical and Fourier transforms that are taken over either the sphere or the Euclidean subspace. The properties of our approach are illustrated in terms of new parametric classes of matrix-valued kernels for product spaces of a hypersphere crossed with a Euclidean space. We also provide two algorithms that allow for fast simulation of approximately Gaussian (in the sense of the central limit theorem) processes in such product spaces.