Power systems' reliable and cost-effective operation depends on the performance of short-term load forecasting models. Probabilistic Load Forecasting (PLF) quantifies the uncertainty of future load, facilitating the derivation of scenario trajectories. This capability aids operators' risk management and decision-making concerning energy supply, load balancing, reserve and ramping scheduling, and other grid management activities. However, traditional PLF models often face challenges stemming from asymmetry, multi-modality, heavy tails, and non-negativity in load series. To address this issue, we propose an innovative approach by incorporating PLF-tailored kernel functions into Mixture Density Networks (MDNs) through two distinct methods: (i) the introduction of the versatile four-parameter Sinh-Arcsinh distribution to enhance MDN flexibility and (ii) the strategic left-truncation of kernel functions at zero, effectively integrating prior knowledge about load series characteristics. Expanding the utility of MDNs, the study also delves into multi-step load scenario generation, pushing the boundaries of their application. Rigorous assessment against both traditional and state-of-the-art benchmarks highlights the superior performance of the proposed methods across diverse load classes. Notably, the synergy between using Sinh-Arcsinh and truncated distributions in an MDN emerges as the standout performer for both PLF and scenario generation.