The 3D geometries of high-redshift galaxies remain poorly understood. We build a differentiable Bayesian model and use Hamiltonian Monte Carlo to efficiently and robustly infer the 3D shapes of star-forming galaxies in James Webb Space Telescope Cosmic Evolution Early Release Science observations with log M-*/M-circle dot = 9.0-10.5 at z = 0.5-8.0. We reproduce previous results from the Hubble Space Telescope Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey in a fraction of the computing time and constrain the mean ellipticity, triaxiality, size, and covariances with samples as small as similar to 50 galaxies. We find high 3D ellipticities for all mass-redshift bins, suggesting oblate (disky) or prolate (elongated) geometries. We break that degeneracy by constraining the mean triaxiality to be similar to 1 for log M-*/M-circle dot = 9.0-9.5 dwarfs at z > 1 (favoring the prolate scenario), with significantly lower triaxialities for higher masses and lower redshifts indicating the emergence of disks. The prolate population traces out a "banana" in the projected b/a-loga diagram with an excess of low-b/a, large-log a galaxies. The dwarf prolate fraction rises from similar to 25% at z = 0.5-1.0 to similar to 50%-80% at z = 3-8. Our results imply a second kind of disk settling from oval (triaxial) to more circular (axisymmetric) shapes with time. We simultaneously constrain the 3D size-mass relation and its dependence on 3D geometry. High-probability prolate and oblate candidates show remarkably similar Sersic indices (n similar to 1), nonparametric morphological properties, and specific star formation rates. Both tend to be visually classified as disks or irregular, but edge-on oblate candidates show more dust attenuation. We discuss selection effects, follow-up prospects, and theoretical implications.