We report the discovery of a new ultra-faint stellar system found near the Magellanic Clouds in the DECam Local Volume Exploration Survey. This new system, DELVE J0155-6815 (DELVE 2), is located at a heliocentric distance of D-circle dot = 71 +/- 4 kpc, which places it at a 3D physical separation of 12 +/- 3 kpc from the center of the Small Magellanic Cloud and 28(-3)(+4) kpc from the center of the Large Magellanic Cloud ( LMC). DELVE 2 is identified as a resolved overdensity of old (tau > 13.3 Gyr) and metal-poor ([Fe/H] = -2.0(-0.5)(+0.2) dex) stars with a projected half-light radius of r(1/2) = 21(-3)(+4) pc and an absolute magnitude of M-V = -2.1(-0.5)(+0.4) mag. The size and luminosity of DELVE 2 are consistent with both the population of recently discovered ultra-faint globular clusters and the smallest ultra-faint dwarf galaxies. However, its photometrically derived age and metallicity would place it among the oldest and most metal-poor globular clusters in the Magellanic system. In the absence of spectroscopic measurements of the system's metallicity dispersion and internal kinematics, we are unable to conclusively classify this system at this time. DELVE 2 is detected in Gaia DR2 with a clear proper-motion signal, with multiple blue horizontal-branch stars near the centroid of the system with proper motions consistent with the systemic mean. We measure the system proper motion to be (mu(alpha) cos delta, mu(delta)) = (1.02(-0.25)(+0.24), -0.85(-0.19)(+0.18) mas yr(-1). We compare the spatial position and proper motion of DELVE 2 with simulations of the accreted satellite population of the LMC and find that it is very likely to be associated with the LMC.