The reduced state of a small system strongly coupled to a thermal bath may be athermal and used as a small battery once disconnected. The unitarily extractable energy (a.k.a. ergotropy) will be negligible if the disconnecting process is too slow. To study the efficiency of this battery, we consider the cycle of disconnecting, extracting, and connecting the battery back to the bath. Efficiency, i.e. the ratio between ergotropy and connecting plus disconnecting work, is a function of disconnecting time. We consider the Caldeira-Leggett model of a quantum battery in two scenarios. In the first scenario, we assume that the discharged battery is uncorrelated to the bath when connecting back and find that the efficiency peaks at an optimal disconnecting time. In the second scenario, the discharged battery is correlated to the bath, and see that the optimal efficiency corresponds to an instantaneous disconnection. On top of these results, we analyze various thermodynamic quantities for these Caldeira-Leggett quantum batteries and express the first and second laws of thermodynamics for the cycles in simple form despite the system-bath initial correlations and strong coupling regime of the working device.