The Background Imaging of Cosmic Extragalactic Polarization (Bicep) experiment was designed specifically to search for the signature of inflationary gravitational waves in the polarization of the cosmic microwave background (CMB). Using a novel small-aperture refractor and 49 pairs of polarization-sensitive bolometers, Bicep has completed three years of successful observations at the South Pole beginning in 2006 February. To constrain the amplitude of the inflationary B-mode polarization, which is expected to be at least 7 orders of magnitude fainter than the 3 K CMB intensity, precise control of systematic effects is essential. This paper describes the characterization of potential systematic errors for the Bicep experiment, supplementing a companion paper on the initial cosmological results. Using the analysis pipelines for the experiment, we have simulated the impact of systematic errors on the B-mode polarization measurement. Guided by these simulations, we have established benchmarks for the characterization of critical instrumental properties including bolometer relative gains, beam mismatch, polarization orientation, telescope pointing, sidelobes, thermal stability, and timestream noise model. A comparison of the benchmarks with the measured values shows that we have characterized the instrument adequately to ensure that systematic errors do not limit Bicep's two-year results, and identifies which future refinements are likely necessary to probe inflationary B-mode polarization down to levels below a tensor-to-scalar ratio r = 0.1.