On four occasions, twice in 1991 (near solar maximum) and twice in 1994 (near solar minimum), one COMPTEL D1 detector module was used as an omnidirectional detector to measure the high-energy (> 12.8 MeV) neutron flux near an altitude of 450 km. The D1 modules are cylindrical, with radius 13.8 cm and depth 8 cm, and are filled with liquid scintillator (NE213A). The combined flux measurements can be fit reasonably well by a product of the Mt. Washington neutron monitor rate, a linear function in the spacecraft geocenter zenith angle, and an exponential function of the vertical geomagnetic cutoff rigidity in which the coefficient of the rigidity is a linear function of the neutron monitor rate. When pointed at the nadir, the flux is consistent with that expected from the atmospheric neutron albedo alone. When pointed at the zenith the flux is reduced by a factor of about 0.54. Thus the production of secondary neutrons in the massive (16000 kg) Compton Gamma-Ray Observatory spacecraft is negligible. Rather, the mass of the spacecraft provides shielding from the earth albedo.