Lorentz invariance is the fundamental symmetry of Einstein's theory of
special relativity, and has been tested to great level of detail. However,
theories of quantum gravity at the Planck scale indicate that Lorentz symmetry
may be broken at that scale motivating further tests. While the Planck energy
is currently unreachable by experiment, tiny residual effects at attainable
energies can become measurable when photons propagate over sufficiently large
distances. The Standard-Model Extension (SME) is an effective field theory
approach to describe low-energy effects of quantum gravity theories. Lorentz
and CPT symmetry violating effects are introduced by adding additional terms to
the Standard Model Lagrangian. These terms can be ordered by the mass dimension
of the corresponding operator, and the leading terms of interest have dimension
d = 5. Effects of these operators are a linear variation of the speed of light
with photon energy, and a rotation of the linear polarization of photons
quadratic in photon energy, as well as anisotropy. We analyzed optical
polarization data from 72 AGN and GRBs and derived the first set of limits on
all 16 coefficients of mass dimension d = 5 of the SME photon sector. Our
constraints imply a lower limit on the energy scale of quantum gravity of
$10^6$ times the Planck energy, severly limiting the phase space for any theory
that predicts a rotation of the photon polarization quadratic in energy.