During an epidemic, metrics such as R0, doubling time, and case fatality rates are important in understanding and predicting the course of an epidemic. However, if collected over country or regional scales, these metrics hide important smaller-scale, local dynamics. We examine how commonly used epidemiological metrics differ for each individual state within the United States during the initial COVID-19 outbreak. We found that the detected case number and trajectory of early detected cases differ considerably between states. We then test for correlations with testing protocols, interventions and population characteristics. We find that epidemic dynamics were most strongly associated with non-pharmaceutical government actions during the early phase of the epidemic. In particular, early social distancing restrictions, particularly on restaurant operations, was correlated with increased doubling times. Interestingly, we also found that states with little tolerance for deviance from enforced rules saw faster early epidemic growth. Together with other correlates such as population density, our results highlight the different factors involved in the heterogeneity in the early spread of COVID-19 throughout the United States. Although individual states are clearly not independent, they can serve as small, natural experiments in how different demographic patterns and government responses can impact the course of an epidemic.