Forest disturbances—driven by pests, pathogens, and discrete events—have led to billions of dollars in lost ecosystem services and management costs. To understand the patterns and severity of these stressors across complex landscapes, there must be an increase in reliable data at scales compatible with management actions. Unmanned aerial systems (UAS or UAV) offer a capable platform for collecting local scale (e.g., individual tree) forestry data. In this study, we evaluate the capability of UAS multispectral imagery and freely available National Agricultural Imagery Program (NAIP) imagery for differentiating coniferous healthy, coniferous stressed, deciduous healthy, deciduous stressed, and degraded individual trees throughout a complex, mixed-species forests. These methods are first compared to assessments of crown vigor in the field, to evaluate the potential in supplementing this resource intensive practice. This investigation uses the random forest and support vector machine (SVM) machine learning algorithms to classify the imagery into the five forest health classes. Using the random forest classifier, the UAS imagery correctly classified five forest Health classes with an overall accuracy of 65.43%. Using similar methods, the high-resolution airborne NAIP imagery achieved an overall accuracy of 50.50% for the five health classes, a reduction of 14.93%. When these classes were generalized to healthy, stressed, and degraded trees, the accuracy improved to 71.19%, using UAS imagery, and 70.62%, using airborne imagery. Further analysis into the precise calibration of UAS multispectral imagery, a refinement of image segmentation methods, and the fusion of these data with more widely distributed remotely sensed imagery would further enhance the potential of these methods to more effectively and efficiently collect forest health information from the UAS instead of using field methods.