The coupled thermosphere‐ionosphere magnetosphere (CMIT) model and the Thermosphere Ionosphere Nested Grid (TING) model have been run to simulate the 15 May 1997 interplanetary coronal mass ejection's (ICME) effects on the Earth's ionosphere and thermosphere. Comparisons were made between these model runs, the IRI‐2007 model, and geomagnetic middle‐latitude ionosonde data (NmF2) from the World Data Center to determine how well the models simulated the event and to understand the causes of model‐data disagreement. The following conclusions were drawn from this study: (1) skill scores were more often negative than positive on average; (2) the best and the worst skill scores occurred on the recovery day; (3) the line plots comparing models to data look better than the skill scores might suggest; (4) skill scores are significantly affected by timing issues and large, short‐duration variability; (5) skill scores give an indication of the relative ability of one model relative to another, rather than an absolute statement of model accuracy; (6) the models capture negative storm effects better than they capture positive storm effects; (7) the TING model captured many short duration features seen in the data at high middle latitude stations that result from changes in the size of the auroral oval; (8) CMIT overestimates the energy driving changes in NmF2, whereas TING provides approximately the correct energy input as a result of the saturation effects on potential that are included in TING; and (9) both TING and CMIT electron densities decreased too rapidly after sunset.
