We present a study of the time evolution of TGFs observed by both RHESSI and BATSE and compare the data with Monte Carlo simulations of relativistic runaway electron avalanches that record the time of flight for individual photons. Our analysis of the RHESSI data shows that the average delay between the main, hard peak and the secondary, soft tail in the RHESSI data is 28 ± 3 μs. We show that this is consistent with Compton downscattering of hard photons in the atmosphere in our simulations when the source region is located at or below 21 km, which is consistent with previous spectroscopic analysis of the RHESSI data. We also show that the much longer delays (nominally 100 μs) reported for the BATSE TGFs can be understood by the same simulations that are consistent with the RHESSI data once the effects of instrumental deadtime are considered