I see this a bit different they have access to written accounts of what happens when a desolation is near, not by necessity experience of past Radiants and almost certainly not personal experience.
Its the number of surviving heralds that makes me think the Radiants were only around for the desolation in the prelude, if the radiants were around for more than one desolation I can't help but think that the last one was an elaborate trap to get the heralds to abandon the oathpact which would most likely imply a traitor amongst them, if nothing was different to previous desolations why is Taln the only herald death Kalak considered it a real possibility that he and Jezrien were the only survivors of what he describes as a furious battle one of the worst when he saw Jezrien alone at the meeting place.