As more trains catch up to air travel, time-wise, one thing that can put them over the top is the time saved avoiding the hassles of getting to the airport, parking, security, waiting, etc.
But what if one of the first mid- or long-range train systems suffers some kind of attack, or even threat of attack? A pipe bomb, a lone gunman, what have you. Immediately would come the metal detectors, shoe examinations, long lines, and the rest of it. Train travel would become frustrating too.
Cheap mass travel — safe enough to satisfy American anxieties — is probably just fated to be annoying.