Activities that maintain cardio fitness - such as running, swimming and cycling - led to better thinking skills and memory 20 years on.
Scientists say the research, reported in Neurology, adds to evidence the brain benefits from good heart health.
Cardio fitness is a measure of how well the body absorbs oxygen during exercise and transports it to the muscles.
Researchers at the University of Minnesota, Minneapolis, tested almost 3,000 healthy people with an average age of 25.
They underwent treadmill tests of cardiovascular fitness during the first year of the study and again 20 years later.
They were asked to run for as long as possible before they became exhausted or short of breath.
People who ran for longer on the treadmill performed better at tests of memory and thinking skills 25 years on, even after adjusting for factors such as smoking, diabetes and high cholesterol.
People who had smaller time differences in their treadmill test 20 years later were more likely to perform better on the executive function test than those who had bigger differences.
"Many studies show the benefits to the brain of good heart health," said study author Dr David Jacobs.
"This is one more important study that should remind young adults of the brain health benefits of cardio fitness activities such as running, swimming, biking or cardio fitness classes."
Dr Jacobs said a concept was emerging of total fitness, incorporating social, physical and mental aspects of health.
"It's really a total package of how your body is and the linkage of that entire package of performance - that's related to cognitive function many years later and in mid-life," he told BBC News.
A spokeswoman for the Alzheimer's Society said: "A growing body of evidence suggests exercise may reduce the risk of cognitive decline and dementia, and much research has shown a link between healthy habits in mid-life and better health in old age.
"Investment in research is vital to better understand how we can protect our brains as we age."