The researchers studied 983 adults over 65 living in the Washington Heights area of New York who had four or fewer years of schooling.
Visiting the participants' homes, scientists conducted tests of memory, language, and visual or spatial skills. During these visits, they made diagnoses of dementia based on standard criteria.
The illiterate performed worse in these tests.
In establishing the basic measures, those who never learned to read or write were almost three times more likely to have dementia than those who could read.
And among those who had no dementia at the start of the study, the illiterate section of the cohort was twice as likely to develop it.
The authors write that one reason for brain decline is that those who do not learn to read have "a lower range of cognitive function" than those who are literate.
The findings are part of a long-term aging study.
Jennifer Manly, professor of neuropsychology at Columbia University and lead author of the study, told CNN that scientists have been monitoring a cohort of adults over 65, many from diverse backgrounds in the Washington Heights area, since 1992.
Over the past three decades, they have studied 6,500 New Yorkers as they get older, she said.
While it has long been known that schooling may be linked to better health outcomes, one of the main goals of the study was to determine how literacy may or may not correlate with one's ability to maintain brain health during the golden years.
For example, many of the illiterate adults in Washington Heights came from the Dominican Republic, she said, and may have had to drop out of school to work.
She says more research is needed to corroborate her team's findings, but they can create a public health case for early school leavers and enrolling in adult literacy courses to help maintain dementia protections.
Policymakers should take note, says one scientist
Manly said the study has implications for how nations think about their educational policy.
"The reason they didn't go to school was because of Dominican educational policy," she said.
Manly said that in the US, policymakers should consider the fact that "educational quality shapes the brain health of adulthood."
"Increasing opportunities for children and adults to gain literacy can protect brain health later in life," she said.
Manly compared the positive effects that learning to read can have in mind with the positive effects exercise can have on the body.
"For individuals and families, health behaviors must include education," she said.