Forget the Score: How You Take A Memory Test May Predict Dementia

It’s Not About Getting It Right, It’s About How You Get It Wrong

Most people would not think twice about remembering the gist of a story but fumbling the exact words. A sweeping new study says they should. How precisely someone recalls the specific phrasing of a short story, not just the general idea, may be a potentially powerful early signal that dementia is approaching, detectable years before any formal diagnosis.

Researchers analyzing decades of data from the Framingham Heart Study found that people who scored higher on a measure of word-for-word story recall had about 76% lower odds (a statistical measure, not direct risk reduction) of developing dementia over the following years compared to those who struggled with exact details. A broader measure of overall cognitive ability, captured through the specific types of mistakes people made during a battery of brain tests, was linked to roughly 85% lower odds for each standard unit of improvement.

When researchers fed this error data into machine learning models, those models correctly flagged more than 99% of the 222 people who would eventually receive a dementia diagnosis, within this dataset using cross-validation, though the researchers caution that independent validation would be needed before any clinical application.

Published in the Journal of the International Neuropsychological Society, the findings argue that the standard approach to cognitive testing, boiling performance down to a single score on each task, misses critical clues hiding in plain sight. Patterns of errors, thinking strategies, and even partial successes on memory tasks all carry diagnostic weight that conventional scoring leaves on the table.

How Memory Tests Miss What Matters

Most cognitive tests work like a school exam: a patient completes a task, and a clinician records a summary score. Did the person name 25 out of 30 pictures correctly? Could they repeat seven digits backward? These scores judge whether memory, attention, or language abilities fall within a normal range. But a single number cannot capture how someone arrived at the answer.

Two people might score identically on a story recall test, yet one remembered the exact wording while the other filled in gaps with related but inaccurate details. Under traditional scoring, those differences vanish.

Developed by the late neuropsychologist Edith Kaplan, the Boston Process Approach was designed to fix this problem. Rather than tallying correct answers, clinicians trained in this method document the types of errors a person makes. Did they mix up words from different parts of the test? Did they describe an object they could not name, circling around the right word without landing on it? Each behavior offers a window into which brain systems might be struggling.

Since 2005, the Framingham Heart Study, a landmark community health study that began in 1948, has been collecting this detailed error-and-strategy data using an extensive scoring system and weekly team meetings to maintain consistency. The result is an unusually rich dataset: 169 distinct variables drawn from a battery of well-known brain tests.

What Mistakes on Memory Tests Reveal About Dementia Risk

Researchers analyzed data from 2,363 participants who were free of dementia at the time of their cognitive assessment. These were primarily older adults with an average age of about 71.5 years; 54.2% were female and 96.5% identified as non-Hispanic White.

Using a specialized modeling technique suited to uneven, count-based data, the team uncovered hidden patterns in how errors cluster together. One model met all predetermined quality standards, revealing a general cognitive factor, essentially a person’s overall tendency to produce correct or strategic responses and avoid errors, plus seven specific factors tied to individual tests. Across subgroups, the model performed largely consistently, whether comparing younger versus older participants, men versus women, or those with more versus less education, though not perfectly identical in all respects.

Among the 2,311 participants with at least one follow-up evaluation, 222, about 9.6%, were eventually diagnosed with dementia over a median of 5.2 years. Two factors stood out as protective: higher scores on the general cognitive factor and stronger word-for-word story recall.

One counterintuitive finding deserves attention. “Near miss” errors on story memory, cases where a person recalled something conceptually related to the correct detail but got it slightly wrong, appeared to reflect some preserved memory capacity according to the model. Even getting it almost right was a meaningful signal. Prior research suggests that gist-based memory may function as a backup system when precise recall begins to fail, a system that is itself compromised in Alzheimer’s disease.

Machine Learning Models Caught 99% of Future Dementia Cases Within This Dataset

When the team trained machine learning models to predict dementia, models using only traditional summary scores correctly identified 95.9% of people who eventually developed dementia. Adding detailed error data pushed that figure to 99.4%, within this dataset using cross-validation. The researchers stress that multiple independent samples would need to validate these models before they could be considered reliable for clinical use.

One particularly telling comparison: a clinical diagnosis of mild cognitive impairment at the time of testing was a surprisingly poor standalone marker. Only 46.3% of those participants later developed dementia, and the diagnosis correctly identified just 22.5% of all eventual cases.

Mistakes on cognitive tests are not noise to be discarded. They are patterned data that clinicians have been systematically setting aside for decades. Paying closer attention to how people think, not just how well, may turn out to be one of the most practical tools available in the push for earlier detection.

---

Disclaimer: This article is based on a published peer-reviewed study and is intended for informational purposes only. It should not be taken as medical advice. If you have concerns about memory or cognitive health, consult a qualified healthcare professional.

---