NIH Public Access: Author Manuscript
NIH Public Access: Author Manuscript
NIH Public Access: Author Manuscript
Author Manuscript
Arch Neurol. Author manuscript; available in PMC 2009 July 1.
Published in final edited form as:
NIH-PA Author Manuscript
Sid E. O'Bryant, Ph.D.1, Joy D. Humphreys, M.A.2, Glenn E. Smith, Ph.D.3, Robert J. Ivnik,
Ph.D.3, Neill R. Graff-Radford, M.D.4, Ronald C. Petersen, M.D., Ph.D.5, and John A. Lucas,
Ph.D.6
1Department of Neuropsychiatry and Behavioral Science, Texas Tech University Health Science Center,
Lubbock, TX
2Department of Psychology, Texas Tech University, Lubbock, TX
3Department of Psychiatry & Psychology, Mayo Clinic College of Medicine, Rochester, MN
4Department of Neurology, Mayo Clinic College of Medicine, Jacksonville, FL
NIH-PA Author Manuscript
Abstract
Objectives—To evaluate the utility of Mini-Mental State Examination (MMSE) scores in detecting
cognitive dysfunction in a sample of highly educated individuals.
Design—Archival data were reviewed on 4248 participants enrolled in the Mayo Clinic Alzheimer's
Disease Research Center (ADRC) and Alzheimer's Disease Patient Registry (ADPR).
Patients—1141 primarily Caucasian (93%) individuals with 16 or more years of self-reported
education were identified. These included 307 (164 males and 143 females) dementia cases (any
type), 176 patients with Mild Cognitive Impairment (106 males and 70 females), and 658
nondemented controls (242 males and 416 females).
Setting—Mayo Clinic ADRC and ADPR cohort.
Main Outcome Measures—Diagnostic accuracy estimates (sensitivity, specificity, positive and
NIH-PA Author Manuscript
Keywords
Alzheimer's disease; dementia; Mini-Mental State Examination; diagnosis
NIH-PA Author Manuscript
Method
Archival data were reviewed from 4248 consecutive participants recruited into the Mayo Clinic
Alzheimer's Disease Research Center (ADRC) and Alzheimer's Disease Patient Registry
(ADPR) database. The Rochester Mayo ADPR is responsible for recruiting dementia patients
and non-demented control subjects for studies on the progression of Alzheimer's disease
through the Department of Community and Internal Medicine and does not operate in
Jacksonville. The Rochester and Jacksonville ADRC sites acquire dementia patients from
NIH-PA Author Manuscript
Behavioral Neurology. The Jacksonville ADRC site also recruits community controls via
churches and community agencies. The same inclusion/exclusion criteria are applied for
normal controls across both recruitment sites and has been published extensively through
analyses of the MOANS16-19 and MOAANS20-22 data. Patients with memory concerns
raised by either the patient themselves, a family member, or a physician undergo a
comprehensive neurological evaluation and neuropsychological testing to confirm or rule out
dementia and Alzheimer disease.
A total of 1141 individuals with 16 or more self-reported years of education were identified.
The sample included 1064 (93%) individuals who self-identified as Caucasian and 77 (7%)
who self-identified as African-American. Of the 1141 participants, 658 individuals (242 males
and 416 females) had no dementia and were considered cognitively normal (see Ivnik et al.
19 for full criteria used to define normal cognition). The remaining 307 (164 males and 143
females) carried diagnoses of dementia established via consensus among ADRC investigators
and based on published diagnostic criteria. Diagnoses included 202 (66%) patients with
probable Alzheimer's disease, 48 (16%) with dementia with Lewy bodies, 18 (6%) with
frontotemporal dementia, 13 (4%) with vascular dementia, and 25 (8%) with other dementia
NIH-PA Author Manuscript
etiologies. A sample of 176 patients (106 males and 70 females) diagnosed with Mild Cognitive
Impairment (MCI) was also included for comparison purposes.
The total sample included 512 (45%) males and 629 (55%) females, with a mean age of 75.9
(SD=7.2) years and a mean self-reported education of 17.1 (SD=1.5) years. There were no
significant between-group differences (dementia vs. no dementia) in terms of age, gender, or
education.
While the MMSE was available in diagnostic meetings, the diagnosis of dementia (and
particular subtype) was arrived at via consensus-based judgment taking into account
information from the neurological examination, clinical interview, lab results, imaging,
informant ratings of activities of daily living (ADLs), as well as neuropsychological test data.
Therefore, the MMSE had minimal impact on diagnostic decisions in the dementia cohort and
was not considered at all as part of the determination of control status.
Results
Estimates of sensitivity and specificity were calculated for MMSE cut-scores from 16 (i.e., 15
NIH-PA Author Manuscript
and below) to 30 (i.e., 29 and below). Results comparing non-demented controls to those
diagnosed with some form of dementia are presented in Table 1 and illustrated via receiver
operating characteristic (ROC) plot in Figure 1. The traditional cut-score of 24 (23 or below)
yielded a moderate estimate of sensitivity (.66) with very high specificity (.99) and an overall
correct classification rate of 88.9%. The modest test sensitivity reflects the failure of the
traditional cut score to identify a sizeable number of dementia patients in this highly educated
sample. Specifically, 104 (34%) dementia cases in this sample were misclassified as normal.
An optimal balance between sensitivity (.89) and specificity (.91) was obtained with a cut-
score of 27 (26 or below). This yielded only slight improvement in the overall correct
classification rate (90.1%) but identified 70 of the 104 dementia patients who were missed
using the traditional cutoff. The cut score of 27 yields a likelihood ratio of 9.6, indicating that
college graduates with an MMSE score of 26 and with complaints of cognitive decline (self-
or other-report) are nearly 10 times more likely to have dementia that those who obtain a score
of 27 or better.
As expected, the improved sensitivity obtained when the cut score is raised to 27 is achieved
at the sacrifice of specificity. As a result 61 (9%) non-demented patients fall below the higher
cutoff, as compared to only 3 (<1%) false positive identifications with the traditional cut-score
NIH-PA Author Manuscript
of 24.
Next, because clinicians regularly evaluate patients with cognitive dysfunction with and
without dementia, the above-mentioned analyses were calculated on a cognitively impaired
group (MCI and dementia) versus non-demented controls to determine if an appropriate cut-
score could be obtained. Estimates of sensitivity and specificity are presented in Table 2. As
can be seen from Table 2, the traditional cut-score of 24 yields a very poor sensitivity (.45) but
perfect specificity (1.0). Raising the cut-score to 27 yields an increased sensitivity (.69) with
a concomitant decline, though still impressive, specificity (.91).
Although sensitivity and specificity measures are important to establish the diagnostic validity
of test measures such as the MMSE, the diagnostic utility of a particular score earned by a
particular patient is represented by the test's predictive values. Positive predictive values (PPV)
represent the probability that a patient with a score below cutoff actually has the condition of
interest. Conversely, negative predictive values (NPV) represent the probability that a patient
with a score above cutoff does not have the condition of interest. Unlike sensitivity and
specificity, PPV and NPV are influenced by the base rate of the condition of interest in the
NIH-PA Author Manuscript
target population. In the current study, where the base rate of dementia (dementia only group)
was 32%, the PPV and NPV for the traditional cutoff of 24 were 96.9% and 86.2%, respectively.
Using a cutoff of 27 yielded a lower PPV (81.7%) but a higher NPV (94.5%). When looking
at the cognitively impaired group (MCI + dementia), the standard cut-score of 24 yields a very
low SN (.45), but perfect SP (1.0). The optimal balances SN and SP were found at cut-scores
of 27 (PPV=.78, NPV = .86) or 28 (PPV=.63, NPV=.88). Table 3 presents predictive value
calculations from the both groups for clinicians who wish to apply these data in settings where
base rates of cognitive impairment and/or dementia differ from that of the current study.
Discussion
The current findings suggest that the traditional MMSE cut-score of 24 does not yield optimal
classification accuracy in highly educated Caucasian dementia patients. Instead, a more
stringent cut-score of 27 yields greater clinical utility with regard to identifying dementia in
well-educated individuals. Although there is an expected concomitant increase in false-positive
identifications using the higher cut score, a sacrifice in specificity in exchange for a significant
gain in sensitivity is preferred when the goal of the mental status screen is early detection of
possible dementia.
NIH-PA Author Manuscript
The current analyses also demonstrate that, when MCI is entered into the equation, obtaining
an optimal balance between SN and SP is very difficult indeed. Table 2 demonstrates that
optimal balances between SN and SP are found at cut-scores of either 27 (SN=.69, SP=.91) or
28 (SN and SP = .78). One might note that the NPV and PPV for the cognitively impaired
group using the traditional cut-score of 24 is quite impressive even at low base rates; however,
this is a function of a perfect SP and the low base rates. What this translates to for practicing
clinicians is a very high false negative rate (often 50% or more) meaning that, because of the
small number of true cases in low base rate settings, a large portion of those individuals actually
suffering from cognitive dysfunction will not be detected and referred on for a comprehensive
evaluation and/or treatment. Table 3 allows the individual clinician to make the determination
as to what cut-score(s) s/he wishes to implement given the nature of the clinic population (e.g.,
demographics, appropriate base rate), additional information obtained in the medical
examination (i.e., screening for cognitive impairment versus dementia if information regarding
functional change is obtained), as well as his/her preferences for potential diagnostic error (i.e.,
false negative and false positive rates).
The vast majority of the published literature examining the relationship between cognitive test
NIH-PA Author Manuscript
There was not enough data in the current sample to test the comparative accuracy of individual
cut-scores among highly educated individuals across ethnic groups. Therefore, the current
findings with Caucasian individuals must be tested within ethnic minority populations before
NIH-PA Author Manuscript
The current findings are not intended to encourage the diagnosis of cognitive impairment or
dementia based on total MMSE scores alone. Instead these results provide practitioners with
revised criteria for appropriate management of highly educated Caucasian elders. Specifically,
older patients who present with memory complaints (self- or other-report) that have attained
a college degree or higher level of education and who score below 27 on the MMSE are at
increased risk of cognitive dysfunction and dementia and should be referred for a
comprehensive evaluation, including formal neuropsychological studies. When early
identification is the primary goal of screening, the cost associated with performing further
evaluation on individuals subsequently found to have no dementia is outweighed by the benefit
of identifying a considerably larger number of individuals who are in the earliest stages of
NIH-PA Author Manuscript
dementia, where early intervention and/or participation in clinical trials may provide maximum
benefit.
Acknowledgment
This study was supported by grants P50 AG16574 and U01 AG06786 from the National Institute on Aging, and by
the Robert and Clarice Smith and Abigail Van Buren Alzheimer's Disease Research Program. Dr. O'Bryant has had
full access to the data analyzed in this study and takes responsibility for the integrity of the data and accuracy of the
analyses.
References
1. Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”. A practical method for grading the
cognitive state of patients for the clinician. Journal of Psychiatric Research 1975;12(3):189–198.
[PubMed: 1202204]
2. Strauss, E.; Sherman, EMS.; Spreen, O. A Compendium of Neuropsychological Tests: Administration,
norms, and commentary. 3rd ed. Oxford University Press; Oxford: 2006.
3. Harvan JR, Cotter V. An evaluation of dementia screening in the primary care setting. Journal of the
American Academy of Nurse Practitioners 2006;18(8):351–360. [PubMed: 16907696]
NIH-PA Author Manuscript
9. Lezak, MD.; Howieson, DB.; Loring, DW. Neuropsychological Assessment. 4th ed. Oxford University
Press; Oxford: 2004.
10. Bravo G, Hebert R. Age- and education-specific reference values for the Mini-Mental and modified
NIH-PA Author Manuscript
Figure 1.
Receiver operating characteristic curve for Mini-Mental State Examination scores (indicated
NIH-PA Author Manuscript
Table 1
Sensitivity and Specificity Estimates for Detecting Dementia using the MMSE
Table 2
Sensitivity and Specificity Estimates for Detecting Cognitive Impairment (MCI + Dementia) using the MMSE
Base Rate
.01 .02 .05 .10 .15 .20 .25 .30 .40 .50
Dementia
Cut Score = 24
PPV .40 .57 .78 .88 .92 .94 .96 .97 .98 .99
O'Bryant et al.
NPV 1.00 .99 .98 .96 .94 .92 .90 .87 .81 .74
Cut Score = 27
PPV .09 .17 .34 .52 .64 .71 .77 .81 .87 .91
NPV 1.00 1.00 .99 .99 .98 .97 .96 .95 .93 .89
Cognitive Impairment*
Cut Score = 24
PPV 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
NPV .99 .99 .97 .94 .90 .87 .83 .79 .71 .62
Cut Score = 29
PPV .03 .07 .16 .28 .38 .47 .54 .60 .70 .78
NPV 1.0 .99 .99 .97 .95 .93 .91 .89 .84 .78
*
Note: Cognitive Impairment group includes patients diagnosed with MCI and dementia