Comparison of Screening Tools

• WHO and EGIR clinical criteria rely mainly on insulin resistance. IDF criteria rely on abdominal obesity, while NCEP-ATP III gives equal weight to each clinical criterion of the metabolic syndrome.

• The IDF and NCEP-ATP III approaches use the same cutoff values for lipids, fasting glucose, and blood pressure. However, the IDF has proposed ethnic-specific cutoffs for waist circumference. Waist circumference is also a mandatory IDF criterion.

• A few prospective studies have compared metabolic syndrome criteria in assessing CVD risk. Although most criteria have a similar relationship to CVD risk, NCEP-ATP III criteria seem to have the strongest ties to CVD.

• Independent of the clinical criteria studied, the metabolic syndrome better predicts type 2 diabetes risk than CVD risk.

• Further studies are needed to compare various metabolic syndrome clinical criteria using different statistical models and in all populations of the world.

Although the metabolic syndrome clinical criteria proposed by major organizations are similar, some of them put special emphasis on certain variables by using different cutoff values. Most organizations have criteria for obesity (mostly abdominal), insulin resistance, dyslipidemia, and blood pressure. The proposals put forward by the National Cholesterol Education Program (NCEP) expert panel on detection, evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel III or ATP III) and the International Diabetes Federations (IDF) use virtually the same criteria and cutoff values. The exception is waist circumference, for which the IDF cutoff value is lower. The IDF has also proposed different waist circumference cutoffs for various regions of the world in order to address the question of ethnicity. In addition, the IDF makes waist circumference a mandatory criterion. To be diagnosed with the metabolic syndrome, one should have a waist circumference above the proposed IDF waist cutoff plus two other components of the metabolic syndrome. Consequently, in a given population, metabolic syndrome prevalence can be expected to be higher when IDF criteria are applied. Moreover, since the American Association of Clinical Endocrinologists (AACE) did not propose a working definition, its criteria can hardly be tested in population studies unless investigators decide to specify their criteria/cutoffs in advance. Therefore, when comparing the criteria for the metabolic syndrome and incident cardiovascular disease (CVD) or diabetes, investigators can choose from the clinical criteria of NCEP-ATP III, IDF, World Health Organization (WHO), European Group for the Study of Insulin Resistance (EGIR), and hypertriglyceridemic waist.

One of the few prospective population studies to compare NCEP-ATP III, IDF, WHO, and hypertriglyceridemic waist criteria is the Insulin Resistance Atherosclerosis Study (IRAS) [1]. This study followed 822 subjects 40 to 69 years of age without diabetes for 5.2 years. A total of 148 individuals developed type 2 diabetes over the follow-up period. The best predictor of incident diabetes was impaired glucose tolerance. The prevalence of the metabolic syndrome was 27.5%, 34.4%, 39.5%, and 18.4% with NCEP-ATP III, WHO, IDF, and hypertriglyceridemic waist criteria, respectively. This study is valuable in that it used many statistical models to evaluate the association between metabolic syndrome criteria and incidence of diabetes. NCEP-ATP III criteria showed the strongest association with incident diabetes with an odds ratio (OR) of 4.14 (95% CI, 2.79-6.16). The ORs were 3.68 (95% CI, 2.48-5.45), 3.40 (95% CI, 2.28-5.06), and 2.51 (95% CI, 1.68-3.75) for WHO, IDF, and hypertriglyceridemic waist clinical criteria, respectively. The area under the receiver operating characteristic curve (AROC), which plots the true-positive rate vs. the false positive rate graphically, was also higher with NCEP-ATP III criteria (0.69). The AROC was 0.68 for both WHO and IDF criteria and 0.64 for hypertriglyceridemic waist. The population attributable risk (PAR), which is an estimate of the proportion of CVD in a population attributable to the metabolic syndrome, was very similar between NCEP-ATP III, WHO, and IDF (46.3%, 48.0%, and 48.7%, respectively) criteria and lower with hypertriglyceridemic waist criteria (21.7%). The authors of the IRAS study concluded that IDF and NCEP-ATP III criteria predicted diabetes at least as well as WHO criteria, suggesting that measuring insulin sensitivity with clamp techniques and measuring microalbuminuria are not necessary to predict diabetes from a clinical diagnosis of the metabolic syndrome.

Lorenzo et al. [3] used data from the San Antonio Heart Study to compare NCEP-ATP III, IDF, and WHO screening tools in predicting CVD and diabetes incidence. The San Antonio Heart Study recruited 2,559 Mexican American and non-Hispanic white individuals and followed them for an average of 7.4 years. Metabolic syndrome prevalence was higher when IDF criteria were used and lower when WHO criteria were used. During the follow-up, 93 men and 63 women developed CVD events and 195 subjects developed diabetes. NCEP-ATP III, IDF, and WHO clinical criteria yielded an OR for CVD events of 2.00 (95% CI, 1.33-3.01), 1.69 (95% CI, 1.13-2.54), and 1.73 (95% CI, 1.12-2.67), respectively. The various criteria predicted CVD incidence independent of age, sex, ethnicity, type 2 diabetes, non-HDL cholesterol (VLDL + IDL + LDL cholesterol fractions), smoking, and personal and family history of CVD. With respect to diabetes risk, the respective ORs were 6.90 (95% CI, 4.97-9.58), 5.76 (95% CI, 4.11-9.07), and 6.67 (95% CI, 4.75-9.35) for NCEP-ATP III, IDF, and WHO criteria. In subjects free of CVD at baseline, the ORs for CVD and diabetes risk were similar with all three proposed screening tools, but the IDF tools showed greater sensitivity and higher false positive rates. The authors explored whether the presence of the metabolic syndrome enhanced risk prediction in addition to age. Selected age cutoff values were 45 years for men and 55 years for women. The authors found that the metabolic syndrome was better able to predict CVD in men 45 and over and in women 55 and over. They also suggested that, for both men and women, adding diagnosis of the metabolic syndrome to traditional risk factors included in the Framingham risk score could enhance CVD prediction. The combination of traditional CVD risk markers and emerging markers could heighten sensitivity, particularly in older individuals.

The Hoorn Study is a population-based cohort study of diabetes that included 615 asymptomatic men and 749 asymptomatic women 50 to 75 years of age [4]. The study investigated the incidence of fatal and nonfatal CVD associated with metabolic syndrome clinical criteria proposed by NCEP-ATP III, WHO, EGIR, and a modified version of AACE. The results of the Hoorn Study were published at the same time as the proposed IDF criteria, and therefore the authors could not compare IDF criteria to the others. In men, metabolic syndrome prevalence was 19%, 32%, 19%, and 41%, respectively, when NCEP-ATP III, WHO, EGIR, and AACE criteria were used. In women, the respective prevalence of the metabolic syndrome was 26%, 26%, 17%, and 35%. The Hoorn Study was the first study to compare metabolic syndrome prevalence between criteria using Cohen’s agreement test (κ). In men, the best coefficient of concordance was found between WHO and EGIR criteria (both criteria relying mostly on insulin resistance) with κ=0.66 (95% CI, 0.59-0.72). The poorest coefficient of concordance was found between WHO and NCEP-ATP III criteria with κ=0.37 (95% CI, 0.29-0.45). In women, the strongest coefficient of concordance was between NCEP-ATP III and AACE criteria with κ=0.78 (95% CI, 0.73-0.82), while the poorest coefficient of concordance was between EGIR and AACE criteria with κ=0.36 (95% CI, 0.29-0.43).

During the follow-up, 132 men and 95 women developed fatal or nonfatal CVD. In men, the hazard ratios (HR) for incident fatal or nonfatal CVD were 1.91 (95% CI, 1.31-2.79), 1.45 (95% CI, 1.02-2.05), 1.49 (95% CI, 1.01-2.21), and 1.30 (95% CI, 0.92-1.83) for NCEP-ATP III, WHO, EGIR, and AACE clinical criteria, respectively. Only the AACE screening tool yielded a non-significant HR for incident CVD. As for women, the HRs for incident CVD were 1.68 (95% CI, 1.11-2.55), 1.31 (0.85-2.00), 1.34 (95% CI, 0.85-2.14), and 1.84 (95% CI, 1.22-2.78) for NCEP-ATP III, WHO, EGIR, and AACE, respectively. In women, both WHO and EGIR criteria, two tools with insulin resistance as a central component, were not significantly associated with incident CVD. The authors also sought to determine whether the control for traditional risk factors included in the Framingham risk score would affect the HR for metabolic syndrome-related incident CVD. In men, all HRs decreased slightly after this adjustment was made. NCEP-ATP III and WHO criteria remained significant, while EGIR and AACE criteria did not. After adjusting for traditional risk factors, HRs were no longer significant in women, regardless of the screening tools used. Although sensitivity and specificity were not calculated, NCEP-ATP III clinical criteria seemed to yield more significant HRs than other available criteria.

The Diabetes Epidemiology: Collaborative analysis Of Diagnostic criteria in Europe (DECODE) study compared the ability of NCEP-ATP III, WHO, and IDF criteria to predict CVD deaths [5]. The DECODE cohort included pooled data from European epidemiological studies on diabetes and impairment of glucose regulation in which oral glucose tolerance tests were performed. A total of 4,715 men and 5,554 women 30 to 89 years of age were followed for a period ranging from 7 to 16 years. Using WHO criteria, metabolic syndrome prevalence was 27.0% in men and 19.7% in women. Using NCEP-ATP III criteria, metabolic syndrome prevalence was 32.2% in men and 28.5% in women. Using IDF criteria, these figures were 35.9% in men and 34.1% in women. With respect to CVD deaths, the HRs for WHO clinical criteria were 2.09 (95% CI, 1.59-2.76) in men and 1.60 (95% CI, 1.01-2.51) in women. With NCEP-ATP III criteria, the corresponding HRs were 1.72 (95% CI, 1.31-2.26) and 1.09 (95% CI, 0.70-1.69), and the HRs were 1.51 (95% CI, 1.15-1.99) and 1.53 (95% CI, 0.99-2.36) using IDF criteria. Paired homogeneity tests were performed to compare proposed criteria and CVD death risk. In men, there was no difference between NCEP-ATP III and WHO criteria in predicting CVD death, while risk was lower in women with NCEP-ATP III criteria. Compared to IDF criteria, NCEP-ATP III criteria predicted risk similarly in both men and women. With WHO and IDF criteria, risk prediction was similar in women, while in men, risk prediction was stronger with WHO criteria. In the DECODE study, WHO clinical criteria seemed to predict CVD death risk best, and this association was generally stronger in men than in women.

WHO, NCEP-ATP III, and IDF screening criteria were compared in another very large population-based study of 10,592 men 50 to 59 years of age from France and Belfast (Southern and Northern Europe) followed for 5 years [6]. Entitled the prospective epidemiological study of myocardial infarction-étude PRospective de l’Infarctus du MyocardE (PRIME), it involved a nested case-control study cohort that matched 296 cases of coronary heart disease (CHD) to 540 CHD-free controls according to age, recruitment centre, and date. Metabolic syndrome prevalence was 36%, 30%, and 39% in cases and 29%, 23%, and 32% in controls according to WHO, NCEP-ATP III, and IDF clinical criteria, respectively. The corresponding ORs for future CHD were 1.40 (95% CI, 1.01-1.94), 1.46 (95% CI, 1.04-2.04), and 1.41 (95% CI, 1.02-1.95).

In the 10-year longitudinal follow-up of the Strong Heart Study, de Simone et al. [7] compared the ability of WHO, NCEP-ATP III, and IDF criteria to predict CVD events in 2,825 asymptomatic individuals (1,700 individuals had diabetes at the beginning of follow-up). In diabetic and non-diabetic men, metabolic syndrome prevalence was 48%, 44%, and 59% using WHO, NCEP-ATP III, and IDF screening tools. In women, the respective prevalence was 53%, 63%, and 73%. In participants without diabetes, the HRs for incident CVD were 1.54 (95% CI, 1.32-1.80), 1.42 (95% CI, 1.22-1.66), and 1.37 (95% CI, 1.17-1.61) for WHO, NCEP-ATP III, and IDF criteria, respectively. The HRs were definitely higher in participants with diabetes, reaching 2.29 (95% CI, 1.97-2.67), 2.45 (95% CI, 2.12-2.84), and 2.59 (95% CI, 2.25-2.98). Coefficients of concordance varied widely. The strongest coefficient of concordance was between NCEP-ATP III and IDF criteria in women (κ=0.77), while the poorest coefficient of concordance was between IDF and WHO criteria in men (κ=0.49).

Tankó et al. [8] sought to compare NCEP-ATP III criteria and the hypertriglyceridemic waist phenotype in predicting accelerated atherogenesis and related cardiovascular mortality in 557 postmenopausal women followed for 8.5±0.3 years. At baseline, hypertriglyceridemic waist prevalence was 15.8% and metabolic syndrome prevalence was 17.6% with NCEP-ATP III criteria. After adjusting for age, smoking, and LDL cholesterol levels, hypertriglyceridemic waist was associated with an HR of 4.7 (95% CI, 2.2-9.8, p<0.001), while the corresponding HR was 3.2 (95% CI, 1.5-6.5, p<0.001) for NCEP-ATP III. Interestingly, the authors found that women with hypertriglyceridemic waist who did not meet NCEP-ATP III criteria were nevertheless at increased risk compared to women who did meet NCEP-ATP III criteria but did not have the hypertriglyceridemic waist phenotype. The authors concluded that hypertriglyceridemic waist provides greater, if not at least similar, sensitivity than NCEP-ATP III criteria at much lower cost to healthcare systems.

In conclusion, the metabolic syndrome as diagnosed by any of the major organizations increases the risk of developing CVD events and type 2 diabetes in particular. Generally, NCEP-ATP III clinical criteria predict CVD risk better than other criteria. However, very few studies have compared the ability of clinical criteria to predict diabetes risk. Since WHO criteria usually indicate a CVD risk that is similar to that of the other screening tools, measuring microalbuminuria and insulin sensitivity with clamp techniques does not seem necessary in order to diagnose the metabolic syndrome.

When comparing metabolic syndrome criteria, investigators mainly rely on ORs or HRs to assess the ability of each approach to predict CVD risk. Additional studies should be conducted in various populations worldwide and in all age groups in order to compare the proposed criteria using additional statistical models and evaluate their sensitivities and false-positive rates. For now, most organizations agree that their criteria for clinical diagnosis of the metabolic syndrome are not perfect (particularly regarding identification of metabolic syndrome patients at increased risk of type 2 diabetes and CVD) and that further studies will refine our ability to optimally diagnose the metabolic syndrome in clinical practice.