Using the Kalman Filter to Improve Disparity Estimates for Rare Racial/Ethnic Minorities: An Application to American Indians / Alaska Natives and Chinese Americans Using the National Health Interview Survey

Marc Elliott, RAND
Daniel McCaffrey, RAND
Brian K. Finch, San Diego State University
David Klein, RAND
Nathan Orr, RAND
Nicole Lurie, RAND

While most health surveys collect substantial data on large race-ethnic groups, little is known about the extent of disparities for smaller groups, such as Asian subgroups and American Indians/Alaska Natives. One solution to improving the precision of estimates is to pool data over time. The simplest solution, using the average of several recent years to estimate the present year, is biased in the presence of a linear trend and inefficient in the presence of autocorrelation. We propose and evaluate a Modified Kalman Filter (MKF) approach in which we model racial/ethnic group means from 1997-2004 NHIS data as having a linear trend and autocorrelated group-year level variance components in addition to individual-level variances. We demonstrated that 1997-2004 averages are poor estimators of 2004 values and that for 7 of 18 outcomes examined the MKF substantially improves the accuracy of 2004 estimates for AI/AN and Chinese Americans.