How the COVID-19 Pandemic Impacted the 2020 ACS 1-year PUMS Data

by

By Danika Brockman and Megan Schouweiler

One of the highlights of the past IPUMS USA release was the 2020 ACS 1-year Public Use Microdata Sample (PUMS) file. Due to the effects of the COVID-19 pandemic on 2020 ACS data collection and data quality, the Census Bureau did not release the standard PUMS data. Instead, they released the 2020 ACS 1-year data with experimental weights designed to account for the impact of the COVID-19 pandemic on data quality. In this blog post, we discuss the impact of the COVID-19 pandemic on the 2020 ACS and the development of the experimental weights, and we provide some recommendations for using the 2020 ACS 1-year PUMS file.

Impact of the Covid-19 Pandemic on Data Collection and Data Quality and the Development of the Experimental Weights

The COVID-19 pandemic severely disrupted data collection for the 2020 ACS. All methods of data collection were either shut down or significantly reduced from March 2020 through the end of the year. Data collection for group quarters was particularly disrupted by the COVID-19 pandemic; in-person visits to group quarter facilities were suspended or greatly reduced from March 2020 through the end of the year, and telephone interviews were not conducted due to logistical constraints. Beyond the impacts to data collection methods, the 2020 ACS had significant variability across the 2020 data collection year in response rates for both housing units and group quarters, and had the lowest overall response rate in the history of the ACS1.

Because of these disruptions to data collection, the Census Bureau reviewed the data and determined that the 2020 ACS data had numerous data quality issues, including lower coverage rates for underrepresented populations, decreased sample size and response rates, increased rate of item allocation for all variables (especially household level variables), and a lower response rate for group quarters. These issues had a particularly marked effect on measures of socioeconomic status, building structure, marital status, educational attainment, Medicaid coverage, citizenship, income, and poverty, resulting in biased estimates in the original data1.

To address these data quality issues, the Census Bureau developed and applied additional weighting methodology to the 2020 ACS 1-year file to address the biases present in the data. The weights were designed to address multiple areas of concern, including the issues listed above, the make-up of sample panel groups, an overestimation of vacant housing units, an under-estimation of occupied housing units, an increase in non-response rates, and a decrease in group quarters response rates1.

One area of concern that was especially important to account for with the experimental weights was the high level of non-response bias. To address the non-response bias, the Census Bureau used administrative, third-party, and decennial census data to develop entropy balance weights using methodology tested on the Annual Social and Economic Supplement (ASEC) of the Current Population Survey (CPS). The weights were tested against standard benchmarks, such as income distribution, and were found to reduce the non-response bias in the 2020 sample. Then, the weights were applied to the 2019 ACS 1-year data for evaluation on a more standard ACS data sample. The large changes between 2019 and 2020 that were observed prior to applying these weights reduced to normal levels, suggesting that the weights were reducing the abnormal effects of the pandemic on the 2020 ACS 1-year data file without changing the normal level of expected change2.

Guidelines for How and When to Use the 2020 ACS 1-Year PUMS File and the Experimental Weights

The experimental weights developed for the 2020 ACS 1-year data address the known data quality issues present in the data. However, the Census Bureau still advises that users proceed with caution when using the 2020 ACS 1-year file and the experimental weights.

Considerations for using the 2020 ACS 1-year PUMS file:

  • The IPUMS-harmonized 2020 ACS 1-year PUMS file is formatted and accessible via IPUMS USA like the other ACS PUMS files.
  • The sample size in the 2020 ACS 1-year PUMS file is smaller than previous ACS 1-year PUMS files; not all states represent one percent of the 2020 population3. See the American Community Survey 2020 1-year experimental PUMS file ReadMe for more information.
  • The Census Bureau advises against making comparisons between the 2020 ACS 1-year PUMS file and any other ACS or Census data products. Despite the experimental weights, the full extent of data quality issues is not yet known and may not be adequately accounted for in the experimental weights4. See Comparing 2020 American Community Survey Data for more information about this guideline.
  • The Census Bureau suggests that users who typically use the ACS 1-year estimates to allocate funds, conduct program evaluations, make decisions, or for any other program purposes use an alternative data source (e.g., the 2019 ACS 1-year, the 2015-2019 ACS 5-year). See 2020 American Community Survey 1-Year Estimates: What You Need To Know for a flowchart that helps users determine whether or not their application of the data may be impacted by the changes to the 2020 ACS 1-year file5.

Considerations for using the PUMS experimental weights:

  • The 2020 experimental weights are accessible via the standard IPUMS USA weight variables HHWT, REPWT, PERWT, REPWTP.
  • The experimental weights should be applied in the same way as the standard weights for other ACS PUMS files. See Sample Designs and Sample Weights for more information about weights in the ACS PUMS.
  • Exercise caution when applying the experimental weights to lower levels of geography. The weights were designed to produce estimates for states and larger counties, not PUMAs3.
  • To calculate standard errors and/or margin of error, users must apply the successive difference replication (SDR) formula to the replicate weights. The Census Bureau is not publishing design factors for the 2020 1-year PUMS file, so the generalized variance formula (GVF) will not be available for this sample3.
  • Variance estimates may be smaller than those for previous ACS PUMS samples3.

We hope that this information helps you understand how the 2020 ACS 1-year PUMS file was affected by the COVID-19 pandemic and gives you guidance on how to use these data for your research! Happy data-ing!

– The IPUMS USA team

Sources

  1. Shin, H. B., Daily, D., Cantwell, P., Battle, K., & Waddington, D. G. (2021). An assessment of the COVID-19 pandemic’s impact on the 2020 ACS 1-year data. ACS Research and Evaluation Report Memorandum Series # ACS21-RER-04. U.S. Census Bureau, Washington, D.C.
  2. Rothbaum, J., Eggleston, J., Bee, A., Klee, M., & Mendez-Smith, B. (2021). Addressing nonresponse bias in the American Community Survey during the pandemic using administrative data. ACS Research and Evaluation Report Memorandum Series # ACS21-RER-05 and SEHSD Working Paper #2021-24. U.S. Census Bureau, Washington, D.C.
  3. American Community Survey Office. (2021). American Community Survey 2020 1-year experimental PUMS file- ReadMe. U.S. Census Bureau, Washington, D.C.
  4. U.S. Census Bureau. (2021, November 23). Comparing 2020 American Community Survey data. https://www.census.gov/programs-surveys/acs/guidance/comparing-acs-data/2020.html.
  5. U.S. Census Bureau. (2021, October 8). 2020 American Community Survey 1-year estimates: What you need to know. https://www.census.gov/programs-surveys/acs/library/flyers/flow-chart.html.
Post Views: 1,497