Disruptions and Remedies to the 2020 American Community Survey 1-Year: What to Know

Forsyth County, North Carolina   |   August, 2021

In our previous post, The COVID-19 Pandemic has Disrupted Critical Sources of Local Data, we highlighted how the COVID-19 pandemic’s disruption of the 2020 American Community Survey (ACS) has caused growing concern over access to the latest information about households in Forsyth County.

From March through September, 2020, the Census Bureau’s typical data collection operations were disrupted, leading to the lowest household response rate in the survey’s history [1]. As a consequence, some specific individual characteristics (such as marital status or educational attainment) produced “unexpected trends” and/or were considerably different from the same measurement published in other data sources [2]. For example, the Centers for Medicare and Medicaid Services released that an additional 9.8 million people between February, 2020, to January, 2021, began receiving Medicaid while the 2020 ACS’ production weights presented a decline by 2.4 million [3].

The Census Bureau has since concluded that the 2020 data collection disruptions were not completely random [4]. Rather, a clear correlation among specific characteristics revealed that estimates were likely biased towards households that:

• “had higher income, 
• were less likely to have multiple jobs, 
• were more likely to have a household member over 60, 
• were more likely to be married, and 
• were less likely to have a child under the age of 10” [5].

This means that the people who responded to the 2020 ACS were unlike the people who did not respond in potentially important ways. 

In an attempt to correct the issues posed by the biased responses, the Census Bureau leveraged other administrative data and a new survey weighting procedure called ‘entropy balancing’ to better align the 2020 estimates with expectations [6]. Despite some improvements, issues still persisted for estimates within areas such as “employment, marital status, educational attainment, and Medicaid coverage” [7].

Table: 2020 Measures that were Reweighted but Still Show Discrepancies to External Benchmarks

To learn more about how individual measures were affected in the 2020 ACS 1-year survey and the Census Bureau’s attempts to align estimates with past survey years and external benchmarks, please reach out to info@forsythfutures.org.

Notably, the Census Bureau recommends users to not compare the 2020 ACS 1-year experimental results to previous ACS 1-year standard estimates since the different survey weighting techniques do not allow for direct comparison [8]. However, the 2016-2020 ACS 5-year’s results can be contrasted with the 2011-2015 ACS 5-year’s as an alternative means of comparison [9]. In addition, 2019 ACS 1-year experimental weights were released in June, 2022 to allow for comparisons strictly to the 2020 ACS 1-year’s experimental results.

Forsyth Futures’ Response

For new projects and data requests, we will assess which data sources provide the best answers to each request’s research questions. Depending on the information needed, this may include using the 2020 ACS 1-year experimental data — strictly when a metric is sought specifically for 2020 only. If comparisons over time are necessary, we will recommend alternate methods, based on best practices, to obtain estimates for the year 2020 that can be acceptably compared to past survey years. 

If you have questions, please reach out to info@forsythfutures.org for additional guidance.