Even More IPUMS Data Available in the SDA Online Data Analysis Tool

December 19, 2024 by mpcblog

By Daniel Backman

Beyond offering the ability to create and download customized datasets from the IPUMS microdata collections, we also support web-based analysis of the data through the SDA (Survey Documentation and Analysis) online data analysis tool. SDA empowers users to analyze IPUMS data directly from their web browsers without the need for additional software or advanced programming skills. Whether you’re a seasoned researcher or a student exploring data for the first time, the SDA tool makes it easier than ever to unlock insights from our datasets. If you’re a current SDA user and ready to get started, check out the new datasets from IPUMS CPS and IPUMS MEPS. Otherwise, read on to learn more about SDA and how to use this tool to analyze IPUMS data.

About IPUMS & SDA

What is SDA?

The SDA tool is a web-based interface that allows you to generate frequency tables, cross-tabulations, and summary statistics; create customized data visualizations, including bar charts, line graphs, and scatter plots; perform regression analysis; and export results as a CSV file for presentations or further analysis.

SDA increases the accessibility of data by allowing users to analyze data through a web-interface without needing to use (or purchase!) statistical software. There is detailed guidance on how to use the tool for analyses and how to manipulate variables. Additionally, it provides exceptionally fast real-time processing of data, making it ideal for use in the classroom or other interactive settings. See our data training exercises page for exercises that will guide you through using SDA to analyze IPUMS data.

What’s new?

IPUMS CPS

In addition to the previously available ASEC data for 1962-present, IPUMS CPS has released 17 additional datasets covering BMS data and supplement topics.

All BMS Samples: 1976-Present. This dataset contains every month and year of the BMS sample (588 and counting) and all BMS person and household-level core variables.
CPS Supplement Specific Datasets. These datasets are tailor-made to include all available samples for each specific supplemental topic available via IPUMS CPS. There are 16 supplement-specific datasets, such as Tobacco Use, Food Security, and Education. Supplement datasets include supplement-specific weights and all supplement variables offered by IPUMS CPS in addition to the BMS core person and household-level variables.

IPUMS MEPS

IPUMS MEPS now offers the ability to analyze person-level variables derived from the Full Year Consolidated (FYC) files (i.e., those listed under the “Annual” variable drop down menu).

All MEPS Combined Samples: 1996-Present. This dataset contains all years of MEPS FYC data and all person-level annual variables available from IPUMS MEPS in those years. This combined dataset allows for pooled and time trend analysis.
Single-year MEPS samples. These datasets each contain only one year of MEPS and their corresponding variables and weights. The single year datasets allow for faster data analysis.

These are just the newest additions of IPUMS data to SDA; they augment previously available datasets available for online analysis from IPUMS USA (decennial censuses and ACS), IPUMS CPS (ASEC), IPUMS International, IPUMS Time Use (ATUS and MTUS), IPUMS NHIS, and IPUMS DHS.

Let’s Look at an Example!

Because IPUMS MEPS are newly available for analysis with SDA, let’s use MEPS as an example.

First, How to Get Started

Using IPUMS MEPS SDA as our example, follow these steps to begin exploring data with the SDA tool:

Choose a dataset. From the IPUMS MEPS SDA page, select your dataset. Let’s use the All MEPS Combined dataset. This dataset contains all years of MEPS data, including the relevant technical survey variables, such as weights, primary sampling unit, and strata variables. For more information on technical variable considerations when using these SDA datasets, view the documentation on our IPUMS MEPS SDA page.

Select variables for analysis. You can use the SDA built-in variable search and selection tools to identify variables of interest. For this analysis, let’s look at the distribution of insulin users by sex across the past ten years of the MEPS data.

To discover variables, you can use the search tool (in the upper-left corner of the SDA interface in the Variable Selection pane) or explore the drop-down menus (below the search bar in the Variable Selection pane – note that these topical groupings correspond to those on the IPUMS MEPS website). If you know the variable name or prefer to explore variables through the IPUMS MEPS user interface, you can enter variable names directly in the fields for the SDA program you are interested in running.

Screenshot of SDA home page for the data collection All MEPS: 1996-2022 highlighting three methods of variable discovery. With arrows pointing to the view variable metadata in the selected text box, the variable group menu, or entering the variable name directly in the Row box. — Figure 1: Screenshot of SDA home page for the data collection All MEPS: 1996-2022 highlighting three methods of variable discovery: view variable metadata in Selected text box, find variables in the variable group menu, or enter variable name directly in **Row** box to begin analysis.

Run your analysis. For this example, let’s create a cross-tabulation showing the sex distribution (SEX) of insulin users (INSULIN==2 “Yes, now taking insulin”) over the years of 2007-2017. In 2018, there was a change in MEPS to how conditions were reported which includes diabetes, so for this example, we will look at 2007-2017. We will also calculate the confidence intervals and standard errors. Under the Tables tab (this is the default view in SDA), we will enter the criteria for our cross-tabulation as follows:

Row: SEX
Column: YEAR(2007-2017)
Control: INSULIN(2)
Selection Filter(s):
WEIGHT: DIABWEIGHT

Appending parentheses that contain a subset of codes after the variable is a shortcut to define a filter (see the SDA variable manipulation guidance for other tips). INSULIN is part of the MEPS Diabetes Care Supplement which is fielded only to MEPS household members who were ever diagnosed with diabetes, and requires the use of the Diabetes Care weight, so you will need to select DIABWEIGHT from the weight drop-down menu (PERWEIGHT is the default weight).

To calculate confidence intervals and standard errors, check the applicable boxes under the Output Options accordion menu. SDA will automatically apply the appropriate sample design variables to correctly estimate standard errors. There are lots of ways to customize your output through the Chart Options, Decimal Options and Create and Download CSV file, but we won’t use those for this example. When you are ready to generate your cross-tab, click the “Run the Table” box!

Screenshot of SDA home page with Output Options menu open. Arrows pointing to Row: Sex, Column: year(2007-2017), and Control: insulin(2). Under the Output Options arrows point to selecting "Confidence intervals - Level: 95 percent and Standard error of each percent. Arrow and highlight box around "Run the Table" — Figure 2: Screenshot of SDA home page with Output Options menu open. Variables displayed will create a table that cross-tabulates sex and year for adults currently using insulin from 2007-2017. Under the output options menu, the checkboxes for ‘Confidence Intervals’ and ‘Standard error of each percent’ are selected.

Visualize results. After you “run the table,” SDA should generate your results within seconds. By default, the cross-tabulation output will include column percentages, weighted population count, and a bar chart. Because we selected the confidence intervals and standard error options, those will be displayed as well. At the bottom of the results page, you will see that our standard error calculations were produced using STRATAPLD and PSUPLD, which are the default technical survey variables for this combined years dataset.

We can see that the proportion of females currently using insulin increased between the years of 2007-2017.

Figure 3: Screenshot of SDA generated cross-tabulation table and bar chart using variable definitions entered on Figure 2. The cells within the table display column percentages, Confidence Interval and Standard error and the weighted population.

Quick Tips

Read the documentation. Familiarize yourself with SDA’s capabilities and limitations to make the most of its features. The SDA online analysis homepage for each IPUMS data collection includes links to relevant technical documentation for that specific data collection.
Use SDA on its own or with IPUMS extracts. Frequency tables and descriptive statistics are a great way to explore the data before you make an extract to run more complex analyses. For example, you can confirm that there is sufficient sample size for your analysis by showing the unweighted number of observations.
Leverage the CSV export option. If you want to produce figures with your SDA output outside of the SDA tool, save yourself the hassle of cleaning up output that you have copy-pasted into Excel. Instead, export the output as a CSV.
Customize your options. The output, chart, and decimal options drop-down menus allow you to tailor the display of your results. Explore the choices to help drive home the most important takeaways from your results.

Updated Land Cover Summaries for Census Tracts, County Subdivisions, Counties, and Places

December 2, 2024 by mpcblog

By David Van Riper, ISRDI Director of Spatial Analysis

What’s new?

We just released updated land cover summaries for census tracts, county subdivisions, counties and places. Our land cover summaries describe the proportion of a particular geographic unit (e.g., a county or a census tract) that is covered by a particular land cover class (e.g., deciduous forest, evergreen forest, or cultivated crops). This release provides users with land cover summaries from nine vintages of the National Land Cover Database (NLCD) – 2001, 2004, 2006, 2008, 2011, 2013, 2016, 2019, and 2021. Summaries are available for 2010, 2020, and 2022 census tract, county subdivisions, counties, and places. We include 2022 versions of these geographic units because that was the year the Census Bureau began identifying planning regions in Connecticut. These regions replaced Connecticut’s historical counties, which have long had no official administrative function. These new planning regions changed the unique identifiers for census tracts, county subdivisions, and counties.

Why did IPUMS NHGIS create these land cover summaries?

Land cover data is commonly used to study the impacts of natural events such as hurricanes or human modifications such as converting forest to agriculture or agricultural land to developed land. Land cover data is typically released as a high spatial resolution, gridded spatial dataset where each grid cell (or pixel) is assigned to a land cover category (Figure 1, Panel A). The gridded data almost never align with the geographic units, and the high spatial resolution yields massive files that can be slow to process. A single NLCD file is 25 gigabytes in size.

We summarized nine versions of the NLCD to multiple sets of geographic units so that users can easily integrate the data into analyses already structured around geographic units. This reduces the burden on individual users to create such summaries themselves.

Census Data for Good: Analysis to Action

October 9, 2024 by mpcblog

By Lara Cleveland

IPUMS International regularly asks representatives of National Statistical Offices (NSOs) around the world to share their data with the research community. While IPUMS offers a license payment to countries for the right to redistribute microdata, NSO representatives are most interested in how sharing data with IPUMS will benefit the people of their countries. After 30 years of harmonizing data that NSOs have shared with us, IPUMS can indeed point to innovative research from data users all over the world, many at major universities in these partner countries. Directors of statistical offices, especially those with close ties to academia, are thrilled that the data are used for scholarly scientific production and for the purpose of educating the next generation. However, most of these leaders are much more interested in how data sharing leads to effective policy. And they want examples. They are essentially asking how the data have been “used for good,” as the original IPUMS tagline, “Use it for good!” implores.

IPUMS supports the Sustainable Development Goals

In response, IPUMS has been following data-to-policy trails where we can find them. The United Nations’ efforts to establish and measure the Sustainable Development Goals (SDGs) have provided wins in this area. Early in the life of the SDGs, colleagues from the World Health Organization visited IPUMS to leverage detailed information in the occupational variables for locating the health workforce. Microdata from censuses helped them measure the density of a range of health worker classifications at subnational levels. The International Organization for Migration (IOM) did similar work to disaggregate census-based SDGs by migratory status. At the start of the pandemic, The United Nations Population Fund (UNFPA) used IPUMS census microdata to spin up a dashboard showing the living arrangements of older adults, again at subnational levels. Each of these applications of IPUMS International data resulted in policy recommendations, informed by additional data, additional policy research, and pilot projects.

Constructing comparable intimate partner violence indicators across the DHS, MICS, and PMA health surveys

September 17, 2024September 16, 2024 by mpcblog

By Miriam King, Anna Bolgrien, Mehr Munir, and Devon Kristiansen

The three data series comprising IPUMS Global Health—IPUMS DHS, IPUMS PMA, and IPUMS MICS—contain intersecting subjects related to women’s and children’s health, while retaining distinct patterns of temporal and geographic coverage. This content overlap opens the door to combining harmonized data across the three surveys, to extend time series and/or increase the number of countries in comparative analyses. However, there are important yet subtle differences between these survey types, in sample frames, questionnaire wording, and variable responses and universes, which require cautious consideration. As the example below demonstrates, researchers must use extra care to avoid errors when combining data across IPUMS DHS, MICS, and PMA.

A July 2024 article in the Journal of Public Health Policy, “Constructing Comparable Intimate Partner Violence Indicators across DHS, MICS, and PMA Health Surveys,” describes some challenges and solutions to combining data across these IPUMS databases, using measures of intimate partner violence as an example. The piece, authored by Devon Kristiansen and colleagues at IPUMS, notes two necessary steps in combining data across survey types:

Identify and combine only variables with similar question wording
Adjust the samples to include only comparable subpopulations

Harmonized Malaria Indicator Survey (MIS) Data Now in IPUMS DHS

September 5, 2024September 3, 2024 by mpcblog

By Miriam King, Senior Research Scientist

Malaria is a pressing global health problem, with nearly 250 million malaria cases in 2022, according to the World Health Organization. Approximately 95 percent of malaria deaths were in Africa, with three-quarters of those deaths to children under 5. Climate change is increasing the transmission of mosquito-borne diseases, such as malaria. When IPUMS DHS recently received supplemental funding to support research on Climate Change Effects on Health, adding data on malaria was a top priority. Specifically, IPUMS DHS chose to integrate data from the DHS Malaria Indicator Surveys (MIS).

MIS have been fielded in nearly 30 African countries during the twenty-first century. Developed under an international partnership coordinating efforts to fight malaria, MIS surveys include some standard DHS variables on topics such as demographics, fertility, and household characteristics. MIS questionnaires also include hundreds of questions related to malaria. People’s knowledge about malaria causes, symptoms, and prevention; use of bednets; diagnosis and treatment of malaria, especially for pregnant women and children; exposure to public health messaging; and diagnostic blood testing for malaria in children under 5 are among the topics covered.

Map of Africa with the countries with MIS data in IPUMS DHS filled in with purple — Figure 1: Countries with MIS Data in IPUMS DHS

IPUMS DHS users now have access to harmonized data from 38 MIS samples, with geographic coverage shown in Figure 1. We prioritized harmonizing responses to MIS questions that matched variables already in the IPUMS DHS database, for approximately 700 widely available variables.

Digitizing and Exploring Qatar’s Population Censuses

June 24, 2024June 24, 2024 by mpcblog

By Shine Min Thant

Qatar, a small yet influential state in the Middle East, is a very interesting case study for demographic research because of its rapid development over the past thirty years. Qatar occupies a peninsula only slightly larger than the U.S. state of Rhode Island that juts out into the Persian Gulf from its border with Saudi Arabia. The country has experienced relatively rapid economic growth since the late 20th century, mainly due to its vast reserves of natural gas and oil. This newfound wealth allowed Qatar to invest heavily in its healthcare, infrastructure, and education – therefore making the country an ideal case study for social change and development. Additionally, a recent surge in Qatar’s immigrant population (which constitutes over 78 percent of the population) also makes it an ideal country to study social mobility and social change.

As part of the ISRDI Diversity Fellowship Program, I worked with Dr. Tracy Kugler, Professor Steven Manson, Professor Evan Roberts, and undergraduate student Rawan AlGahtani on a project to examine Qatar’s change using census data from 1984, 1997, and 2004. Summary tables from all three censuses were previously only available as printed documents. As a first step, we needed to transform the data from a hard-to-get printed format to widely accessible IPUMS IHGIS format. This process included multiple steps from conducting optical character recognition (OCR) to conducting data quality checks using R scripts (Figure 1).

Figure 1: IPUMS IHGIS Workflow

New Data Release from IPUMS International – From Mexico to MOSAIC

May 9, 2024May 9, 2024 by mpcblog

By Lara Cleveland and Jane Lee

IPUMS International has released new data! Eighteen new census samples have been added to the collection, including data from Côte d’Ivoire, which is new to IPUMS International. Newly released census samples include Cambodia (2019), Côte d’Ivoire (1988, 1998), Denmark (1845, 1880, 1885), Laos (1995, 2015), Mexico (2020), Peru (2017), Puerto Rico (2015, 2020), Switzerland (2011), United Kingdom (1961, 1971), United States (2015, 2020) and Vietnam (2019). As always, we gratefully acknowledge the national statistical offices of all the countries partnering with IPUMS International to make data available for research.

New geography variables are also now available with harmonized migration variables at the second-administrative level; the codes for the newly released migration variables match existing IPUMS International geography codes and labels. As an example, the geographic units in the migration variable for Mexico at the municipo level (place of residence 5 years ago, MIG2_5_MX) are reconciled to the boundaries for place of current residence (GEO2_MX).

This is a map showing the 2020 census 5-year migration rates for GEO1 in Mexico, and GEO2 in Nuevo Leon state — 2020 census 5-year migration rates for GEO1 in Mexico, and GEO2 in Nuevo Leon state. Map by Quinn Heimann