mpcblog – Page 6 – Use It for Good

New to IPUMS USA: The Adjust Monetary Values Feature

September 22, 2022September 22, 2022 by mpcblog

By Danika Brockman and the Adjust Monetary Values Team

Introducing the Adjust Monetary Values feature

The team at IPUMS is excited to introduce a brand-new extract feature, Adjust Monetary Values, which gives you the option to adjust monetary variables to constant units in the IPUMS data extract system. We know firsthand how tedious it can be to compare things like income and rent over time when you have to manually adjust for inflation. This feature allows you to request pre-adjusted monetary variables (e.g., INCWAGE) as part of your extract request! The feature is first being released on IPUMS USA, where you will be able to adjust monetary variables to 2010 dollars.

What does the Adjust Monetary Values feature do?

This feature gives you the option to adjust the monetary variables you have added to your data cart into constant dollars, so that all samples in your data cart are comparable across time for your selected monetary variables. IPUMS USA variables are adjusted to 2010 dollars using the Consumer Price Index for All Urban Consumers (CPI-U). For more information about why the CPI-U was chosen as the pricing index for this feature, see the Monetary Adjustment Feature page.

When you add an inflation-adjusted version of a variable to your data cart, the IPUMS data extract system applies the appropriate CPI-U adjustment factor for each sample year to the variable(s) you’ve selected. Your extract will include both the original monetary variable and the inflation-adjusted monetary variable. Special codes (e.g., NIU, missing) will not be affected by the inflation adjustment. Inflation-adjusted versions of variables will assign all specialty (i.e., non-monetary) codes to a code comprised exclusively of “9’s” with a width two digits greater than the largest value in the original variable (e.g., a variable where the maximum monetary value is “8500,” would assign all specialty codes to “999999” and apply a label of “Non-monetary.”) For details on the original specialty codes and their labels, consult the documentation for the original variable on the IPUMS USA website or cross-tab the adjusted and original variables in your statistical program (note that you may want to include a qualifying if statement so you see only the non-monetary codes).

Automating monthly workflows using IPUMS CPS and the IPUMS Microdata Extract API

September 22, 2022September 9, 2022 by mpcblog

By Renae Rodgers

As many readers will know, the Current Population Survey (CPS) is a monthly labor force survey that is, among other things, the data source for the monthly jobs report (or more formally the Employment Situation reports) from the Bureau of Labor Statistics.

In this blog post, I will show you how to create a reproducible, sustainable monthly workflow to update previous analyses using new data with IPUMS CPS data, IPUMS Microdata Extract API, and the ipumspy Python library.

If this is not your first CPS rodeo, you may already have a monthly workflow for working with IPUMS CPS data that suits your needs just fine – perhaps written in Stata. Did you know you can use ipumspy to make IPUMS CPS extracts from Stata?! Check out the set up instructions and template .do file in this blog post and optimize your monthly analysis even more with the IPUMS Microdata Extract API!

But I digress. In this blog post, I will first walk through a simple analysis using the IPUMS Microdata Extract API and ipumspy. I will then show you how to package that workflow so that it can be simply executed monthly when the most recent data becomes available from IPUMS CPS for refreshed analysis including the newest data.

An example IPUMS CPS, IPUMS Microdata Extract API workflow: teleworking due to COVID-19

Let’s suppose that we’re interested in looking at trends in telework due to COVID-19 over the course of the pandemic. The IPUMS CPS variable COVIDTELEW indicates whether the respondent worked from home at any time during the past 4 weeks due to COVID-19. This example will show us the overall trend in remote work due to COVID-19 as well as how teleworking breaks down by educational attainment. First we’ll define an IPUMS CPS extract that contains COVIDTELEW and EDUC variables and all months from May 2020 to June 2022.

Helpful Functions for Constructing Reproducible Workflows in the Absence of an IPUMS Microdata Metadata API

September 22, 2022August 22, 2022 by mpcblog

by Renae Rodgers

As you may have heard, the IPUMS Microdata Extract API is now in beta for our IPUMS USA and IPUMS CPS collections! Yay! You may have also heard that an IPUMS Metadata API is not yet available. Bummer. This means that users still need to do data discovery via the IPUMS web sites, which means a lot of clicking. It also makes it difficult to set up a reproducible workflow that can be updated as new data become available (and who doesn’t want that?). This blog post will show you some easy functions you can use to retrieve sample metadata to create a monthly or annual workflow that doesn’t require visiting an IPUMS website, and walk through some ipumspy functionality to access metadata for variables already included in your IPUMS extracts. The workflow in this blog post is using ipumspy v0.2.1.

If Stata is your stats package of choice, you can leverage ipumspy to make IPUMS extracts from a Stata do file! You can spruce up your workflow with any of the functions in the blog post below by incorporating them into the template .do file offered in our blog post on making IPUMS extracts from Stata.

Sample Metadata

To get started, import the utilities module from ipumspy. All of the helper functions in this blog post will be using the CollectionInformation class from this module. The sample_ids attribute of CollectionInformation returns a dictionary with sample descriptions as keys and sample ids as values. This information is pulled from the sample ID page for the specified IPUMS data collection. This page is the source of the sample_ids dictionary for IPUMS CPS, and this page is the source of the sample_ids dictionary for IPUMS USA.

from ipumspy import utilities

Functions for retrieving IPUMS CPS sample IDs

First, let’s take a look at the sample ID dictionary for IPUMS CPS.

IPUMS International has brand new low level geographic variables and shapefiles

September 22, 2022July 8, 2022 by mpcblog

By Quinn Heimann

Map showing percentage of households with internet access in the 2014 Myanmar census by township — Map of Myanmar Internet Access

An ongoing goal and challenge for IPUMS-International (IPUMSI) is providing users with the most detailed geography possible. A unique obstacle to this is the confidentiality requirements agreed upon in order to distribute these census and survey samples. Nevertheless, IPUMSI has started launching lower-level geographic variables in samples where data is sufficient and confidentiality thresholds are still met. As of spring 2022, twenty samples have been released with third administrative level geographic data, covering ten countries across Africa and Asia. In addition, accompanying shapefiles are also being distributed to supplement these variables. Shapefiles can be used in conjunction with these more granular geographic variables to map out population trends in greater detail.

Screenshot to IPUMS International third level download page — IPUMSI third level download page

Many of these countries have multiple samples with lower level geography variables available. It is always a goal of IPUMSI to provide users with as much detail as possible for each sample, but this is sometimes hindered by a lack of sufficient data or detail. Some countries, such as Bangladesh and Mali, contain sufficient detail to provide lower level geography for all available samples in IPUMSI. More recent samples often contain more detail and more thorough documentation, whereas oftentimes this level of information is not present for samples produced longer ago.

Reproducible Research with R Markdown, ipumsr, and the IPUMS API

June 14, 2022June 14, 2022 by mpcblog

By Dan Ehrlich

Have you ever wanted to share a project using IPUMS data with a colleague, but then thought, “Oh wait! It is against the terms of use to redistribute my IPUMS data file!”

Maybe you’d like a colleague to explore your findings. Or maybe you’re a teacher with an exercise you’d like your students to review and replicate. In the past, if you wanted someone to use the same IPUMS data that you did, you would need to provide a list of samples and variables and instructions for your collaborator on how to navigate the online data extract system.

If you’re thinking that sounds like a pain, don’t worry, the brand new IPUMS microdata API makes it easier than ever to share your extract definitions with fellow IPUMS users!!! Using the microdata API, you and your collaborators can:

Save an extract definition as a .json file that can be shared freely
Submit a new extract request based on a .json definition
Download data and metadata directly into your project directory (this feature is a personal favorite)

Sharing IPUMS Extract Definitions Using ipumspy

June 10, 2022 by mpcblog

By Renae Rodgers

What is an Extract?

IPUMS users will already be familiar with the concept of an extract, but for those who may just be joining us, we’ll do a brief recap. Public Use data files are often large, unwieldy blocks of data, many variables wide and many many records long. Most analyses will only require a small subset of the available variables in any given dataset, but downloading public data from government agencies is an all-or-nothing endeavor. In addition to offering public use data that is harmonized across time and place, IPUMS allows users to choose only their variables of interest for download. These individualized datasets and accompanying metadata are IPUMS extracts.

What is an Extract Definition?

In short, an IPUMS extract definition is all the information needed to create a user’s personalized extract data file and accompanying metadata – everything short of those files themselves.

An IPUMS extract is defined by:

The name of the IPUMS collection (e.g. “usa”, “cps”)
A list of sample names or IDs (to be) included in the extract file
A list of variable names (to be) included in the extract file
An extract description (e.g. “2022 ACS demographic variables”)

IPUMS users build these extract definitions piece by piece when they create an extract through the IPUMS website, selecting samples, variables, and formats.

Making IPUMS extracts from Stata

June 8, 2022June 8, 2022 by mpcblog

By Renae Rodgers

IPUMS has released a beta version of the Extract API that supports the IPUMS USA and IPUMS CPS microdata collections! Check out An Introduction to the IPUMS Extract API for Microdata for a brief introduction to the IPUMS Extract API for microdata. This blog post will demonstrate how to leverage the IPUMS Extract API and the ipumspy Python library to make IPUMS extracts directly from your Stata .do files! No prior knowledge of or interest in learning Python is required, but you will need Stata 16 or higher, an IPUMS user account, and an API key. All the code in the examples below is available in a template .do file if you would like to follow along.

Setting up Python for use with Stata

While you don’t need to be a Python user to make an IPUMS extract via Stata, there is a little bit of Python set up required. Chuck Huber at Stata has put together a great series of blog posts about how to use Python in Stata and this section is heavily inspired by the first post in that series.

Step 1: Download and install Miniconda

Even if you already have Python installed on your computer, I highly recommend setting up a separate python installation in a conda environment for your ipumspy-in-Stata work. Miniconda is a light-weight version of the package manager Anaconda that will allow you to install Python, ipumspy, and all of the necessary dependencies in a separate environment that you can access from Stata without disturbing anything else on your system that might be using Python. Download Miniconda for your operating system and install.