Data Integration – Page 2 – Use It for Good

IPUMS IHGIS: Unlocking International Population and Agricultural Census Data

December 21, 2021November 5, 2020 by mpcblog

By Tracy Kugler

Nearly all countries throughout the world conduct population and housing censuses at least every ten years, and most also conduct agricultural censuses or surveys regularly. These censuses collect information on demographics, education, employment, housing characteristics, migration, agricultural land ownership, agricultural workforce, livestock, crops, and more. The resulting data can be used to study a wide range of questions, from the character of demographic transitions within and across countries, to utilization of irrigation, to educational trends among women.

Unfortunately, this wealth of data has remained largely inaccessible to researchers. The data are typically published in reports as tables summarizing population characteristics. In recent decades, many of these reports have been published as PDF documents and made available on national statistical office websites. While the reports are available, data from a PDF document cannot be easily imported into a statistical or GIS package. Furthermore, the table structures are highly heterogeneous, both across countries and even within the same report.

The International Historical Geographic Information System (IPUMS IHGIS) is designed to provide easy access to these data in a way that researchers can easily use for analysis. In the early phases, IHGIS was known internally as “Project Mako,” named after the Mako shark, which has a global range, voracious appetite, and a reputation for a broad-ranging diet. Like the shark, IHGIS (née Project Mako) will encompass the world and ingest all kinds of data tables.

In the Archive: “25 Years of IPUMS Data”

December 18, 2019July 22, 2019 by mpcblog

“25 Years of IPUMS Data,” the current IPUMS/MPC archive exhibit, highlights a dynamic quarter center history of data innovation at the University of Minnesota. In the late 1980s, the Social History Research Laboratory at the University of Minnesota’s History Department proposed “the creation of a single integrated microdata series composed of public use samples for every year … with the exception of the 1890 census, which was destroyed by fire.” The primary aim was to make the U.S. census microdata “as compatible over time as possible while losing little, if any, of the detail in the original datasets” (Integrated Public Use Microdata Series: A Prospectus).

Wrangling International Census Tables

November 17, 2016November 17, 2016 by mpcblog

aggdatasheets2 IPUMS-International currently disseminates census microdata from 82 countries around the world. It’s an impressive collection. But it still only covers about half the world’s countries. Under the TerraPop project, we are working to assemble a truly comprehensive global collection of population data.

New IPUMS CPS Variable Links Records Across Data Files

October 4, 2016August 26, 2016 by mpcblog

Screenshot 2016-08-25 15.43.37

IPUMS CPS has added a new variable to unlock longitudinal information in CPS data called CPSIDP. The identifier, the result of a major initiative from the Minnesota Population Center, uniquely identifies individuals who are in the CPS and is assigned to the individual in each of the (up to) eight times over sixteen months that the individual is in the survey.

Learning from Data: From Systemization to Investigation

August 17, 2016August 18, 2016 by mpcblog

Benjamin Hartman is a 2016 Summer Diversity Fellow at the MPC. As part of his fellowship, he learned how to take unprocessed data to produce harmonized IPUMS-I data and documentation, make GIS maps, and conduct his own case study investigating the spatial dimensions of internal migration using the Cambodian census. Hartman worked with his colleagues in IPUMS International to create this blog post.

A History of Data: Information Technology and the MPC

July 14, 2016July 14, 2016 by mpcblog

MPCSuperman (1)

Bethel University Professor of History Diana L. Magnuson is documenting the growth of the Minnesota Population Center. Believing that preserving institutional memory is vital, the Center is supporting Magnuson’s work to capture oral histories of past and present MPC faculty and staff.

This is the second in a three-part series, with oral histories from the information technology (IT) side of the MPC. For over 16 years, the IT staff has collaborated with the MPC research staff to recode and disseminate data, develop specialized software, and make research more efficient. The “secret sauce of the MPC” is the longstanding synergistic collaboration between IT and research staff.

Data Release Details: IPUMS-DHS now includes 90 surveys from 20 African countries and India

June 16, 2016June 16, 2016 by mpcblog

The time is now – mums waiting for family planning services. Photo by UK Department for International Development – CC 2.0

As of May 2016, IPUMS-DHS includes over 2000 integrated variables from 90 Demographic and Health Surveys (DHS) in 21 countries, and allows researchers to select women, children, or births as their unit of analysis.