Census Data
From charlesreid1
This page covers the use of the census reporter API to obtain census data: https://github.com/censusreporter/censusreporter
Note that the Census Map page also contains some similar information, but focuses on drawing maps with the resulting data. This page focuses specifically on the API and extracting data from the API.
Contents
Overview
Let's start with what we're obtaining from the API and what we need to draw a map.
The census API provides a gateway to census data, which is essentially a very large number of tables of data (the columns), broken down into components and sub-components, for hundreds of thousands of geographical entities (the rows). We can think of each piece of data about a particular location as a cell in a giant spreadsheet.
Both the data and the geographical entities are organized hierarchically. For example, a single table about poverty might have poverty statics broken down by demographics or education level, resulting in several dozen columns. The geographical entities are likewise organized in a hierarchy including the country, states, counties, and census tracts.
Useful Information and Lists
The census reporter API code has a long list of useful constants, such as hierarchy names and state fips codes:
Geographical Information: References
Starting with some references:
The Census Reporter Geography page (https://censusreporter.org/topics/geography/) contains a list of codes for geographical hierarchies:
- 010 - United States
- 040 - States
- 050 - Counties
- 140 - Census tracts
- 310 - Metropolitan and micropolitan areas (cities)
- 500 - Congressional district
- 860 - Zip codes
The Missouri Census Data Center's Geography page has a much more extensive overview:
- codes: https://factfinder.census.gov/help/en/summary_level_code_list.htm
- hierarchies: http://mcdc.missouri.edu/geography/sumlevs/index.shtml
The official census documentation contains a page on geographic terms and concepts:
Link to documentation chapter for each hierarchy level:
The GeoIDs of different states are called FIPS codes:
- FIPS codes for each state are listed on page 16 of this PDF: https://www2.census.gov/geo/pdfs/reference/GARM/Ch4GARM.pdf
- Small JSON file containing all 50 FIPS codes: https://raw.githubusercontent.com/charlesreid1/a-shrubbery/master/census/ca/fips.json
If you need the GeoID of a county or a city, you can also search. For example, to find the metropolitan area corresponding to New York City, you could pass the code for state level entity and pass the code for New York state, and ask for all child metropolitan areas. These could then be parsed and the names searched for a keyword, "New York City".
Geographical Information: API
Provide Parent, Get Children
This section covers how to provide a geographical entity at one level of the hierarchy and receive a set of children geographical entities at another level of the hierarchy.
When we call the geo API, we have a lot of ways to obtain information. We must specify the following:
- What geographical hierarchy level do we want information about? (The final results will be aggregated by this hierarchy type)
- What geographical entity ID are we passing in? (We have to specify some entity with some ID to limit the results, so we decide what geographical hierarchy level)
- The specific ID of the geographical entity
The base API call looks like this:
https://api.censusreporter.org/1.0/geo/show/tiger2016?geo_ids=<level to return>|<level of geoid>00US<geoid>
Here is an example where we request geographical information about county-level boundaries. To limit the results, we specify that we will provide a geogrpahical entity that is a metropolitan-area code (310), then we provide the geoid of that entity (41860, corresponding to San Francisco)
https://api.censusreporter.org/1.0/geo/show/tiger2016?geo_ids=050|31000US41860
^^^
050 return county level information
^^^
310 use metro geoid that will be passed
^^ ^^
00 US
^^^^^
41860 your city metro ID (SF, in this case)
Provide/Receive Single Entity
If we wish to provide a single geographical entity and get its boundaries, we can do that by providing two pieces of information:
- Geographical hierarchy level of entity you will specify
- GeoID of entity you are specify
Then call the base API as follows:
curl "http://api.censusreporter.org/1.0/geo/tiger2016/<hierarchy level>00US<geoid>?geom=True
(Note that if you leave out the geom=True, you won't get any GeoJson shape information.)
For example, to get a geojson file containing the boundaries of Yuma County, you can call the API as follows:
curl "https://api.censusreporter.org/1.0/geo/tiger2016/05000US04027?geom=True" > yuma.geojson
here is the result:
{"geometry":{"type":"Polygon","coordinates":[[[-114.763998,32.643364],[-114.763512,32.645995],[-114.76495,32.649391],[-114.75831,32.655178],[-114.751079,32.659789],[-114.74948,32.66178],[-114.748,32.664184],[-114.747817,32.667777],[-114.745344,32.67219],[-114.744487,32.678683],[-114.730453,32.698844],[-114.72981,32.700282],[-114.730086,32.704298],[-114.72241,32.713597],[-114.717695,32.721547],[-114.714522,32.73039],[-114.709074,32.735456],[-114.705717,32.74158],[-114.702223,32.74541],[-114.69879,32.744846],[-114.68823,32.73753],[-114.682614,32.737348],[-114.665921,32.734028],[-114.654247,32.73357],[-114.635006,32.731372],[-114.617395,32.728257],[-114.61567,32.728454],[-114.615112,32.734515],[-114.581784,32.734946],[-114.581736,32.74232],[-114.564508,32.742274],[-114.564447,32.749554],[-114.539224,32.749812],[-114.539092,32.756949],[-114.526856,32.757094],[-114.528443,32.767276],[-114.532432,32.776922],[-114.531669,32.791185],[-114.529633,32.795477],[-114.522031,32.801675],[-114.515389,32.811439],[-114.510327,32.816488],[-114.494116,32.823287],[-114.468971,32.845155],[-114.465711,32.873681],[-114.462929,32.907944],[-114.464448,32.913128],[-114.47664,32.923628],[-114.479005,32.928291],[-114.480925,32.936276],[-114.478456,32.940555],[-114.470768,32.949424],[-114.468061,32.95526],[-114.467315,32.96406],[-114.468605,32.971649],[-114.470511,32.973858],[-114.476532,32.975174],[-114.480303,32.973748],[-114.481315,32.972064],[-114.488625,32.969946],[-114.490129,32.969884],[-114.492938,32.971781],[-114.495712,32.980075],[-114.499797,33.003905],[-114.501226,33.007556],[-114.507956,33.019708],[-114.516454,33.027617],[-114.371523,33.027489],[-114.370783,33.030012],[-114.268748,33.029815],[-114.269164,33.215034],[-114.268983,33.289532],[-114.268183,33.289429],[-114.267815,33.463574],[-113.958042,33.464655],[-113.958046,33.377578],[-113.434499,33.376879],[-113.33392,33.377425],[-113.334217,33.25004],[-113.333603,33.24763],[-113.333212,33.22721],[-113.334066,33.140282],[-113.333767,32.038607],[-113.78168,32.179034],[-114.250775,32.323909],[-114.813613,32.494276],[-114.813991,32.497231],[-114.812316,32.500054],[-114.813753,32.50426],[-114.816591,32.507696],[-114.814321,32.509023],[-114.807726,32.508726],[-114.804076,32.510375],[-114.802211,32.513191],[-114.802238,32.515206],[-114.80367,32.516374],[-114.809672,32.517567],[-114.810374,32.518391],[-114.810482,32.521758],[-114.813348,32.524186],[-114.802181,32.536414],[-114.802018,32.53946],[-114.804776,32.541659],[-114.80583,32.546354],[-114.803883,32.548001],[-114.795635,32.550956],[-114.793769,32.552329],[-114.791551,32.557023],[-114.792955,32.562085],[-114.792088,32.568497],[-114.793224,32.569459],[-114.794684,32.568703],[-114.795253,32.56662],[-114.79766,32.564444],[-114.801311,32.562865],[-114.80683,32.55888],[-114.810318,32.558628],[-114.812914,32.560049],[-114.813995,32.562201],[-114.813968,32.566209],[-114.812995,32.568706],[-114.804421,32.572941],[-114.801877,32.576009],[-114.801471,32.578255],[-114.803879,32.580889],[-114.803987,32.582652],[-114.800441,32.588079],[-114.799737,32.592177],[-114.801548,32.598591],[-114.805932,32.600721],[-114.807906,32.602783],[-114.807879,32.605416],[-114.809042,32.608806],[-114.809393,32.617119],[-114.80739,32.621332],[-114.799302,32.625115],[-114.791179,32.621833],[-114.787715,32.623573],[-114.782573,32.624304],[-114.781766,32.625613],[-114.782235,32.630215],[-114.780049,32.632725],[-114.768199,32.639874],[-114.763998,32.643364]]]},"type":"Feature","properties":{"awater":13171316,"display_name":"Yuma County, AZ","simple_name":"Yuma County","sumlevel":"050","population":203292,"full_geoid":"05000US04027","aland":14281265636}}
Census Tables: References
Please keep in mind that the tables get quite complicated.
Big honkin list of all tables:
Census reporter lists a large number of table codes (topics):
An example of what a single table can contain:
Another table example: B07007: "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States":
Census Tables: API
Tables beginning with B have the most detailed column breakdown, while a C table for the same numbers will have fewer columns.
Tabulation Endpoint
To get basic information about a table, use the tabulation endpoint:
https://api.censusreporter.org/1.0/tabulation/<table-id>
such as 07007. Example:
$ curl "https://api.censusreporter.org/1.0/tabulation/07007"
{"tabulation_code": "07007", "table_title": "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States", "tables_by_release": {"five_yr": ["B07007", "B07007PR"], "one_yr": ["B07007", "B07007PR"], "three_yr": []}, "universe": "Population 1 Year and Over in the United States", "topics": ["migration", "citizenship"], "subject_area": "Residence Last Year - Migration", "simple_table_title": "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States"}
And prettified:
{
"tabulation_code": "07007",
"table_title": "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States",
"tables_by_release": {
"five_yr": [
"B07007",
"B07007PR"
],
"one_yr": [
"B07007",
"B07007PR"
],
"three_yr": []
},
"universe": "Population 1 Year and Over in the United States",
"topics": [
"migration",
"citizenship"
],
"subject_area": "Residence Last Year - Migration",
"simple_table_title": "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States"
}
Table Endpoint
To get column information about a tale, use the table endpoint:
$ curl "https://api.censusreporter.org/1.0/table/<table-id-plus-letter>"
such as B07007. Example:
$ curl "https://api.censusreporter.org/1.0/table/B07007"
{"table_id": "B07007", "table_title": "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States", "simple_table_title": "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States", "subject_area": "Residence Last Year - Migration", "universe": "Population 1 Year and Over in the United States", "denominator_column_id": "B07007001", "topics": ["migration", "citizenship"], "columns": {"B07007001": {"column_title": "Total:", "indent": 0, "parent_column_id": null}, "B07007002": {"column_title": "Native", "indent": 2, "parent_column_id": "B07007001"}, "B07007003": {"column_title": "Foreign born:", "indent": 2, "parent_column_id": "B07007001"}, "B07007004": {"column_title": "Naturalized U.S. citizen", "indent": 3, "parent_column_id": "B07007003"}, "B07007005": {"column_title": "Not a U.S. citizen", "indent": 3, "parent_column_id": "B07007003"}, "B07007006": {"column_title": "Same house 1 year ago:", "indent": 1, "parent_column_id": "B07007001"}, "B07007007": {"column_title": "Native", "indent": 2, "parent_column_id": "B07007006"}, "B07007008": {"column_title": "Foreign born:", "indent": 2, "parent_column_id": "B07007006"}, "B07007009": {"column_title": "Naturalized U.S. citizen", "indent": 3, "parent_column_id": "B07007008"}, "B07007010": {"column_title": "Not a U.S. citizen", "indent": 3, "parent_column_id": "B07007008"}, "B07007011": {"column_title": "Moved within same county:", "indent": 1, "parent_column_id": "B07007001"}, "B07007012": {"column_title": "Native", "indent": 2, "parent_column_id": "B07007011"}, "B07007013": {"column_title": "Foreign born:", "indent": 2, "parent_column_id": "B07007011"}, "B07007014": {"column_title": "Naturalized U.S. citizen", "indent": 3, "parent_column_id": "B07007013"}, "B07007015": {"column_title": "Not a U.S. citizen", "indent": 3, "parent_column_id": "B07007013"}, "B07007016": {"column_title": "Moved from different county within same state:", "indent": 1, "parent_column_id": "B07007001"}, "B07007017": {"column_title": "Native", "indent": 2, "parent_column_id": "B07007016"}, "B07007018": {"column_title": "Foreign born:", "indent": 2, "parent_column_id": "B07007016"}, "B07007019": {"column_title": "Naturalized U.S. citizen", "indent": 3, "parent_column_id": "B07007018"}, "B07007020": {"column_title": "Not a U.S. citizen", "indent": 3, "parent_column_id": "B07007018"}, "B07007021": {"column_title": "Moved from different state:", "indent": 1, "parent_column_id": "B07007001"}, "B07007022": {"column_title": "Native", "indent": 2, "parent_column_id": "B07007021"}, "B07007023": {"column_title": "Foreign born:", "indent": 2, "parent_column_id": "B07007021"}, "B07007024": {"column_title": "Naturalized U.S. citizen", "indent": 3, "parent_column_id": "B07007023"}, "B07007025": {"column_title": "Not a U.S. citizen", "indent": 3, "parent_column_id": "B07007023"}, "B07007026": {"column_title": "Moved from abroad:", "indent": 1, "parent_column_id": "B07007001"}, "B07007027": {"column_title": "Native", "indent": 2, "parent_column_id": "B07007026"}, "B07007028": {"column_title": "Foreign born:", "indent": 2, "parent_column_id": "B07007026"}, "B07007029": {"column_title": "Naturalized U.S. citizen", "indent": 3, "parent_column_id": "B07007028"}, "B07007030": {"column_title": "Not a U.S. citizen", "indent": 3, "parent_column_id": "B07007028"}}}
Prettified:
{
"table_id": "B07007",
"table_title": "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States",
"simple_table_title": "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States",
"subject_area": "Residence Last Year - Migration",
"universe": "Population 1 Year and Over in the United States",
"denominator_column_id": "B07007001",
"topics": [
"migration",
"citizenship"
],
"columns": {
"B07007001": {
"column_title": "Total:",
"indent": 0,
"parent_column_id": null
},
"B07007002": {
"column_title": "Native",
"indent": 2,
"parent_column_id": "B07007001"
},
"B07007003": {
"column_title": "Foreign born:",
"indent": 2,
"parent_column_id": "B07007001"
},
"B07007004": {
"column_title": "Naturalized U.S. citizen",
"indent": 3,
"parent_column_id": "B07007003"
},
"B07007005": {
"column_title": "Not a U.S. citizen",
"indent": 3,
"parent_column_id": "B07007003"
},
"B07007006": {
"column_title": "Same house 1 year ago:",
"indent": 1,
"parent_column_id": "B07007001"
},
"B07007007": {
"column_title": "Native",
"indent": 2,
"parent_column_id": "B07007006"
},
"B07007008": {
"column_title": "Foreign born:",
"indent": 2,
"parent_column_id": "B07007006"
},
"B07007009": {
"column_title": "Naturalized U.S. citizen",
"indent": 3,
"parent_column_id": "B07007008"
},
"B07007010": {
"column_title": "Not a U.S. citizen",
"indent": 3,
"parent_column_id": "B07007008"
},
"B07007011": {
"column_title": "Moved within same county:",
"indent": 1,
"parent_column_id": "B07007001"
},
"B07007012": {
"column_title": "Native",
"indent": 2,
"parent_column_id": "B07007011"
},
"B07007013": {
"column_title": "Foreign born:",
"indent": 2,
"parent_column_id": "B07007011"
},
"B07007014": {
"column_title": "Naturalized U.S. citizen",
"indent": 3,
"parent_column_id": "B07007013"
},
"B07007015": {
"column_title": "Not a U.S. citizen",
"indent": 3,
"parent_column_id": "B07007013"
},
"B07007016": {
"column_title": "Moved from different county within same state:",
"indent": 1,
"parent_column_id": "B07007001"
},
"B07007017": {
"column_title": "Native",
"indent": 2,
"parent_column_id": "B07007016"
},
"B07007018": {
"column_title": "Foreign born:",
"indent": 2,
"parent_column_id": "B07007016"
},
"B07007019": {
"column_title": "Naturalized U.S. citizen",
"indent": 3,
"parent_column_id": "B07007018"
},
"B07007020": {
"column_title": "Not a U.S. citizen",
"indent": 3,
"parent_column_id": "B07007018"
},
"B07007021": {
"column_title": "Moved from different state:",
"indent": 1,
"parent_column_id": "B07007001"
},
"B07007022": {
"column_title": "Native",
"indent": 2,
"parent_column_id": "B07007021"
},
"B07007023": {
"column_title": "Foreign born:",
"indent": 2,
"parent_column_id": "B07007021"
},
"B07007024": {
"column_title": "Naturalized U.S. citizen",
"indent": 3,
"parent_column_id": "B07007023"
},
"B07007025": {
"column_title": "Not a U.S. citizen",
"indent": 3,
"parent_column_id": "B07007023"
},
"B07007026": {
"column_title": "Moved from abroad:",
"indent": 1,
"parent_column_id": "B07007001"
},
"B07007027": {
"column_title": "Native",
"indent": 2,
"parent_column_id": "B07007026"
},
"B07007028": {
"column_title": "Foreign born:",
"indent": 2,
"parent_column_id": "B07007026"
},
"B07007029": {
"column_title": "Naturalized U.S. citizen",
"indent": 3,
"parent_column_id": "B07007028"
},
"B07007030": {
"column_title": "Not a U.S. citizen",
"indent": 3,
"parent_column_id": "B07007028"
}
}
}
Data Show API
The data show API combines the two functions above to get tables of data for specified geographical entities.
To query the API for table information about a particular geographical entity, you should provide three pieces of information:
- the ID of the table of data to query
- the hierarchical level of the geo entity you're passing in (state, county, census tract, etc.)
- the geo ID of the geo entity you're passing in
The URL is the /data/show URL endpoint, and it is requested using the base API format:
https://api.censusreporter.org/1.0/data/show/latest?table_ids=<table-ids-here>&geo_ids=<geo-ids-here>
Reference: https://github.com/censusreporter/census-api/blob/master/API.md
Data Show Example: Single Table, Single Geo Entity
For example, this requests table B13016 from the state (040) with geoid 55 (Wisconsin):
curl "https://api.censusreporter.org/1.0/data/show/latest?table_ids=B13016&geo_ids=04000US55"
Data Show Example: Single Table, Multiple Geo Entities
We can also obtain tables of data for multiple geographical entities at a time by listing them in a comma separated list:
curl "https://api.censusreporter.org/1.0/data/show/latest?table_ids=B07007&geo_ids=04000US06,04000US28"
Data Show Example: Multiple Tables, Multiple Geo Entities
Obtaining multiple tables for multiple geographic entities is also possible by constructing comma separated lists for both table_ids and geo_ids arguments:
curl "https://api.censusreporter.org/1.0/data/show/latest?table_ids=B07007,B13016&geo_ids=04000US06,04000US28"
This returns the GeoJSON shown below:
{
"release": {
"id": "acs2016_1yr",
"name": "ACS 2016 1-year",
"years": "2016"
},
"tables": {
"B07007": {
"title": "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States",
"universe": "Population 1 Year and Over in the United States",
"denominator_column_id": "B07007001",
"columns": {
"B07007001": {
"name": "Total:",
"indent": 0
},
"B07007002": {
"name": "Native",
"indent": 2
},
"B07007003": {
"name": "Foreign born:",
"indent": 2
...
"B07007029": {
"name": "Naturalized U.S. citizen",
"indent": 3
},
"B07007030": {
"name": "Not a U.S. citizen",
"indent": 3
}
}
},
"B13016": {
"title": "Women 15 to 50 Years Who Had a Birth in the Past 12 Months by Age",
"universe": "Women 15 to 50 Years",
"denominator_column_id": "B13016001",
"columns": {
"B13016001": {
"name": "Total:",
"indent": 0
},
"B13016002": {
"name": "Women who had a birth in the past 12 months:",
"indent": 1
},
"B13016003": {
"name": "15 to 19 years old",
"indent": 2
},
...
"B13016016": {
"name": "40 to 44 years old",
"indent": 2
},
"B13016017": {
"name": "45 to 50 years old",
"indent": 2
}
}
}
},
"data": {
"04000US06": {
"B07007": {
"estimate": {
"B07007001": 38783436.0,
"B07007002": 28111036.0,
...
"B07007029": 17337.0,
"B07007030": 233086.0
},
"error": {
"B07007001": 10247.0,
"B07007002": 47145.0,
...
"B07007029": 2450.0,
"B07007030": 10773.0
}
},
"B13016": {
"estimate": {
"B13016001": 9646626.0,
"B13016002": 471829.0,
...
"B13016016": 12733.0,
"B13016017": 8178.0
}
}
},
"04000US28": {
"B07007": {
"estimate": {
"B07007001": 2951081.0,
...
"B07007030": 1631.0
},
"error": {
"B07007001": 3274.0,
...
"B07007030": 556.0
}
},
"B13016": {
"estimate": {
"B13016001": 714223.0,
...
"B13016017": 111081.0
},
"error": {
"B13016001": 5643.0,
...
"B13016017": 3006.0
}
}
}
},
"geography": {
"04000US06": {
"name": "California"
},
"04000US28": {
"name": "Mississippi"
}
}
}
In Depth Examples
In-depth examples contained in the census-data repo: https://git.charlesreid1.com/charlesreid1/census-data
This parses data for two data repos:
maps (geo data for maps): https://git.charlesreid1.com/data/maps
census (table data for maps): https://git.charlesreid1.com/data/census
Geo Examples
The following script populates a directory with one GeoJSON file per state, for all 50 states, and puts GeoJSON information for each county in the state's file: https://git.charlesreid1.com/charlesreid1/census-data/src/master/geo/populate_us_county.py
Start with the first function:
def main():
fipscodes = get_fipscodes()
for k in fipscodes:
v = fipscodes[k]
if(len(k)==2):
print(k)
fips_download(k,v)
Fips code json file can be constructed using this script: https://git.charlesreid1.com/charlesreid1/census-data/src/master/fips/get_fips_codes.py
This takes a given fips code, for a given state, and calls the censusreporter.org API endpoint to get county information for a given state.
Table Examples
This script populates census data in JSON format for a given group of geographic entities