Revision as of 13:43, 21 February 2018

This page covers the use of the census reporter API to obtain census data: https://github.com/censusreporter/censusreporter

Note that the Census Map page also contains some similar information, but focuses on drawing maps with the resulting data. This page focuses specifically on the API and extracting data from the API.

Overview

Let's start with what we're obtaining from the API and what we need to draw a map.

The census API provides a gateway to census data, which is essentially a very large number of tables of data (the columns), broken down into components and sub-components, for hundreds of thousands of geographical entities (the rows). We can think of each piece of data about a particular location as a cell in a giant spreadsheet.

Both the data and the geographical entities are organized hierarchically. For example, a single table about poverty might have poverty statics broken down by demographics or education level, resulting in several dozen columns. The geographical entities are likewise organized in a hierarchy including the country, states, counties, and census tracts.

Geographical Information: References

Starting with some references:

The Census Reporter Geography page (https://censusreporter.org/topics/geography/) contains a list of codes for geographical hierarchies:

010 - United States
040 - States
050 - Counties
140 - Census tracts
310 - Metropolitan and micropolitan areas (cities)
500 - Congressional district
860 - Zip codes

The Missouri Census Data Center's Geography page has a much more extensive overview:

codes: https://factfinder.census.gov/help/en/summary_level_code_list.htm
hierarchies: http://mcdc.missouri.edu/geography/sumlevs/index.shtml

The official census documentation contains a page on geographic terms and concepts:

https://www.census.gov/geo/reference/gtc/gtc_cousub.html

Link to documentation chapter for each hierarchy level:

https://www.census.gov/geo/reference/garm.html

The GeoIDs of different states are called FIPS codes:

FIPS codes for each state are listed on page 16 of this PDF: https://www2.census.gov/geo/pdfs/reference/GARM/Ch4GARM.pdf
Small JSON file containing all 50 FIPS codes: https://raw.githubusercontent.com/charlesreid1/a-shrubbery/master/census/ca/fips.json

If you need the GeoID of a county or a city, you can also search. For example, to find the metropolitan area corresponding to New York City, you could pass the code for state level entity and pass the code for New York state, and ask for all child metropolitan areas. These could then be parsed and the names searched for a keyword, "New York City".

Geographical Information: API

Provide Parent, Get Children

This section covers how to provide a geographical entity at one level of the hierarchy and receive a set of children geographical entities at another level of the hierarchy.

When we call the geo API, we have a lot of ways to obtain information. We must specify the following:

What geographical hierarchy level do we want information about? (The final results will be aggregated by this hierarchy type)
What geographical entity ID are we passing in? (We have to specify some entity with some ID to limit the results, so we decide what geographical hierarchy level)
The specific ID of the geographical entity

The base API call looks like this:

https://api.censusreporter.org/1.0/geo/show/tiger2016?geo_ids=<level to return>|<level of geoid>00US<geoid>

Here is an example where we request geographical information about county-level boundaries. To limit the results, we specify that we will provide a geogrpahical entity that is a metropolitan-area code (310), then we provide the geoid of that entity (41860, corresponding to San Francisco)

https://api.censusreporter.org/1.0/geo/show/tiger2016?geo_ids=050|31000US41860
                                                             ^^^
                                                             050 return county level information

                                                                 ^^^
                                                                 310 use metro geoid that will be passed
                                                                     ^^ ^^ 
                                                                     00 US
                                                                         ^^^^^
                                                                         41860 your city metro ID (SF, in this case)

Provide/Receive Single Entity

If we wish to provide a single geographical entity and get its boundaries, we can do that by providing two pieces of information:

Geographical hierarchy level of entity you will specify
GeoID of entity you are specify

Then call the base API as follows:

curl "http://api.censusreporter.org/1.0/geo/tiger2016/<hierarchy level>00US<geoid>?geom=True

(Note that if you leave out the geom=True, you won't get any GeoJson shape information.)

For example, to get a geojson file containing the boundaries of Yuma County, you can call the API as follows:

curl "https://api.censusreporter.org/1.0/geo/tiger2016/05000US04027?geom=True" > yuma.geojson

here is the result:

{"geometry":{"type":"Polygon","coordinates":[[[-114.763998,32.643364],[-114.763512,32.645995],[-114.76495,32.649391],[-114.75831,32.655178],[-114.751079,32.659789],[-114.74948,32.66178],[-114.748,32.664184],[-114.747817,32.667777],[-114.745344,32.67219],[-114.744487,32.678683],[-114.730453,32.698844],[-114.72981,32.700282],[-114.730086,32.704298],[-114.72241,32.713597],[-114.717695,32.721547],[-114.714522,32.73039],[-114.709074,32.735456],[-114.705717,32.74158],[-114.702223,32.74541],[-114.69879,32.744846],[-114.68823,32.73753],[-114.682614,32.737348],[-114.665921,32.734028],[-114.654247,32.73357],[-114.635006,32.731372],[-114.617395,32.728257],[-114.61567,32.728454],[-114.615112,32.734515],[-114.581784,32.734946],[-114.581736,32.74232],[-114.564508,32.742274],[-114.564447,32.749554],[-114.539224,32.749812],[-114.539092,32.756949],[-114.526856,32.757094],[-114.528443,32.767276],[-114.532432,32.776922],[-114.531669,32.791185],[-114.529633,32.795477],[-114.522031,32.801675],[-114.515389,32.811439],[-114.510327,32.816488],[-114.494116,32.823287],[-114.468971,32.845155],[-114.465711,32.873681],[-114.462929,32.907944],[-114.464448,32.913128],[-114.47664,32.923628],[-114.479005,32.928291],[-114.480925,32.936276],[-114.478456,32.940555],[-114.470768,32.949424],[-114.468061,32.95526],[-114.467315,32.96406],[-114.468605,32.971649],[-114.470511,32.973858],[-114.476532,32.975174],[-114.480303,32.973748],[-114.481315,32.972064],[-114.488625,32.969946],[-114.490129,32.969884],[-114.492938,32.971781],[-114.495712,32.980075],[-114.499797,33.003905],[-114.501226,33.007556],[-114.507956,33.019708],[-114.516454,33.027617],[-114.371523,33.027489],[-114.370783,33.030012],[-114.268748,33.029815],[-114.269164,33.215034],[-114.268983,33.289532],[-114.268183,33.289429],[-114.267815,33.463574],[-113.958042,33.464655],[-113.958046,33.377578],[-113.434499,33.376879],[-113.33392,33.377425],[-113.334217,33.25004],[-113.333603,33.24763],[-113.333212,33.22721],[-113.334066,33.140282],[-113.333767,32.038607],[-113.78168,32.179034],[-114.250775,32.323909],[-114.813613,32.494276],[-114.813991,32.497231],[-114.812316,32.500054],[-114.813753,32.50426],[-114.816591,32.507696],[-114.814321,32.509023],[-114.807726,32.508726],[-114.804076,32.510375],[-114.802211,32.513191],[-114.802238,32.515206],[-114.80367,32.516374],[-114.809672,32.517567],[-114.810374,32.518391],[-114.810482,32.521758],[-114.813348,32.524186],[-114.802181,32.536414],[-114.802018,32.53946],[-114.804776,32.541659],[-114.80583,32.546354],[-114.803883,32.548001],[-114.795635,32.550956],[-114.793769,32.552329],[-114.791551,32.557023],[-114.792955,32.562085],[-114.792088,32.568497],[-114.793224,32.569459],[-114.794684,32.568703],[-114.795253,32.56662],[-114.79766,32.564444],[-114.801311,32.562865],[-114.80683,32.55888],[-114.810318,32.558628],[-114.812914,32.560049],[-114.813995,32.562201],[-114.813968,32.566209],[-114.812995,32.568706],[-114.804421,32.572941],[-114.801877,32.576009],[-114.801471,32.578255],[-114.803879,32.580889],[-114.803987,32.582652],[-114.800441,32.588079],[-114.799737,32.592177],[-114.801548,32.598591],[-114.805932,32.600721],[-114.807906,32.602783],[-114.807879,32.605416],[-114.809042,32.608806],[-114.809393,32.617119],[-114.80739,32.621332],[-114.799302,32.625115],[-114.791179,32.621833],[-114.787715,32.623573],[-114.782573,32.624304],[-114.781766,32.625613],[-114.782235,32.630215],[-114.780049,32.632725],[-114.768199,32.639874],[-114.763998,32.643364]]]},"type":"Feature","properties":{"awater":13171316,"display_name":"Yuma County, AZ","simple_name":"Yuma County","sumlevel":"050","population":203292,"full_geoid":"05000US04027","aland":14281265636}}

Census Information Tables: References

Please keep in mind that the tables get quite complicated.

Census reporter lists a large number of topics for data tables:

https://censusreporter.org/topics/table-codes/

An example of what a single table can contain:

https://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml?src=bkmk

Another table example: B07007: "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States":

https://censusreporter.org/tables/B07007/

Census Information Tables: API

Tabulation Endpoint

To get basic information about a table, use the tabulation endpoint:

https://api.censusreporter.org/1.0/tabulation/<table-id>

such as 07007. Example:

$ curl "https://api.censusreporter.org/1.0/tabulation/07007"
{"tabulation_code": "07007", "table_title": "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States", "tables_by_release": {"five_yr": ["B07007", "B07007PR"], "one_yr": ["B07007", "B07007PR"], "three_yr": []}, "universe": "Population 1 Year and Over in the United States", "topics": ["migration", "citizenship"], "subject_area": "Residence Last Year - Migration", "simple_table_title": "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States"}

And prettified:

{
    "tabulation_code": "07007",
    "table_title": "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States",
    "tables_by_release": {
        "five_yr": [
            "B07007",
            "B07007PR"
        ],
        "one_yr": [
            "B07007",
            "B07007PR"
        ],
        "three_yr": []
    },
    "universe": "Population 1 Year and Over in the United States",
    "topics": [
        "migration",
        "citizenship"
    ],
    "subject_area": "Residence Last Year - Migration",
    "simple_table_title": "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States"
}

=Table Endpoint

To get column information about a tale, use the table endpoint:

$ curl "https://api.censusreporter.org/1.0/table/<table-id-plus-letter>"

such as B07007. Example:

$ curl "https://api.censusreporter.org/1.0/table/B07007"
{"table_id": "B07007", "table_title": "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States", "simple_table_title": "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States", "subject_area": "Residence Last Year - Migration", "universe": "Population 1 Year and Over in the United States", "denominator_column_id": "B07007001", "topics": ["migration", "citizenship"], "columns": {"B07007001": {"column_title": "Total:", "indent": 0, "parent_column_id": null}, "B07007002": {"column_title": "Native", "indent": 2, "parent_column_id": "B07007001"}, "B07007003": {"column_title": "Foreign born:", "indent": 2, "parent_column_id": "B07007001"}, "B07007004": {"column_title": "Naturalized U.S. citizen", "indent": 3, "parent_column_id": "B07007003"}, "B07007005": {"column_title": "Not a U.S. citizen", "indent": 3, "parent_column_id": "B07007003"}, "B07007006": {"column_title": "Same house 1 year ago:", "indent": 1, "parent_column_id": "B07007001"}, "B07007007": {"column_title": "Native", "indent": 2, "parent_column_id": "B07007006"}, "B07007008": {"column_title": "Foreign born:", "indent": 2, "parent_column_id": "B07007006"}, "B07007009": {"column_title": "Naturalized U.S. citizen", "indent": 3, "parent_column_id": "B07007008"}, "B07007010": {"column_title": "Not a U.S. citizen", "indent": 3, "parent_column_id": "B07007008"}, "B07007011": {"column_title": "Moved within same county:", "indent": 1, "parent_column_id": "B07007001"}, "B07007012": {"column_title": "Native", "indent": 2, "parent_column_id": "B07007011"}, "B07007013": {"column_title": "Foreign born:", "indent": 2, "parent_column_id": "B07007011"}, "B07007014": {"column_title": "Naturalized U.S. citizen", "indent": 3, "parent_column_id": "B07007013"}, "B07007015": {"column_title": "Not a U.S. citizen", "indent": 3, "parent_column_id": "B07007013"}, "B07007016": {"column_title": "Moved from different county within same state:", "indent": 1, "parent_column_id": "B07007001"}, "B07007017": {"column_title": "Native", "indent": 2, "parent_column_id": "B07007016"}, "B07007018": {"column_title": "Foreign born:", "indent": 2, "parent_column_id": "B07007016"}, "B07007019": {"column_title": "Naturalized U.S. citizen", "indent": 3, "parent_column_id": "B07007018"}, "B07007020": {"column_title": "Not a U.S. citizen", "indent": 3, "parent_column_id": "B07007018"}, "B07007021": {"column_title": "Moved from different state:", "indent": 1, "parent_column_id": "B07007001"}, "B07007022": {"column_title": "Native", "indent": 2, "parent_column_id": "B07007021"}, "B07007023": {"column_title": "Foreign born:", "indent": 2, "parent_column_id": "B07007021"}, "B07007024": {"column_title": "Naturalized U.S. citizen", "indent": 3, "parent_column_id": "B07007023"}, "B07007025": {"column_title": "Not a U.S. citizen", "indent": 3, "parent_column_id": "B07007023"}, "B07007026": {"column_title": "Moved from abroad:", "indent": 1, "parent_column_id": "B07007001"}, "B07007027": {"column_title": "Native", "indent": 2, "parent_column_id": "B07007026"}, "B07007028": {"column_title": "Foreign born:", "indent": 2, "parent_column_id": "B07007026"}, "B07007029": {"column_title": "Naturalized U.S. citizen", "indent": 3, "parent_column_id": "B07007028"}, "B07007030": {"column_title": "Not a U.S. citizen", "indent": 3, "parent_column_id": "B07007028"}}}

Prettified:

{
    "table_id": "B07007",
    "table_title": "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States",
    "simple_table_title": "Geographical Mobility in the Past Year by Citizenship Status for Current Residence in the United States",
    "subject_area": "Residence Last Year - Migration",
    "universe": "Population 1 Year and Over in the United States",
    "denominator_column_id": "B07007001",
    "topics": [
        "migration",
        "citizenship"
    ],
    "columns": {
        "B07007001": {
            "column_title": "Total:",
            "indent": 0,
            "parent_column_id": null
        },
        "B07007002": {
            "column_title": "Native",
            "indent": 2,
            "parent_column_id": "B07007001"
        },
        "B07007003": {
            "column_title": "Foreign born:",
            "indent": 2,
            "parent_column_id": "B07007001"
        },
        "B07007004": {
            "column_title": "Naturalized U.S. citizen",
            "indent": 3,
            "parent_column_id": "B07007003"
        },
        "B07007005": {
            "column_title": "Not a U.S. citizen",
            "indent": 3,
            "parent_column_id": "B07007003"
        },
        "B07007006": {
            "column_title": "Same house 1 year ago:",
            "indent": 1,
            "parent_column_id": "B07007001"
        },
        "B07007007": {
            "column_title": "Native",
            "indent": 2,
            "parent_column_id": "B07007006"
        },
        "B07007008": {
            "column_title": "Foreign born:",
            "indent": 2,
            "parent_column_id": "B07007006"
        },
        "B07007009": {
            "column_title": "Naturalized U.S. citizen",
            "indent": 3,
            "parent_column_id": "B07007008"
        },
        "B07007010": {
            "column_title": "Not a U.S. citizen",
            "indent": 3,
            "parent_column_id": "B07007008"
        },
        "B07007011": {
            "column_title": "Moved within same county:",
            "indent": 1,
            "parent_column_id": "B07007001"
        },
        "B07007012": {
            "column_title": "Native",
            "indent": 2,
            "parent_column_id": "B07007011"
        },
        "B07007013": {
            "column_title": "Foreign born:",
            "indent": 2,
            "parent_column_id": "B07007011"
        },
        "B07007014": {
            "column_title": "Naturalized U.S. citizen",
            "indent": 3,
            "parent_column_id": "B07007013"
        },
        "B07007015": {
            "column_title": "Not a U.S. citizen",
            "indent": 3,
            "parent_column_id": "B07007013"
        },
        "B07007016": {
            "column_title": "Moved from different county within same state:",
            "indent": 1,
            "parent_column_id": "B07007001"
        },
        "B07007017": {
            "column_title": "Native",
            "indent": 2,
            "parent_column_id": "B07007016"
        },
        "B07007018": {
            "column_title": "Foreign born:",
            "indent": 2,
            "parent_column_id": "B07007016"
        },
        "B07007019": {
            "column_title": "Naturalized U.S. citizen",
            "indent": 3,
            "parent_column_id": "B07007018"
        },
        "B07007020": {
            "column_title": "Not a U.S. citizen",
            "indent": 3,
            "parent_column_id": "B07007018"
        },
        "B07007021": {
            "column_title": "Moved from different state:",
            "indent": 1,
            "parent_column_id": "B07007001"
        },
        "B07007022": {
            "column_title": "Native",
            "indent": 2,
            "parent_column_id": "B07007021"
        },
        "B07007023": {
            "column_title": "Foreign born:",
            "indent": 2,
            "parent_column_id": "B07007021"
        },
        "B07007024": {
            "column_title": "Naturalized U.S. citizen",
            "indent": 3,
            "parent_column_id": "B07007023"
        },
        "B07007025": {
            "column_title": "Not a U.S. citizen",
            "indent": 3,
            "parent_column_id": "B07007023"
        },
        "B07007026": {
            "column_title": "Moved from abroad:",
            "indent": 1,
            "parent_column_id": "B07007001"
        },
        "B07007027": {
            "column_title": "Native",
            "indent": 2,
            "parent_column_id": "B07007026"
        },
        "B07007028": {
            "column_title": "Foreign born:",
            "indent": 2,
            "parent_column_id": "B07007026"
        },
        "B07007029": {
            "column_title": "Naturalized U.S. citizen",
            "indent": 3,
            "parent_column_id": "B07007028"
        },
        "B07007030": {
            "column_title": "Not a U.S. citizen",
            "indent": 3,
            "parent_column_id": "B07007028"
        }
    }
}

Flags

Census Data: Difference between revisions

From charlesreid1