D3 Map
From charlesreid1
Contents
Installing
First step was prerequisites.
Prerequisites
I had to install GDAL (software for converting different geographic formats) and NPM (node package manager):
brew install gdal brew install npm
Next, I used npm to install TopoJSON:
npm -g topojson
Then I checked that the installations went ok with:
which ogr2ogr which topojson
Converting
GeoJson to TopoJson
To convert a GeoJson to a TopoJson and preserve properties:
topojson in.geojson -o out.json -p
Once I converted my GeoJson into TopoJson I was ready to start creating a map from it in D3.
My First D3 Map
I started with a basic map tutorial from Mike Bostock, creator of D3: http://bost.ocks.org/mike/map/
Initially I wasn't able to get it working, but it was because I was using GeoJson. Switching to TopoJson helped me make sense of some of the Javascript - there's a lot that's happening implicitly even in the simplest D3 examples. Opening the TopoJson to understand its structure and keys, how properties were stored in it, made the D3 more clear.
Here is the final map created for the tutorial: http://bost.ocks.org/mike/map/step-7.html
The main point that gave me trouble was figuring out the projection. With Leaflet, I can pass latitude and longitude to center the map on my region of interest. But this wasn't possible (or, more likely, I couldn't figure out how to do it) with D3.
I ended up using a US projection, which led to this funny rotation of California, out there on the left coast:
When I tried rotating or transforming it, it would disappear for any setting more than very tiny numbers, and it is really inefficient to position a map like you draw with an etch-a-sketch. Need to figure out how to use latitude and longitude. And maybe how to add tiles?
Step 1: The Data
First, I was using a GeoJson file from the US Census. This encoded some data about method of commuting to work by poverty status. I wanted to test whether I could preserve properties encoded in the GeoJson when I converted it to TopoJson, since the whole point of using D3 is to take advantage of its data-crunching capabilities.
When I ran the topojson command to convert GeoJson to TopoJson, I used the -p flag.
Step 2: The D3 Javascript
I began with specifying information about the canvas, projection, and setup of D3:
var width = 800,
height = 600;
var projection = d3.geo.albersUsa()
.scale(1400)
.translate([width*3/4, height/2]);
var path = d3.geo.path()
.projection(projection)
.pointRadius(2);
// This is inserted as <body> is being constructed,
// so <svg> occurs wherever this .js file is called.
var svg = d3.select("body").append("svg")
.attr("width", width)
.attr("height", height);
Next, we iterate over each TopoJson feature and extract properties. We use the geoid property to create a unique style for each California county. If we had multiple states, we could give each state's counties unique CSS classes. This would ease changing styles on a state-by-state basis. I've also encoded the geoid, which is unique to that county, so that we can map colors to data, or pick out one particular county (as with San Bernadino County in the D3 example on a-shrubbery).
var prefix = "/";
// this Json file is TopoJson:
d3.json(prefix+"d3basicmap.json", function(error, ca) {
//console.log(ca);
var subunits = topojson.feature(ca, ca.objects.d3basicmap);
console.log(subunits);
svg.selectAll(".subunit")
.data(subunits.features)
.enter().append("path")
.attr("class", function(d) { return "subunit subunit" + d.properties.geoid; })
.attr("d", path);
});
This can all be found on Github: https://github.com/charlesreid1/a-shrubbery/blob/master/pelican/maps/d3basicmap.js
Step 3: D3 HTML
I added all of this to an HTML document as follows.
The script above appends an <svg> tag to the <body> tag wherever I place the <script>.
I add the following to an HTML document, wherever I want my D3 content to go:
<script src="http://d3js.org/d3.v3.min.js"></script>
<script src="http://d3js.org/topojson.v1.min.js"></script>
<script src="{ { SITEURL } }/d3basicmap.js" type="text/javascript"></script>
where <code>{{ SITEURL }}</code> is Jinja template markup, part of using Pelican to manage all the maps on the page.
Pelican/Jinja template on Github here: https://github.com/charlesreid1/a-shrubbery/blob/master/pelican/maps/d3basicmap.html
Workflow: D3 for Topo Maps
If this works, it'll be great:
http://stackoverflow.com/questions/18300527/d3js-how-to-design-topographic-maps
I'm going to test it out on visualizing contours of the Barry Goldwater bombing range in southwest Arizona, using data obtained from the National Map Viewer.
Requirements
The page lists the following requirements:
- make
- curl
- unzip
- gdal
- nodejs
- topojson
- ogr
I used homebrew to install most of these...
Creating make file
Next step was to dump all the shapefiles in a working directory, and create a makefile there:
$ ls barrygoldwater/ Elev_Contour.dbf Elev_Contour.prj Elev_Contour.shp Elev_Contour.shx topo.mk
My makefile was significantly simpler than the one given in the example, mainly because I was dealing with a single shapefile and not doing all of the tif contour rastering nonsense. Here's my topo.mk:
# topojsoning:
final.json: Elev_Contour.json
topojson --id-property none --simplify=0.5 -p -o barrygoldwater_new.json -- Elev_Contour.json
# simplification approach to explore further. Feedbacks welcome.
# shp2jsoning:
Elev_Contour.json: Elev_Contour.shp
ogr2ogr -f GeoJSON Elev_Contour.json Elev_Contour.shp
clean:
rm `ls | grep -v 'zip' | grep -v 'Makefile'`
# Makefile v4b (@Lopez_lz)
Don't forget, in Vim you can use Control + T in insert mode to insert hard tabs, just in case you've told vim to automatically use spaces everywhere it sees tabs. Makefiles MUST use tabs, or else they'll throw errors. No four-spaces allowed!
I ran into a problem with the ogr2ogr command:
$ make -f ./topo.mk ogr2ogr -f GeoJSON Elev_Contour.json Elev_Contour.shp dyld: Library not loaded: /usr/local/lib/libproj.0.dylib Referenced from: /usr/local/bin/ogr2ogr Reason: image not found make: *** [Elev_Contour.json] Trace/BPT trap: 5
This ended up being a problem with the brew installed version of GDAL (see this issue on Github: [1]).
I fixed it with:
$ brew uninstall proj $ brew install proj Xcode can be updated from the App Store. ==> Downloading https://downloads.sf.net/project/machomebrew/Bottles/proj-4.8.0.mountain_lion.bottle.tar.gz Already downloaded: /Library/Caches/Homebrew/proj-4.8.0.mountain_lion.bottle.tar.gz ==> Pouring proj-4.8.0.mountain_lion.bottle.tar.gz Error: The `brew link` step did not complete successfully The formula built, but is not symlinked into /usr/local You can try again using: brew link proj $ brew link proj Linking /usr/local/Cellar/proj/4.8.0... Error: Could not symlink bin/proj Target /usr/local/bin/proj already exists. You may want to remove it: rm /usr/local/bin/proj To force the link and overwrite all conflicting files: brew link --overwrite proj To list all files that would be deleted: brew link --overwrite --dry-run proj $ brew link --overwrite proj Linking /usr/local/Cellar/proj/4.8.0... 42 symlinks created
Hooray! Now it will work:
$ make -f ./topo.mk ogr2ogr -f GeoJSON Elev_Contour.json Elev_Contour.shp topojson --id-property none --simplify=0.5 -p -o barrygoldwater_new.json -- Elev_Contour.json bounds: -114.0000000001562 32 -112.99999999962512 32.99999999981907 (spherical) pre-quantization: 0.111m (0.00000100°) 0.111m (0.00000100°) topology: 34880 arcs, 4083014 points post-quantization: 11.1m (0.000100°) 11.1m (0.000100°) simplification: retained 69760 / 3702887 points (2%) prune: retained 34880 / 34880 arcs (100%)
Init D3 HTML Page
Now time for making the D3 HTML page.
I copied and pasted most of what the example gave, with a few small modifications for my own data. Here's the final HTML file, barrygoldwater.html:
<style>
svg { border: 5px solid #333; background-color: #C6ECFF;}
/* TOPO */
path.Topo_1 { fill:#ACD0A5; stroke: #0978AB; stroke-width: 1px; }
path.Topo_50 {fill: #94BF8B; }
path.Topo_100 {fill: #BDCC96; }
path.Topo_200 {fill: #E1E4B5; }
path.Topo_500 {fill: #DED6A3; }
path.Topo_1000 {fill:#CAB982 ; }
path.Topo_2000 {fill: #AA8753; }
path.Topo_3000 {fill: #BAAE9A; }
path.Topo_4000 {fill: #E0DED8 ; }
path.Topo_5000 {fill: #FFFFFF ; }
.download {
background: #333;
color: #FFF;
font-weight: 900;
border: 2px solid #B10000;
padding: 4px;
margin:4px;
}
</style>
<body>
<script src="http://code.jquery.com/jquery-2.0.2.min.js"></script>
<script src="http://d3js.org/d3.v3.min.js"></script>
<script src="http://d3js.org/topojson.v1.min.js"></script>
<script>
// 1. -------------- SETTINGS ------------- //
// Geo-frame_borders in decimal degrees
var WNES = { "W": 32.0, "N":-113.0, "E": 33.0, "S": -114.0 };
// Geo values of interest :
var latCenter = (WNES.S + WNES.N)/2,
lonCenter = (WNES.W + WNES.E)/2,
geo_width = (WNES.E - WNES.W),
geo_height= (WNES.N - WNES.S);
// HTML expected frame dimensions
var width = 600,
height = width * (geo_height / geo_width);
// Projection: projection, reset scale and translate
var projection = d3.geo.equirectangular()
.scale(1)
.translate([0, 0]);
// SVG injection:
var svg = d3.select("body").append("svg")
.attr("width", width)
.attr("height", height);
// Path
var path = d3.geo.path()
.projection(projection)
.pointRadius(4);
// Data (getJSON: TopoJSON)
d3.json("barrygoldwater_new.json", showData);
// 2. ---------- FUNCTION ------------- //
function showData(error, contours) {
var contour = topojson.feature(contours, contours.objects.Elev_Contour);
var onecontour = contour.features.filter(function(d) {
return d.properties.CONTOURELE === 1840;
});
// Focus area box compute for derive scale & translate.
// [[left, bottom], [right, top]] // E W N S
var b = path.bounds(contour[0]),
s = 1 / Math.max((b[1][0] - b[0][0]) / width, (b[1][1] - b[0][1]) / height),
t = [(width - s * (b[1][0] + b[0][0])) / 2,
(height - s * (b[1][1] + b[0][1])) / 2];
console.log("bounds = "+b);
// Projection update
projection
.scale(s)
.translate(t);
console.log("Scale factor = "+s);
// black border around map
svg.append("rect").attr('width', width).attr('height', height)
.style('stroke', 'black').style('fill', 'none');
svg.selectAll("path")
.data(contour.features)
.enter().append("path")
.attr("d",path)
.attr("class", function(d) { return "contour contour" + d.properties.CONTOURELE; })
.style("stroke-width", "1")
.style("stroke", "black")
}
</script>
<br />
<div>
<a class="download ac-icon-download" href="javascript:javascript: (function () { var e = document.createElement('script'); if (window.location.protocol === 'https:') { e.setAttribute('src', 'https://raw.github.com/NYTimes/svg-crowbar/gh-pages/svg-crowbar.js'); } else { e.setAttribute('src', 'http://nytimes.github.com/svg-crowbar/svg-crowbar.js'); } e.setAttribute('class', 'svg-crowbar'); document.body.appendChild(e); })();"><!--⤋--><big>⇩</big> Download</a> -- Works on Chrome. Feedback me for others web browsers ?
</div>
<br />
</body>
</html>
This failed. I kept seeing errors about the path elements, as well as problems with the computed bounds and scales:
I finally ended up, once again, adjusting the whole map by hand to get the view centered on the interesting data. Only problem was, it wasn't that interesting...:
