2018/Data Project: Difference between revisions

From charlesreid1

(Created page with "=Project Overview= The 2018 data project is an ongoing effort to figure out how to set up "painless" dashboards. First, we set up Netdata to dump to a Prometheus dat...")
 
Line 8: Line 8:


We then set up [[MongoDB]] and [[MongoExpress]] in Docker containers, and have MongoDB listening over the VPN for incoming data. We used this to scrape information from the charlesreid1 wiki.
We then set up [[MongoDB]] and [[MongoExpress]] in Docker containers, and have MongoDB listening over the VPN for incoming data. We used this to scrape information from the charlesreid1 wiki.
* [[MongoDB]]
* [[MongoExpress]]
* [[Pywikibot]]
* Link to MongoDB docker files: https://charlesreid1.com:3000/docker/d-mongodb
* Link to MongoDB docker files: https://charlesreid1.com:3000/docker/d-mongodb
* Link to MongoExpress docker files: https://charlesreid1.com:3000/docker/d-mongoexpress
* Link to MongoExpress docker files: https://charlesreid1.com:3000/docker/d-mongoexpress
* Link to wiki scraping scripts: https://charlesreid1.com:3000/wiki/charlesreid1-wiki-data
* Link to wiki scraping scripts: https://charlesreid1.com:3000/wiki/charlesreid1-wiki-data

Revision as of 20:32, 10 February 2018

Project Overview

The 2018 data project is an ongoing effort to figure out how to set up "painless" dashboards.

First, we set up Netdata to dump to a Prometheus database. That was fine, but not very transparent. Netdata has a fantastic dashboard with all kinds of stuff all ready to go, but it's custom-built for monitoring compute nodes, and not for general visualization. Prometheus was not a particularly outstanding tool, don't know much about how to use it.

We then set up MongoDB and MongoExpress in Docker containers, and have MongoDB listening over the VPN for incoming data. We used this to scrape information from the charlesreid1 wiki.

Retrieved from "https://charlesreid1.com/w/index.php?title=2018/Data_Project&oldid=24120"