The 2018 data project is an ongoing effort to figure out how to set up "painless" dashboards for monitoring metrics.
Stage 1: Collecting System Data
Stage 2: Spy
- lightweight server (may want larger disk, okay if non-free)
- running mongodb
- running mongoexpress
- running prometheus
- running grafana
- running netdata
additional components before real world testing:
- netdata on the build node
- netdata python plugin from another process, monitoring.....???
- is snakemake running (binary yes/no)
- current stage of snakemake (adjust snakemake file to write into a dotfile)
- cpu/memory/network/disk io
netdata python plugin workflow?
- does it need to be installed and netdata restarted, or can it push data into netdata?
- get a yeti node
- debug the snakemake file one step at a time using already-downloaded files (faster step)
- let the snakemake file run with netdata and friends running