From charlesreid1

Project Overview

The 2018 data project is an ongoing effort to figure out how to set up "painless" dashboards for monitoring metrics.

Stage 1: Collecting System Data

See 2018/Data Project/Stage 1

Stage 2: Spy

dahak-spy project:

  • lightweight server (may want larger disk, okay if non-free)
  • running mongodb
  • running mongoexpress
  • running prometheus
  • running grafana
  • running netdata

additional components before real world testing:

  • netdata on the build node
  • netdata python plugin from another process, monitoring.....???
  • metrics:
    • is snakemake running (binary yes/no)
    • current stage of snakemake (adjust snakemake file to write into a dotfile)
    • cpu/memory/network/disk io

netdata python plugin workflow?

  • does it need to be installed and netdata restarted, or can it push data into netdata?

real yeti:

  • get a yeti node
  • debug the snakemake file one step at a time using already-downloaded files (faster step)
  • let the snakemake file run with netdata and friends running