Netdata/MongoDB/API

The Netdata url schema exposes all metrics being measured by Netdata as a JSON-exportable REST url.

Notes

Querying Netdata API with Python

To obtain the data that Netdata is reading, then, is a simple matter of making a URL request and translating the result into JSON. This is a breeze with the Python 3 requests library:

import requests, json
my_url = 'http://10.6.0.1:19999/api/v1/allmetrics?format=json&help=yes'
r = requests.get(url=my_url)

# dump resulting json
with open('output.json','w') as f:
    json.dump( r.json(), f, indent=4 )

# print resulting json
print(r.json())

This displays a huge dictionary full of key-value pairs - all the quantities netdata is monitoring.

At this point, the data can be inserted into the database, or it can be parsed to extract particular quantities of interest. Each key has a timestamp associated with it, in Unix epoch format (e.g., 1518321718).

$ head -n30 output.json
{
    "ipv4.tcpofo": {
        "name": "ipv4.tcpofo",
        "context": "ipv4.tcpofo",
        "units": "packets/s",
        "last_updated": 1518321718,
        "dimensions": {
            "TCPOFOQueue": {
                "name": "inqueue",
                "value": 0.0
            },
            "TCPOFODrop": {
                "name": "dropped",
                "value": 0.0
            },
            "TCPOFOMerge": {
                "name": "merged",
                "value": 0.0
            },
            "OfoPruned": {
                "name": "pruned",
                "value": 0.0
            }
        }
    },
    "cgroup_happy_mongo.merged_ops": {
        "name": "cgroup_happy_mongo.merged_ops",
        "context": "cgroup.merged_ops",
        "units": "operations/s",
        "last_updated": 1518321718,

Parsing Netdata Output

In [1]: import requests, json

In [2]: my_url = 'http://10.6.0.1:19999/api/v1/allmetrics?format=json&help=yes'
   ...:

In [3]: r = requests.get(url=my_url)

In [4]: with open('output.json','w') as f:
   ...:         json.dump( r.json(), f, indent=4 )
   ...:

In [5]: d = r.json()

In [6]: print(len(d.keys()))
248

In [7]: print(d.keys())
dict_keys(['ipv4.tcpofo', 'cgroup_happy_mongo.merged_ops', 'cgroup_mex.merged_ops', 'cgroup_mex.throttle_serviced_ops', 'cgroup_mex.throttle_io', 'cgroup_mex.net_packets_eth0', 'cgroup_mex.serviced_ops', 'cgroup_mex.net_eth0', 'cgroup_mex.io', 'cgroup_mex.mem_usage', 'cgroup_mex.pgfaults', 'cgroup_mex.mem_activity', 'cgroup_mex.writeback', 'cgroup_mex.mem', 'cgroup_mex.cpu_per_core', 'cgroup_mex.cpu', 'cgroup_happy_mongo.queued_ops', 'cgroup_happy_mongo.throttle_serviced_ops', 'cgroup_happy_mongo.throttle_io', 'cgroup_happy_mongo.serviced_ops', 'cgroup_happy_mongo.io', 'cgroup_happy_mongo.mem_usage', 'cgroup_happy_mongo.pgfaults', 'cgroup_happy_mongo.mem_activity', 'cgroup_happy_mongo.net_packets_eth0', 'cgroup_happy_mongo.writeback', 'cgroup_happy_mongo.net_eth0', 'cgroup_happy_mongo.mem', 'cgroup_happy_mongo.cpu_per_core', 'cgroup_happy_mongo.cpu', 'ipv4.sockstat_tcp_mem', 'net_packets.docker0', 'net.docker0', 'sensors.coretemp-isa-0000_temperature', 'cpu.cpu1_cpuidle', 'cpu.cpu0_cpuidle', 'cpu.cpufreq', 'netdata.runtime_sensors', 'netdata.runtime_cpuidle', 'netdata.runtime_cpufreq', 'disk_svctm.dm-1', 'disk_avgsz.dm-1', 'disk_await.dm-1', 'disk_svctm.dm-0', 'disk_avgsz.dm-0', 'disk_await.dm-0', 'disk_svctm.sda', 'disk_avgsz.sda', 'disk_await.sda', 'groups.pipes', 'groups.sockets', 'groups.files', 'netdata.compression_ratio', 'netdata.response_time', 'groups.lwrites', 'groups.lreads', 'netdata.net', 'groups.pwrites', 'netdata.requests', 'netdata.clients', 'netdata.server_cpu', 'netdata.plugin_proc_cpu', 'groups.preads', 'groups.minor_faults', 'netdata.plugin_proc_modules', 'system.ipc_semaphore_arrays', 'system.ipc_semaphores', 'groups.major_faults', 'system.io', 'disk_iotime.dm-1', 'groups.cpu_system', 'disk_util.dm-1', 'groups.cpu_user', 'disk_backlog.dm-1', 'disk_ops.dm-1', 'disk.dm-1', 'disk_iotime.dm-0', 'groups.processes', 'disk_util.dm-0', 'disk_backlog.dm-0', 'groups.threads', 'disk_qops.dm-0', 'disk_ops.dm-0', 'disk.dm-0', 'disk_iotime.sda', 'groups.vmem', 'disk_mops.sda', 'disk_util.sda', 'groups.mem', 'disk_backlog.sda', 'groups.cpu', 'disk_qops.sda', 'disk_ops.sda', 'disk.sda', 'netfilter.conntrack_sockets', 'cpu.cpu1_softnet_stat', 'users.pipes', 'cpu.cpu0_softnet_stat', 'system.softnet_stat', 'users.sockets', 'ipv6.ect', 'users.files', 'ipv6.icmptypes', 'ipv6.icmpmldv2', 'ipv6.icmpneighbor', 'ipv6.icmprouter', 'users.lwrites', 'users.lreads', 'ipv6.icmperrors', 'ipv6.icmp', 'users.pwrites', 'ipv6.mcastpkts', 'ipv6.mcast', 'users.preads', 'users.minor_faults', 'users.major_faults', 'ipv6.udperrors', 'ipv6.udppackets', 'users.cpu_system', 'ipv6.packets', 'system.ipv6', 'users.cpu_user', 'ipv4.udplite_errors', 'ipv4.udplite', 'users.processes', 'users.threads', 'ipv4.udperrors', 'users.vmem', 'ipv4.udppackets', 'ipv4.tcphandshake', 'users.mem', 'ipv4.tcpopens', 'users.cpu', 'ipv4.tcperrors', 'ipv4.tcppackets', 'ipv4.tcpsock', 'apps.pipes', 'apps.sockets', 'ipv4.icmpmsg', 'ipv4.icmp_errors', 'ipv4.icmp', 'ipv4.errors', 'ipv4.fragsin', 'ipv4.fragsout', 'apps.files', 'ipv4.packets', 'ipv4.ecnpkts', 'ipv4.bcastpkts', 'ipv4.mcastpkts', 'apps.lwrites', 'ipv4.bcast', 'ipv4.mcast', 'system.ipv4', 'ipv6.sockstat6_raw_sockets', 'ipv6.sockstat6_udp_sockets', 'ipv6.sockstat6_tcp_sockets', 'ipv4.sockstat_udp_mem', 'ipv4.sockstat_udp_sockets', 'ipv4.sockstat_tcp_sockets', 'apps.lreads', 'ipv4.sockstat_sockets', 'system.net', 'net_packets.wlx7cdd906c3ef0', 'net.wlx7cdd906c3ef0', 'net_packets.master', 'net.master', 'mem.slab', 'mem.kernel', 'apps.pwrites', 'mem.writeback', 'mem.committed', 'system.swap', 'apps.preads', 'mem.available', 'system.ram', 'mem.pgfaults', 'system.pgpgio', 'apps.minor_faults', 'cpu.cpu1_softirqs', 'cpu.cpu0_softirqs', 'apps.major_faults', 'system.softirqs', 'apps.cpu_system', 'cpu.cpu1_interrupts', 'apps.cpu_user', 'apps.processes', 'cpu.cpu0_interrupts', 'apps.threads', 'services.merged_io_ops_write', 'services.merged_io_ops_read', 'services.queued_io_ops_write', 'services.queued_io_ops_read', 'services.throttle_io_ops_write', 'services.throttle_io_ops_read', 'services.throttle_io_write', 'services.throttle_io_read', 'services.io_ops_write', 'services.io_ops_read', 'services.io_write', 'services.io_read', 'services.mem_usage', 'services.cpu', 'netdata.plugin_diskspace_dt', 'netdata.plugin_diskspace', 'system.interrupts', 'apps.vmem', 'system.entropy', 'disk_inodes._boot', 'system.active_processes', 'disk_space._boot', 'system.load', 'disk_inodes._run_lock', 'system.uptime', 'disk_space._run_lock', 'apps.mem', 'cpu.core_throttling', 'disk_inodes._dev_shm', 'system.processes', 'system.forks', 'system.ctxt', 'disk_space._dev_shm', 'system.intr', 'netdata.private_charts', 'disk_inodes._', 'apps.cpu', 'netdata.tcp_connected', 'disk_space._', 'netdata.apps_children_fix', 'cpu.cpu1', 'netdata.tcp_connects', 'disk_inodes._run', 'netdata.plugin_tc_time', 'netdata.apps_fix', 'netdata.statsd_packets', 'disk_space._run', 'cpu.cpu0', 'netdata.plugin_tc_cpu', 'netdata.statsd_bytes', 'netdata.apps_sizes', 'disk_inodes._dev', 'netdata.statsd_reads', 'netdata.apps_cpu', 'disk_space._dev', 'system.cpu', 'netdata.plugin_cgroups_cpu', 'netdata.statsd_events', 'netdata.statsd_metrics', 'system.idlejitter'])

If we insert this dictionary directly into the MongoDB database, this list of keys will become the columns, and each row will become an observation. These observations are nested JSON objects/dictionaries themselves. Here's what one (example) observation looks like:

In [8]: d['cpu.cpu1']
Out[8]:
{'context': 'cpu.cpu',
 'dimensions': {'guest': {'name': 'guest', 'value': 0.0},
  'guest_nice': {'name': 'guest_nice', 'value': 0.0},
  'idle': {'name': 'idle', 'value': 98.989899},
  'iowait': {'name': 'iowait', 'value': 0.0},
  'irq': {'name': 'irq', 'value': 0.0},
  'nice': {'name': 'nice', 'value': 0.0},
  'softirq': {'name': 'softirq', 'value': 0.0},
  'steal': {'name': 'steal', 'value': 0.0},
  'system': {'name': 'system', 'value': 1.010101},
  'user': {'name': 'user', 'value': 0.0}},
 'last_updated': 1518323931,
 'name': 'cpu.cpu1',
 'units': 'percentage'}

In [9]: d['ipv4.sockstat_tcp_sockets']
Out[9]:
{'context': 'ipv4.sockstat_tcp_sockets',
 'dimensions': {'alloc': {'name': 'alloc', 'value': 26.0},
  'inuse': {'name': 'inuse', 'value': 11.0},
  'orphan': {'name': 'orphan', 'value': 0.0},
  'timewait': {'name': 'timewait', 'value': 0.0}},
 'last_updated': 1518323931,
 'name': 'ipv4.sockstat_tcp_sockets',
 'units': 'sockets'}