Collectd

Collectd is similar to Netdata in that it is a logging program that collects system metrics. It is implemented as a unix-style daemon, with lots of plugins.

Linode uses collectd to provide users with system status dashboards.

Link: https://collectd.org/

installing

debian

binary

install with aptitude:

apt-get install collectd

To install plugins:

apt-get install collectd-utils

source

Installing from source is required if you want any/all the plugins.

To install from source: https://git.charlesreid1.com/data/collectd

The script to do everything: https://git.charlesreid1.com/data/collectd/src/master/doit.sh

Dependencies:

sudo apt-get install -y libbson-1.0-0 libbson-dev
sudo apt-get install -y libmongoc-1.0-0 libmongoc-dev
sudo apt-get install -y libgcrypt20-dev
sudo apt-get install -y libgpg-error-dev
sudo apt-get install -y byacc
sudo apt-get install -y bison libbison-dev 
sudo apt-get install -y cloog-ppl libcloog-ppl1 libppl1c4 libppl13v5
sudo apt-get install -y libgmpxx4ldbl
sudo apt-get install -y flex
sudo apt-get install -y libfl-dev

Now run the included build script to create a configure script, then configure:

./build.sh

./configure \
    --enable-cpu --enable-cpufreq --enable-cpusleep \
    --enable-csv --enable-df --enable-disk \
    --enable-ethstat --enable-filecount \
    --enable-load --enable-logfile \
    --enable-memory \
    --enable-network \
    --enable-python \
    --enable-processes \
    --enable-swap \
    --enable-syslog \
    --enable-tail \
    --enable-write_graphite \
    --enable-write_mongodb \
    --enable-write_log 

The final steps are to make and make install:

make

make install

This will install into /opt/collectd. It is helpful to change ownership to the regular user you are running the script as.

mac

install with homebrew:

brew install collectd

openwrt

opkg install collectd

config and usage

As you can deduce from the name, collectd is a daemon, so it runs as a background process.

The way you use it is to edit the configuration file, set up any plugins, and then let the daemon service run.

config

Link: https://collectd.org/wiki/index.php/First_steps#Configuration

Edit the config file at /etc/collectd.conf or /etc/collectd/collectd.conf

If you installed from source, it will be at /opt/collectd/etc/collectd.conf

Each plugin (see below) has a LoadPlugin line in the configuration. Plugins are bits of code that tell collectd how to collect various metrics.

Most plugins are commented out to keep things lean. The number of comment characters is significant.

• Two comment characters ## mean the plugin has not been built
• One comment character means the plugin has been built
• Default enabled plugins: cpu, interface, load, memory

The config file sets all kinds of parameters. The code for the plugin will parse these parameters when they run, and use those to collect data and send it back to collectd.

plugins

when talking about plugins, we must distinguish between two kinds of plugins: collect plugins, and write plugins.

collect plugins provide collectd with a way to ask a particular server or piece of software for statistics. This type of plugin allows collectd to monitor that software.

write plugins provide collectd with a way to write its data to a particular piece of software. This type of plugin allows collectd to use a particular piece of software as a backend.

as an example:

• the regular mongodb plugin at https://collectd.org/wiki/index.php/Plugin:MongoDB allows collectd to monitor a mongodb server
• the write mongodb plugin at https://collectd.org/wiki/index.php/Plugin:Write_MongoDB allows collectd to write its collected data to a mongodb database

write plugins

mongodb

Main article: Collectd/Mongo

The write plugin for mongodb allows collectd to write to a mongodb database.

Link: https://collectd.org/wiki/index.php/Plugin:Write_MongoDB

graphite

Main article: Graphite

Link to collectd wiki page: https://collectd.org/wiki/index.php/Plugin:Write_Graphite

Here is the section of collectd.conf that contains the directives for graphite. Note that several parameters included by default will raise errors; these are commented out:

LoadPlugin write_graphite

<Plugin write_graphite>
	<Node "default">
		Host "localhost"
		Port "2003"
		Protocol "tcp"
		#ReconnectInterval 0
		LogSendErrors true
		Prefix "collectd"
		Postfix "collectd"
		StoreRates true
		AlwaysAppendDS false
		EscapeCharacter "_"
		SeparateInstances false
		#PreserveSeparator false
		#DropDuplicateFields false
	</Node>
</Plugin>

prometheus

https://collectd.org/wiki/index.php/Plugin:Write_Prometheus

collect plugins

Full table of plugins: https://collectd.org/wiki/index.php/Table_of_Plugins

Custom plugins: http://docs.rightscale.com/cm/rs101/create_custom_collectd_plug-ins_for_linux.html