MongoDB

The short version:

Stop wasting your time with these idiotic esoteric database configurations where 99% of it works perfectly but the one critical component (authentication) has some sloppy documentation.

Just use docker.

https://charlesreid1.com:3000/docker/d-mongodb

https://charlesreid1.com:3000/docker/d-mongoexpress

Summary

The brief summary:

• MongoDB provides a nosql unstructured data store for arbitrarily complicated json structures
• Listens on port 27017
• Install from mongodb.org debian repos
• Config handles file paths, logging, security, networking
• Multiple ways to interface (command line shell in Javascript, or via language bindings)
• Users must be created per-database, or a system-wide admin account added
• Enable user access controls, expose to private management LAN interfaces

Setting Up

Installing

Debian/Ubuntu

MongoDB provides instructions for installing on Debian/Ubuntu. The short version: don't do apt-get install mongodb.

Here's what you do:

• Add the mongodb aptitude repositories to your aptitude
• Update your aptitude
• Install a mongodb package from mongodb.org

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2930ADAE8CAF5059EE73BB4B58712A2291FA4AD5
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.6 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list
sudo apt-get update
sudo apt-get install -y mongodb-org

These assume you have ubuntu xenial, see link [1] for other LTS releases.

Fixing Stupid Issues

On Ubuntu there is a stupid issue with the startup service - a mistake.

Edit the file /lib/systemd/system/mongod.service

Change the line

ExecStart=/usr/bin/mongod -f /etc/mongod.conf

to the following (note the name of the conf file):

ExecStart=/usr/bin/mongod -f /etc/mongodb.conf

Re-load the service from the edited file by running:

sudo systemctl start mongod

Pretty stupid, ey?

Homebrew

Was able to install this ok with Homebrew: https://docs.mongodb.com/manual/tutorial/install-mongodb-on-os-x/#install-mongodb-community-edition-with-homebrew

brew update
brew install mongodb

or to install the development version:

brew update
brew install mongodb --devel

Configuring

Link to documentation page on config options: https://docs.mongodb.com/manual/reference/configuration-options/

By default, MongoDB will not require a config file, and if you don't specify one, it makes some weird decisions.

To start mongodb with a specified config file, use the --config or -f options:

mongod --config /etc/mongod.conf
mongod -f /etc/mongod.conf

Core mongodb config sections:

• systemLog
• net

systemLog

# default:

systemLog:
  destination: file
  path: /var/log/mongodb/mongod.log
  logAppend: true

can also set verbosity (0-5):

systemLog:
    destination: file
    path: /var/log/mongodb/mongod.log
    verbosity: 2
    logRotate: rename

If using logrotate util, set logRotate: reopen

Can further customize log behavior for specific components (access, commands, etc.)

net

net:
    port: 27017
    bindIp: 10.0.0.1
    ipv6: True

There are also several options for SSL. Those go into an ssl subsection of the net section of the config file.

Starting

MongoDB can be started with systemd, or using the init.d startup scripts. I hate systemd so I went with the latter.

Start by creating the directory where MongoDB will keep all of its data. For example, I used /opt/mongodb. Set the permissions so that the mongodb user/group can read/write to this directory:

sudo chown -R mongodb:mongodb mongodb/

Now start the service, which is defined in /etc/init.d/mongodb:

sudo service mongodb start

You can issue the status command in place of the start command to check if the process is running:

$ sudo service mongodb status
● mongodb.service - LSB: An object/document-oriented database
   Loaded: loaded (/etc/init.d/mongodb; bad; vendor preset: enabled)
   Active: active (running) since Tue 2018-01-30 16:59:10 PST; 1min 38s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 1962 ExecStart=/etc/init.d/mongodb start (code=exited, status=0/SUCCESS)
 Main PID: 13596 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/mongodb.service
           └─1973 /usr/bin/mongod --config /etc/mongodb.conf

Jan 30 16:59:09 jupiter systemd[1]: Starting LSB: An object/document-oriented database...
Jan 30 16:59:09 jupiter mongodb[1962]:  * Starting database mongodb
Jan 30 16:59:10 jupiter mongodb[1962]:    ...done.
Jan 30 16:59:10 jupiter systemd[1]: Started LSB: An object/document-oriented database.

You can also turn on logging, and look at the log files in /var/log/mongodb/mongodb.log

Create Users and Enable Authentication

It is a good idea to set up users and user authentication to control access to the data in the database.

To create a system-wide mongodb user admin, create a user with the role userAdminAnyDatabase (no other roles!).

Start the mongo shell from localhost (which will not require authentication to begin with) or using --noauth flag by using the mongo command:

$ mongo
> 

Now create a user for the admin database (run these commands directly in the shell). This will create a user "darthvader" with password "secretpass":

> use admin
> db.createUser(
  {
    user: "darthvader",
    pwd: "secretpass",
    roles: [ { role: "userAdminAnyDatabase", db: "admin" } ]
  }
)

Now you can enable client access control.

As per the mongodb documentation [2], to enable authorization you can either pass --auth when starting mongod or you can set security.authorization in the mongodb config file as follows:

security:
    authorization: enabled

List Users

Start up the mongo shell and tell it to use the admin database. Then use the show users command:

> use admin
> show users
{
	"_id" : "admin.admin",
	"user" : "admin",
	"db" : "admin",
	"roles" : [
		{
			"role" : "userAdminAnyDatabase",
			"db" : "admin"
		}
	]
}

Drop Users

To drop users:

> db.dropUser('admin')
true

Selecting an Interface

The first thing you have to decide before interacting with the database is how you want to interact.

The mongodb shell is a javascript shell that can be used from a command line on the mongodb server.

Mongodb also has python language bindings. there are multiple non-mongo-provided third party APIs and libraries too, so there are multiple options.

Basic CRUD Operations

MongoDB performs CRUD (create, read, update, delete) transactions/operations on the data that it stores.

Mongo/CRUD

Basic Collections Operations

Basic operations on collections:

Mongo/Collections

Basic Database Operations

Notes on basic database operations:

Mongo/Databases

Monitoring

MongoDB as a Monitoring Target

MongoDB has several mechanisms for monitoring the state of the database (per second operations, cache sizes, disk and memory usage, etc.)

Utilities like Netdata and Collectd have plugins written for MongoDB that can collect this information as part of scraping the system status.

MongoDB as a Monitoring Data Store

collectd has a Write_MongoDB plugin to allow collectd to write its data to MongoDB.

Plugin link: https://collectd.org/wiki/index.php/Plugin:Write_MongoDB

APIs

Python API: Pymongo

Java API: MongoDB/Java

References

pymodm: https://pymodm.readthedocs.io/en/latest/getting-started.html

Database design patterns: https://docs.mongodb.com/manual/applications/data-models/

Cheat sheet: https://blog.codecentric.de/files/2012/12/MongoDB-CheatSheet-v1_0.pdf

Related Page

• MongoExpress