From charlesreid1

This page covers an Ansible playbook for a full stack example. This full stack example will run the following services:

  • Django web server
  • Celery task queue
  • RabbitMQ message queue
  • Postgresql for data storage

Setup

See Ansible/Full Stack Playbook/Vagrant Setup for details of setting up Vagrant to test out this configuration

Full Stack Playbook

Overview of Full Stack

Services

For this particular full stack deployment, we have the following services:

  • Django app + http server
  • Nginx web server
  • Celery task queue
  • RabbitMQ (Celery backend)
  • Postgres (persistent store)

Environments

We have three environments to deploy to:

  • vagrant (local testing)
  • staging (testing)
  • production

Architecture

Here is the architecture used for the above apps:

  • Web application is run on multiple hosts for better performance, with a load balancer in front
  • Task queue servers are run on multiple hosts for better performance
  • Celery, RabbitMQ, Postgres all on separate servers
  • 2 Postgres hosts - primary and replica

That is a total of 10 hosts (1 load balancer, 3 web servers, 3 task queues, 1 mq server, 2 database servers).

In the production environment: 10 separate hosts

In the staging environment: use fewer hosts (only 2 hosts: web server and task queue on one host, rabbitmq and postgres on another)

In the vagrant environment: use 3 servers (1 for web app, 1 for task queue, and 1 for postgres)

Ansible hosts file

Note that this initial hosts file (below) can be streamlined using suggestions from Ansible/Groups, Ansible/Group Variables, and Ansible/Host Naming, so consider it a first pass.

Using the information given above about the architecture, we have the following hosts file for our Django app:

[production]
delaware.example.com
georgia.example.com
maryland.example.com
newhampshire.example.com
newjersey.example.com
newyork.example.com
northcarolina.example.com
pennsylvania.example.com
rhodeisland.example.com
virginia.example.com

[staging]
redblue.example.com
orangered.example.com

[vagrant]
vagrant1 ansible_host=127.0.0.1 ansible_port=2222
vagrant2 ansible_host=127.0.0.1 ansible_port=2200
vagrant3 ansible_host=127.0.0.1 ansible_port=2201

[lb]
delaware.example.com

[web]
georgia.example.com
newhampshire.example.com
newjersey.example.com
redblue.example.com
vagrant1

[task]
newyork.example.com
northcarolina.example.com
maryland.example.com
redblue.example.com
vagrant2

[rabbitmq]
pennsylvania.example.com
orangered.example.com
vagrant3

[db]
rhodeisland.example.com
virginia.example.com
orangered.example.com
vagrant3

Note that we make the vagrant machines part of the vagrant group when their behavioral inventory parameters are specified, but we can later add them to other groups by adding their short name (only) to the group.

In this way, we've defined which machines are in each of the 3 environments (production, staging, vagrant) and which one runs which service.

Passing credentials via the host file

Now that we're deploying these services, we have more details to take care of:

  • Web servers must have hostname, port, username, password of Postgres, and name of database
  • Task queues must have hostname, port, username, password of Postgres, and name of database
  • Web servers must have hostname, port of RabbitMQ server
  • task queues must have hostname, port of RabbitMQ server
  • Postgres primary must have hostname, port, username, password of replica Postgres server (production only)

This information can be properly distributed by using groups, and assigning different variable values for each different group.

Specifically, things break down best by environment (vagrant, staging, and production).

Here's what the Vagrant hosts file looks like, grouping by environment:

[all:vars]
ntp_server=ntp.ubuntu.com

[production:vars]
db_primary_host=rhodeisland.example.com
db_primary_port=5432
db_replica_host=virginia.example.com
db_name=widget_production
db_user=widgetuser
db_password=pFmMxcyD;Fc6)6
rabbitmq_host=pennsylvania.example.com
rabbitmq_port=5672

[staging:vars]
db_primary_host=orangered.example.com
db_primary_port=5432
db_name=widget_staging
db_user=widgetuser
db_password=L@4Ryz8cRUXedj
rabbitmq_host=redblue.example.com
rabbitmq_port=5672

[vagrant:vars]
db_primary_host=vagrant3
db_primary_port=5432
db_name=widget_vagrant
db_user=widgetuser
db_password=password
rabbitmq_host=vagrant3
rabbitmq_port=5672

Flags