Ansible/Full Stack Playbook
From charlesreid1
This page covers an Ansible playbook for a full stack example. This full stack example will run the following services:
- Django web server
- Celery task queue
- RabbitMQ message queue
- Postgresql for data storage
Contents
Setup
See Ansible/Full Stack Playbook/Vagrant Setup for details of setting up Vagrant to test out this configuration
Full Stack Playbook
Overview of Full Stack
Services
For this particular full stack deployment, we have the following services:
- Django app + http server
- Nginx web server
- Celery task queue
- RabbitMQ (Celery backend)
- Postgres (persistent store)
Environments
We have three environments to deploy to:
- vagrant (local testing)
- staging (testing)
- production
Architecture
Here is the architecture used for the above apps:
- Web application is run on multiple hosts for better performance, with a load balancer in front
- Task queue servers are run on multiple hosts for better performance
- Celery, RabbitMQ, Postgres all on separate servers
- 2 Postgres hosts - primary and replica
That is a total of 10 hosts (1 load balancer, 3 web servers, 3 task queues, 1 mq server, 2 database servers).
In the production environment: 10 separate hosts
In the staging environment: use fewer hosts (only 2 hosts: web server and task queue on one host, rabbitmq and postgres on another)
In the vagrant environment: use 3 servers (1 for web app, 1 for task queue, and 1 for postgres)
Ansible hosts file
Note that this initial hosts file (below) can be streamlined using suggestions from Ansible/Groups, Ansible/Group Variables, and Ansible/Host Naming, so consider it a first pass.
Using the information given above about the architecture, we have the following hosts file for our Django app:
[production] delaware.example.com georgia.example.com maryland.example.com newhampshire.example.com newjersey.example.com newyork.example.com northcarolina.example.com pennsylvania.example.com rhodeisland.example.com virginia.example.com [staging] redblue.example.com orangered.example.com [vagrant] vagrant1 ansible_host=127.0.0.1 ansible_port=2222 vagrant2 ansible_host=127.0.0.1 ansible_port=2200 vagrant3 ansible_host=127.0.0.1 ansible_port=2201 [lb] delaware.example.com [web] georgia.example.com newhampshire.example.com newjersey.example.com redblue.example.com vagrant1 [task] newyork.example.com northcarolina.example.com maryland.example.com redblue.example.com vagrant2 [rabbitmq] pennsylvania.example.com orangered.example.com vagrant3 [db] rhodeisland.example.com virginia.example.com orangered.example.com vagrant3
Note that we make the vagrant machines part of the vagrant group when their behavioral inventory parameters are specified, but we can later add them to other groups by adding their short name (only) to the group.
In this way, we've defined which machines are in each of the 3 environments (production, staging, vagrant) and which one runs which service.
Passing credentials via the host file
Now that we're deploying these services, we have more details to take care of:
- Web servers must have hostname, port, username, password of Postgres, and name of database
- Task queues must have hostname, port, username, password of Postgres, and name of database
- Web servers must have hostname, port of RabbitMQ server
- task queues must have hostname, port of RabbitMQ server
- Postgres primary must have hostname, port, username, password of replica Postgres server (production only)
This information can be properly distributed by using groups, and assigning different variable values for each different group.
Specifically, things break down best by environment (vagrant, staging, and production).
Here's what the Vagrant hosts file looks like, grouping by environment:
[all:vars] ntp_server=ntp.ubuntu.com [production:vars] db_primary_host=rhodeisland.example.com db_primary_port=5432 db_replica_host=virginia.example.com db_name=widget_production db_user=widgetuser db_password=pFmMxcyD;Fc6)6 rabbitmq_host=pennsylvania.example.com rabbitmq_port=5672 [staging:vars] db_primary_host=orangered.example.com db_primary_port=5432 db_name=widget_staging db_user=widgetuser db_password=L@4Ryz8cRUXedj rabbitmq_host=redblue.example.com rabbitmq_port=5672 [vagrant:vars] db_primary_host=vagrant3 db_primary_port=5432 db_name=widget_vagrant db_user=widgetuser db_password=password rabbitmq_host=vagrant3 rabbitmq_port=5672
Flags
