Ansible/Roles

Playbook Roles

What are Ansible roles?

Roles allow you to split your playbook into different parts for different servers.

For example, a webapp with a database backend can define a webserver role and a database role, and it becomes much easier to modify the playbook to run these on the same host or on different hosts.

Feature: pre-tasks and post-tasks

In the playbook you can specify a pre-task and a post-task for a role.

For example, suppose you want to update aptitude before deploying the web server, and you want to send a notification to Slack when you are finished.

Then you could use the following playbook, which defines a pre_tasks list of things to do before the roles are defined, and a post_task list of things to do once the roles have been carried out.

- name: deploy mezzanine on vagrant
  hosts: web
  vars_files:
    - secrets.yml
  pre_tasks:
    - name: update the apt cache
      apt: update_cache=yes
  roles:
    - role: mezzanine
      database_host: "{{ hostvars.db.ansible_eth1.ipv4.address }}"
      live_hostname: 192.168.33.10.xip.io
      domains:
        - 192.168.33.10.xip.io
        - www.192.168.33.10.xip.io
  post_tasks:
    - name: notify Slack that the servers have been updated
      local_action: >
        slack
        domain=acme.slack.com
        token={{ slack_token }}
        msg="web server {{ inventory_hostname }} configured"

Feature: role files

If a role is particularly complicated or has details of its own to take care of, you can put all of the files specific to one particular role into a directory.

Suppose you have two roles, webserver and database. Then your directory structure would look like this:

playbooks/

        roles/
            database/
                tasks/
                    main.yml
                handlers/
                    main.yml
                files/
                    pg_hba.conf
                    postgresql.conf

            webserver/
                ...

Tasks, handlers, files, templates, etc. are all put into subfolders with the same name as the role.

Example

Let's look at an example of deploying an application with Ansible that a classic architecture: a web frontend with a database backend.

The web frontend can use an nginx web server playbook (see Ansible/Nginx Playbook for sample nginx playbook).

Example Playbook: Single Host

Here is an example playbook with two roles, web and database.

- name: deploy webapp on vagrant
  hosts: web
  vars_files:
    - secrets.yml
  roles:
    - role: database
      database_name: "{{ mezzanine_proj_name }}"
      database_user: "{{ mezzanine_proj_name }}"
    - role: webserver
      live_hostname: 192.168.33.10.xip.io
      domains:
        - 192.168.33.10.xip.io
- www.192.168.33.10.xip.io

Example Playbook: Single Host with Pre and Post Tasks

Here is the same playbook but running a pre/post task before/after the roles are defined. These pre/post tasks will be run once per host.

- name: deploy mezzanine on vagrant
  hosts: web
  vars_files:
    - secrets.yml

  pre_tasks:
    - name: update the apt cache
      apt: update_cache=yes

  roles:
    - role: database
      database_name: "{{ mezzanine_proj_name }}"
      database_user: "{{ mezzanine_proj_name }}"

    - role: webserver
      live_hostname: 192.168.33.10.xip.io
      domains:
        - 192.168.33.10.xip.io
        - www.192.168.33.10.xip.io

  post_tasks:
    - name: notify Slack that the servers have been updated
      local_action: >
        slack
        domain=acme.slack.com
        token={{ slack_token }}
        msg="web server {{ inventory_hostname }} configured"

Example Playbook: Multiple Hosts

Here is the prior playbook adapted to multiple hosts:

- name: deploy postgres on vagrant
  hosts: db
  vars_files:
    - secrets.yml
  roles:
    - role: database
      database_name: "{{ mezzanine_proj_name }}"
      database_user: "{{ mezzanine_proj_name }}"

- name: deploy mezzanine on vagrant
  hosts: web
  vars_files:
    - secrets.yml
  roles:
    - role: mezzanine
      database_host: "{{ hostvars.db.ansible_eth1.ipv4.address }}"
      live_hostname: 192.168.33.10.xip.io
      domains:
        - 192.168.33.10.xip.io
        - www.192.168.33.10.xip.io

Database role

The database role has multiple files associated with it, to define the tasks, default variable values, handlers, files, and templates used by Ansible.

These should go in playbooks/roles/database/.

The Ansible/Directory Layout article gives much more detail on the directory structure that should be in place for a role.

Tasks File

The main thing we want to define for our database role are the tasks required to set up the database server. These will be specific to the database role and are not executed by the webserver.

The task file is located in playbooks/roles/database/tasks/main.yml and are defined like normal tasks:

• install packages
• copy postgres config file into machine
• copy client authentication config file
• create project locale
• create postgresql user
• create the database as the postgresql user

playbooks/roles/database/tasks/main.yml:

- name: install apt packages
  apt: pkg={{ item }} update_cache=yes cache_valid_time=3600
  become: True
  with_items:
    - libpq-dev
    - postgresql
    - python-psycopg2

- name: copy configuration file
  copy: >
    src=postgresql.conf dest=/etc/postgresql/9.3/main/postgresql.conf
    owner=postgres group=postgres mode=0644
  become: True
  notify: restart postgres

- name: copy client authentication configuration file
  copy: >
    src=pg_hba.conf dest=/etc/postgresql/9.3/main/pg_hba.conf
    owner=postgres group=postgres mode=0640
  become: True
  notify: restart postgres

- name: create project locale
  locale_gen: name={{ locale }}
  become: True

- name: create a user
  postgresql_user:
    name: "{{ database_user }}"
    password: "{{ db_pass }}"
  become: True
  become_user: postgres

- name: create the database
  postgresql_db:
    name: "{{ database_name }}"
    owner: "{{ database_user }}"
    encoding: UTF8
    lc_ctype: "{{ locale }}"
    lc_collate: "{{ locale }}"
    template: template0
  become: True
  become_user: postgres

Note that copying the configuration file includes a notify: restart postgres. These are handler notifications (see next section).

Handlers File

Handlers can be defined in the handlers folder of the directory for this role.

Recall that handlers are simply tasks that are dependent on other tasks.

We define a task to restart the postgresql service. Whenever the configuration files are changed, this handler is notified/run.

playbooks/roles/database/handlers/main.yml:

- name: restart postgres
  service: name=postgresql state=restarted
  become: True

Defaults

To define defaults, we can use the defaults folder following the same patterns as above.

roles/database/defaults/main.yml:

database_port: 5432

Variables

Note that we defined a vars_files variable at the top of the playbook:

- name: deploy mezzanine on vagrant
  hosts: web
  vars_files:
    - secrets.yml

The secrets.yml file will contain, we presume, secrets like the database password.

The variables we referred to were:

• database_name
• database_user
• db_pass
• locale

The rest of the variables can be defined as follows:

• If the variable will be the same across all roles, define the variable in playbooks/group_vars/all

• If the variable will change from role to role, put it in the playbooks/<role-name>/defaults/main.yml file

Webapp role

Here we will go through the webserver or webapp role, which runs the application that operates the web frontend for our application.

All files related to this role should go in playbooks/roles/webapp/.

Below,"Mezzanine" refers to the name of a hypothetical webapp.

Files

The files we will use with this role are as follows:

Here are the files that the role comprises:

Default variable values:

• roles/mezzanine/defaults/main.yml

Role variables:

• roles/mezzanine/vars/main.yml

Handlers:

• roles/mezzanine/handlers/main.yml

Tasks:

• roles/mezzanine/tasks/django.yml
• roles/mezzanine/tasks/main.yml
• roles/mezzanine/tasks/nginx.yml

Templates:

• roles/mezzanine/templates/gunicorn.conf.py.j2
• roles/mezzanine/templates/local_settings.py.filters.j2
• roles/mezzanine/templates/local_settings.py.j2
• roles/mezzanine/templates/nginx.conf.j2
• roles/mezzanine/templates/supervisor.conf.j2

Variables

There are two places to add variables:

• in the role-specific variables file playbooks/<role-name>/vars/main.yml
• in the variable defaults file playbooks/<cole-name>/defaults/main.yml

roles/mezzanine/vars/main.yml:

# vars file for mezzanine
mezzanine_user: "{{ ansible_user }}"
mezzanine_venv_home: "{{ ansible_env.HOME }}"
mezzanine_venv_path: "{{ mezzanine_venv_home }}/{{ mezzanine_proj_name }}"
mezzanine_repo_url: git@github.com:lorin/mezzanine-example.git
mezzanine_proj_dirname: project
mezzanine_proj_path: "{{ mezzanine_venv_path }}/{{ mezzanine_proj_dirname }}"
mezzanine_reqs_path: requirements.txt
mezzanine_conf_path: /etc/nginx/conf
mezzanine_python: "{{ mezzanine_venv_path }}/bin/python"
mezzanine_manage: "{{ mezzanine_python }} {{ mezzanine_proj_path }}/manage.py"
mezzanine_gunicorn_port: 8000

All variables in the mezzanine-specific role file are prefixed with "mezzanine" - good practice to keep from getting confused or causing unexpected behavior when using variables across multiple roles.

Defaults

The default variable values just defines one variable, to turn TLS (HTTPS) on:

playbooks/roles/mezzanine/defaults/main.yml:

tls_enabled: True

Tasks

This task is broken up into three YML files.

The main (top-level) task list defines how to install all packages (using a for loop over packages), then transcludes django and nginx task files.

roles/mezzanine/tasks/main.yml

- name: install apt packages
  apt: pkg={{ item }} update_cache=yes cache_valid_time=3600
  become: True
  with_items:
    - git
    - libjpeg-dev
    - libpq-dev
    - memcached
    - nginx
    - python-dev
    - python-pip
    - python-psycopg2
    - python-setuptools
    - python-virtualenv
    - supervisor

- include: django.yml

- include: nginx.yml

The reason for splitting the tasks out is that they get long and fairly involved.

Here is the django task list:

roles/mezzanine/tasks/django.yml:

- name: create a logs directory
  file: path="{{ ansible_env.HOME }}/logs" state=directory

- name: check out the repository on the host
  git:
    repo: "{{ mezzanine_repo_url }}"
    dest: "{{ mezzanine_proj_path }}"
    accept_hostkey: yes

- name: install Python requirements globally via pip
  pip: name={{ item }} state=latest
  with_items:
    - pip
    - virtualenv
    - virtualenvwrapper

- name: install required python packages
  pip: name={{ item }} virtualenv={{ mezzanine_venv_path }}
  with_items:
    - gunicorn
    - setproctitle
    - psycopg2
    - django-compressor
    - python-memcached

- name: install requirements.txt
  pip: >
    requirements={{ mezzanine_proj_path }}/{{ mezzanine_reqs_path }}
    virtualenv={{ mezzanine_venv_path }}

- name: generate the settings file
  template: src=local_settings.py.j2 dest={{ mezzanine_proj_path }}/local_settings.py

- name: apply migrations to create the database, collect static content
  django_manage:
    command: "{{ item }}"
    app_path: "{{ mezzanine_proj_path }}"
    virtualenv: "{{ mezzanine_venv_path }}"
  with_items:
    - migrate
    - collectstatic

- name: set the site id
  script: scripts/setsite.py
  environment:
    PATH: "{{ mezzanine_venv_path }}/bin"
    PROJECT_DIR: "{{ mezzanine_proj_path }}"
    PROJECT_APP: "{{ mezzanine_proj_app }}"
    WEBSITE_DOMAIN: "{{ live_hostname }}"

- name: set the admin password
  script: scripts/setadmin.py
  environment:
    PATH: "{{ mezzanine_venv_path }}/bin"
    PROJECT_DIR: "{{ mezzanine_proj_path }}"
    PROJECT_APP: "{{ mezzanine_proj_app }}"
    ADMIN_PASSWORD: "{{ admin_pass }}"

- name: set the gunicorn config file
  template: src=gunicorn.conf.py.j2 dest={{ mezzanine_proj_path }}/gunicorn.conf.py

- name: set the supervisor config file
  template: src=supervisor.conf.j2 dest=/etc/supervisor/conf.d/mezzanine.conf
  become: True
  notify: restart supervisor

- name: ensure config path exists
  file: path={{ mezzanine_conf_path }} state=directory
  become: True
  when: tls_enabled

- name: install poll twitter cron job
  cron: >
    name="poll twitter" minute="*/5" user={{ mezzanine_user }}
    job="{{ mezzanine_manage }} poll_twitter"

The next set of tasks relate to the nginx server.

These install the nginx configuration files and certificate, and restart the nginx service.

roles/mezzanine/tasks/nginx.yml:

- name: set the nginx config file
  template: src=nginx.conf.j2 dest=/etc/nginx/sites-available/mezzanine.conf
  notify: restart nginx
  become: True

- name: enable the nginx config file
  file:
    src: /etc/nginx/sites-available/mezzanine.conf
    dest: /etc/nginx/sites-enabled/mezzanine.conf
    state: link
  notify: restart nginx
  become: True

- name: remove the default nginx config file
  file: path=/etc/nginx/sites-enabled/default state=absent
  notify: restart nginx
  become: True

- name: create tls certificates
  command: >
    openssl req -new -x509 -nodes -out {{ mezzanine_proj_name }}.crt
    -keyout {{ mezzanine_proj_name }}.key -subj '/CN={{ domains[0] }}' -days 3650
    chdir={{ mezzanine_conf_path }}
    creates={{ mezzanine_conf_path }}/{{ mezzanine_proj_name }}.crt
  become: True
  when: tls_enabled
  notify: restart nginx

Handlers

roles/mezzanine/handlers/main.yml:

- name: restart supervisor
  supervisorctl: name=gunicorn_mezzanine state=restarted
  become: True

- name: restart nginx
  service: name=nginx state=restarted
  become: True

Dependent Roles

Suppose two particular roles need to perform a set of tasks that are the same for both machines (e.g., installing NTP on both a web server and a database server).

Hard-coding the tasks in each role's playbook would be unnecessary duplication.

Creating a separate ntp role would require us to separately apply this role each time (can forget).

What we want is to specify a "sub-role" (a dependent role) that will be executed before the main role.

The meta folder of each role can store information about dependencies.

Dependent Roles Example

Let's look at an example of dependent roles.

Web Server

We can specify a dependency for all machines playing a web server role (role "web") by editing roles/web/meta/main.yml:

roles/web/meta/main.yml:

dependencies:
    - { role: ntp, ntp_server=ntp.ubuntu.com }

Django

Suppose we have a role for a Django server, and we want to specify two dependent roles. Then we can create the following dependent roles list:

roles/django/meta/main.yml:

dependencies:
    - { role: web }
    - { role: memcached }