Ansible : organize my playbooks, inventories and roles

Rocky Linux General
1

Hi,

I’ve spent this last couple of months experimenting extensively with Ansible. So far I’ve only worked on a series of VMs using Vagrant and KVM on my workstation. I’ve become reasonably proficient with ansible.cfg, inventory files and playbooks. I’ve already written my first roles and know how to properly organize them in a directory subtree with tasks, files, handlers etc. Looks like I’m beginning to wrap my head around this, though I must say Ansible is a hell of a beast and the learning curve is quite steep.

I feel the time has become to use it for my everyday work. Before I dive head first into using inventories and playbooks and roles for my everyday work, I’m facing the following problem. How do I organize all this wealth of files to come intelligently? How will all this looks like in a year? How can I avoid having myriads of redundant playbooks scattered all over semi-redundant Git repositories?

I’ve given this some reflection for a couple days, and here’s what I came up with.

1. One central Git repository where I store pretty much everything except roles. Here’s what this would look like:

$ tree -F
.
├── campanula/
│   ├── ansible.cfg
│   ├── inventory
│   └── roles/
├── dedibox/
│   ├── ansible.cfg
│   ├── inventory
│   └── roles/
├── microlinux/
│   ├── alphamule.yml
│   ├── ansible.cfg
│   ├── inventory
│   ├── nestor.yml
│   ├── proxy.yml
│   └── roles/
└── scholae/
    ├── ansible.cfg
    ├── inventory
    ├── pc-direction.yml
    ├── pc-info.yml
    ├── portable.yml
    ├── proxy.yml
    ├── roles/
    ├── sauvegarde.yml
    └── serveur.yml

A little explanation on what is what here:

  • microlinux is my own office, with a proxy, a backup server and my main workstation.
  • scholae is our local school with about three dozen machines (servers, desktop clients, laptops) all running Rocky Linux
  • campanula is a small company in Belgium
  • dedibox is the directory for all my public Internet-facing server

2. All these various playbooks use a series of roles. I would store all the roles separately in another Git repository and aim for reusability. So I would have something like a rockylinux-8-roles repo containing a series of roles like configure_shell, configure_repos, update_system, install_base, install_extras, etc.

Notice the directory tree above has a series of empty roles/ directories. Now this is where I would import the roles from GitHub (with the corresponding .gitignore files of course).

So far this makes sense to me in theory, but before diving into this, I thought I’d share this idea with you. I’m curious about your way of doing things and your eventual suggestions.

Cheers from the sunny South of France,

Niki

2

I don’t find it very efficient if you have servers in different locations that require same base configurations, that way you would have to copy some roles over to the other repo when you make changes to tasks that that are used across multiple locations.
Why not just have all the roles in the same ansible repo and have a common role with the configurations that are the same for all the servers across locations since it’s possible to have servers in multiple groups or you can create playbooks for each location that includes specific roles and then use host_vars and group_vars to help with the logic of your tasks/templates.
https://docs.ansible.com/ansible/2.8/user_guide/playbooks_best_practices.html#content-organization
https://docs.ansible.com/ansible/2.8/user_guide/playbooks_best_practices.html#group-and-host-variables

2 Likes
3

Better is to have one ansible.cfg, one inventory. You list all the hosts in the inventory file, you can even put hosts in a group if they are similar to each other, eg: webservers.

Roles you share between all your hosts. When making your playbook, you can then apply these to certain hosts or certain host groups so that they don’t get applied to the hosts where you do not want them to be applied.

Eg, in inventory:

[webapps]
campanula
dedibox

then for a task, or importing tasks/roles, etc:

  tasks:
    - name: Creating /etc/myissue if in webapps group
      copy:
        content: "Web Application Server"
        dest: /etc/myissue
      when: '"webapps" in group_names'

or:

  tasks:
    - name: Adding repositories
      import_tasks: tasks/add_repos.yml

you can see the when option. You can also use things like ansible facts if you want to pull out the hostname to apply it to a single host rather than group.

The idea is to keep it simple to save repeating yourself for each host, group or whatever. Make it more dynamic.

You can also use ansible-galaxy to create directory structures for roles, eg:

root@redmine:~/playbooks# ansible-galaxy role init mytestrole
- Role mytestrole was created successfully

root@redmine:~/playbooks# cd mytestrole/

root@redmine:~/playbooks/mytestrole# tree
.
├── defaults
│   └── main.yml
├── handlers
│   └── main.yml
├── meta
│   └── main.yml
├── README.md
├── tasks
│   └── main.yml
├── tests
│   ├── inventory
│   └── test.yml
└── vars
    └── main.yml

6 directories, 8 files

you can then set up that role with default variables, handlers for restarting services once changes are applied, relevant tasks that need to be run, eg: ensure firewalld installed and running, or httpd installed and running. That role can then be used for multiple hosts. Even in a when field, you can set up tasks o be for multiple versions of Linux, so:

when: ansible_distribution == "Debian"

Depending on ansible version, instead of using the apt or dnf option, there is a package option so irrespective of distro, be it Debian, Ubuntu, Rocky, it will use the appropriate package manager itself and save you duplicating tasks when using apt or dnf in the playbook to install packages. With that example, you don’t even need to use when to differentiate Debian/Ubuntu with say Rocky which would use dnf - because package will automatically know.

Eg:

  tasks:
    - name: Install firewalld
      package:
        name: firewalld
        state: present

instead of:

  tasks:
    - name: Install firewalld
      dnf:
        name: firewalld
        state: present
      when: ansible_distribution == "Rocky"

with when you can also use or and, so you could apply something to multiple distributions.

when: (ansible_distribution == "RedHat" or ansible_distribution == "Rocky") and ansible_distribution_major_version == "8"
2 Likes
4

Thanks for all the feedback. As it happens sometimes, you start with a complicated layout before being able to do things in a more simple and straightforward manner.

Here’s a start for my “real” Ansible repo. For now I have just one host to test things, and then I’ll evolve from there.

https://gitlab.com/kikinovak/ansible

Cheers,

Niki

1 Like
5

The way I see it, there are essentially three: plays, roles, and inventories.

The plays may use some roles, but can ideally be used with any inventory. Plays are ok to share too, since they should not contain any “site-specific” data.

The roles can be in many places. The rhel-system-roles package has some. There are many roles in various git repos (and in Ansible Galaxy). You can have long list of paths in roles_path in ansible.cfg(s). The plays can have necessary roles and collections listed in requirements.yml, so that one can conveniently pull in all the roles and modules that the plays need.

So far I’ve kept the roles that I’ve written with the plays in one repo. Ideally, they would be shared, separately.

Inventories – my plays are used in multiple unrelated sites, so each has their own inventory in its own repo. The inventories might contain sensitive information (recommendation is to have that crypted with ansible-vault) so that is not something that one shares. If I have access to multiple inventories in same control host, I can select inventory with the -i option, or could have one directory of plays for each inventory and ansible.cfg in it pointing to the relevant inventory.

What I have not figured out yet is files. Ideally, roles have the files (or better yet templates) to deploy to target hosts. However, when a site has very specific files to deploy (that I have no role for or not feasible), where the put those? They can’t be “inside” inventory, which is a file – or files in directory (tree). The ansible commands do look files from $(pwd)/files, but that is “subdir of plays”.

1 Like
6

One more thing:
Ansible 101 - Standards - Ansible Junky is a nice read.

3 Likes
7

Hello Niki,
You can take a look at our ansible documentation:
https://docs.rockylinux.org/books/learning_ansible/06-large-scale-infrastructure/

This part of our Ansible book is based on the recommendations in the ansible documentation, and this organization works for at least 400 servers.

Cheers from the cloudy west of France :wink:

2 Likes
8

Ha ! Merci Antoine ! Un gentil bonjour de la garrigue gardoise !