The discussion thus far on IRC/Matrix has been concerning whether to use Ansible (with Tower) and existing playbooks or develop infrastructure in puppet instead. Moving this discussion here for further comment.
Running tally of top options:
Ansible (config management)
– including vault (secrets)
– including tower (management / auth)
FreeIPA (Identity Management)
I believe we’ve confirmed now that we’ll be using Ansible, 90% sure we decided on Tower, it’s the bits that float around it we’re waiting on now I think.
Yes, 100% ansible.
We are going to also do tower (AWX) to make sure things are nice and secure and we’re locking it down properly.
So, requirements around AWX sounds like Ansible and Docker, Python 3, Node 14.x, NPM 6.x, make, and git. Hardware requirements are 2 CPUs, 4GB of RAM, 20GB of storage, and AWX itself needs to run on a system running Docker, K8s or OpenShift.
Is this something that needs to be highly available/run in a cluster behind a load balancer?
To get things rolling (ie. deploy AWX), someone needs to start with plain old Ansible playbooks in a git repo 
Once AWX (and probably Identity with FreeIPA or something) is deployed, it can be used to deploy the rest of the stuff.
If AWX only handles configuration management and orchestration, then it’s probably not a component that can’t recover from some minutes of downtime every now and then. rather than HA and redundancy for it, I’d focus on good backups first.
Looking at the components, it’s a web UI on top of PostgreSQL, and both of those can be migrated into an HA setup later too.
It’s also possible to use a Managed Kubernetes solution somewhere, or set up our own, but to be honest I would not want to complicate the infrastructure with full-blown container orchestration right off the bat.
There are also non-techical things like naming schemes and such to decide, which are surprisingly important when it comes to building understandable systems.
Agreed, oranenj, especially with what is sounding like it’ll end up as a more distributed architecture… we need to make sure we encode good naming conventions into things.
I’m personally a fan of location-environment-subenvironment-name/function-arity.subenvironment.environment.location.rockylinux.org… e.g.
use1-dev-ci-builder001.ci.dev.use1.rockylinux.org (or similar) though pieces can be removed/added/changed as required.
Spent the last hour or so mapping all the dependencies and taking some notes on the CentOS Ansible Repo mentioned above. Below are all the playbooks, Their interconnects (Not documented well we need to NOT do that ) and a few i identified that would need a rebrand if we adopted the playbook.
FreeIPA can also handle DNS responsibilties in addition to Identity Management)
Yes, we have came to the decision to use FreeIPA for identity management. It will likely be used for internal infrastructure DNS, but not external DNS (it doesn’t support views).
Roger that.
Good point.
Vince
I mentioned on another thread, I’ve been doing Puppet for a decade in a very large scale environment… And I think we should do Ansible. Seems like there’s no opposition to Ansible, though…
As an Ansible expert working for everyone’s favorite chromatic head wear company, I’m all about being here to support Ansible!
It’s worth pointing out I’ve had success with running AWX on Podman on Centos8, if we’re wanting to keep it self hosted (as Centos8/RockyToBe doesn’t support docker as standard) - https://github.com/ikke-t/awx_pod
Having AWX in a container may be useful for us. We will keep note of this for when we get the infra setup.
Looks great. But “Role is tested on Fedora server.” Is this role usable with CentOS 8 as is?
A coworker recently setup AWX with Docker on CentOS 8. It was painful.
As I said - yes I got it up and working relatively easily on a Centos 8.2 install.
Hi folks. I work on the AWX/Tower team. I’m excited to see y’all are considering using it. I just came here to share this announcement regarding upcoming changes to the project’s architecture and deployment model: https://groups.google.com/g/awx-project/c/47MjWSUQaOc
Thank you for letting us know Shane!
I’m personally not a huge of Ansible because with proper time and coding it can’t really do anything you could not do already with SSH commands and bash scripting. Even less now that Red Hat belongs to IBM. It is very likely IBM will start to cut down on open source projects in the future.
I’m currently using awx in an rpm based setup for production. Multiple instances in groups, and ha (replication) postgresql backend. Not quite as simple as containers, but quite scalable. I’ve got the setup notes somewhere and can post up if needed.