As always: depends on your needs and usage 
But remember, we’re just random people here so far with multiple ideas and different experiences. For me Jenkins wins only about flexibility. But loses with simplicity and resource usage.
I would strongly suggest using Pagure and Koji for the sources and package build tool. Especially in order to ensure everything will build and release as expected, and it’d be trivial to reuse the existing integrations built in Fedora and CentOS for this.
Just wondering about the yum/dnf repositories.
And the installation ISO’s and VM images.
Isn’t that the biggest problem due to the large amount of data?
We will be using Koji. We may not use Pagure as development for it has slowed. But I don’t know if that decision is final yet.
2 Likes
Does Koji finally allows to login using username/password or some oAuth2? Or still only SSL Certs?
I’m gonna bump Gitea as a git forge as I’m a maintainer of it. I can help implement any features that we may need.
According to docs, still Kerberos and ssl certs https://docs.pagure.org/koji/server_howto/#koji-authentication-selection
It hasn’t stopped, and people are actively contributing and working on releases: https://pagure.io/pagure/commits/master
There are a number of projects running Pagure instances now, and some of those folks are contributing to Pagure to support it.
Zabbix’ biggest failure imo is automatic host enrollment in a secure way. If that isn’t necessary it should be able to most of the actual monitoring with the right hooks in. Agent-based is better from a security standpoint imo as well.
Icinga has a much higher curve for getting things running initially but if you’re willing to put in the effort and have some people who are intimately familiar with it it might be better in the long run. I wouldn’t attempt to use it without director though, which is another hump on the learning curve.
If you want pretty dashboards you’re going to need something else on top of these but I’ve never found them necessary except when the C-levels wanted to show someone our amazing system shrug
If it matters, would vote for Zabbix as quickest up and running option.
1 Like
I’ll also add to the bandwagon against OpenStack. Like everyone else here said, it’s an absolute monster. If you’re just looking for ‘self-hosted cloud’ KVM with a gluster backing and freeipa-based GSSAPI authentication can be setup on cheap used hardware inside of a day. There’s some trickiness to migration without third party helpers (I really REALLY want a few weeks to be able to dive into oVirt) but I’ve been running a company on that exact setup for three years now (with a low five-figure hardware budget!) and it is rock solid.
I’d like to echo recommendations for the following stacks:
Virtualization: oVirt
Configuration Management: Ansible / AWX
authentication / authorization: FreeIP/Keycloak
DNS: FreeIPA
Monitoring: I’d say Prometheus/grafana over Zabbix
Secret Management: Vault
Container Orchestration (if required): Kubernetes clusters deployed using kubespray
I am willing to help in the deployment of any of the above.
+1 for Prometheus and FreeIPA.
Having spent around a decade on Puppet infrastructure, I think I’d rather see Ansible/AWX employed here.
I’m somewhat ambivalent about most of the other infrastructure.
1 Like
I haven’t used Prometheus myself but have heard a lot of good. I found zabbix to “just work” for simple topologies but it’s been a number of years since I’ve had occasion to stand it up.
1 Like
Prometheus is super simple to set up, definitely simpler than Zabbix last time I did it. Prometheus’ database is built in, rather than needing to run a separate instance of MySQL or another RDBMS. It’s also written in Go and provides its own HTTP server, so no need to deal with setting up an HTTP proxy (although we probably would want to for HTTPS support, although that might have landed upstream).
Prometheus also scales very well. With the plethora of prexisting “exporters” (agents), there’s support for basically every major open source software out there. When I was first learning Prometheus, understanding “everything as a timeseries” was pretty mind bending, but it pays huge dividends. Prometheus alert manager also has a number of integrations, like PagerDuty.
I’m happy to talk at length about Prometheus.
2 Likes
Don’t deploy new Gluster. It’s being put to pasture. Go with Ceph.
2 Likes
What is the alerting capability of Prometheus/Grafana? I’ve typically had to support a system like Zabbix which is capable of doing more complex triggering, more in depth than “metric X over threshold”. I’ve used the alerting in Grafana but it simply isn’t as powerful as more complex systems which can run arbitrary scripts etc.
Of course, this only matters if you care about some level of SLA, idk if Centos has something like that?
So, is a greater feature set than metric collection required, and if so does Prometheus provide that?
although I like gluster, you are right its being abandoned by rh
the only reason rh still pouring money in it is due to support contracts.
ceph is the way to go for distributed storage
although you have to consider the fact that the upstream of ceph is seriously lagging behind the production version of rh.
Grafana supports a list of notifiers.
Since it lists email and Web hooks, basically it supports every known notification means.
+1 for Ceph, but very dependent on requirements and infrastructure
1 Like
As for the FreeIPA stuff - maybe we should go with noggin [1] (self-service portal for FreeIPA), it is planned to be used as core of the IDM infra for the Fedora, CentOS and OpenSuSE
[1] https://github.com/fedora-infra/noggin
1 Like