There have been a number of documented issues with the Rocky Linux errata. Many of the issues with Rocky errata stem from the current state of the Apollo system which handles pulling in Red Hat errata information and then cloning it to Rocky. The Red Hat Hydra API is used as the initial source of truth for pulling in Red Hat advisory information as it contains information on security, bug, and enhancement advisories. However, this API is not always 100% accurate and can lead to inaccurate Rocky advisories.
The CIQ Open Source Program Office (OSPO) would like to work with the Rocky team to re-work Apollo to increase the accuracy of Rocky advisories starting with security advisories. This would entail moving from the Hydra API to using the Red Hat CSAF VEX files as the source of truth for Rocky Security advisories. The long term goal would be to get all errata to a more accurate and stable place but reducing the scope to only security advisories for the time being.
The ask we have from the community is this:
What are your thoughts on temporarily shifting the focus of the Rocky Errata to be more focussed on security errata and trying to make these as accurate as possible? What are the downsides to doing this for the time being? Any concerns or thoughts? We want to ensure that any rebuild of this service provides what the community needs.
Weāll leave this thread for open comment until 4.16.25, after which weāll present a proposal. We know this situation with errata needs to be fixed, and weāre eager to dive in.
Yes please! This is very timely, as I was looking at determining patches for our upcoming patch day this morning! For instance, the newest security updates for Rocky8 on errata.rockylinux.org show as February 26th, so either they are not being tagged or not being pushed.
Focusing on security updates errata is my highest concern with Rocky Linux, let us know if we can help in any way.
Hi,
Yes, it is very important for us to be able to manage security errata.
It allows us to have a stable environment in production and use Foreman for patching.
We rely on errata to determine security patches, not other kinds of changes. So temporary shift in focus to stabilize this subset makes total sense to me.
Hello,
Same here, it would be really nice to have security erratas as soon as possible (to integrate them in foreman content views in my case).
Concerning bug and enhancement advisories, we can wait to publish new content views with all RPMs that will be tested and slowly published to production env.
I will be glad to help or test if needed too.
Thanks for the efforts on this subject.
I feel that resolving processing issues with security errata and ensuring its stability/reliability as a first and foremost priority is proper and a correct action plan. It becomes very difficult to accurately paint a complete security picture to oneās stakeholders without the pertinent and verifiable data. Additionally, compliance activities such as audits and reports are a more manual process. The major third-party security tooling providers which also depends on this data to be readily available canāt provide the automation to run their tools against the errata, leading to missing information.
The pitfall to be cognizant of would be that once the security errata has been restored to a stable and functional process, is to not delay and apply the corrected methods to bug fixes and enhancement errata as resources permit. While not as pressing as security matters, these items are still crucial to the overall structure and health of the Rocky Linux OS. Itās very tidy to use when you caught in a unique situation to patch a bug or release an enhancement feature and want to see if this resolves any issues at hand.
Iām very pleased to see that there is a focused effort in resolving this issue with a long term plan and roadmap!
I will add my emphasis to the need for Enterprise Errata and OVAL files/outputs. The current OVAL files derived from the errata: Index of /pub/oval/ do not work in OpenSCAP. Lacking the ability to verify the Rocky Linux system is locked down and is up to data on patches, means the software is NOT enterprise ready in the eyes of our organization. The lock-down check from the STIG if fine, but the OVAL generates errors and cannot be used. I greatly appreciate efforts to get this fully functional again, so do not take this as slander to those donating their time to develop. I am happy to test and provide any insight I can, but I am not a developer. Some Issues are identified here: GitHub - rocky-linux/oval: OVAL file generator. Thanks again for your support!
The public comment period is now closed. As there has been no disagreement to the proposal of focusing on improving the accuracy of security updates, this is the approach we will take.
Weāre going to work on the existing tooling (Apollo) to improve security update accuracy. We need to get the tooling to a state where updates are reliably identified and matched to their associated builds and the information for them is appropriately enriched.
Simultaneously, weāre going to work on implementing additional automation to the current build process so that when updates are made available, manual effort is not required to ingest the updates, patch them, and then push the new build. This will ensure updates are made available as soon as possible.
Every Tuesday at 1300 EST we will have a standup with the engineers leading these efforts, with the first set for 4.22.25. These will be open to the public to attend, and posted on the Rocky calendar. Anyone can drop in to get status updates, progress, etc.
Weāll also be updating this thread as we make progress so the community is kept up-to-date along the way.
Progress on this continues to go well. After a period of discovery, as planned weāve outlined where things are at with Apollo, current issues, and a path forward broken into phases. That document is public, and can be viewed here:
By moving to Red Hat CSAF instead of Hydra for updates, we believe weāll be able to pull in not only security, but bug fixes and other advisories as well.
Weāll look to have an update out by end of next week on the status of phase 1.
Hereās a PR covering the first phase in the above linked tracker, the CSAF ingestion proof of concept:
Unless thereās an objection, weād like to merge this by next Tuesday, the 20th. Itās gone through extensive review by multiple CIQ engineers, but weād like community and Rocky engineer reviews as well.
Replaces HYDRA-based security advisory ingestion with a CSAFv2-driven workflow.
Streams and parses CSAF files directly from Red Hat, filtered to the past 30 days.
Adds new logic for extracting affected products (including noarch expansion and arch mapping).
Improves logging and error handling throughout ingestion and advisory processing.
Adds shared NEVRA parsing utilities to reduce duplication.
Includes expanded test coverage for CSAF parsing, edge cases, and advisory creation.
Lays groundwork for future parallelization and backfill support.
This work aligns with Phase 2 of the ongoing advisory ingestion overhaul and sets the stage for production rollout in a future phase. Apollo Refactor Doc
Thanks in advance to anyone who has time to review!
