Rocky 9 | Only latest kernel in AppStream

Hey folks, new forum user here with some questions.

I have a system that compiles a .ko for every kernel-devel package I can get my hands on, when rocky 9.6 came out I noticed that some kernel-devel packages were moved from AppStream to Devel, and only the latest still remains in AppStream for both x86_64 and aarch64. (kernel-devel-5.14.0-570.32.1.el9_6)

I was wondering if this is intentional? will only the latest kernel-devel be in the AppStream repo going forward? would it be safe to install kernel-devel packages from the Devel repo? is there an alternative repo or mirror I can get them from? getting them manually is not an option for me in this case.

I saw that in the vault there were some of the older kernels for rocky 9.6 (except one), but its repodata is not available (403).

Please assist as you can I know asked a bunch of questions, any relevant info would be welcome!
Thanks!

Both appstream, and devel only have the latest kernel-devel or kernel-devel-matched packages. No it’s not safe to leave the devel repo enabled:

root@rocky9:~# dnf --enablerepo=devel list kernel-devel*
Rocky Linux 9 - Devel WARNING! FOR BUILDROOT ONLY DO NOT LEAVE ENABLED                  9.2 MB/s |  16 MB     00:01    
Last metadata expiration check: 0:00:04 ago on Thu 14 Aug 2025 05:01:08 PM CEST.
Installed Packages
kernel-devel.x86_64                                       5.14.0-570.26.1.el9_6                               @appstream
kernel-devel.x86_64                                       5.14.0-570.28.1.el9_6                               @appstream
kernel-devel.x86_64                                       5.14.0-570.32.1.el9_6                               @appstream
Available Packages
kernel-devel-matched.x86_64                               5.14.0-570.32.1.el9_6                               appstream 
kernel-devel-matched.x86_64                               5.14.0-570.32.1.el9_6                               devel

as you can see devel is only for build root. Also, kernel-devel-matched isn’t needed. You cannot see previous kernels so anything older than 9.6 now because it’s all moved to the vault. The vault repositories can be used by configuring repo files accordingly, but it’s not recommended. The only supported version is the latest, so 9.6. I expect your repo files were not configured correctly to use the vault properly, hence the 404’s. Otherwise it would work. But either way, it’s not supported by Rocky to do that.

When 9.7 is released, all the 9.6 packages will move to vault. This has been the same for every Rocky 8.x and 9.x release, and will also be the same for 10.x as well. Therefore you should really only be building the kernel module for the latest release, in this instance 9.6 - and just forget about previous kernels from previous 9.x releases.

This is not RHEL where there is release pinning. If you are looking for something like that, then you should be paying for a RHEL subscription and using that.

Alternative B is to build the module the way ELRepo does: built for one 9.6 kernel, but works (usually) with every 9.6 kernel. Then build one of 9.7, etc.

Alternative C is to package to use dkms/akmod, which auto-build the module when new kernel is installed.

1 Like

Thanks for the replies @iwalker and @jlehtone

The alternative solutions would be something to look into when an an opportunity presents itself, meanwhile I’m trying to understand the issue I described.

@iwalker I’m not entirely sure I completely understand what you mean, or that the issue I described was clear enough, so let me clarify what I meant.

A few weeks ago I noticed that some 9.6 kernels that were previously in AppStream were gone and I found them in the Devel repo, I’m confident at least one of them was previously in AppStream since Amazon ships it with their .ami for rocky 9.6. (kernel-devel-5.14.0-570.17.1.el9_6)

Currently the 9.6 kernels in Devel are:

  • kernel-devel-5.14.0-570.17.1.el9_6.x86_64.rpm
  • kernel-devel-5.14.0-570.18.1.el9_6.x86_64.rpm
  • kernel-devel-5.14.0-570.19.1.el9_6.x86_64.rpm
  • kernel-devel-5.14.0-570.21.1.el9_6.x86_64.rpm
  • kernel-devel-5.14.0-570.22.1.el9_6.x86_64.rpm
  • kernel-devel-5.14.0-570.23.1.el9_6.x86_64.rpm
  • kernel-devel-5.14.0-570.25.1.el9_6.x86_64.rpm
  • kernel-devel-5.14.0-570.26.1.el9_6.x86_64.rpm
  • kernel-devel-5.14.0-570.28.1.el9_6.x86_64.rpm
  • kernel-devel-5.14.0-570.30.1.el9_6.x86_64.rpm
  • kernel-devel-5.14.0-570.32.1.el9_6.x86_64.rpm

But AppStream only has:

  • kernel-devel-5.14.0-570.32.1.el9_6.x86_64.rpm

* This also applies for aarch64

Just like you said, kernel-devel-matched is not an interest - only the kernel-devel packages, and as you can see I’m only referring to el9_6 kernels and not previous rocky versions.

As for the vault - I can’t access the repository since the repodata is Forbidden (403) but this seems in line with 9.6 being the latest, so I assume that’s why its not public.

Hopefully now you understand what I meant - I’m having a bit of an issue getting older 9.6 kernels cleanly since they are no longer in AppStream, and the solutions @jlehtone described sound useful but I wont be able to look into them for some time.

Thanks again for the replies, and if anyone has some info on the described issue please let me know ^^

Linkage: (new user so only 2, sorry for the inconvenience)

Except that they are not in devel, since I already showed this using the dnf commands. Again:

root@rocky9:~# dnf --enablerepo=devel list available kernel-devel* --showduplicates
Last metadata expiration check: 0:00:19 ago on Sun 17 Aug 2025 10:51:09 AM CEST.
Available Packages
kernel-devel.x86_64                                       5.14.0-570.32.1.el9_6                                appstream
kernel-devel.x86_64                                       5.14.0-570.32.1.el9_6                                devel    
kernel-devel-matched.x86_64                               5.14.0-570.32.1.el9_6                                appstream
kernel-devel-matched.x86_64                               5.14.0-570.32.1.el9_6                                devel

if you mean by browsing the repo, eg as in this link: Index of /pub/rocky/9/devel/x86_64/os/Packages/k/ then yes you can see them there. But they are not included in the repodata.

To use vault, you would have to create a repo file similar to this:

[appstream-9.5]
name=Rocky Linux 9.5 - AppStream
baseurl=http://dl.rockylinux.org/vault/rocky/9.5/AppStream/$basearch/os/
gpgcheck=1
enabled=1
countme=1
metadata_expire=6h
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Rocky-9

and then repeat for all previous releases. Then when I do this:

root@rocky9:~# dnf repolist | grep -i appstream
appstream                   Rocky Linux 9 - AppStream
appstream-9.5               Rocky Linux 9.5 - AppStream

and now when I search with dnf:

root@rocky9:~# dnf --enablerepo=devel list available kernel-devel* --showduplicates
Rocky Linux 9.5 - AppStream                                                             9.8 MB/s |  16 MB     00:01    
Last metadata expiration check: 0:00:05 ago on Sun 17 Aug 2025 10:55:29 AM CEST.
Available Packages
kernel-devel.x86_64                                     5.14.0-503.14.1.el9_5                              appstream-9.5
kernel-devel.x86_64                                     5.14.0-503.15.1.el9_5                              appstream-9.5
kernel-devel.x86_64                                     5.14.0-503.16.1.el9_5                              appstream-9.5
kernel-devel.x86_64                                     5.14.0-503.21.1.el9_5                              appstream-9.5
kernel-devel.x86_64                                     5.14.0-503.22.1.el9_5                              appstream-9.5
kernel-devel.x86_64                                     5.14.0-503.23.1.el9_5                              appstream-9.5
kernel-devel.x86_64                                     5.14.0-503.23.2.el9_5                              appstream-9.5
kernel-devel.x86_64                                     5.14.0-503.26.1.el9_5                              appstream-9.5
kernel-devel.x86_64                                     5.14.0-503.29.1.el9_5                              appstream-9.5
kernel-devel.x86_64                                     5.14.0-503.31.1.el9_5                              appstream-9.5
kernel-devel.x86_64                                     5.14.0-503.33.1.el9_5                              appstream-9.5
kernel-devel.x86_64                                     5.14.0-503.34.1.el9_5                              appstream-9.5
kernel-devel.x86_64                                     5.14.0-503.38.1.el9_5                              appstream-9.5
kernel-devel.x86_64                                     5.14.0-503.40.1.el9_5                              appstream-9.5
kernel-devel.x86_64                                     5.14.0-570.32.1.el9_6                              appstream    
kernel-devel.x86_64                                     5.14.0-570.32.1.el9_6                              devel        
kernel-devel-matched.x86_64                             5.14.0-503.14.1.el9_5                              appstream-9.5
kernel-devel-matched.x86_64                             5.14.0-503.15.1.el9_5                              appstream-9.5
kernel-devel-matched.x86_64                             5.14.0-503.16.1.el9_5                              appstream-9.5
kernel-devel-matched.x86_64                             5.14.0-503.21.1.el9_5                              appstream-9.5
kernel-devel-matched.x86_64                             5.14.0-503.22.1.el9_5                              appstream-9.5
kernel-devel-matched.x86_64                             5.14.0-503.23.1.el9_5                              appstream-9.5
kernel-devel-matched.x86_64                             5.14.0-503.23.2.el9_5                              appstream-9.5
kernel-devel-matched.x86_64                             5.14.0-503.26.1.el9_5                              appstream-9.5
kernel-devel-matched.x86_64                             5.14.0-503.29.1.el9_5                              appstream-9.5
kernel-devel-matched.x86_64                             5.14.0-503.31.1.el9_5                              appstream-9.5
kernel-devel-matched.x86_64                             5.14.0-503.33.1.el9_5                              appstream-9.5
kernel-devel-matched.x86_64                             5.14.0-503.34.1.el9_5                              appstream-9.5
kernel-devel-matched.x86_64                             5.14.0-503.38.1.el9_5                              appstream-9.5
kernel-devel-matched.x86_64                             5.14.0-503.40.1.el9_5                              appstream-9.5
kernel-devel-matched.x86_64                             5.14.0-570.32.1.el9_6                              appstream    
kernel-devel-matched.x86_64                             5.14.0-570.32.1.el9_6                              devel

you can clearly see here all the files listed from the appstream-9.5 repo file that I created. So that will now allow me to install them, and there is no 404 error. It would suggest if you are getting the 404 error, that you have something configured incorrectly.

Hey @iwalker, Thanks for the quick reply!

You’re right that dnf --enablerepo=devel list available kernel-devel* --showduplicates only shows the latest kernel with the command - that was my miss!, but it does not answer the question of ‘what happened to previous kernels?’.

kernel-devel-5.14.0-570.17.1.el9_6 for example is used in production (Amazon image for rocky9.6) but is no longer available through AppStream, and some of the other may be in the same boat, but I wasn’t able to verify that…

r.e the vault, I’m not interested in 9.5 kernels, only 9.6 - what I tried to add was:

[appstream-vault]
name=Rocky Linux 9.6 - AppStream Vault
baseurl=http://dl.rockylinux.org/vault/rocky/9.6/AppStream/$basearch/os/
gpgcheck=1
enabled=1
metadata_expire=6h
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Rocky-9

And here i get a 403 error, not a 404!
(As it’s a vault for the latest release, I guess this makes sense.)

So my question becomes, are the el9_6 kernels I listed not relevant for rocky 9.6? (except the latest, …570-32), if they are why are they not in AppStream? or where can I get them not manually?

Looking at the release page for the supported kernels for rocky9.6, It does indeed start from ..570-17.el9_7, but the relevant kernel-devel packages aren’t in AppStream.

Thanks again for the help, I really want to get to the bottom of this and I appreciate your help so far!

As I already said, previous kernels are moved to the vault. This is not Red Hat, whereby you can see and view older packages. You have to remember and realise, the only release is the current one, so 9.6. When 9.7 is released, all 9.6 packages will go to the vault. As I already explained. Only the latest packages exist, so when a current one is replaced, the previous version is no longer accessible. That’s how it is, accept the situation here. Otherwise, pay for RHEL and use that if you want historical packages accessible, or manually configure and use the vault.

You cannot use 9.6 from the vault, you can only use 9.5 or earlier so no wonder it doesn’t work. 9.6 is not in the vault.

This is what I was looking for, thanks for clarifying that it’s intended…

One last question, is it ‘safe’ to use the older kernel-devel packages from the ‘Devel’ repo? (those associated with rocky 9.6)

All I was looking for is clarity that only the latest kernel will be available, and where I can get older kernel-devel packages that I can use safely, if the answer is nowhere, or the ‘Devel’ repo - than that’s the way it is.

If you ensure that you do not enable devel globally, but only when you need to install packages, eg:

dnf --enablerepo=devel install <pkgname>

then it should be safe. Enabling it globally to install without providing the additional parameter apparently would not be safe based on the repo title. If a build/dev machine, then you could do it, but not on something being used in production.

Thanks! Just clarifying about the quote - do you mean not usable for production when enabling the repo?

Or that dnf --enablerepo=devel install should also not be used in production?

Enabling it globally, I mean doing:

dnf config-manager --enable devel

that would be OK for a dev/build system that you don’t really care about too much. For production, you should use the command:

dnf --enablerepo=devel

as and when you need to install packages, or update them.

Got it, Thanks for the help and patience!

1 Like