# RL 9.3 kernel-devel rpm incomplete? **URL:** \ **Category:** Rocky Linux Help & Support\ **Tags:** rocky-linux-9\ **Created:** [November 29, 2023, 2:18pm UTC](https://forums.rockylinux.org/t/rl-9-3-kernel-devel-rpm-incomplete/12025 "2023-11-29T14:18:36Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![felfert](https://sea2.discourse-cdn.com/flex020/user_avatar/forums.rockylinux.org/felfert/32/2259_2.png) [@felfert](https://forums.rockylinux.org/u/felfert)\ **Post date:** [November 29, 2023, 2:18pm UTC](https://forums.rockylinux.org/t/rl-9-3-kernel-devel-rpm-incomplete/12025/1 "2023-11-29T14:18:36Z")
Hi guys, After upgrading from RL 9.2 to RL 9.3, I can’t build any of my (proprietary) kernel modules anymore. The code itself is not the problem. It builds fine, **BUT** in the usual modpost phase, there is a tool named **resolve\_btfids** which normally gets build and then used to resolve exported symbols. It fails to build due to missing files in the source tree which should be provided by the kernel-devel package. Here is the output of the relevant tree on RL 9.3: ```auto rpm -ql kernel-devel | grep /tools/bpf /usr/src/kernels/5.14.0-362.8.1.el9_3.x86_64/tools/bpf /usr/src/kernels/5.14.0-362.8.1.el9_3.x86_64/tools/bpf/Makefile /usr/src/kernels/5.14.0-362.8.1.el9_3.x86_64/tools/bpf/bpftool /usr/src/kernels/5.14.0-362.8.1.el9_3.x86_64/tools/bpf/bpftool/Documentation /usr/src/kernels/5.14.0-362.8.1.el9_3.x86_64/tools/bpf/bpftool/Documentation/Makefile /usr/src/kernels/5.14.0-362.8.1.el9_3.x86_64/tools/bpf/bpftool/Makefile /usr/src/kernels/5.14.0-362.8.1.el9_3.x86_64/tools/bpf/resolve_btfids /usr/src/kernels/5.14.0-362.8.1.el9_3.x86_64/tools/bpf/resolve_btfids/Build /usr/src/kernels/5.14.0-362.8.1.el9_3.x86_64/tools/bpf/resolve_btfids/Makefile /usr/src/kernels/5.14.0-362.8.1.el9_3.x86_64/tools/bpf/resolve_btfids/main.c /usr/src/kernels/5.14.0-362.8.1.el9_3.x86_64/tools/bpf/runqslower /usr/src/kernels/5.14.0-362.8.1.el9_3.x86_64/tools/bpf/runqslower/Makefile ``` For comparison, here is the output of the relevant tree on RL 9.2 (of course from a different kernel version): ```auto rpm -ql kernel-devel | grep /tools/bpf /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/Makefile /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/bpftool /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/bpftool/Documentation /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/bpftool/Documentation/Makefile /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/bpftool/Makefile /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/.fixdep.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/Build /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/Makefile /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/fixdep /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/bpf_helper_defs.h /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/include /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/include/bpf /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/include/bpf/bpf.h /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/include/bpf/bpf_core_read.h /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/include/bpf/bpf_endian.h /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/include/bpf/bpf_helper_defs.h /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/include/bpf/bpf_helpers.h /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/include/bpf/bpf_tracing.h /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/include/bpf/btf.h /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/include/bpf/libbpf.h /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/include/bpf/libbpf_common.h /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/include/bpf/libbpf_legacy.h /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/include/bpf/libbpf_version.h /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/include/bpf/skel_internal.h /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/include/bpf/usdt.bpf.h /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/libbpf.a /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/staticobjs /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/staticobjs/.bpf.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/staticobjs/.bpf_prog_linfo.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/staticobjs/.btf.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/staticobjs/.btf_dump.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/staticobjs/.gen_loader.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/staticobjs/.hashmap.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/staticobjs/.libbpf.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/staticobjs/.libbpf_errno.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/staticobjs/.libbpf_probes.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/staticobjs/.linker.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/staticobjs/.netlink.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/staticobjs/.nlattr.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/staticobjs/.relo_core.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/staticobjs/.ringbuf.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/staticobjs/.str_error.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/staticobjs/.strset.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libbpf/staticobjs/.usdt.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libsubcmd /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libsubcmd/.exec-cmd.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libsubcmd/.help.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libsubcmd/.pager.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libsubcmd/.parse-options.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libsubcmd/.run-command.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libsubcmd/.sigchain.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libsubcmd/.subcmd-config.o.d /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/libsubcmd/libsubcmd.a /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/main.c /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/resolve_btfids/resolve_btfids /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/runqslower /usr/src/kernels/5.14.0-284.30.1.el9_2.x86_64/tools/bpf/runqslower/Makefile ``` This indicates to me, that there are a lot of files missing in kernel-devel-5.14.0-362.8.1.el9\_3.x86\_64 Or am I missing something? Thanks -Fritz ---
**Author:** ![felfert](https://sea2.discourse-cdn.com/flex020/user_avatar/forums.rockylinux.org/felfert/32/2259_2.png) [@felfert](https://forums.rockylinux.org/u/felfert)\ **Post date:** [November 29, 2023, 4:07pm UTC](https://forums.rockylinux.org/t/rl-9-3-kernel-devel-rpm-incomplete/12025/2 "2023-11-29T16:07:11Z")
Examining the kernel.spec file, I could find: ```auto * Wed Jun 28 2023 Jan Stancek [5.14.0-333.el9] ... - spec: remove resolve_btfids from kernel-devel (Viktor Malik) [2178930] - tools/resolve_btfids: Fix setting HOSTCFLAGS (Viktor Malik) [2178930] - tools/resolve_btfids: Pass HOSTCFLAGS as EXTRA_CFLAGS to prepare targets (Viktor Malik) [2178930] - tools/resolve_btfids: Tidy HOST_OVERRIDES (Viktor Malik) [2178930] - tools/resolve_btfids: Compile resolve_btfids as host program (Viktor Malik) [2178930] - tools/resolve_btfids: Alter how HOSTCC is forced (Viktor Malik) [2178930] - tools/resolve_btfids: Install subcmd headers (Viktor Malik) [2178930] ``` Assuming, that the number in square brackets is a bugzilla ID, I went to [bugzilla.redhat.com](http://bugzilla.redhat.com) and looked for that, but I don’t have permission to view that bug ☹ This begs the question about the rationale for this change. So: Anyone with more permissions on bugzilla can explain **WHY** they decided to remove that? ---
**Author:** ![jlehtone](https://avatars.discourse-cdn.com/v4/letter/j/e9a140/32.png) [@jlehtone](https://forums.rockylinux.org/u/jlehtone)\ **Post date:** [November 29, 2023, 4:36pm UTC](https://forums.rockylinux.org/t/rl-9-3-kernel-devel-rpm-incomplete/12025/3 "2023-11-29T16:36:01Z")
> [@felfert](#): > > I don’t have permission to view that bug Aren’t all kernel-bugs non-public? ---
**Author:** ![Ritov](https://avatars.discourse-cdn.com/v4/letter/r/3ec8ea/32.png) [@Ritov](https://forums.rockylinux.org/u/Ritov)\ **Post date:** [November 29, 2023, 5:21pm UTC](https://forums.rockylinux.org/t/rl-9-3-kernel-devel-rpm-incomplete/12025/4 "2023-11-29T17:21:31Z")
@felfert you could open a bug report in rh’s bugzilla and explain your use case as generic issue and the necessity of the mentioned files … ---
**Author:** ![label](https://avatars.discourse-cdn.com/v4/letter/l/c5a1d2/32.png) [@label](https://forums.rockylinux.org/u/label)\ **Post date:** [November 29, 2023, 5:37pm UTC](https://forums.rockylinux.org/t/rl-9-3-kernel-devel-rpm-incomplete/12025/5 "2023-11-29T17:37:44Z")
What it sounds like is you need to run `make prepare`. I would look at these links: - [spec: fix building and packaging of resolve\_btfids (!2490) · Merge requests · cki-project / kernel-ark · GitLab](https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2490) - [https://www.spinics.net/linux/fedora/fedora-kernel/msg17547.html](https://www.spinics.net/linux/fedora/fedora-kernel/msg17547.html) ---
**Author:** ![felfert](https://sea2.discourse-cdn.com/flex020/user_avatar/forums.rockylinux.org/felfert/32/2259_2.png) [@felfert](https://forums.rockylinux.org/u/felfert)\ **Post date:** [November 30, 2023, 7:55am UTC](https://forums.rockylinux.org/t/rl-9-3-kernel-devel-rpm-incomplete/12025/6 "2023-11-30T07:55:51Z")
Thanks a lot for those links. That explains it. However I disagree with the reasoning given there: - After all, the kernel-devel package is not a noarch package and therefore any tools can stay in there. - Running `make prepare` in the build tree rebuilds **all** tools and - as a side effct - recreates the kernel’s config which is a no-go if one wants to make shure a thirdparty module is built against an **unmodified** kernel. - It also happens, that make prepare currently fails earlier in building objtool. =\> So they either should revert that patch or add the tools into another package. Removing them completely is not a valid option. =\> Will file a bug on bugzilla… ---
**Author:** ![felfert](https://sea2.discourse-cdn.com/flex020/user_avatar/forums.rockylinux.org/felfert/32/2259_2.png) [@felfert](https://forums.rockylinux.org/u/felfert)\ **Post date:** [November 30, 2023, 8:08am UTC](https://forums.rockylinux.org/t/rl-9-3-kernel-devel-rpm-incomplete/12025/7 "2023-11-30T08:08:00Z")
As an interim solution, I have disabled the BTF step in the modpost phase. This can be easily done by NOT providing an uncompressed vmlinux in the toplevel build tree. The build then prints an additional hint: `Skipping BTF generation for /.ko due to unavailability of vmlinux` ---
**Author:** ![system](https://us1.discourse-cdn.com/flex020/uploads/rockylinux/original/1X/91b7219eec10e30013422e4df76c1d898711a5d5.svg) [@system](https://forums.rockylinux.org/u/system)\ **Post date:** [January 29, 2024, 8:08am UTC](https://forums.rockylinux.org/t/rl-9-3-kernel-devel-rpm-incomplete/12025/8 "2024-01-29T08:08:28Z")
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.