Dnf update failing again

Doing my regular check for updates (I know I can automate this, but I prefer manually as that way it’s clearer to me what changed on each server) I get the following in response to dnf upgrade:

DigitalOcean Agent                                                                                                           43 kB/s | 3.3 kB     00:00
DigitalOcean Droplet Agent                                                                                                   49 kB/s | 3.3 kB     00:00
Error:
 Problem: cannot install the best update candidate for package fwupd-plugin-flashrom-1.8.16-1.el9.rocky.0.2.x86_64
  - nothing provides fwupd(x86-64) = 1.8.16-1.el9.rocky.0.3 needed by fwupd-plugin-flashrom-1.8.16-1.el9.rocky.0.3.x86_64 from appstream
(try to add '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)

Is this an issue with something on this box, and so I should know how to fix it, or something with the latest dependency chain (and so I am just supposed to report it)?

Thanks as always for all help,
Paul

Added: a similar box also at Digital Ocean succeeds in its regular update (with a new kernel). Both boxes were last updated about five days ago.

New Installed Modules (also 15 upgraded not shown here):
kernel-5.14.0-362.24.1.el9_3.0.1.x86_64
kernel-core-5.14.0-362.24.1.el9_3.0.1.x86_64
kernel-modules-5.14.0-362.24.1.el9_3.0.1.x86_64
kernel-modules-core-5.14.0-362.24.1.el9_3.0.1.x86_64

I would try to run dnf clean all on the box that is having issues and then try to update again. If you’re having issues still, check dnf repolist -v to see where your repos are pointing to. If they are pointing to a digital ocean url, it will be an issue with how they manage their mirrors of our repositories.

[paul@ironwood ~]$ sudo dnf clean all
69 files removed
[paul@ironwood ~]$ sudo dnf upgrade
Netdata                                                                                                                     282 kB/s | 155 kB     00:00
Netdata Repository Config                                                                                                   3.8 kB/s | 1.6 kB     00:00
DigitalOcean Agent                                                                                                           20 kB/s | 6.1 kB     00:00
DigitalOcean Droplet Agent                                                                                                   11 kB/s | 3.2 kB     00:00
Extra Packages for Enterprise Linux 9 - x86_64                                                                               40 MB/s |  21 MB     00:00
Extra Packages for Enterprise Linux 9 openh264 (From Cisco) - x86_64                                                        3.9 kB/s | 2.5 kB     00:00
Rocky Linux 9 - BaseOS                                                                                                      5.4 MB/s | 2.2 MB     00:00
Rocky Linux 9 - AppStream                                                                                                    12 MB/s | 7.4 MB     00:00
Rocky Linux 9 - Extras                                                                                                       38 kB/s |  14 kB     00:00
Error:
 Problem: package fwupd-plugin-flashrom-1.8.16-1.el9.rocky.0.2.x86_64 from @System requires fwupd(x86-64) = 1.8.16-1.el9.rocky.0.2, but none of the providers can be installed
  - cannot install both fwupd-1.8.16-1.el9.rocky.0.3.x86_64 from baseos and fwupd-1.8.16-1.el9.rocky.0.2.x86_64 from @System
  - cannot install the best update candidate for package fwupd-plugin-flashrom-1.8.16-1.el9.rocky.0.2.x86_64
  - cannot install the best update candidate for package fwupd-1.8.16-1.el9.rocky.0.2.x86_64
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
[paul@ironwood ~]$
[paul@ironwood ~]$ sudo dnf repolist -v
Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, groups-manager, needs-restarting, playground, repoclosure, repodiff, repograph, repomanage, reposync, system-upgrade
DNF version: 4.14.0
cachedir: /var/cache/dnf
Last metadata expiration check: 0:02:36 ago on Mon 08 Apr 2024 11:59:19 PM UTC.
Repo-id            : appstream
Repo-name          : Rocky Linux 9 - AppStream
Repo-revision      : 1710191856
Repo-updated       : Mon 11 Mar 2024 09:17:36 PM UTC
Repo-pkgs          : 5,659
Repo-available-pkgs: 5,524
Repo-size          : 7.6 G
Repo-mirrors       : https://mirrors.rockylinux.org/mirrorlist?arch=x86_64&repo=AppStream-9
Repo-baseurl       : https://mirror.reenigne.net/rocky/9.3/AppStream/x86_64/os/ (45 more)
Repo-expire        : 21,600 second(s) (last: Mon 08 Apr 2024 11:59:17 PM UTC)
Repo-filename      : /etc/yum.repos.d/rocky.repo

Repo-id            : baseos
Repo-name          : Rocky Linux 9 - BaseOS
Repo-revision      : 1712331641
Repo-updated       : Fri 05 Apr 2024 03:40:41 PM UTC
Repo-pkgs          : 1,157
Repo-available-pkgs: 1,157
Repo-size          : 1.2 G
Repo-mirrors       : https://mirrors.rockylinux.org/mirrorlist?arch=x86_64&repo=BaseOS-9
Repo-baseurl       : https://rocky-linux-northamerica-northeast2.production.gcp.mirrors.ctrliq.cloud/pub/rocky//9.3/BaseOS/x86_64/os/ (45 more)
Repo-expire        : 21,600 second(s) (last: Mon 08 Apr 2024 11:59:15 PM UTC)
Repo-filename      : /etc/yum.repos.d/rocky.repo

Repo-id            : digitalocean-agent
Repo-name          : DigitalOcean Agent
Repo-revision      : 1695057921
Repo-updated       : Mon 18 Sep 2023 05:25:21 PM UTC
Repo-pkgs          : 36
Repo-available-pkgs: 36
Repo-size          : 187 M
Repo-baseurl       : https://repos.insights.digitalocean.com/yum/do-agent/x86_64
Repo-expire        : 300 second(s) (last: Mon 08 Apr 2024 11:59:02 PM UTC)
Repo-filename      : /etc/yum.repos.d/digitalocean-agent.repo

Repo-id            : droplet-agent
Repo-name          : DigitalOcean Droplet Agent
Repo-revision      : 1695281092
Repo-updated       : Thu 21 Sep 2023 07:24:53 AM UTC
Repo-pkgs          : 15
Repo-available-pkgs: 15
Repo-size          : 32 M
Repo-baseurl       : https://repos-droplet.digitalocean.com/yum/droplet-agent/x86_64
Repo-expire        : 300 second(s) (last: Mon 08 Apr 2024 11:59:02 PM UTC)
Repo-filename      : /etc/yum.repos.d/droplet-agent.repo

Repo-id            : epel
Repo-name          : Extra Packages for Enterprise Linux 9 - x86_64
Repo-revision      : 1712540920
Repo-updated       : Mon 08 Apr 2024 01:49:36 AM UTC
Repo-pkgs          : 20,821
Repo-available-pkgs: 20,821
Repo-size          : 18 G
Repo-metalink      : https://mirrors.fedoraproject.org/metalink?repo=epel-9&arch=x86_64&infra=genclo&content=pub/rocky
  Updated          : Mon 08 Apr 2024 11:59:03 PM UTC
Repo-baseurl       : http://mirror.xenyth.net/epel/9/Everything/x86_64/ (171 more)
Repo-expire        : 172,800 second(s) (last: Mon 08 Apr 2024 11:59:03 PM UTC)
Repo-filename      : /etc/yum.repos.d/epel.repo

Repo-id            : epel-cisco-openh264
Repo-name          : Extra Packages for Enterprise Linux 9 openh264 (From Cisco) - x86_64
Repo-revision      : 1684772361
Repo-updated       : Mon 22 May 2023 04:19:21 PM UTC
Repo-pkgs          : 4
Repo-available-pkgs: 4
Repo-size          : 904 k
Repo-metalink      : https://mirrors.fedoraproject.org/metalink?repo=epel-cisco-openh264-9&arch=x86_64
  Updated          : Mon 08 Apr 2024 11:59:15 PM UTC
Repo-baseurl       : https://codecs.fedoraproject.org/openh264/epel/9/x86_64/os/ (0 more)
Repo-expire        : 1,209,600 second(s) (last: Mon 08 Apr 2024 11:59:15 PM UTC)
Repo-filename      : /etc/yum.repos.d/epel-cisco-openh264.repo

Repo-id            : extras
Repo-name          : Rocky Linux 9 - Extras
Repo-revision      : 1708531863
Repo-updated       : Wed 21 Feb 2024 04:11:03 PM UTC
Repo-pkgs          : 46
Repo-available-pkgs: 46
Repo-size          : 3.0 M
Repo-mirrors       : https://mirrors.rockylinux.org/mirrorlist?arch=x86_64&repo=extras-9
Repo-baseurl       : https://rocky-linux-northamerica-northeast1.production.gcp.mirrors.ctrliq.cloud/pub/rocky//9.3/extras/x86_64/os/ (45 more)
Repo-expire        : 21,600 second(s) (last: Mon 08 Apr 2024 11:59:19 PM UTC)
Repo-filename      : /etc/yum.repos.d/rocky-extras.repo

Repo-id            : netdata
Repo-name          : Netdata
Repo-revision      : 1711987306
Repo-updated       : Mon 01 Apr 2024 04:01:46 PM UTC
Repo-pkgs          : 217
Repo-available-pkgs: 217
Repo-size          : 797 M
Repo-baseurl       : https://repo.netdata.cloud/repos/stable/el/9/x86_64
Repo-expire        : 172,800 second(s) (last: Mon 08 Apr 2024 11:59:01 PM UTC)
Repo-filename      : /etc/yum.repos.d/netdata.repo

Repo-id            : netdata-repoconfig
Repo-name          : Netdata Repository Config
Repo-revision      : 1711969898
Repo-updated       : Mon 01 Apr 2024 11:11:38 AM UTC
Repo-pkgs          : 6
Repo-available-pkgs: 6
Repo-size          : 42 k
Repo-baseurl       : https://repo.netdata.cloud/repos/repoconfig/el/9/x86_64
Repo-expire        : 172,800 second(s) (last: Mon 08 Apr 2024 11:59:01 PM UTC)
Repo-filename      : /etc/yum.repos.d/netdata.repo
Total packages: 27,961

I’m afraid I’m not familiar enough with how this is supposed to look to understand what could be wrong.

I’m happy to open a ticket with Digital Ocean if I can describe the issue to them in an intelligible way!

Looking at it, it looks like it may not be a digital ocean issue. I figured they may have used their own mirrors internally.

Looks like the mirror chosen for baseos doesn’t seem to have the fwupd updates. You could try to do a dnf clean all and try again… Otherwise you could modify /etc/yum.repos.d/rocky.conf and use just the baseurl.

In the meanwhile, I’ll do some digging on how in sync some of our mirrors are, since you had one system that seemed to update just fine.

Sorry that you are having this issue!

On such conflicts I usually look at from which repo the packages come from. For example:

dnf list --showduplicates fwupd\*

Then I check whether I can uninstall some/all of the packages without cascade of other packages leaving due to dependency.
Usually it has been possible to reinstall the removed packages as it is now a “fresh addition”.

Obviously, if the repo (mirrors) offer conflicting versions, then that has to be sorted out before reinstall.

Is it normal/expected that the version number formats are somewhat rearranged in the reported package names?

I see

Installed Packages
fwupd.x86_64                                                                 1.8.16-1.el9.rocky.0.2                                                  @baseos
fwupd-plugin-flashrom.x86_64                                                 1.8.16-1.el9.rocky.0.2                                                  @appstream
Available Packages
fwupd.x86_64                                                                 1.8.16-1.el9.rocky.0.3                                                  baseos
fwupd-plugin-flashrom.x86_64                                                 1.8.16-1.el9.rocky.0.3                                                  appstream

But the actual missing package report says

Error:
 Problem: cannot install the best update candidate for package fwupd-plugin-flashrom-1.8.16-1.el9.rocky.0.2.x86_64
  - nothing provides fwupd(x86-64) = 1.8.16-1.el9.rocky.0.3 needed by fwupd-plugin-flashrom-1.8.16-1.el9.rocky.0.3.x86_64 from appstream

This is the same version but in a different format than what’s available.

Available: fwupd.x86_64 1.8.16-1.el9.rocky.0.3
Needed: fwupd(x86-64) = 1.8.16-1.el9.rocky.0.3

Is this usual?

PS: I have tried again this morning; the problem persists.

Success. I tried a clean all / update combination one more time in despair and this time it worked.

Interestingly, it simply reported nothing to do, complete (!)

Don’t know if this is a result of a mirror being corrected / brought up to date, a different mirror being selected, or something at my end that got resolved by my twiddling. Whatever it is, at least my updates work again!

Yes.

The dnf list does show three columns of data:

name.arch     version     [@]repoid

and here:

name:    fwupd-plugin-flashrom
arch:    x86_64
version: 1.8.16-1.el9.rocky.0.2
repoid:  appstream

but in error message (and some other places) the format is:

name-version.arch