Fstab nfs share entry refuses to mount on reboot/boot; mount -a required

Rocky Linux Help & Support
1

Hi-

Host 1- Rocky 9.2 with autofs (Linux automount version 5.1.7-58.el9) running. Excluded one nfs share from autofs conf file and put it in /etc/fstab. The nfs share entry in /etc/fstab mounts on boot/reboot just fine. The /etc/fstab syntax:
export.host.com:/path/on/export /home/users nfs defaults,vers=3 0 0

Autofs running for different shares on same host and no issues.

Host 2- Rocky 9.5 with autofs (Linux automount version 5.1.7-58.el9) running. Excluded one nfs share from autofs conf file and put it in /etc/fstab. The nfs share entry in /etc/fstab FAILS to mount on boot/reboot. Syntax:
export.host.com:/path/on/export /home/users nfs defaults,vers=3 0 0

Share refuses to mount on boot/reboot. User needs to run command “mount -a” after reboot and share mounts. I have tried to add:
nfs defaults,vers=3,x-systemd.requires=network-online.target,_netdev,nofail

but nothing works.

on Host 2 /var/log/messages:
Jul 24 23:10:26 Host 2 systemd[1]: Finished Network Manager Wait Online.
Jul 24 23:10:26 Host 2 systemd[1]: Reached target Network is Online.
Jul 24 23:10:26 Host 2 systemd[1]: Mounting /home/users…
Jul 24 23:10:26 Host 2 systemd[1]: Starting Login and scanning of iSCSI devices…
Jul 24 23:10:26 Host 2 systemd[1]: Starting Notify NFS peers of a restart…
Jul 24 23:10:26 Host 2 systemd[1]: Starting System Logging Service…
Jul 24 23:10:26 Host 2 sm-notify[1856]: Version 2.5.4 starting
Jul 24 23:10:26 Host 2 systemd[1]: Started Notify NFS peers of a restart.
Jul 24 23:10:26 Host 2 mount[1855]: mount.nfs: Failed to resolve server export.host.com: Name or service not known
Jul 24 23:10:26 Host 2 iscsiadm[1854]: iscsiadm: No records found
Jul 24 23:10:26 Host 2 systemd[1]: home-users.mount: Mount process exited, code=exited, status=32/n/a
Jul 24 23:10:26 Host 2 systemd[1]: home-users.mount: Failed with result ‘exit-code’.
Jul 24 23:10:26 Host 2 systemd[1]: Failed to mount /home/users.

But immediately after boot, mount -a mounts that share. That share pings and resolves via nslookup from the same host.

What is going on here? Thank for your assistance.

EDIT: I added an IP from the export.host.com to /etc/hosts. Still cannot mount on boot. Mount -a post-boot works as before. The error in /var/log/messages is different now:
Jul 24 23:25:54 Host 2 systemd[1]: Mounting /home/users…
Jul 24 23:25:54 Host 2 systemd[1]: Starting Login and scanning of iSCSI devices…
Jul 24 23:25:54 Host 2 systemd[1]: Starting Notify NFS peers of a restart…
Jul 24 23:25:54 Host 2 systemd[1]: Starting System Logging Service…
Jul 24 23:25:54 Host 2 sm-notify[1852]: Version 2.5.4 starting
Jul 24 23:25:54 Host 2 systemd[1]: Started Notify NFS peers of a restart.
Jul 24 23:25:54 Host 2 iscsiadm[1850]: iscsiadm: No records found
Jul 24 23:25:54 Host 2 systemd[1]: Finished Login and scanning of iSCSI devices.
Jul 24 23:25:54 Host 2 systemd[1]: Reached target Remote File Systems.
Jul 24 23:25:54 Host 2 systemd[1]: Starting Crash recovery kernel arming…
Jul 24 23:25:54 Host 2 systemd[1]: TPM2 PCR Barrier (User) was skipped because of an unmet condition check (ConditionPathExists=/sys/firmware/efi/efivars/StubPcrKernelImage-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f).
Jul 24 23:25:54 Host 2 systemd[1]: Starting Permit User Sessions…
Jul 24 23:25:54 Host 2 systemd[1]: Finished Permit User Sessions.
Jul 24 23:25:54 Host 2 systemd[1]: Reached target Host and Network Name Lookups.
Jul 24 23:25:54 Host 2 systemd[1]: Started Deferred execution scheduler.
Jul 24 23:25:54 Host 2 systemd[1]: Started Command Scheduler.
Jul 24 23:25:54 Host 2 systemd[1]: Starting GNOME Display Manager…
Jul 24 23:25:54 Host 2 systemd[1]: Starting Hold until boot process finishes up…
Jul 24 23:25:54 Host 2 systemd[1]: Starting NFS status monitor for NFSv2/3 locking…
Jul 24 23:25:54 Host 2 kdumpctl[1868]: kdump: No memory reserved for crash kernel
Jul 24 23:25:54 Host 2 kdumpctl[1868]: kdump: Starting kdump: [FAILED]
Jul 24 23:25:54 Host 2 systemd[1]: kdump.service: Main process exited, code=exited, status=1/FAILURE
Jul 24 23:25:54 Host 2 systemd[1]: kdump.service: Failed with result ‘exit-code’.
Jul 24 23:25:54 Host 2 systemd[1]: Failed to start Crash recovery kernel arming.
Jul 24 23:25:54 Host 2 rpc.statd[1875]: Version 2.5.4 starting
Jul 24 23:25:54 Host 2 rpc.statd[1875]: Flags: TI-RPC
Jul 24 23:25:54 Host 2 rsyslogd[1853]: [origin software=“rsyslogd” swVersion=“8.2310.0-4.el9” x-pid=“1853” x-info=“https://www.rsyslog.com”] start
Jul 24 23:25:54 Host 2 systemd[1]: Started System Logging Service.
Jul 24 23:25:54 Host 2 systemd[1]: Started GNOME Display Manager.
Jul 24 23:25:54 Host 2 systemd[1]: Started NFS status monitor for NFSv2/3 locking..
Jul 24 23:25:54 Host 2 mount[1884]: Created symlink /run/systemd/system/remote-fs.target.wants/rpc-statd.service → /usr/lib/systemd/system/rpc-statd.service.
Jul 24 23:25:54 Host 2 systemd[1]: Reloading.
Jul 24 23:25:54 Host 2 rsyslogd[1853]: imjournal: journal files changed, reloading… [v8.2310.0-4.el9 try https://www.rsyslog.com/e/0 ]
Jul 24 23:25:54 Host 2 systemd-rc-local-generator: /etc/rc.d/rc.local is not marked executable, skipping.
Jul 24 23:25:54 Host 2 systemd[1]: /usr/lib/systemd/system/falcon-sensor.service:12: PIDFile= references a path below legacy directory /var/run/, updating /var/run/falcond.pid → /run/falcond.pid; please update the unit file accordingly.
Jul 24 23:25:54 Host 2 systemd[1]: Finished Hold until boot process finishes up.
Jul 24 23:25:54 Host 2 mount[1851]: mount.nfs: Network is unreachable
Jul 24 23:25:54 Host 2 systemd[1]: Started PCoIP Agent.
Jul 24 23:25:54 Host 2 systemd[1]: home-users.mount: Mount process exited, code=exited, status=32/n/a
Jul 24 23:25:54 Host 2 systemd[1]: home-users.mount: Failed with result ‘exit-code’.
Jul 24 23:25:54 Host 2 systemd[1]: Failed to mount /home/users.

“Network is unreachable”? I am at a loss.

2

It seems that when the mount.nfs runs, the network is not properly up yet. (Hence no access to the DNS and NFS servers.)

Look at the network configuration. What differs there?

How about:

nfs defaults,vers=3,noauto,nofail,x-systemd.automount,x-systemd.idle-timeout=300 0 0

That would use automounting (with systemd, rather than autofs), i.e. mount on need only, which should be after boot.

Currently the only supported Rocky 9 is based on 9.6. Can you reproduce the error after dnf up?

3

Host 2’s internal/local OS network config is identical to Host 1.

We are version-locked in a corp. studio env. OS version cannot be updated at this time. autotmount is not an option for this share (user homedir) as other team members want to rule-out automount/autofs potential issues with user homedirs.

We had a similar issue in 2020 with some hosts not mounting /etc/fstab nfs shares on boot. Our network engineer mentioned something about “spanning tree” and made some networking change (external to our hosts)- thereafter, the nfs shares in /etc/fstab did mount on boot. I am not familiar with “spanning tree” or what change he made. We are now using autofs for all NFS shares except the share in question. I am waiting to hear back from our network engineer re: whether whatever “spanning tree” change is still in place or whether it is missing for the routing Host 2 uses.

4

I’ve seen “spanning tree” mentioned with network switches. (This includes the bridge.stp option on bridges.)

If you connect three switches to form a cycle: A-B, B-C, C-A, and then send broadcast packet to A, A will forward to B (and C), B forwards to C, C forwards to A, … a “storm”.
Switches may use spanning tree protocol (STP) to disarm cycles. Switches do send STP packets to their neighbours and if they spot a cycle, then they agree to not use some links. (E.g. A and C auto-disable link between them and talk only via B.

Linux software bridges do support STP too. It is usually disabled on virtualization hosts as the admin of the host promises to not create cycles with bridges.

The software bridge has an another option: bridge.forward-delay. That is, if new traffic/client shows up, the bridge delays sending it forward. (There is no point to forward across bridge if receiver is on the same side as the sender.) There is no delay when sender and receiver are known to be on different sides of bridge.

I’ve had the delay hit me. New VM guest install starts on libvirt/KVM and requests PXEboot data. Bridge between VM and DHCP/TFTP server delays. Request timed out. However, that was many versions ago.

5

OK- problem solved. Dell 5860 was connected via 10G nic, not the 1G nic (host as two nics). Connecting to the 1G nic and viola- nfs share now mounts on boot. Interesting that the 10G nic still provided networking, but only post-boot. Anyway- sorry for the confusion!