I've done something daft (TM) [and hosed my network connection]

Rocky Linux General
1

With great embarrassment I admit to mixing my repos - I thought I’d been very cleaver (TM, another) with a whole lot of cascading priorities in my repo files that were intended to ensure nothing went wrong…

In an attempt to install a pika-backup rpm from Mandriva on my RL 8.7 little home server running on an old laptop, I allowed a raft of f36 rpms to install - I did wonder if I would live to regret it… (I’ve actually just run uname and discovered my kernel is fc32…)

The first symptom was actually a regular pika backup failing over ssh from my workday laptop. Trying to login over ssh I got refused:

$ ssh admin@frontserver
Connection closed by 192.168.0.20 port 22
$ ssh admin@192.168.0.20
Connection closed by 192.168.0.20 port 22
$

So, I tried from the server - I can ssh locally from one account to another if I use the IP address, but the name doesn’t resolve (my local router is set up to do that). I can ping 1.1.1.1 but not one.one.one.one .
Interestingly, I get a different result from frontserver and frontserver.lan and the second is similar, but a bit more interesting, to using ssh from my workday laptop:

img_20230306_234015_crop
img_20230306_234015_crop1920×1079 214 KB

I figured the easiest way out of this mess was to dnf history rollback , but alas none of the repos resolve. Rollback seems the best solution to me - but, I’m open to suggestions (nothing rude please) - so, my question is, what’s the easiest way to get dnf to work without DNS resolution - I started replacing domain names with IPs in /etc/yum.repos.d , but immediately ran into difficulty with https and ssl certification.

Thanks for listening,
M

2

Get the network connections and DNS resolution back up, manually?

You do have IP address, know the router’s IP address, and for DNS you could use the google (8.8.8.8)?

3

Thanks @jlehtone I’m sure the problems is internal to the server rather than external DNS configuration - I did enter 1.1.1.1 and 2606:4700:4700:0:0:0:0:1111 manually, but it’s made no difference - I have a few services running on this via podman (squeeze server and nextcloud) and they’re continuing to perform as expected - as is cockpit. So, it’s the terminal, ssh etc that’s somehow getting blocked from inside the server.

What would be the best way to run dnf without ssl and dns?

4

If openssl and dnf are corrupted, then copy of select RPMs and install with rpm might still be on the table.

The name resolution is essentially in /etc/resolv.conf. Is that sane?

You do have a backup of user data and procedure to easily redeploy config for server&services?

5

Well… I do have a back up - but, it’s so long since I set it up it’d take me longer to workout what I did than solve the name resolution problem (I think) - that was the whole reason for trying to get pika on to the server…

As for /etc/resolv.conf, it’s pointing at my local router still - which is because although I entered the external servers, I didn’t take it off ‘auto’ so still using the DHCP assigned DNS - but, I just went to change that in Cockpit (I was using nmtui on the server previously) and Cockpit report that I’d break my connection to the server with the change and simultaneously my Radio 6 listen pleasure via Squeeze Server stopped… So, I don’t think I want to change that…

6

Can you try dig @1.1.1.1 google.com? What’s the result then?

Another thing you can try is using option -vvv with ssh to see the debug output.

7

Thanks @zaffiro, here’s the outputs of dig and various permutaions of ssh -vvv. To me, the difference between $ ssh -vvv root@frontserver and $ ssh -vvv root@frontserver.lan look most interesting - adding my local TLD seems to cause the dns querry to go outside the box and back and return a Connection closed response (cf. the result from my machine) instead of Could not resolve hostname like the rest (which is as it should be):

$ dig @1.1.1.1 google.com 
[admin@frontserver yum.repos.d]$ dig @1.1.1.1 google.com

; <<>> DiG 9.11.36-RedHat-9.11.36-5.el8_7.2 <<>> @1.1.1.1 google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 6634
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;google.com.                    IN      A

;; ANSWER SECTION:
google.com.             62      IN      A       142.250.180.14

;; Query time: 10 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Mon Mar 13 20:51:41 GMT 2023
;; MSG SIZE  rcvd: 55

[admin@frontserver yum.repos.d]$
$ ssh -vvv root@frontserver 
[admin@frontserver yum.repos.d]$ ssh -vvv root@frontserver
OpenSSH_8.0p1, OpenSSL 1.1.1k  FIPS 25 Mar 2021
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 52: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug2: checking match for 'final all' host frontserver originally frontserver
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: not matched 'final'
debug2: match not found
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1]
debug1: configuration requests final Match pass
debug1: re-parsing configuration
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 52: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug2: checking match for 'final all' host frontserver originally frontserver
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: matched 'final'
debug2: match found
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1]
debug2: resolving "frontserver" port 22
ssh: Could not resolve hostname frontserver: Name or service not known
[admin@frontserver yum.repos.d]$
$ ssh -vvv root@frontserver.lan 
[admin@frontserver yum.repos.d]$ ssh -vvv root@frontserver.lan
OpenSSH_8.0p1, OpenSSL 1.1.1k  FIPS 25 Mar 2021
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 52: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug2: checking match for 'final all' host frontserver.lan originally frontserver.lan
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: not matched 'final'
debug2: match not found
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1]
debug1: configuration requests final Match pass
debug1: re-parsing configuration
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 52: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug2: checking match for 'final all' host frontserver.lan originally frontserver.lan
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: matched 'final'
debug2: match found
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1]
debug2: resolving "frontserver.lan" port 22
debug2: ssh_connect_direct
debug1: Connecting to frontserver.lan [fe80::6875:11ed:760:a836%nm-bridge1] port 22.
debug1: Connection established.
debug1: identity file /home/admin/.ssh/id_rsa type -1
debug1: identity file /home/admin/.ssh/id_rsa-cert type -1
debug1: identity file /home/admin/.ssh/id_dsa type -1
debug1: identity file /home/admin/.ssh/id_dsa-cert type -1
debug1: identity file /home/admin/.ssh/id_ecdsa type -1
debug1: identity file /home/admin/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/admin/.ssh/id_ed25519 type -1
debug1: identity file /home/admin/.ssh/id_ed25519-cert type -1
debug1: identity file /home/admin/.ssh/id_xmss type -1
debug1: identity file /home/admin/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.0
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.0
debug1: match: OpenSSH_8.0 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to frontserver.lan:22 as 'root'
debug3: hostkeys_foreach: reading file "/home/admin/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /home/admin/.ssh/known_hosts:4
debug3: load_hostkeys: loaded 1 keys from frontserver.lan
debug3: hostkeys_foreach: reading file "/etc/ssh/ssh_known_hosts"
debug3: order_hostkeyalgs: have matching best-preference key type ecdsa-sha2-nistp256-cert-v01@openssh.com, using HostkeyAlgorithms verbatim
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc
debug2: ciphers stoc: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc
debug2: MACs ctos: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
debug2: MACs stoc: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc
debug2: ciphers stoc: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc
debug2: MACs ctos: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
debug2: MACs stoc: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: aes256-gcm@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: aes256-gcm@openssh.com MAC: <implicit> compression: none
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:wiTwrVZx1f3CbXSsWozaex5YDQbCoAS1eYkKMyO6IAo
debug3: hostkeys_foreach: reading file "/home/admin/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /home/admin/.ssh/known_hosts:4
debug3: load_hostkeys: loaded 1 keys from frontserver.lan
debug3: hostkeys_foreach: reading file "/etc/ssh/ssh_known_hosts"
debug3: hostkeys_foreach: reading file "/home/admin/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /home/admin/.ssh/known_hosts:4
debug3: load_hostkeys: loaded 1 keys from fe80::6875:11ed:760:a836%nm-bridge1
debug3: hostkeys_foreach: reading file "/etc/ssh/ssh_known_hosts"
debug1: Host 'frontserver.lan' is known and matches the ECDSA host key.
debug1: Found key in /home/admin/.ssh/known_hosts:4
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 4294967296 blocks
debug1: Will attempt key: /home/admin/.ssh/id_rsa 
debug1: Will attempt key: /home/admin/.ssh/id_dsa 
debug1: Will attempt key: /home/admin/.ssh/id_ecdsa 
debug1: Will attempt key: /home/admin/.ssh/id_ed25519 
debug1: Will attempt key: /home/admin/.ssh/id_xmss 
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
Connection closed by fe80::6875:11ed:760:a836%nm-bridge1 port 22
[admin@frontserver yum.repos.d]$
$ ssh -vvv root@localhost 
[admin@frontserver yum.repos.d]$ ssh -vvv root@localhost
OpenSSH_8.0p1, OpenSSL 1.1.1k  FIPS 25 Mar 2021
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 52: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug2: checking match for 'final all' host localhost originally localhost
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: not matched 'final'
debug2: match not found
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1]
debug1: configuration requests final Match pass
debug1: re-parsing configuration
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 52: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug2: checking match for 'final all' host localhost originally localhost
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: matched 'final'
debug2: match found
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1]
debug2: resolving "localhost" port 22
debug2: ssh_connect_direct
debug1: Connecting to localhost [::1] port 22.
debug1: Connection established.
debug1: identity file /home/admin/.ssh/id_rsa type -1
debug1: identity file /home/admin/.ssh/id_rsa-cert type -1
debug1: identity file /home/admin/.ssh/id_dsa type -1
debug1: identity file /home/admin/.ssh/id_dsa-cert type -1
debug1: identity file /home/admin/.ssh/id_ecdsa type -1
debug1: identity file /home/admin/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/admin/.ssh/id_ed25519 type -1
debug1: identity file /home/admin/.ssh/id_ed25519-cert type -1
debug1: identity file /home/admin/.ssh/id_xmss type -1
debug1: identity file /home/admin/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.0
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.0
debug1: match: OpenSSH_8.0 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to localhost:22 as 'root'
debug3: hostkeys_foreach: reading file "/home/admin/.ssh/known_hosts"
debug3: hostkeys_foreach: reading file "/etc/ssh/ssh_known_hosts"
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc
debug2: ciphers stoc: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc
debug2: MACs ctos: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
debug2: MACs stoc: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc
debug2: ciphers stoc: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc
debug2: MACs ctos: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
debug2: MACs stoc: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: aes256-gcm@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: aes256-gcm@openssh.com MAC: <implicit> compression: none
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:wiTwrVZx1f3CbXSsWozaex5YDQbCoAS1eYkKMyO6IAo
debug3: hostkeys_foreach: reading file "/home/admin/.ssh/known_hosts"
debug3: hostkeys_foreach: reading file "/etc/ssh/ssh_known_hosts"
The authenticity of host 'localhost (::1)' can't be established.
ECDSA key fingerprint is SHA256:wiTwrVZx1f3CbXSsWozaex5YDQbCoAS1eYkKMyO6IAo.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 4294967296 blocks
debug1: Will attempt key: /home/admin/.ssh/id_rsa 
debug1: Will attempt key: /home/admin/.ssh/id_dsa 
debug1: Will attempt key: /home/admin/.ssh/id_ecdsa 
debug1: Will attempt key: /home/admin/.ssh/id_ed25519 
debug1: Will attempt key: /home/admin/.ssh/id_xmss 
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
Connection closed by ::1 port 22
[admin@frontserver yum.repos.d]$

And, to round it out, from my workday machine:

$ ssh -vvv admin@frontserver 
morgan@morgansmachine:~$ ssh -vvv admin@frontserver
OpenSSH_8.4p1 Debian-5+deb11u1, OpenSSL 1.1.1n  15 Mar 2022
debug1: Reading configuration data /home/morgan/.ssh/config
debug3: kex names ok: [diffie-hellman-group1-sha1]
debug3: kex names ok: [diffie-hellman-group1-sha1]
debug3: kex names ok: [diffie-hellman-group1-sha1]
debug3: kex names ok: [diffie-hellman-group1-sha1]
debug3: kex names ok: [diffie-hellman-group1-sha1]
debug3: kex names ok: [diffie-hellman-group1-sha1]
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug3: kex names ok: [diffie-hellman-group1-sha1]
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/home/morgan/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/home/morgan/.ssh/known_hosts2'
debug2: resolving "frontserver" port 22
debug2: ssh_connect_direct
debug1: Connecting to frontserver [192.168.0.20] port 22.
debug1: Connection established.
debug1: identity file /home/morgan/.ssh/id_rsa type 0
debug1: identity file /home/morgan/.ssh/id_rsa-cert type -1
debug1: identity file /home/morgan/.ssh/id_dsa type 1
debug1: identity file /home/morgan/.ssh/id_dsa-cert type -1
debug1: identity file /home/morgan/.ssh/id_ecdsa type -1
debug1: identity file /home/morgan/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/morgan/.ssh/id_ecdsa_sk type -1
debug1: identity file /home/morgan/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /home/morgan/.ssh/id_ed25519 type -1
debug1: identity file /home/morgan/.ssh/id_ed25519-cert type -1
debug1: identity file /home/morgan/.ssh/id_ed25519_sk type -1
debug1: identity file /home/morgan/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /home/morgan/.ssh/id_xmss type -1
debug1: identity file /home/morgan/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.4p1 Debian-5+deb11u1
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.0
debug1: match: OpenSSH_8.0 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to frontserver:22 as 'admin'
debug3: hostkeys_foreach: reading file "/home/morgan/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /home/morgan/.ssh/known_hosts:1
debug3: load_hostkeys: loaded 1 keys from frontserver
debug3: order_hostkeyalgs: have matching best-preference key type ecdsa-sha2-nistp256-cert-v01@openssh.com, using HostkeyAlgorithms verbatim
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,ssh-ed25519,sk-ssh-ed25519@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc
debug2: ciphers stoc: aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc
debug2: ciphers stoc: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc
debug2: MACs ctos: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
debug2: MACs stoc: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: aes128-ctr MAC: umac-128-etm@openssh.com compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: umac-128-etm@openssh.com compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:wiTwrVZx1f3CbXSsWozaex5YDQbCoAS1eYkKMyO6IAo
debug3: hostkeys_foreach: reading file "/home/morgan/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /home/morgan/.ssh/known_hosts:1
debug3: load_hostkeys: loaded 1 keys from frontserver
debug3: hostkeys_foreach: reading file "/home/morgan/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /home/morgan/.ssh/known_hosts:2
debug3: load_hostkeys: loaded 1 keys from 192.168.0.20
debug1: Host 'frontserver' is known and matches the ECDSA host key.
debug1: Found key in /home/morgan/.ssh/known_hosts:1
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 4294967296 blocks
debug1: Skipping ssh-dss key /home/morgan/.ssh/id_dsa - not in PubkeyAcceptedKeyTypes
debug1: Will attempt key: /home/morgan/.ssh/id_rsa RSA SHA256:vvO67R+R2ElfssxQl3iaVVPMAQ/pWp1lwQpX1jaFLRM agent
debug1: Will attempt key: /home/morgan/.config/gsconnect/private.pem RSA SHA256:KhU92hBlLoYN21ZHtIASofxzGQt5tnbRymy+hzX5N2g agent
debug1: Will attempt key: /home/morgan/.ssh/id_ecdsa 
debug1: Will attempt key: /home/morgan/.ssh/id_ecdsa_sk 
debug1: Will attempt key: /home/morgan/.ssh/id_ed25519 
debug1: Will attempt key: /home/morgan/.ssh/id_ed25519_sk 
debug1: Will attempt key: /home/morgan/.ssh/id_xmss 
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
Connection closed by 192.168.0.20 port 22
morgan@morgansmachine:~$

Any suggestions?
Thx

8

The basics of name resolution. You run a program that has to get address for name “XYZ”.

  • system looks at /etc/nsswitch.conf for hosts line. Typically:
    hosts: files dns myhostname
  • First value is ‘files’. Lets look from /etc/hosts. First match of “XYZ” is used. If none,
  • Second ‘dns’. Read /etc/resolv.conf Lets say it has:
# Generated by NetworkManager
search dum lan
nameserver 1.2.3.4
nameserver 8.8.8.8
  • System sends DNS queries to server 1.2.3.4 until it gets a positive answer:
    • XYZ.dum
    • XYZ.lan
    • XYZ
  • If the 1.2.3.4 does not answer, then the queries are sent to 8.8.8.8 (after connection timeouts)
  • If none of the names did give answer, then the “Could not resolve hostname” back to user

Where does your “workday machine” get the 192.168.0.20 from?

What does ‘dig’ give with dig google.com and dig fronserver.lan?

If you do have command ‘host’ … it neatly shows the names it queries with: host -v frontserver

Neither ‘dig’ nor ‘host’ does look at /etc/hosts. They don’t use nameservers from /etc/resolv.conf if you give address of DNS server on the command line.

1 Like
9

Thanks @jlehtone, that reveals:

[admin@frontserver ~]$ cat /etc/nsswitch.conf
# This file is part of systemd.

passwd: files
shadow: files
group:  files
hosts:  files mymachines resolve myhostname
[admin@frontserver ~]$ cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
[admin@frontserver ~]$ cat /etc/resolv.conf
# Generated by NetworkManager
search lan
nameserver 192.168.0.10
nameserver fe80::ea94:f6ff:febe:8674%nm-bridge1
[admin@frontserver ~]$

And suggests (to me) something is up with 192.168.0.10 which is my home router.
But, that is where my work machine (and everything else) gets 192.168.0.20 from and I’ve not made any changes there for a year or so - it’s a Archer C7 running OpenWRT 19.07.7.

`$ dig google.com` & `dig fronserver.lan` 
[admin@frontserver ~]$ dig google.com

; <<>> DiG 9.11.36-RedHat-9.11.36-5.el8_7.2 <<>> google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39499
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;google.com.                    IN      A

;; ANSWER SECTION:
google.com.             93      IN      A       172.217.16.238

;; Query time: 11 msec
;; SERVER: 192.168.0.10#53(192.168.0.10)
;; WHEN: Tue Mar 14 16:32:32 GMT 2023
;; MSG SIZE  rcvd: 55

[admin@frontserver ~]$ dig fronserver.lan

; <<>> DiG 9.11.36-RedHat-9.11.36-5.el8_7.2 <<>> fronserver.lan
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 42573
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;fronserver.lan.                        IN      A

;; Query time: 0 msec
;; SERVER: 192.168.0.10#53(192.168.0.10)
;; WHEN: Tue Mar 14 16:32:49 GMT 2023
;; MSG SIZE  rcvd: 43

[admin@frontserver ~]$

And, from my machine:

`$ dig google.com` & `dig fronserver.lan` 
morgan@morgansmachine:~$ dig google.com

; <<>> DiG 9.16.37-Debian <<>> google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16247
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;google.com.			IN	A

;; ANSWER SECTION:
google.com.		268	IN	A	172.217.16.238

;; Query time: 16 msec
;; SERVER: 192.168.0.10#53(192.168.0.10)
;; WHEN: Tue Mar 14 16:36:35 GMT 2023
;; MSG SIZE  rcvd: 55

morgan@morgansmachine:~$ dig fronserver.lan

; <<>> DiG 9.16.37-Debian <<>> fronserver.lan
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 27889
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;fronserver.lan.			IN	A

;; Query time: 4 msec
;; SERVER: 192.168.0.10#53(192.168.0.10)
;; WHEN: Tue Mar 14 16:36:46 GMT 2023
;; MSG SIZE  rcvd: 43

morgan@morgansmachine:~$

Here’s the same with nslookup, which does return an answer:

admin@frontserver 
[admin@frontserver ~]$ nslookup google.com
Server:         192.168.0.10
Address:        192.168.0.10#53

Non-authoritative answer:
Name:   google.com
Address: 172.217.16.238
Name:   google.com
Address: 2a00:1450:4009:821::200e

[admin@frontserver ~]$ nslookup frontserver.lan
Server:         192.168.0.10
Address:        192.168.0.10#53

Name:   frontserver.lan
Address: 192.168.0.20

[admin@frontserver ~]$
morgan@morgansmachine 
morgan@morgansmachine:~$ nslookup google.com
Server:		192.168.0.10
Address:	192.168.0.10#53

Non-authoritative answer:
Name:	google.com
Address: 172.217.16.238
Name:	google.com
Address: 2a00:1450:4009:822::200e

morgan@morgansmachine:~$ nslookup frontserver.lan
Server:		192.168.0.10
Address:	192.168.0.10#53

Name:	frontserver.lan
Address: 192.168.0.20

morgan@morgansmachine:~$

And, with host:

admin@frontserver ~$ `host -v frontserver' 
[admin@frontserver ~]$ host -v frontserver
Trying "frontserver.lan"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 55792
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;frontserver.lan.               IN      A

;; ANSWER SECTION:
frontserver.lan.        0       IN      A       192.168.0.20

Received 49 bytes from 192.168.0.10#53 in 2 ms
Trying "frontserver.lan"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4141
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;frontserver.lan.               IN      AAAA

Received 33 bytes from 192.168.0.10#53 in 0 ms
Trying "frontserver.lan"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4596
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;frontserver.lan.               IN      MX

Received 33 bytes from 192.168.0.10#53 in 0 ms
[admin@frontserver ~]$
morgan@morgansmachine:~$ `host -v frontserver` 
morgan@morgansmachine:~$ host -v frontserver
Trying "frontserver.lan"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34883
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;frontserver.lan.		IN	A

;; ANSWER SECTION:
frontserver.lan.	0	IN	A	192.168.0.20

Received 49 bytes from 192.168.0.10#53 in 4 ms
Trying "frontserver.lan"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64642
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;frontserver.lan.		IN	AAAA

Received 33 bytes from 192.168.0.10#53 in 0 ms
Trying "frontserver.lan"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17897
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;frontserver.lan.		IN	MX

Received 33 bytes from 192.168.0.10#53 in 4 ms
morgan@morgansmachine:~$

The only one that looks wrong to me is dig frontserver.lan which isn’t returning an answer - does dig work differently to nslookup and host?
But, then again - I’ve just rerun it and and it seems erratic - first returning an answer and then not…:

'dig frontserver.lan' x 2 
[admin@frontserver ~]$ dig frontserver.lan

; <<>> DiG 9.11.36-RedHat-9.11.36-5.el8_7.2 <<>> frontserver.lan
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64336
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;frontserver.lan.               IN      A

;; ANSWER SECTION:
frontserver.lan.        0       IN      A       192.168.0.20

;; Query time: 0 msec
;; SERVER: 192.168.0.10#53(192.168.0.10)
;; WHEN: Tue Mar 14 18:11:22 GMT 2023
;; MSG SIZE  rcvd: 60

[admin@frontserver ~]$ dig fronserver.lan

; <<>> DiG 9.11.36-RedHat-9.11.36-5.el8_7.2 <<>> fronserver.lan
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 20356
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;fronserver.lan.                        IN      A

;; Query time: 0 msec
;; SERVER: 192.168.0.10#53(192.168.0.10)
;; WHEN: Tue Mar 14 18:15:34 GMT 2023
;; MSG SIZE  rcvd: 43

[admin@frontserver ~]$

???

10

Notice the difference:

dig fronserver
dig frontserver

My post had a typo – missing ‘t’ …

11

That’s a trick question… :slight_smile:

12

But, my nfs shares are also refused:

morgan@morgansmachine:~$ sudo mount -a
[sudo] password for morgan: 
mount.nfs: access denied by server while mounting frontserver.lan:/mnt/RAID1-699GiB/nfs-share
mount.nfs: access denied by server while mounting frontserver.lan:/mnt/RAID1-699GiB/av
morgan@morgansmachine:~$

But, https is okay, because I’m able to use Cockpit

morgan@morgansmachine:~$ telnet frontserver.lan 9090
Trying 192.168.0.20...
Connected to frontserver.lan.
Escape character is '^]'.
^]
telnet> Connection closed.
morgan@morgansmachine:~$

I’m lost…

13

There are at least four levels with a service:

  • Does firewall allow connection to port?
  • Does SELinux allow service process listen a port?
  • Does service process listen a port?
  • Does service process allow connection?

However, NFS should not be the first priority. You want to get just enough network so that you can get in packages to reinstall. To replace the non-Rocky packages.

If dnf cannot connect to repos, then give URL to packet to ‘rpm’, or download packages (with wget, curl, rsync, …) and install with dnf (or rpm, if dnf is totally broken).

In fact, if you still have the Rocky install media (USB) and it was the “DVD1” version – fat – you have packages at hand. Mount that media and install packages from it.

14 
morgan@morgansmachine:~$ ssh admin@frontserver
Web console: https://frontserver.lan:9090/ or https://192.168.0.20:9090/

Last login: Tue Mar 14 22:52:31 2023 from ::ffff:192.168.0.140
[admin@frontserver ~]$ echo hello world
hello world
[admin@frontserver ~]$

The answer was to track down the IPs of all the base URLs in the repos and use sed to swap them for the domain names - manual dns…

[edit: And, adding sslverify=false to /etc/yum.conf else ssl certification will cause dnf to spit the dummy on IPs.]

15

The hosts line in your /etc/nsswitch.conf doesn’t have dns option, as @jlehtone has explained. I think that’s the root cause here. I tried this myself and get the same result as in your case (dig and host work but other utilities don’t):

$ grep hosts /etc/nsswitch.conf
# Valid databases are: aliases, ethers, group, gshadow, hosts,
#hosts:      files dns myhostname
hosts:      files myhostname  # I removed the dns here

$ dig google.com +short
142.250.4.101
142.250.4.139
142.250.4.102
142.250.4.113
142.250.4.138
142.250.4.100

$ ping google.com
ping: google.com: Name or service not known

$ ssh google.com
ssh: Could not resolve hostname google.com: Name or service not known
16

Oh, I misunderstood the three bullet points at the top of @jlehtone 's post - understanding (wrongly) that if hosts failed then the second file to look at is resolv.conf and take the the dns entry from that - never mind, thank you for the clarification @zaffiro - I wonder where the dns entry went, perhaps I lost that here:

?

It’s back in now:

$ grep hosts /etc/nsswitch.conf
hosts:  files dns mymachines resolve myhostname

$ ping google.com
PING google.com (172.217.16.238) 56(84) bytes of data.
64 bytes from lhr48s28-in-f14.1e100.net (172.217.16.238): icmp_seq=1 ttl=116 time=10.1 ms

$ ssh google.com
^C

And, dnf works again :slight_smile:
Thank you @jlehtone and @zaffiro.

17

Note: Starting el8 the /etc/nsswitch.conf is preferentially managed with tool: authselect
The file has comments about how to edit the config.

The el9 version has even more text about “service providers”:

# Valid service provider entries include (in alphabetical order):
#
#       compat                  Use /etc files plus *_compat pseudo-db
#       db                      Use the pre-processed /var/db files
#       dns                     Use DNS (Domain Name Service)
#       files                   Use the local files in /etc
#       hesiod                  Use Hesiod (DNS) for user lookups
#
# See `info libc 'NSS Basics'` for more information.
#
# Commonly used alternative service providers (may need installation):
#
#       ldap                    Use LDAP directory server
#       myhostname              Use systemd host names
#       mymachines              Use systemd machine names
#       mdns*, mdns*_minimal    Use Avahi mDNS/DNS-SD
#       resolve                 Use systemd resolved resolver
#       sss                     Use System Security Services Daemon (sssd)
#       systemd                 Use systemd for dynamic user option
#       winbind                 Use Samba winbind support
#       wins                    Use Samba wins support
#       wrapper                 Use wrapper module for testing

These do explain what the files dns mymachines resolve myhostname are.

There is package that is not installed by default:

# dnf info systemd-resolved
Summary      : System daemon that provides network name resolution to local applications
Description  : systemd-resolved is a system service that provides network name
             : resolution to local applications. It implements a caching and
             : validating DNS/DNSSEC stub resolver, as well as an LLMNR and
             : MulticastDNS resolver and responder.

In other words, the resolve looks like an alternative for dns, if it is installed and configured properly.
Something that Fedora might have, but RHEL does not default to.

1 Like
18

Thanks @jlehtone I don’t have systemd-resolved installed, but I might have - it was once a fedora machine until I decided I was too old for that much instability in my life… (swings and roundabouts I guess).

My authselect doesn’t look good, but that’ll have to be homework for another day:

$ authselect check
[error] [/etc/authselect/system-auth] has unexpected content!
[error] [/etc/authselect/password-auth] has unexpected content!
[error] [/etc/authselect/fingerprint-auth] has unexpected content!
[error] [/etc/authselect/smartcard-auth] has unexpected content!
[error] [/etc/authselect/postlogin] has unexpected content!
[error] [/etc/authselect/nsswitch.conf] has unexpected content!
[error] [/etc/authselect/dconf-db] has unexpected content!
[error] [/etc/authselect/dconf-locks] has unexpected content!
[error] [/etc/nsswitch.conf] is not a symbolic link!
[error] [/etc/nsswitch.conf] was not created by authselect!
Current configuration is not valid. It was probably modified outside authselect.
$

:-/