I spinned up a Azure Virtual Machine with Rocky Linux ver 9 last week. It was all running fine until yesterday and all of sudden seeing Guest OS error in the serial console.
It seems to be related to kernel error as per the serial consolse logs
[ OK ] Started The Apache HTTP Server.
[ 20.260931] BUG: unable to handle page fault for address: ff263ab855b92d88
[ 20.262988] #PF: supervisor write access in kernel mode
[ 20.264362] #PF: error_code(0x0002) - not-present page
[ 20.265686] PGD 476001067 P4D 476002067 PUD 0
[ 20.266913] Oops: 0002 [#1] PREEMPT SMP NOPTI
[ 20.268147] CPU: 2 PID: 922 Comm: falcon-sensor-b Not tainted 5.14.0-427.16.1.el9_4.x86_64 #1
[ 20.270502] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS Hyper-V UEFI Release v4.1 11/28/2023
[ 20.273598] RIP: 0010:backtrack_insn+0x408/0x800
[ 20.274933] Code: 30 00 0f 85 64 fd ff ff 41 ba 01 00 00 00 b9 01 00 00 00 45 8d 48 ff 44 89 d0 d3 e0 85 c2 74 0f 89 c6 f7 d6 21 d6 89 74 bb 0c <42> 09 44 8b 0c 83 c1 01 83 f9 06 0f 84 71 01 00 00 44 89 d0 8b 54
[ 20.280109] RSP: 0018:ff2e3d8e404239d8 EFLAGS: 00010246
[ 20.281527] RAX: 0000000000000002 RBX: ff263ab455b92d80 RCX: 0000000000000001
[ 20.283397] RDX: 0000000000000002 RSI: 0000000000000000 RDI: 0000000000000000
[ 20.285187] RBP: ff263ab455b90000 R08: 0000000000000000 R09: 00000000ffffffff
[ 20.287091] R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000058
[ 20.288987] R13: ff263ab455b90a90 R14: ff263ab455166a80 R15: ff2e3d8e5221e300
[ 20.290747] FS: 00007f96e35e9740(0000) GS:ff263ab7afb00000(0000) knlGS:0000000000000000
[ 20.292948] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 20.294443] CR2: ff263ab855b92d88 CR3: 0000000106dea005 CR4: 0000000000371ee0
[ 20.296271] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 20.298138] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 20.299942] Call Trace:
[ 20.300598]
[ 20.301171] ? show_trace_log_lvl+0x1c4/0x2df
[ 20.302366] ? show_trace_log_lvl+0x1c4/0x2df
[ 20.303510] ? __mark_chain_precision+0x166/0x630
[ 20.304741] ? __die_body.cold+0x8/0xd
[ 20.305703] ? page_fault_oops+0x134/0x170
[ 20.306776] ? kernelmode_fixup_or_oops+0x84/0x110
[ 20.308105] ? exc_page_fault+0xa8/0x150
[ 20.309136] ? asm_exc_page_fault+0x22/0x30
[ 20.310276] ? backtrack_insn+0x408/0x800
[ 20.311502] ? backtrack_insn+0x386/0x800
[ 20.312568] ? copy_array+0x4d/0xb0
[ 20.313502] ? __pfx_verbose+0x10/0x10
[ 20.314561] ? __pfx_disasm_kfunc_name+0x10/0x10
[ 20.315810] __mark_chain_precision+0x166/0x630
[ 20.317002] check_cond_jmp_op+0x738/0xbd0
[ 20.318095] ? is_state_visited+0x400/0x740
[ 20.319226] do_check+0x85b/0xac0
[ 20.320137] do_check_common+0x2a9/0x340
[ 20.321101] bpf_check+0xf7c/0x10a0
[ 20.322046] ? __kmem_cache_alloc_node+0x1c7/0x2d0
[ 20.323362] ? _raw_spin_lock_irq+0x10/0x30
[ 20.324421] bpf_prog_load+0x636/0x970
[ 20.325333] __sys_bpf+0x5e8/0xe20
[ 20.326161] __x64_sys_bpf+0x1a/0x30
[ 20.327027] do_syscall_64+0x5c/0x90
[ 20.327934] ? syscall_exit_work+0x103/0x130
[ 20.328987] ? syscall_exit_to_user_mode+0x22/0x40
[ 20.330135] ? do_syscall_64+0x69/0x90
[ 20.331045] ? syscall_exit_work+0x103/0x130
[ 20.332135] ? syscall_exit_to_user_mode+0x22/0x40
[ 20.333297] ? do_syscall_64+0x69/0x90
[ 20.334211] ? do_syscall_64+0x69/0x90
[ 20.335148] ? do_syscall_64+0x69/0x90
[ 20.336065] ? __irq_exit_rcu+0x46/0xc0
[ 20.336998] ? sysvec_hyperv_stimer0+0x3c/0x90
[ 20.338134] entry_SYSCALL_64_after_hwframe+0x72/0xdc
[ 20.339442] RIP: 0033:0x7f96e323ee5d
[ 20.340345] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 93 af 1b 00 f7 d8 64 89 01 48
[ 20.344815] RSP: 002b:00007fff8077b048 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
[ 20.346640] RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 00007f96e323ee5d
[ 20.348335] RDX: 0000000000000090 RSI: 00007fff8077b0d0 RDI: 0000000000000005
[ 20.350041] RBP: 0000000000000090 R08: 00007fff8077b200 R09: 00000000000002c5
[ 20.351714] R10: 0000000000000070 R11: 0000000000000246 R12: 00007fff8077b0d0
[ 20.353465] R13: 00007fff8077b200 R14: 0000000000000005 R15: 00005630c1b23470
[ 20.355273]
[ 20.355843] Modules linked in: rfkill ext4 xt_conntrack nf_conntrack mbcache nf_defrag_ipv6 jbd2 nf_defrag_ipv4 xt_owner nft_counter nft_compat nf_tables nfnetlink vfat fat intel_rapl_msr intel_rapl_common intel_uncore_frequency_common nfit libnvdimm mlx5_ib kvm_intel kvm ib_uverbs ib_core hyperv_drm irqbypass rapl drm_shmem_helper drm_kms_helper syscopyarea sysfillrect sysimgblt hv_balloon fb_sys_fops hv_utils joydev pcspkr drm fuse xfs libcrc32c mlx5_core mlxfw psample tls pci_hyperv pci_hyperv_intf sd_mod t10_pi sg hv_netvsc hv_storvsc serio_raw hid_hyperv scsi_transport_fc hyperv_keyboard crct10dif_pclmul crc32_pclmul crc32c_intel hv_vmbus ghash_clmulni_intel dm_mirror dm_region_hash dm_log dm_mod
[ 20.370312] CR2: ff263ab855b92d88
[ 20.371244] —[ end trace b3ff6b206998dad1 ]—
[ 20.371245] RIP: 0010:backtrack_insn+0x408/0x800
[ 20.371247] Code: 30 00 0f 85 64 fd ff ff 41 ba 01 00 00 00 b9 01 00 00 00 45 8d 48 ff 44 89 d0 d3 e0 85 c2 74 0f 89 c6 f7 d6 21 d6 89 74 bb 0c <42> 09 44 8b 0c 83 c1 01 83 f9 06 0f 84 71 01 00 00 44 89 d0 8b 54
[ 20.371249] RSP: 0018:ff2e3d8e404239d8 EFLAGS: 00010246
[ 20.371253] RAX: 0000000000000002 RBX: ff263ab455b92d80 RCX: 0000000000000001
[ 20.371254] RDX: 0000000000000002 RSI: 0000000000000000 RDI: 0000000000000000
[ 20.371255] RBP: ff263ab455b90000 R08: 0000000000000000 R09: 00000000ffffffff
[ 20.371256] R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000058
[ 20.371257] R13: ff263ab455b90a90 R14: ff263ab455166a80 R15: ff2e3d8e5221e300
[ 20.371258] FS: 00007f96e35e9740(0000) GS:ff263ab7afb00000(0000) knlGS:0000000000000000
[ 20.371260] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 20.371261] CR2: ff263ab855b92d88 CR3: 0000000106dea005 CR4: 0000000000371ee0
[ 20.371261] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 20.371262] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 20.371267] Kernel panic - not syncing: Fatal exception
[ 21.461924] Kernel Offset: 0x2ac00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[ 21.495789] —[ end Kernel panic - not syncing: Fatal exception ]—