Hi All, today i upgraded rocky linux from 8.5 after the upgrade i reboot the system and failed to boot. heres the screenshot of the errors.
If im going to use SystemrescueCD, do you think it can fixed the system?
thank you in advance
The kernel messages seem to refer to your disk having issues. Your disk may be starting to fail. I would suggest booting into a rescue media and checking the SMART data.
like the systemRescueCd sir ?
Kind of, but don’t forget you can use the Rocky install media as a bootable rescue cd, the advantage being that every package will be compatible with the “broken” installatoin.
Thank very much sir! 'still struggling to revive the disk here 
Hi all, 
Im trying to understand the log result here, but i need really your expertise mates
# smartctl -a /dev/sdc
smartctl 7.0 2018-12-30 r4883 [x86_64-linux-3.10.0-1160.15.2.el7.x86_64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Seagate Barracuda 3.5
Device Model: ST2000DM006-2DM164
Serial Number: Z4Z91ZSD
LU WWN Device Id: 5 000c50 0a474e5a3
Firmware Version: CC26
User Capacity: 2,000,398,934,016 bytes [2.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 7200 rpm
Form Factor: 3.5 inches
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-2, ACS-3 T13/2161-D revision 3b
SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Thu Aug 11 09:32:52 2022 PST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 89) seconds.
Offline data collection
capabilities: (0x73) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
No Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: ( 232) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x1085) SCT Status supported.
SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 093 093 006 Pre-fail Always - 136969175
3 Spin_Up_Time 0x0003 097 096 000 Pre-fail Always - 0
4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 181
5 Reallocated_Sector_Ct 0x0033 079 079 010 Pre-fail Always - 26688
7 Seek_Error_Rate 0x000f 083 060 030 Pre-fail Always - 226872390
9 Power_On_Hours 0x0032 091 085 000 Old_age Always - 8326
10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0
12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 89
183 Runtime_Bad_Block 0x0032 095 095 000 Old_age Always - 5
184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0
187 Reported_Uncorrect 0x0032 001 001 000 Old_age Always - 247
188 Command_Timeout 0x0032 099 099 000 Old_age Always - 1 3 4
189 High_Fly_Writes 0x003a 092 092 000 Old_age Always - 8
190 Airflow_Temperature_Cel 0x0022 071 057 045 Old_age Always - 29 (Min/Max 29/29)
191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0
192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 127
193 Load_Cycle_Count 0x0032 087 087 000 Old_age Always - 26630
194 Temperature_Celsius 0x0022 029 043 000 Old_age Always - 29 (0 20 0 0 0)
197 Current_Pending_Sector 0x0012 001 001 000 Old_age Always - 23800
198 Offline_Uncorrectable 0x0010 001 001 000 Old_age Offline - 23800
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0
240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 21157h+50m+33.352s
241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 7822365613
242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 35518177192
SMART Error Log Version: 1
ATA Error Count: 250 (device log contains only the most recent five errors)
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.
Error 250 occurred at disk power-on lifetime: 8325 hours (346 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 ff ff ff 4f 00 00:00:09.537 READ FPDMA QUEUED
60 00 90 ff ff ff 4f 00 00:00:09.534 READ FPDMA QUEUED
60 00 38 ff ff ff 4f 00 00:00:09.534 READ FPDMA QUEUED
60 00 18 ff ff ff 4f 00 00:00:09.534 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 00:00:09.534 READ FPDMA QUEUED
Error 249 occurred at disk power-on lifetime: 8325 hours (346 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 ff ff ff 4f 00 00:27:35.773 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 00:27:35.770 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 00:27:35.764 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 00:27:35.761 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 00:27:35.748 READ FPDMA QUEUED
Error 248 occurred at disk power-on lifetime: 8325 hours (346 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 ff ff ff 4f 00 00:01:53.813 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 00:01:53.810 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 00:01:53.804 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 00:01:53.801 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 00:01:53.788 READ FPDMA QUEUED
Error 247 occurred at disk power-on lifetime: 8325 hours (346 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 53 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 ff ff ff 4f 00 00:00:55.340 READ FPDMA QUEUED
ef 10 02 00 00 00 a0 00 00:00:55.340 SET FEATURES [Enable SATA feature]
27 00 00 00 00 00 e0 00 00:00:55.339 READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
ec 00 00 00 00 00 a0 00 00:00:55.339 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 00 00:00:55.339 SET FEATURES [Set transfer mode]
Error 246 occurred at disk power-on lifetime: 8325 hours (346 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 53 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 ff ff ff 4f 00 00:00:51.698 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 00:00:51.690 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 00:00:51.681 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 00:00:51.681 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 00:00:51.676 READ FPDMA QUEUED
SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
Mybe try these simpified commands
smartctl --health /dev/sdc
smartctl --test=short /dev/sdc
wait, wait, wait
smartctl --log=error /dev/sdc
smartctl --log=selftest /dev/sdc
Thanks mate.
Now my challenge on mounting gpt. still googling for possible guide.
The GPT is a format of partition table. One does not mount partition table.
One does mount filesystem that usually is in partition.
gdisk -l /dev/sdc
lsblk
lsblk -o NAME,SIZE,TYPE,FSTYPE,UUID,MOUNTPOINTS
blkid
Thanks mate, i am able to mount /dev/sdc1 the file system(the one failed to boot).
my question is , Is it safe to delete everything inside EFI and replace it from a copy of a fresh-new installed filesystem from Rocky?
You havn’t shown the results of the smart tests above yet, nor the output of the gdisk and lsblk commands, nor have you made it clear if you’ve understood why it was showing disk errors.
Regarding the EFI partition, you should be able to do this from within the GUI installer, there’s an option to re-use and re-format, but it’s better to start with a completely clean disk, install, restore from backups.
Heres the result with smartctl -H /dev/sdc
smartctl 7.3 2022-02-28 r5338 [x86_64-linux-5.18.0-2-amd64] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
and heres the Errlog with smartctl
smartctl 7.3 2022-02-28 r5338 [x86_64-linux-5.18.0-2-amd64] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
smartctl 7.3 2022-02-28 r5338 [x86_64-linux-5.18.0-2-amd64] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF READ SMART DATA SECTION ===
SMART Error Log Version: 1
ATA Error Count: 2541 (device log contains only the most recent five errors)
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.
Error 2541 occurred at disk power-on lifetime: 8366 hours (348 days + 14 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 ff ff ff 4f 00 00:07:10.138 READ FPDMA QUEUED
60 00 08 00 f0 32 40 00 00:07:10.120 READ FPDMA QUEUED
60 00 48 ff ff ff 4f 00 00:07:10.118 READ FPDMA QUEUED
60 00 50 ff ff ff 4f 00 00:07:10.118 READ FPDMA QUEUED
60 00 30 ff ff ff 4f 00 00:07:10.118 READ FPDMA QUEUED
Error 2540 occurred at disk power-on lifetime: 8356 hours (348 days + 4 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 ff ff ff 4f 00 00:00:09.965 READ FPDMA QUEUED
60 00 48 b8 0d 29 42 00 00:00:09.951 READ FPDMA QUEUED
60 00 50 60 0d 29 42 00 00:00:09.951 READ FPDMA QUEUED
60 00 30 28 0d 29 42 00 00:00:09.951 READ FPDMA QUEUED
60 00 a8 78 0c 29 42 00 00:00:09.951 READ FPDMA QUEUED
Error 2539 occurred at disk power-on lifetime: 8355 hours (348 days + 3 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 53 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 ff ff ff 4f 00 14:40:32.271 READ FPDMA QUEUED
ef 10 02 00 00 00 a0 00 14:40:32.270 SET FEATURES [Enable SATA feature]
27 00 00 00 00 00 e0 00 14:40:32.270 READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
ec 00 00 00 00 00 a0 00 14:40:32.270 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 00 14:40:32.269 SET FEATURES [Set transfer mode]
Error 2538 occurred at disk power-on lifetime: 8355 hours (348 days + 3 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 53 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 ff ff ff 4f 00 14:40:28.573 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 14:40:28.572 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 14:40:28.565 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 14:40:28.564 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 14:40:28.559 READ FPDMA QUEUED
Error 2537 occurred at disk power-on lifetime: 8355 hours (348 days + 3 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 53 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 ff ff ff 4f 00 14:40:24.672 READ FPDMA QUEUED
ef 10 02 00 00 00 a0 00 14:40:24.672 SET FEATURES [Enable SATA feature]
27 00 00 00 00 00 e0 00 14:40:24.672 READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
ec 00 00 00 00 00 a0 00 14:40:24.672 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 00 14:40:24.671 SET FEATURES [Set transfer mode]
In your earlier post:
I’d replace the disk ASAP, since it has had so many reallocation events (and attempts) already.
Do you have backups of the data?
One last ditch backup procedure is:
dd entire content of the faulty disk to new disk. E.g. ‘dd if=/dev/sdc of=/dev/sdd’ (if sdd is the new, empty disk)The main point is that access to faulty is minimized; only one read (with dd)
Thank you for this very detailed/useful reply mate. i follow this Thanks again!
Maybe I’m missing something, but aren’t the errors in your original screenshot referring to sda? Why are smartctl commands being run against sdc?
that was an image taken when the subject drive failed to boot. The errLogs was reproduced using gparted live cd right after.