Partition /dev/mapper/rl-root

Smoky · December 15, 2024, 4:08pm

Above mentioned partition was 100%.
I was able to delete some unnecessary stuff and have now 2,1G available. 98% used.

[root@home /]# df -h
Filesystem           Size  Used Avail Use% Mounted on
devtmpfs             4.0M     0  4.0M   0% /dev
tmpfs                7.7G     0  7.7G   0% /dev/shm
tmpfs                3.1G   34M  3.1G   2% /run
/dev/mapper/rl-root   70G   68G  2.1G  98% /
/dev/mapper/rl-home  853G   30G  823G   4% /home
/dev/nvme0n1p1      1014M  448M  567M  45% /boot
/dev/sda1            1.8T  940G  800G  55% /mnt/media
/dev/sdb1            1.8T  964G  777G  56% /mnt/backup02
/dev/sdc             1.8T  965G  869G  53% /mnt/backup01
tmpfs                1.6G  4.0K  1.6G   1% /run/user/1000

Are 70G for this partition enough or should I better increase it? As you see fomr the above I have a lot of space available in /home which I will not need as I put all media files on external hard drives.
I tried to find with du command what else I could delete, but did no get far.

Any suggestion / insight is welcome.

Thanks, Wolfgang

jlehtone · December 15, 2024, 5:10pm

What takes space from that filesystem?

This is a desktop that admittedly does not have much in it:

$ sudo du -d1 -hx /
8.0K	/mnt
579M	/var
8.3G	/usr
1.4M	/root
32M	/etc
4.0K	/srv
1.4G	/opt
104K	/tmp
4.0K	/media
16K	/lost+found
4.0K	/afs
11G	/

Smoky · December 15, 2024, 7:06pm

It is not a desktop.
It is a headless home server.
What is a good command to list biggest files on this partition?

Wolfgang

jlehtone · December 15, 2024, 11:08pm

Start with:

Then, if for example the /var looks big, look at it with:
sudo du -d1 -hx /var

Smoky · December 16, 2024, 4:25pm

Thanks for that hint.
I followed your suggestion and made config changes.
I run a media server. I moved all multimedia files to /home. I also deleted some big log files.
I do have a bit more space on / and available 11G, telling me that 60G are used.

Filesystem           Size  Used Avail Use% Mounted on
devtmpfs             4.0M     0  4.0M   0% /dev
tmpfs                7.7G     0  7.7G   0% /dev/shm
tmpfs                3.1G  9.1M  3.1G   1% /run
/dev/mapper/rl-root   70G   60G   11G  86% /
/dev/mapper/rl-home  853G  123G  731G  15% /home
/dev/nvme0n1p1      1014M  449M  566M  45% /boot
/dev/sda1            1.8T  945G  795G  55% /mnt/media
/dev/sdb1            1.8T  969G  772G  56% /mnt/backup02
/dev/sdc             1.8T  970G  864G  53% /mnt/backup01
tmpfs                1.6G  4.0K  1.6G   1% /run/user/1000

At the same time when I run du -d1 -hx / ir tells me that / is 20G big.

[root@home /]# du -d1 -hx /
30M	/etc
18M	/root
16G	/var
3.4G	/usr
0	/media
0	/mnt
679M	/opt
0	/srv
36M	/tmp
0	/afs
20G	/

Can this be correct?

Wolfgang

hs303 · December 16, 2024, 4:58pm

If you have so much unused space lying around in /home you could backup this data in one of the backup directories, destroy /dev/mapper/rl-home and recreate it with ~200GiB.
Than mount it back and move the data back to your /home. Then expand /dev/mapper/rl-root by 100 GiB.
This leaves you some big chunk of space for future use, so next time something like this happens you are still flexible.

Smoky · December 17, 2024, 6:57am

Hello hs303,

this is the plan for XMas vacation :).

Coming back yo my quesiton, df -h tells me I used up 61GB.

[root@home ~]# df -h
Filesystem           Size  Used Avail Use% Mounted on
devtmpfs             4.0M     0  4.0M   0% /dev
tmpfs                7.7G     0  7.7G   0% /dev/shm
tmpfs                3.1G   26M  3.1G   1% /run
/dev/mapper/rl-root   70G   61G  9.4G  87% /
/dev/mapper/rl-home  853G  123G  730G  15% /home

At the same time command du -d1 -hx / tells me that only 21G are used. Is this correct?

30M     /etc
27M     /root
16G     /var
3.4G    /usr
0       /media
0       /mnt
679M    /opt
0       /srv
36M     /tmp
0       /afs
21G     /

Wolfgang

hs303 · December 17, 2024, 7:53am

Although these outputs will always differ a bit, this does not seem correct. df reports free space seen from file system level, while du adds up the size of the files in directory trees. This might happen when deleted files are still held by some processes and the free space is not committed to the disk. Maybe lsof | grep deleted will show you some.
I guess xfs_spaceman should provide the most accurate data on the free space.

[hs303@rocky-vm ~]$ sudo xfs_spaceman -c "freesp -s" / | grep "free blocks"
total free blocks 2840457
[hs303@rocky-vm ~]$ sudo xfs_spaceman -c "info" / | grep -oE "bsize=[0-9]{1,10}" | uniq
bsize=4096
[hs303@rocky-vm ~]$ python3 -c "print(f'{2840457*4096:,} blocks sum up to {2840457*4096/1024/1024/1024:.2f} GiB')"
11,634,511,872 blocks sum up to 10.84 GiB

while df reports

[hs303@rocky-vm ~]$ df --sync --block-size=4096 -h / && df --sync --block-size=4096 /
Filesystem           Size  Used Avail Use% Mounted on
/dev/mapper/rl-root   17G  6.3G   11G  37% /
Filesystem          4K-blocks    Used Available Use% Mounted on
/dev/mapper/rl-root   4439040 1637751   2801289  37% /
[hs303@rocky-vm ~]$ sudo du -d0 -hx /
6.1G    /

Smoky · December 17, 2024, 1:24pm

Thank you again for responding.
If I understand correctly, then I have ~10,8GB availalble.

[root@home tmp]# sudo xfs_spaceman -c "freesp -s" / | grep "free blocks"
total free blocks 2444440
[root@home tmp]# sudo xfs_spaceman -c "info" / | grep -oE "bsize=[0-9]{1,10}" | uniqbsize=4096
[root@home tmp]# python3 -c "print(f'{2840457*4096:,} blocks sum up to {2840457*4096/1024/1024/1024:.2f} GiB')"
11,634,511,872 blocks sum up to 10.84 GiB

Unfortunately I cannot upload a .txt file with the output of “lsof | grep deleted”
Some of the files referenced here should not be pritend as I removed them from the harddisk like e.g. Collabora and coolwsd.
How can I commit the free space to the disk?

dbus-brok    880                                dbus   12u      REG                0,1   2097152       4096 /memfd:dbus-broker-log (deleted)
firewalld    888                                root    9u      REG                0,1      4096          2 /memfd:libffi (deleted)
firewalld    888   1163 gmain                   root    9u      REG                0,1      4096          2 /memfd:libffi (deleted)
php-fpm     1014                                root    3u      REG              253,0         0  201326855 /tmp/.ZendSem.h6hS3u (deleted)
mariadbd    1118                               mysql    6u      REG              253,0         0   67887136 /var/tmp/#67887136 (deleted)
mariadbd    1118                               mysql    7u      REG              253,0         0   67887137 /var/tmp/#67887137 (deleted)
mariadbd    1118                               mysql    8u      REG              253,0         0   67887138 /var/tmp/#67887138 (deleted)
mariadbd    1118                               mysql   11u      REG              253,0         0   67887139 /var/tmp/#67887139 (deleted)
mariadbd    1118   1170 mariadbd               mysql    6u      REG              253,0         0   67887136 /var/tmp/#67887136 (deleted)
mariadbd    1118   1170 mariadbd               mysql    7u      REG              253,0         0   67887137 /var/tmp/#67887137 (deleted)
mariadbd    1118   1170 mariadbd               mysql    8u      REG              253,0         0   67887138 /var/tmp/#67887138 (deleted)
mariadbd    1118   1170 mariadbd               mysql   11u      REG              253,0         0   67887139 /var/tmp/#67887139 (deleted)
mariadbd    1118   1200 mariadbd               mysql    6u      REG              253,0         0   67887136 /var/tmp/#67887136 (deleted)
mariadbd    1118   1200 mariadbd               mysql    7u      REG              253,0         0   67887137 /var/tmp/#67887137 (deleted)
mariadbd    1118   1200 mariadbd               mysql    8u      REG              253,0         0   67887138 /var/tmp/#67887138 (deleted)
mariadbd    1118   1200 mariadbd               mysql   11u      REG              253,0         0   67887139 /var/tmp/#67887139 (deleted)
mariadbd    1118   1232 mariadbd               mysql    6u      REG              253,0         0   67887136 /var/tmp/#67887136 (deleted)
mariadbd    1118   1232 mariadbd               mysql    7u      REG              253,0         0   67887137 /var/tmp/#67887137 (deleted)
mariadbd    1118   1232 mariadbd               mysql    8u      REG              253,0         0   67887138 /var/tmp/#67887138 (deleted)
mariadbd    1118   1232 mariadbd               mysql   11u      REG              253,0         0   67887139 /var/tmp/#67887139 (deleted)
mariadbd    1118   1237 mariadbd               mysql    6u      REG              253,0         0   67887136 /var/tmp/#67887136 (deleted)
mariadbd    1118   1237 mariadbd               mysql    7u      REG              253,0         0   67887137 /var/tmp/#67887137 (deleted)
mariadbd    1118   1237 mariadbd               mysql    8u      REG              253,0         0   67887138 /var/tmp/#67887138 (deleted)
mariadbd    1118   1237 mariadbd               mysql   11u      REG              253,0         0   67887139 /var/tmp/#67887139 (deleted)
mariadbd    1118   1343 mariadbd               mysql    6u      REG              253,0         0   67887136 /var/tmp/#67887136 (deleted)
mariadbd    1118   1343 mariadbd               mysql    7u      REG              253,0         0   67887137 /var/tmp/#67887137 (deleted)
mariadbd    1118   1343 mariadbd               mysql    8u      REG              253,0         0   67887138 /var/tmp/#67887138 (deleted)
mariadbd    1118   1343 mariadbd               mysql   11u      REG              253,0         0   67887139 /var/tmp/#67887139 (deleted)
mariadbd    1118   1348 mariadbd               mysql    6u      REG              253,0         0   67887136 /var/tmp/#67887136 (deleted)
mariadbd    1118   1348 mariadbd               mysql    7u      REG              253,0         0   67887137 /var/tmp/#67887137 (deleted)
mariadbd    1118   1348 mariadbd               mysql    8u      REG              253,0         0   67887138 /var/tmp/#67887138 (deleted)
mariadbd    1118   1348 mariadbd               mysql   11u      REG              253,0         0   67887139 /var/tmp/#67887139 (deleted)
mariadbd    1118   8090 mariadbd               mysql    6u      REG              253,0         0   67887136 /var/tmp/#67887136 (deleted)
mariadbd    1118   8090 mariadbd               mysql    7u      REG              253,0         0   67887137 /var/tmp/#67887137 (deleted)
mariadbd    1118   8090 mariadbd               mysql    8u      REG              253,0         0   67887138 /var/tmp/#67887138 (deleted)
mariadbd    1118   8090 mariadbd               mysql   11u      REG              253,0         0   67887139 /var/tmp/#67887139 (deleted)
mariadbd    1118   8177 mariadbd               mysql    6u      REG              253,0         0   67887136 /var/tmp/#67887136 (deleted)
mariadbd    1118   8177 mariadbd               mysql    7u      REG              253,0         0   67887137 /var/tmp/#67887137 (deleted)
mariadbd    1118   8177 mariadbd               mysql    8u      REG              253,0         0   67887138 /var/tmp/#67887138 (deleted)
mariadbd    1118   8177 mariadbd               mysql   11u      REG              253,0         0   67887139 /var/tmp/#67887139 (deleted)
mariadbd    1118  39718 mariadbd               mysql    6u      REG              253,0         0   67887136 /var/tmp/#67887136 (deleted)
mariadbd    1118  39718 mariadbd               mysql    7u      REG              253,0         0   67887137 /var/tmp/#67887137 (deleted)
mariadbd    1118  39718 mariadbd               mysql    8u      REG              253,0         0   67887138 /var/tmp/#67887138 (deleted)
mariadbd    1118  39718 mariadbd               mysql   11u      REG              253,0         0   67887139 /var/tmp/#67887139 (deleted)
mariadbd    1118  39720 mariadbd               mysql    6u      REG              253,0         0   67887136 /var/tmp/#67887136 (deleted)
mariadbd    1118  39720 mariadbd               mysql    7u      REG              253,0         0   67887137 /var/tmp/#67887137 (deleted)
mariadbd    1118  39720 mariadbd               mysql    8u      REG              253,0         0   67887138 /var/tmp/#67887138 (deleted)
mariadbd    1118  39720 mariadbd               mysql   11u      REG              253,0         0   67887139 /var/tmp/#67887139 (deleted)
mariadbd    1118  39721 mariadbd               mysql    6u      REG              253,0         0   67887136 /var/tmp/#67887136 (deleted)
mariadbd    1118  39721 mariadbd               mysql    7u      REG              253,0         0   67887137 /var/tmp/#67887137 (deleted)
mariadbd    1118  39721 mariadbd               mysql    8u      REG              253,0         0   67887138 /var/tmp/#67887138 (deleted)
mariadbd    1118  39721 mariadbd               mysql   11u      REG              253,0         0   67887139 /var/tmp/#67887139 (deleted)
mariadbd    1118 424704 mariadbd               mysql    6u      REG              253,0         0   67887136 /var/tmp/#67887136 (deleted)
mariadbd    1118 424704 mariadbd               mysql    7u      REG              253,0         0   67887137 /var/tmp/#67887137 (deleted)
mariadbd    1118 424704 mariadbd               mysql    8u      REG              253,0         0   67887138 /var/tmp/#67887138 (deleted)
mariadbd    1118 424704 mariadbd               mysql   11u      REG              253,0         0   67887139 /var/tmp/#67887139 (deleted)
mariadbd    1118 425376 mariadbd               mysql    6u      REG              253,0         0   67887136 /var/tmp/#67887136 (deleted)
mariadbd    1118 425376 mariadbd               mysql    7u      REG              253,0         0   67887137 /var/tmp/#67887137 (deleted)
mariadbd    1118 425376 mariadbd               mysql    8u      REG              253,0         0   67887138 /var/tmp/#67887138 (deleted)
mariadbd    1118 425376 mariadbd               mysql   11u      REG              253,0         0   67887139 /var/tmp/#67887139 (deleted)
mariadbd    1118 425377 mariadbd               mysql    6u      REG              253,0         0   67887136 /var/tmp/#67887136 (deleted)
mariadbd    1118 425377 mariadbd               mysql    7u      REG              253,0         0   67887137 /var/tmp/#67887137 (deleted)
mariadbd    1118 425377 mariadbd               mysql    8u      REG              253,0         0   67887138 /var/tmp/#67887138 (deleted)
mariadbd    1118 425377 mariadbd               mysql   11u      REG              253,0         0   67887139 /var/tmp/#67887139 (deleted)
mariadbd    1118 425378 mariadbd               mysql    6u      REG              253,0         0   67887136 /var/tmp/#67887136 (deleted)
mariadbd    1118 425378 mariadbd               mysql    7u      REG              253,0         0   67887137 /var/tmp/#67887137 (deleted)
mariadbd    1118 425378 mariadbd               mysql    8u      REG              253,0         0   67887138 /var/tmp/#67887138 (deleted)
mariadbd    1118 425378 mariadbd               mysql   11u      REG              253,0         0   67887139 /var/tmp/#67887139 (deleted)
php-fpm     1318                              apache    3u      REG              253,0         0  201326855 /tmp/.ZendSem.h6hS3u (deleted)
php-fpm     1320                              apache    3u      REG              253,0         0  201326855 /tmp/.ZendSem.h6hS3u (deleted)
php-fpm     1321                              apache    3u      REG              253,0         0  201326855 /tmp/.ZendSem.h6hS3u (deleted)
php-fpm     1323                              apache    3u      REG              253,0         0  201326855 /tmp/.ZendSem.h6hS3u (deleted)
php-fpm     1325                              apache    3u      REG              253,0         0  201326855 /tmp/.ZendSem.h6hS3u (deleted)
dovecot     1671                                root  164u      REG               0,24         0       1632 /run/dovecot/login-master-notifydbbcf302fe536346 (deleted)
dovecot     1671                                root  169u      REG               0,24         0       1633 /run/dovecot/login-master-notifyc70eeb467771f398 (deleted)
pmlogger    2477                                 pcp    1w      REG              253,0         0  134795463 /var/tmp/pmlogger_check.Knu5kxcEw/out (deleted)
php-fpm     2857                              apache    3u      REG              253,0         0  201326855 /tmp/.ZendSem.h6hS3u (deleted)
php-fpm     4261                              apache    3u      REG              253,0         0  201326855 /tmp/.ZendSem.h6hS3u (deleted)
php-fpm     4293                              apache    3u      REG              253,0         0  201326855 /tmp/.ZendSem.h6hS3u (deleted)
php-fpm     8110                              apache    3u      REG              253,0         0  201326855 /tmp/.ZendSem.h6hS3u (deleted)
php-fpm     8153                              apache    3u      REG              253,0         0  201326855 /tmp/.ZendSem.h6hS3u (deleted)
php-fpm     8174                              apache    3u      REG              253,0         0  201326855 /tmp/.ZendSem.h6hS3u (deleted)
php-fpm     8197                              apache    3u      REG              253,0         0  201326855 /tmp/.ZendSem.h6hS3u (deleted)
bash      415464                              apache    8w      REG              253,0         0  201326760 /tmp/coolwsd.lock (deleted)
AppRun    415465                              apache    8w      REG              253,0         0  201326760 /tmp/coolwsd.lock (deleted)
Collabora 415468                              apache    8w      REG              253,0         0  201326760 /tmp/coolwsd.lock (deleted)
Collabora 415468 415469 Collabora             apache    8w      REG              253,0         0  201326760 /tmp/coolwsd.lock (deleted)
coolwsd   415555                              apache    8w      REG              253,0         0  201326760 /tmp/coolwsd.lock (deleted)
coolwsd   415555 415568 asyncdns              apache    8w      REG              253,0         0  201326760 /tmp/coolwsd.lock (deleted)
coolwsd   415555 415569 prisoner_             apache    8w      REG              253,0         0  201326760 /tmp/coolwsd.lock (deleted)
coolwsd   415555 415578 remotefon             apache    8w      REG              253,0         0  201326760 /tmp/coolwsd.lock (deleted)
coolwsd   415555 415579 accept_po             apache    8w      REG              253,0         0  201326760 /tmp/coolwsd.lock (deleted)
coolwsd   415555 415580 websrv_po             apache    8w      REG              253,0         0  201326760 /tmp/coolwsd.lock (deleted)
coolwsd   415555 415581 admin                 apache    8w      REG              253,0         0  201326760 /tmp/coolwsd.lock (deleted)
forkit    415570                              apache    8w      REG              253,0         0  201326760 /tmp/coolwsd.lock (deleted)
forkit    415570                              apache   18u      REG              253,0      1034   68581494 /tmp/lu41557018q5dc.tmp/user/extensions/bundled/extensions.pmap (deleted)
kit_spare 415577                              apache    8w      REG              253,0         0  201326760 /tmp/coolwsd.lock (deleted)
kit_spare 415577                              apache   18u      REG              253,0      1034   68581494 /tmp/lu41557018q5dc.tmp/user/extensions/bundled/extensions.pmap (deleted)
kit_spare 416404                              apache    8w      REG              253,0         0  201326760 /tmp/coolwsd.lock (deleted)
kit_spare 416404                              apache   18u      REG              253,0      1034   68581494 /tmp/lu41557018q5dc.tmp/user/extensions/bundled/extensions.pmap (deleted)
kit_spare 416405                              apache    8w      REG              253,0         0  201326760 /tmp/coolwsd.lock (deleted)
kit_spare 416405                              apache   18u      REG              253,0      1034   68581494 /tmp/lu41557018q5dc.tmp/user/extensions/bundled/extensions.pmap (deleted)
kit_spare 416406                              apache    8w      REG              253,0         0  201326760 /tmp/coolwsd.lock (deleted)
kit_spare 416406                              apache   18u      REG              253,0      1034   68581494 /tmp/lu41557018q5dc.tmp/user/extensions/bundled/extensions.pmap (deleted)
cockpit-s 421064                                root    7r      REG                0,1       122       7383 /memfd:cockpit login messages (deleted)
cockpit-b 421088                                root    3u      REG                0,1      8192       3266 /memfd:libffi (deleted)
cockpit-b 421088 421089 cockpit-b               root    3u      REG                0,1      8192       3266 /memfd:libffi (deleted)
cockpit-b 421088 421090 cockpit-b               root    3u      REG                0,1      8192       3266 /memfd:libffi (deleted)
cockpit-b 421088 421095 cockpit-b               root    3u      REG                0,1      8192       3266 /memfd:libffi (deleted)
dbus-brok 421098                                root   12u      REG                0,1   2097152       1230 /memfd:dbus-broker-log (deleted)

hs303 · December 17, 2024, 1:44pm

So it seems the files are being held by the mariadb process (PID 1118). So if it is safe for your application you can just run systemctl restart mariadb and it should free the kept files.
And also you did not change the values for free blocks in the short python command. You can calculate the free disk size by taking free blocks x block size in bytes.
In your case (assuming block size of 4096 bytes) and taking your number of free blocks 2444440 it would perfectly allign with your output of df.

[jp@rocky-wsl ~]$ python3 -c "print(f'{2444440*4096:,} blocks sum up to {2444440*4096/1024/1024/1024:.2f} GiB')"
10,012,426,240 blocks sum up to 9.32 GiB

jlehtone · December 17, 2024, 5:03pm

Note: the coreutils has numfmt, although it rounds differently:

X=$[2444440*4096] ; echo "$(numfmt --grouping $X) blocks sum up to $(numfmt --to=iec-i $X)" ; unset X
10,012,426,240 blocks sum up to 9.4Gi

Smoky · December 20, 2024, 10:25am

Finally I had a bit of time to dig in further. There is still a big gap bewteen “df -h” and
“sudo du -d1 -hx /” I cannot re-concile. The gap is 41GB (57G - 16G)
I also re-started mariadb.

df -h
Filesystem           Size  Used Avail Use% Mounted on
devtmpfs             4.0M     0  4.0M   0% /dev
tmpfs                7.7G     0  7.7G   0% /dev/shm
tmpfs                3.1G  135M  3.0G   5% /run
/dev/mapper/rl-root   70G   57G   14G  81% /
/dev/mapper/rl-home  853G  123G  730G  15% /home
/dev/nvme0n1p1      1014M  449M  566M  45% /boot
/dev/sda1            1.8T  946G  794G  55% /mnt/media
/dev/sdb1            1.8T  970G  771G  56% /mnt/backup02
/dev/sdc             1.8T  971G  863G  53% /mnt/backup01
tmpfs                1.6G  4.0K  1.6G   1% /run/user/1000

sudo du -d1 -hx /
30M	/etc
27M	/root
12G	/var
3.4G	/usr
0	/media
0	/mnt
679M	/opt
0	/srv
36M	/tmp
0	/afs
16G	/

I analyzed /opt and /usr.
From what I see, I think 21GB comes close to the space actually used.

sudo du -d1 -hx /usr/ (biggest directories only)
415M	/usr/bin
852M	/usr/lib
976M	/usr/lib64
880M	/usr/share

sudo du -d1 -hx /opt/
625M	/opt/emby-server
54M	    /opt/remi
679M	/opt

Could it be that the partition is corrupt? How can I check this?
What else can cause the gap?

Thanks,

Wolfgang

iwalker · December 20, 2024, 10:41am

Sometimes a reboot is needed to release space - have had that before, have you restarted the machine yet?

Smoky · December 20, 2024, 3:19pm

Hello iwalker, I shutdown the server and rebooted. Result is the same

Last login: Fri Dec 20 11:03:43 2024 from 10.5.2.145
[wp.rauchholz@home ~]$ df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 4.0M 0 4.0M 0% /dev
tmpfs 7.7G 0 7.7G 0% /dev/shm
tmpfs 3.1G 9.1M 3.1G 1% /run
/dev/mapper/rl-root 70G 58G 13G 82% /
/dev/mapper/rl-home 853G 123G 730G 15% /home
/dev/nvme0n1p1 1014M 449M 566M 45% /boot
/dev/sda1 1.8T 946G 794G 55% /mnt/media
/dev/sdb1 1.8T 970G 771G 56% /mnt/backup02
/dev/sdc 1.8T 971G 863G 53% /mnt/backup01
tmpfs 1.6G 4.0K 1.6G 1% /run/user/1000

sudo du -d1 -hx /
30M /etc
27M /root
13G /var
3.4G /usr
0 /media
0 /mnt
679M /opt
0 /srv
24K /tmp
0 /afs
17G /

iwalker · December 20, 2024, 5:27pm

Just in case there are files instead of just directories at /, can you try this instead?

cd /
du -sh *

rbtnichols · December 21, 2024, 2:49pm

Another possibility is files hidden under an active mount point directory, perhaps due to a backup being performed while the backup device was not actually mounted. Try this:

mkdir /tmp/tmpmount
mount --bind / /tmp/tmpmount
du -h -d 1 /tmp/tmpmount

That will show you what the root filesystem would look like with nothing mounted. After checking and perhaps cleaning up, just

umount /tmp/tmpmount
rmdir /tmp/tmpmount

Look at “Bind mount operation” in the bind manpage for details.

Smoky · December 23, 2024, 12:34pm

Hello iwalker,
this is how it looks. I can’t see anything out of the normal.

sudo du -sh *

0 afs
0 bin
409M boot
0 dev
4.0K dhparams.pem
30M etc
117G home
0 lib
0 lib64
0 media
2.9T mnt
679M opt
du: cannot access ‘proc/1317905/task/1317905/fd/4’: No such file or directory
du: cannot access ‘proc/1317905/task/1317905/fdinfo/4’: No such file or directory
du: cannot access ‘proc/1317905/fd/4’: No such file or directory
du: cannot access ‘proc/1317905/fdinfo/4’: No such file or directory
0 proc
27M root
102M run
0 sbin
0 srv
0 sys
36M tmp
3.4G usr
13G var

iwalker · December 23, 2024, 12:47pm

Yeah me either, seems really strange.

hs303 · December 23, 2024, 4:15pm

Maybe the file system is corrupt. You can check it with xfs_repair -n /dev/mapper/rl-root (on an unmounted file system).
man 8 xfs_repair

-n No modify mode. Specifies that xfs_repair should not modify the filesystem but should only scan the filesystem and indicate what repairs would have been made.

If you should want to repair it, it is probably best to back the data up first. I guess you can only run xfs_repair on an unmounted file system, so you would need to boot from an install media in order to run it.

Smoky · December 27, 2024, 2:05pm

Thank you rbtnichols
Finally I had some time after Xmas to look into this again.
This is the output.

0 /tmp/tmpmount/boot
0 /tmp/tmpmount/dev
0 /tmp/tmpmount/home
0 /tmp/tmpmount/proc
0 /tmp/tmpmount/run
0 /tmp/tmpmount/sys
30M /tmp/tmpmount/etc
27M /tmp/tmpmount/root
16G /tmp/tmpmount/var
3.4G /tmp/tmpmount/usr
0 /tmp/tmpmount/media
40G /tmp/tmpmount/mnt
679M /tmp/tmpmount/opt
0 /tmp/tmpmount/srv
24K /tmp/tmpmount/tmp
0 /tmp/tmpmount/afs
60G /tmp/tmpmount

I am not sure I understand this right. There are 40G in tmp/tmpmount/mnt.
3 external USB harddisks are mounted under /mnt. 2 backups and one to store movies, pictures, etc…
I am not sure where these 40G come from.
I changed the directory into /media, but cannot find anything there.

pwd
/mnt
ls -ahl
total 96Kdrwxr-xr-x. 5 root root 51 Dec 27 14:42 .
dr-xr-xr-x. 18 root root 255 Dec 16 09:32 …
dr-xr-xr-x. 8 root root 12K Jun 18 2024 backup01
drwx------. 7 wp.rauchholz wp.rauchholz 4.0K Nov 12 16:22 backup02
drwxr-xr-x. 5 emby emby 80K Dec 16 16:27 media

You mentioned in your email the possibility that a backup could have been performed while the backup device was not actually mounted.
This is actually the case. One of the external harddisks broke and I had to change it.

Thanks