Install glusterfs with ssl encryption on rocky linux

I would like to set up glusterfs for /home and have secure communication. Two (or maybe later three) systems (server-011, server-033), both need to be simultaneously server and client.

The problem seems to be the SSL connection, what I have done so far:

I looked at these two FAQs:

https://www.cyberciti.biz/faq/how-to-enable-tlsssl-encryption-with-glusterfs-storage-cluster-on-linux/

As a starting point:

[root@server-011 ~]# lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
sda           8:0    0  100G  0 disk 
├─sda1        8:1    0    1G  0 part /boot
└─sda2        8:2    0   99G  0 part 
  ├─rl-root 253:0    0   70G  0 lvm  /
  ├─rl-swap 253:1    0    3G  0 lvm  [SWAP]
  └─rl-home 253:2    0  126G  0 lvm  /home
sdb           8:16   0  100G  0 disk 
└─sdb1        8:17   0  100G  0 part 
  └─rl-home 253:2    0  126G  0 lvm  /home
sr0          11:0    1 1024M  0 rom  

I want to use the sdb drive for Glusterfs, so the partitions are adjusted accordingly:

[root@server-033 ~]# lsblk
NAME         MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
sda            8:0    0  100G  0 disk 
├─sda1         8:1    0    1G  0 part /boot
└─sda2         8:2    0   99G  0 part 
  ├─rl-root  253:0    0   70G  0 lvm  /
  ├─rl-swap  253:1    0    3G  0 lvm  [SWAP]
  └─rl-home  253:2    0   26G  0 lvm  /home
sdb            8:16   0  100G  0 disk 
└─sdb1         8:17   0  100G  0 part 
  └─rl-ghome 253:3    0  100G  0 lvm  
sr0           11:0    1 1024M  0 rom  

glusterfs:

yum install centos-release-gluster -y
yum --enablerepo=crb install python3-pyxattr -y
yum install glusterfs-server -y

fstab:

/dev/mapper/rl-home     /homeBackup                   xfs     defaults        0 0
/dev/mapper/rl-ghome     /mnt/glusterfs-data                   xfs     defaults        0 0
localhost:/home /home glusterfs defaults,_netdev 0 0

firewall
firewall-cmd --zone=public --add-service=glusterfs --permanent

I needed to use force by creating the home volume (“parent directory is already part of a volume”):
gluster volume create home replica 2 transport tcp server-011:/mnt/glusterfs-data/gluster-brick server-033:/mnt/glusterfs-data/gluster-brick force

At this point, the volume is working, but I need a secure communication and only authorized servers should communicate to each other.

SSL setup:

cd /etc/ssl/
openssl genrsa -out glusterfs.key 2048
openssl req -new -x509 -key glusterfs.key -subj "/CN=${HOSTNAME}" -out glusterfs.pem -days 3650

only on server-033:

scp root@server-011:/etc/ssl/glusterfs.pem /etc/ssl/server-011.pem
cat /etc/ssl/glusterfs.pem /etc/ssl/server-011.pem > /etc/ssl/glusterfs.ca

only on server-011:

scp root@server-033:/etc/ssl/glusterfs.pem /etc/ssl/server-033.pem
cat /etc/ssl/glusterfs.pem /etc/ssl/server-033.pem > /etc/ssl/glusterfs.ca
touch /var/lib/glusterd/secure-access
systemctl restart glusterfsd.service
systemctl restart glusterd.service

Somehow I need the glusterfs.key on another path, than in the FAQ. glusterd.log:
E [socket.c:4241:ssl_setup_connection_params] 0-management: could not load private key at /etc/pki/tls/glusterfs.key
I created:

ln -s /etc/ssl/glusterfs.pem /etc/pki/tls/glusterfs.pem
ln -s /etc/ssl/glusterfs.key /etc/pki/tls/glusterfs.key

The peers are still not connected:

[root@server-033 glusterfs]# gluster peer status
Number of Peers: 1

Hostname: server-011
Uuid: 342181d2-0a34-4e96-b75c-3d4048ba6529
State: Peer in Cluster (Disconnected)

New error from the gluster logfile:

[2024-03-26 14:45:42.475243 +0000] I [socket.c:4107:ssl_setup_connection_params] 0-management: SSL support for MGMT is ENABLED IO path is ENABLED certificate depth is 1 for peer 192.168.56.104:24007
[2024-03-26 14:45:42.486170 +0000] E [socket.c:4249:ssl_setup_connection_params] 0-management: could not load CA list
[2024-03-26 14:45:42.486828 +0000] I [socket.c:4107:ssl_setup_connection_params] 0-socket.management: SSL support for MGMT is ENABLED IO path is ENABLED certificate depth is 1 for peer 192.168.56.104:49151
[2024-03-26 14:45:42.492996 +0000] E [socket.c:4249:ssl_setup_connection_params] 0-socket.management: could not load CA list
[2024-03-26 14:45:42.493142 +0000] E [socket.c:222:ssl_dump_error_stack] 0-socket.management:   error:0A000126:SSL routines::unexpected eof while reading
[2024-03-26 14:45:45.495282 +0000] I [socket.c:4107:ssl_setup_connection_params] 0-management: SSL support for MGMT is ENABLED IO path is ENABLED certificate depth is 1 for peer 192.168.56.104:24007
[2024-03-26 14:45:45.498217 +0000] E [socket.c:4249:ssl_setup_connection_params] 0-management: could not load CA list

If I check the ssl cert:

openssl s_client -showcerts -connect server-011:24007 -servername server-011
...
read R BLOCK
80CBF0E6997F0000:error:0A000126:SSL routines:ssl3_read_n:unexpected eof while reading:ssl/record/rec_layer_s3.c:320:

What did I do wrong in the configuration?

I missed this one:

ln -s /etc/ssl/glusterfs.ca /etc/pki/tls/glusterfs.ca

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.