I used the RL8.5 instructions to install Mate per this post here:
Install Mate DE
It may be that you are having trouble with the window manager and or wayland. Mate does not work with wayland so you need to have all the necessary Xorg files. Using the instructions in the link above should bring them in.