The Display Manager (gdm) is graphical. Hence it runs with some Window Manager, even though the user can choose WM (X11 or Wayland). Logically, it uses Wayland?
There is config file /etc/gdm/custom.conf that has “comment”:
#WaylandEnable=false
If one uncomments that:
WaylandEnable=false
and reboots, then both gdm and all user sessions would be forced to use X11.
Would that make a difference? I don’t know.
Does the vmware offer multiple “video device” options? Would some of them behave better than the one you use now?
When it says “boots up”, that implies the text based console before you get to Gnome Display Manager (gdm), and then you log in (using gui), and at that point you get the real desktop.
But what does “display” mean anyway, when it’s a guest VM; some client is able to connect to the guest and interpret a screen widget and display it on that client?