It seemed like a simple request. All I wanted was for my Linux virtual machine to stay at the 1920×1080 resolution I specified. VMware Tools, however, had different ideas and resized my VM’s screen resolution every time I resized the window in the remote console. Normally this is no biggie, but users were connecting into this box with VNC and the VNC server was freaking out when the resolution changed. Google has dozens of blog posts and forum questions about resolving this issue in Windows VMs and in VMware Fusion/Workstation, but I couldn’t find anything on the fixing this in ESXi 6.7 even after an hour of searching.
Finally I dug into my local filesystem and found the culprit: a dynamic library called libresolutionKMS.so. On my CentOS 7 VM, it was located in /usr/lib64/open-vm-tools/plugins/vmsvc. Like the Windows equivalent, moving or deleting this file and rebooting was sufficient to stop this issue in its tracks. In my case, I moved it back to my home directory to make sure nothing broke, but it appears that this stopped the auto-resizing without affecting any other operations, and VMware still reported the VMtools as running.
As this is Linux, open-vm-tools is updated by my package manager, so this file will need to be moved or deleted each time after open-vm-tools is updated. Given the lack of any other way to disable this behavior, this seems like a workable kludge for my situation.
FYI: On a Centos 6 host I additionally had to move a file called libresolutionSet.so. But, thanks very much for this tip! Just incredible that Vmware doesn’t provide a configuration operation to turn off guest re-scale.