[ubuntu] Seamless VM desktop integration with Libvirt VGA Passthrough?

I'm running a triple-head Ubuntu 16.10 system with a virtualized Windows 7 guest using VGA passthrough. That means the Windows guest has its own dedicated display adapter, so it can have high-performance 3D. In order to get VGA passthrough, I had to switch hypervisors from VirtualBox to Qemu/KVM/Libvirt. If these concepts are new to you, here's some terminology.

The main thing that I miss is VirtualBox's (relatively) seamless fullscreen monitor/keyboard/mouse integration. Mouse cursor on the monitor that has a Windows desktop? Mouse/keyboard captured. Mouse cursor not on a Windows Desktop? Clicks and keystrokes go to the host. Just switched? Clipboard is persistent. Awesome.

VGA passthrough is a different animal. The UX feels like two totally separate physical machines sharing a box. I physically change the input on my monitor. Physically switch keyboard and mouse to the VM. No clipboard integration at all. It's just icky, there's no fluid UX between the machines at all.

I'm here for suggestions and/or discussion on whether there's a way to improve integration of the VGA-Passthrough VM with the Linux desktop. Here's one idea that occurred to me, but I'd really like to hear other suggestions.