Looks like SPICE could do it, found this comment on Wikipedia:
Xspice
The X.Org Server driver for the QXL framebuffer device includes a wrapper script[11] which makes it possible to launch an Xorg server whose display is exported via the SPICE protocol. This enables use of SPICE in a remote desktop environment, without requiring QEMU/KVM virtualization.