The way the websockets TLS code was integrated into the VNC server
made it essentially useless. The only time that the websockets TLS
support could be used is if the primary VNC server had its existing
TLS support disabled. ie QEMU had to be launched with:
# qemu -vnc localhost:1,websockets=5902,x509=/path/to/certs
Note the absence of the 'tls' flag. This is already a bug, because
the docs indicate that 'x509' is ignored unless 'tls' is given.
If the primary VNC server had TLS turned on via the 'tls' flag,
then this prevented the websockets TLS support from being used,
because it activates the VeNCrypt auth which would have resulted
in TLS being run over a TLS session. Of course no websockets VNC
client supported VeNCrypt so in practice, since the browser clients
cannot setup a nested TLS session over the main HTTPS connection,
so it would not even get past auth.
This patch causes us to decide our auth scheme separately for the
main VNC server vs the websockets VNC server. We take account of
the fact that if TLS is enabled, then the websockets client will
use https, so setting up VeNCrypt is thus redundant as it would
lead to nested TLS sessions.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
f9148c8ae7