qemu/ui
Daniel P. Berrange ada8d2e436 ui: fix VNC client throttling when forced update is requested
The VNC server must throttle data sent to the client to prevent the 'output'
buffer size growing without bound, if the client stops reading data off the
socket (either maliciously or due to stalled/slow network connection).

The current throttling is very crude because it simply checks whether the
output buffer offset is zero. This check is disabled if the client has requested
a forced update, because we want to send these as soon as possible.

As a result, the VNC client can cause QEMU to allocate arbitrary amounts of RAM.
They can first start something in the guest that triggers lots of framebuffer
updates eg play a youtube video. Then repeatedly send full framebuffer update
requests, but never read data back from the server. This can easily make QEMU's
VNC server send buffer consume 100MB of RAM per second, until the OOM killer
starts reaping processes (hopefully the rogue QEMU process, but it might pick
others...).

To address this we make the throttling more intelligent, so we can throttle
full updates. When we get a forced update request, we keep track of exactly how
much data we put on the output buffer. We will not process a subsequent forced
update request until this data has been fully sent on the wire. We always allow
one forced update request to be in flight, regardless of what data is queued
for incremental updates or audio data. The slight complication is that we do
not initially know how much data an update will send, as this is done in the
background by the VNC job thread. So we must track the fact that the job thread
has an update pending, and not process any further updates until this job is
has been completed & put data on the output buffer.

This unbounded memory growth affects all VNC server configurations supported by
QEMU, with no workaround possible. The mitigating factor is that it can only be
triggered by a client that has authenticated with the VNC server, and who is
able to trigger a large quantity of framebuffer updates or audio samples from
the guest OS. Mostly they'll just succeed in getting the OOM killer to kill
their own QEMU process, but its possible other processes can get taken out as
collateral damage.

This is a more general variant of the similar unbounded memory usage flaw in
the websockets server, that was previously assigned CVE-2017-15268, and fixed
in 2.11 by:

  commit a7b20a8efa
  Author: Daniel P. Berrange <berrange@redhat.com>
  Date:   Mon Oct 9 14:43:42 2017 +0100

    io: monitor encoutput buffer size from websocket GSource

This new general memory usage flaw has been assigned CVE-2017-15124, and is
partially fixed by this patch.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-id: 20171218191228.31018-11-berrange@redhat.com
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2018-01-12 13:48:54 +01:00
..
keycodemapdb@10739aa260 ui: pull in latest keycodemapdb 2017-10-23 10:50:02 +02:00
shader opengl: add flipping vertex shader 2017-10-17 10:25:42 +02:00
cocoa.m ui/cocoa.m: Send ctrl-alt key combos to guest if QEMU isn't using them 2017-11-07 10:14:14 +00:00
console-gl.c ui: use QEMU_IS_ALIGNED macro 2017-11-10 14:27:29 +01:00
console.c ui: fix dcl unregister 2017-11-10 11:06:43 +01:00
curses.c console: purge curses bits from console.h 2017-09-29 10:36:33 +02:00
curses_keys.h ui: add next and prior keysyms 2017-07-27 14:23:09 +02:00
cursor.c ui: use DIV_ROUND_UP 2017-08-31 12:29:07 +02:00
cursor_hidden.xpm ui: move files to ui/ and include/ui/ 2012-12-19 08:31:30 +01:00
cursor_left_ptr.xpm ui: move files to ui/ and include/ui/ 2012-12-19 08:31:30 +01:00
egl-context.c egl: explicitly ask for core context 2017-05-12 12:02:48 +02:00
egl-headless.c egl-headless: add dmabuf support 2017-10-17 10:25:42 +02:00
egl-helpers.c egl-helpers: add egl_texture_blit and egl_texture_blend 2017-10-17 10:25:42 +02:00
gtk-egl.c opengl: move shader init from console-gl.c to shader.c 2017-10-17 10:25:42 +02:00
gtk-gl-area.c ui: opengl updates for dma-buf support. 2017-10-19 12:09:53 +01:00
gtk.c ui/gtk: Fix deprecation of vte_terminal_copy_clipboard 2017-10-16 14:50:54 +02:00
input-keymap.c ui: generate qcode to linux mappings 2017-12-14 15:24:30 -08:00
input-legacy.c ui: fix crash with sendkey and raw key numbers 2017-10-23 10:50:02 +02:00
input-linux.c ui: move qemu_input_linux_to_qcode() 2017-07-27 14:23:09 +02:00
input.c ui: normalize the 'sysrq' key into the 'print' key 2017-10-23 10:50:02 +02:00
keymaps.c General warn report fixups 2017-09-19 14:09:34 +02:00
keymaps.h ps2: fix sending of PAUSE/BREAK scancodes 2017-07-27 14:24:05 +02:00
Makefile.objs buildsys: Move sdl cflags/libs to per object 2017-09-22 10:20:34 +08:00
qemu-pixman.c pixman: drop submodule 2017-09-13 10:15:43 +02:00
qemu-x509.h ui: move files to ui/ and include/ui/ 2012-12-19 08:31:30 +01:00
sdl.c shutdown: Add source information to SHUTDOWN and RESET 2017-05-23 13:28:17 +02:00
sdl2-2d.c SDL2: add bgrx pixel format 2016-06-03 08:23:26 +02:00
sdl2-gl.c opengl: move shader init from console-gl.c to shader.c 2017-10-17 10:25:42 +02:00
sdl2-input.c ui: Clean up includes 2016-02-04 17:01:04 +00:00
sdl2-keymap.h sdl2: keymap fixups 2014-09-16 08:07:05 +02:00
sdl2.c sdl2: Fix broken display updating after the window is hidden 2017-11-16 09:57:47 +01:00
sdl_keysym.h ui/sdl2 : initial port to SDL 2.0 (v2.0) 2014-03-05 09:52:05 +01:00
sdl_zoom.c all: Remove unnecessary glib.h includes 2016-06-07 18:19:24 +03:00
sdl_zoom.h Clean up ill-advised or unusual header guards 2016-07-12 16:20:46 +02:00
sdl_zoom_template.h sdl: Fix heap smash in sdl_zoom_rgb{16,32} for int > 32 bits 2013-01-15 18:25:30 -06:00
shader.c opengl: add flipping vertex shader 2017-10-17 10:25:42 +02:00
spice-core.c QAPI patches for 2017-06-09 2017-06-22 11:34:39 +01:00
spice-display.c opengl: move shader init from console-gl.c to shader.c 2017-10-17 10:25:42 +02:00
spice-input.c ui: correctly detect spice PAUSE scancode sequence 2017-07-28 12:35:40 +02:00
trace-events ui: add tracing of VNC authentication process 2017-09-29 10:36:34 +02:00
vgafont.h ui: move files to ui/ and include/ui/ 2012-12-19 08:31:30 +01:00
vnc-auth-sasl.c ui: fix VNC client throttling when forced update is requested 2018-01-12 13:48:54 +01:00
vnc-auth-sasl.h ui: track how much decoded data we consumed when doing SASL encoding 2018-01-12 13:48:54 +01:00
vnc-auth-vencrypt.c ui: Always remove an old VNC channel watch before adding a new one 2017-10-04 13:21:53 +01:00
vnc-auth-vencrypt.h Clean up ill-advised or unusual header guards 2016-07-12 16:20:46 +02:00
vnc-enc-hextile-template.h pixman/vnc: use pixman images in vnc. 2012-11-01 14:00:04 +01:00
vnc-enc-hextile.c ui: Clean up includes 2016-02-04 17:01:04 +00:00
vnc-enc-tight.c vnc: use DIV_ROUND_UP 2017-08-31 12:29:07 +02:00
vnc-enc-tight.h Clean up header guards that don't match their file name 2016-07-12 16:19:16 +02:00
vnc-enc-zlib.c ui: Clean up includes 2016-02-04 17:01:04 +00:00
vnc-enc-zrle-template.c ui: Clean up includes 2016-02-04 17:01:04 +00:00
vnc-enc-zrle.c vnc: simple clean up 2017-05-12 12:34:31 +02:00
vnc-enc-zrle.h Clean up header guards that don't match their file name 2016-07-12 16:19:16 +02:00
vnc-enc-zywrle-template.c ui: Clean up includes 2016-02-04 17:01:04 +00:00
vnc-enc-zywrle.h Clean up header guards that don't match their file name 2016-07-12 16:19:16 +02:00
vnc-jobs.c ui: fix VNC client throttling when forced update is requested 2018-01-12 13:48:54 +01:00
vnc-jobs.h ui/vnc: Drop unused vnc_has_job() and vnc_jobs_clear() 2017-02-08 14:59:36 +01:00
vnc-palette.c all: Remove unnecessary glib.h includes 2016-06-07 18:19:24 +03:00
vnc-palette.h all: Clean up includes 2016-02-23 12:43:05 +00:00
vnc-ws.c ui: Always remove an old VNC channel watch before adding a new one 2017-10-04 13:21:53 +01:00
vnc-ws.h Clean up ill-advised or unusual header guards 2016-07-12 16:20:46 +02:00
vnc.c ui: fix VNC client throttling when forced update is requested 2018-01-12 13:48:54 +01:00
vnc.h ui: fix VNC client throttling when forced update is requested 2018-01-12 13:48:54 +01:00
vnc_keysym.h ui: add next and prior keysyms 2017-07-27 14:23:09 +02:00
x_keymap.c ui: Clean up includes 2016-02-04 17:01:04 +00:00
x_keymap.h Delete useless 'extern' qualifiers for functions 2011-01-23 16:21:20 +00:00