vnc-enc-tight: Fix divide-by-zero in tight_detect_smooth_image{16,24,32}

Spotted by Coverity:

(1) Event assignment:  Assigning: "pixels" = "0".
(2) Event cond_true:  Condition "y < h", taking true branch
(3) Event cond_false:  Condition "x < w", taking false branch
(4) Event loop_end:  Reached end of loop
(5) Event divide_by_zero:  In expression "(stats[0] + stats[1]) * 100U / pixels",
division by expression "pixels" which may be zero has undefined behavior.

290     DEFINE_DETECT_FUNCTION(16)
291     DEFINE_DETECT_FUNCTION(32)

Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
This commit is contained in:
Gonglei 2014-05-28 21:21:35 +08:00 committed by Gerd Hoffmann
parent 4006617552
commit b52991537c

View file

@ -181,6 +181,10 @@ tight_detect_smooth_image24(VncState *vs, int w, int h)
}
}
if (pixels == 0) {
return 0;
}
/* 95% smooth or more ... */
if (stats[0] * 33 / pixels >= 95) {
return 0;
@ -267,7 +271,9 @@ tight_detect_smooth_image24(VncState *vs, int w, int h)
y += w; \
} \
} \
\
if (pixels == 0) { \
return 0; \
} \
if ((stats[0] + stats[1]) * 100 / pixels >= 90) { \
return 0; \
} \