We were storing the output description into the name field, which was
harmless but did cause a leak.
Signed-off-by: Daniel Stone <daniels@collabora.com>
This adds a new test checking that output decorations (used in the
wayland-backend, here tested with headless-backend) are drawn correctly.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Only the 'image' field of struct buffer is ever used here, so just
pass pixman_image_t instead of struct buffer.
This allows a future test to mangle a screenshot (e.g. convert pixel
format) before feeding it in.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Migrate the tst suite to using the new screenshooting protocol. This
ensures the new protocol and implementation work, and removes a user of
the old protocol so that the old protocol can be removed in the future.
Now that the compositor chooses the pixel format,
capture_screenshot_of_output() needs to convert to the expected format
in the tests when necessary.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Instead of starting yet another hand-crafted pixel format mapping table,
use the one we have.
Following patches want to be able to create XRGB8888 buffers, and later
even other formats.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
These have been in wayland a while back with version 1.20.0.
We also need to update the test client helper with this bump, as
those bind to version 4.
Signed-off-by: Marius Vlad <marius.vlad@collabora.com>
The new field in struct scalar_stat allows recording all tested values
into a file. This is intended to replace ad hoc dumping code like in
alpha-blending-test.c.
To make it easy to set up, also offer a helper to open a writable file
whose name consists of a custom prefix and test name.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
These are the last places in weston-test-client-helper.c where using
image_header_from() will reduce the code line count and simplify the
code a little.
No change in behaviour.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
I will be needing it in a new test, so let's share it.
For convenience, this also adds client back-pointer in struct surface so
that I don't need to pass client explicitly.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
skip() is a left-over from an old test harness design, the comment even
refers to automake. Calling skip() cannot do anything good anymore,
because it wouldn't print the skips in the TAP report, so it would
probably be considered a failure.
Delete this unused and nowadays incorrect function, so it doesn't
confuse people.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Store the pointer serial for events that provide one, so that it can be
used by tests to send requests that require it (e.g., setting the cursor
surface).
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
expect_protocol_error() ensures that the connection has failed in the
expected way. To satisfy ASan leak detection, we still need to tear down
everything, including call client_destroy().
client_destroy() needs to check that tear-down does not cause protocol
errors, so it does one last roundtrip that checks that is succeeds. But
if the connection is already down in an expected way, this roundtrip
cannot succeed and must be skipped.
Also moves the roundtrip under 'if (wl_display)', assuming the 'if' is
there for a reason - but obviously that reason was never used as it
would have crashed.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Fixes a leak found by ASan in alpha-blending-test.
Direct leak of 160 byte(s) in 2 object(s) allocated from:
#0 0x7f511fe11518 in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0xe9518)
#1 0x7f511fc76373 in zalloc ../../git/wayland/src/wayland-private.h:232
#2 0x7f511fc76373 in proxy_create ../../git/wayland/src/wayland-client.c:422
#3 0x7f511fc79dcc in create_outgoing_proxy ../../git/wayland/src/wayland-client.c:651
#4 0x7f511fc79dcc in wl_proxy_marshal_array_constructor_versioned ../../git/wayland/src/wayland-client.c:736
#5 0x7f511fc7b17b in wl_proxy_marshal_constructor_versioned ../../git/wayland/src/wayland-client.c:873
#6 0x5583e5348f43 in wl_registry_bind /home/pq/local/include/wayland-client-protocol.h:1165
#7 0x5583e534cfbe in handle_global ../../git/weston/tests/weston-test-client-helper.c:800
#8 0x7f511f34b8ed in ffi_call_unix64 (/lib/x86_64-linux-gnu/libffi.so.6+0x68ed)
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This refactors a new function verify_image() out of
verify_screen_content().
verify_image() will be useful with a test that verifies a screenshot
against a reference image but also wants to do additional testing on the
screenshot.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Const has documentary value saying the code will not modify the
parameter contents. Everything that can be const, should be const.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Do not write out PNG files for successful screenshot tests. It clutters
the build directory, but the biggest reason is to keep the CI artifacts
smaller.
internal-screenshot test still writes a PNG, it's good to keep one PNG
written so that we exercise the PNG writing code always.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Until now we had two different protocol extensions: one for the
test suite screenshooter and other for the screenshooter client.
As they are identical and this won't change, make the test suite
use the same protocol that the screenshooter client uses.
Besides the cleanup, this change will also allow us to use the
DRM writeback screenshooter in the test suite, as the test suite
implementation is hardcoded to use a renderer based screenshooter.
Signed-off-by: Leandro Ribeiro <leandro.ribeiro@collabora.com>
The 'struct test' has a field 'int buffer_copy_done', but it
is in a fact a boolean. Change it to 'bool buffer_copy_done'.
Signed-off-by: Leandro Ribeiro <leandro.ribeiro@collabora.com>
The wl_drm protocol is not being used by the test client. So
remove 'bool has_wl_drm' from 'struct client' and also the
branch that initializes this variable in handle_global().
Signed-off-by: Leandro Ribeiro <leandro.ribeiro@collabora.com>
There will be a new test program using viewports and would like to share this
bit of code.
There are two behavioral changes:
- Compositor wp_viewporter interface version is no longer checked.
- client_create_viewport() does not leak the viewporter object.
test_viewporter_double_create needs to call bind_to_singleton_global() itself
so that the viewporter object still exists when the error event arrives.
Otherwise error verification fails.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
When a test fails and it produces a difference image, also compute the min/max
per-channel signed difference statistics. These numbers can be used to adjust
the fuzz needed for fuzzy_match_pixels() to pass. Otherwise one would have to
manually inspect the reference and result images and figure out the values.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This goes through all output transforms with two different buffer transforms
and verifies the visual output against reference images.
This commit introduces a new test input image 'basic-test-card.png'. It is a
small image with deliberately odd and indivisible dimensions to provoke bad
assumptions about image sizes. It contains red, green and blue areas which are
actually text that makes it very obvious if you have e.g. color channels
swapped. It has a white thick circle to highlight aspect ratio issues, and an
orange cross to show a mixed color. The white border is for contrast and a 1px
wide detail. The whole design makes it clear if the image happens to be rotated
or flipped in any way.
The image has one pixel wide transparent border so that bilinear sampling
filter near the edges of the image would produce the same colors with both
Pixman- and GL-renderers which handle the out-of-image samples fundamentally
differently: Pixman assumes (0, 0, 0, 0) samples outside of the image, while
GL-renderer clamps sample coordinates to the edge essentially repeating the
edge pixels.
It would have been "easy" to create a full matrix of
every output scale & transform x every buffer scale & transform, but that
would have resulted in 2 renderers * 8 output transforms * 3 output scales *
8 buffer transforms * 3 buffer scales = 1152 test cases that would have all
ran strictly serially because our test harness has no parallelism inside one
test program. That would have been slow to run, and need a lot more reference
images too.
Instead, I chose to iterate separately through all output scales & transforms
(this patch) and all buffer scales & transforms (next patch). This limits the
number of test cases in this patch to 56, and allows the two test programs to
run in parallel.
I did not even pick all possible scale & transform combinations here, but just
what I think is a representative sub-set to hopefully exercise all the code
paths.
https://gitlab.freedesktop.org/wayland/weston/issues/52
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Running with Mesa 20.1.0-devel (git-c7617d8908) GL renderer:
Radeon RX 550 Series (POLARIS11, DRM 3.27.0, 4.19.0-2-amd64, LLVM 8.0.1)
I found output-tranform test (a future patch) to produce exactly this much more
difference between Pixman and GL rendererers.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
It turns out that if the client is not explicitly destroyed, it will remain
connected until the compositor shuts down because there is no more a client
process that would terminate.
Usually this is not a problem, but if a test file has multiple screenshooting
tests, the windows from earlier tests in the file will remain on screen. That
is not wanted, hence implement client destruction.
To properly destroy a client, we also need a list of outputs. They used to be
simply leaked. This does not fix wl_registry.global_remove for wl_outputs, that
is left for a time when a test will actually need that.
This patch makes only ivi-shell-app test use the new client_destroy() to show
that it actually works. The added log scopes prove it: destroy requests get
sent. Sprinkling client_destroy() around in all other tests is left for a time
when it is actually necessary.
ivi-shell-app is a nicely simple test doing little else, hence I picked it.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Allow the reference image to be NULL or missing so that it does not even
attempt to load a reference image or compare it. You cannot just point the
reference image to an arbitrary image because the comparison functions can
abort due to size mismatch. This makes bootstrapping new tests easier when you
do not yet have a reference image.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
The old name felt too... short.
The return type is changed to bool; fits better for a success/failure.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This will be useful in more tests.
No changes to the code, aside from dropping one 'static'.
Copyright 2017 is taken from git-blame of the moved code.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
The fuzzy range will be used with GL-renderer testing, as it may produce
slightly different images than Pixman-renderer yet still correct results.
Such allowed differences are due to different rounding.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Instead of relying on Meson setting up environment so that Weston and tests
find all their files, build those values into the tests. This way one can
execute a test program successfully wihtout Meson, simply by running it.
The old environment variables are still honoured if set. This might change in
the future.
Baking the source or build directory paths into the tests should not regress
reproducible builds, because the binaries where test-config.h values are used
will not be installed.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
When we move on to TAP, stdout will be reserved for TAP and stderr is for free
chatter. Set up an example that tests should use testlog() instead of fprintf
or printf to chat in the right place.
Most statements were already printing to stderr, so this just makes then a
little shorter. There are also some statements that printed to stdout and are
now corrected.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This reverts 50b7b70835.
We didn't make Meson create a logs directory, so writing the images fails
because the directory does not exist. If you run a test without Meson, there is
even less expectation that it would write somewhere else than CWD by default.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
It is a public installed header used by libweston.h.
See "Rename compositor.h to libweston/libweston.h" for rationale.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
There is no automake anymore, I suppose it is ninja that handles it now.
There are still a couple references to automake left to point out where the
conventions originated, e.g. the exit code 77.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Introduce helper test code to implement the client side of the
input_timestamps_unstable_v1 protocol. This helper will be used in
upcoming commits to test the server side implementation of the protocol
in libweston.
The input_timestamps_unstable_v1 protocol was introduced in version 1.13
of wayland-protocols, so this commit updates the version dependency in
configure.ac accordingly.
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
The current test client code waits for all wl_seat globals to arrive
before checking them and deciding which one is the test seat global to
use for the input object. Test code that needs to add/remove test seats
would have to call the client_set_input() function for any seat changes
to take effect. Although we could allow this by making
client_set_input() public, we would be exposing unecessary
implementation details.
This commit applies any seat changes immediately upon arrival of the
seat name, freeing test code from needing to call extra functions like
client_set_input(). To achieve this the call to input_data_devices() is
moved from client_set_input() to the seat name event handler.
This commit also moves the check that all seats have names to an
explicit test. To support this test, inputs corresponding to non-test
seats are not destroyed (unless their seat global is removed), as
was previously the case.
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
The current test client code completely ignores removal of globals.
This commit updates the code to properly handle removal of globals in
general, and of seat globals in particular. This ensures that the test
client objects are in sync with the server and any relevant resources
are released accordingly.
This update will be used by upcoming tests to check that seat removal
and re-addition is working properly.
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Add test to verify the server correctly emits pointer axis events. This
requires updating the weston-test protocol with a new request for
pointer axis events.
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Add test to verify that the server correctly sets the timestamps of
touch events. This requires updating the weston-test protocol with a new
request for touch events.
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Add test to verify that the server correctly sets the timestamps of
keyboard key events. This requires updating the weston-test protocol to
support passing key event timestamps.
simple_keyboard_test now uses the create_client_with_keyboard_focus()
helper function which changes the initial state of the surface to be
focused. This leads to one additional iteration of the test loop when
starting, during which the surface is deactivated, i.e., loses focus.
After this initial iteration the test continues as before.
Furthermore, simple_keyboard_test now uses the send_key() helper
function which performs a roundtrip internally. To account for this, the
client_roundtrip() function is now directly called in the loop only when
it is still required, i.e., when deactivating the surface.
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Enhance the existing pointer motion and button event tests to
additionally verify the event timestamps. This requires updating the
weston-test protocol to support passing motion and button event
timestamps.
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
This doesn't attach a buffer to the surface. This is needed for the
next commit, where we have a test case with a surface that doesn't
have a buffer attached.
Signed-off-by: Emilio Pozuelo Monfort <emilio.pozuelo@collabora.co.uk>
Reviewed-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Logs is where we write all our custom test logs, let's also put the
screenshots in the same place by default from cluttering the base
directory.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Reviewed-by: Emilio Pozuelo Monfort <emilio.pozuelo@collabora.co.uk>
Reviewed-by: Micah Fedke <micah.fedke@collabora.co.uk>
Reviewed-by: Quentin Glidic <sardemff7+git@sardemff7.net>
buffer-count was introduced in line with a Mesa change which forced
an earlier block on frame events to try to enforce double-buffering
where available.
The Mesa change has since been reverted (Mesa commit 9ca6711faa), as
this had unpleasant interactions with buffer_age in particular, so this
test is no longer valid.
Additionally, it only worked on backends which initialised EGL (not
headless-backend, where tests generally run), which can be flaky due to
initialisation races. Not only that, but on the DRM backend, we can
legitimately enter triple-buffering due to promoting the surface to a
hardware plane, skipping GPU composition.
In light of all this, just remove the test.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Derek Foreman <derekf@osg.samsung.com>
Useful for pointing out where the image comparisons fail.
Internal-screenshot-test is modified to save the visualization if the
test fails.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Reviewed-by: Daniel Stone <daniels@collabora.com>
