Commit graph

1428 commits

Author SHA1 Message Date
Martin Aeschlimann ad928a4e0f
adopt server script in integration tests 2022-01-19 18:08:04 +01:00
Benjamin Pasero c333d44ecd
update integration test docs how to compile 2022-01-19 07:29:38 +01:00
Sandeep Somavarapu 066671a902
Fix #140838 2022-01-18 17:43:20 +01:00
Benjamin Pasero 158cd6da42
smoke - log code download (#140769)
* smoke - log progress/errors of vscode download

//cc @connor4312

* reduce spam
2022-01-15 11:31:13 +01:00
Benjamin Pasero 9be2757019
🆙 playwright@1.18 (#140767)
* tests - use `@playwright/test@1.18.0-rc1`

* fix bad dependency

* clear more weird ts dependencies
2022-01-15 09:35:39 +01:00
Daniel Imms f2c755a878 Skip smoke test
Part of #139076
2022-01-14 09:47:56 -08:00
Daniel Imms 0b7bc6d73c Re-enable auto replies tests with retries on Windows only
Fixes #139076
2022-01-14 08:56:11 -08:00
Benjamin Pasero eecbd962ef
debt - rename iconv-lite-umd package (#140650) 2022-01-14 07:28:41 +01:00
João Moreno 8c9870d298
Smoke Test: forcefully kill electron if exit doesn't work (#140570)
* smoke test: forcefully kill electron if exit doesn't work

* use tree-kill

* roll back veto
2022-01-12 17:02:52 +01:00
Daniel Imms 860e935d1a Skip notebooks smoke test suite
Part of #140575
2022-01-12 07:35:56 -08:00
Benjamin Pasero af5de88a44
tests - add --sequential option and use in CI runs 2022-01-11 10:48:20 +01:00
Matt Bierner 9f867c3ed8
Revert service worker usage of MessageChannel (#140351)
* Revert service worker usage of MessageChannel

Reverts 66b6adf035

While I'm not 100% about this, I think 66b6adf035 causes resourses to occasionally not load. I believe this can happen if the service worker is unitilized while the webview remains active. I can't reproduce this myself so it may be related to memory pressure or resource usage, however relying on the service worker not being reinitilized does seem like a potentially bad idea https://stackoverflow.com/questions/34775105/what-causes-the-global-context-of-a-service-worker-to-be-reset

Will investigate if there's another way to achive this since using MessagePort did clean up the code and slightly improve performance

* Bump webview commit versions
2022-01-10 14:22:49 -08:00
Benjamin Pasero a224ef4fb3
smoke - fix selectTab to select tab? 2022-01-04 09:47:09 +01:00
Benjamin Pasero 18a7108073
smoke - select tab over quick pick (#139944)
* smoke - further try to reduce usages of quick pick

* poke
2021-12-30 22:38:59 +01:00
Benjamin Pasero 99fabbdb27
smoke - more logging 2021-12-29 17:46:52 +01:00
Benjamin Pasero bd43c04262
smoke - log electron stdout into log file 2021-12-29 17:35:54 +01:00
Benjamin Pasero 9c0cce1dae
smoke - take screenshot when waiting long for window ready 2021-12-28 15:32:35 +01:00
Benjamin Pasero b2e9e87c2e
smoke - skip flaky notebook test (#139672) 2021-12-23 17:12:29 +01:00
Benjamin Pasero b2e79a699f
smoke - improve app lifecycle (#139693)
* smoke - make sure app is closed when `app.start` fails

* smoke - `capturePage` to fallback to active window

* smoke - more logging around remote resolving
2021-12-23 16:38:54 +01:00
Daniel Imms daf4ba622f
Merge pull request #138516 from microsoft/tyriar/137847
Try removing timeout in checkWindowReady
2021-12-22 10:38:41 -08:00
Benjamin Pasero bb39ae82cf
smoke - improve runCommand and disable more tests 2021-12-20 08:25:08 +01:00
Benjamin Pasero 2d77c9e23a
smoke - more robust usage of quick input / access (#139367)
* smoke - more robust `openQuickAccessAndWait`

* tweaks

* 💄

* trigger

* fix fluke

* poke

* fix terminal weirdness

* poke

* address feedback
2021-12-17 18:49:17 +01:00
Daniel Imms 88fcbd84e0 Fix auto replies smoke tests
This was flaky because the shell's wrapping character may have come after the
input (eg. 'foo> bar'), the fix is to allow characters between the matching
string and reply via regex.

Fixes #139083
2021-12-17 09:42:50 -08:00
Benjamin Pasero 7141436baa
smoke - skip failing terminal test (#139353) 2021-12-17 08:21:58 +01:00
Benjamin Pasero 1a4a2714bf
smoke - skip failing terminal test (#139349) 2021-12-17 07:34:20 +01:00
Daniel Imms d33e3d2fbc Re-enable flaky auto replies tests
Need a new playwright trace from a failure to be able to diagnose

Part of #139083
2021-12-16 11:36:53 -08:00
Benjamin Pasero ae8aeb653d
smoke - use absolute paths for opening files (#139322) 2021-12-16 20:14:12 +01:00
Benjamin Pasero 9e22f1ccac
smoke - disable flaky notebook tests (#139270) 2021-12-16 15:55:15 +01:00
Daniel Imms dfc4077fb0 Use dom renderer for terminal smoke tests
Part of #139083
2021-12-16 06:38:11 -08:00
Sandeep Somavarapu 995d40b12d
#139183 enable the test and add a log statement 2021-12-16 14:34:47 +01:00
Benjamin Pasero fa739870a2
smoke - remove doubleClick usage 2021-12-16 14:27:21 +01:00
Benjamin Pasero fdc4b7101d
smoke - actually retry quick access usage 2021-12-16 14:25:12 +01:00
Benjamin Pasero 51eda13af4
smoke - move editor test into language test 2021-12-16 10:29:46 +01:00
Rob Lourens 86bdc567e9 Remove obsolete comment 2021-12-15 16:41:20 -08:00
Daniel Imms da308a9a7f Format 2021-12-15 19:35:16 +00:00
Daniel Imms fa61def811
Merge branch 'main' into tyriar/137847 2021-12-15 09:29:49 -08:00
Johannes Rieken 1b2c610ffe
skip bad test, https://github.com/microsoft/vscode/issues/139183 2021-12-15 10:51:01 +01:00
Rob Lourens b823936907 Wait for window to connect before starting browser smoketests
Fix #138045
2021-12-14 20:23:44 -08:00
Matt Bierner 8cf4dcdade
Re-enable notebook smoke tests (#139127)
Fixes #126371
2021-12-14 16:50:22 -08:00
Connor Peet 2b19344369
testing: debug test failures
For #137853
2021-12-14 11:29:14 -08:00
Daniel Imms 2eb4cec44b Wait for terminal to be ready before changing color
Fixes #139081
2021-12-14 10:14:06 -08:00
Daniel Imms 9c99eaa0b2 Also log exception in itSkipOnFail 2021-12-14 08:15:54 -08:00
Daniel Imms c29df4c2d5 Add itSkipOnFail util
Part of #139076
2021-12-14 08:05:43 -08:00
Benjamin Pasero 5526cd558f
smoke - remove only 2021-12-14 07:38:08 +01:00
Daniel Imms 9d1a7abe35 Some jsdoc 2021-12-13 16:57:14 -08:00
Daniel Imms 50f96a3c6b Clean up terminal creation in smoke tests 2021-12-13 16:54:42 -08:00
Daniel Imms 73a19337ac Merge remote-tracking branch 'origin/main' into tyriar/133524 2021-12-13 16:28:23 -08:00
Daniel Imms 9566e299dd Add auto replies test 2021-12-13 16:20:34 -08:00
Megan Rogge a2f5ec29b2
Revert "Revert "be sure that terminal target gets set for the instance (#138875)""
This reverts commit 8605d7aeff.
2021-12-13 17:08:29 -06:00
Megan Rogge 0602587ff2
re-enable smoke tests, keep editor fix (#139012) 2021-12-13 12:50:58 -06:00
Rob Lourens 7e4942dcb7 Revert "Re-enable additional notebook smoke tests"
This reverts commit b3b9d4e2e7.
2021-12-13 09:18:17 -08:00
Daniel Imms 8605d7aeff Revert "be sure that terminal target gets set for the instance (#138875)"
This reverts commit 578eb0d4ae.
2021-12-13 08:15:32 -08:00
Matt Bierner b3b9d4e2e7
Re-enable additional notebook smoke tests
For #126371

These pass for me locally
2021-12-10 17:08:02 -08:00
Matt Bierner 02c3551a59
Re-enable notebook smoke suite
For #126371

This single test passes for me. Will look at re-enabling individual tests next
2021-12-10 17:01:02 -08:00
Matt Bierner 66b6adf035
Switch webview service-worker to use message channel (#138811)
* Switch webview service-worker to use message channel

This change hooks the service worker used for loading webview resources directly up to the main VS Code process over a message channel. Previously this communication had to go through an extra hop through the webview

This simplifies the logic somewhat (although this change required adding extra logic to exchange the message port). It also improves performance a little

* Update webview content commit version
2021-12-10 16:35:13 -08:00
Megan Rogge 86ee106054
skip failing tests 2021-12-10 16:06:13 -08:00
Megan Rogge 578eb0d4ae
be sure that terminal target gets set for the instance (#138875) 2021-12-10 14:51:45 -08:00
Daniel Imms 69dcf8b0f6 Revert "Re-enable tests now that runCommandWithValue is more reliable"
This reverts commit 4554eac76d.
2021-12-10 13:05:53 -08:00
Daniel Imms 7d35da6eb9 Wait for host on web as well 2021-12-10 11:15:59 -08:00
Daniel Imms da7ab25150 Merge remote-tracking branch 'origin/main' into tyriar/137847 2021-12-10 11:11:49 -08:00
Daniel Imms 4554eac76d Re-enable tests now that runCommandWithValue is more reliable
Part of #138658
2021-12-10 10:59:40 -08:00
Daniel Imms 866b9df390 Wait for quick input to open again after runCommandWithValue
Fixes #138658
2021-12-10 10:59:40 -08:00
Rob Lourens 9893857d4b Handle "clear search results" cancelling running search
Fix #138841
2021-12-10 10:51:26 -08:00
Benjamin Pasero c5ea0fa337
fix some recent test failures 2021-12-10 14:34:14 +01:00
Benjamin Pasero 05de5834b0
smoke - add test to verify an editor can be saved 2021-12-10 13:45:08 +01:00
Benjamin Pasero 4667cac14b
smoke - more obvious tests setup 2021-12-10 13:32:08 +01:00
Rob Lourens e5637ce9af Fix #138748 2021-12-09 10:00:25 -08:00
Johannes Rieken ee3e404b71
disable failing smoketest, https://github.com/microsoft/vscode/issues/138748 2021-12-09 12:53:27 +01:00
Benjamin Pasero c04c29d496
smoke - mention playwright trace website 2021-12-09 12:36:41 +01:00
Benjamin Pasero 9c7387bb5a
smoke - bring back stable smoke tests 2021-12-09 11:05:41 +01:00
Benjamin Pasero 27e38a8e27
smoke - always create screenshot on failure on desktop 2021-12-09 09:51:55 +01:00
Benjamin Pasero ae0ee547ba
smoke - reimplement test handlers 2021-12-09 09:33:51 +01:00
Benjamin Pasero 158bd2507e
smoke - always have a logger 2021-12-09 08:48:42 +01:00
Benjamin Pasero 15415b6de6
smoke - align killing 2021-12-09 08:03:48 +01:00
Benjamin Pasero ec3ba7a320
smoke - rewrite killing 2021-12-09 07:55:25 +01:00
Benjamin Pasero acc0803f38
cleanup automation 2021-12-09 07:03:29 +01:00
Rob Lourens 1341ca3d7f Remove timeouts in search smoketest.
Improvements came from waiting for search to complete before clicking buttons, clearing previous results before starting the search, and adding a retry around the clicking.
2021-12-08 15:34:35 -08:00
Daniel Imms 465402dd4b Merge remote-tracking branch 'origin/main' into tyriar/137847 2021-12-08 06:28:35 -08:00
Benjamin Pasero 18cba4882d
smoke - more logging 2021-12-08 15:23:27 +01:00
Benjamin Pasero 64562e2a0e
smoke - reimplement logging 2021-12-08 14:13:41 +01:00
Benjamin Pasero 2c68d4de7e
smoke - always log with trace 2021-12-08 10:50:27 +01:00
Benjamin Pasero fbad065eea
Playwright: record a trace per failing test, not suite (#138600)
* smoke - record traces per test and not entire suite

* smoke - only persist failing tests

* smoke - cleanup

* smoke - more logging

* smoke - push a test failure to proof the point

* smoke - switch back to chrome for smoke tests

* smoke - warn when exit takes 10s

* Revert "smoke - push a test failure to proof the point"

This reverts commit e572a0c40d.
2021-12-08 10:09:30 +01:00
Matt Bierner c2f4daf348
Update webview bits version for tests 2021-12-07 16:16:54 -08:00
Tyler James Leonhardt 224701f74d
try remove timeouts in smoke tests for quickaccess (#138615) 2021-12-07 12:25:35 -08:00
Benjamin Pasero 13c0ed811e
🆙 playwright & set trace names (#138539) 2021-12-06 19:42:05 +01:00
Daniel Imms 110345a97b Try removing timeout in checkWindowReady
Part of #137847
2021-12-06 06:35:57 -08:00
Benjamin Pasero 2085b19a51
smoke - allow 30s max for deleting test path (#137725) 2021-12-06 14:09:02 +01:00
Benjamin Pasero e8e0fb874d
smoke - disable test that is flaky still 2021-12-06 13:44:01 +01:00
Benjamin Pasero fe227dacb7
lifecycle - introduce and adopt finalVeto for backups (#138055) 2021-12-06 11:16:57 +01:00
Benjamin Pasero e1caacebce
backup - cancel pending operations on shutdown and handle cancellation better (#138055) 2021-12-06 10:55:33 +01:00
Benjamin Pasero ff970e88e5
Revert "smoke - 🆙 retry timeout"
This reverts commit d60b844aa3.
2021-12-02 07:38:04 +01:00
Benjamin Pasero 9d427e4df4
smoke test tweaks
- remove timeout on shutdown
- log exactly where time is spend on shutdown (web)
- stop spawning code when terminated
2021-12-02 07:34:13 +01:00
Megan Rogge 4a21019476
fix #138238, skip failing test 2021-12-01 08:50:26 -08:00
Megan Rogge c0f21fa378
fix #138239, skip test 2021-12-01 08:38:28 -08:00
Benjamin Pasero d60b844aa3
smoke - 🆙 retry timeout 2021-12-01 11:34:25 +01:00
Megan Rogge 7b3078d931
fix #137795 2021-11-30 12:48:58 -08:00
Megan Rogge 7f7fad2f9e
fix #137798 2021-11-30 12:38:52 -08:00
Daniel Imms faf6f0e1fd Setup terminal tests for 3 retries to minimize impact on build
This isn't a great long term solution but it should help reduce failures
for now.

Part of #137725
2021-11-30 12:19:10 -08:00
Megan Rogge 005947e3b8
fix #137722
fix #137808
2021-11-30 11:54:28 -08:00
Megan Rogge e89889fe54
fix #137799 2021-11-30 11:21:01 -08:00