Commit graph

293 commits

Author SHA1 Message Date
Benjamin Pasero b7cdfce7e5
Smoke test runner fails or hangs without error (fix #141148) 2022-01-22 08:43:21 +01:00
Martin Aeschlimann ef4549b141
Merge branch 'main' into aeschli/noremoteindicatorweb 2022-01-21 17:19:52 +01:00
Alex Dima 715dbb0c16
Refuse launching unless a connection token is provided 2022-01-21 13:25:20 +01:00
Martin Aeschlimann f9cac7c28a
adopt --server-data-dir 2022-01-21 12:11:47 +01:00
Martin Aeschlimann 8047f9c877
no remote indicator for code server web 2022-01-21 10:00:20 +01:00
Benjamin Pasero 295984b33b
tests - adopt new URL syntax when web is connected to a remote 2022-01-20 15:55:18 +01:00
Benjamin Pasero 13461f8c61
Web: change yarn web to run with our server instead of playground (#139725)
* web - first cut `yarn web` via our server

* properly pipe output

* web - remove traces of web playground

* web - remember last opened workspace for convinience

* use vscode-test-web for server less, clean up web commands

* fix comment

* fix `yarn web`

* rename to code-server

* open system browser

* code-server script: use minimist

* test resolver: use ./scripts/code-server

* integartion tests: fix code-server command name

Co-authored-by: Martin Aeschlimann <martinae@microsoft.com>
2022-01-20 13:48:08 +01:00
Martin Aeschlimann ad928a4e0f
adopt server script in integration tests 2022-01-19 18:08:04 +01:00
Sandeep Somavarapu 066671a902
Fix #140838 2022-01-18 17:43:20 +01:00
Benjamin Pasero 9be2757019
🆙 playwright@1.18 (#140767)
* tests - use `@playwright/test@1.18.0-rc1`

* fix bad dependency

* clear more weird ts dependencies
2022-01-15 09:35:39 +01:00
João Moreno 8c9870d298
Smoke Test: forcefully kill electron if exit doesn't work (#140570)
* smoke test: forcefully kill electron if exit doesn't work

* use tree-kill

* roll back veto
2022-01-12 17:02:52 +01:00
Matt Bierner 9f867c3ed8
Revert service worker usage of MessageChannel (#140351)
* Revert service worker usage of MessageChannel

Reverts 66b6adf035

While I'm not 100% about this, I think 66b6adf035 causes resourses to occasionally not load. I believe this can happen if the service worker is unitilized while the webview remains active. I can't reproduce this myself so it may be related to memory pressure or resource usage, however relying on the service worker not being reinitilized does seem like a potentially bad idea https://stackoverflow.com/questions/34775105/what-causes-the-global-context-of-a-service-worker-to-be-reset

Will investigate if there's another way to achive this since using MessagePort did clean up the code and slightly improve performance

* Bump webview commit versions
2022-01-10 14:22:49 -08:00
Benjamin Pasero a224ef4fb3
smoke - fix selectTab to select tab? 2022-01-04 09:47:09 +01:00
Benjamin Pasero 18a7108073
smoke - select tab over quick pick (#139944)
* smoke - further try to reduce usages of quick pick

* poke
2021-12-30 22:38:59 +01:00
Benjamin Pasero 99fabbdb27
smoke - more logging 2021-12-29 17:46:52 +01:00
Benjamin Pasero bd43c04262
smoke - log electron stdout into log file 2021-12-29 17:35:54 +01:00
Benjamin Pasero 9c0cce1dae
smoke - take screenshot when waiting long for window ready 2021-12-28 15:32:35 +01:00
Benjamin Pasero b2e79a699f
smoke - improve app lifecycle (#139693)
* smoke - make sure app is closed when `app.start` fails

* smoke - `capturePage` to fallback to active window

* smoke - more logging around remote resolving
2021-12-23 16:38:54 +01:00
Daniel Imms daf4ba622f
Merge pull request #138516 from microsoft/tyriar/137847
Try removing timeout in checkWindowReady
2021-12-22 10:38:41 -08:00
Benjamin Pasero bb39ae82cf
smoke - improve runCommand and disable more tests 2021-12-20 08:25:08 +01:00
Benjamin Pasero 2d77c9e23a
smoke - more robust usage of quick input / access (#139367)
* smoke - more robust `openQuickAccessAndWait`

* tweaks

* 💄

* trigger

* fix fluke

* poke

* fix terminal weirdness

* poke

* address feedback
2021-12-17 18:49:17 +01:00
Benjamin Pasero ae8aeb653d
smoke - use absolute paths for opening files (#139322) 2021-12-16 20:14:12 +01:00
Benjamin Pasero fa739870a2
smoke - remove doubleClick usage 2021-12-16 14:27:21 +01:00
Benjamin Pasero fdc4b7101d
smoke - actually retry quick access usage 2021-12-16 14:25:12 +01:00
Daniel Imms da308a9a7f Format 2021-12-15 19:35:16 +00:00
Daniel Imms fa61def811
Merge branch 'main' into tyriar/137847 2021-12-15 09:29:49 -08:00
Rob Lourens b823936907 Wait for window to connect before starting browser smoketests
Fix #138045
2021-12-14 20:23:44 -08:00
Matt Bierner 8cf4dcdade
Re-enable notebook smoke tests (#139127)
Fixes #126371
2021-12-14 16:50:22 -08:00
Daniel Imms 2eb4cec44b Wait for terminal to be ready before changing color
Fixes #139081
2021-12-14 10:14:06 -08:00
Daniel Imms 9d1a7abe35 Some jsdoc 2021-12-13 16:57:14 -08:00
Daniel Imms 50f96a3c6b Clean up terminal creation in smoke tests 2021-12-13 16:54:42 -08:00
Daniel Imms 73a19337ac Merge remote-tracking branch 'origin/main' into tyriar/133524 2021-12-13 16:28:23 -08:00
Daniel Imms 9566e299dd Add auto replies test 2021-12-13 16:20:34 -08:00
Megan Rogge a2f5ec29b2
Revert "Revert "be sure that terminal target gets set for the instance (#138875)""
This reverts commit 8605d7aeff.
2021-12-13 17:08:29 -06:00
Daniel Imms 8605d7aeff Revert "be sure that terminal target gets set for the instance (#138875)"
This reverts commit 578eb0d4ae.
2021-12-13 08:15:32 -08:00
Megan Rogge 578eb0d4ae
be sure that terminal target gets set for the instance (#138875) 2021-12-10 14:51:45 -08:00
Daniel Imms 7d35da6eb9 Wait for host on web as well 2021-12-10 11:15:59 -08:00
Daniel Imms da7ab25150 Merge remote-tracking branch 'origin/main' into tyriar/137847 2021-12-10 11:11:49 -08:00
Daniel Imms 866b9df390 Wait for quick input to open again after runCommandWithValue
Fixes #138658
2021-12-10 10:59:40 -08:00
Rob Lourens 9893857d4b Handle "clear search results" cancelling running search
Fix #138841
2021-12-10 10:51:26 -08:00
Rob Lourens e5637ce9af Fix #138748 2021-12-09 10:00:25 -08:00
Benjamin Pasero 27e38a8e27
smoke - always create screenshot on failure on desktop 2021-12-09 09:51:55 +01:00
Benjamin Pasero 158bd2507e
smoke - always have a logger 2021-12-09 08:48:42 +01:00
Benjamin Pasero 15415b6de6
smoke - align killing 2021-12-09 08:03:48 +01:00
Benjamin Pasero ec3ba7a320
smoke - rewrite killing 2021-12-09 07:55:25 +01:00
Benjamin Pasero acc0803f38
cleanup automation 2021-12-09 07:03:29 +01:00
Rob Lourens 1341ca3d7f Remove timeouts in search smoketest.
Improvements came from waiting for search to complete before clicking buttons, clearing previous results before starting the search, and adding a retry around the clicking.
2021-12-08 15:34:35 -08:00
Daniel Imms 465402dd4b Merge remote-tracking branch 'origin/main' into tyriar/137847 2021-12-08 06:28:35 -08:00
Benjamin Pasero 18cba4882d
smoke - more logging 2021-12-08 15:23:27 +01:00
Benjamin Pasero 64562e2a0e
smoke - reimplement logging 2021-12-08 14:13:41 +01:00
Benjamin Pasero fbad065eea
Playwright: record a trace per failing test, not suite (#138600)
* smoke - record traces per test and not entire suite

* smoke - only persist failing tests

* smoke - cleanup

* smoke - more logging

* smoke - push a test failure to proof the point

* smoke - switch back to chrome for smoke tests

* smoke - warn when exit takes 10s

* Revert "smoke - push a test failure to proof the point"

This reverts commit e572a0c40d.
2021-12-08 10:09:30 +01:00
Tyler James Leonhardt 224701f74d
try remove timeouts in smoke tests for quickaccess (#138615) 2021-12-07 12:25:35 -08:00
Benjamin Pasero 13c0ed811e
🆙 playwright & set trace names (#138539) 2021-12-06 19:42:05 +01:00
Daniel Imms 110345a97b Try removing timeout in checkWindowReady
Part of #137847
2021-12-06 06:35:57 -08:00
Benjamin Pasero ff970e88e5
Revert "smoke - 🆙 retry timeout"
This reverts commit d60b844aa3.
2021-12-02 07:38:04 +01:00
Benjamin Pasero 9d427e4df4
smoke test tweaks
- remove timeout on shutdown
- log exactly where time is spend on shutdown (web)
- stop spawning code when terminated
2021-12-02 07:34:13 +01:00
Benjamin Pasero d60b844aa3
smoke - 🆙 retry timeout 2021-12-01 11:34:25 +01:00
Megan Rogge 7b3078d931
fix #137795 2021-11-30 12:48:58 -08:00
Megan Rogge 005947e3b8
fix #137722
fix #137808
2021-11-30 11:54:28 -08:00
Megan Rogge e89889fe54
fix #137799 2021-11-30 11:21:01 -08:00
Megan Rogge 18725fc147
set default profile to pwsh for only windows for #137225 2021-11-30 10:57:59 -08:00
Megan Rogge 4be98a3bce
set default to pwsh instead of bash for #137725 2021-11-30 10:21:15 -08:00
Daniel Imms d5bfd6d31c Remove unneeded logs in test 2021-11-29 11:48:14 -08:00
Daniel Imms e208c8604e Remove timeout from automation terminal util
I don't believe this was ever required, awaiting writeInTerminal should make sure the
data event is triggered in xterm, the following enter should just trigger another data
event. This doesn't guarantee it was accepted by the process yet, just that xterm.js
emitted the event.

Part of #137847
2021-11-29 11:46:57 -08:00
Benjamin Pasero 1af6cb0edc
smoke test tweaks 2021-11-29 17:53:11 +01:00
Benjamin Pasero 18c6692316
tests - print main process pid when killing fails 2021-11-29 13:48:27 +01:00
MalikIdreesHasanKhan 595d893fdc
Fixed typos. (#135729) 2021-11-29 11:32:14 +01:00
Benjamin Pasero e71b28a444
Smoke test lifecycle changes (#137969)
* rewrite teardown

* 💄

* avoid spam errors

* 💄

* split spawning browser from electron

* await copyExtension

* cleanup

* refactor methods

* cleanup

* cleanup

* cleanup

* 🆙 deps
2021-11-28 10:30:52 +01:00
Benjamin Pasero df4b5d6d04
smoke - strengthen shutdown path 2021-11-26 18:08:27 +01:00
Benjamin Pasero a11814c57a
smoke - switch away from potentially flaky editors.selectTab method 2021-11-26 09:22:19 +01:00
Benjamin Pasero 6f2239307b
Smoke test tweaks (#137809)
* smoke - move data migration tests into one and align

* fix app starting

* `createWorkspaceFile` is not async

* 💄

* support screenshot on failure even for stable app

* smoke - try to remove timeout (#137847)

* improve exit call
2021-11-25 14:37:22 +01:00
Rob Lourens 2932ec095d Add retry to quickaccess tests
Fix #137733
2021-11-23 15:35:48 -08:00
Daniel Imms ef440e563d Merge remote-tracking branch 'origin/main' into tyriar/persist_tests 2021-11-23 12:47:36 -08:00
Benjamin Pasero 954ce2c3f4
proper await exit 2021-11-23 10:31:46 +01:00
Megan Rogge 4e303fcc52
fix #137590 2021-11-21 10:04:21 -08:00
Daniel Imms 5dec5256d7 Add some detach/attach tests 2021-11-19 10:33:43 -08:00
Megan Rogge f66bc86da8
add terminal editor and tab smoke tests (#137393)
* Add a bunch of tests

* tweak tests

* add profile tests, now breaking (no profiles showing up)

* uncomment tests

* update expected

* fix test

* merge

* all passing

* bunch of improvements

* improve readability

* all should be passing and cleaned up

* all passing

* fix #137243 and clean-up

* remove .only

* fix #137247

* get rid of enum

* see if this fixes timing issue

* use keybinding for show terminal instead

* remove delay, implement better accept function

* get rid of unneeded index

* try something else

* wait for empty editor to be active

* remove .only

* decrease redundancy in names

* re-add xterm selector

* try something else

* undo change that broke things worse

* add terminal-tabs test

* all tabs tests passing locally

* kill all

* await before running each

* fix tests

* remove .only

* try waiting for focused xterm

* remove .only

* clean up

* 🧹

* fix profiles tests by using this.app.props

* revert to use default shell

* re-add conditional

* Get tests passing on WebKit

On WebKit the smoke tests were failing because the quick pick was
triggering a focus event on the terminal after the blur event had
already happened, this caused the view service to think the terminal
was still focused when it wasn't. The fix was to reset the context key
also in IViewsService.closeView.

* Safari -> WebKit

* Prefer arrow functions

* remove beforeEach timeout

* add timeouts to contributed profile tests

* add terminal editor tests, fix #137377

* add a bunch of tests

* refactor

* get rid of profile specific command

* more polish

* refactor getTabs -> assertTerminalGroups

* fix failing test

* more polish

* remove .only

* add assertSingleTab

* remove unused import

* fix error

* fix more failures

* more changes

* fix almost all except for plus button

* fix failing plus button test

* all passing

* modify error that gets thrown

* modify error message again

* remove unused wildcard/ ANY_NAME, fix icon code

* fix icon code

* large refactor, improvements

* finish polish

* 1 failing

* get test to pass

Co-authored-by: Daniel Imms <2193314+Tyriar@users.noreply.github.com>
2021-11-18 15:11:40 -08:00
Megan Rogge 239a94cee8
remove debug log 2021-11-17 14:01:31 -08:00
Megan Rogge 00fa317c43
Add terminal profile smoke tests (#137079) 2021-11-17 14:00:01 -08:00
Benjamin Pasero 6f28593438
smoke - retry quick open search (fix #136499) 2021-11-15 07:52:12 +01:00
Daniel Imms c65c8738f9 Copy and use IWindowDriver in PlaywrightDriver 2021-11-12 06:11:18 -08:00
Daniel Imms 634dae736e Merge remote-tracking branch 'origin/main' into tyriar/smoke_test_improvements 2021-11-12 05:45:20 -08:00
Daniel Imms 5bfaf29f2b Fix layering problem 2021-11-10 08:38:02 -08:00
Daniel Imms 6a6a1b65df Use handles in playwright driver 2021-11-10 08:03:22 -08:00
Benjamin Pasero bef4dba21f
Parcel update and CI improvements (#136830)
* 🆙 product

* 🆙 parcel/watcher

* 🆙 parcel/watcher

* tests - separate remote logs from others

* store server logs too

* more tweaks

* fix name

* error when 10s passed
2021-11-10 08:13:56 +01:00
Daniel Imms bd98431baf Write basic terminal default profile test
Co-authored-by: Megan Rogge <merogge@microsoft.com>
2021-11-09 10:10:54 -08:00
Daniel Imms 628f486b1d Move playwright driver impl to class
Part of #136064
2021-11-09 08:45:08 -08:00
Benjamin Pasero 7ca8b0f613
Statusbar - checks if 'Go to Line' works if called from the status bar: (fix #135962) 2021-10-28 07:22:45 +02:00
Benjamin Pasero be2f951fde debt - introduce Promises.withAsyncBody and adopt in a few places 2021-10-11 13:27:43 +02:00
Sandeep Somavarapu 77b1fb79f5
Fix #132233 2021-09-06 14:32:19 +02:00
Logan Ramos 8bf9cfa21b
Fix spacing 2021-08-11 15:26:20 -04:00
Logan Ramos 2ee9010437
Disable telemetry in tests (#130591)
* Disable telemetry in tests

* Remove the console log check
2021-08-11 11:49:23 -04:00
Benjamin Pasero ec9a8fe5b7
web smoke - guard against failing tracing 2021-08-07 15:10:50 +02:00
Benjamin Pasero 5e99049e66
smoke - fix web smoke tests 2021-08-05 11:31:04 +02:00
Benjamin Pasero b96644b15f
smoke tests - drop 'tmp' and try to deal with tmp path ourself (#130102) 2021-08-05 10:04:00 +02:00
Benjamin Pasero 6c3bb99c63
smoke - better exit application handling 2021-08-05 09:05:17 +02:00
Benjamin Pasero c28ea165cd
smoke tests - try other ways of ensuring the editor is open for data migration (#129279) 2021-08-04 15:32:21 +02:00
Benjamin Pasero db4cda10a8
web - guard against error from stopping tracing on tear down of tests 2021-08-03 14:15:55 +02:00
Benjamin Pasero 0390d98a4e
web tests - print page errors to console
//cc @lramos15 this would have detected the missing module
2021-08-03 09:10:47 +02:00
João Moreno a28d028d2d
Enable automated smoketests on Windows (#128224) 2021-07-08 19:10:25 +02:00
Benjamin Pasero fa032d752d
Update playwright to 1.12.x and enable tracing for smoke tests (#128061)
* 🆙 playwright

* create trace
2021-07-07 13:23:11 +02:00
Dirk Baeumer ae52915f0c
Improve localization tests 2021-07-06 13:15:43 +02:00
João Moreno ca61a42383
web smoke test: bump up port number 2021-06-30 12:01:31 +02:00
João Moreno a44611d4c6
tests: hide output 2021-06-30 11:07:35 +02:00
Dirk Baeumer 1f2a6cc331
Improve localization smoke test 2021-06-21 13:54:16 +02:00
Benjamin Pasero 81317867a3
web - disable welcome in smoke tests (fix #126221)
//cc @JacksonKearl
2021-06-15 10:57:29 +02:00
Benjamin Pasero 2ddc1e4fc5
Produce logs during test runs (integration, smoke) and publish as artefacts (#126065)
* first cut upload log files from integration test runs

* always publish log files

* move logs into integration-tests folder

* also produce logs for smoke tests

* store remote log files

* wire in logspath arg properly

* always publish logs

* enable logs for browser based smoke tests

* log only in verbose
2021-06-15 08:33:26 +02:00
João Moreno 85a36ae074 enable smoketest on linux 2021-06-11 12:19:17 +02:00
Ladislau Szomoru 291ec03bc3
Smoke tests - run each suite in its own user-data-dir (#125670)
* Initial commit for isolated smoke tests

* Do not wait for the explorer view when restarting

* Debug failed localization test

* User data path suffix length

* Kill server when application exits

* Do not run yarn install

* Refactor application start/stop

* Reverted some changes

* Missed one

* One more change to revert

* Restored one more file

* Pull request feedback
2021-06-10 06:40:57 -07:00
Benjamin Pasero 33f84b36fd
tests - disable getting started in integration tests and smoke tests (#125808)
* tests - disable getting started in integration tests and smoke tests

* --skip-getting-started => --skip-welcome

Co-authored-by: Jackson Kearl <jakearl@microsoft.com>
2021-06-09 18:36:42 +00:00
Ladislau Szomoru 4e0cf885d4
Don't wait for the explorer when restarting 2021-06-07 15:34:37 +02:00
Ladislau Szomoru b0509196f6
Disable workspace trust during tests 2021-06-03 10:21:19 +02:00
João Moreno 541a5d4fa7
smoke: trust dialog should appear in 5 seconds, and we can ignore if it doesn't 2021-05-28 11:38:10 +02:00
SteVen Batten e5f70e8e2a
fix smoke tests (#124814) 2021-05-28 09:30:47 +02:00
Jackson Kearl 852b70eafe
Try fix (#124384) 2021-05-21 17:36:20 +00:00
Jackson Kearl 89c8f91988
Add smoke test for #124146 2021-05-20 22:20:48 -07:00
Jackson Kearl ac0008a25b
Remove waiting for welcome page while we are experimenting with what the welcome page will be
Fixes #118748
2021-03-11 09:23:38 -08:00
Jackson Kearl 556e6d8310
Fixup search smoke tests 2021-03-09 12:50:36 -08:00
Sandeep Somavarapu 9d3a1d9f59 Fix #118443 2021-03-09 09:30:56 +01:00
Sandeep Somavarapu 1a9df6127c fix smoke tests 2021-03-05 15:27:03 +01:00
Sandeep Somavarapu c6202bb3cc Adopt to use new table widget 2021-03-04 19:51:03 +01:00
Benjamin Pasero 5fabc0d9dc smoke test - remove code only needed for stable january release 2021-02-25 09:31:55 +01:00
Benjamin Pasero 041e4d9b1f fix smoke tests with stable build (fix #117420) 2021-02-23 19:04:51 +01:00
Jackson Kearl 096e5555b6 Allow welcome page to not be active on account of focus potentially going to a getting started item. 2021-02-18 18:57:24 -08:00
Benjamin Pasero 9de38fc9a8
Disable keytar in our integration tests (#116852)
* tests - extract and use ALL_PLATFORMS_API_TESTS_EXTRA_ARGS

* tests - throw when accessing keytar from API tests
2021-02-17 15:09:40 +01:00
Jackson Kearl d10f2e6032 Fix #116412: Use getting started page as default startupEditor. 2021-02-11 12:34:14 -08:00
Jackson Kearl 4623a50995 Fix #116154 2021-02-08 17:46:12 -08:00
Sandeep Somavarapu 63310d91d3 remove install extension 2021-02-04 20:48:25 +01:00
Sandeep Somavarapu 4e6d9ce2b7 add server cli test 2021-02-03 21:42:12 +01:00
Sandeep Somavarapu 066c584405 enable remote smoke tests 2021-02-03 19:20:44 +01:00
Martin Aeschlimann 4bd2d367e7 Remote smoketest timeout (fixes #115159) 2021-01-27 15:52:25 +01:00
Sandeep Somavarapu 0ac5e95251 Fix #109644 2020-12-02 17:50:49 +01:00
Sandeep Somavarapu 47ce2a8d6d #111845 fix localization tests 2020-11-30 20:26:00 +01:00
Sandeep Somavarapu 2890fd321f Fix #111485 2020-11-30 11:58:43 +01:00
Martin Aeschlimann 8012c255e5 fix keybinding smoke test after icon name changes 2020-11-27 12:19:03 +01:00
Sandeep Somavarapu 6a2535e0ab Fix #107965 2020-10-28 19:49:42 +01:00
Sandeep Somavarapu 758240249a Fix #109210 2020-10-23 09:20:36 +02:00
Benjamin Pasero 781e0e3192 windows - remove not needed and confusing disable-restore-windows CLI arg 2020-10-01 11:36:45 +02:00
Sandeep Somavarapu 2277c8e2a3 Fix #104230 2020-08-07 12:31:03 +02:00
isidor 55e09d6bf8 update extensions aria label to contain version and publisher display name
fixes #102224
2020-07-28 19:16:02 +02:00
rebornix 5e96337121 re #93070. test for command arguments. 2020-07-13 10:52:57 -07:00
Rob Lourens 8b5e6ff8c5 Fix notebook smoketest
Esc doesn't do the right thing because a notification showed up
Fix #99941
2020-06-12 07:56:18 -05:00
Sandeep Somavarapu 6876a0de40 Fix #99840 2020-06-11 10:13:21 +02:00
Benjamin Pasero d41ca1ce5f tests - also use --no-cached-data for normal integration tests 2020-06-09 10:29:01 +02:00
Rob Lourens 4d6da3154c Get notebook smoketest working in web
Fix #98364
2020-06-07 17:27:47 -05:00
Rob Lourens b877c4d411 Fix notebooks and notebook smoketest in web 2020-05-21 08:56:22 -05:00
Rob Lourens 9bc11824c3 Add notebook smoketests 2020-05-20 12:01:52 -05:00
Benjamin Pasero 869929e8c6 tests - log the actual build used for tests 2020-05-14 17:00:51 +02:00
Rob Lourens 7a5baba094 Fix replace smoketest
Fix #97120
2020-05-07 09:54:43 -05:00
Benjamin Pasero 6a8b201c8a smoke - restore skip-release-notes CLI arg (#96459) 2020-04-29 12:40:04 +02:00