Benjamin Pasero
96206bb5a8
smoke - 🆙 timeouts
2022-01-19 19:45:03 +01:00
Connor Peet
bdc81fc0a9
tests: log progress as stable code downloads
2022-01-19 09:16:39 -08:00
Benjamin Pasero
158cd6da42
smoke - log code download ( #140769 )
...
* smoke - log progress/errors of vscode download
//cc @connor4312
* reduce spam
2022-01-15 11:31:13 +01:00
Daniel Imms
f2c755a878
Skip smoke test
...
Part of #139076
2022-01-14 09:47:56 -08:00
Daniel Imms
0b7bc6d73c
Re-enable auto replies tests with retries on Windows only
...
Fixes #139076
2022-01-14 08:56:11 -08:00
Daniel Imms
860e935d1a
Skip notebooks smoke test suite
...
Part of #140575
2022-01-12 07:35:56 -08:00
Benjamin Pasero
18a7108073
smoke - select tab over quick pick ( #139944 )
...
* smoke - further try to reduce usages of quick pick
* poke
2021-12-30 22:38:59 +01:00
Benjamin Pasero
b2e9e87c2e
smoke - skip flaky notebook test ( #139672 )
2021-12-23 17:12:29 +01:00
Benjamin Pasero
b2e79a699f
smoke - improve app lifecycle ( #139693 )
...
* smoke - make sure app is closed when `app.start` fails
* smoke - `capturePage` to fallback to active window
* smoke - more logging around remote resolving
2021-12-23 16:38:54 +01:00
Benjamin Pasero
bb39ae82cf
smoke - improve runCommand
and disable more tests
2021-12-20 08:25:08 +01:00
Benjamin Pasero
2d77c9e23a
smoke - more robust usage of quick input / access ( #139367 )
...
* smoke - more robust `openQuickAccessAndWait`
* tweaks
* 💄
* trigger
* fix fluke
* poke
* fix terminal weirdness
* poke
* address feedback
2021-12-17 18:49:17 +01:00
Daniel Imms
88fcbd84e0
Fix auto replies smoke tests
...
This was flaky because the shell's wrapping character may have come after the
input (eg. 'foo> bar'), the fix is to allow characters between the matching
string and reply via regex.
Fixes #139083
2021-12-17 09:42:50 -08:00
Benjamin Pasero
7141436baa
smoke - skip failing terminal test ( #139353 )
2021-12-17 08:21:58 +01:00
Benjamin Pasero
1a4a2714bf
smoke - skip failing terminal test ( #139349 )
2021-12-17 07:34:20 +01:00
Daniel Imms
d33e3d2fbc
Re-enable flaky auto replies tests
...
Need a new playwright trace from a failure to be able to diagnose
Part of #139083
2021-12-16 11:36:53 -08:00
Benjamin Pasero
ae8aeb653d
smoke - use absolute paths for opening files ( #139322 )
2021-12-16 20:14:12 +01:00
Benjamin Pasero
9e22f1ccac
smoke - disable flaky notebook tests ( #139270 )
2021-12-16 15:55:15 +01:00
Daniel Imms
dfc4077fb0
Use dom renderer for terminal smoke tests
...
Part of #139083
2021-12-16 06:38:11 -08:00
Sandeep Somavarapu
995d40b12d
#139183 enable the test and add a log statement
2021-12-16 14:34:47 +01:00
Benjamin Pasero
51eda13af4
smoke - move editor test into language test
2021-12-16 10:29:46 +01:00
Rob Lourens
86bdc567e9
Remove obsolete comment
2021-12-15 16:41:20 -08:00
Johannes Rieken
1b2c610ffe
skip bad test, https://github.com/microsoft/vscode/issues/139183
2021-12-15 10:51:01 +01:00
Matt Bierner
8cf4dcdade
Re-enable notebook smoke tests ( #139127 )
...
Fixes #126371
2021-12-14 16:50:22 -08:00
Daniel Imms
2eb4cec44b
Wait for terminal to be ready before changing color
...
Fixes #139081
2021-12-14 10:14:06 -08:00
Daniel Imms
9c99eaa0b2
Also log exception in itSkipOnFail
2021-12-14 08:15:54 -08:00
Daniel Imms
c29df4c2d5
Add itSkipOnFail util
...
Part of #139076
2021-12-14 08:05:43 -08:00
Benjamin Pasero
5526cd558f
smoke - remove only
2021-12-14 07:38:08 +01:00
Daniel Imms
50f96a3c6b
Clean up terminal creation in smoke tests
2021-12-13 16:54:42 -08:00
Daniel Imms
73a19337ac
Merge remote-tracking branch 'origin/main' into tyriar/133524
2021-12-13 16:28:23 -08:00
Daniel Imms
9566e299dd
Add auto replies test
2021-12-13 16:20:34 -08:00
Megan Rogge
a2f5ec29b2
Revert "Revert "be sure that terminal target gets set for the instance ( #138875 )""
...
This reverts commit 8605d7aeff
.
2021-12-13 17:08:29 -06:00
Megan Rogge
0602587ff2
re-enable smoke tests, keep editor fix ( #139012 )
2021-12-13 12:50:58 -06:00
Rob Lourens
7e4942dcb7
Revert "Re-enable additional notebook smoke tests"
...
This reverts commit b3b9d4e2e7
.
2021-12-13 09:18:17 -08:00
Daniel Imms
8605d7aeff
Revert "be sure that terminal target gets set for the instance ( #138875 )"
...
This reverts commit 578eb0d4ae
.
2021-12-13 08:15:32 -08:00
Matt Bierner
b3b9d4e2e7
Re-enable additional notebook smoke tests
...
For #126371
These pass for me locally
2021-12-10 17:08:02 -08:00
Matt Bierner
02c3551a59
Re-enable notebook smoke suite
...
For #126371
This single test passes for me. Will look at re-enabling individual tests next
2021-12-10 17:01:02 -08:00
Megan Rogge
86ee106054
skip failing tests
2021-12-10 16:06:13 -08:00
Megan Rogge
578eb0d4ae
be sure that terminal target gets set for the instance ( #138875 )
2021-12-10 14:51:45 -08:00
Daniel Imms
69dcf8b0f6
Revert "Re-enable tests now that runCommandWithValue is more reliable"
...
This reverts commit 4554eac76d
.
2021-12-10 13:05:53 -08:00
Daniel Imms
4554eac76d
Re-enable tests now that runCommandWithValue is more reliable
...
Part of #138658
2021-12-10 10:59:40 -08:00
Benjamin Pasero
c5ea0fa337
fix some recent test failures
2021-12-10 14:34:14 +01:00
Benjamin Pasero
05de5834b0
smoke - add test to verify an editor can be saved
2021-12-10 13:45:08 +01:00
Benjamin Pasero
4667cac14b
smoke - more obvious tests setup
2021-12-10 13:32:08 +01:00
Rob Lourens
e5637ce9af
Fix #138748
2021-12-09 10:00:25 -08:00
Johannes Rieken
ee3e404b71
disable failing smoketest, https://github.com/microsoft/vscode/issues/138748
2021-12-09 12:53:27 +01:00
Benjamin Pasero
9c7387bb5a
smoke - bring back stable smoke tests
2021-12-09 11:05:41 +01:00
Benjamin Pasero
27e38a8e27
smoke - always create screenshot on failure on desktop
2021-12-09 09:51:55 +01:00
Benjamin Pasero
ae0ee547ba
smoke - reimplement test handlers
2021-12-09 09:33:51 +01:00
Benjamin Pasero
158bd2507e
smoke - always have a logger
2021-12-09 08:48:42 +01:00
Benjamin Pasero
acc0803f38
cleanup automation
2021-12-09 07:03:29 +01:00
Rob Lourens
1341ca3d7f
Remove timeouts in search smoketest.
...
Improvements came from waiting for search to complete before clicking buttons, clearing previous results before starting the search, and adding a retry around the clicking.
2021-12-08 15:34:35 -08:00
Benjamin Pasero
18cba4882d
smoke - more logging
2021-12-08 15:23:27 +01:00
Benjamin Pasero
64562e2a0e
smoke - reimplement logging
2021-12-08 14:13:41 +01:00
Benjamin Pasero
2c68d4de7e
smoke - always log with trace
2021-12-08 10:50:27 +01:00
Benjamin Pasero
fbad065eea
Playwright: record a trace per failing test, not suite ( #138600 )
...
* smoke - record traces per test and not entire suite
* smoke - only persist failing tests
* smoke - cleanup
* smoke - more logging
* smoke - push a test failure to proof the point
* smoke - switch back to chrome for smoke tests
* smoke - warn when exit takes 10s
* Revert "smoke - push a test failure to proof the point"
This reverts commit e572a0c40d
.
2021-12-08 10:09:30 +01:00
Benjamin Pasero
13c0ed811e
🆙 playwright & set trace names ( #138539 )
2021-12-06 19:42:05 +01:00
Benjamin Pasero
2085b19a51
smoke - allow 30s max for deleting test path ( #137725 )
2021-12-06 14:09:02 +01:00
Benjamin Pasero
e8e0fb874d
smoke - disable test that is flaky still
2021-12-06 13:44:01 +01:00
Benjamin Pasero
fe227dacb7
lifecycle - introduce and adopt finalVeto
for backups ( #138055 )
2021-12-06 11:16:57 +01:00
Benjamin Pasero
e1caacebce
backup - cancel pending operations on shutdown and handle cancellation better ( #138055 )
2021-12-06 10:55:33 +01:00
Benjamin Pasero
9d427e4df4
smoke test tweaks
...
- remove timeout on shutdown
- log exactly where time is spend on shutdown (web)
- stop spawning code when terminated
2021-12-02 07:34:13 +01:00
Megan Rogge
4a21019476
fix #138238 , skip failing test
2021-12-01 08:50:26 -08:00
Megan Rogge
c0f21fa378
fix #138239 , skip test
2021-12-01 08:38:28 -08:00
Megan Rogge
7b3078d931
fix #137795
2021-11-30 12:48:58 -08:00
Megan Rogge
7f7fad2f9e
fix #137798
2021-11-30 12:38:52 -08:00
Daniel Imms
faf6f0e1fd
Setup terminal tests for 3 retries to minimize impact on build
...
This isn't a great long term solution but it should help reduce failures
for now.
Part of #137725
2021-11-30 12:19:10 -08:00
Megan Rogge
005947e3b8
fix #137722
...
fix #137808
2021-11-30 11:54:28 -08:00
Megan Rogge
e89889fe54
fix #137799
2021-11-30 11:21:01 -08:00
Megan Rogge
18725fc147
set default profile to pwsh for only windows for #137225
2021-11-30 10:57:59 -08:00
Megan Rogge
4be98a3bce
set default to pwsh instead of bash for #137725
2021-11-30 10:21:15 -08:00
Benjamin Pasero
7b6c1065c7
smoke - temporary bring back timeout
2021-11-30 11:04:39 +01:00
Benjamin Pasero
1af6cb0edc
smoke test tweaks
2021-11-29 17:53:11 +01:00
Benjamin Pasero
e241749320
smoke - print more backup info
2021-11-29 14:05:40 +01:00
Benjamin Pasero
5bb653374f
smoke - diagnose failing backup test
2021-11-29 09:56:19 +01:00
Benjamin Pasero
cbbba8e69a
smoke - unique user data dir for stable tests too
2021-11-29 09:11:23 +01:00
Benjamin Pasero
c42793d035
smoke - do not fail build when unable to delete workspace folder (workaround #137725 )
2021-11-29 08:49:37 +01:00
Benjamin Pasero
a5efd80ee0
Revert "smoke - print task list when deleting workspace path fails"
...
This reverts commit 368154bd0b
.
2021-11-29 08:46:35 +01:00
Benjamin Pasero
368154bd0b
smoke - print task list when deleting workspace path fails
2021-11-29 08:04:30 +01:00
Benjamin Pasero
e71b28a444
Smoke test lifecycle changes ( #137969 )
...
* rewrite teardown
* 💄
* avoid spam errors
* 💄
* split spawning browser from electron
* await copyExtension
* cleanup
* refactor methods
* cleanup
* cleanup
* cleanup
* 🆙 deps
2021-11-28 10:30:52 +01:00
Benjamin Pasero
13ff6baa3f
fix smoke tests
2021-11-26 14:45:35 +01:00
Benjamin Pasero
8d250e99e7
smoke - wait for tab becoming dirty before exit
2021-11-26 11:00:15 +01:00
Benjamin Pasero
a11814c57a
smoke - switch away from potentially flaky editors.selectTab
method
2021-11-26 09:22:19 +01:00
Benjamin Pasero
6f2239307b
Smoke test tweaks ( #137809 )
...
* smoke - move data migration tests into one and align
* fix app starting
* `createWorkspaceFile` is not async
* 💄
* support screenshot on failure even for stable app
* smoke - try to remove timeout (#137847 )
* improve exit call
2021-11-25 14:37:22 +01:00
Daniel Imms
eeae10e56c
Skip flaky test
...
Part of #137795
2021-11-24 10:04:38 -08:00
Daniel Imms
653e5cdaee
Skip flaky test
...
Part of #137808
2021-11-24 09:59:22 -08:00
Daniel Imms
f880669743
Merge pull request #137763 from microsoft/tyriar/cleanup_smoke
...
Move terminal smoke tests into a single suite and reduce boilterplate
2021-11-24 09:54:50 -08:00
Daniel Imms
07d986443d
Skipping flaky test
...
Part of #137799
2021-11-24 09:50:26 -08:00
Daniel Imms
42bd3e884b
More clean up of terminal smoke test init
2021-11-23 15:46:51 -08:00
Rob Lourens
2932ec095d
Add retry to quickaccess tests
...
Fix #137733
2021-11-23 15:35:48 -08:00
Daniel Imms
d7f10626b8
Clean up terminal smoke test setup
...
Part of #136064
2021-11-23 15:18:38 -08:00
Daniel Imms
68e491f99f
Re-enable other smoke tests
2021-11-23 14:50:52 -08:00
Daniel Imms
78d7ece7aa
Add basic detach/attach terminal tests
...
Part of #136064
2021-11-23 13:23:54 -08:00
Daniel Imms
ef440e563d
Merge remote-tracking branch 'origin/main' into tyriar/persist_tests
2021-11-23 12:47:36 -08:00
Benjamin Pasero
1b40be3377
smoke - enable graceful-fs ( #137734 )
2021-11-23 17:29:08 +01:00
Rob Lourens
3bac53642b
Disable flaky search smoke test #137195
2021-11-22 14:25:23 -08:00
Megan Rogge
4e303fcc52
fix #137590
2021-11-21 10:04:21 -08:00
Daniel Imms
2f7ab5c0e0
Disable should update color of the tab
...
Flake in CI
2021-11-19 13:58:01 -08:00
Daniel Imms
5dec5256d7
Add some detach/attach tests
2021-11-19 10:33:43 -08:00
Megan Rogge
f66bc86da8
add terminal editor and tab smoke tests ( #137393 )
...
* Add a bunch of tests
* tweak tests
* add profile tests, now breaking (no profiles showing up)
* uncomment tests
* update expected
* fix test
* merge
* all passing
* bunch of improvements
* improve readability
* all should be passing and cleaned up
* all passing
* fix #137243 and clean-up
* remove .only
* fix #137247
* get rid of enum
* see if this fixes timing issue
* use keybinding for show terminal instead
* remove delay, implement better accept function
* get rid of unneeded index
* try something else
* wait for empty editor to be active
* remove .only
* decrease redundancy in names
* re-add xterm selector
* try something else
* undo change that broke things worse
* add terminal-tabs test
* all tabs tests passing locally
* kill all
* await before running each
* fix tests
* remove .only
* try waiting for focused xterm
* remove .only
* clean up
* 🧹
* fix profiles tests by using this.app.props
* revert to use default shell
* re-add conditional
* Get tests passing on WebKit
On WebKit the smoke tests were failing because the quick pick was
triggering a focus event on the terminal after the blur event had
already happened, this caused the view service to think the terminal
was still focused when it wasn't. The fix was to reset the context key
also in IViewsService.closeView.
* Safari -> WebKit
* Prefer arrow functions
* remove beforeEach timeout
* add timeouts to contributed profile tests
* add terminal editor tests, fix #137377
* add a bunch of tests
* refactor
* get rid of profile specific command
* more polish
* refactor getTabs -> assertTerminalGroups
* fix failing test
* more polish
* remove .only
* add assertSingleTab
* remove unused import
* fix error
* fix more failures
* more changes
* fix almost all except for plus button
* fix failing plus button test
* all passing
* modify error that gets thrown
* modify error message again
* remove unused wildcard/ ANY_NAME, fix icon code
* fix icon code
* large refactor, improvements
* finish polish
* 1 failing
* get test to pass
Co-authored-by: Daniel Imms <2193314+Tyriar@users.noreply.github.com>
2021-11-18 15:11:40 -08:00
Megan Rogge
00fa317c43
Add terminal profile smoke tests ( #137079 )
2021-11-17 14:00:01 -08:00
Benjamin Pasero
bacf6ed2c1
smoke - properly create screenshots and stop instances ( #137220 )
2021-11-15 12:15:25 +01:00
Daniel Imms
3c9b18e135
Remove test only
2021-11-12 06:16:17 -08:00
Daniel Imms
634dae736e
Merge remote-tracking branch 'origin/main' into tyriar/smoke_test_improvements
2021-11-12 05:45:20 -08:00
Daniel Imms
e6caf0da49
Pass terminal profiles test
2021-11-10 12:27:38 -08:00
Daniel Imms
5bfaf29f2b
Fix layering problem
2021-11-10 08:38:02 -08:00
Benjamin Pasero
bef4dba21f
Parcel update and CI improvements ( #136830 )
...
* 🆙 product
* 🆙 parcel/watcher
* 🆙 parcel/watcher
* tests - separate remote logs from others
* store server logs too
* more tweaks
* fix name
* error when 10s passed
2021-11-10 08:13:56 +01:00
Daniel Imms
bd98431baf
Write basic terminal default profile test
...
Co-authored-by: Megan Rogge <merogge@microsoft.com>
2021-11-09 10:10:54 -08:00
Daniel Imms
28da3051b3
Tidy up smoke test imports
2021-11-09 08:26:39 -08:00
Daniel Imms
281bacb074
Stub out some terminal smoke tests
...
Part of #136064
2021-11-09 08:25:15 -08:00
Benjamin Pasero
f9e8dc2095
🆙 playwright
2021-11-04 09:27:22 +01:00
Benjamin Pasero
7ca8b0f613
Statusbar - checks if 'Go to Line' works if called from the status bar: ( fix #135962 )
2021-10-28 07:22:45 +02:00
Rob Lourens
83fec0220c
Add retry logic for flaky git operations
...
Fix #132218
2021-10-12 17:08:20 -07:00
Benjamin Pasero
769d5fb521
smoke - disable search smoke tests on windows ( #132218 )
2021-09-06 18:07:58 +02:00
Sandeep Somavarapu
77b1fb79f5
Fix #132233
2021-09-06 14:32:19 +02:00
Johannes Rieken
70e30bc64e
disable flaky/failing smoketest, https://github.com/microsoft/vscode/issues/132233
2021-09-03 11:38:14 +02:00
Benjamin Pasero
a3acef919f
smoke - ignore any failure from capturing screenshot
2021-09-03 09:31:01 +02:00
Benjamin Pasero
90443f7a8a
tests - try to make some tests more robust
2021-08-12 07:48:02 +02:00
Benjamin Pasero
6957e52e4b
smoke - always create test folder
2021-08-05 10:31:07 +02:00
Benjamin Pasero
b96644b15f
smoke tests - drop 'tmp' and try to deal with tmp path ourself ( #130102 )
2021-08-05 10:04:00 +02:00
Benjamin Pasero
70e6d3f85d
smoke tests - try to fix data migration test ( #129279 )
2021-08-05 07:56:55 +02:00
Benjamin Pasero
bf4a4a6b05
smoke tests - retry data migration tests on windows ( #129279 )
2021-08-04 18:10:00 +02:00
Benjamin Pasero
2f58131321
smoke tests - fix compile issue
2021-08-04 15:46:10 +02:00
Benjamin Pasero
c28ea165cd
smoke tests - try other ways of ensuring the editor is open for data migration ( #129279 )
2021-08-04 15:32:21 +02:00
João Moreno
132e3783d0
smoke: fix vscode-test cache location
2021-07-16 15:23:48 +02:00
João Moreno
a9e4cb0874
oops
2021-07-16 12:50:07 +02:00
João Moreno
3c4fdb58a1
fix screenshots in smoketests
...
related to #122685
2021-07-16 11:58:52 +02:00
Rob Lourens
50b3811fdc
Revert "Fix running smoketest multiple times"
...
See https://github.com/microsoft/vscode/pull/124666/files#r662070269
This reverts commit 4d694d7dc8
.
2021-07-14 21:32:51 -07:00
Rob Lourens
4d694d7dc8
Fix running smoketest multiple times
2021-07-14 21:21:27 -07:00
Robo
9520f0a515
chore: update to electron 13 ( #124666 )
...
* chore: bump electron@13.0.1
* chore: update cache for native modules
* chore: bump electron@13.1.2
* chore: bump electron@13.1.4
* chore: fix user data dir length for validateIPCHandleLength check
* chore: always disable uploadToServer on OSS
* chore: use process.contextIsolated property
* chore: bump electron@13.1.6
* fix: getUserDataPath helper
2021-07-13 20:01:10 -07:00
Benjamin Pasero
d85bc647ef
smoke tests - automatically test against stable build ( #127799 )
...
* smoke tests - automatically test against stable build
* fix path on Linux/Windows
* 💄
* retry migration tests
* smoke: always download latest previous stable
* hm
Co-authored-by: João Moreno <joao.moreno@microsoft.com>
2021-07-12 05:44:50 -07:00
João Moreno
d737dc4db3
smoke: simplify localization test
2021-07-12 12:20:34 +02:00
João Moreno
a28d028d2d
Enable automated smoketests on Windows ( #128224 )
2021-07-08 19:10:25 +02:00
Dirk Baeumer
c9aea49a87
Re-enable enhanced localization tests
2021-07-07 10:49:52 +02:00
João Moreno
2c7fc24d63
smoke: ctrl-c should kill all processes
2021-07-06 20:30:20 +02:00
Dirk Baeumer
1a3fd783fe
Disable locale info check.
2021-07-06 15:38:21 +02:00
Dirk Baeumer
ae52915f0c
Improve localization tests
2021-07-06 13:15:43 +02:00
Benjamin Pasero
7a0714a6e5
smoke - harden data migration test
2021-07-06 12:10:08 +02:00
Sandeep Somavarapu
062e6519f8
use vscode smoke test check extension
2021-07-02 10:46:31 +02:00
Dirk Baeumer
a63e0a0a21
Reenable German translation test based on new API
2021-06-29 09:23:52 +02:00
Dirk Baeumer
9cfc4bcb5c
Improve wording for failing test case
2021-06-22 13:30:29 +02:00
Dirk Baeumer
1f2a6cc331
Improve localization smoke test
2021-06-21 13:54:16 +02:00
João Moreno
9d4274bf64
skip the entire notebook test suite so Code isn't even launched
2021-06-15 14:07:35 +02:00
Benjamin Pasero
81317867a3
web - disable welcome in smoke tests ( fix #126221 )
...
//cc @JacksonKearl
2021-06-15 10:57:29 +02:00
João Moreno
85a36ae074
enable smoketest on linux
2021-06-11 12:19:17 +02:00
Ladislau Szomoru
291ec03bc3
Smoke tests - run each suite in its own user-data-dir ( #125670 )
...
* Initial commit for isolated smoke tests
* Do not wait for the explorer view when restarting
* Debug failed localization test
* User data path suffix length
* Kill server when application exits
* Do not run yarn install
* Refactor application start/stop
* Reverted some changes
* Missed one
* One more change to revert
* Restored one more file
* Pull request feedback
2021-06-10 06:40:57 -07:00
Ladislau Szomoru
e2d0e05038
Do not run yarn install as part of the smoke tests
2021-06-06 21:14:37 +02:00
Benjamin Pasero
5eec3c86e3
smoke test - disable experiments
2021-05-27 07:30:23 +02:00
Benjamin Pasero
4cda850edf
disable welcome editor in smoke tests ( fix #124674 )
2021-05-26 14:22:13 +02:00
Jackson Kearl
5ac4f5f5be
Fix didn't work
2021-05-21 11:21:27 -07:00
Jackson Kearl
852b70eafe
Try fix ( #124384 )
2021-05-21 17:36:20 +00:00