From 9ee44989d82ea0a0288cf4d6e41fb09bdc4eab1c Mon Sep 17 00:00:00 2001 From: Joao Moreno Date: Mon, 25 Sep 2017 22:50:39 +0200 Subject: [PATCH] smoke: more pitfalls --- test/smoke/README.md | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/test/smoke/README.md b/test/smoke/README.md index a779e9450ec..f58f2113924 100644 --- a/test/smoke/README.md +++ b/test/smoke/README.md @@ -19,16 +19,12 @@ By default, screenshots are not captured. To run tests with screenshots use the ## Pitfalls -- Beware of **state**. Invariably, your test suite will leave the workspace in a different state than the one it started - with. Try to avoid these situations and strive to leave the workspace in the same state as before the test suite ran. +- Beware of **state**. The tests within a single suite will share the same state. -- Beware of **singletons**. This evil can manifest itself under the form of FS paths, TCP ports, IPC handles. - Whenever writing a test, make sure it can run simultaneously with any other tests and even itself. - All test suites should be able to run many times in parallel. +- Beware of **singletons**. This evil can, and will, manifest itself under the form of FS paths, TCP ports, IPC handles. Whenever writing a test, or setting up more smoke test architecture, make sure it can run simultaneously with any other tests and even itself. All test suites should be able to run many times in parallel. -- Beware of **focus**. Never depend on DOM elements having focus using `.focused` classes or `:focus` pseudo-classes, since they will lose that state as soon as another window appears on top of the running VS Code window. - A safe approach which avoids this problem is to use the `waitForActiveElement` API. Many tests use this whenever they need to wait for a specific element to _have focus_. +- Beware of **focus**. **Never** depend on DOM elements having focus using `.focused` classes or `:focus` pseudo-classes, since they will lose that state as soon as another window appears on top of the running VS Code window. A safe approach which avoids this problem is to use the `waitForActiveElement` API. Many tests use this whenever they need to wait for a specific element to _have focus_. -- **Never** wait longer than a couple of seconds for anything, unless it's justified. Think of it as a human - using Code. Would a human take 10 minutes to run through the Search viewlet smoke test? Then, the computer - should even be faster. **Don't** use `setTimeout` just because. Think about what you should wait for in the DOM to be ready, then wait for that instead. \ No newline at end of file +- Beware of **timing**. You need to read from or write to the DOM... yeah I know. But is it the right time to do that? Can you 100% promise that that `input` box will be visible and in the DOM at this point in time? Or are you just hoping that it will be so? Every time you want to interact with the DOM, be absolutely sure that you can. Eg. just because you triggered Quick Open, it doesn't mean that it's open; you must wait for the widget to be in the DOM and for its input field to be the active element. + +- Beware of **waiting**. **Never** wait longer than a couple of seconds for anything, unless it's justified. Think of it as a human using Code. Would a human take 10 minutes to run through the Search viewlet smoke test? Then, the computer should even be faster. **Don't** use `setTimeout` just because. Think about what you should wait for in the DOM to be ready, then wait for that instead. \ No newline at end of file