vscode/test/smoke/README.md

# VS Code Smoke Test

## How to run

```
# Dev
yarn smoketest

# Build
yarn smoketest --build "path/to/code"
```

Screenshots can be captured when tests fail. In order to get them, you need to use the argument `--screenshots SCREENSHOT_DIR`.

## How to debug issues

You can use a `--verbose` flag to log all the low level driver calls make to Code.

You can also use any mocha argument. For example, use `-f Git` to filter all tests except the `Git` tests.

## Pitfalls

- Beware of workbench **state**. The tests within a single suite will share the same state.

- Beware of **singletons**. This evil can, and will, manifest itself under the form of FS paths, TCP ports, IPC handles. Whenever writing a test, or setting up more smoke test architecture, make sure it can run simultaneously with any other tests and even itself.	All test suites should be able to run many times in parallel.

- Beware of **focus**. **Never** depend on DOM elements having focus using `.focused` classes or `:focus` pseudo-classes, since they will lose that state as soon as another window appears on top of the running VS Code window. A safe approach which avoids this problem is to use the `waitForActiveElement` API. Many tests use this whenever they need to wait for a specific element to _have focus_.

- Beware of **timing**. You need to read from or write to the DOM... but is it the right time to do that? Can you 100% guarantee that that `input` box will be visible at that point in time? Or are you just hoping that it will be so? Hope is your worst enemy in UI tests. Example: just because you triggered Quick Open with `F1`, it doesn't mean that it's open and you can just start typing; you must first wait for the input element to be in the DOM as well as be the current active element.

- Beware of **waiting**. **Never** wait longer than a couple of seconds for anything, unless it's justified. Think of it as a human using Code. Would a human take 10 minutes to run through the Search viewlet smoke test? Then, the computer should even be faster. **Don't** use `setTimeout` just because. Think about what you should wait for in the DOM to be ready and wait for that instead.
update smoketest docs 2017-09-25 15:36:58 +00:00			`# VS Code Smoke Test`
Added step to run npm install 2017-06-30 13:32:45 +00:00
update smoketest docs 2017-09-25 15:36:58 +00:00			`## How to run`
smoke: update wiki 2017-09-08 17:04:52 +00:00
update smoketest launch 2017-09-04 09:32:41 +00:00			```
update smoketest docs 2017-09-25 15:36:58 +00:00			`# Dev`
improve smoketest docs 2018-04-18 10:01:17 +00:00			`yarn smoketest`
update smoketest launch 2017-09-04 09:32:41 +00:00
improve smoketest docs 2018-04-18 10:01:17 +00:00			`# Build`
			`yarn smoketest --build "path/to/code"`
smoke: update wiki 2017-09-08 17:04:52 +00:00			```
Moved contribution section to readme. 2017-06-01 13:50:23 +00:00
update README.md 2018-05-02 09:33:49 +00:00			Screenshots can be captured when tests fail. In order to get them, you need to use the argument `--screenshots SCREENSHOT_DIR`.
improve smoketest docs 2018-04-18 10:01:17 +00:00
update README.md 2018-05-02 09:33:49 +00:00			`## How to debug issues`
Moved contribution section to readme. 2017-06-01 13:50:23 +00:00
update README.md 2018-05-02 09:33:49 +00:00			You can use a `--verbose` flag to log all the low level driver calls make to Code.

			You can also use any mocha argument. For example, use `-f Git` to filter all tests except the `Git` tests.
Added explanation where to look for test data. 2017-06-14 12:06:42 +00:00
update smoketest docs 2017-09-25 15:36:58 +00:00			`## Pitfalls`
Moved contribution section to readme. 2017-06-01 13:50:23 +00:00
improve smoketest docs 2018-04-18 10:01:17 +00:00			`- Beware of workbench state. The tests within a single suite will share the same state.`
Added smoke test source code. 2017-05-22 11:56:23 +00:00
smoke: more pitfalls 2017-09-25 20:50:39 +00:00			`- Beware of singletons. This evil can, and will, manifest itself under the form of FS paths, TCP ports, IPC handles. Whenever writing a test, or setting up more smoke test architecture, make sure it can run simultaneously with any other tests and even itself. All test suites should be able to run many times in parallel.`
update smoketest launch 2017-09-04 09:32:41 +00:00
smoke: more pitfalls 2017-09-25 20:50:39 +00:00			- Beware of focus. Never depend on DOM elements having focus using `.focused` classes or `:focus` pseudo-classes, since they will lose that state as soon as another window appears on top of the running VS Code window. A safe approach which avoids this problem is to use the `waitForActiveElement` API. Many tests use this whenever they need to wait for a specific element to _have focus_.
smoke: pitfalls 2017-09-25 20:46:02 +00:00
improve smoketest docs 2018-04-18 10:01:17 +00:00			- Beware of timing. You need to read from or write to the DOM... but is it the right time to do that? Can you 100% guarantee that that `input` box will be visible at that point in time? Or are you just hoping that it will be so? Hope is your worst enemy in UI tests. Example: just because you triggered Quick Open with `F1`, it doesn't mean that it's open and you can just start typing; you must first wait for the input element to be in the DOM as well as be the current active element.
message 2017-10-10 12:41:11 +00:00
improve smoketest docs 2018-04-18 10:01:17 +00:00			- Beware of waiting. Never wait longer than a couple of seconds for anything, unless it's justified. Think of it as a human using Code. Would a human take 10 minutes to run through the Search viewlet smoke test? Then, the computer should even be faster. Don't use `setTimeout` just because. Think about what you should wait for in the DOM to be ready and wait for that instead.