mirror of
https://github.com/Microsoft/vscode
synced 2024-10-04 02:14:06 +00:00
update smoketest docs
This commit is contained in:
parent
0a27975c25
commit
b55705bd65
|
@ -1,61 +1,30 @@
|
|||
# VS Code Automated Smoke Testing
|
||||
# VS Code Smoke Test
|
||||
|
||||
## Framework
|
||||
* Smoke tests are written using the [Spectron](https://electron.atom.io/spectron/) and [Mocha](https://mochajs.org/) frameworks.
|
||||
* Spectron is used to control the lifecycle of VS Code and also to query the DOM elements of VS Code renderer window using the [WebriverIO](http://webdriver.io/) API that is wrapped in Spectron API.
|
||||
* Mocha is used to launch the smoke tests.
|
||||
## How to run
|
||||
|
||||
## Code Organization
|
||||
* All smoke test code is present under `/test/smoke/` folder. Code is organized into indvidual areas. Each area contains a facade to to access it functionality and a test file to test it. For e.g. `debug` area has `debug.ts` facade class that provides utility methods to access debug functionalities. It also has `debug.test.ts` that tests the debug functionalities.
|
||||
|
||||
* `application.ts` provides APIs to start and stop the VS Code application. It also provides access to various utility APIs like
|
||||
* client
|
||||
* webclient
|
||||
* screenCapturer
|
||||
* workbench
|
||||
|
||||
* `client.ts` class wraps WebDriverIO APIs and provides enhanced APIs for accessing DOM elements. For e.g. it has methods like `waitForElement(selector, accept)` that will query the DOM with the given selector and waits until the element is found or time out after configured time (`5s`).
|
||||
|
||||
* `screenCapturer.ts` allows you capture the screenshots. Capturing is done only if argument `--screenshots` is passed while launching smoke tests. When run out of sources, screenshots are captured under `screenshots` folder in the parent directory of vscode workspace.
|
||||
|
||||
* `workbench.ts` provides utlities to access workbench functionality and also access to other functionality areas like `scm`, `debug`, `editor`. **Note**: All areas should be able to be accessible from workbench either directly or through area traversal.
|
||||
|
||||
### Adding new area and tests
|
||||
To contribute a new smoke test area, add `${area}` folder under `./areas/`. All tests and facades related to this area should go under this folder. Newly added tests should be listed in `main.ts` as imports so that mocha can pick and run them.
|
||||
|
||||
## Running "Out of Sources"
|
||||
```
|
||||
# Dev
|
||||
npm run smoketest
|
||||
```
|
||||
|
||||
## Running Insiders
|
||||
```
|
||||
npm run smoketest -- --build "path/to/code-insiders"
|
||||
```
|
||||
|
||||
## Running Stable
|
||||
```
|
||||
# Specific build
|
||||
npm run smoketest -- --build "path/to/code"
|
||||
```
|
||||
|
||||
To run 'Data Migration' tests, specify the path of the version to be migrated using `--stable` argument
|
||||
|
||||
```
|
||||
# Data Migration tests
|
||||
npm run smoketest -- --build "path/to/code-insiders" --stable "path/to/code"
|
||||
```
|
||||
|
||||
By default screenshots are not captured. To run tests with screenshots use the argument `--screenshots`
|
||||
The script calls mocha, so all mocha arguments should work fine. For example, use `-f Git` to only run the `Git` tests.
|
||||
|
||||
```
|
||||
npm run smoketest -- --screenshots
|
||||
```
|
||||
By default, screenshots are not captured. To run tests with screenshots use the argument `--screenshots`.
|
||||
|
||||
To run a specific test suite use `-f ${suiteName}` argument
|
||||
## Pitfalls
|
||||
|
||||
```
|
||||
npm run smoketest -- -f Git
|
||||
```
|
||||
- Beware of **state**. Invariably, your test suite will leave the workspace in a different state than the one it started
|
||||
with. Try to avoid these situations and strive to leave the workspace in the same state as before the test suite ran.
|
||||
|
||||
# Debugging
|
||||
- Beware of **singletons**. This evil can manifest itself under the form of FS paths, TCP ports, IPC handles.
|
||||
Whenever writing a test, make sure it can run simultaneously with any other tests and even itself.
|
||||
All test suites should be able to run many times in parallel.
|
||||
|
||||
Update
|
||||
- Beware of **focus**. Never depend on DOM elements having focus using `.focused` classes or `:focus` pseudo-classes, since they will lose that state as soon as another window appears on top of the running VS Code window.
|
||||
A safe approach which avoids this problem is to use the `waitForActiveElement` API. Many tests use this whenever they need to wait for a specific element to _have focus_.
|
Loading…
Reference in a new issue