Velocity places significant emphasis on testing. Our goal is to create a reliable e-commerce platform that operates predictably in a myriad of use cases. To achieve this, we utilize End-to-End (E2E) testing extensively as our primary method of ensuring application robustness. We chose Playwright as our E2E test framework due to its versatility and ease of use. Playwright is an open-source testing tool designed for modern web applications, supporting multiple browsers and providing features like real-time user interaction simulation, device emulation, network control, and more.

Before executing the tests, it's necessary to install Playwright's browser dependencies by running the following command: npx playwright install. Tests can be executed using the command pnpm test, and Playwright CLI arguments can be passed as well with pnpm test -- <playwright arguments>. For example, to select a specific test case, you can run pnpm test -- -g <test-name>.

During development, you can prefix the test script with DEV=1 to reduce the number of browsers being tested to Chrome desktop & mobile (e.g., DEV=1 pnpm test). This feature can speed up test execution during development.

This chapter covers our testing setup, special features, and established conventions to help you write tests efficiently and maintainably.

Isolated Testing: One Application per Test Case

Unlike conventional E2E testing setups that run a single application instance and share it among all tests, we run one application per test case. This ensures perfect isolation and lets us inject changes into each application individually.

We accomplish this by running each application in its own Node.js Virtual Machine (VM), using the standard Node.js VM API. Each application runs on its own dynamically assigned port, which means each test operates in a completely independent scope. This approach also allows us to inject custom code and exchange data between tests.

For example, it is possible to overwrite connectors using the useTestInterceptor. This feature enables us to create scenarios typically not achievable with conventional E2E testing setups, which are more common in unit testing.

HTTP Request Recording and Replaying

Another important feature in our testing setup is the ability to intercept HTTP requests to record and replay them. This practice significantly increases the execution speed and overall test stability.

To utilize this feature, you need to explicitly activate the recorder via startRecorder and stop it with stopRecorder. If stopRecorder is not called, the recording is automatically stopped at the end of the test.

To refresh the recordings or if you choose not to use them, set the UPDATE_RECORDING environment variable, like so: UPDATE_RECORDING=1 pnpm test. If you run a subset of the tests, only the recordings of the affected test cases will be re-recorded.

Remember that all tests should be able to function with the recordings. This condition can be particularly tricky for account creation tests as using the same email address can conflict with pre-existing accounts, while recording these test cases will cause issues when they are re-recorded.

Testing Conventions

• Test Independence: All test cases should be runnable independently from each other. This guideline improves both performance and maintainability. To respect this, avoid writing tests that depend on the state created by other tests.
• Deterministic Tests: Write tests that produce the same results given the same input. Avoid relying on real-time data that might change and affect test outcomes.
• Avoiding Sleeps: Don't use sleeps (playwright.waitForTimeout) if it can be avoided. Tests using sleeps are often fragile and can lead to unreliable results.
• Test Organization: All tests are located under the test directory and should be grouped by domain. This practice makes it easier to navigate through tests and understand the functionality being tested.
• Test Coverage: You can get the test coverage by running the coverage script with pnpm coverage. This command will execute all tests to determine their coverage. However, it's important to note that frontend JavaScript is not included in the coverage calculation. Our goal is to reach close to 100% test coverage.
• Code Reusability: Consider creating reusable functions for common test actions or setups. This approach reduces duplication and improves maintainability.
• Descriptive Naming: Use descriptive naming for test cases and test files. Clear naming helps clarify what each test is meant to verify.

Writing tests for your code is as crucial as writing the code itself. A well-tested codebase is easier to debug, simpler to understand, and much safer to modify. With Velocity's advanced and comprehensive testing setup, you have a strong foundation for creating robust, reliable applications.

Playwright Errors

If you get errors that playwright cannot find the browser executable, try installing it at the version that is defined in the projects package.json, i.e.

npx playwright@1.38.0 install