In passing, I’ve been revising a couple of JupyterLab extensions I’ve cobbled together in the past and started looking at producing some simple Playwright tests for them.
For years now, there has been a testing helper framework called Galata, now bundled as part of JupyterLab, although the docs are scarce, the examples few, and things you might expect to be there are lacking. (There are also seemingly obvious signals lacking in the core JupyterLab codebase — you can find out when a notebook code cell is queued to run, stops running, and whether it has successfully or unsuccessfully run, but not find out when it actually starts running — but that’s another story.)
Anyway… I wasted a whole chunk of time trying to use Galata when it would probably have been easier just writing simple, native Playwright tests, and then a chunk of time more trying to use Galata in a minimal demo of hooking up a node.js Playwright Docker container to a container running a JupyterLab environment so I can run tests in the Playwright container with the testing framework completely isolated from the Jupyter container.
Minimally, I used a minimal Docker Compose script (run using docker-compose up) to create the two containers (I used a node Playwright container, put into a holding position) and networked them together. A shared folder on host containing the tests was mounted in into the playwright container (I could also have mounted some test notebooks from another directory into the Jupyter container).
# ./docker-compose.yml
networks:
playwright-test-network:
driver: bridge
services:
playwright:
image: mcr.microsoft.com/playwright:v1.43.0-jammy
stdin_open: true
tty: true
# Perhaps tighten up things by requiring the container
# to be tested to actually start running:
#depends_on:
#- tm351
volumes:
- ./:/home/pwuser
networks:
- playwright-test-network
tm351:
image: mmh352/tm351:23j
networks:
- playwright-test-network
A playwright.config.js file of the form:
// ./playwright.config.js
module.exports = {
use: {
// Browser options
headless: true,
},
};
and an example test file showing how to log in to the auth’ed Jupyter environment:
// tests/demo-login.spec.ts
import { test, expect } from "@playwright/test";
test("form submission test", async ({ page }) => {
// If we run this from the docker-compose.yml entrypoint,
// we need to give the other container time to start up
// and publish the Jupyter landing page.
// await page.waitForTimeout(10000);
// Does playwright have a function that polls a URL a
// few times every so often for a specified time or
// number of poll attempts?
// That would be a more effective way of waiting.
// Navigate to the webpage
await page.goto("http://tm351:8888/login?next=%2Flab");
// Enter a value into the form
await page.fill("#password_input", "TM351-23J");
await page.screenshot({ path: "screenshot1.png" });
// Click the button
await page.click("#login_submit");
// Wait for response or changes on the page
await page.waitForResponse((response) => response.status() === 200);
await page.waitForTimeout(10000);
// Take a screenshot
await page.screenshot({ path: "screenshot2.png" });
});
Inside the Playwright container, we can run npx install @jupyterlab/galata to install the Galata test framework and use galata test functions and helpers. However, the Galata test package just seemed to mess things up for me and not work, in a way that the Playwright test package didn’t, and did just work.
After looking up the name of the Playwright container (docker ps) I logged into it (docker exec -it playtest-playwright-1 ; by default this was as root, but we can force that by adding -u root to the exec command) and then manually ran the tests: npx playwright test. Ctrl-C to stop the containers and docker-compose rm to delete them.
We can run the tests directly from the docker-compose.yml:
networks:
playwright-test-network:
driver: bridge
services:
playwright:
image: mcr.microsoft.com/playwright:v1.43.0-jammy
#stdin_open: true
#tty: true
entrypoint: ["npx","playwright","test"]
working_dir: /home/pwuser
volumes:
- ./:/home/pwuser
networks:
- playwright-test-network
tm351:
image: mmh352/tm351:23j
networks:
- playwright-test-network
Ideally, we’d just want to start up both containers, run the tests, then shut them down, but docker-compose doesn’t seem to offer that sort of facility (does kubernetes?).
In passing, I also note that there is a pre-built Docker container that uses the Python Playwright API (mcr.microsoft.com/playwright/python:v1.42.0-jammy), which creates opportunities for using something like playwright-pytest. I also note that within a notebook, we could inject and run Python code into notebooks (or include Python scripts in pre-written notebooks) that control the JupyterLab UI using ipylab. The ipylab package lets you script and manipulate the behaviour of the JupyterLab UI from a notebook code cell. This creates the interesting possibility of having notebook based Python scripts control the JupyterLab UI, and whose execution is controlled by the test script (either a Python test script or a Typescript test script).
And related to pytest testing, via (who else but?!) @simonw here, I note inline-snapshot, a rather handy tool that lets you specify a test against a thing, and if the thing doesn’t exist, it will grab a gold-master copy of the thing for you. Playwright does this natively for screenshots. for example, the first time a screenshot comparison test is run as part of its visual comparisons offering.
OUseful.Info, the blog... 