Browser Testing

Request specs test what the server sends. Browser specs test what the user gets: a real browser loads the page, runs its JavaScript, and you drive it with helpers that read like the HTTP ones you already use.

describe("checkout", fn() {
  test("a customer can place an order", fn() {
    visit("/cart")
    fill_in("Coupon", "SAVE10")
    click_button("Apply")

    assert_text("Discount applied")
    assert_no_page_errors()
  })
})

Soli speaks the Chrome DevTools protocol from the binary itself. The only requirement is a Chromium-family browser on the machine — nothing to install into your project, and nothing to keep in sync.

Running browser specs

Browser specs are opt-in: they need a browser and cost seconds rather than milliseconds.

soli test --browser        # run everything, browser specs included
soli test --headed         # same, but watch it happen in a window

A spec counts as a browser spec when a directory called browser appears anywhere in its path. Plain soli test sets those aside and says so — a project with no browser installed still runs green.

tests/
  users_spec.sl              # always runs
  browser/
    checkout_spec.sl         # only with --browser

$ soli test
Skipping 1 browser spec(s) — add --browser to run them.

Choosing a browser

Soli looks for google-chrome, chromium, microsoft-edge and brave-browser on PATH (plus the usual /Applications paths on macOS). Point it elsewhere with SOLI_CHROME_PATH. If nothing is found, --browser fails immediately with what it looked for, rather than thirty seconds later on the first visit().

Navigating

visit("/posts")                  # relative to this worker's test server
visit("https://example.com")     # absolute URLs work too

page_path()                      # "/posts?page=2"
page_url()                       # full URL
page_title()                     # the <title>
page_text()                      # visible text, as the user sees it
page_html()                      # full markup

visit returns once the document has finished loading, so any script the page runs on boot has already run.

Viewport

Every spec runs at a fixed 1280×800 — not “whatever the browser opens with” — so a responsive layout renders the same on your machine and in CI. Declare a different one in the spec:

describe("navigation on a phone", fn() {
  viewport("mobile")               # applies to every test below

  test("the menu collapses", fn() {
    visit("/")
    assert_selector(".menu-toggle")
    assert_no_selector(".sidebar")
  })
})

The declaration belongs to the suite, like before_each: every test in the describe starts in it, and a nested describe inherits it unless it declares its own.

describe("dashboard", fn() {
  viewport("mobile")

  context("on a wide screen", fn() {
    viewport("wide")               # overrides, for this block only
    test("shows both panes", fn() { ... })
  })

  test("stacks the panes", fn() { ... })   # still the phone
})

Sizes can be a preset, a "WxH" string, or two numbers:

viewport("iphone_se")
viewport("1024x768")
viewport(1024, 768)
viewport(1024, 768, {"scale": 2, "mobile": true})

Names are matched loosely, so "iPhone SE" and "iphone-se" are the same request.

Resize inside a test when the resize is the thing under test. viewport() with no arguments reads the current one back as {"width": 390, "height": 844, "scale": 3, "mobile": true}.

test("the sidebar collapses when the window narrows", fn() {
  visit("/dashboard")
  assert_selector(".sidebar")

  viewport("mobile")
  assert_no_selector(".sidebar")
})

Interacting

click("#save")                   # CSS selector
click_link("Edit")               # a link by its text
click_button("Save")             # a button by its label or value

fill_in("#title", "Hello")       # by selector
fill_in("Full name", "Ada")      # by label text
fill_in("email", "a@b.c")        # by name or placeholder

select_option("#role", "admin")  # by value or visible text
check("#agree")
uncheck("#agree")
choose("#plan_pro")
press("Enter")
press("Alt+d")                   # chords: Alt, Ctrl, Shift, Meta/Cmd

Selectors resolve leniently: a CSS selector first, then a matching <label>, then a field's name or placeholder. Write what you see on the page rather than what the markup happens to call it.

Asserting

assert_text("Saved")             # visible text contains this
assert_no_text("Error")
assert_selector("#toast")        # element is present
assert_no_selector(".error")
assert_page_path("/posts/1")
assert_no_page_errors()          # no uncaught exception or console.error

Positive assertions wait. assert_text("Saved") polls until the text appears or the timeout expires, so a spec never has to guess how long a round trip takes. Negative assertions check immediately — waiting for something to stay absent would only slow every passing test down.

assert_text("Report ready", {"timeout": 30})   # seconds; default is 10
wait_for("#chart", {"timeout": 30})

Waiting explicitly

You need these when the next thing you do is not an assertion — evaluate reads the DOM as it is right now and does not wait.

click("#increment")
wait_for_text("count=1")                                # wait first…
assert_eq(evaluate("document.title"), "Counter — 1")    # …then read

Escape hatches

evaluate("window.appVersion")       # any expression, value comes back
screenshot("/tmp/checkout.png")     # PNG of the current view
page_errors()                       # array of captured JS errors

evaluate preserves JavaScript's types: a string stays a string even when it looks like a number, so evaluate("el.textContent") on <span>0</span> gives you "0", not 0.

Signing in

The browser shares the request helpers' cookie jar, so the sign-in you already have keeps working. Cookies flow both ways: a sign-in performed in the browser is visible to a later get() and to signed_in().

before_each(fn() {
  login("ada@example.com", "secret")     # a real POST /login
})

test("the dashboard greets the user", fn() {
  visit("/dashboard")                    # arrives already signed in
  assert_text("Welcome back, Ada")
})

What resets between tests

Each test starts with a clean page-error list, empty sessionStorage and localStorage, and the viewport its suite declared. Without that, a panel one test collapsed — or a resize one test performed — would carry into the rest of the suite and results would depend on test order.

Cookies are not cleared automatically, matching the existing convention for request specs — sign out explicitly when a test needs a guest.

The browser is reused for the whole worker rather than relaunched per test, which is why the reset exists at all — and why a suite of thirty browser specs takes seconds rather than a minute.

In CI

- uses: browser-actions/setup-chrome@v1
  with:
    chrome-version: stable

- run: soli test tests/browser --browser --no-coverage

Browser specs parallelise like any other: each test worker gets its own server and its own browser, so --jobs 3 means three browsers.

Troubleshooting