Testing a Plugin

A plugin’s tests come in two tiers. Unit tests are *.test.ts files beside your source. They run in Node with vitest and import their harnesses from plumix/test. End-to-end tests live in your plugin’s e2e/ folder. They drive a running site in Chromium with Playwright and import from plumix/test/playwright. Test a behaviour once, in the lowest tier that can observe it.

Unit tests in Node

A unit test calls your code the way the site calls it: a procedure with its input, a route through the dispatcher, an event or filter handler with its arguments. A procedure’s input can be a File, and fakeImage builds one:

src/photos.test.ts

import { base, v } from "plumix/plugin";
import { createRpcHarness, fakeImage } from "plumix/test";
import { expect, test } from "vitest";

const PNG_SIGNATURE = [0x89, 0x50, 0x4e, 0x47];

const uploadPhoto = base
  .input(v.object({ photo: v.file() }))
  .handler(async ({ input }) => {
    const head = new Uint8Array(await input.photo.slice(0, 4).arrayBuffer());
    return {
      name: input.photo.name,
      isPng: PNG_SIGNATURE.every((byte, i) => head[i] === byte),
    };
  });

test("an uploaded PNG reaches the procedure whole", async () => {
  const h = await createRpcHarness({ authAs: "editor" });
  const upload = uploadPhoto.callable({ context: h.context });

  const result = await upload({ photo: fakeImage("dish.png") });

  expect(result).toEqual({ name: "dish.png", isPng: true });
});

In your plugin, uploadPhoto lives in its server module and the test imports it.

Setting up

Install vitest:

pnpm add -D vitest

The whole vitest config is one call:

vitest.config.ts

import { defineTestConfig } from "plumix/vite";

export default defineTestConfig();

Run pnpm exec vitest run. It runs two projects in Node: unit, every *.test.ts and *.test.tsx under src/, and feature, those under tests/feature/. Run one with --project unit or --project feature.

defineTestConfig takes overrides and merges them over its config, so a setup file, an alias or a define goes there:

vitest.config.ts

import { defineTestConfig } from "plumix/vite";

export default defineTestConfig({
  test: {
    setupFiles: ["./test/setup.ts"],
    exclude: ["fixtures/**"],
  },
});

The overrides apply to both projects. Narrow with exclude, never with include: Vitest adds your include to each project’s own.

A server test

A server test boots your plugin in the dispatcher harness, seeds through the factories, and asserts on what a request answers:

src/recipes.test.ts

import { definePlugin } from "plumix/plugin";
import { createDispatcherHarness } from "plumix/test";
import { expect, test } from "vitest";

const recipes = definePlugin("recipes", {
  setup: (ctx) => {
    ctx.registerEntryType("recipe", {
      label: "Recipes",
      isPublic: true,
      hasArchive: "recipes",
    });
  },
});

test("a published recipe is listed on its archive", async () => {
  const h = await createDispatcherHarness({ config: { plugins: [recipes] } });
  const author = await h.factory.author.create();
  await h.factory.entry.create({
    type: "recipe",
    slug: "caponata",
    title: "Caponata",
    status: "published",
    authorId: author.id,
  });

  const res = await h.fetch("/recipes");

  res.assertStatus(200);
  expect(await res.text()).toContain("Caponata");
});

Events and filters

testSite from plumix/test boots core and the plugins you list. It records every event the site emitted and every filter it applied, so a test asserts on them after the call, with no listener set up first. Each assertion takes the value your plugin declared with defineEvent or defineFilter, never its id as a string, so renaming or removing one breaks the test when it compiles:

src/rating.test.ts

import { testSite } from "plumix/test";
import { test } from "vitest";

import { recipe, recipes } from "./index.js";

const site = await testSite({ plugins: [recipes()] });

test("rating a recipe announces the rating", async () => {
  const dish = await site.factory.published.create({ type: "recipe" });

  await site.actingAs("member").rpc.recipes.rate({ id: dish.id, stars: 5 });

  site.events.assertEmitted(recipe.rated, { entryId: dish.id, stars: 5 });
  site.filters.assertApplied(recipe.stars, { entryId: dish.id });
});

Assertion	Passes when
site.events.assertEmitted(event, payload?)	the event was emitted, with a payload that has every key you pass.
site.events.assertNotEmitted(event)	the event was never emitted.
site.filters.assertApplied(filter, input?)	the filter was applied, with an input that has every key you pass.
site.filters.assertNotApplied(filter)	the filter was never applied.

site.events.emitted(event) returns each payload the event was emitted with, in order. A key you pass is matched deeply, and a key you leave out is not checked. An event or filter you scope with .for(name) matches only what fired for that name, while the unscoped one matches it for any name, as its handlers do. A failed assertion throws a plain Error that lists everything emitted or applied, with its payload or input. Each test starts with nothing recorded.

Recording changes nothing: every handler still runs. To change what a filter returns in a test, add a handler the way a plugin does, in a plugin the test installs:

import { definePlugin } from "plumix/plugin";

const generous = definePlugin("generous", {
  setup: (ctx) => {
    ctx.filters.add(recipe.stars, (stars) => Math.min(stars + 1, 5));
  },
});

const site = await testSite({ plugins: [recipes(), generous] });

Uploads

fakeFile and fakeImage build an upload as a value. fakeFile(name, { content, size, type, lastModified }) returns a real File, typed from its extension unless you say otherwise. fakeImage(name, { width, height }) returns a valid PNG of that size, which a browser decodes and a magic-byte check accepts. Node has File, Blob, FormData and fetch, so the procedure gets the same File a browser would send.

Components and the RPC client

A component keeps its logic in functions it calls: which state it shows, such as a validation error, an empty list or a control a role may not use, is decided by a function, and the unit test calls that function once per state. What the rendered page does when a user acts on it belongs in an end-to-end test.

Your admin code calls its procedures through createPluginRpcClient. stubPluginRpc answers those calls at the fetch boundary through oRPC’s own handler, so one test can check that a call crosses the wire intact. Pass it the router type your server module hands ctx.registerRpcRouter, imported with import type, and the route map is checked against it. The client builds its URL from location, which Node does not have, so the test gives it one:

src/admin/photo-rpc.test.ts

import { createPluginRpcClient } from "plumix/admin";
import { fakeFile, stubPluginRpc } from "plumix/test";
import { afterEach, expect, test, vi } from "vitest";

import type { RecipesRouter } from "../server/router.js";

afterEach(() => {
  vi.unstubAllGlobals();
});

test("an uploaded file reaches the procedure whole", async () => {
  vi.stubGlobal("location", new URL("https://cms.example/admin"));
  let received: File | undefined;
  stubPluginRpc<RecipesRouter>("recipes", {
    "photos/upload": ({ file }) => {
      received = file;
      return { size: file.size };
    },
  });
  const rpc = createPluginRpcClient<RecipesRouter>("recipes");

  await rpc.photos.upload({ file: fakeFile("a.txt", { content: "hello" }) });

  expect(received).toBeInstanceOf(File);
  expect(received?.name).toBe("a.txt");
  expect(await received?.text()).toBe("hello");
});

A responder that throws PluginRpcError answers with that code, status and data, so a test can drive the failure your code handles. A procedure the route map does not name answers 404. That way a test cannot pass on a call it never declared.

End-to-end tests

An end-to-end test drives your plugin’s pages the way a user does, against a small site in your plugin’s playground/ folder that installs it. It covers the journeys a user takes, not every state your unit tests already cover. The suite’s globalSetup.ts signs a user in with actingAs, and every page starts with that session:

e2e/globalSetup.ts

import { writeFile } from "node:fs/promises";
import { resolve } from "node:path";
import { actingAs, openPlaygroundDb } from "plumix/test/playwright";

export default async function globalSetup(): Promise<void> {
  const db = await openPlaygroundDb({
    cwd: resolve(process.cwd(), "playground"),
  });
  const { storageState } = await actingAs(db, "editor");
  await writeFile(
    resolve(process.cwd(), "storageState.json"),
    JSON.stringify(storageState),
    "utf8",
  );
}

e2e/recipes.spec.ts

import { expect, test } from "plumix/test/playwright";

test("an editor opens the recipes page", async ({ page }) => {
  await page.goto("pages/recipes");

  await expect(page.getByTestId("recipe-list")).toBeVisible();
});

openPlaygroundDb opens the playground’s migrated database. actingAs(db, role) creates a user of that role, or takes a user you pass, and returns its session as a Playwright storageState. A test that needs rows of its own writes them at its start through openPlaygroundDb. Past seeding, a test acts through the page and asserts on what the page shows, so a behaviour only a browser has, such as decoding an image or a drag, is tested here.

The suite’s config

Install Playwright’s test runner, then download Chromium once:

pnpm add -D @playwright/test
pnpm exec playwright install chromium

definePlumixE2EConfig from plumix/test/playwright runs a plugin’s admin pages against a real site. Point playground at a small site that installs your plugin, and pass configDir: import.meta.dirname. With a playground, it runs globalSetup.ts beside the config and starts every page from the storageState.json it writes; pass seedAdminSession: false to skip both:

import { definePlumixE2EConfig } from "plumix/test/playwright";

export default definePlumixE2EConfig({
  port: 3010,
  configDir: import.meta.dirname,
  playground: "../playground",
});

The server command it builds is:

cd <playground> && rm -rf -- <wipe> && plumix migrate fresh --yes && <extraSetup> && plumix dev --port <port>

It deletes the playground’s local state, at each path its runtime’s plumix.e2e.wipe names, then plumix migrate fresh --yes applies core’s and every plugin’s shipped migrations to the empty database. <extraSetup> runs only when you pass extraSetup, such as a seed command. Nothing is generated and nothing outside the runtime’s local state is deleted, so a playground’s committed drizzle/ stays as it is. A runtime that runs its CLI some other way, as Bun does with bun --bun, supplies its own prefix in place of plumix.

Every spec shares the playground’s one database. The suite runs on a single worker, and the plumixDbBaseline fixture restores the database after each attempt. It leaves each owner’s __drizzle_migrations_* table alone. Pass sharedDatabase: false for a site that never serves from that database, such as one that creates a database per session. The suite then runs on parallel workers with no snapshot.

Every page runs in UTC, so a date a spec expects renders the same on every machine. Set use.timezoneId on a project to test another zone.

A drag

dragTo from plumix/test/playwright drags with the mouse from one element to another. It waits until both elements have a box that has stopped moving, presses at the source, moves to the target in steps, and releases there. A list that re-renders as the page settles cannot make it aim at a stale position.

This spec drags the first row of a sortable list below the second:

e2e/recipes.spec.ts

import { dragTo, expect, test } from "plumix/test/playwright";

test("dragging a recipe below the next one reorders the list", async ({
  page,
}) => {
  await page.goto("pages/recipes");
  const handle = page.getByTestId("recipe-drag-caponata");
  const next = page.getByTestId("recipe-row-panzanella");

  await dragTo(page, handle, next, {
    targetPosition: (box) => ({ x: box.width / 2, y: box.height * 0.75 }),
  });

  await expect(page.getByTestId("recipe-list")).toHaveAttribute(
    "data-order",
    "panzanella,caponata",
  );
});

sourcePosition and targetPosition give the point to press and release, measured from the top-left of that element’s box. Both default to the centre, and a point may fall outside the box, as when a drag pans a canvas past its own handle. steps sets how many moves lead to the target (10 by default). beforeDrop runs at the target with the button still held, so a test can check that the drag is live before it drops. dragTo does not scroll, so bring both elements into view first.

Testing your blocks in the editor

The editor fixture from plumix/test/playwright drives the real block editor: it opens or creates an entry, inserts blocks from the catalog by click or drag, selects them, works the selection toolbar and the Layers rail, fills the inspector, styles blocks per device, edits rich text, and saves. Each action waits on what the editor shows, never on a timer. editor.stored() returns the entry’s block tree as the server stores it, so a test can assert on what was persisted.

This spec inserts a site’s own demo/rating block, configures it, and checks what was saved:

e2e/rating.spec.ts

import { expect, test } from "plumix/test/playwright";

test("a rating is inserted, configured and stored", async ({ editor }) => {
  await editor.create("post");

  const rating = await editor.insert("demo/rating");
  await editor.inspector.set("score", 5);
  await editor.inspector.set("symbol", "heart");
  await editor.inspector.set("label", "Would read again");
  await editor.save();

  await expect(rating).toContainText("Would read again");
  expect(await editor.stored()).toMatchObject([
    {
      name: "demo/rating",
      attrs: { score: 5, symbol: "heart", label: "Would read again" },
    },
  ]);
});

inspector.set(field, value) and inspector.get(field) address an input by the name your block declares, the key stored() returns its value under, not by its visible label. They handle every control the inspector renders: text, number, URL, select, radio, checkbox, switch, combobox, and a media picker, which takes a media id.

• Navigation. editor.open({ type, id }) opens an existing entry, and editor.create(type) starts a new one.
• Canvas. editor.canvas.block(name, index?) returns a locator for a block inside the canvas iframe, editor.canvas.blocks() the names of the top-level blocks in order, and editor.canvas.focus(block) selects and clicks a block so the keys pressed next start inside the canvas.
• Insert. editor.insert(name, { at, via }) puts the block at "start", at "end" (the default), { after: block }, { before: block } or { into: block, slot? }, and returns its locator. via: "drag" drops it from the catalog with dragTo; a click reaches a nested position only through an empty slot.
• Selection. editor.select(block), then editor.toolbar.moveUp(), .moveDown(), .selectParent() and .remove().
• Structure. editor.inspector.press(button) presses a button that reshapes the selected block: "add-column" on a columns block, or "add-table-row", "add-table-column", "remove-table-row" and "remove-table-column" on a table or any block inside it.
• Layers. editor.layers.row(block) returns a block’s row, .focus(block) focuses it so the keys pressed next (Delete, F2) act on it, .rename(block, label) labels it through its menu, .act(block, action) runs "copy", "paste", "duplicate" or "delete" from its menu, and .drag(block, onto) drags it onto another row’s place.
• Styles. editor.styles.set(property, value) sets a style property of the selected block through its control, whether that control takes a typed value, a keyword, a toggle or, for the fill image with the media plugin installed, the id of a library image; null clears it. .token(property, id) picks a theme token, .mark(mark) presses a text formatting toggle, .textShadow(shadow) sets the text shadow, .hide(device, hidden) hides the block on a device, .className(value), .declare(property, value), .tagName(tag) and .attribute(name, value) work the Custom CSS and HTML attributes sections, and .inherited(property) reads the value an empty control inherits from a wider device.
• Devices. editor.device(device) sizes the canvas to "desktop", "tablet" or "mobile". A style set after it lands in that device’s bucket, large, medium or small in stored().
• Rich text. editor.richText(block) returns type(text), select() (the whole text, for keys pressed next), format(format) (a mark such as "bold", "heading-2", "bullet-list" or "quote"), link(url) and unlink().
• History. editor.undo() and editor.redo() press the header’s Undo and Redo buttons.
• Saving. editor.save() resolves once every edit has reached the server, editor.publish() publishes the entry or its pending edits, and editor.reload() reloads the editor.
• Lifecycle. editor.back() leaves with the header’s Back button, editor.discard() drops a published entry’s pending edits, and editor.restoreRevision(revision) previews a revision from the Revisions sheet and restores it. A test seeds that revision with the revision factory: .transient({ of: entry }).create({ content }) writes it as a save of entry would, with the content it had back then.
• Page tab. editor.document.setVisibility(policy) picks a visibility policy by key (null for the type’s default) and editor.document.toggleTerms(taxonomy, termIds) toggles terms in a taxonomy’s picker. editor.document.visibility() and editor.document.terms(taxonomy) return each picker, whose text is the chosen label.
• Revisions. editor.revisions.open(tab?) opens the revisions sheet on its "all" or "publishes" tab, and editor.revisions.rows locates its rows in order. loadMore() pages through it and loadMoreButton locates its button while another page is left, comment(id, text) saves a revision’s comment and commentOf(id) locates it, and diff(id) opens a revision’s diff and returns its revision and current panes.
• Stale draft. editor.staleDraft.dialog locates the dialog the editor opens when a pending draft predates the last publish, and editor.staleDraft.useTheirs() discards the draft for the published entry.
• Stored state. Beside editor.stored()’s blocks, editor.storedDocument() returns the entry’s stored title, meta (the visibility and template choices included) and term ids by taxonomy, and editor.storedRevisions() every one of its revisions, newest first, with each one’s title and comment.

Mail

A test checks the mail your plugin sends by name, the same name ctx.mail.send takes. It never reads a mail server.

Faking mail

The dispatcher and RPC harnesses each install a mail fake and expose it as h.mail. It records every mail ctx.mail.send sent, with its name, its props, the recipient and the rendered message. With no mailer in the harness config, the fake is the site’s mailer too, so a site configured for magic links boots and nothing is delivered.

src/welcome.test.ts

import { createDispatcherHarness } from "plumix/test";
import { test } from "vitest";

test("requesting a magic link mails the user, and only that", async () => {
  const h = await createDispatcherHarness({
    config: { auth: { magicLink: { siteName: "Recipes" } } },
  });
  await h.factory.user.create({ email: "ann@example.test" });

  await h.fetch("/_plumix/auth/magic-link/request", {
    method: "POST",
    json: { email: "ann@example.test" },
  });

  h.mail.assertSent("magicLink", { to: "ann@example.test" });
  h.mail.assertNotSent("emailChange");
});

To test the transport itself, pass your own mailer in the harness config. Your mailer then carries the mail and h.mail still records it. Pass mailer: undefined to test a site that has no mailer. fakeMail() from plumix/test builds a fake on its own, for a context you build with createTestContext. There, pass the fake as the mailer and its record as onMailSent.

Assertions

Every assertion is typed against the mail registry, so a misspelled name or a where written for the wrong props fails to compile. A failing assertion throws a plain Error that lists the mail that was sent, by name and recipient, so it reads the same in Vitest, Bun’s test runner or Playwright.

Assertion	Passes when
assertSent(name, { to, where, times })	a matching mail was sent, exactly times times if you pass it.
assertNotSent(name, { to, where })	no matching mail was sent.
assertNothingSent()	no mail of any name was sent.
assertSentCount(n)	n mails were sent, whatever their names.

to matches the recipient’s address and where takes the mail’s props. sent(name) returns each mail of that name as { props, to, subject, text, html }, for a check the assertions don’t cover:

const [mail] = h.mail.sent("magicLink");
expect(mail?.props.url).toContain("/_plumix/auth/magic-link/verify");

Testing mail content

renderMail(name, props, options) from plumix/test renders a mail the way sending does, with no recipient and no send. It returns { subject, text, html }. locale picks the language, siteName names the site, and config carries the plugins that declare mail and the theme’s and site’s overrides, which apply as they do on a send:

src/welcome-content.test.ts

import { renderMail } from "plumix/test";
import { expect, test } from "vitest";

test("the magic-link subject is in German", async () => {
  const { subject } = await renderMail(
    "magicLink",
    { url: "https://example.com/verify?token=t", ttlSeconds: 900 },
    { locale: "de", siteName: "Recipes" },
  );

  expect(subject).toBe("Bei Recipes anmelden");
});

Mail in end-to-end tests

plumix dev catches the mail of a site that has no mailer. The mailbox fixture of test from plumix/test/playwright reads that mail. mailbox.latest(name, { to }) waits until a matching mail arrives, or the test times out. It returns the newest one, with its props and its rendered message. mailbox.clear() empties the caught mail, so a test starts from none:

e2e/sign-in.spec.ts

import { expect, test } from "plumix/test/playwright";

test.use({ storageState: { cookies: [], origins: [] } });

test("a user signs in through the magic link", async ({ page, mailbox }) => {
  await mailbox.clear();
  await page.goto("login");
  await page.getByTestId("login-email-input").fill("ann@example.test");
  await page.getByTestId("login-magic-link-submit").click();

  const { props } = await mailbox.latest("magicLink", {
    to: "ann@example.test",
  });
  await page.goto(props.url);

  await expect(page.getByTestId("dashboard-welcome-heading")).toBeVisible();
});

A site configured for magic links needs a mailer to build. To keep catching its mail under plumix dev, leave the mailer out there only: mailer: process.env.PLUMIX_DEV ? undefined : yourMailer.

Passkeys in end-to-end tests

The passkey fixture of test from plumix/test/playwright is the user’s device. Ask for it and the page gets a Chromium virtual authenticator, so a passkey sign-in or registration runs with no hardware. passkey.credentials() lists the credentials the authenticator holds, so a test can assert that a ceremony stored one. The authenticator is removed when the test ends, and a test that doesn’t ask for passkey gets none. Virtual authenticators need Chromium: asking for passkey in another browser throws.

e2e/bootstrap.spec.ts

import { expect, test } from "plumix/test/playwright";

test("the first admin signs up with a passkey", async ({ page, passkey }) => {
  await page.goto("bootstrap");
  await page.getByTestId("bootstrap-email-input").fill("ann@example.test");
  await page.getByTestId("bootstrap-name-input").fill("Ann");
  await page.getByTestId("bootstrap-submit").click();

  await expect(page.getByTestId("dashboard-welcome-heading")).toBeVisible();
  await expect.poll(() => passkey.credentials()).toHaveLength(1);
});

Related

Overview covers installing a plugin and the registration context your plugin’s setup receives. Project Structure lists every plumix subpath, plumix/test and plumix/vite among them. Content Modelling covers writing the descriptor these tests boot.

Next steps

Write the unit tests first. They cover what your plugin registers, what its procedures answer, and the logic behind each admin screen. Then add an end-to-end test for each journey a user takes through your plugin’s pages.