Testing and CI

CI runs through GitHub Actions and mainly covers static checks, type checks, tests, builds, automatic format fixes, and documentation deployment. Local automation uses Taskfile.yml as the main entrypoint.

Taskfile entrypoints

task check
task test
task build
task ci

task check covers Ruff, Ruff format check, Markdown lint, Turbo lint, Pyright, ty, and Turbo type check. task test covers Python pytest and docs Vitest. task ci runs check, test, and build in order.

Python / code focused checks

uv run -m ruff check . --output-format=github
uv run -m ruff format --check .
uv run -m pyright .
uv run -m ty check --output-format github
uv run -m pytest

If only one module changed, run the related test file first, then broaden the scope as needed.

Database testing

The project supports six database backends for testing via the SQLALCHEMY_DATABASE_URL environment variable, following NoneBot database testing best practices.

Local multi-database testing

By default, tests use SQLite. To test against PostgreSQL / MySQL / MariaDB / Oracle / SQL Server locally:

Install the database drivers (already in the test dependency group):

uv sync --all-groups

Start a database server (via Docker or local install).
Set the SQLALCHEMY_DATABASE_URL environment variable and run tests:

# PostgreSQL
export SQLALCHEMY_DATABASE_URL="postgresql+psycopg://postgres:postgres@localhost:5432/postgres"
uv run -m pytest

# MySQL
export SQLALCHEMY_DATABASE_URL="mysql+aiomysql://mysql:mysql@localhost:3306/mymysql"
uv run -m pytest

# MariaDB (SQLAlchemy auto-detects MariaDB via VERSION() without dedicated driver)
export SQLALCHEMY_DATABASE_URL="mysql+aiomysql://mariadb:mariadb@localhost:3306/mymariadb"
uv run -m pytest

# Oracle (oracledb 2.0+ default Thin mode, no Oracle Client required)
export SQLALCHEMY_DATABASE_URL="oracle+oracledb://system:oracle@localhost:1521/?service_name=FREEPDB1"
uv run -m pytest

# SQL Server (requires aioodbc + pyodbc + system ODBC Driver 18)
export SQLALCHEMY_DATABASE_URL="mssql+aioodbc://sa:Password!1@localhost:1433/master?driver=ODBC+Driver+18+for+SQL+Server&TrustServerCertificate=yes"
uv run -m pytest

Run migrations before testing:

uv run nb orm upgrade

System dependencies

Oracle: uses ghcr.io/gvenzl/oracle-free:23-slim (GitHub Container Registry to avoid Docker Hub rate limits); requires Oracle 12.2+ (128-char identifiers + MERGE INTO).
SQL Server: Linux needs the msodbcsql18 system package (ACCEPT_EULA=Y apt-get install -y msodbcsql18 mssql-tools18 unixodbc-dev); macOS uses the same name from brew; Windows is built-in.

CI multi-database matrix

CI runs tests on 10 jobs across 6 database engines, covering both LTS and latest versions:

SQLite (default, no service required; version pinned by Python sqlite3/aiosqlite)
PostgreSQL 16 (postgres:16-alpine, supported until Nov 2028) and PostgreSQL 18 (postgres:18-alpine, latest)
MySQL 8.4 LTS (mysql:8.4, until ~2032) and MySQL 9.7 LTS (mysql:9.7, until ~2034)
MariaDB 11.4 LTS (mariadb:11.4, until May 2029) and MariaDB 11.8 LTS (mariadb:11.8, until Jun 2028)
Oracle 23ai (ghcr.io/gvenzl/oracle-free:23-slim via GitHub Container Registry; only freely available version, no LTS split)
SQL Server 2022 and SQL Server 2025 (mcr.microsoft.com/mssql/server:2022-latest / 2025-latest + apt msodbcsql18; migrated off the deprecated azure-sql-edge image)

Each matrix entry carries an engine + image field; service containers select their image via ${{ matrix.db.engine == '<engine>' && matrix.db.image || '' }} so multiple versions of the same engine coexist in one matrix. The fail-fast: false strategy ensures all databases are tested even if one fails. Oracle / SQL Server start slowly (health-start-period 90-180s); a full matrix run takes roughly 8-15 minutes.

Docs / frontend focused checks

The docs site uses pnpm, Vitest, Playwright, ESLint, Fumadocs, and Turbo:

pnpm --filter docs lint
pnpm --filter docs test
pnpm --filter docs run test:e2e:hook
pnpm --filter docs run test:e2e
pnpm --filter docs run lint:links
pnpm turbo run check-types --filter=docs
pnpm turbo run build --filter=docs

Vitest covers component and library behavior under apps/docs/src/__tests__. Playwright tests live under apps/docs/e2e and cover browser-level docs navigation. Local hooks run the Chromium smoke command (test:e2e:hook) on docs changes; CI runs all configured Playwright browser projects and uploads the HTML report plus traces.

Write Playwright tests with role/text locators and web-first assertions such as toBeVisible() and toHaveAttribute(). Avoid sleeps, screenshot assertions, and broad visual checks unless the change specifically needs them.

Markdown checks

pnpm exec markdownlint-cli2

Globs and ignores are configured centrally in .markdownlint-cli2.jsonc at the repository root, so no path arguments are needed.

Internationalization checks

After changing translatable strings, run:

task i18n

If you only change documentation that describes i18n, gettext catalogs do not need to be regenerated.

CI workflows

🧪 Python CI: A shared detect-changes composite action (.github/actions/detect-changes) outputs boolean flags per file type (python/markdown/frontend/frontend-code/frontend-style/frontend-content/frontend-tsx), then conditionally runs Static Analysis (on Python or markdown changes, via task ci:static) and Tests & Type Check (on Python changes, across the multi-database matrix). Auto Fix & Format runs on main/dev pushes. Conditions align with pre-commit v3 NEEDS_LINT/NEEDS_TYPE_CHECK/NEEDS_DOCS_TEST.
🧪 Frontend CI: Uses the same detect-changes action, then conditionally runs Docs Check — ESLint on code/style, check-types on any frontend, link validation on content, Vitest on code/content.
🎭 Playwright: runs docs E2E tests for apps/docs and packages changes, installs browser dependencies, and uploads Playwright report artifacts.
👷 CI-builds: runs task ci:build; on pushes to main, dev, or releases/**, it also syncs the version from the latest tag, bumps the dev version, writes core_version, builds, archives artifacts, generates provenance, and tags the version.
📚 Docs Deploy: when docs-related paths are pushed to main or dev, it runs pnpm/Turbo lint, docs test, docs build, and deploys GitHub Pages.

Handling CI failures

Open the failing job logs and locate the command, rule, and line number. Fix only the smallest related scope and rerun the corresponding local command.

Testing and CI

On this page