日本語版はこちら

🔎 Latest CI reports: JUnit summary / Flaky ranking / Coverage HTML

Demos

01

Spec to Cases

A pipeline that turns specification Markdown into test case JSON with LLM drafting and rule-based post-processing.

• Schema validation and type-preserving transformation logic.
• CLI helpers and JSON samples for quick onboarding.

Evidence →

02

LLM to Playwright

A proof of concept where an LLM expands acceptance criteria and generates Playwright tests automatically.

• Robust selector strategy based on data-testid with a11y scanning baked in.
• Data-driven tests via JSON / CSV drivers for a minimal setup.

Evidence →

03

CI Flaky Analyzer

A CLI that detects flaky tests from CI logs and automatically produces HTML reports and ticket templates.

• Streaming analysis for large JUnit XML files with scoring stored as JSONL / HTML.
• One-command generation of HTML reports, JSONL history, and GitHub Issue templates.

Evidence →

04

LLM Adapter — Provider Orchestration

An adapter that orchestrates provider calls and comparison runs while keeping production fallbacks intact.

• Supports OpenAI, Gemini, Ollama, and OpenRouter behind a unified tracing layer.
• Run prompts with llm-adapter --provider adapter/config/providers/openai.yaml --prompts datasets/golden/tasks.jsonl to replay the shared JSONL dataset.
• Launch batch comparisons via python adapter/run_compare.py --prompts datasets/golden/tasks.jsonl python adapter/run_compare.py --providers adapter/config/providers/openai.yaml --prompts examples/prompts/ja_one_liner.jsonl for side-by-side provider diffs.

Evidence →

Weekly Summary

Weekly QA Snapshot — 2025-10-13

• TotalTests: 126
• PassRate: 97.62%
• NewDefects: 0
• TopFailureKinds: rate_limit, http_5xx, timeout Read the weekly summary →

01. Spec to Cases

• Minimal pipeline that generates test cases from specification Markdown.
• Deliverable: cases.sample.json
• Extra materials: spec.sample.md

02. LLM to Playwright

• PoC that enriches acceptance criteria with an LLM and auto-generates Playwright tests.
• Deliverables: tests/generated/
• Samples: blueprint.sample.json / demo/
• References: tests/README.md

03. CI Flaky Analyzer

• Detects flaky tests from CI logs and enables re-runs and automated ticketing end-to-end.
• Deliverable: Running npx flaky analyze generates projects/03-ci-flaky/out/index.html (HTML/CSV/JSON) retrievable from CI artifacts.
• Sample workflow: Ingest any JUnit XML via npx flaky parse --input <path-to-xml> and store the history for later analysis.

04. LLM Adapter — Provider Orchestration

• Connects OpenAI, Gemini, Ollama, and OpenRouter with resilient fallback strategies and shared telemetry hooks.
• llm-adapter --provider adapter/config/providers/openai.yaml --prompts datasets/golden/tasks.jsonl replays the JSONL dataset with a single provider.
• python adapter/run_compare.py --prompts datasets/golden/tasks.jsonl records comparison metrics from the JSONL prompt list for audits.
• python adapter/run_compare.py --providers adapter/config/providers/openai.yaml --prompts examples/prompts/ja_one_liner.jsonl records comparison metrics from the JSONL prompt list for audits.
• Reference: evidence/llm-adapter

View all weekly summaries →

Evidence Library

• QA Evidence Catalog
• Test Plan
• Defect Report Sample

Operations Notes

• The weekly-qa-summary.yml workflow automatically updates docs/weekly-summary.md.
• tools/generate_gallery_snippets.py generates highlight cards from the weekly summary.
• .github/workflows/pages.yml deploys everything under docs/ to GitHub Pages.