Brief, mostly facts

Slack ran 200 agentic E2E tests and found Playwright MCP most reliable

Slack Engineering compared Playwright MCP, Playwright CLI and generated Playwright tests over more than 200 runs. Generated tests degraded sharply on complex flows.

Read the source at slack.engineering

What the research found

Slack Engineering ran over 200 agentic E2E runs comparing Playwright MCP, Playwright CLI and generated Playwright tests, finding MCP most reliable and generated tests degrading sharply on complex flows.

Claims and how well the evidence supports them

Slack ran more than 200 agentic E2E test executions using Playwright MCP, Playwright CLI, and generated Playwright tests 73%
Agent-driven E2E runs cost $15-30 per run and take over 10 minutes 69%
Playwright MCP had 0% failure on thread reply and about 12% on search discovery 98%
Generated Playwright tests had about 48% failure rate on search discovery 99%

Evidence

slack.engineering https://slack.engineering/agentic-testing-where-agents-fit-in-the-e2e-testing-stack

Actionability rated medium.