KaneAI
The Story of Redefining Test Automation
Kane AI, built by TestMu AI, is the world's first GenAI-native testing agent you describe what to test in plain English, and it authors, runs, and heals those tests on real browsers and devices.
Industry: SaaS
Team: 1 Designer + 1 PM + 8 Engineers + 5 QA
Timeline: 6 Months
Business Pressure
Why Kane AI, Why Now
AI was helping developers write code faster than ever, but testing remained slow and manual. While features could be built in minutes, QA teams still spent hours creating and maintaining automation scripts.
Together with the Product Manager, we saw an opportunity to build an AI-native testing experience that could make test automation accessible to everyone—not just automation engineers.
Problem
Existing testing tools struggled with both usability and reliability.
key challenges:
Only 15–20% of QA engineers could write automation scripts.
Creating a new automated test took 30–60 minutes.
Locator changes caused flaky tests and unreliable CI pipelines.
Existing AI tools either made incorrect decisions or required constant user intervention.
The real challenge wasn't generating tests—it was helping users trust AI.
Research Insights
To understand the problem, I partnered with the PM to research customer workflows.
We interviewed QA engineers, analyzed support tickets, observed live testing sessions, and reviewed competing AI testing tools.
Key Insights
Users trusted accuracy more than speed.
AI needed the right context before taking action.
Users preferred reviewing uncertain decisions rather than fixing broken tests later.
Recovering from AI failures was more valuable than preventing every failure.
These insights made one thing clear: users didn't need a fully autonomous testing agent, they needed an AI they could understand and trust.
Users
Four people, different pains, one shared anxiety “can I trust what the AI just did?”
Manual QA Tester : Can't code, feels replaceable. Kane AI unlocks automation for them through plain English.
SDET / Automation Engineer : Drowning in flaky tests. Auto-heal and reusable modules cut maintenance hours.
QA Lead / Manager : Coverage gaps, can't hire fast enough. Generates tests straight from PRDs and Jira tickets.
PM / Business Stakeholder : QA is a black box to them. Readable plain-English tests bring them into the loop.
Constraints
Natural language can be ambiguous. A short instruction may not contain enough detail for reliable browser actions.
Users have different technical confidence. Manual testers need readable steps; engineers may need logs and execution detail.
A passed test needs proof. QA teams need evidence of what happened before relying on a result.
Browser flows are unpredictable. Websites change, elements move, and unexpected states occur—so recovery cannot be an afterthought.
My Role
I partnered with the Product Manager to shape the product direction, define MVP priorities, and simplify complex AI workflows into intuitive user experiences.
My ownership included:
Defining AI interaction patterns and user journeys.
Designing trust-building experiences for AI-assisted testing.
Working with engineering to balance usability with technical feasibility.
Validating solutions through user research and iterative feedback.
Driving design quality from concept through implementation.
Solution
Instead of building a fully autonomous testing agent, we focused on helping users understand what the AI was doing, when it needed help, and how they could recover without losing progress.
Start with intent, then turn it into visible steps
Users can describe a testing goal in natural language. KaneAI turns that goal into readable browser actions, allowing the user to see what the test is expected to do.
Let users show an action when describing it is difficult
Manual Interaction provides a second authoring method. When a user cannot clearly explain an action, they can perform it in the browser and KaneAI captures it as a test step.
Verify the outcome with execution evidence
After execution, the Automation dashboard shows pass/fail status, individual commands, browser configuration, logs, and video. This helps users verify that the test actually ran, rather than simply trusting an AI-generated claim.
Keep recovery close to the affected step
During correction or issue handling, observed step-level actions such as Edit, Delete, and Run from here help users modify the relevant part of a flow instead of starting again.
Key features
Test Planner
Write test scenarios in plain English, or point Kane AI at a PRD or Jira ticket, and it generates the steps automatically.
Web Agent
Tests run in real time on a live cloud browser, with instant feedback and quick edits as they execute
.
Modules
Save and reuse test steps like login flows once, so suites scale without duplicate work.
Parameters & Variables
Run the same test with different data inputs flexible tests without copies.
API Testing
Validate backend APIs and responses, not just the UI.
Smart Test Execution
Run, debug, and optimize tests at scale with real time feedback and AI-driven insights.
Tradeoffs
Fast prompts vs clear intent
Natural language is quick, but can be vague. KaneAI keeps the start simple and shows generated steps for review.
Write it vs show it
Typing is faster for simple tasks; Manual Interaction is clearer for complex actions.Simple results vs deep debugging
Automation shows steps, video, and pass/fail clearly; HyperExecute provides technical logs when needed.No-code ease vs engineering flexibility
KaneAI reduces scripting work, while code export and existing workflow support help engineers retain ownership.
Outcome
$1M+ ARR within the first year of public launch.
60% reduction in average test authoring time across pilot customers.
20% more non-technical users (PMs, developers) contributing to test automation.
KaneAI helped secure $38M in additional funding; featured in the Gartner Magic Quadrant for
AI-Augmented Software Testing.
Lessons Learned
Designing AI products isn't about removing humans from the workflow—it's about knowing when users need confidence, context, and control.
The biggest lesson from KaneAI was that users trusted AI more when they understood what it was doing, why it was doing it, and when they should step in.
