KaneAI Manual Interaction Mode

Overview

Manual Interaction support in KaneAI enables users to manually interact with the applications in real-time during authoring or playground session to record test steps. This mode lets you momentarily pause the agent, take full control of the application under test, and reach an exact UI state before resuming automated authoring. Whether you're testing Web or Mobile apps, Manual Interaction allows for actions like clicking, typing, scrolling and navigating through the application, simulating real user behaviors.

note

Manual Interaction is only supported in Web Browser on desktop & Mobile App testing (Android). Support for manual interaction on iOS app testing will be coming soon.

Guide for using Manual Interaction:

While in Authoring mode, simply interact with the app yourself, after enabling manual interaction by clicking on Manual Interaction button - the Agent will record your actions and turn them into test steps automatically.

If you want to switch back to natural language instructions then you can click on Turn off Manual Interaction. Instructions for assertions currently are only supported in natural language.

At each step performed, it is highlighted if the instruction was recorded using manual interaction or not with the help of an icon.

Example Use Cases:

• Execute the workflow manually once and have the agent convert it into an automated test script.
• When the intended action is too intricate to articulate clearly in natural language.
• Navigating to a screen accessible only through an long tap sequence.
• Working with dynamic UI elements such as infinite-scroll lists or carousels which render items on demand.

** Note** that following operations are not present in the Manual Interaction for now:

Web Browser on Desktop:

• Drag and drop on canvas elements
• Double click
• Right click
• Hover

Mobile App:

• Two fingers scroll
• Click and hold
• Double click
• Copy paste
• Recent apps
• Mobile keyboard support
• Using backspace
• Typed characters will be considered in the instruction even if they didn't get typed in the input box. eg: In an Input box where only numbers are allowed if you try to type alphabets then those will get included in the instruction but not in the input box.
• System popups

Tutorial: