Qpilot Coordinate Actions: Tap, Drag, Slide | Pcloudy Docs

Summarize with:ChatGPTPerplexityGrokClaude

Last updated on : 29 Oct 2026

Overview

QPilot now supports Coordinate Actions, enabling testers to automate application UI interactions using screen coordinates and AI vision, even when traditional locators or XPath are unavailable. It can perform actions such as tap, drag and drop, and slide by identifying interaction points on the screen. This is particularly useful for Unity based gaming applications, where the entire UI is often rendered as a single element, making conventional locator based automation challenging.

Steps

Once you start a Qpilot playground session simply enter the prompt to perform the actions (mentioned below) using coordinates

Action	What it Does?	Example Prompts
tap_on_element	Taps a target on screen by its position, even with no XPath or when all screen text sits in one element	"tap on login", "tap on submit"
drag_drop_coordinate	Drags one item and drops it onto another using coordinates	"drag Group1 and drop onto Group2 on the screen using coordinates", "drag a and drop onto b using coordinates"
slide_right_coordinate	Performs a right slide/swipe on the screen using coordinates	"Slide right on the screen using coordinates", "Slide right using coordinates"

Benefits

• Automate screens that expose no usable locators, such as canvas, game, custom-rendered or WebView screens where the whole screen is one element.
• Click any visible target with a plain-language prompt instead of hunting for an XPath.
• Cover gestures like drag-and-drop and slides that element-based actions cannot reach.
• Fewer blocked test cases and less locator maintenance.