Skip to main content
Documentation›Test Creation Agent›Qpilot_ask_ai
Search
Summarize with:ChatGPTPerplexityGrokClaude

1. Overview

Ask AI is a step type in QPilot that lets you query an AI model at any point during a test run. Instead of writing assertions manually, you describe what you want to check or extract in plain English — the AI analyses the app screen or your uploaded data and returns a structured result.

Ask AI works with three optional inputs you can combine in any way:

InputWhat it does
PromptYour plain-English question or instruction — the only required field
Skill File (.zip)Structured agent skill following the Agent Skills standard — packages instructions, workflows, and domain logic that tell the AI how to perform a specific task
Context File (.txt / .pdf / etc.)Reference data the AI reads to answer your question —
Capture Screenshot (toggle)Sends the current device screen to the AI alongside the prompt

2. Step 1 — Upload Files to My Data

Before using a Skill File or Context File in an Ask AI step, you must first upload the file to Pcloudy. Files uploaded here become available in the Ask AI dropdowns across all your tests.

2.1 How to Upload

  1. Go to Functional Experience — from the left navigation menu.
  2. Click on the Files/Testdata tab — in the Select Application panel on the left side of the screen.
  3. Click Upload — and select your file from your local machine.
  4. The file appears in the list — with its name, upload date, and file size once the upload is complete.
  5. Note the name of uploaded file

2.2 Supported File Types

File TypeUse in Ask AI as
.zipSkill File — a packaged Agent Skill following the Agent Skills standard (SKILL.md + optional scripts and resources)
.txtContext File — plain text data such as order records, user data, or reference content
.csvContext File — tabular data such as product catalogues, pricing sheets, or test datasets
.mdContext File — structured reference content such as feature specs, release notes, or documentation
.pdfContext File — documents such as compliance specs or terms and conditions
caution

Note

Files uploaded under Files/Testdata are shared across your Functional Experience session.

Once uploaded, note the file name and mention it in Ask AI dialog box.

3. Step 2 — Add an Ask AI Step

Once your files are uploaded, add an Ask AI step to your test at the point where you need AI analysis.

Open your test in QPilot — and enter / and choose AI step.

The Ask AI dialog opens — — configure the fields described in Section 4.

Click Add Step — to save the step and continue building your test.

4. The Ask AI Dialog — Fields

FieldDescription
PromptRequired. A plain-English question or instruction. Maximum 1500 characters. Be specific about what you want checked and what format you want returned.
Skill FileTextbox. Enter the name of .zip files you have uploaded to Files/Testdata.
Context FileTextbox. Enter the name of files you have uploaded to Files/Testdata.
Capture ScreenshotToggle. When ON, QPilot captures the current device screen and sends it to the AI as part of the input. Use this when your prompt refers to what is visible on screen.
Retry Just This StepToggle. Visible during retry only. When ON, only this step re-executes — the rest of the test is not re-run.

5. Capture Screenshot

The Capture Screenshot toggle controls whether the AI can see your app's current screen.

note

Toggle OFF

The AI uses only the Prompt, Skill File, and Context File.

Use this when your question is about data — for example, querying a context file for a specific value, without needing to look at the screen.

info

Toggle ON

QPilot takes a screenshot of the live device screen at the moment this step runs and sends it to the AI.

Use this when your question refers to something visible on screen — a label, a button colour, a count, a layout check.

6. Context File — Query Your Own Data

The Context File lets you give the AI a reference document to read when answering your prompt. The AI does not look at the screen — it reads the file and returns an answer based on its contents.

This is the right approach when your question is about data that exists outside the app — an order record, an API response, a user database, or a product catalogue.

6.1 How It Works

  1. Upload your data file— to Files/Testdata (Section 2). For example: test.txt containing order records.
  2. Add an Ask AI step —Enter the name of uploaded file
  3. Write a prompt — that asks a question about the data in the file. Keep Capture Screenshot OFF — the AI reads the file, not the screen.
  4. The AI reads the file — answers your question, and stores the result as a variable for use in later steps.

6.2 Real Example — Order Data Lookup

Context file (test.txt) contains a list of customer orders. The prompt asks a specific question about one record.

Context file: test.txt
Alice Johnson | Order #1042 | Item: Laptop Stand | Qty: 2 | Unit Price: 49.99 | Total: 99.98
Bob Smith     | Order #1043 | Item: USB Hub      | Qty: 1 | Unit Price: 29.99 | Total: 29.99
Carol White   | Order #1044 | Item: Mouse Pad    | Qty: 3 | Unit Price: 12.50 | Total: 37.50
note

Ask AI step configuration

Context File: test.txt

Capture Screenshot: OFF

Prompt: What is the total price of Alice Johnson's order?

Output: {{totalPrice}} = {"total_price": 99.98}

The output variable {{totalPrice}} can now be used in a subsequent Assert step to verify the amount shown in the app matches the expected value from the data file.

6.3 When to Use Context File vs Capture Screenshot

ScenarioUse
Question is about data in a file (order records, API response, user list)Context File — Screenshot OFF
Question is about what is visible on the device screenCapture Screenshot ON — no Context File needed
Question is about what is visible on the device screenContext File + Capture Screenshot ON (Upcoming)
Apply domain rules from a skill file to what is on screenSkill File + Capture Screenshot ON (Upcoming release)

7. Skill File — Domain Knowledge for the AI

A Skill File is a .zip package built to the Agent Skills open standard (agentskills.io). It is not just a knowledge document — it is a structured set of instructions and resources that tells the AI agent how to perform a specific task, step by step.

This is fundamentally different from a Context File. A Context File gives the AI data to read. A Skill File gives the AI a workflow to follow.

info

Agent Skills Standard

Agent Skills is an open format developed by Anthropic and adopted across AI platforms.

A skill is a folder containing a SKILL.md file (with metadata and instructions) plus optional scripts, reference materials, and assets. The folder is zipped and uploaded as the Skill File.

7.1 Skill File Structure

Contents of a Typical Skill .zip
my-skill/
├── SKILL.md      # Required: skill name, description, and instructions
├── scripts/      # Optional: executable code
├── references/   # Optional: supporting documentation
└── assets/       # Optional: templates, rules, and resources

The SKILL.md file is the core. It contains:

  • name — the skill identifier
  • description — what the skill does (used by the AI to decide when to activate it)
  • instructions — the step-by-step procedure the AI follows when the skill is active

7.2 Context File vs Skill File — Key Difference

Context FileSkill File
Raw data — the AI reads it as inputStructured instructions — the AI follows them as a workflow
Defines how the AI should perform a taskDefines how the AI should perform a task
Any file type: .txt, .csv, .pdf, .mdAlways a .zip built to the Agent Skills standard
Example: order_records.txt — Alice's total is 99.98Example: brand_audit.zip — SKILL.md instructs AI to check colours, then typography, then component rules in sequence
No special format requiredMust contain SKILL.md at minimum

7.3 Real Example

name: brand-guidelines-audit
description: Verifies mobile app screens...

instructions:
  - Check primary CTA colour (#FF5722)
  - Check heading typography (Poppins 600)
info

Ask AI step configuration

Skill File: brand_guidelines.zip

Capture Screenshot: ON

Prompt: Run the brand guidelines audit on this screen.

ScreenResultWhat the Skill Caught
CheckoutFAIL'Place order' CTA is #2196F3 (blue). Brand requires #FF5722. Severity: High.
Order SuccessWARN'Order Placed!' heading is font-weight 400. Skill requires Poppins 600. Severity: Medium.
HomePASSAll checks passed — colour, typography, logo, copy casing correct.

7.4 Why Ask AI + Skill File Beats Pixel Diff Tools (Upcoming)

Traditional pixel diff (e.g. Applitools)Ask AI + Skill File
Requires a baseline screenshot per screenNo baseline needed — reasons from guidelines directly
Intentional redesigns break baselines, causing false failuresIntentional changes don't cause failures — rules, not pixels, define pass/fail
Cannot detect wrong font weight, incorrect copy casing, or tone of voice errorsCatches semantic errors invisible to pixel comparison
Baseline maintenance overhead grows with the appUpdate the skill file once — all tests inherit it
info

Skill files are reusable

Write payment_rules.zip once. Attach it to every payment-related Ask AI step across your suite.

When a business rule changes, update the skill file in one place. All tests pick up the change automatically.

8. Step Output and Variables

Every Ask AI step stores its result as a named variable, visible in the step output and usable in any subsequent step.

  • The variable name is auto-generated from your prompt — e.g. {{totalPrice}}, {{loginKeywordCount}}, {{termsSummary}}, {{acceptWordCount}}.
  • The output is displayed inline under the step — e.g. {{totalPrice}} = {"total_price": 99.98}.
  • Reference the variable in a later Assert step, a log step, or another Ask AI step.
Variable Chaining Example

Step A — Ask AI

Context File: test.txt\

Capture Screenshot: OFF

Prompt:

What is the total price of Alice Johnson's order?

Output:

{
  "total_price": 99.98
}

Variable:

{{totalPrice}}

Step B — Assert

{{totalPrice.total_price}}

equals 99.98

Tip: Ask the AI to return JSON in your prompt. This makes the output clean and easy to reference with dot notation in later steps.

9. Example Prompts

Count keyword occurrences on screen

info

Prompt

How many times has the LOG-IN keyword appeared on this screen? Return JSON: { "log\_in\_count": <number> }

Summarise on-screen content

info

Prompt

Summarise the terms and conditions shown on this screen in 2–3 sentences. Return JSON: { "summary": "<text>" }

Look up a value from a context file

info

Prompt

What is the total price of Alice Johnson's order? Return JSON: { "total\_price": <number> }

Verify screen against brand guidelines

info

Prompt

Verify this screen against brand_guidelines.zip. List any violations with severity. Return JSON: { "result": "PASS/FAIL/WARN", "violations": [ { "element": "", "issue": "", "severity": "High/Medium/Low" } ] }

Cross-reference screen with context file

Prompt

The context file contains the expected order details.

Verify that the order ID, items, and total shown on screen match the context file exactly.

{
  "match": true,
  "mismatches": []
}

10. Retrying an Ask AI Step

  1. Click the retry icon — (circular arrow) next to the Ask AI step in the test runner.
  2. The Ask AI dialog reopens — pre-filled with the existing configuration.
  3. A warning banner appears — — 'Ensure app is in correct state before retrying' — check that the device is on the correct screen before proceeding.
  4. Enable 'Retry just this step' — to re-run only this step without re-running the full test.
  5. Click Retry Step — to execute.
warning

Important

If Capture Screenshot is ON, the AI will use whatever is currently visible on the device — screen state matters.

If using a Context File, the file content is re-read from your Files/Testdata at retry time.

11. Quick Reference — Which Inputs to Use

Note: As of now user can either use Screenshot or Context or Skill File. Combining two or more features is not yet supported and is planned for a future release

**What you want to do ****Recommended Configuration **
Count or extract something visible on screenPrompt + Capture Screenshot ON
Summarize on-screen contentPrompt + Capture Screenshot ON
Look up a value from your own data filePrompt + Context File + Screenshot OFF
Verify screen matches expected data from a filePrompt + Context File + Capture Screenshot ON
Check screen against brand / business rulesPrompt + Skill File + Capture Screenshot ON
Check screen against brand / business rulesPrompt + Skill File + Context File + Screenshot OFF

Did this page help you?