Skip to content

Foundational Level — AI-Based Testing Interview (Q&A)

A plain-English Q&A pack for a QA/SDET preparing for an AI-based testing interview — no heavy AI theory required.

What is a Large Language Model (LLM)?

An LLM is a computer program trained on a huge amount of text so it can predict the next word (or "token") in a sentence. In plain words: it is a very advanced autocomplete that has read most of the internet. It does not "look up" facts in a database; it generates the most likely next words based on patterns it learned. Because it works on probability, the same question can give slightly different answers each time — it is non-deterministic. This also means it can sound very confident and still be wrong, so a human must verify important output.

Example: You type "Write a login test case for a banking app" and the LLM produces steps and expected results by predicting what a good test case usually looks like — it did not fetch a real one from a file.

Remember: LLM = smart autocomplete, not a fact database — probabilistic, so verify.

Yes. The best-known families in early 2026 are Anthropic's Claude (Opus, Sonnet, and Haiku in the 4.x family), OpenAI's GPT-4 and GPT-5 series, Google's Gemini, Meta's Llama, and Mistral. There is also GitHub Copilot, which is a coding assistant built on top of LLMs and lives inside your editor. In plain words: these are the "brands" of AI text models you will hear about. Some are closed/hosted (Claude, GPT, Gemini) and some are open-weight (Llama, Mistral) that teams can run themselves. For testing work, Claude and GPT are common for reasoning and test-case drafting, while Copilot is popular for writing code inline.

Example: A tester might use Claude to draft test scenarios, then use GitHub Copilot inside VS Code to autocomplete the Selenium code for those scenarios.

Remember: Claude, GPT, Gemini, Llama, Mistral — plus Copilot for coding.

What is Generative AI?

Generative AI is AI that creates brand-new content — text, code, images, or audio — instead of just sorting or scoring existing data. In plain words: old-style AI mostly answered "is this spam, yes or no?"; generative AI can write the whole email. LLMs are one type of generative AI focused on text and code. This is powerful for testers because it can draft test cases, generate test data, and write automation code. But because it invents content, it can also invent things that are wrong, so review is essential.

Example: Traditional AI classifies a transaction as "fraud" or "not fraud." Generative AI writes a fresh set of negative test cases for the fraud-detection screen.

Remember: Generative AI creates content; traditional AI just labels or predicts.

How is AI being used in software testing today?

AI helps testers speed up the boring and repetitive parts of the job. In plain words: it acts like a fast junior teammate who drafts things you then check. Common uses: generating test cases from requirements, creating test data, converting manual steps into automation code (Selenium/Playwright), suggesting locators, explaining stack traces and error logs, and summarizing failures in CI runs. Some tools also do "self-healing" locators and flaky-test detection. Importantly, AI is used as an assistant — a human still designs the strategy and verifies the output.

Example: After a nightly Playwright run fails, AI summarizes 40 failing tests into "3 root causes" so the tester investigates faster.

Remember: AI is the fast drafting assistant; the tester stays the decision-maker.

What are the limitations of AI-generated test cases?

AI-generated test cases can look polished but miss the point. In plain words: it writes what usually appears, not necessarily what your app needs. Limits include: it may hallucinate (invent) fields, buttons, or rules that don't exist; it often misses edge cases and negative scenarios unless prompted; it doesn't know your business rules, compliance needs, or hidden system dependencies; and results vary run to run. It can also produce duplicates or shallow "happy path" tests. So AI output is a starting draft, never a finished, trusted suite.

Example: For a funds-transfer feature, AI writes 10 happy-path cases but forgets "transfer more than the daily limit" and "transfer to a closed account" — the risky banking edge cases.

Remember: Great first draft, weak on edge cases and your business rules.

What is AI-assisted testing?

AI-assisted testing means a human tester uses AI tools to work faster while staying in control of decisions. In plain words: the AI helps, you decide. The tester frames the problem, gives context, and reviews and edits everything the AI produces. It is the opposite of "fully autonomous testing," where AI would run unchecked. This model keeps quality and accountability with the human. It is the honest, realistic way AI is used in QA today.

Example: You ask AI to convert 15 manual regression steps into Playwright code, then you review the selectors and assertions before committing.

Remember: AI assists, human decides — assistant, not replacement.

How would you validate AI-generated test cases?

Treat AI output like a pull request from a junior engineer: review before trusting. In plain words: check that each test is real, relevant, and runnable. Steps: (1) map each case back to an actual requirement or acceptance criterion; (2) confirm the fields, screens, and rules actually exist in the app; (3) check coverage — are edge, negative, and boundary cases present; (4) run the tests and see if they pass/fail for the right reasons; (5) remove duplicates and fix wrong expected results. If a case can't be traced to a real requirement, drop it. This turns a raw draft into a trustworthy suite.

Example: AI generates a test for a "remember me" checkbox; you check the actual login page, find no such checkbox, and delete that case as a hallucination.

Remember: Trace to requirements, verify against the real app, then run it.

What risks exist when relying entirely on AI for testing?

(Note: this question appears twice in the source; answered once here.) Relying 100% on AI is risky because AI is probabilistic and can be confidently wrong. In plain words: if nobody checks, mistakes ship. Risks: hallucinated tests that validate features that don't exist; missed critical edge and compliance cases; false confidence from a green suite that tests the wrong things; over-fitting to the "happy path"; leaking sensitive data into external AI tools; and loss of human accountability if something breaks in production. In regulated areas like banking, unchecked AI can cause real compliance failures. The safe model is always human-in-the-loop.

Example: An all-AI suite passes fully, but never tested "withdraw beyond overdraft limit" — a real customer overdraws and the bank takes a loss.

Remember: Full trust in AI = confident, invisible mistakes; always keep a human in the loop.

What is a prompt?

A prompt is the instruction or question you give an AI to get a response. In plain words: it is what you type to tell the AI what you want. A prompt can include the task, context, examples, constraints, and the format you want back. The clearer and more specific the prompt, the better the output. Vague prompts give vague, generic answers. Good prompting is a real skill for testers using AI.

Example: Prompt: "Write 5 negative test cases for a login API using RestAssured, cover invalid password, locked account, and expired token; output as a table."

Remember: Prompt = your instruction to the AI; specific in, specific out.

Why is prompt quality important?

Because the quality of your prompt directly controls the quality of the answer. In plain words: garbage in, garbage out. A good prompt gives the AI context (what app, what feature, what rules), a clear task, constraints, and the output format you want — so you get usable results instead of generic fluff. A weak prompt like "write some tests" produces shallow, off-target output you have to redo. Better prompts save time, reduce hallucinations, and cut down your review effort. For a tester, prompt quality is the difference between a helpful draft and wasted time.

Example: "Write tests" gives 5 generic cases; "Write boundary tests for an amount field that accepts 1 to 100000, INR only, 2 decimals" gives exactly the cases you need.

Remember: Better prompt in, less rework out.

Have you used ChatGPT, Claude, Gemini, or Copilot for testing?

(Sample first-person answer to adapt.) Yes, I use these tools regularly as part of my day-to-day QA work. In plain words: I treat them as a fast assistant, not an oracle. I use Claude and ChatGPT to draft test cases from requirements and to generate realistic test data like account numbers, IFSC codes, and edge-case amounts. When a CI run fails, I paste the stack trace or error log and ask the AI to explain the likely root cause, which speeds up triage. I also convert manual test steps into Selenium or Playwright code and ask it to write RestAssured assertions for API responses. GitHub Copilot helps me autocomplete boilerplate automation code inside the editor. In every case, I review, run, and correct the output before it goes into the codebase, because AI can be confidently wrong.

Example: I gave Claude a failing Playwright log; it pointed to a stale locator, I confirmed it in the DOM, and fixed the selector — saving 20 minutes of manual digging.

Remember: Use AI to draft, data, explain, and convert code — but always review before you trust.

What testing activities can AI automate?

AI can speed up or partly automate many repetitive testing tasks. In plain words: the drafting and grunt work, not the judgment. It can generate test cases and test data, convert manual steps into automation scripts, suggest and self-heal locators, write API assertions, summarize failures and logs, detect flaky tests, and help write documentation. It can also assist with coverage gap analysis and boilerplate code. What it should NOT own alone: test strategy, risk-based prioritization, business-rule validation, and final sign-off — those stay with the tester.

Example: AI regenerates broken locators after a UI redesign (self-healing) so most of the regression suite keeps running without manual fixes.

Remember: AI automates the repetitive drafting; humans keep strategy and sign-off.

Give an example where AI helped improve your productivity.

(Sample STAR-style story to adapt.) Situation: Our team migrated a banking web app and 60+ manual regression steps needed to become Playwright tests before a tight release. Task: I had two days to automate them without slipping the deadline. Action: I fed the manual steps into Claude in small batches and asked it to draft Playwright + TypeScript tests with proper waits and assertions, in our project's page-object style. I then reviewed each generated test, fixed a few wrong selectors and one hallucinated field, and wired them into CI. Result: I converted the whole suite in about a day instead of three, and used the saved time to add the negative and boundary cases the AI had missed. In plain words: AI did the boring first draft, I did the thinking and the checking.

Example: Converting "Login → transfer funds → verify balance" manual steps into a reviewed Playwright test in minutes instead of an hour.

Remember: AI drafts fast, you review and add the risky edge cases — big time saver.

Night-before recap

  • LLM = probabilistic next-word predictor, not a fact database — verify its output.
  • Popular LLMs: Claude, GPT, Gemini, Llama, Mistral; Copilot for coding.
  • Generative AI creates new content; traditional AI just classifies/predicts.
  • AI in testing today: draft cases, gen data, convert to code, explain logs, summarize failures.
  • AI-generated tests are strong drafts but weak on edge cases and your business rules.
  • AI-assisted testing = AI helps, human decides (assistant, not replacement).
  • Validate AI tests: trace to requirements, check against the real app, then run them.
  • Full reliance on AI risks hallucinated tests, missed edge cases, and false confidence.
  • Prompt = your instruction to the AI; be specific.
  • Prompt quality controls output quality — garbage in, garbage out.
  • Use ChatGPT/Claude/Gemini/Copilot to draft, generate, explain, and convert — always review.
  • AI automates repetitive drafting; humans keep strategy, prioritization, and sign-off.
  • Productivity win: let AI do the first draft, you verify and add the risky cases.