Arrowhead — R1 Interview Prep (Your Answers)¶

Round: R1 (Profile / Projects / Hands-on) · Time: 6:30 PM Format: Discussion — profile accuracy, tool knowledge, project understanding, practical implementation.

How to use this: read each answer out loud 2–3 times so it sounds like you, not a script. Keep answers to 45–90 seconds. Lead with the headline, then one concrete detail, then the impact. Don't over-claim — the honest "here's what I'd improve" lines below win trust.

0. 60-second self-introduction (say this first)¶

"I'm Rohan, a QA / SDET with about [X] years of experience, currently working as an SDET-2 at Questt. My background spans three areas. First, classic UI and API automation — I've built a Java + Selenium + TestNG + REST Assured framework for a B2B e-commerce platform, and a Playwright + TypeScript suite for our AI-agent product, both using Page Object Model, data-driven tests, and CI pipelines. Second, security testing — I run VAPT engagements with OWASP ZAP, Nikto, Nmap and Burp Suite, mapping findings to the OWASP Top 10 and verifying fixes on retest. And most recently, AI/LLM testing — I built a Python framework that tests our LLM chatbot itself, because you can't assertEquals a non-deterministic answer, so I use an LLM-as-judge plus an agentic 'oracle' that re-runs the business logic against the live database to fact-check the bot's numbers. The common thread through all of it is the same QA mindset: define what 'correct' means, automate the check, gate the release, and track regressions over time."

Adjust [X] years to your real number before the call.

1. Previous roles & responsibilities¶

Answer:

"At Questt I work as an SDET-2. My day-to-day covers the full test lifecycle — I own test strategy and test-case design for the features in my area, build and maintain the automation frameworks (UI, API, and our AI-eval framework), and I'm hands-on in triage: reproducing defects, doing root-cause analysis, and driving them to closure with the dev team. I also maintain our CI pipelines so the suites run on every build, and I do release sign-off. On top of functional work I take on security testing (VAPT) for our apps. So my role is a mix of individual-contributor automation, quality gatekeeping for releases, and cross-functional defect investigation."

Follow-up — "manual or automation?"

"Both. I design manual/exploratory test scenarios first — that's where I find the interesting edge cases and negative paths — then I automate the stable, high-value regression paths. I don't automate blindly; I automate what's repeatable and high-risk."

Follow-up — "Agile process?"

"Yes — sprint-based. I get involved early in refinement so I can raise testability and edge-case questions before code is written (shift-left), I write test cases against acceptance criteria, and I keep the regression suite green as part of the definition of done."

2. Projects & your contributions¶

Keep a short list ready and let them pick which to go deep on. One line each:

AI/LLM Evaluation Framework (Python) — automated evaluation of our LLM analytics chatbots across telecom, FMCG, and retail domains. LLM-as-judge + an agentic "oracle" that verifies the bot's numbers against a live Postgres DB. Stack: Python, pytest, OpenAI/Anthropic, PostgreSQL, sqlglot, Langfuse.
Questt REST Assured API framework (Java) — REST Assured + TestNG + Maven; ~60 tests across API versions v1–v4, a helper/service layer with GET/POST/PATCH wrappers, and 19+ JSON-schema validators.
Morrie / Playwright automation (TypeScript) — E2E + API automation for our AI-agent product; Page Object Model, custom fixtures, Axios for API calls, Jenkins CI.
Avysh B2B — Selenium framework (Java) — UI + API regression/smoke for a B2B e-commerce platform; Selenium 4
TestNG + REST Assured + ExtentReports + Maven, POM with a helper/facade layer, data-driven.
VAPT — security testing of multiple web apps with ZAP/Nikto/Nmap/Burp, OWASP Top 10 mapping, remediation retests.

My contribution line (say this):

"In each of these I was the one who designed the framework structure and wrote the tests — not just executing someone else's suite. For the AI-eval work I designed the scoring rubric and the oracle logic from scratch."

3. Tools / frameworks (be ready to go deep on any)¶

Have a crisp 2–3 sentence answer for each. Table = quick recall; talk in full sentences.

Tool	What you say
Selenium (Java)	"Selenium 4 + TestNG, Page Object Model with a helper/facade layer so tests read like business steps. I use explicit waits (WebDriverWait / ExpectedConditions), never Thread.sleep, and TestNG for grouping, priorities, data providers, and a retry analyzer for flaky cases."
Playwright (TypeScript)	"Playwright + TS for our AI-agent product. I lean on its auto-waiting, use role/text/test-id locators over brittle CSS/XPath, custom fixtures for setup, and storage-state to reuse login. Traces and screenshots on failure for debugging."
TestNG	"Test lifecycle annotations, groups (smoke/regression), priorities, parameterization via @DataProvider, parallel execution via the XML suite, and IRetryAnalyzer for known-flaky tests."
Pytest	"I use it for the AI-eval framework — fixtures for setup/teardown, parametrize for data-driven cases, markers to slice suites, and conftest for shared config. Assertions read cleanly and the plugin ecosystem is strong."
REST Assured	"API automation in Java — given/when/then style. I validate status codes, headers, and response bodies with JSONPath, and I do JSON-schema validation to catch contract changes. Built a service/helper layer so auth and base config aren't repeated."
SQL	"I write SQL to set up and verify test data and to validate what the app actually persisted — the DB is my source of truth. In the AI-eval framework I even re-run the business logic as SQL against Postgres to fact-check the chatbot's answers."
Jenkins / CI-CD	"Jenkins runs my Playwright suite on each build/PR; the eval projects run on GitHub Actions. Pipelines do install → lint/type-check → test → publish report, and fail the build on regressions so nothing broken merges."
Reporting	"ExtentReports for the Java suites (with failure screenshots), Playwright HTML report + traces, and Langfuse for the AI-eval scores/traces."

If asked "Selenium vs Playwright?":

"Playwright has auto-waiting built in, no separate WebDriver, faster and more stable out of the box, and tracing/ video/screenshots are first-class — so it's less flaky with less effort. Selenium is more mature, has the biggest ecosystem and language support, and is often what's already in place in enterprise stacks. I'm comfortable in both; I pick based on what the team already runs."

4. Automation scenarios you implemented¶

Pick 2 concrete ones and tell them like mini-stories. Below is a strong pair, including the role-based Dashboard scenarios from your screenshot (great because it shows functional + security thinking).

Scenario A — Role-based Dashboard testing (super_admin vs Distributor)¶

"One feature I tested was a role-based Dashboard where different roles — super_admin and Distributor — see different content. I designed both positive and negative scenarios: - Content correctness per role: after a successful login I verified the Dashboard shows the right content for super_admin, and separately for a Distributor — the two roles should not see the same data. - Static links / footer: clicked the Privacy Policy and Terms & Conditions links at the bottom of the page and verified each redirects to the correct page. - Notifications: verified the notification bell shows a red-dot only when there's a new notification. - Card layout consistency: each card renders with a unique logo but a consistent card size, and clicking a card routes to the correct detail page. - The important one — authorization / broken access control: as a Distributor I tried to reach the Users section directly by editing the URL. A Distributor shouldn't have access, so the expected result is a redirect or 'access denied' — not the page loading. That's a classic Broken Access Control check (OWASP Top 10), and it's exactly the kind of negative test that catches real security bugs, not just UI bugs."

Why this answer lands: it shows you test functional correctness, UI consistency, AND authorization — and you naturally connect it to OWASP, which ties into your VAPT background.

How you'd automate it (if they ask):

"In Playwright I'd use storage-state to log in once per role, then run the same spec against each role via a fixture/parameter. For the URL-tamper test I'd navigate directly to /users as the Distributor session and assert the response/redirect, not the element. For the footer links I'd assert the resulting URL. For the red-dot I'd assert the badge is visible only when a notification exists — driving that state through the API or a seeded fixture."

Scenario B — API contract regression (REST Assured / Playwright API)¶

"For our APIs I built a data-driven suite that validates status codes, headers, and response bodies with JSONPath, plus JSON-schema validation so a contract change — a renamed or dropped field — fails the build immediately instead of silently breaking a consumer. I chained tests where needed: create a resource via POST, capture the ID, then GET/ PATCH/DELETE it and assert each step. That schema-validation layer caught real breaking changes before release."

5. RCA / defect investigation examples¶

Use STAR (Situation, Task, Action, Result). Two ready-made stories:

RCA Story 1 — Flaky test that was actually a real bug¶

S: A UI test intermittently failed on a data table that sometimes showed stale values. T: Everyone assumed it was a flaky wait; I had to prove root cause before we suppressed it. A: I added tracing/logging, correlated the failures with the API response timing, and checked the DB directly with SQL. The UI was rendering before a background refresh completed — a race condition in the app, not the test. R: I filed it with the trace + DB evidence, dev fixed the refresh sequencing, and I replaced the hard wait with a proper condition-based wait. The 'flaky' test became a reliable regression guard. Lesson: don't mute a flaky test until you've ruled out a real defect."

RCA Story 2 — Wrong number from the LLM chatbot (AI-eval)¶

S: Our analytics chatbot occasionally returned a confident but wrong figure. T: I needed an automated way to know when the answer was factually wrong, not just badly worded. A: I built an 'oracle' — I re-ran the canonical business logic as SQL against the live Postgres DB and compared it to the bot's number within a tolerance, alongside an LLM-as-judge for the qualitative parts. When they diverged, I traced the bot's tool calls in Langfuse to find where it picked the wrong table/filter. R: RCA showed the bot was joining on the wrong dimension for one KPI. We fixed the prompt/tool mapping, and the eval suite now catches that class of regression automatically before release."

One-liner on your RCA method (if they ask "how do you approach RCA?"):

"Reproduce reliably, isolate the layer — UI, API, or DB — with evidence at each layer (logs, traces, SQL), find the first point where actual diverges from expected, then confirm the fix with a regression test so it can't come back."

6. AI / LLM testing experience (your differentiator — lead with it if they're interested)¶

Core answer:

"The hard part of testing an LLM is that the output is non-deterministic — you can't assert one exact string. So I reframed 'correct' as a rubric plus a fact-check. I use two mechanisms: 1. LLM-as-judge — a model with a strict rubric at temperature 0 scores the answer on dimensions like factual accuracy, relevance, and completeness, so the grading is consistent and repeatable. 2. An agentic oracle — for the numbers, I don't trust another model; I re-run the actual business logic as SQL against the live database and compare the bot's figure to ground truth within a tolerance. I persist every run's scores to a store and compare across runs, so it behaves like a regression suite for AI — if a prompt or model change drops accuracy, we see the trend. I use Langfuse to trace the bot's tool calls, which is how I do RCA when an answer is wrong. I applied this same framework across three domains — telecom, FMCG, and retail."

Terms to be ready to define crisply: - LLM-as-judge: a model scoring another model's output against a rubric, temp 0 for consistency. - Oracle: the source of truth you compare against — here, the business logic re-run against the DB. - Non-determinism handling: tolerances for numbers, semantic matching for text, multiple runs for consistency. - RAG / embeddings, Langfuse, sqlglot — mention only if asked; don't over-reach.

Honest boundary (say this if pushed):

"I'm strong on the evaluation and testing side of LLMs — I'm not the one training models. My value is treating an AI feature like any other system under test: define correctness, automate the check, catch regressions."

7. Likely quick-fire questions — one-liners¶

"How do you handle flaky tests?" → "Prefer auto-waiting / explicit condition-based waits over sleeps, stable locators (role/test-id), isolate test data so tests don't collide, retry only known-transient cases, and always investigate whether the flake is actually a real bug first."
"How do you decide what to automate?" → "High-value, repeatable, stable regression paths first; leave rapidly- changing UI and one-off exploratory checks manual. ROI-driven, not automate-everything."
"Page Object Model — why?" → "Locators + actions live in one class per page, so tests read like business steps and a UI change is a one-line fix in one place instead of across every test."
"How do you test an API?" → "Validate status, headers, and body with JSONPath; JSON-schema validation for contracts; positive + negative + boundary cases; chain CRUD flows; run it in CI."
"How do you use SQL in testing?" → "Seed and clean test data, and verify the app persisted what it should — the DB is my source of truth, especially for backend/data validation."
"CI/CD?" → "Jenkins/GitHub Actions run install → lint/type-check → tests → report on every build/PR, and fail the build on regressions so broken code doesn't merge."
"Biggest strength?" → "I define what 'correct' means before I automate — I've done it for UI, APIs, security, and even non-deterministic AI. That framing is what makes my suites actually catch bugs."

8. Questions to ask THEM (always have 2–3)¶

"What does the current test stack look like — Selenium, Playwright, or a mix — and where do you want to take it?"
"Is this role more feature-testing, framework-building, or a mix? How much manual vs automation?"
"Are you starting to test any AI/LLM features, and would that eval experience be useful here?"
"What does the CI/CD and release process look like today?"

9. Final checklist before the call¶

[ ] Fill in your real years of experience in the intro.
[ ] Have your resume open — every tool you name, they can drill into; know each project on it.
[ ] Pick your top 2 projects to go deep on (suggest: AI-eval + one of Selenium/Playwright).
[ ] Rehearse the 60-sec intro and 2 STAR stories out loud.
[ ] Accept the calendar invite.
[ ] Test camera/mic, quiet room, water nearby, join 3–4 min early.
[ ] Be honest about boundaries — "I'd approach it like this" beats pretending to know.

All the best, Rohan. You've got real depth here — talk like the engineer who built these, not a candidate reciting features.