Project Portfolio — Interview Prep (QA / SDET / Automation)¶
This folder contains deep-dive interview notes for six projects. Each file has: an elevator pitch, the problem it solves, the tech stack and why, an architecture walkthrough, the end-to-end flow, key components, and 15–20 likely interview Q&A with strong answers, follow-ups, and honest "what I'd improve" responses.
How to use this: Read the pitch + architecture out loud until you can deliver them without notes. For each project, be able to draw the architecture on a whiteboard in 60 seconds. Memorize 2–3 "challenge / STAR" stories per project — those win interviews more than feature lists.
The six projects at a glance¶
| # | Project | What it is | Core stack | Your role / story |
|---|---|---|---|---|
| 1 | bkg-chat-eval | Automated evaluation framework for an LLM analytics chatbot (telecom/5G tower-deployment domain) | Python, pytest, OpenAI/Anthropic, PostgreSQL, Langfuse, Streamlit | Built a "test harness for AI" — grading non-deterministic LLM answers |
| 2 | FMCG-chat-evaluation | Same eval framework applied to an FMCG analytics chatbot (SimCo sales/distribution data) | Python, pytest, WebSocket, PostgreSQL, sqlglot, Langfuse | LLM-as-judge + agentic "oracle" verification against a live DB |
| 3 | hoad-ai-automation | Same eval framework for a retail-analytics chatbot (House of Anita Dongre fashion replenishment) | Python, pytest, Playwright, WebSocket, OpenAI/Anthropic, Langfuse | Multi-channel (API + UI) eval, RAG/BKG hybrid retrieval |
| 4 | Morrie automation | E2E + API test automation for an AI-agent platform (questt.ai) | Playwright + TypeScript, Axios, Jenkins | Page Object Model, custom fixtures, CI pipeline |
| 5 | VAPT | Web app security testing (Vulnerability Assessment & Penetration Testing) of 4+ apps | OWASP ZAP, Nikto, Nmap, Burp Suite | Authenticated DAST + manual testing, OWASP Top 10, remediation retests |
| 6 | B2BProjectTest | UI + API regression/smoke automation for a B2B e-commerce platform (Avysh) | Java 8, Selenium 4, TestNG, REST Assured, ExtentReports, Maven | POM + helper-layer framework, data-driven, REST Assured API tests |
Your 60-second self-introduction (sample)¶
"I'm a QA/automation engineer with a range from classic UI/API test frameworks to security testing and, most recently, evaluation frameworks for AI products. On the automation side I've built a Java/Selenium/TestNG/REST Assured framework for a B2B e-commerce platform and a Playwright + TypeScript suite for an AI-agent product, both with Page Object Model, data-driven tests, and CI pipelines. On the security side I run VAPT engagements with OWASP ZAP, Nikto, Nmap and Burp Suite, mapping findings to the OWASP Top 10 and verifying fixes on retests. Most recently I built a Python framework that tests our LLM chatbot itself — because you can't
assertEqualsa non-deterministic answer, so I use an LLM-as-judge plus an agentic 'oracle' that re-runs the canonical business logic against the live database to fact-check the bot's numbers. The thread through all of it is the same QA mindset: define what 'correct' means, automate the check, gate the release, and track regressions over time."
Cross-cutting themes interviewers love (use these as bridges)¶
- "What does correct mean?" — In every project you had to define the oracle: for B2B it's expected field values; for API it's status codes + JSON paths; for security it's "no High/Medium findings"; for AI it's a multi-dimensional rubric because there's no single right string. Showing you think about the definition of done is senior-level signal.
- Determinism vs. non-determinism. Traditional automation assumes a deterministic system; LLM eval forced you to handle non-determinism (tolerances, semantic matching, multi-run consistency, temperature=0). Great contrast story.
- Flakiness & reliability. Waits/auto-waiting in Playwright, Angular sync in Selenium, retry analyzers, rate-limit handling — be ready to talk about why tests flake and how you stabilized them.
- CI/CD & shift-left. Jenkins pipeline (Morrie), GitHub Actions (the eval projects), pre-commit hooks, lint+type-check+test gates.
- Regression tracking. The eval projects persist run history to SQLite and compare runs/trends — frame this as the same instinct as a regression suite.
- Honest tech-debt awareness. Each file lists real weaknesses (no BaseTest, hardcoded creds, hard waits, stale docs). Interviewers respect "here's what I'd refactor next" far more than "it's perfect."
Quick glossary (so you never stumble on a term)¶
- POM (Page Object Model): design pattern that wraps each page's locators + actions in a class, so tests read like business steps and locator changes live in one place.
- LLM-as-judge: using a large language model (with a strict rubric, temperature 0) to score another model's output on quality dimensions.
- BKG (Business Knowledge Graph): a JSON export of the chatbot's domain model — entities, KPIs, table/column mappings, and executable reference functions for each metric. Used both as context for judges and as the ground-truth "oracle."
- Oracle (in testing): the source of truth you compare actual output against. In the eval projects, the oracle is the BKG's reference logic re-run against the live DB.
- DAST: Dynamic Application Security Testing — scanning a running app (ZAP). vs SAST = static, source-code scanning.
- Langfuse: an LLM observability/tracing platform; here it stores the chatbot's tool-call traces (read) and the eval's scores (write).
- Regression test / regression store: re-running checks to confirm new changes didn't break old behavior; the eval projects keep a SQLite history to detect score regressions across runs.