Skip to content

Interview Prep — Rohan Dsouza (SDET-2)

Simple, easy-to-speak answers based on the updated resume. 5.5+ years | SDET-2 at Questt | API + UI + AI/LLM + CI/CD

1. Self Introduction (Full)

Good morning, and thank you for the chance.

My name is Rohan Dsouza. I am an SDET / QA Automation Engineer with about 5.5 years of experience in web, mobile, and API testing. Out of these, around four years I have spent building test frameworks from scratch.

I did my MCA from NMAMIT, Nitte in 2020.

I started my career at Avysh for about two years. There I learned the basics — writing test cases, doing manual, smoke, and regression testing. I also started automation with Selenium and Java, and did some performance testing with JMeter.

For the last 3.5 years I have been at Questt as an SDET-2. Here my role has grown a lot. I own the QA for an AI-based, multi-tenant SaaS product used by big global clients. It has chatbots, invoice processing, and forecasting tools.

  • I built and still maintain the API tests in Rest Assured, TestNG, and Maven, and the UI and end-to-end tests in Playwright with TypeScript. All of these run in Jenkins, so we catch bugs on every build.
  • I test user roles and access (RBAC) and third-party integrations. I check backend data using SQL in Redash and Python in Jupyter. I also do API security and negative testing based on OWASP.
  • In the last year, I built a Python framework that checks our AI chatbots — whether their answers are correct, based on real data, and safe.

I was also the runner-up in the Questt Hackathon, and the app I worked on won the Google Best App Award in 2022.

So in short, I am strong in API testing, UI testing, AI/LLM testing, and CI/CD, and I like to learn fast and deliver quality work. Thank you.

2. Self Introduction (1 Minute — HR Round)

Hi, I am Rohan Dsouza, an SDET with about 5.5 years of experience in manual and automation testing for web, mobile, and API.

I did my MCA in 2020. I started at Avysh for two years doing manual and Selenium testing.

For the last 3.5 years I am at Questt as an SDET-2, where I own QA for an AI-based SaaS product. I have built API tests in Rest Assured and TestNG, UI tests in Playwright with TypeScript, and run them all in Jenkins. Recently I also built a Python framework that checks AI chatbots for correct and safe answers.

I am a Hackathon runner-up, and our app won the Google Best App Award 2022. I am now looking for a role where I can keep growing in automation and AI testing. Thank you.

3. Self Introduction (Technical Round)

Hi, I am Rohan Dsouza, a QA Automation Engineer with 5.5 years of experience in API, UI, AI testing, and CI/CD.

  • API: I build frameworks from scratch using Rest Assured, TestNG, and Maven. I cover CRUD flows, login, schema checks, and negative/OWASP tests. For manual API testing I use Postman.
  • UI / E2E: I use Playwright with TypeScript — Page Object Model, fixtures, login through saved storage state, parallel runs, and reports.
  • AI / LLM: I built a Python framework that scores chatbot answers and checks the numbers against the real database.
  • CI/CD: I use Jenkins — tests run on every build, on schedule, and on pull requests.
  • Data check: I write SQL in Redash and Python in Jupyter to check backend data.
  • Earlier I used Selenium and JMeter. Databases: MySQL, PostgreSQL, MongoDB.

Memory hook: API → UI → AI → CI/CD → Data → Domain

4. Roles & Responsibilities

As an SDET-2 at Questt, I own full QA for an AI-based SaaS product used by big global clients. My main work is:

  1. Test planning — I plan and write test cases for new features, and decide what to automate and what to test by hand.
  2. Automation — I build and maintain API tests (Rest Assured/TestNG) and UI tests (Playwright/TypeScript).
  3. CI/CD — I keep all tests running in Jenkins, so we catch bugs on every build.
  4. AI testing — I run my Python framework to check if chatbot answers are correct and safe.
  5. Backend & security — I check user roles (RBAC), integrations, and backend data with SQL and Python, and do API security testing.
  6. Teamwork — I work in Agile sprints with developers, product managers, and clients, and track bugs in JIRA.

So my role covers the full testing cycle — planning, manual, automation, and AI quality.

5. A Typical Day (Day-to-Day Work)

My day is a mix of testing, automation, and checking results.

  • Morning: I first check the Jenkins runs from last night. If any test failed, I check if it is a real bug, a flaky test, or an environment issue, and raise it in JIRA. Then I join the daily stand-up with the team.
  • Mid-day: Based on the sprint, I either write test cases for new features, add new automation (API in Rest Assured or UI in Playwright), or do manual testing on AI features that are hard to automate.
  • For our AI products, I run my evaluation framework, see where the chatbot's answers go wrong, and check the numbers against the database using SQL and Python.
  • Afternoon: I do security and negative testing on APIs, check roles and integrations, and work with developers on fixes. Once fixed, I re-test.
  • All through the day, I watch for flaky tests and keep the framework healthy, so the team gets fast and reliable feedback.

So every day is a little different, but the goal is the same: catch bugs early, keep automation stable, and make sure the AI answers we ship are correct and safe.

6. "Why are you looking for a change?"

Stay positive. Never blame the company.

I have had a good 3.5 years at Questt. I grew from API testing into owning QA for an AI product and building our chatbot evaluation framework, which I am proud of.

I am looking for a change mainly for bigger scale and more learning. So far I have built things from scratch, and now I want to work in a bigger or more mature team — larger systems, stronger QA practices, and seniors I can learn from.

I am also very interested in roles that mix automation with AI testing, because that is where I have been learning a lot and where I see the industry going.

So it is not about leaving something — it is about finding a place where I can keep growing.

Short version: "I have grown a lot at Questt, but I want a bigger-scale team where I can go deeper in automation and AI testing and take more ownership."

7. "Walk me through your AI / LLM Evaluation Framework"

Your strongest topic. Speak in 4 simple parts: Problem → Idea → How it works → Result.

The problem: Our product has AI chatbots that answer questions on client data. The risk is that the chatbot can sound confident but give a wrong, made-up, or unsafe answer. Checking this by hand for every client is not possible. So I built a Python framework to check the answers automatically.

The main idea — one code, many clients: The framework is config-driven. The same code works for all our chatbots (telecom, FMCG, retail) by just changing the config. So I don't write a new framework for each client — this keeps it easy to maintain.

How it works: For each answer, the framework gives a score on six quality points — like correctness, is it based on real data, is it complete, and is it safe. For this scoring I use an LLM-as-a-judge.

The best part is the "oracle." Instead of trusting the chatbot's numbers, the oracle runs the real KPI query on the database (PostgreSQL) and compares the real number with the chatbot's number. So I check facts against the real source, not against another AI's guess.

Two problems I solved: 1. Some facts were being missed during checking. I added a two-pass check, which reduced missed facts by about 30–50%. 2. I also checked the KPI SQL itself using a tool called sqlglot plus an LLM judge, and this found real bugs in our own KPI code.

Tools and result: It runs on OpenAI and Anthropic models, and I use Langfuse to trace everything. The result — we can check chatbot quality for many clients from one codebase, catch wrong answers before clients do, and it even found real bugs in our own data code.

If they ask "how do you trust an AI judge?": That is exactly why the oracle exists. For any number, I don't trust the AI judge alone — I check it against the real SQL result from the database. The AI judge only handles language points like grounding and relevance.

8. Strengths & Weaknesses

Strengths (pick 2, with proof):

  1. Building frameworks from scratch. "I have built three frameworks from zero — Rest Assured, Playwright, and the Python AI framework. I design them to be reusable, like the config that lets one codebase serve many clients."
  2. Mixing normal QA with AI testing. "I have strong automation basics — API, UI, CI/CD — and I have also moved into AI testing. I can test both an API and whether a chatbot's answer is correct. This mix is still rare."

Weakness (real, with a fix):

"Earlier, I used to build big automation before checking if it was the most important coverage — I would over-engineer tests that were not the top risk. I fixed this by first making a risk-based plan and talking to the PM about priorities. Now my effort goes where it matters most."

Other weakness (if you want a technical one):

"My performance testing is basic — I have used JMeter for simple load checks, but not for big-scale performance work. I know this gap, so I am reading more about it and would like a role where I can build this skill."

9. Quick Memory Hooks

  • Self Intro flow: Name + experience → Education → Avysh → Questt → Recent AI work + Awards
  • Technical flow: API → UI → AI → CI/CD → Data → Domain
  • Framework flow: Problem → One code (config) → Six scores + Oracle → Two fixes → Result
  • Why change: Bigger scale + AI focus + more ownership (stay positive)