Looking for a job

Sumaira Bashir

@sumairabashir

Message

AI evaluation and QA engineer at Source Gate Limited who measures RAG quality against Python-built ground truth.

Oman

Message

At Source Gate Limited, I test a multi-tenant AI platform and evaluate its RAG features against expected answers and independent ground truth I build from source documents in Python. I use citation and debug traces to distinguish retrieval faults from answer-construction faults, and track groundedness, hallucinations and run-to-run variance.

Before focusing on AI evaluation, I worked in software QA at Bentley Systems, Avenir Technologies and Teralight. At Bentley Systems, I led the Framework Certification QA team and tested AutoPLANT applications; I’ve also built Playwright automation projects with TypeScript and GitHub Actions.

Experience

Work history, roles, and key accomplishments

Current

Software Quality Assurance Engineer

Current

Source Gate Limited

Aug 2026 - Present (3 months)

Tested a multi-tenant AI platform end to end, designed evaluation sets with expected answers, built ground truth in Python, and isolated retrieval vs generation faults. Automated evaluation runs with Playwright and REST APIs.

RAG Evaluation LLM Evaluation Python Playwright REST APIs Testing Automation Ground Truth Dataset Creation Retrieval Fault Isolation Run to Run Variance GitHub Actions

Software Quality Analyst II

Bentley Systems

Nov 2008 - Jan 2013 (4 years 2 months)

Led the Framework Certification QA team, performed functional, system, integration, performance and regression testing for AutoPLANT applications. Authored test plans, managed sprint backlog, and trained new hires.

Functional Testing Regression Testing Integration Testing Performance Testing Test Planning Defect Tracking Scrum VersionOne PVCS Tracker VMWare

Quality Assurance Engineer

Avenir Technologies

Oct 2007 - Nov 2008 (1 year 1 month)

Developed QA plans, test cases and error/testing reports based on black-box testing techniques. Mapped customer requirements to product functionality and authored system overview documents and user guides.

Black Box Testing Test Case Design Requirement Mapping Documentation Bugzilla RoboHelp

Technical Specification Writer & Associate QA Analyst

Teralight

Feb 2006 - Oct 2007 (1 year 8 months)

Acted as primary QA resource for department supervisors, handling issue identification and reporting. Produced technical specifications, user manuals and product description reports.

Technical Writer QA Issue Identification Process Improvement