Ayan Sau
@ayansau
Generative AI Data Analyst specializing in LLM evaluation, RLHF, and multimodal alignment.
IndiaWhat I'm looking for
I’m a Generative AI Data Analyst with 3+ years of experience driving LLM evaluation and training quality through LLM evaluation, RLHF, and human preference evaluation. I work across text, image, audio, and video to assess accuracy, safety, reasoning, instruction-following, and user preference—turning findings into actionable improvements.
I’ve led QA/QC audits for annotations and model evaluations, performing response ranking and re-evaluation to maintain consistency and inter-annotator agreement. I actively identify hallucinations, bias, misinformation, and policy violations, providing structured feedback that supports model alignment and response reliability.
I also bring strong multilingual localization experience, conducting evaluations across English, Bengali, Hindi, and more to improve cultural relevance and alignment. Alongside GenAI work, I’ve built a technical support foundation and data labeling expertise—supporting high-quality training data for large-scale AI systems while meeting accuracy and productivity targets.
Experience
Work history, roles, and key accomplishments
Evaluated AI-generated content across text, image, audio, and video for accuracy, safety, and overall quality. Compared and ranked model responses, conducted multilingual localization evaluations, and performed QC/QA audits including hallucination and policy-violation checks.
Evaluated AI-generated Bengali responses for accuracy, grammar, and cultural relevance, and provided rewritten improvements. Created Bengali prompts and responses to improve conversational datasets while collaborating with a global team on localization quality.
Performed RLHF tasks to improve large language model reasoning, safety, and instruction adherence. Evaluated and ranked responses, wrote prompts across domains, identified model failure patterns, and provided structured feedback for alignment improvement.
Evaluated and compared AI-generated responses across domains for accuracy, reasoning, factuality, and instruction following. Performed response ranking and preference evaluation, conducted multimodal/text/audio evaluation, ran error analysis (hallucinations and safety issues), and completed QA/QC reviews to support multilingual model alignment.
Conducted high-precision data annotation for machine learning applications including sentiment analysis, NER, content moderation, and text classification. Reviewed datasets for consistency, balance, and linguistic accuracy, supported QA/inter-annotator agreement processes, and delivered expert labeling for large-scale client projects.
Trained and fine-tuned large language models using RLHF. Created and improved prompts, evaluated outputs for accuracy/safety/instruction following, and performed annotations and quality checks using tools including Python, OpenAI API, and Hugging Face.
Provided B2B and B2C technical support for Autodesk customers via chat and email, troubleshooting connectivity, serial activation, licensing errors, account verification, and installation issues. Supported Autodesk products (e.g., AutoCAD, Revit, Fusion 360, Navisworks, Inventor, 3ds Max, Maya, Civil 3D, BIM 360) and assisted with licensing management, multi-user network deployments, and complianc
Supported Zomato backend operations by resolving order-related and system-level discrepancies and acting as a key liaison between customer-facing teams and technical backend teams. Used Excel and Google Suite to maintain daily reporting for order tracking and escalation patterns and supported chatbot development through video/image annotation.
Operation and Audit Intern
PlanetSpark
Jan 2023 - May 2023 (4 months)
Evaluated over 10 demo sessions daily against predefined quality parameters and provided feedback to leadership on teacher skills and methodologies. Resolved teacher inquiries via phone and email and handled technical/platform-related concerns while collaborating with trainers and coaches.
Provided end-to-end technical support to Flipkart customers for electronics such as smartphones, laptops, desktops, smartwatches, IoT devices, and related products. Handled voice and chat inquiries, resolved hardware/software setup and usage issues, escalated complex cases to authorized service centers, and documented resolutions in internal CRM tools.
Education
Degrees, certifications, and relevant coursework
NSOU
Bachelor of science, Zoology
2018 - 2021
Grade: 7.4
West Bengal Council of Higher Secondary Education (WBCHSE)
Higher Secondary in Science, Science
Grade: 58.8%
Completed Higher Secondary in Science under WBCHSE, scoring 58.8%.
Tech stack
Software and tools used professionally
Availability
Location
IndiaAuthorized to work in
Social media
Job categories
Skills
Interested in hiring Ayan?
You can contact Ayan and 90k+ other talented remote workers on Himalayas.
Message AyanGet matched with your dream remote job
Sign up now and join over 250,000+ remote workers who receive personalized job alerts, curated job matches, and more for free!
