# Leaky Barak

Leaky Barak is a remote worker on Himalayas, AI Evaluation Analyst / Machine Learning Engineer at Defined.ai, based in Kenya.

Source: https://himalayas.app/@leakybarak1

```text
AI Evaluation Analyst at Defined.ai who evaluates LLMs and builds frameworks for consistent assessment.
```

| Detail | Value |
| --- | --- |
| Availability | Looking for a job |
| Location | Kenya |
| Authorized to work in | Kenya |
| Salary expectations | 60k+ USD |
| Job categories | AI Evaluation Analyst, Machine Learning Engineer, AI Evaluation Engineer, AI ML Evaluation Specialist |
| Skills | Large Language Model (LLM) Evaluation, AI Benchmarking, Model Validation, Evaluation Framework Design, Error Analysis, Data Quality Assessment, Annotation Guidelines Development, AI Analysis, Human Feedback Evaluation, Statistical Analysis, Benchmarking and Testing, Quality Assurance, Natural Language Processing, Generative AI, Human In The Loop (HITL) Systems |

## About

```text
At Defined.ai, I evaluate large language models and design frameworks, annotation guidelines, and scoring rubrics for consistent assessment. I analyze model responses for factual errors, reasoning failures, and other weaknesses, then share recommendations to support refinement.

Previously, I developed machine learning data workflows at AfterQuery and evaluated AI outputs at Scale AI and Appen. Across these roles, I’ve benchmarked models, reviewed dataset quality, and contributed to clearer evaluation standards and more reliable AI systems.
```

## Experience

### AI Evaluation Analyst / Machine Learning Engineer (current)

[Defined.ai](https://himalayas.app/companies/defined-ai). Jun 2025 - Present

```text
Evaluates large language models and AI systems across reasoning, factual accuracy, instruction following, and response quality. Designs evaluation frameworks, annotation guidelines, and scoring rubrics, and performs benchmark testing and error analysis.
```

Skills: LLM-Evaluation, AI-Benchmarking, Model-Validation, AI-Analysis, Error-Analysis, Evaluation-Framework-Design, Annotation-Guidelines, Python, Pandas, NumPy, scikit-learn, PyTorch, TensorFlow, Statistical-Analysis, Data-Visualization

### Machine Learning Engineer

AfterQuery. Jan 2023 - Dec 2024

```text
Developed Python-based workflows for preparing, processing, validating, and analyzing machine learning datasets. Built data pipelines, conducted model comparison experiments using metrics like accuracy, precision, recall, and F1-score, and applied statistical analysis to identify model strengths and weaknesses.
```

Skills: Python, Pandas, NumPy, scikit-learn, Data-Pipelines, Model-Evaluation, Statistical-Analysis, Data-Processing, Quality-Assurance

### AI Evaluation Engineer

[Scale AI](https://himalayas.app/companies/scale). Jan 2021 - Dec 2023

```text
Evaluated AI-generated responses and machine learning outputs using structured quality criteria. Designed and maintained benchmark tasks, performed qualitative and quantitative analysis, and created annotation guidelines to improve evaluation consistency.
```

Skills: AI-Evaluation, Benchmarking, Annotation-Guidelines, Python, Data-Analysis, Model-Comparison, Quality-Assurance

### Machine Learning Evaluation Specialist

[Appen](https://himalayas.app/companies/appen). Jan 2019 - Dec 2021

```text
Evaluated AI-generated outputs for accuracy, relevance, and quality. Reviewed and annotated large-scale datasets, performed data quality reviews, and contributed feedback to improve annotation workflows and labeling guidelines.
```

Skills: Data-Annotation, Data-Quality, AI-Evaluation, Quality-Assurance, Human-In-The-Loop, Python

## Education

### United States International University – Africa (USIU-Africa)

Bachelor of Science, Artificial Intelligence. 2017 - 2021

```text
Completed a comprehensive undergraduate program focused on artificial intelligence, machine learning, computational systems, and software development.
```

### DeepLearning.AI

Machine Learning Specialization, Machine Learning

```text
Completed the Machine Learning Specialization, covering supervised and unsupervised learning, model evaluation, and validation methods.
```

### DeepLearning.AI

Generative AI with Large Language Models, Generative AI

```text
Completed the Generative AI with Large Language Models course, covering LLM fundamentals, prompt engineering, and evaluation techniques.
```

## Tech stack

- NumPy
- Pandas
- Python
- TensorFlow
- PyTorch
- scikit-learn

To contact Leaky, message them from this profile with a Himalayas employer account: [Hire on Himalayas](https://himalayas.app/hire).

## People also viewed

- [Frankline Mutuma](https://himalayas.app/@franklinemutuma): AI Rater / Evaluator, Telus International
- [John Wangechi](https://himalayas.app/@johnwangechi): AI Evaluation Specialist, Micro1
- [Fredrick Maina](https://himalayas.app/@fredrickmaina1): AI Data Annotation & Evaluation, Freelance
- [RICHARD MUMBA](https://himalayas.app/@richardmumba): AI Evaluation & Data Annotation Specialist, Mercor
- [Patrick User](https://himalayas.app/@patrickuser21): Senior AI Content Evaluator & RLHF Specialist, Outlier / Scale AI
- [Peter Kigen](https://himalayas.app/@peterkigen): AI Data Annotation & Evaluation Specialist, Unknown
- [Fabian Munene](https://himalayas.app/@fabianmunene): Senior AI Quality & LLM Evaluation Engineer, Tavily AI
- [Josbrian Udoto](https://himalayas.app/@josbrianudoto): Gen AI Model Trainer / AI Evaluation Specialist, OneForma
- [Miles Mailo](https://himalayas.app/@milesmailo): Remote AI Trainer & Data Evaluator, Independent Contractor
- [MUCHERU PIUS MACHARIA](https://himalayas.app/@mucherupiusmacharia): Senior Data Annotation Specialist, Remote AI Data Projects