# Asif Shahid

Asif Shahid is a remote worker on Himalayas, Founder and Principal Engineer at KwikQuery LLC, based in United States.

Source: https://himalayas.app/@asifshahid

```text
Principal Engineer at KwikQuery building TabbyDB, an Apache Spark fork with 35% TPC-DS benchmark improvement.
```

| Detail | Value |
| --- | --- |
| Availability | Looking for a job |
| Location | United States |
| Authorized to work in | United States |
| Salary expectations | 115k+ USD |
| Job categories | Software Engineer |
| Skills | Apache Spark, Spark SQL, Catalyst Optimization, Distributed Systems, Distributed Databases, Query Engine, SQL, Apache Iceberg, Java, Scala, Low Latency Systems, Concurrent Programming, Lock Free Programming, Query Optimization, Constraint Propagation |
| Website | https://kwikquery.com |

## About

```text
At KwikQuery LLC, I started the TabbyDB venture and developed an Apache Spark fork that addresses complex query compilation issues and improves runtime performance with dynamic file pruning for Broadcast Hash Joins. Internal TPC-DS testing on 1 TB and 3 TB datasets showed 35% improvement compared with stock Spark.

At Workday Inc, I implemented a constraint propagation algorithm for Spark’s Catalyst optimizer that reduced compilation time by a factor of 10–100 in problematic cases. I also proposed an improvement to open-source Spark and presented the solution at Databricks Spark Summit 2021.

Across my work at Cloudera Inc, SnappyData, Pivotal Labs, and GemStone Systems, I contributed to Spark SQL and Catalyst, distributed databases, and query engines. My projects included approximate query processing, Spark and GemFireXD integration, and Java systems using concurrency and low-latency techniques.
```

## Experience

### Founder and Principal Engineer (current)

KwikQuery LLC. Jul 2025 - Present

```text
Founded KwikQuery LLC and developed TabbyDB, a high-performance fork of Apache Spark that addresses complex query compilation issues and improves runtime performance through dynamic file pruning for Broadcast Hash Joins.
```

Skills: Apache-Spark, Catalyst, SQL, Query-Optimization, Java, Scala

### Principal Engineer

[Cloudera Inc](https://himalayas.app/companies/cloudera). Sep 2022 - Apr 2025

```text
Worked on Apache Spark SQL, Core, and Catalyst layers to fix functional and performance issues. Debugged customer environment issues and contributed to Iceberg core and Spark-Iceberg layer.
```

Skills: Apache-Spark, SQL, Catalyst, Iceberg, Debugging, Performance-Optimization

### Principal Engineer

[Workday Inc](https://himalayas.app/companies/workday). Aug 2020 - Sep 2022

```text
Optimized the Catalyst layer of an Apache Spark fork, implementing a new algorithm for Constraint Propagation that speeds up compilation time by 10-100x in problematic cases. Presented the solution at Databricks Spark Summit 2021.
```

Skills: Apache-Spark, Catalyst, Query-Optimization, Java, Scala

### Principal Architect

[Tibco](https://himalayas.app/companies/tibco-software). Mar 2019 - Mar 2020

```text
Worked on computeDB (formerly SnappyData), implementing an optimized HashAggregateExec for Spark using serialized data in a byte-based HashMap. Gained deep knowledge of Spark's code generation architecture.
```

Skills: Apache-Spark, Catalyst, SQL, Java, Scala, Code-Generation

### Staff Engineer

SnappyData. Jun 2017 - Mar 2019

```text
Implemented approximate query processing and error estimation by enhancing the Catalyst querying framework of Apache Spark for aggregate queries on stratified sample tables. Developed a connector for GemFire and implemented policy-based row-level security.
```

Skills: Apache-Spark, Catalyst, SQL, AQP, Gemfire, Java, Scala

### Staff Engineer

Pivotal Labs. Sep 2015 - Jun 2017

```text
Worked on a new generation OLAP/OLTP SQL-based database on top of Apache Spark, which became SnappyData.
```

Skills: Apache-Spark, SQL, Java, Scala

### Sr. Java Backend Developer

eBuddy. Sep 2011 - Nov 2012

```text
Enhanced eBuddy chat performance to be 6 times faster through optimal synchronization and efficient byte data handling. Reduced memory footprint of image data by 50% and worked on Android.
```

Skills: Java, Android, Performance-Optimization, Synchronization

### Manager R & D

GemStone Systems. Sep 2004 - Sep 2011

```text
Architected design, managed team, and implemented core Java code for GemFire. Implemented JTA spec for Transaction Manager, enhanced Object Query Engine with intelligent index usage, and improved disk-based persistence performance using Oplogging.
```

Skills: Java, Gemfire, JTA, OQL, SQLFire, Distributed-Systems

### Team Leader

[eGain Corporation](https://himalayas.app/companies/egain-corporation). Sep 2000 - Sep 2004

```text
Led design and development of a collaboration proxy server for co-navigation, involving HTTP protocol, cookie management, and SSL. Implemented pushlet mechanism for asynchronous messaging and designed a chat server.
```

Skills: Java, HTTP, SSL, Pushlet, Web-Technologies

### Sr. Engineer Tech Support

Onward Technologies Ltd. Jan 1999 - Dec 1999

```text
Learned the PIMS system of Aspen Tech, a linear programming package for identifying refinery throughputs and yields based on profit maximization functions.
```

Skills: PIMS, Linear-programming, Refinery-Operations

### Production Engineer

Indian Oil Corporation. Jan 1997 - Jan 1999

```text
Handled the Feed Preparation Unit (FPU) providing feed to the Fluidized Catalytic Cracking Unit (FCC). Oversaw functioning of distillation columns and furnaces.
```

Skills: FPU, FCC, Distillation, Furnaces

## Education

### Indian Institute of Technology Bombay

B. Tech, Chemical Engineering. 1993 - 1997. Grade: 7.57

```text
Pursued a Bachelor of Technology in Chemical Engineering from 1993 to 1997, achieving a grade of 7.57.
```

## Tech stack

- Apache Spark
- Java
- C++
- C
- Scala
- SQL
- Apache Iceberg
- Android

To contact Asif, message them from this profile with a Himalayas employer account: [Hire on Himalayas](https://himalayas.app/hire).

## People also viewed

- [Rohit Kotecha](https://himalayas.app/@rohitkotecha): Research Assistant, Lucas School of Business SJSU
- [Michael Livingston](https://himalayas.app/@michaellivingston3): Big Data & Senior Software Engineer, AAA Auto Club Enterprises
- [raju vadali](https://himalayas.app/@rajuvadali): Vice President – Global Adjustment Inventory & Analytics (GAI) / Data Engineer, Citi Bank
- [Sai P](https://himalayas.app/@saip): Software Engineer, Netflix
- [Orhan Kislal](https://himalayas.app/@orhankislal): Founder / Principal Engineer, QueryScale AI
- [Yagnika Kancherla](https://himalayas.app/@yagnikakancherla): AI/Data Engineer, State of Ohio
- [Shalabh Raj](https://himalayas.app/@shalabhraj): Application Software Consultant, Intuit
- [Vonny User](https://himalayas.app/@vonnyuser): Staff Software Engineer, Magic Leap
- [Siddhant Chaturvedi](https://himalayas.app/@siddhantchaturvedi): Senior Software Engineer, Snowflake, Inc.
- [Shikhar Kapoor](https://himalayas.app/@shikharkapoor): ML Engineer, University of Alabama at Birmingham
