Available for new opportunities

Spoorthi
Basu

Software Engineer · Distributed Systems · Real-Time Data Infrastructure

I build distributed systems that move data at scale. Real-time pipelines, high-throughput APIs, and the infrastructure that keeps them reliable.

Speaker · Data Streaming Summit 2026 InfoQ Published Author Apache Flink CDC Contributor 100M+ Events / Day
Get in Touch Resume

Indianapolis, IN · Open to relocation

Spoorthi Basu
5+
Years of
experience
100M+
Events processed
per day
99.99%
Uptime
delivered
5K+ RPS
APIs
engineered

What I Build

Real-time Stream Processing

Apache Kafka + Flink pipelines ingesting hundreds of millions of events daily into schema-driven, analytics-ready Iceberg tables on S3.

KafkaFlinkIcebergAvro

Distributed Systems

Fault-tolerant, multi-region Java microservices with circuit breakers, event partitioning, and 99.99% uptime across sustained production workloads.

JavaResilience4jMulti-region

High-Scale API Engineering

Spring Boot REST APIs handling 5K+ RPS with Redis cache sharding that cuts database load by 50%, built for reliability under sustained traffic.

Spring BootRedisREST

Data Infrastructure

End-to-end data platforms: event modeling, schema consolidation, download APIs, and analytics infrastructure that scales to hundreds of millions of events per day.

AWS S3DynamoDBSchema Design

About Me

I'm a Software Engineer with 5+ years building the infrastructure that makes data move: reliably, at scale, and in real time. At Genesys, I've engineered systems processing hundreds of millions of events per day, APIs that handle 5K+ requests per second, and testing infrastructure that went from zero to production-grade in weeks.

My work spans the full data path, from Kafka topics and Flink jobs to Iceberg tables and the REST APIs that serve data to end users. I care deeply about correctness, fault tolerance, and the operational rigor that makes systems boring, which is exactly what production needs.

I recently built the company's first end-to-end data validation framework, now adopted across three teams. I've also published in InfoQ on schema proliferation in Kafka and Flink pipelines, and I'm an active contributor to Apache Flink CDC. This October I'll be speaking at Data Streaming Summit 2026 in San Francisco on why the pipelines behind AI agent memory fail silently.

Currently at
Genesys
Software Engineer
Mar 2021 · Present
Core Stack
JavaApache KafkaApache Flink Apache IcebergAWSRedis Spring BootDockerTerraform

Speaking & Credentials

Upcoming Talk Accepted

Your Agent’s Memory Is Only as Good as the Pipeline Behind It

Data Streaming Summit 2026 Oct 7–8, 2026 San Francisco, CA

I don’t build agents. I build the pipelines that feed them. Durable memory and continuous context are CDC problems wearing new clothes. Kafka into a vector store, Flink into a lakehouse, a change log materialized into something an agent reads. That path breaks without ever tripping an alarm. Exactly-once is a guarantee about processing, not about what ends up in your table. An agent doesn’t crash on wrong memory. It answers confidently, and you never trace it back.

  • Selected through the open Call for Proposals
  • Correctness checks across Kafka, Flink, and Iceberg
Data Streaming Summit

Education

M.S. in Computer Science
Cal Poly Pomona
2018 – 2020 · GPA 3.66
B.S. in Computer Science
Dr. Ambedkar Institute of Technology
2014 – 2018 · GPA 4.0

Publication

Schema Proliferation in Kafka and Flink Pipelines
InfoQ · 2026

Explores schema proliferation in event-driven systems and presents a discriminator-based consolidation approach for scalable Kafka and Flink pipelines.

  • #1 Top Article in InfoQ's weekly Round-Up
  • 550K+ monthly readers
  • Peer-reviewed
Read on InfoQ

Open Source

Apache Flink CDC Contributor
apache/flink-cdc · 2026

Fixed silent data duplication in the Iceberg sink during in-checkpoint schema changes. Also fixed a JobManager OOM on large MySQL CDC tables caused by retained snapshot-split metadata.

  • Merged, shipping in Flink CDC 3.7
  • Streaming-data internals
  • Reviewed by a committer
View Pull Requests

Work Experience

Software Engineer

Genesys · Cloud Contact Center Platform
Mar 2021 · Present
  • Built Flink pipelines turning company-wide Kafka streams into schema-driven Iceberg/S3 datasets at hundreds of millions of events/day.
  • Architected and built the company's first end-to-end data validation framework, catching data issues before customers; adopted by 3 teams.
  • Scaled fault-tolerant Java/Kafka microservices to 10M+ events/day, cutting latency 25%.
  • Engineered customer-facing REST APIs at 5K+ RPS with Redis cache sharding, halving DB load.
  • Stood up org-wide integration testing 0→1 (LocalStack/AWS), cutting test creation to <1 min for 5+ teams.
  • Delivered 99.99% uptime via multi-region deployment and circuit breakers (Resilience4j).
  • Drove 10+ customer-facing features with Product and QA, growing daily active users 25%.
  • Mentored a junior engineer (halved onboarding) and held a 15-min P0/P1 SLA.

Software Engineer

Coding Minds, Inc · Ed-Tech Platform
Jul 2020 · Feb 2021
  • Developed academic system (React/Java/Node.js) deployed on Heroku, serving 500+ daily active users.
  • Built RESTful APIs (Java/Spring) with MySQL CRUD operations, achieving 95% test coverage.
  • Led full SDLC from requirements to deployment using Agile/Scrum, delivering 3 major releases.
  • Enhanced React performance via memoization and lazy loading, improving page load speed by 30%.
  • Implemented automated testing, catching 20+ critical bugs pre-production.

Selected Projects

Featured Project

Kafka Flink Schema Consolidation

Reference implementation for discriminator-based schema consolidation in Kafka and Flink pipelines. Collapses twelve schemas into one consolidated Avro record, enabling single filtered queries over Apache Iceberg on S3. Companion to the InfoQ article on schema proliferation.

Apache FlinkApache KafkaApache IcebergAvroJava
GitHub InfoQ Article
ConsolidatedRide.avsc 
// Discriminator-based union field
{
  "name": "standardRideAttributes",
  "type": ["null", {
    "type": "record",
    "name": "StandardRideAttributes",
    "fields": [
      { "name": "vehicleClass",
        "type": "string" },
      { "name": "surgeMultiplier",
        "type": "double" }
    ]
  }],
  "default": null
}
Research Artifact

MOR Faithfulness

When a CDC stream is materialized by a merge-on-read table, is the “current” row per key actually correct, and can you tell from the physical table alone? A Lean 4 and Mathlib development proves the ordering value must linearly extend logical version order, and that in general you cannot tell. Ships with a read-only checker, a FLINK-38450 reproduction, and an audit-preserving compaction mechanism for Iceberg.

Lean 4Apache IcebergApache HudiDelta LakePython
GitHub
MorFaithful/*.lean 
/-- Def 6. Faithful: the materialized set is
    exactly the current version. -/
def Faithful : Prop :=
  distinct M.Zphys = Finsupp.single M.cur 1

/-- Def 7. LinearExtension: seq strictly
    increases along logical order. -/
def LinearExtension : Prop :=
  ∀ i j, i < j → M.s i < M.s j

/-- Corrected MAIN. -/
theorem prefixFaithful_iff_linear
    (inj : Function.Injective M.d) :
    M.PrefixFaithful ↔ M.LinearExtension :=
  ⟨M.linear_of_prefixFaithful inj,
   M.prefixFaithful_of_linear⟩

-- No `sorry`. 15 theorems audited
-- against the 3 standard axioms.

Kafka Flink Audit Trail

Flink pipeline writing profile change events to an append-only Iceberg table. Old values are rebuilt at query time with LAG and window functions instead of stored in previous-value columns.

FlinkKafkaIcebergSQL

Health Web

Clinic website deployed on AWS enabling patients to find nearby doctors and book appointments. Java backend with MySQL via JDBC.

JavaJSPMySQLAWS

Ecommerce Order Processing

Order microservices using Spring Boot and JPA with PostgreSQL. Swagger docs, Docker containerized, Log4j logging.

Spring BootPostgreSQLDocker

Hair & Skin Segmentation

Deep autoencoder using U-NET for hair/skin segmentation with Keras, tested on Celeb-A dataset with data augmentation and transfer learning.

KerasU-NETNumPy

Safe Driving: Collision Prevention

Demo vehicle using Renesas Microcontroller with ultrasonic sensors. Proximity alerts via CubeSuite++, with AWS storing messages to nearby vehicles.

Embedded CRenesasAWS

Skills & Technologies

Streaming & Data

Apache Kafka Apache Flink Apache Iceberg Avro Schema Registry

Languages

Java Python JavaScript SQL

Cloud & Infrastructure

AWS Docker Terraform Kubernetes CI/CD

Frameworks

Spring Boot Spring Cloud React gRPC

Databases & Caching

DynamoDB Redis PostgreSQL MySQL S3

Testing & Observability

JUnit 5 Mockito Testcontainers LocalStack Sumo Logic New Relic Prometheus

Community & Mentorship

Keys to Success

Cal Poly Pomona · Incoming Graduate Student Welcome

  • Talked 50+ incoming students through my path into Computer Science
  • Focused on the transition from undergraduate to graduate school
Cal Poly Pomona

Judge, Game Gala 2021

K–12 Coding Competition

  • Evaluated 20+ K-12 developers on digital game projects
  • Provided feedback on code quality and game performance
  • Helped select the competition winner
Game Gala 2021