Gollan KurulkarLet’s talk

AI / ML engineer / Israel

There’s usually a bottleneck.

I like finding it.

I’m Gollan. I build and evaluate LLM systems, with a focus on GPU inference, performance, and reliability. My starting point is always the same: measure what’s actually happening.

Explore my work GitHub
From the lab · courseworkExperiment 03 · where the step time went

Four GPUs trained
slower than one.

So I found out why.

≈93.5%mean sync ÷ median step time

NCCL found no InfiniBand device and fell back to TCP sockets.

Proof
Evaluation on the same GPUs scaled 3.84×; only training, which syncs gradients, fell to 0.41×.
Fix
Recommended: expose RDMA to the pods, or keep the job on one NVLink node.

NCCL INFO NET/IB : No device found.NCCL INFO Using network Socket

Read the diagnosis

Rank-0 timings: mean all-reduce per step against median step time. The 0.29 s remainder is derived.

Source · ddp-scaling-h100 / summary.json @ 0e3d153

01 / Selected work

The work, with the workings.

Hands-on coursework in AI systems. Every figure links to the commit it came from.

Experiment 01Time / generation

Where does the generation time go?

5.11×faster autoregressive generation

Tesla T4 · 2-layer Llama · 128 tokens↓ Lower is better

Source · gpu-inference-optimization-lab / 02_autoregressive_decode_optimization.ipynb @ 1630d52

GPU performance · 2026

GPU Inference Optimization Lab

Profiling showed the decode loop was bound by kernel launches, not arithmetic, so I targeted launches: torch.compile reached 2.74×, and CUDA-graph capture of a static-cache decode step reached 5.11×, from 552.6 ms to 108.2 ms per generation.

Method & limits

One recorded run on a randomly initialized 2-layer Llama, fp32 baseline against fp16 optimized. Each time is the mean of three timed generations after one warmup. Results depend on hardware and workload.

Hardware
Tesla T4, one GPU
Workload
64-token prompt, 128 new tokens, batch 1
Runs
Mean of 3 timed generations, 1 warmup
Model
Randomly initialized 2-layer Llama
Read the experiment
Experiment 02Output / second

What do smaller weights buy you?

+49%more output tokens per second

Qwen2.5-7B-Instruct · H100 80 GB · 64 concurrent requests↑ Higher is better

Source · quantized-llm-serving / comparison.json @ dc6d32d

LLM inference · 2026

Quantized LLM Serving

Serving the same model in FP8 raised output throughput from 4,805 to 7,159 tokens per second at 64 concurrent requests and cut logged weight memory by 42.8% (14.29 → 8.17 GiB). In synchronous tests, inter-token latency fell 31%, from 6.11 to 4.20 ms.

Method & limits

30-second synthetic runs on one H100 without warmup. GuideLLM’s mean counts only requests that finished inside the window, which favours the ratio. Synchronous median time to first token rose from 21.59 to 24.51 ms. Answer quality was not evaluated.

Hardware
NVIDIA H100 80 GB, one GPU
Workload
512-token prompts, 256 output tokens, 64 concurrent
Runs
One 30 s run per precision
Metric
GuideLLM output tokens/s, mean over completed requests
Read the experiment
Experiment 03
Diagnosed4-GPU slowdown traced to TCP sockets
Fix: RDMA, or one NVLink node

Distributed training · 2026

Why four GPUs trained slower than one

I provisioned a four-node H100 cluster on Kubernetes, ran matched one- and four-GPU GPT-2 Large jobs, and traced why four GPUs trained at 0.41× the speed of one: gradient synchronization. NCCL found no InfiniBand device and fell back to TCP sockets. Evaluation on the same GPUs scaled 3.84×, which points to the gradient all-reduce. The recommended fix is RDMA for the pods or a single NVLink node.

Method & limits
Hardware
4 nodes × 1 H100 80 GB, Kubernetes
Model
GPT-2 Large, 774M params, from scratch
Runs
500 optimizer steps, timings reported by rank 0
Baseline
Same job spec on one node
Read the scaling study
The tellSpeedup on 4 × H100

Same four GPUs. Evaluation never synchronizes gradients; training does, on every step.

Source · ddp-scaling-h100 / summary.json @ 0e3d153

Further experiments

04—07 / Open a row for details

04LLM Inference ObservabilityReliability engineering · 202698.5 → 3.44 sP95 latency

Load-tested a vLLM-backed agent service at 10 requests per second, instrumented it with Prometheus and Grafana, and tuned it: four API workers, vLLM batching settings, and removing a self-correction loop that had not improved accuracy. P95 latency of successful requests fell from 98.5 s to 3.44 s, timeouts went to zero, and success rose from 32% to 87% over 3,000 requests.

NoteThe baseline figures come from the project report; the committed run file is the final 3,000-request test.

  • Prometheus
  • Grafana
  • vLLM
  • FastAPI
View repositorySource · llm-inference-observability / load_test.json @ 19f27fb
05Coding-Agent Evaluation PipelineMLOps · 20262 of 3tasks resolved · validation run

Built an Airflow pipeline that runs coding agents on SWE-bench Verified in Docker-isolated containers and tracks every run in MLflow. An end-to-end validation run resolved two of three tasks; it validates the pipeline rather than benchmarking a model.

  • Airflow
  • MLflow
  • SWE-bench
  • Docker
View repositorySource · coding-agent-evaluation-pipeline / metrics.json @ 5615995
06Financial Document RAG EvaluationRetrieval evaluation · 20260.23 → 0.34LLM-judged correctness

Built and evaluated a RAG system over 42 SEC filings on 100 FinanceBench questions, testing retrieval, reranking, and prompt policy. The best configuration raised LLM-judged answer correctness from 0.23 to 0.34, and filtering retrieval to the right filing lifted page-hit@1 from 0.21 to 0.29.

NoteFaithfulness was scored on a 20-question subsample with 200-character context excerpts; for the best-correctness configuration it fell from 0.48 to 0.39.

  • LangChain
  • FAISS
  • RAGAS
  • FinanceBench
View repositorySource · financebench-rag / assignment2_improvement_cycles.xlsx @ a1d9296
07Glass-Box PPO on GPT-2Reinforcement learning · 202615.9 vs 6.2reward with vs without critic

Extended a course lab that implements PPO and RLHF by hand on GPT-2 (124M): added verified solutions to its five predict-then-test experiments and an H100 port with bf16 autocast and TF32. Removing the critic cut final reward from 15.9 to 6.2, and a word-count reward showed reward hacking in action: mean reward rose from 0.25 to 15.5 as completions collapsed into repeated “great”.

NoteOne run per setting on the H100. The CPU/MPS notebook reproduces the critic effect with a smaller gap (14.4 vs 8.4).

  • PyTorch
  • Transformers
  • PPO
  • RLHF
View repositorySource · glassbox-ppo-llm / ppo_glassbox_llm_h100.ipynb @ bf1d6e9

02 / Approach

Curiosity first.
Evidence next.

I connect low-level performance with the reliability of the whole system. A faster result is useful when I can explain why it happened.

  1. 01
    Profile the bottleneckExp 03 · NCCL log: Using network Socket
  2. 02
    Change one leverExp 02 · same harness, BF16 → FP8
  3. 03
    Verify the resultExp 01 · fp32 loop checked for equal output before timing

  • 01GPU & LLM performance

    Benchmarking and optimization across model serving, autoregressive decoding, distributed training, quantization, and GPU execution.

    • PyTorch
    • CUDA Graphs
    • vLLM
    • H100
    • NCCL

  • 02Evaluation & reliability

    Controlled experiments that find the bottleneck, quantify the tradeoffs, and explain why a result happened.

    • MLflow
    • RAGAS
    • SWE-bench
    • GuideLLM
    • Langfuse

  • 03Applied LLM systems

    Retrieval, agent, and inference services designed around measurable quality, typed interfaces, and reproducible behavior.

    • LangGraph
    • FastAPI
    • FAISS
    • FastMCP
    • Pydantic

  • 04MLOps & experiment systems

    Repeatable evaluation workflows with orchestration, experiment tracking, containers, tests, and CI-ready project structure.

    • Airflow
    • MLflow
    • Docker
    • GitHub Actions
    • pytest

03 / A little context

Before model pipelines,
production lines.

I investigate manufacturing processes for a living. That habit of tracing a problem back to its source also shapes how I work with AI.

My professional background is in process engineering and data analysis; the AI / ML work here comes from hands-on coursework. Whether it’s an assembly line or a decode loop, I want to understand what the system is actually doing.

More about my background
Gollan.Engineer. Persistent question-asker.

Professional experience

  1. 2024 — Present

    Process Engineer

    UNI — Sophisticated Electronic Assembling

    Investigate production and quality problems across PCB assembly from inspection data, using root-cause analysis and process knowledge alongside production, quality, and engineering teams.

  2. 2023

    Data Analyst

    Eleos Health

    Built an automated PDF-to-data pipeline, analyzed PostgreSQL datasets, and enabled A/B testing that measured 53% faster documentation generation.

  3. 2020 — 2022

    Data Analyst / Developer

    Freelance

    Delivered Python data-collection and validation pipelines for software-development clients, producing reusable JSON and CSV datasets.

04 / What’s next

A good problem
starts a conversation.

Open to AI / ML engineering roles and focused collaborations in evaluation and performance.

gollankurulkar@gmail.com