I build the infrastructure that lets large language models run efficiently across multi-GPU clusters — disaggregated serving, RDMA communication, and cross-platform inference tooling.
I work on the infrastructure layer beneath large language models: how they run across multi-GPU clusters, why they stall at particular layers, and how to move that efficiency to edge hardware without losing accuracy.
At MulticoreWare, I work on a Microsoft-collaborative effort to disaggregate LLM serving — separating Attention and FFN computation across GPUs with different hardware profiles, and building the RDMA communication layer that makes the separation transparent to the inference runtime.
The real bottleneck in distributed inference isn't compute — it's the cost of moving tensors between nodes. Once you see that, everything else follows.
Before that, I spent five months at IIIT Hyderabad under Dr. CK Raju, studying Transformer internals from first principles — the linear algebra behind self-attention and positional encoding, not just the API surface. That habit still shapes how I debug distributed systems: I start with the math, not the stack trace.
Outside of work, I build independent projects — including LearnKit, an open-source procedural-memory system for AI agents, and an interactive visualizer for DeepSeek's model architecture.
My work sits at the boundary between machine learning and systems. Less about new model architectures than about the systems that execute them — what running a model actually costs, where that cost is avoidable, and how serving infrastructure and kernels should be designed around the answer.
Rethinking how a model's execution is partitioned across machines — splitting stages by what each actually demands of hardware, memory bandwidth versus compute, rather than replicating whole models. I'm drawn to the design question underneath: how far a system can be disaggregated before coordination cost overtakes the gain, and what a topology built for that limit would look like.
Where theoretical speedups are actually won or lost. Fusion decisions, memory access patterns and occupancy, quantized kernel behaviour, and the gap between what a compiler emits and what the hardware could sustain. Writing and tuning kernels rather than treating the runtime as a black box.
Treating the network as a first-class scheduling constraint. Micro-batch pipelining, overlapping RDMA transfer with GPU compute, and the question of how much communication latency a scheduler can hide before it becomes structurally exposed.
Why a quantized model that should be 4× faster often isn't. Runtime-specific graph rewriting, cross-platform behavioural differences, and building the measurement tooling needed to attribute a performance gap to a specific layer rather than guessing.
Agents re-plan from scratch on tasks they have already solved. I'm interested in procedural memory as a systems problem — what to store, how to gate it on real outcomes rather than model self-assessment, and when reuse should override fresh inference.
Attention and FFN operations have different hardware needs — Attention is memory-bandwidth bound, FFN is compute bound. Running both on the same GPU wastes one or the other. This physically disaggregates the two across GPU groups, connected by an A2F/F2A communication pipeline over RDMA with a micro-batch overlap scheduler that hides transfer latency behind computation.
Result: FFN GPUs went from ~40% utilization (idle, waiting on Attention) to running near peak concurrently with Attention — removing the scaling wall that capped single-node serving around 70B parameters.
Built as part of a Microsoft-collaborative infrastructure effort at MulticoreWare — code is company-confidential, not open source.
LearnKit gives tool-using AI agents procedural memory. It captures the tool-call sequence that solves a task, replays it with zero LLM calls on exact repeats, and guides similar tasks with a distilled playbook instead of re-planning from scratch. Memory is typed across seven record kinds (skills, facts, failures, strategies, preferences, traces, heuristics), quality-gated on real tool outcomes rather than an LLM judge, and lifecycle-managed — new records are quarantined, then promoted or decayed. Retrieval is hybrid SQLite + FTS5. Adapters cover LangChain, LangGraph, AutoGen, CrewAI, LlamaIndex, and the OpenAI Agents SDK, plus plugin integrations for Claude Code, GitHub Copilot CLI, and Codex CLI.
Result: on a reproducible agentic benchmark across Qwen2.5-14B/32B and Llama-3.3-70B, playbook guidance produced up to +2.25 quality lift with 38% fewer pooled LLM planning calls, at equal task success.
Engineers deploying models to edge hardware were running six separate profiling tools by hand and still couldn't explain why a runtime underperformed its benchmark prediction. Perfalign profiles layer-by-layer across six runtimes at once, with a custom layer-fusion tracker that surfaces when operations silently merge into a single kernel — invisible to standard profilers.
Result: cut hardware evaluation from multiple days to under an hour; fusion tracking showed that roughly 30% of expected quantization speedups were being lost to unfused kernels. A vLLM-backed RAG chatbot lets engineers query results conversationally. Since adopted internally at MulticoreWare and by external customers, including Emass, for cross-platform hardware evaluation.
Commercial MulticoreWare product — source not public.
CCTV systems record continuously but flag nothing — detection and response were entirely manual. SecureSight runs real-time weapon detection with YOLOv11 on live feeds. Decoupling detection from database writes via async workers cut end-to-end alert latency significantly; duplicate-report auditing removes false-alarm fatigue before alerts reach dispatch.
Result: alert latency dropped from 800ms to under 200ms (4×). Won the Make UC Global Hackathon (University of Cincinnati); the work directly informed two filed patents.
Short write-ups on specific problems from the work above — the parts that don't fit in a resume bullet.
Attention and FFN look like two halves of the same transformer block, so the instinct is to run them on identical hardware. In practice they want opposite things. Attention over a long KV-cache is dominated by loading state from memory — it's bandwidth-bound, and the compute units mostly wait. FFN is the opposite: dense matrix multiplies that keep compute busy and barely touch memory. Colocating them means provisioning every GPU for the worse of the two profiles, and getting neither.
Disaggregating them sounds simple — Attention on one set of GPUs, FFN on another — but it turns a local memory read into a network operation. That's the actual engineering problem: A2F/F2A traffic has to cross RDMA fast enough, and often enough, that the FFN side is never sitting idle waiting on a tensor. The scheduling half of this — micro-batching the transfer so it overlaps with compute on both sides — ended up mattering more than the placement decision itself. In our numbers, that's the difference between FFN GPUs idling around 40% utilization and running near peak concurrently with Attention.
The part I didn't expect going in: correctness here is easy, performance is hard, and almost all of the hard part lives in the scheduling logic, not the split itself.
A common assumption is that if quantization gives you an N% theoretical speedup on paper, you should see close to N% on the target hardware. We kept seeing meaningfully less, and standard profilers weren't explaining why — they report timing per layer, not whether the compiler on that specific runtime actually fused the operations it should have.
Perfalign's fusion tracker sits underneath that: for each layer, log whether the operations expected to fuse into one kernel actually loaded as one kernel launch, per runtime. Doing that across six runtimes at once made the pattern visible instead of anecdotal — roughly 30% of the theoretical quantization speedup was disappearing specifically to unfused kernels the developer didn't know they had, not to any accuracy tradeoff or hardware limitation.
The generalization that stuck with me: an on-paper speedup estimate assumes its own reference implementation. The moment you cross to a different runtime or vendor toolchain, fusion behaviour stops being a hardware property and becomes a compiler property — and that's rarely reported anywhere.