My Work
Language models from scratch, LLM-driven game agents, computer vision, and tools that explain how they work.
Deep learning
LLM from Scratch
In progress: fine-tuning for classification
A GPT-2 style decoder-only transformer written by hand in PyTorch, following Sebastian Raschka's Build a Large Language Model From Scratch.
- params
- 124M
- layers
- 12
- heads
- 12
- emb dim
- 768
- vocab
- 50,257
- context
- 1,024
How it works
- 1Text is tokenized with GPT-2's BPE vocabulary and sampled into input/target pairs with a sliding window.
- 2Multi-head causal self-attention, LayerNorm, GELU and the feed-forward block are each implemented as their own nn.Module, not imported.
- 3Pretrained on unlabeled text with next-token prediction; a sibling repo, shakespeareGPT, trains the same architecture on Shakespeare.
class MultiHeadAttention(nn.Module):
def forward(self, x):
b, num_tokens, d_in = x.shape
keys = self.W_key(x) # Shape: (b, num_tokens, d_out)
queries = self.W_query(x)
values = self.W_value(x)
# (b, num_tokens, d_out) -> (b, num_heads, num_tokens, head_dim)
keys = keys.view(b, num_tokens, self.num_heads, self.head_dim).transpose(1, 2)
queries = queries.view(b, num_tokens, self.num_heads, self.head_dim).transpose(1, 2)
values = values.view(b, num_tokens, self.num_heads, self.head_dim).transpose(1, 2)
# Scaled dot-product attention with a causal mask
attn_scores = queries @ keys.transpose(2, 3)
mask_bool = self.mask.bool()[:num_tokens, :num_tokens]
attn_scores.masked_fill_(mask_bool, -torch.inf)
attn_weights = torch.softmax(attn_scores / keys.shape[-1]**0.5, dim=-1)
attn_weights = self.dropout(attn_weights)
context_vec = (attn_weights @ values).transpose(1, 2)
context_vec = context_vec.contiguous().view(b, num_tokens, self.d_out)
return self.out_proj(context_vec)Interactive explainer
Inside the Transformer
A step-by-step walkthrough of what GPT-2 does to a prompt, from raw text to the next-token distribution. Type a sentence and follow it through the model.
- stages
- 11
- dependencies
- 0
How it works
- 1Eleven stages: BPE tokenization, vocabulary lookup, token and positional embeddings, the transformer block, Q·K·V self-attention, the feed-forward network, the 12-layer stack and the output distribution.
- 2Every stage renders tensors at GPT-2's real shapes (768-dim embeddings, 12 heads, 50,257-token vocabulary).
- 3The numbers are deterministic illustrations, not a live model: the goal is to show the mechanics, not to run inference.
Games + LLM agents
Court of Whispers
A political espionage RPG where every NPC is an LLM agent with its own goals. You have five days to talk four of them into a coup while one of them gathers proof against you. Built with Usaib Ahmed.
- agents
- 4
- days
- 5
- whispers / day
- 5
- targets
- Web + Android
How it works
- 1The Godot client only handles input, rendering and physics. A stateless server on Cloudflare Workers generates dialogue and is the arbiter of game state.
- 2Each conversation moves trust, suspicion and proof scores; the win condition is computed server-side and synced to the client at day transitions.
- 3NPCs run a physics-driven state machine: bounded wander zones, navigation-mesh snapping on spawn and displacement-based stuck recovery.
- 4Agent replies are capped to a short sentence, which keeps the cost of several concurrent LLM calls per player under control.
Computer vision
Physiotherapy Guidance System
A doctor records an exercise once; the system learns the correct form from that video and then scores a patient's reps live from their webcam.
- landmarks
- 33 × 3D
- inference
- in-browser
- form score
- 0 to 100
How it works
- 1MediaPipe Pose Landmarker extracts 33 3D landmarks per frame in the browser; a custom One Euro Filter removes jitter without lagging fast movement.
- 2Joint and segment angles are computed from the landmarks, then K-Means clusters the doctor's reference video into a sequence of exercise states with expected angle ranges.
- 3The patient's live angles are matched to those states, per-rep deviation is normalized against each angle's range and rolled into a form score.
- 4Feedback comes back as audio cues and a skeleton overlay, and sessions are stored in Supabase.
Webcam feed
-> MediaPipe Pose Landmarker L ∈ ℝ^(33×3)
-> One Euro Filter adaptive smoothing
-> Angle computation θ_joint, θ_segment
-> K-Means state matching s*(t) = argmin |x_primary − μ_primary|
-> Deviation vector ε(t), normalized per angle range
-> Form score F = max(0, 100 − mean(‖ε‖₁) / 2)
-> Feedback audio cues + skeleton overlayTooling
gif2ascii
Converts animated GIFs into ASCII animations, entirely in the browser. Nothing is uploaded; a Rust CLI is in the works.
- char ramp
- @%#*+=-:.
- server
- none
How it works
- 1Each frame is drawn to a canvas and downsampled to a character grid.
- 2Per-cell brightness is 0.299R + 0.587G + 0.114B, shifted by the luminosity setting and stretched around the midpoint by the contrast setting.
- 3Brightness is normalized against the frame's own min and max, then mapped onto a 10-character ramp.
- 4Frames are re-encoded to a GIF with gifenc, or exported as text.
Design
Side quests
Ghost Pastel
A dark pastel theme for Omarchy with a companion VS Code theme.
Featured on
Fictional plant/farm site
A pastel farm and plant-themed site concept, rebuilt as an interactive HTML page: toggle the theme, drag pots, pick seed cards and plant the grid. A personal design exercise, not a real product.
Archive
Everything else
Pethia
Sassiest discord bot known to man
Bool
One of the first things I ever coded. Bool is just a simple rule based bot bud
Relic
Search through an entire historical archive of data to correlate events from the past to history
