Implementing

Recently, I watched Andrew Kelley’s talk on Practical Data Oriented Design. It goes into some of the architectural changes he’s been making to the Zig compiler, with pretty significant performance benefit. Would definitely recommend checking out the talk, even if … Read more

GRPO training with minimal dependencies. We implement almost everything from scratch and only depend on tokenizers for tokenization and pytorch for training. No transformers and vLLM dependencies! The default config is set to run on a single A40 GPU (48GB … Read more

Introduction# I created a Game Boy emulator in Ruby and released it as a gem called rubyboy! (I’d be happy if you could give it a star!) And now it runs in the browser using WebAssembly! Try the demo in … Read more

The previous iteration of the European Strategy for Particle Physics (ESPP) gave as its first recommendation: “An electron-positron Higgs factory is the highest-priority next collider.” This recommendation is, in our view, best addressed by constructing an e+e− Higgs factory that … Read more

I’ve been on a spaced repetition (SR) kick as of late. Mochi, the SR software I use, uses a modified version of the SuperMemo SM-2 algorithm to schedule cards. The differences are: The ease factor (EF) of the cards is … Read more

This is the first post in a multi-part series describing the Raft distributed consensus algorithm and its complete implementation in Go. Here is a complete list: Raft is a relatively new algorithm (2014), but it’s already being used quite a … Read more

Hello! This will be a first attempt at coming back to writing some blog posts about interesting topics I end up rabbitholing about. All the older stuff has been sadly lost to time (and “time” here mostly means a bad … Read more

Bouyed by the surprisingly good performance of neural networks with quantization aware training on the CH32V003, I wondered how far this can be pushed. How much can we compress a neural network while still achieving good test accuracy on the … Read more

Software for physics, geometry, finance, and probability often calls library functions like sin, cos, exp, and log. These library functions should be fast and accurate for science, engineering, and finance to work best. Unsurprisingly, lots of algorithms have been invented … Read more