OpenAI postmortem – Unexpected responses from ChatGPT
On February 20, 2024, an optimization to the user experience introduced a bug with how the model processes language. LLMs generate responses by randomly sampling words based in part on probabilities. Their “language” consists of numbers that map to tokens. In this case, the bug was in the step where the model chooses these numbers. […]
Go Run
It may seem silly, but go run is my favorite part about go. Want to run your code? go run main.go. It is so stupidly simple that I could tell my mom about this command, and she would immediately understand. Like with most things in go, the real power in this command is in the […]
An Introduction to the Theory and Practice of Poker (2020)
Lecture 9, Jan 24, 2020 Topic: An actual turn hand, River play, hand names Lecture slides: Click for PDF Lecture video: To see the embedded video clips while watching, click on the “i” when you see the placeholder.
Moving a billion Postgres rows on a $100 budget
Inspired by the 1BR Challenge, I wanted to see how much it would cost to transfer 1 billion rows from Postgres to Snowflake. Moving 1 billion rows is no easy task. The process involves not just the transfer of data but ensuring its integrity, error recovery and consistency post-migration. Central to this task is the […]
Neural Network Diffusion
Abstract:Diffusion models have achieved remarkable success in image and video generation. In this work, we demonstrate that diffusion models can also textit{generate high-performing neural network parameters}. Our approach is simple, utilizing an autoencoder and a standard latent diffusion model. The autoencoder extracts latent representations of a subset of the trained network parameters. A diffusion model […]
The killer app of Gemini Pro 1.5 is video
21st February 2024 Last week Google introduced Gemini Pro 1.5, an enormous upgrade to their Gemini series of AI models. Gemini Pro 1.5 has a 1,000,000 token context size. This is huge—previously that record was held by Claude 2.1 (200,000 tokens) and gpt-4-turbo (128,000 tokens)—though the difference in tokenizer implementations between the models means this […]
Useful Uses of Cat
When I write shell one-liners that transform the contents of some file, they often look something like In[1]: cat access.log | head -n 500 | grep mail | perl -e … This is what a lot of people reflexively call a useless use of cat1 And more thoughtful people will call a useless use of […]
Webhooks suck, but here are alternatives
Webhooks are a foundational part of the web’s infrastructure. Coined in a blog post in 2007, webhooks were suggested as a simple way to consume asynchronous feeds with a protocol all web developers are familiar with — HTTP. It’s now 2024 and webhooks have more or less remained the same, yet they have become the […]
Unconventional Strategies to Get Unstuck from a Creative Rut
Great ideas aren’t found. They’re made, through consistent creative practice. Creative thinking comes from creative doing. Influential painter Chuck Close said in an interview for Inside the Painter’s Studio: Inspiration is for amateurs—the rest of us just show up and get to work. And the belief that things will grow out of the activity itself […]
Inside The Funding Frenzy at Anthropic, One of A.I.’S Hottest Startups
Last May, Anthropic, one of the world’s hottest artificial intelligence start-ups, raised $450 million from investors including Google and Salesforce. It was the beginning of an astonishing funding spree. By August, Anthropic had landed $100 million from two Asian telecoms. Then Amazon committed $4 billion to it, followed by $2 billion more from Google. This […]