Just Ask for Generalization
This blog post outlines a key engineering principle I’ve come to believe strongly in for building general AI systems with deep learning. This principle guides my present-day research tastes and day-to-day design choices in building large-scale, general-purpose ML systems. Discoveries around Neural Scaling Laws, unsupervised pretraining on Internet-scale datasets, and other work on Foundation Models […]
What every programmer should know about how CPUs work [video]
Wind Knitting Factory
From the top of the façade the knitwear drops along the front, until a window has been reached. Trough the window the knitwear gets continues transported inside, where people can see it becoming longer and longer. Occasionally the knitwear gets ‘harvested’ and transformed into scarves. Every scarf gets a label which tells you the time […]
High-Fidelity Simultaneous Speech-to-Speech Translation
Abstract:We introduce Hibiki, a decoder-only model for simultaneous speech translation. Hibiki leverages a multistream language model to synchronously process source and target speech, and jointly produces text and audio tokens to perform speech-to-text and speech-to-speech translation. We furthermore address the fundamental challenge of simultaneous interpretation, which unlike its consecutive counterpart, where one waits for the […]
Development of a transputer ISA board
I developed software for transputers in the years 1993-1996. A few months ago, I wrote some articles about my experiences, and most recently I developed a transputer emulator in Javascript. After my transputer emulator in Javascript was working, I got curious about running my software on a PC computer. I have several PC motherboards with […]
Impact of PCIe 5.0 Bandwidth on GPU Content Creation and LLM Performance
Table of Contents Introduction With the release of the NVIDIA Blackwell GPUs and RDNA 4-based Radeon 9000-series GPUs, we finally have consumer video cards that support the PCIe 5.0 standard. Although we have had motherboards with support for it for some time now, we didn’t have any devices other than storage that took advantage of […]
Opening up ‘Zero-Knowledge Proof’ technology
[{“model”: “blogsurvey.survey”, “pk”: 9, “fields”: {“name”: “AA – Google AI product use – I/O”, “survey_id”: “aa-google-ai-product-use-io_250519”, “scroll_depth_trigger”: 50, “previous_survey”: null, “display_rate”: 75, “thank_message”: “Thank You!”, “thank_emoji”: “✅”, “questions”: “[{“id”: “e83606c3-7746-41ea-b405-439129885ead”, “type”: “simple_question”, “value”: {“question”: “How often do you use Google AI tools like Gemini and NotebookLM?”, “responses”: [{“id”: “32ecfe11-9171-405a-a9d3-785cca201a75”, “type”: “item”, “value”: “Daily”}, {“id”: “29b253e9-e318-4677-a2b3-03364e48a6e7”, […]
The End of Moore’s Law for AI? Gemini Flash Offers a Warning
For the past few years, the AI industry has operated under its own version of Moore’s Law: an unwavering belief that the cost of intelligence would perpetually decrease by orders of magnitude each year. Like clockwork, each new model generation promised to be not only more capable but also cheaper to run. Last week, Google […]
Launch HN: K-Scale Labs (YC W24) – Open-Source Humanoid Robots
Hi HN, I’m Ben, from K-Scale Labs (https://kscale.dev). We’re building open-source humanoid robots. Hardware video: https://www.youtube.com/watch?v=qhZi9rtdEKg Software video: https://www.youtube.com/watch?v=hXi3b3xXJFw Docs: https://docs.kscale.dev Github: https://github.com/kscalelabs HN thread from back in May: https://news.ycombinator.com/item?id=44023680 I started K-Scale because I really wanted a humanoid robot to hack on, so I knew that if I built one, I would have at […]
EBAF – eBPF Based Ad Firewall
“You Wouldn’t Download an Ad”