Today, we release an early checkpoint of LFM2-24B-A2B, our largest LFM2 model. This sparse Mixture of Experts (MoE) model has 24 billion total parameters with 2 billion active per token, showing that the LFM2 architecture scales effectively to larger sizes. … Read more
This is the third part of a series on LLMs in software engineering. First we learned what LLMs and genetics have in common. (part 1) LLMs don’t improve all facets of engineering. So, understanding which areas LLMs do improve (part … Read more
I’m taking a few weeks of pause on my HNSWs developments (now working on some other data structure, news soon). At this point, the new type I added to Redis is stable and complete enough, it’s the perfect moment to … Read more
Observability at scale: Our internal system grew from 19 PiB to 100 PB of uncompressed logs and from ~40 trillion to 500 trillion rows. Efficiency breakthrough: We absorbed a 20× surge in event volume using under 10% of the CPU … Read more
Abstract:We study a novel language model architecture that is capable of scaling test-time computation by implicitly reasoning in latent space. Our model works by iterating a recurrent block, thereby unrolling to arbitrary depth at test-time. This stands in contrast to … Read more
“Postgres was great when we started but now that our service is being used heavily we are running into a lot of ‘weird’ issues” This sentiment is frequently echoed by CTOs and senior engineers at high-growth startups when I speak … Read more
Kimi Team Full Report 🚀 Introducing Kimi k1.5 — an o1-level multi-modal model Sota short-CoT performance, outperforming GPT-4o and Claude Sonnet 3.5 on 📐AIME, 📐MATH-500, 💻 LiveCodeBench by a large margin (up to +550%) Long-CoT performance matches o1 across multiple … Read more
Classic linear programming (LP) problems are one of the most foundational problems in computer science and operations research. With extensive applications across numerous sectors of the global economy, such as manufacturing, networking, and other fields, LP has been the cornerstone … Read more
[embedded content] Do you have a Rails app built on PostgreSQL and need to scale it to millions of users? In this interview, I speak with Andrew Atkinson, one of StepChange’s expert consultants, who brings deep expertise in optimizing Rails … Read more
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. … Read more