There’s a concept in urbanism and sustainability that I recently learned about from the CityNerd YouTube channel called the green transportation hierarchy, which looks like this: The idea is that things lower on the pyramid should defer to the things … Read more
Implementing cosine in C from scratch 7/19/2020 Update 7/20: See the discussion of this post on Reddit. TL;DR: I explored how to implement cosine using a few different approaches. One of the implementations is nearly 3x faster than math.h if … Read more
As part of bringing myself up-to-speed after joining TigerBeetle, I wanted some background on how distributed consensus and replicated state machines protocols work. TigerBeetle uses Viewstamped Replication. But I wanted to understand all popular protocols and I decided to start … Read more
CPUs are very good at doing things in parallel, even in a single core context. Indeed, speculative execution of code and instruction reordering helps the CPU ensure that the pipeline is always full. However, data dependencies in the sequence of … Read more
In December 2021, a company from San Francisco asked me to port a Rust library to Java. The task seemed simple, but it ended up requiring a from-scratch implementation of the MySQL protocol and even fixing a bug in the … Read more
In this blog post I want to show the complete code (in Python3) of how a very simple optimizer for sequences of operations can work. These algorithms could be part of a (really simple) compiler, or a JIT. The architecture … Read more
In C, the length of a string in marked by a 0 byte at the end of the string. Thus to determine the length of the string, one must scan it, looking for the 0 byte. Recent ARM processors have … Read more
Working with data structures is fun — said nobody ever. What about “inventing” our own data structures? It’s gotta be a lot more fun, huh? In this blog post I am gonna go step by step through implementation of one … Read more
tl;dr: Use HyperLogLog, it’s a reasonable approach with great trade-offs and no large architectural liabilities. For a quick & dirty prototype, use hstore, which also performs the best with integer IDs. The year is 2022. You’re head DBA at the … Read more
def get_gradients(text, model, tokenizer): def get_correct_span_mask(correct_index, token_size): span_mask = np.zeros((1, token_size)) span_mask[0, correct_index] = 1 span_mask = tf.constant(span_mask, dtype=‘float32’) return span_mask embedding_matrix = model.bert.embeddings.weights[0] encoded_tokens = tokenizer(text, return_tensors=“tf”) token_ids = list(encoded_tokens[“input_ids”].numpy()[0]) vocab_size = embedding_matrix.get_shape()[0] token_ids_tensor = tf.constant([token_ids], dtype=‘int32’) token_ids_tensor_one_hot = … Read more