Essays
I pulled down Llama 3.1 7B for a long flight recently. It seemed magical in the same way downloading all of Wikipedia seems--like taking an entire universe with you. The missing piece is the context to steer the LLM. We need a Microsoft Access of RAG and LLM system
lab notes
As part of biasing towards action, I thought it was time to reimplement the Prolly Tree in Zig. I wanted to focus on being cache-friendly this time, now that I have lower-level control. But that resulted in a deep dive into data-oriented programming. The question is how to represent a
programming
I was watching Casey Muratori and the Primeagen the other day, and they were talking about the high-level architectural design of the core data inside of Primeagen's game. What Casey says sounds counter-intuitive for anyone from a traditional OOP software background. He suggests not committing to an organization
lab notes
My adherence to immutability has put me on the hard path for everything so far. Not just for the unicit Prolly tree, but also for the CRDT. Most practical CRDT libraries use an op-based CRDT or a delta-based CRDT, but many do not keep the entire history around. I think
programming
Just like CAD didn't make everyone an industrial designer, visual programming isn't going to make everyone a programmer. Whether textual or visual, there'd still be a lot of underlying concepts (and how they are composed) to learn. But still, I think it's
lab notes
I started working on making an LLM Zine. Not to worry, the work on the database is still in the works, but this has been a burgeoning interest and opportunity. LLMs are a fundamental technological breakthrough, and the effects of this technology will take at least a decade to work
programming
💡The Recurse Center encourages recent graduates to write about their experience in the form of a Return Statement. Below is mine. The 12-week experience at RC did remind me of the story of the tall, strong, and unbendable Oak that towers over the short, thin, and flexible Reeds. On an
lab notes
I've now implemented an Op-based and a Pure op-based CRDT. I couldn't quite make the leap before to merkle-based CRDTs. While vectors clocks are mechanically simple, I wasn't quite sure how they mapped to a merkle-DAG node if replica IDs weren't available.
lab notes
This continues my exploration into CRDTs, beyond the typical tutorials. The more I looked, the more threads there were to pull on. I currently have a couple of lines of questioning to answer: * How do we compose CRDTs so they both converge to a single state without coordination but with
lab notes
After the implementation of the Prolly Tree, I set about writing an immutable map. It was pretty straightforward as most of the heavy lifting is done by the Prolly Tree implementation. The immutable map just keeps a simple list of roots after every change. There are a bunch of improvements
lab notes
It took a long time, but the incremental insert for a Prolly Tree is finally done. I don't think I've ever spent this long as such a little part of code. The two aspects that had me struggling were the bottom-up breadth-first traversal, and how to
lab notes
The punchline first: the incremental insert is finally done on the Prolly Tree. I don't think I've ever spent so much time on such a small, core piece of code. It was also a lesson for me to lean into the struggle and talk about it.