Lab note #042 LLM Zine

I started working on making an LLM Zine. Not to worry, the work on the database is still in the works, but this has been a burgeoning interest and opportunity. LLMs are a fundamental technological breakthrough, and the effects of this technology will take at least a decade to work out, just as the internet had. Despite that, they can be hard to work with to get the precise specific output that you want. It's even harder to tell whether they're doing the right thing.

So Sri and I, of the Technium Podcast fame, decided to get together again to work on collating knowledge of how to work with LLMs. But why a zine? After doing the Technium Podcast, we did think we were good at synthesizing new information on new technologies and getting to the heart of why it's interesting. But monetizing on YouTube can be difficult on a topic that's a niche of a niche. So this time, we're taking our skills to put together something we can actually sell to people.

We could have sold consulting-style reports, but we saw that Julia Evans was having some amount of success with zines and started to look into it more. We surmised that her target audience were engineers, but she also sold bundles for HR, corporate events, and conferences. That seemed promising. In addition, given the topic, I think there's potential for premium products later on.

Zines are a good middle between blog posts and infographics in terms of their information density. People want the information delivered quickly, but they also want insights. We think zines are an under-explored avenue for the specific signal-to-noise ratio we're targeting. In addition, they have the potential for world-building that helps draw readers in.

Lastly, zines are underexplored and currently low-status. That's usually a sign to me that there's quite a lot that can be done here.

The first issue is going to be about LLM Task Eval: How do we know the LLM is doing what it's supposed to be doing well? It's set in a forest with a shoggoth representing the LLM living the canopy. Down below, there is a community of technologically advanced forest friends that use the shoggoth for their daily lives, from personal to business. We'll follow around a yet-to-be-named duo of Fox and Bear as they live lives and work on and with the shoggoth and learn about how to do LLM task eval as a result.

We've done some preliminary sketches and layouts to show some friends and colleagues to gauge interest and potential (though it's on a different topic). You can check them out below.

I'll have more updates on the database stuff next week.