Lab note #063 Vibe coding a scheduler
This was a tough past week. I was working on supporting async handlers, even if the reactive part was synchronous. I didn't want the graph to have to wait on network requests.
This turned out to be tough. Also I got food poisoning this past week. The design tension was this: I wanted async handlers, but I didn't want async to infect the API for reactive nodes.
Frustratingly stuck for two days, I decided to see if improvements in code gen would be any help. I tried out CodeBuff and Claude Code.
Most of the polarization around code gen is how useful it could really be. I think it depends on the type of work you do. If it's mainstream web coding with lots of users and training data, or if you don't have strict requirements or taste, you'll probably have an easier time. For something slightly new like a reactive notebook with an effect system, I usually had a hard time getting code gen to do a lot of what I wanted in the way that I wanted.
But stuff improves fast. In addition, I was curious by a Lightcone episode where highly technical founders claim they were spending all their time reviewing code written by AI, rather than actually writing code.
Why I was resistant? Perhaps I still tied the identity of an engineer with being able to write code. But then again, I've always said, AI coding honestly just feels how Geordi La Forge works, and they still call him an engineer.
How did code gen do? It's much more impressive than it was just a few months ago, but it's almost-but-not-quite-there.
CodeBuff tries to one-shot a solution. I wrote up a long thing about my conflicting design goals and let it go to town. It did independently suggest schedulers, and proceeded to implement something. It also wrote tests and was pretty aggressive in getting it to pass. After two independent tries, it had something that looked like a working solution. However, it came up with a solution that just blocked when running async code. 😕
Nonetheless, I think this is the first time I'd been impressed with code gen. Most of the time, they don't even get close. I think there's a lot of other little things I'd want to tweak, but at least this starts to give me an idea of the shape of the solution.
Then I tried Claude Code. It's more iterative in its approach. It does feel more chatbox-like, but it's focused more on reviewing code, instead of chatting, though you can do that too if you want. You can ask it why it did that, or to adjust its approach. I found it easier to carve out the solution that I wanted, but it did require a lot more code review. For the most part, it did a great job with the basic data structures and building up a basic scheduler. However, the last mile integration between the reactive core and the scheduler proved challenging for it. In addition, it can take the wrong approach, such as changing tests to pass them rather than changing the implementation.
The silver lining is that I've gotten to look at different four or five variations of a solution and learned some python API things that I wouldn't have known before, even if I don't end up taking any of their solutions. And definitely, the bar for code gen has been moved up. I expect that it'll keep moving on up.
I don't think this is a slam dunk success, as I had to do a lot of iterating, but this is progress. I got some modicum of a scaffold for a scheduler, and will go back to writing code by hand and will probably delegate to vibe coding for the things that I know it'll be able to handle. But I think the trend is asymptotic towards vibe coding, and it'll still be valuable to drop down to manual coding in some cases. And likely, it'll be more and more valuable to know how to read code.
