Let it rip

Huge LLM user here. I didn’t become one a couple of months ago — I’ve been experimenting with LLMs for a while. But it’s hard to find the courage to write about this experience. Every couple of weeks the LLM-powered workflow shifts — people keep coming up with great ways to use it.

One thing that never changed for me was that an LLM was only a handy assistant, nothing more than that. But this time, I gave the LLM a boring maintenance loop and it actually shipped a lot meaningful improvements without any guidance at all.

(yard|rdoc)-markdown gems

This year, all my time has been going towards one project — Posh TUI.

A Markdown renderer is a big part of an application like this. I’m using the glamour library from the Charm team. Thank god I don’t have to figure out how to render Markdown in a terminal — it’s a crazy complicated problem.

But I do have to figure out how to convert Ruby source code into Markdown documentation. The main issue is that it has to be done reliably. Quick workarounds don’t cut it here. So that led me to write a couple of libraries — yard-markdown and rdoc-markdown — to make that possible.

It seems like I picked the right approach. But to make it work, I went into this loop to generate → compare → tweak → test → repeat (500x) and got tired of it. The libraries got stuck in a “it doesn’t break; looks legit for now” state.

But this is part of the core experience for users — these libraries matter. I just didn’t have enough drive to solve it once and for all.

Let it rip

One weekend I asked an LLM to work on the rdoc-markdown gem. But this time my prompt started with: “Don’t ask any questions. Make your best guess and proceed without assistance.” The plan was to run this session in “yolo” mode.

I laid out all the context:

  • How to generate output
  • That the output should be as close to HTML as possible
  • That there might be bugs
  • Which gems it might use as good test subjects

After that was out of the way, I suggested it might be a good idea to do this in a loop.

With that, I made sure my laptop wouldn’t automatically go to sleep. And I let it rip.

Pupil outdid its master

It took it a couple of hours to do all this boring, repetitive work. I checked on it a couple of times before going to sleep, but I never assisted. There wasn’t a specific goal — it used its best judgment and just kept going.

In the morning, the results were in.

It produced decent Markdown documentation for Rails, then for Minitest, and then for Ruby itself. I couldn’t believe my eyes — it squashed all the bugs I knew about. More than that, it looked great.

Of course, I couldn’t let this slide — I jumped into the code and found some bugs. Now you see who’s the boss around here?

Jokes aside: I ran a similar procedure on the yard-markdown gem the next evening, and the results were similarly good.

Now I’m sitting here relieved that I probably saved myself at least a month of part-time work. But at the same time I’m wondering: would I have found enough motivation to do the same tedious job at the same level of execution? I’m not sure I would.

I didn’t even bother guiding anything in this process. And for a brief moment, the compute became its own master and executed the work the way it saw fit. Bravo, young padawan.