Lessons from the LLM Frontlines: A Developer’s Journey

Hey there, fellow code warriors! I’m back with another tale from the trenches. This time, we’re diving into the wild world of developing software with Large Language Models (LLMs). Buckle up, because this ride’s got more twists than a pretzel factory!

The LLM Toolbox: My Curated Arsenal

Let me tell you, my journey with LLM-powered development tools has been more unpredictable than a game of Minesweeper. After extensive experimentation, I’ve settled on a core set of tools that really get the job done:

1. Aider: The heavyweight champ. It’s like having a senior developer in your pocket, especially since it added support for Claude 3.5 Sonnet.
2. Simon Willison’s LLM tool: The reliable sidekick. Not flashy, but gets the job done when I need something quick and dirty.
3. GitHub Copilot: The plucky underdog. It’s trying its best, bless its heart, and sometimes surprises me with its suggestions.

These three form the backbone of my LLM-powered development workflow. Each has its strengths, and I find myself reaching for different tools depending on the task at hand.

The Road Not Taken: Why I Skipped Some Popular Options

Now, you might be wondering, “What about all those other shiny LLM tools out there?” Well, let me spill the tea on that.

Take Cursor, for example. I gave it a couple of test drives, about an hour each time, but quickly scurried back to whatever I was using at that time. Here’s the deal: Cursor is essentially a VSCode clone, trying to muscle into the same space. But here’s the rub – I’m deeply entrenched in the VSCode ecosystem. I’ve got my devcontainers, my custom configurations, and a whole suite of extensions I can’t live without.

Plus, I’ve got this neat setup with WSL2 where I can summon VSCode with a quick command. Cursor? It just couldn’t keep up. It’s like trying to replace your favorite pair of broken-in coding socks with a fancy new pair that looks great but gives you blisters.

That said, I’m not completely writing off Cursor or other tools (especially Cody or Continue.dev). I’m keeping an eye on them from afar, like that one cousin on Facebook who’s always up to something interesting but you’re not quite sure you want to get involved.

Aider: The New Sheriff in Town

Let me break it down for you. Aider with Claude 3.5 Sonnet support is like having a coding buddy who actually understands you. It’s not just converting your (broken) English to a piece of code which you can run and verify. And it does it so effortlessly, something which is missing with other tools, or (back in the day) with Aider using GPT-4 Turbo or 4o.

Before Aider got its Sonnet upgrade, I was doing this weird dance with Simon Willison’s files-to-prompt tool. Picture this: feed specific parts of the codebase, chat, copy outputs, rinse and repeat. It was like playing telephone with my code. But now? It’s smoother than a freshly refactored codebase.

The LLM Development Playbook: Tips from the Trenches

Alright, gather ‘round, padawans. Here’s some wisdom from your battle-scarred code sensei. These tips aren’t just theoretical mumbo-jumbo; they’re forged in the fires of real-world LLM-assisted development.

1. Convention is King: Have your code conventions ready as a prompt. Save it somewhere your LLM tool can access. It’s like giving your AI pair programmer a clear set of instructions. This isn’t just about coding style – it’s about project-specific quirks too. Maybe you always wrap errors in a custom logger, or you have a particular way of structuring your APIs. Whatever it is, spell it out for your AI buddy. And your AI buddy will include it in the system prompt and it’s bound to follow that, almost most of the time.

2. Context is Queen: Always know which files you need beforehand. Include only those in your context. It’s like giving your LLM a map instead of dropping it in the middle of the Amazon. This is crucial. The more focused your context, the more relevant and accurate the LLM’s output will be. I’ve had sessions go off the rails because I included too much irrelevant code. Now, I’m ruthless about curating the context. Though Aider can figure out that it should deal with a file which is not included, and can ask you to add that file, it would be better when you start your session. If you don’t know, don’t worry, that’s not the end of the world.

3. Know When to Fold ‘Em: If your AI buddy starts going off the rails, don’t be afraid to hit that reset button. It’s not giving up; it’s tactical retreating, akin to going for a walk when you are stuck. This is especially true in long sessions. LLMs can sometimes “forget” earlier context or start to jumble things up. If you find yourself constantly correcting the AI or if its suggestions start to feel random, it’s time for a fresh start. Practical tip: If you know that AI has gone off the rails, evaluate if you can reset its mistakes in one or two messages. If not, manually undo it and start afresh.

4. Copy-Paste > Web Search: When you need to provide specific examples, create a temporary MD file and paste them in. It’s like showing your LLM a picture instead of describing it. This has been a game-changer for me. Instead of relying on the LLM’s potentially outdated or incorrect knowledge, I give it exactly what it needs. This works wonders for API documentation, conversion of code between languages or use cases, specific code patterns, or even error messages you’re trying to debug.

5. Trust, but Verify: Always validate outputs. Run tests, lint your code, actually run the darn thing. Remember, hallucination might be a feature for LLMs, but it’s definitely a bug for software. This can’t be overstated. LLMs are amazing, but they’re not infallible. I’ve (and almost everyone else) had instances where the code looked perfect but had subtle logical errors that only showed up during runtime.

6. Divide and Conquer (with a Side of Brainstorming): Break down your tasks to the smallest unit possible. It’s like eating a giant cake all by yourself - one bite at a time. This isn’t just good software development practice; it’s essential for effective LLM use. I often start by asking the LLM to help me break down a complex task into smaller, manageable chunks. Then we tackle each chunk one by one. This keeps the context focused and makes it easier to track progress. But here’s the kicker – don’t just use LLMs for coding. They’re fantastic brainstorming buddies too. I often use chat interfaces or web-based LLMs as a rubber duck, explaining my problem and asking for suggestions. The responses can spark new ideas or help me see the problem from a different angle.

7. Cache In, Cash Out: Use prompt caching if available, but remember to exit when you’re not active. It’s like turning off the lights when you leave a room - good for your wallet and the environment. This is particularly relevant with tools that offer prompt caching. It’s a great feature for active coding sessions, potentially saving on costs. But don’t forget to close the session when you’re done. There’s no need to keep the meter running when you’re not coding. Aider handles Anthropic’s prompt caching like a pro.

8. Master the Art of Prompt Engineering: Learn to phrase your requests in a way that gets the best results. This is a skill that develops over time. I’ve found that being specific, providing examples, and clearly stating your expectations leads to better outcomes. Instead of “refactor this code,” try “refactor this function to improve performance, focusing on reducing database calls.” The more precise you are, the better the LLM can tailor its response to your needs.

Remember, these AI tools are here to augment your skills, not replace them. They’re like power tools - incredibly useful, but you still need to know which end of the hammer to hold. Used wisely, they can supercharge your productivity and help you tackle problems you might have shied away from before.

So go forth, experiment, and may your code be ever bug-free. And if it’s not? Well, that’s what LLMs are for, right?

Resources

P.S.: Shout out to Tim Kitchens and his YouTube channel. I won’t overstate that I reiterated many points he said in his videos, adding some personal sprinkles.

Here is one specific video I like to add here: Mastering AI for Software Development: Essential Tips & Principles

And his channel is awesome to follow: Coding the future with AI

Aider + Cursor tutorial: SECRET SAUCE of AI Coding? AI Devlog with Aider, Cursor, Bun and Notion

Until next time, may the force (and the LLMs) be with you!