Throughstone brings software engineering discipline to vibe coding and points to a broader lesson: AI works best when guided by structure, process, and expertise.
This blog was created as a companion to my book The Career Toolkit: Essential Skills for Success That No One Taught You. Initially, I wrote only about work topics, not technology, even though I work as a CTO. Over the past few years, however, that line has blurred. AI is rapidly transforming the workplace, along with many other aspects of society, so I added a technology section to the blog. Still, most of my writing has focused on technology’s impact on jobs, work, and society. Today’s article is a little more technical, but still relevant to all of us. The software I’m announcing is a tool for vibe coders, and vibe coding is becoming more and more common. However, the motivation behind it and how it works provides a broader lesson about how we should and should not use AI at work.
Throughstone: a structured methodology and template for using AI coding tools to move from idea, to architecture, to implementation. It lets even non-technical people follow industry best practices, with no technical knowledge required.
For the past two years I’ve been hearing conflicting stories. On the one hand, people kept talking about “vibe coding,” which means writing software simply by telling AI what you want in natural language, as opposed to writing actual code directly. People with no knowledge of coding could quickly create software products. Seasoned developers could supposedly get huge productivity gains. I talked to many people (mostly non-technical folks on the business side) who raved about it.
But there was a flip side to the story. I would see articles like “After 6 Months of ‘Vibe Coding,’ My Codebase Is a Mess. Here’s What Went Wrong” by Devrim Ozcay. Ozcay leads the article off with, “AI wrote 60% of my code. Tech debt increased 340%. Now I’m rewriting everything.” I’ve seen similar reports from engineers even with decades of experience.
Or comments like this one, “[Our VC firm] made the decision to pass on [your company] . . . our engineering team flagged significant concerns around the codebase . . . AI-generated output – no clear architecture, minimal test coverage, and logic that falls apart under load. It’s not a product we can confidently scale capital behind. I genuinely believe in the problem you’re solving, and I think you have real commercial instincts. But the foundation needs to be rebuilt before institutional money makes sense here.” (source) In other words, the VC liked the problem space, the solution, and the team. The startup even got to the due diligence stage. Unfortunately, it was built on a foundation of sand.
I spent the last two years looking into why this was. I kept hearing these two conflicting stories of how it’s amazing and how it’s smoke and mirrors.
Software is simply the process of automation. Early software systems did things like payroll. Each payroll period it would take a list of names, hourly wages, and hours worked and generate all the payroll checks, completing the task much faster than a human. Simple. Other, much more complex processes got automated over time.
Many companies accumulate tech debt: unnecessary complexity that makes simple tasks, like adding a new feature, take more time and effort than they should. One form of technical debt is a misalignment of the process encoded in the software to the process as needed in the real world. (Other sources are poor quality code, duplication, missing tests, missing paths, lack of robustness, and more.) Sometimes the misalignment comes from the real world process shifting and the software not keeping up with it. Other times, it comes from poor initial design of the software in the first place. Either way the result is changes get costly and the system itself is less automated than it should be, often requiring humans to complete part of what should be automatically done by the software. I’ve cleaned up tech debt at multiple companies, in addition to creating greenfield projects where I designed them to avoid introducing tech debt in the first place.
This is the key point of the article. If you don’t understand and think through the process, automating it doesn’t help, and may even hurt. Software just does things at scale, and the wrong system, at scale, becomes inaccurate or misaligned at scale.
The solution in software is to sit down and design the system. Good engineers will spend hours, days, or even weeks at a whiteboard with the business team to define the system. Like an iceberg, most of what they need to think through isn’t visible to non-engineers. Non-engineers usually see only the UI, while most of the code is working behind the scenes.
This applies not just to software, but other projects. You may see the event or ad campaign delivered by the marketing team, but most of the work was internal, to design and execute that initiative: Who’s the audience? What’s the objective? What’s the message? The compelling sales pitch that takes an hour to deliver is a culmination of research, collateral development, objection handling, and practice refined over months across the entire sales team. Whatever the discipline, top practitioners have developed methodologies over time to get them to the best results while novices often try and struggle until they stumble over what works. Mentorship is valuable because it takes that learned wisdom and accelerates less experienced people down the path.
When using AI, if you just throw a problem at it, it will give you an answer. If you point it at a folder with all the data from the past quarter and tell it to create a twenty-page report, it can do so in minutes. It may not be wrong, but will it be right? The report will, for the most part (hallucinations are still a thing), be plausible. That’s not the same as correct. It may not be answering the right questions or looking at things the right way. You are also likely to wind up with workslop. This is why experience still matters (although in coming years AI may close that gap).
There’s a second issue plaguing these projects as well, size. If you read a 1,500 word article, you can keep most of the details in your head right after you finish. If you read an entire book, even in one sitting, it’s hard to remember all the details.
LLMs are not dissimilar. They have context windows and even if you fit everything into the context window, the more information there is, the more LLMs seem to have trouble keeping track of it. This is referred to as the lost in the middle effect (as first documented in “Lost in the Middle: How Language Models Use Long Contexts” by Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang).
Some companies have large codebases, sometimes monolithic, and often full of spaghetti code (very entangled code). One approach is to use microservices, multiple smaller codebases with smaller scope. It has multiple advantages, but the relevant one here is that a human working in one microservice can treat the rest as irrelevant, or at the least, black boxes. This means far less (human working) memory is needed to keep track of what’s relevant.
This applies to LLMs as well. I’ve seen estimates ranging from 3–10 AI tokens per line of code. If we assume 5 per line, then a mere 100,000 lines of code is 500,000 tokens (although with well-organized code the LLMs can selectively consume less of the codebase for a session). Most context windows top out at a few hundred thousand or a million tokens. Of course, the context must also include the request (the feature being used) and other relevant documentation.
Throughstone solves this issue in three ways. First, it encapsulates the code into modular components. Second, the thorough documentation of Throughstone reduces the amount of information that needs to be loaded into context. Together that means prompts are smaller, and so less likely to suffer from the lost in the middle and similar effects.
Third, base LLM calls are stateless. Every prompt submission, even within the same chat session, is effectively “new” to the LLM. Unlike humans, it doesn’t “remember” prior chats. Some newer tools are starting to add memory or persistent project context, but those systems still depend on what information is captured and reintroduced. The Throughstone process helps document decisions, both the what and the why, so there is memory across LLM sessions (and across human users). In other words, the extensive documentation helps every chat, every decision made by the LLM, to be made with all the relevant information so it stays focused on the goal, both the business goals, and the technical guardrails (e.g., coding standards).
The reason companies have CTOs and software architects (and the equivalent in other departments) is because we’ve been around the block. We understand how systems scale and break; we can design not just for today’s need but for the myriad of directions it can go in over time.
This is a subtle but key point. Good architects of early systems don’t try to optimize everything. Rather, a good system designer keeps as many possibilities on the table, for as long as possible, for a reasonable cost. An MIT administrator I used to work with always said, “options cost money.” So the trick is to keep options open—for scaling, for different ways the system may grow, for extended feature sets, etc.—but not in a way that costs significant effort or money now. The art of software design is in finding that balance.
Throughstone is my attempt to automate that wisdom for vibe coders. It systematizes the design process using a wizard-like onboarding process. What’s powerful about AI is that one of the first questions it asks is about the level of experience of the user, ranging from a non-coder to a very experienced developer; it then calibrates the rest of the interview process to ask questions appropriate to that level of experience. The output of the wizard is a series of documents including architecture documents, test strategy, security concerns, UI design, infrastructure plan, and more. These documents bind the whole process and system together as it’s built (hence, the name Throughstone).
A good start is valuable, but more is needed to successfully deliver and maintain software. The Throughstone system also provides a best-practices framework for implementing the code. After a user specifies some high-level milestones (e.g., MVP, V1, V2), the system guides the AI coding tool to break each milestone down into manageable steps. Each of those steps are turned into bite-sized substeps. This reduces the chance of both the human and the LLM getting too far off-track by enabling verification between steps and reducing error compounding.
The architecture documents mean the software gets built in a guided way, instead of the novice approach of throwing in one feature after another. Every step of the process references back to the system design. For example, as new code is added, the glossary helps it understand what the relevant entities are, the scaling information guides how to plan ahead for growth, and the testing guidelines remind the LLM to include tests along with the feature under development.
The system also guides the user to regularly run all tests and check for design drift. In the latter case, Throughstone will compare the code to the documents and vice versa. It’s fine if the system does diverge, but that should be intentional and documentation updated (or code updated if unintentional). By having a process the project stays on track and minimizes tech debt.
Generative AI has moved from being able to do autocomplete on a single line of code, to creating code blocks, and now to generating thousands of lines of code in a single session. What’s been tripping up many development teams is that underspecified requests produce workslop. The larger the amount of code being generated, the more likely it is to become slop unless stronger guidance keeps it focused.
Throughstone provides this guidance for software projects. Like any tool, it doesn’t guarantee success, but it should make it easier to do things right and reduce the risk of creating unmaintainable code. No doubt the tool itself and best practices for this type of development will continue to evolve, as will generative AI itself.
I hope similar guidance tools come out for other projects in different disciplines such as strategy, marketing, and sales plans. Software is typically both an upfront design process and an ongoing build process. Other disciplines may need more upfront planning and less ongoing iterative build than software does (e.g., in marketing, once the plan is created, it’s usually just monitoring the execution of it, as opposed to continuing to enhance it the way software continues to add features).
The Throughstone template itself could be repurposed. It just needs the domain expertise to define what should be created upfront and what ongoing process should continue it. The key is to compartmentalize the components of a project. That will allow an LLM to work on larger-sized projects by encapsulating the elements so an AI with a given context window can navigate the larger project while maintaining fidelity to the plan.
More generally, less-experienced people in different disciplines should look to their mentors for guidance about best practices. Those can be encoded in a process (which may be as simple as planning templates) and then delivered through an interactive session with an LLM.
For those who are developing software, non-coders through CTOs, please give Throughstone a try. I’d appreciate your feedback so we can continue to make Throughstone better. Even if you’re not creating software yourself, consider downloading the project and asking your AI to look at it and map the Throughstone methodology to your work.
Project information can be found at https://throughstone.org/. The GitHub repo is at https://github.com/mherschberg/Throughstone and you can copy the template from https://github.com/mherschberg/Throughstone/generate. With the quickstart guide you can be off and running in just three steps. Good luck, and let me know how it goes.
It’s critical to learn about corporate culture before you accept a job offer but it can be awkward to raise such questions. Learn what to ask and how to ask it to avoid landing yourself in a bad situation.
Investing just a few hours per year will help you focus and advance in your career.
Groups with a high barrier to entry and high trust are often the most valuable groups to join.
