Building an ERP with AI: A Computer Vision Engineer's Tale of Four Pitfalls

I've always wanted to build my own ERP system. For years, it was a distant dream. I studied computer science, but after graduation, I rarely wrote a line of code. The blueprint was in my head, but I couldn't type a single character. Then AI came along and made it real. Now, the system is in internal testing, automatically generating accounting entries from purchase, production, and sales documents. But it wasn't a smooth ride. Here are the four big pitfalls I hit, hoping to save you some pain.

The Architecture Trap: Letting Each Module Do Its Own Accounting

In the early days, I spent most of my energy on the UI and workflow. Buttons, field layouts, interactions—I thought that was the real craft. It wasn't until I started writing the core logic that I realized the real challenge: turning business documents into accounting language.

When a purchase receipt comes in, you need to record inventory and accounts payable. Production material issues move raw materials to work-in-progress. Sales confirmations trigger revenue and receivables. Each business action has its own set of debit-credit rules. In my first version, I let each module generate its own accounting entries. Big mistake. The balances never matched. Purchasing said payables were X, finance said Y, and everyone thought they were right.

So I changed the architecture. No module generates its own entries anymore. Instead, they emit business events, and a central accounting engine translates those events into entries. Each document type maps to a set of account rules. When a purchase receipt comes in, the engine debits inventory and credits payables. When a sale is confirmed, it creates revenue and receivables. All entries go into one unified ledger, and the balance sheet and income statement are just read-only projections of that table.

The essence of integrated business and finance isn't just bolting modules together. It's a translation pipeline from business actions to accounting language. The translation rules must be centralized and configurable, not scattered across modules. Once I had that, adding new modules became a matter of plugging in a mapping, and the entries just flowed.

The Model Hopping Mistake: Mixing AI Models

AI wrote the code, but I made a rookie mistake early on. I thought I'd use different AI models for different parts. Model A for one module, Model B for another. Seemed clever. It wasn't. The interface styles clashed. Naming conventions, structures, error handling—all different. Fixing A would break B, and vice versa. Bugs multiplied faster than features.

Even worse was letting different models take turns editing the same logic. One model wouldn't understand the context left by another, so it rewrote things in its own way, completely messing up the logic. Each snippet looked fine on its own, but together they were a mess. Debugging was a nightmare because every piece individually seemed correct.

After testing a few, I settled on GLM5.2 and used it for the entire project. One model, one style. Consistency made maintenance bearable. I could trace the connections and understand the flow. If something was off, the model's reasoning was consistent, so I could continue its thread. My advice: don't be greedy. One good model beats three smart ones. If I'd known that six months earlier, I'd have saved myself thousands of lines of deleted code.

Version Control: The Cliff Edge of Solo Development

I didn't use Git properly at first. I named files by date and stuffed them in folders—final, final2, real_final, real_final_no_changes. Then a big refactor broke the core accounting engine logic. I wanted to roll back, but there was no clean version. I had to rewrite it from scratch. That happened more than once. Each rewrite was days of work down the drain, and it crushed my spirit. After the second rewrite, I started doubting whether I could ever finish. Plus, it burned tokens.

Eventually, I got serious with Git. I created a branch for each phase. The main branch only contains tested code. Feature branches are for experimentation. If I break something, I can revert in ten minutes. I also made a habit: before wrapping up each day, I merge stable changes into main and leave unstable stuff on a branch overnight.

Without version control discipline, a solo project is like coding on a cliff's edge. You think you're saving time, but you're one mistake away from losing everything. When you're alone, there's no one to back you up. If it breaks, it's really broken.

The MCP Pitfall: Frontend Calculations Don't Survive Conversation

Once the system ran, I hit another wall in MCP integration. Traditional ERPs rely on menus and forms, which I find clunky. So I wrapped purchase, production, sales, and finance modules as MCPs, each with dozens of tools behind them. In WorkBuddy, I can just say, "Check how much of this material is left," and the tool runs—no need to open any UI.

Getting the tool granularity right was tricky. Too coarse, and a single tool does ten things, confusing the intent. Too fine, and dozens of similar tools confuse the caller. I settled on one tool per business action, with clear names.

The sneaky part was where I placed the calculation logic. In the UI, it worked fine because the frontend executed those calculations. But with conversational calls, the dialog layer only passes parameters and retrieves results—it doesn't run frontend logic. So the numbers came out wrong. The accounting entries didn't match, and the reports didn't align with reality.

It took a while to find. In the UI, everything looked perfect. Only when called via conversation did the numbers differ. I stared at the same transaction, checked the UI, then the dialog, and got two different figures. I was baffled.

The fix was to move all business calculations into the backend, no matter how small. The frontend and dialog layer only pass parameters and display results. I also added a double-check: the same number is computed via two different code paths, and if they don't match, the data is blocked from entering the ledger. A belt-and-suspenders approach.

Since then, the numbers have been consistent. The dialog's numbers match the UI's, which match the reports. The lesson: MCP interfaces must be fully backend-driven. Any calculation done on the frontend will evaporate when called through conversation. Conversational access forced me to clean up my interfaces, which was an unexpected bonus.

The Real Lesson: AI Gives You Muscle, Not Judgment

Through this journey, I've learned that AI loosens the reins on coding, but not on business thinking. Many people expect AI to think through the system for them. It doesn't. It writes the code, but you still have to figure out the business logic. That gap is where the pitfalls lie.

None of these four pitfalls was a one-shot deal. Each required rework, costing time, spirit, and those moments of staring at the screen doubting myself. AI provides the physical effort, but judgment is still on you. It doesn't make development easier; it just allows one person to handle the scale of enterprise software. The price is that you have to make every decision yourself, with no one to hit the brakes.

That long-held dream is now running in internal testing. It's not perfect. Modules need polishing, reports need tweaking, and sometimes it mishears my commands. But it actually generates a balance sheet on its own. And that's pretty cool.