A couple of weeks ago I wrote about the ten sites I’ve launched that Claude now autonomously manages, and today I want to expand on what I think is the most broadly useful lesson from that project: how to use a temporary human-in-the-loop phase of deployment in order to minimize the risk of bad outcomes when you go fully autonomous. If you haven’t read that post, I’d recommend doing that before you read this.

Putting Your Trust in AI

The less oversight you have of your AI, the more it can do for you, but also the more damage it can do. Even for someone who leans on AI as heavily as I do, it’s still tough to fully hand things off. That’s one of the reasons I decided to try my little autonomous website experiment — unlike with my actual business, here there aren’t going to be any consequences if this blows up in my face.

That’s not to say I’m not relying on Claude in my core business, just that I have historically done so in places where the consequences of failure won’t be dire. I also tend to automate processes where I have a reasonable level of oversight.

Sometimes that’s because I explicitly set up reports that keep me apprised of what’s happening (plus monitors that check to make sure those reports are going out; only the paranoid survive). In other cases, it’s because I expect to notice if something goes awry anyway — people are always surprised I have Claude do my bookkeeping, but I know the numbers of my business well enough that if it makes any significant mistakes, I’ll notice them on the P&L before I send it to my CPA at tax time.

So, as you think about letting AI take work over, consider a few key points:

• What’s the worst thing that could happen if something goes wrong here?

• Whatever the answer to the above question is: Can I prevent or mitigate that somehow? If yes, figure out the next-worst thing that could happen and repeat until you’ve found the worst non-preventable thing.

• If this is automated and things go awry, how likely am I to catch it?

• If the answer is less than “very likely,” are there systems I can put in place that will help me catch it?

If you really go through this exercise, you’ll find that most tasks, even those that seem high-risk at first, can be automated with minimal risk. Sometimes risk mitigation means only partial automation, but even partial automation can be a huge time saver.

Starting With a Human in the Loop

Even if you’ve thought through the task at hand and decided that the worst case is not so bad, it can still be somewhat unnerving to let AI handle it. So: baby steps. A good starting point is asking your AI to do whatever it is that you’d like automated once and checking the results.

Let’s take one of my current projects as an example: automating my Amazon Ads bid changes. I said earlier that I’ve saved automation for places where the consequences of failure won’t be dire, but this is the first real case where there are some very bad failure cases. Claude is now messing with a system that goes straight to my credit card; there are a series of API calls it could make here that would cost me many thousands of dollars. Thus it is my most measured rollout since I’ve begun throwing AI at my operations.

I have gone through several iterations of bid management with Claude, and each time, the first thing I do after it’s implemented some new logic to decide what bids should be is to have it run through one brand’s account and tell me exactly what bid changes it would make, with a brief explanation of how it came to each bid. If it seems wildly off from my intuition, I discuss with Claude and make changes as needed. If everything seems reasonable, I run it on a few more brands.

Once that looks good, the next step is to see what it would look like on an ongoing basis. I stay fully in control here, with Claude’s system telling me what changes it would make but never implementing without explicit approval. I also start with a single campaign type for one brand. Rolling it out to the whole portfolio would give me more information, but it would also be a lot for me to review. Better to ease into it with an easily digestible amount of information, so no problems slip by me.

If everything’s good, I approve, and if anything looks off then I discuss with Claude. Either it convinces me I’m wrong, or we tweak the system so the problem won’t reoccur. This continues until I’ve approved everything enough days in a row that I’m comfortable removing myself from the process. Since this is high stakes, I’m looking for a few weeks with no issues. That turns out to have been a good call, since the first few versions of this I implemented looked great for a week or so then began to go off the rails.

The good news is the current implementation is working smoothly. After the first few weeks passed without incident, I let it auto-apply changes without my approval while I expanded to more campaign types and brands.

Trust But Verify

That said, just because I’m letting it make changes without my approval doesn’t mean I trust it enough to remove myself fully. When you’re automating critical systems (and advertising is about as critical as they come for me), it’s best not to become entirely complacent. Problems can always crop up, and you’d rather know sooner rather than later.

There are plenty of ways to implement monitoring, but they all start with two questions:

• What could go wrong?

• If something did go wrong, what problems would it cause?

The first is helpful because if you know what issues to expect, it’s often straightforward to check on them. With my ads, the obvious thing that could go wrong is that the system would suddenly decide to make some enormous change to a bid. That shouldn’t happen, because it’s designed to make frequent, small changes based on the last 60 days of data. Even if there is a huge outlier day that gets dropped from the end of that window, it shouldn’t move the needle that much. That one’s easy to monitor: every day a script pulls all bids from the account and compares them to the prior day’s pull. If anything has moved >10%, I get pinged to investigate.

But of course there are other ways that problems can occur. What if, for whatever reason, the system just starts raising every bid 5% every day? Won’t trip my sudden move monitor, but also won’t take that many days before my spending spirals out of control.

This is where the second question comes in — in addition to checking for particular problems, you also need to be checking for the downstream effects of a broken system. In my case, that’s one of two outcomes: bids go too high and money is spent on ads with poor returns, or bids go too low and profitable sales are missed.

Thankfully, neither of these is hard to check for. I’ve got a daily update email, and for each brand that’s running automated ads, it includes 1d/7d/30d spend and return on ad spend (ROAS), plus period-over-prior-period comparisons for each duration. This is both generally useful to me as an indicator of each brand’s health and the best way to spot problems on either a short- or long-term basis.

AI Overseer

The actual processes I’ve described so far are all deterministic. Claude writes the code, but my autonomous ad engine isn’t calling out to AI models, nor are the monitors doing anything but rote checks of numbers.

The next level of monitoring is where dropping in AI is both useful and lots of fun to watch (at least for the sort of person who reads this Substack). Consider what you’re doing when you review the information that your monitors send you.

When I get my daily email, I’m eyeballing the numbers to see if they seem far enough off to be worth investigating. This involves considering a few questions:

• Are they much higher or lower than I expected?

• If they’re out of the expected range for some period of time, how much data do I have during that period? If the numbers are off for a day, it’s probably just normal variance. A week might be worrying, but it depends on how high-volume the particular brand is.

• Did anything happen during the period that would likely explain the numbers? Something like Prime Day, for example, will spike spend in a way that’s not at all a concern.

If my initial assessment tells me something’s off, then I go in and investigate in more detail. Is that issue across the board, or is it just one campaign that’s gone off the rails? Does it appear to be because of recent changes, or have things gone wrong for no discernible reason?

All of these questions, of course, can be answered by AI, so get your favorite model to monitor the monitors. Each week, I have Fable review the data and answer all of the above questions. If anything is off, it does a more thorough deep dive than I possibly could and comes back to me with a full assessment. If the issue appears to stem from the automation system, it does a root cause analysis and presents me with its findings and suggested fixes. I’m that boss who loves to say, “Don’t come to me with problems, come to me with solutions,” and Claude is happy to oblige.

It also gives me a general take on how the system is working. The main objective is to get each campaign’s ROAS to a target; are things heading in the right direction each week? How is our ad spend looking? Are the results what we expected? Do we have sufficient data to draw conclusions yet?

As you’re implementing this, it’s best to start by keeping yourself in the loop. Have the agent do its analysis, you do your own, and where they diverge, figure out why. If the AI analysis is bad, adjust the prompt or the data it receives and have it take another run at it.

Over time, you’ll build up a specific list of checks for it to run that mirrors your own process. Much like with deterministic monitors, I recommend approaching this list at a couple of levels — first, specific things that could go wrong, and second, the outcomes you want to avoid. You can also experiment with throwing in a catch-all, “Let me know if you see anything not captured elsewhere in your instructions that you think I should be aware of,” but in my experience it really tries too hard to fulfill that request and flags non-issues.

With that in place, you can move up in the management chain. Your day-to-day operational responsibilities have been given to Claude; until it brings you an issue to deal with, you can focus on the big picture — improving the whole system. In my case this involves thinking about whether I should be trying new types of ads or different creative. It also means keeping tabs on the competitive landscape and broader needs of the business to see if it makes sense to try sacrificing margin to gain volume or vice versa.

I’ve actually tried implementing an agent to handle those higher-level tasks for me, but the reality is these are simple businesses that don’t really need regular changes in strategy — that’s why I picked them.

Recursive Self-Removal

The broader lesson here is that the way to get AI running your business is one step at a time. Think of it like handing some of your work off to a new human employee who’s smart but not 100% trustworthy. Start with a task that you understand well, document extensively (both the task itself and the context of how and why you make decisions related to it), give it to an agent, and monitor the results. When you reach the point where you’re approving everything the agent does, take yourself out of the loop and add some monitoring.

How long should that process take? It really depends on what you’re trying to automate. If it’s straightforward and low stakes, maybe it’s a day or two. If it’s mission-critical with lots of hairy edge cases that only pop up intermittently, you probably want to stay in the loop longer.

It also depends on how easy it is to monitor. If there are very obvious and immediate signs when problems occur, then you might as well put in good monitoring and wait. If the sorts of problems you might encounter are subtle, difficult to notice and likely to quietly accumulate until something blows up, more vigilance is required.

After you’ve automated the process, think bigger! Can you expand the breadth of your new system? What tasks are you now doing, and can you automate those too? The answer to the latter question is increasingly yes; if it can be done on a computer and you can document it well, Fable can probably handle it.

So: automate what you’re doing, move up a level, automate that, and then repeat ad infinitum until you are a meaningless figurehead atop an organization of agents.

I have neither expertise nor a particular interest in water conditions (except that as a native Californian, I say the obligatory “well, you know we really need it” any time someone complains about rain). Rather, this was a test to see how much of the work of getting the site up and running Claude could handle on its own.

The answer was: a lot! Thanks to Cloudflare’s agent-friendly tools, Claude had everything needed to buy the domain and get the website live. From a technical perspective, it had almost all of the tools it needed (though yours truly still had to grab an API key to get government data and set up a Buttondown account to send emails).

But since this was a first go, I was still fairly hands-on with the editorial side of the project, with plenty of big-picture feedback about site structure and little nits about design. The next step was to try this while truly cutting myself out of the process except when my involvement was absolutely unavoidable.

More Sites!

With The Waterline complete, I had a solid template for how to create this kind of site. Because I started this project by having ChatGPT generate a list of 50 topics, I also had other options ready to go, so I told Claude to take the other nine of the top 10 and spin up agents to build all of those in parallel. After that promptly froze my computer, I told it to cut that back to three at a time.

The process was, roughly:

• Take the first unused idea from the initial 50-topic report

• Search for and evaluate relevant information on the topic that can be used on the site, particularly freely available government data, and have ChatGPT do the same in parallel using Claude in Chrome

• Review both outputs, and build a plan for the site’s content, including:

  • What sources of information will be used and when they’re updated

  • The structure of the site — how the data will be grouped and presented

  • Any relevant evergreen articles on the topic that should be included

• Build the site

• Find relevant domains that are available for purchase and present them to me

• Once I pick a domain, purchase it and launch the site

• Create GA4 and GSC properties for the site and set them up

• Have ChatGPT review the site, checking for:

  • Discrepancies in the data or other factual issues

  • Whether the tone and language of the site are appropriate given the audience (particularly with regard to appropriate use of industry terms)

  • Any other issues with content, layout or structure

• Review ChatGPT’s feedback and make updates as appropriate

• Configure the Buttondown account and newsletter template

• Set up recurring jobs to pull fresh data from the sources, update the site and send out a newsletter the day after new data are released

This went well! The ChatGPT feedback step was really critical to the whole endeavor — Claude would frequently end up with inconsistent math across different pages in the same site or would miss clear opportunities for content. In any case, we now have:

• The Ration

• The Scramble

• The Run

• Stumpage

• The Fill Line

• Know Your Onions

• Monday Diesel

• The Dwell

• The Russet Report

The only places I had to step in were logo creation, Buttondown account creation and domain approval. In the future, I’d just instruct Claude in Chrome to have ChatGPT generate the logo. I’d also switch from Buttondown to something like Amazon SES, where I can sign up for an account once and let Claude build out an email system that doesn’t require a new account for each site. Domain approval will have to stay on my plate until they come up with a model that has better taste.

Automated Improvement

At this point the plan was to let them sit for a while and see if they started ranking on Google. To that end, I had Claude set up a weekly email report giving me numbers on traffic and appearance in search for each site.

Started getting a trickle of traffic right away; pretty neat! I looked through some of the queries, checked the relevant pages on the sites, and found some opportunities to both build new pages and expand the ones we had. Claude dutifully ran with my suggestions, and our rank for those queries was up the next time I checked.

But of course the goal here is for Claude to run this whole thing without me, so the next week when the email landed, I had it do its own analysis and give me a list of suggested improvements to the sites. They were mostly solid, so I offered a little feedback and let it get to work.

Once that was done, I set up a weekly job for Claude to generate with the list of proposals and email those to me for review.

Again, mostly good, though a few bad ideas. One general category of miss was forgetting who the site’s audience was (e.g. The Fill Line ranked for queries related to “propane usage calculator,” but Claude’s suggestion was to create a calculator for consumers to figure out how much propane their home appliances would use, even though individual users of propane are clearly not the audience). Another was reviewer Claude’s tendency to get pedantic about editorial rounding of numbers (“The headline says it’s 94 cents, but it’s actually 94.4 cents”).

I gave Claude feedback with each round of suggestions, which it saved that to an instruction file for the reviewer. By weeks three and four, all of its suggestions were good, so at that point I automated the whole process.

We now have 10 sites that autonomously self-improve each week based on the data available to them. Traffic is still at a trickle but visits and search impressions are up and to the right on most of the sites.

Will this ever turn into something substantive? No idea. I’m just planning to let it simmer for a few months and see what happens. But it’s entirely possible that eventually I’ll get a reasonable number of newsletter subscribers on one or more of these, get a sponsor or two and have myself a tiny little fully Claude-operated business. And even if that doesn’t happen, it’s been a fun way to think about the scope of project that Claude can handle on its own and what it takes to get it to the point of autonomous management.

Limitations

These sites are far from perfect! I’ve been pulling them up as I write this, and I’ve already found a few issues that I flagged to Claude to fix. Some are major, like seeing this on the homepage of The Run.

It turns out this is the result of Claude trying to copy a pattern used in other sites. That problem has proven to be a little bit of a theme, as there is an underlying engine that they all use. On more than one occasion I’ve had to admonish it to remember that it’s never acceptable to compromise the quality of the site in order to make it work with the engine. Code is free! There are no tradeoffs needed for this sort of thing! Modify the engine or customize it for the particular site! Build the site from scratch if you have to! That’s very clearly laid out in the project instructions, but it doesn’t entirely stop Claude from clinging to a vestigial notion of engineering efficiency.

I should probably have ChatGPT take a separate weekly adversarial pass over all of the sites, but there’s no way to invoke ChatGPT using my subscription and I’m too cheap to pay for API usage. Actually writing that inspired me to double-check, and it turns out I can set up a ChatGPT scheduled task to go through them all and email me its findings. Nice! I’ll set that up, review the emails myself until I confirm it’s reliably doing a good job, then have Claude pick them up straight out of my inbox and make any improvements automatically.

The other big issue is that what these sites need to get some real traffic is backlinks. For the SEO-uninitiated, Google tends to rank sites higher if they are linked from other sites that it thinks are high quality. Getting these links mostly involves reaching out to places where your content is relevant, so I did have ChatGPT give me a list of good targets for each of the sites. The problem is that I’m opposed on principle to having AI do email outreach, which makes this difficult to automate. For now, I’m planning to fire off some of those emails myself to see how they’re received, and if I can get some links, then I’ll figure out what to do next.

So: lots to do! First things first, though — run the playbook I’ve developed, starting with getting the Amazon listings cleaned up. These ones were actually better than what I usually inherit, but they had one big issue, which was a total lack of coherent brand identity in the images. To give you an idea of what I mean, here are listing images from four different products:

In fairness to the guy who sold me the brand, this is the kind of thing that’s easy to ignore when you own it (the listings were converting fine, so might as well spend time on other things) but really stands out when you have just acquired it.

I wanted to get these all unified into a single style, and I liked the backpack one in the bottom left of the above image. OpenAI’s image model is currently the best on the market for this sort of thing, so I was definitely going to use that. Two options there:

1. Generate each image myself using ChatGPT

2. Have Claude handle it using the API

This was something like 100 images, so the first one was going to be a day’s worth of work, while the second would’ve cost something in the ballpark of $30-100 depending on how many generations I had to do for each image. The obvious answer here is that my time is clearly worth the cost of having Claude do it using the API.

But as someone who spent more than a decade as a PM at early-stage startups, I am something of an expert at finding non-obvious solutions. After noodling it over a little bit, I figured out the way to get the best of both worlds — having Claude do the heavy lifting while still making use of my ChatGPT subscription to cover the cost. Claude in Chrome!

I laid out the problem to Fable, gave it several of the backpack listing images to convey the style that I was looking for, and then told it to spin up a Sonnet subagent using Claude in Chrome to update this image:

The result?

Not bad! You will notice Claude took some liberties with the text, plus ChatGPT adjusted the photo to give it a wider crop and new background. That led me to take a closer look at exactly what Claude had done. I was surprised to find that it had given the original image to ChatGPT but hadn’t included any of the backpack images as style references, opting instead to describe the style in text. Didn’t nail the subtitle font exactly, but it’s certainly close enough for these purposes.

With that, I had it do the rest of the images for that product. To make life easy for myself, I instructed it to use its existing access to all of my various software to grab everything — Amazon Seller Central for the listing images to be updated, and Google Drive for all of the lifestyle photos it might need. Beyond that, I gave it some general best practices:

The objective is to create a set of new, high-converting listing images in the style of the backpack images. You don’t have to copy the existing listing images exactly in terms of text or photos, though you may if it makes sense to do so. They should retain the same general message as the original images even if the text changes. You can use the existing photos or select new ones from Google Drive.

From there, it spun up a couple of Chrome tabs (I’m not sure why two vs. either one at a time or all six at once — over the course of all of this work it varied between two and three), launched ChatGPT and got to work. Once it pinged me to let me know it was done, I checked the folder it had created and realized that I didn’t remember what the original listing images looked like well enough to review these. I was going to pull up the Amazon listing, but of course it would be silly to do work better handled by my trusty assistant. Instead, I had Claude whip me up a webpage with the original listing image, the new image and the photo it had used for the new image (if it had pulled something from Google Drive rather than just using the original as the input).

The results were good! I gave it some feedback on a couple of images, which it fixed with another pass, and we had one product done. Claude uploaded those to the Amazon listing at my request, and then it started on the rest of them.

Completing the remaining images took a few hours — the big downside of Claude in Chrome is that it involves Claude frequently pausing for five seconds, taking a screenshot and then reviewing that screenshot, which eats up a lot of time for long-running tasks. Still, if you have a case like this where you intend to fully delegate the task with no involvement until it’s complete, a long runtime doesn’t really matter.

It came back with a pretty similar success rate to the first set of images; something like two-thirds of them were immediately good to go. For the rest, I gave feedback and had it do another round of edits.

I will note a couple of failures on my part here. First, I should have had it build a feedback mechanism into the page where I was reviewing the images. I was just giving it poorly punctuated paragraphs of notes in the command line as I went down the list of images, which was mostly fine but wasted time and lacked precision. If I had done it on the page, I wouldn’t have had to type a description of each image before the feedback, and I also could have had it generate a UI where I could annotate the image to clarify exactly what needed changing.

The second error on my part was failing to convey to Claude the lessons I’ve learned about how best to prompt ChatGPT for images. The big one here is that if you repeatedly edit the same image in the same conversation, things go downhill quickly. First, the photo you’re using drifts away from reality with each generation (I realized Claude was reusing the same conversation when the logo started disappearing from the product). Second, the image model gets stuck in its existing context and just can’t make the changes you ask for. Claude described this to me as ChatGPT refusing to follow instructions, which I found weirdly amusing.

There were a couple of images that I ended up handling because they were taking too many tries from Claude, but besides that, I didn’t generate anything myself.

Lessons Learned

This was both a fun experiment and more productive than some of my other AI-enabled harebrained schemes; the images are live on Amazon. It was also, for my purposes, ideal. I got the same result I would’ve via the API without having to spend money!

That said, it was a wildly inefficient use of tokens and another example of the oddities of the labs’ billing models. Anthropic decided to start charging API prices for claude -p and the Agent SDK precisely because it enabled you to use Claude programmatically without paying API prices (though they reversed that decision before it actually kicked in). Same thing here: OpenAI will allow you to generate “virtually unlimited” images for $20/month in the ChatGPT UI but none using the API as part of your subscription. This leads to very dumb arbitrage opportunities that use vastly more tokens than the clearly technically correct way of doing things. But hey, I enjoyed figuring out this silly little loophole, and I think that if he ever read this post, Matt Levine would be proud. Or at least amused.

A surprising outcome of this project is that it seems to have permanently lodged the idea of using Claude in Chrome in Claude (in Code)’s brain. I can’t think of a single time it ever suggested the browser as a tool beforehand, but in the last couple of weeks it has not only suggested it but also just gone ahead and fired up a browser agent of its own initiative. A few days ago I asked it via the mobile app why I couldn’t see some web properties in Google Search Console. After a few minutes it came back and told me that they just needed to be added manually from the UI, but not to worry, it had opened a new tab in Chrome, found that I was already logged into the relevant Google Account, and added them for me.

The great news about this is that nothing can possibly go wrong with an AI agent using logged-in sessions in my browser without my approval!

Models are profitable at the token level; each token can be sold for greater than the cost of inference. The real question is whether margin remains positive after training costs are amortized over the useful life of the model.

The pace of new model releases is rapid and increasing, and if customers move to each new version immediately upon its release, then long-term profitability seems unachievable for the labs.

So how do the labs become profitable? Three ways:

• Selling to the enterprise, where switching costs are meaningful and incentives favor inertia, so the useful lives of models run longer.

• Monetizing the attention of their consumer bases with not only subscriptions, but also the most tried-and-true of internet business models: ads.

• Shifting model economics as SOTA models achieve a level of intelligence necessary for only the most advanced projects.

AI in the Enterprise

Switching models is trivial for consumers; the reality is different for the enterprise. The prompt incorrectly assumes low switching costs because the act of swapping out a model is easy, particularly in comparison to traditional SaaS. But the transition is only part of the cost — performing the evaluation to determine whether to change is expensive for traditional, deterministic software. For probabilistic software, it’s not only more complex, but also nigh impossible to do with 100% confidence.

There are cases in which a switch will be worth it, but only where leveraging the most advanced model available adds value. If you’re pushing the boundaries of science, then you’ll want Opus 4.8 the minute it’s released. For most white-collar work, though, there’s a cap on how much intelligence is useful. Von Neumann wouldn’t do a better job at accounts receivable than the 100 IQ employee in the role now.

As models improve, the number of cases where upgrading to SOTA will yield benefits shrinks. The current generation of models is already sufficient for large swathes of knowledge work; it’s plausible that businesses will still use Opus 4.7 a decade from now.

When model capabilities aren’t a reason to switch, the other incentive is cost. If you can cut your token bill down (net of switching costs) without sacrificing functionality, why wouldn’t you?

Employee incentives create a deeply favorable environment for incumbents. If something goes wrong with the software you’re already using, it’s nobody’s fault. If an employee pushes to replace that software and the replacement fails, it’s that employee’s fault. With probabilistic software, even an intra-lab switch from an Opus to a Haiku carries risk. Switching to another lab’s model carries more. Unless management is pushing cost cutting, employees rarely get rewarded for saving money, so there is no reason to try.

The prompt identifies distillation as a threat, but that wrongly assumes that capabilities and cost are all that matter for purchasing decisions. This is rarely true, especially for products that function as critical pieces of infrastructure. Most distilled, open-source models are from China, which is heavy baggage in the current geopolitically fraught environment. They also lack the salespeople, support and other operational infrastructure of frontier labs, and those matter to enterprise decision makers. Nobody ever got fired for buying IBM OpenAI.

Consumers

Enterprises provide consistent, sticky revenue to drive model-level profitability, but when considering the path to lab-level profits, one mustn’t ignore enormous consumer user bases.

Google and Meta are advertising juggernauts because they each possess valuable information and users’ attention. Google owns high-intent, and thus highly valuable queries. Meta simply knows everything about its users. AI chatbots are the best of both worlds — they have attention, plus chat history with a wealth of personal information, and people frequently ask them the sort of high-intent, highly-monetizable questions.

Meta’s annual ARPU is roughly $60. The average app time/day for an Android user is 67 minutes, which we’ll take as a proxy for average time/day for all users. ChatGPT.com’s average visit duration is 12:34. ChatGPT has 900M WAUs. For simplicity, we’ll assume WAUs visit on average once/day and ChatGPT could achieve revenue/user/minute equal to Meta. That gives us ChatGPT ARPU of $11 — approximately $10B in potential ad revenue. These are unofficial numbers and back-of-the-napkin math, but they’re sufficient to support a meaningful ad business. Add 50M subscribers for roughly $12B annual revenue and you have a real consumer business.

The Intelligence Inflection

Between longer model amortization periods for enterprises and substantial consumer revenue, labs have a clear path to model-level profitability. Still, if they continue with larger training runs, they may remain unprofitable, especially because each new model has value for fewer use cases.

The reality is that once we have a model sufficient for all but the most cutting-edge applications, the economics must shift. Beyond that threshold, models will need to pay themselves off more quickly. That’s feasible, because they will be employed solely by governments and large corporations for their most valuable work.

This will necessarily create a market equilibrium. Sophisticated buyers of these models will understand their value and be willing to pay accordingly; labs will understand their economics. If there isn’t a token price at which both labs and customers can operate profitably, labs will slow the pace of training runs to create time for models to cover their costs at prices the market will bear.

Conclusion

So when does the profit start? Dario suggests that when “the models can’t get larger… then it’ll be a large, very profitable business.“ This is directionally correct, but models will plateau in value for most use cases before they can’t get larger. At that point, the business changes from building broadly-available, general-purpose technology to offering an expensive, niche product to a handful of deep-pocketed customers. When that switch happens, labs will have two sets of products: sticky legacy models continuing to generate profit after their training costs are paid off, and SOTA models with pricing power.

Model and Graphs

This model helped me think through what the above-described scenario might look like in practice with actual numbers.

Share

In any case, this seems like an excellent time to address one of the biggest challenges of using LLMs: figuring out which combination of model, settings and harness is right for the task at hand. This is one of those topics that makes me realize how difficult it must be for normies to keep up with the latest goings-on in AI. You’ve got Haiku, Sonnet, Opus and Fable. Each of them can operate at a thinking level of low, medium, high, xhigh and max. There’s also ultracode, but ultracode is actually xhigh with more subagents and workflows. Then you’ve got the original Claude on the web, as well as Cowork and Code.

How on earth is anyone supposed to figure out which of those to use? I spend hours every day with Claude in all of its various forms, and I still couldn’t tell you whether Sonnet on max is better than Opus on low for any particular query. And then of course we’re at Opus 4.8 but recently went from Sonnet 4.6 to Sonnet 5, so that upends whatever intuition I’d managed to build up.

Anthropic has charts like the above, which are sort of helpful, though given the incredibly jagged nature of AI capabilities, they really aren’t that useful in telling you what you should use for the specific thing that you need to work on right now. The only way to actually know whether one model beats another is to run the same task past both of them, ideally multiple times, and judge the outputs. But of course if you want to get work done instead of evaluating models, that is a terrible use of time and you just kinda have to take your best guess.

Up to this point, my strategy has primarily been to just pay for the Max subscription and use Opus on max thinking all the time. Inefficient on both cost and time, but I’m not going to quibble over a couple of dozen dollars a month for the most powerful technology in the history of mankind! Plus I’ve usually got a few tasks running in parallel, so slower-than-necessary responses don’t bother me.

Unfortunately Fable really blew up that plan — that thing uses tokens like it is nine-year-old me playing that one Ninja Turtles arcade game at Chuck-E-Cheese.

I had a number of projects that I really wanted to throw at it during the brief window of subscription availability, but it rapidly became apparent that I was going to have to ration my usage.

Which Model, Harness and Thinking Level To Use

When I say that prior to Fable’s launch I was always using Opus on max, that’s not quite true. I’m really just talking about my conversations in Claude Code and web. That does not mean there aren’t valid uses for other models, though, particularly when you’re using AI within a product or recurring job.

• Haiku: Useful for simple tasks where you need intelligence embedded in an application. I just signed up for a couple of services that send daily emails with requests from reporters looking for sources in order to get free PR for some of my products. I’ll be using Haiku to filter these down to the ones I care about and tag them with the relevant product.

• Sonnet: If you’re concerned about token usage, Sonnet is pretty good at simple coding tasks and research questions. I typically use it like Haiku, within applications but where they need a little more intelligence than Haiku provides. In the reporter email filtering application, I’m using Sonnet to draft responses to queries (though I will definitely edit those prior to submission).

• Opus: My daily driver for most things.

• Fable: If you’ve got deeper pockets than I, just use Fable for everything. There is something about it that’s difficult to quantify, but it just feels deeply intelligent. If you want to keep your costs under control, though, use Fable for planning and review, not execution. More on that below.

From a harness perspective, I haven’t done enough with Cowork (or any of the many external harnesses like OpenClaw and Hermes) to comment, so I can really only tell you about Claude web and Claude Code.

The major difference between those two, at least in my experience, is that Code is best at execution — writing code, making API calls, managing files — but lacking in creativity and not much of a problem solver. Claude web, on the other hand, feels clever in a way that Code is not. You give it a problem, and it will be much more thorough and thoughtful as it considers possible solutions. Claude web is also better, though meaningfully worse than ChatGPT, when it comes to web search.

As to the final variable, thinking level, I really have very little sense of when to change it. I tend to just pick a model and use it on max, except that every once in a while I’ll drop Opus to xhigh or Fable to high/xhigh if it intuitively feels like the problem doesn’t require the maximum level of thinking. That is an unsatisfactory answer, I know, but sometimes you have to limit the variables available to you to simplify your life. For what it’s worth, I think having five levels of thinking is a terrible user experience choice by Anthropic, but it’s easy to opine from my Substack while they’re creating superintelligence.

Making Them Work Together

With all of that in mind, let me get a little bit more specific about how I approach a new project.

A few weeks ago, I listened to an episode of the Odd Lots podcast (highly recommended), in which they interviewed the founder of HayWire, a website that tracks hay prices in the US, which turns out to be much more complex than you might guess. The first version of the site started with him pulling publicly available data that was buried in USDA PDFs and turning it into something easily accessible. One of the hosts made an offhand comment about how there were almost certainly other similar sources of government data that some clever entrepreneur could use to create the HayWire for other industries.

I am one such entrepreneur! Plus my thoughts are now largely filtered through the lens of, “would that be an interesting thing to test AI capabilities?” Obvious yes here!

First off, gotta find these opportunities. For this, I used ChatGPT. I’ve focused on Claude models to this point because they’re 90% of my usage. Still, the one thing you should really know about ChatGPT is that it is far and away the best model for research. It will search more pages, spend more time on each search result and come back with higher-quality, more comprehensive answers than Claude.

For a question like this where you really want it to scour the internet and draw conclusions from the information it finds, you should use Deep Research. If you have access to it, setting the thinking level to Pro will get you the best result of any model out there, hands down.

Before starting on that, it’s helpful to work with ChatGPT to get a good prompt. I told it about HayWire and the Odd Lots episode, then explained that I wanted to get a Deep Research report identifying other, similar opportunities. It asked me questions about what exactly would make for a good topic, then turned that into a fairly extensive prompt, which I won’t quote here because it’s 117 lines, but if you’re interested in reading it in full, here you go.

I launched the Deep Research query, and it searched for 20 minutes through 384 sources. It came back with a 20-page report covering 60 possible topics, helpfully stack ranked according to my criteria.

With that, it was time to get building. The goal here was to get a few of the top-ranked options built with as little input from me as possible, then see how much of the marketing could be automated as well. Big-picture planning of things like architecture and marketing strategies is exactly where Fable shines, and they’re also where you want to use Claude web.

I gave it the full report and the following prompt:

Read this report and lmk if you have questions. The objective here is to pick out the best 3 - you can use the rankings in the report, but also really optimize for free/easy ways of marketing and the ability to automate as much as possible with AI, from pulling the data to formatting/structuring it to writing a newsletter/updating a webpage to marketing via posts on social media, email outreach, etc. This is my experiment in trying to have AI basically run a business and get it to profitability. Ultimately the goal here is to have you build out three plans including specs for all code that claude code needs to write to handle everything related to the business. Also note anything that is not possible to handle via AI/code that I would have to do manually.

It did some additional research on the accessibility of the data sources and reasoned through the marketing side of things, and it ultimately landed on PropaneWire, WesternWaterWire and OnionWire (I will note that ChatGPT just slapped Wire on the end of everything in the original report, copying HayWire without understanding why it’s clever).

The selection criteria were good. On the data acquisition piece, it prioritized data that are publicly available via API or PDF but still require a lot of digging and/or structuring that’s annoying for people but ideal for LLMs. For marketing, it decided to focus on opportunities that were most SEO-friendly, since that’s the most automatable of all possible marketing strategies.

I didn’t try the same question on Claude Code, but I would wager heavily that it would have ignored SEO and focused on social media and email outreach since those are what I gave as examples in the prompt. It also wouldn’t have been as thoughtful about which newsletter options to choose. Fable explained why it dropped some options that were towards the top of the list, like AlaskaSalmonWire because it’s seasonal. That’s just not the sort of critical thinking you get with the Code harness.

That’s the big weakness of Code — it’s very good at executing what you tell it, but it doesn’t think creatively about problems and almost always sticks to the specifics of your request, even in a case like this where it was clear that I was giving examples of marketing channels rather than an exhaustive list.

Fable asked me a couple of questions about my architecture preferences and, after getting my responses, built out the full plan. It opted for a single engine that could power each newsletter with minor adaptation for the specifics. I suspect Claude Code would have just gone with a separate codebase for each one. If you go to a surgeon and ask about a health problem, he’s probably going to recommend surgery. If you go to Claude Code and ask about your project, it’s going to write a lot of code.

With the spec ready, I fired up Claude Code, selected Fable and had it give the whole thing a review. One advantage I’ve found to using two different harnesses is they’re surprisingly good at catching each other’s errors and suggesting improvements (though this is more true if you’re using Opus, since Fable makes fewer errors in the first place). It pointed out that we should probably lock down a domain first, so I instructed it to use Cloudflare’s domain API to find me options.

First it told me that Cloudflare only has an API to fetch domains that you already own. This is wrong and probably the most common error I run into when using Claude Code; it is incredibly lazy about retrieving information on APIs it needs to use (which seems bizarre given the code-centric purpose of the harness) and will frequently tell you things that are absolutely untrue. Sometimes it’s that an API doesn’t exist, and even more frequently it will send a malformed request, get back an error response and conclude that the endpoint is broken.

It still does this despite my note in CLAUDE.md that “YOU ARE BAD AT EVALUATING APIS. IF I TELL YOU AN API EXISTS, IT EXISTS AND YOU SHOULD NOT TELL ME OTHERWISE UNTIL YOU HAVE EXHAUSTIVELY SEARCHED. IF YOU GET AN ERROR RESPONSE BACK FROM AN API, YOU SHOULD ASSUME THAT THE ISSUE IS WITH YOUR REQUEST AND NOT TELL ME THE API ENDPOINT DOES NOT WORK UNLESS YOU HAVE DONE EXTREMELY THOROUGH TESTING.” Does all caps help? Apparently not, but it makes me feel better.

I specifically asked for a broad range and some that were clever and punny. This did not go well — even where it actually came up with sort of good ones, it added Wire on the end of all of them (no, Claude, TheWaterwayWire.com is not going to be a winner here). This is, once again, classic Claude Code. The spec had Wire at the end, and it locked in on that.

Back to Fable on the web, which did reasonably well:

Western water — The Watermark (the mark showing water level — literal — plus the "high/low watermark" milestone idiom; closest to Haywire-quality), The Waterline (the level line, plus "line" as a news dispatch), Drawdown (the literal term for a falling reservoir, plus the finance sense — note Project Drawdown exists), The Runoff (snowmelt runoff drives supply, plus election/general runoff).

I gave these back to Fable in Claude Code, it searched for domains, and we settled on the-waterline.com.

With that, I told it to start building, with specific instructions that it should spin up subagents using lesser models for all of the code and restrict its own role to orchestration and review. This really is the key to using Fable to build things without breaking the bank. I highly recommend that you add a note at the top of your CLAUDE.md instructing Fable to only ever have subagents with non-Fable models write code, lest you forget to mention it and burn through your monthly token allocation in an hour.

It fired up some Opuses and Sonnets, I walked my dog, and when I got back I had a website, as well as scripts to pull data from public sources and update the site weekly. The design even looks much better than what Claude typically delivers, though I’m not sure how much of that is attributable to Fable vs. the fact that it clearly took a lot of inspiration from HayWire.

The weekly scripts to pull data and add new articles are where the lower-end models come in. Two of the data sets come only in the form of free-text press releases, so Sonnet is used to parse those in order to extract the relevant information. It gets strict instructions to only cite things exactly as they exist in the text and never infer anything. If I wanted an extra layer of security, I would run a second pass where a Haiku instance gets Sonnet’s output and searches the releases to confirm the numbers are there, but so far I haven’t seen any issues that would necessitate that.

Sonnet then writes the weekly newsletter, and Haiku writes some explanatory notes beneath each graph on the site. While I am generally against using AI for writing (every word and em dash here is my own, though Claude serves as an excellent copy editor), this is formulaic stuff. Its only purpose is to clearly convey information, so it doesn’t need to have a compelling authorial voice or do any thoughtful analysis. For the sake of turning data into structured prose, Sonnet is perfectly sufficient. The graph notes are even simpler and could probably be handled by deterministic code, but there are enough variations that it would be a little annoying, so Haiku’s easier.

Once everything was done, I turned back to Fable for one last pass (on the web, thinking level high because it’s just a review, not complex problem solving). I gave it the URL, reminded it of the general goal and asked for feedback. It came back with some solid notes:

The allocations page has duplicate entities with conflicting values. You've got "CVP South of Delta Irrigation" (20%), "CVP South of Delta Agricultural" (25%), and "CVP South of Delta Irrigation Water Service and Repayment" (20%) as separate entries — likely the same contract class parsed under different names, one stale. Same pattern on the M&I side (65% / 70% / 75% across three near-identical names). Dedupe these.

A few like this — the data come from different sources and use slightly different names. Opus wasn’t smart enough to figure out they were the same, but Fable picked it up.

Streamflow URLs are raw USGS gauge IDs (/streamflow/09380000). Zero keyword value. Use descriptive slugs like /streamflow/colorado-river-at-lees-ferry (keep the ID at the end if you want uniqueness). This is your highest-leverage SEO fix.

Several SEO items like this. Technical stuff that I wouldn’t have found but that is important given that SEO is key to the project.

This is tremendous value from Fable — didn’t take a ton of tokens, but identified a number of clear, specific issues. Upfront planning and post-completion review get you real bang for your buck.

Just Do What Works For You

I hope this has been useful in helping you think through how to select a model for whatever task you’re working on — I always find concrete examples to be better than theoretical advice or comparison charts. AI is a weird technology that is both very simple and deeply complex to use. With four models, three harnesses and five thinking levels, you have 60 options, and that’s ignoring the older versions of each model and ultracode option.

That’s just one family of model. Once you throw in GPT, Grok and all of the open-source Chinese models, it’s nigh-impossible to keep a good handle on the totality of the AI landscape. But that’s okay! The best advice I have is to figure out what works for you given your constraints. If you can spend $100/month on a Claude subscription, you can likely just use Opus at max and only put thought into when it’s worth moving to Fable. If $20/month is your ceiling, you can probably get similar results by being a bit more thoughtful about when to use Sonnet and reduce the amount of thinking.

In any case, don’t forget that if for some reason you need to stay up to date on water levels and drought conditions in the western half of the US, just come pay a visit to The Waterline.

My second apology is because this post is about using AI to add slop to the internet. I do not feel good about it! But the older I get, the more I prioritize practicality over principles, and the reality here is that I am not moving the needle in the grand scheme of slop. If throwing some AI-written pages up on an e-commerce website blog drives a few extra sales (and gives me a new task on which to judge AI capabilities/write a Substack post), then that’s what I shall do. I beg your forgiveness.

My first foray into e-commerce was during the pandemic, when I decided to start a business making dog treats. That business is Cooper’s Treats, and it sells frozen dog treat mix — basically a powder in a jar that you mix with water and freeze. I began with a Shopify store and found customers mostly through Meta ads.

I knew that SEO was important, but it wasn’t really my area of expertise. I did the basic stuff I was aware of, which mostly involved alt tags on images, a few blog posts with some relevant search terms and a bunch of outreach to try and get links. The last effort was somewhat successful, and I even made it on some local news segments.

Still, customers came in from organic search at a pace for which “trickle” is perhaps too generous a description. I focused on Meta ads and email marketing, found some loyal customers and still make a little money from the business today. For the most part, though, I have entirely neglected the Shopify site (most of my sales come from Amazon now) as I’ve moved into buying other e-commerce brands.

Enter Claude

A month or so ago, I was in the shower (the location of most of my best thoughts) and realized that Claude could probably do SEO better than I. I don’t have the exact prompt I used, but it was something along the lines of:

I have a Shopify site for Cooper’s Treats. I don’t really know much about SEO. Can you do SEO stuff for it?

Never be afraid to ask dumb questions!

Unsurprisingly, the answer was yes. It gave me three general categories of work it could do:

1. SEO cleanup and optimization. Get rid of test pages that I published but never linked from the site, make sure all the images have alt text, set SEO titles and meta descriptions, etc.

2. Run through my Google Search Console results and publish a bunch of informative blog posts slop about dog-related search queries.

3. Reach out to other websites to inquire about having them link to mine.

I went with options one and two. Having my AI agent start emailing people is a bridge too far for me. Adding blog posts to the internet isn’t great, but at least it isn’t directly annoying people. I’m not totally devoid of principles!

First things first, Claude needed access to Shopify and Google Search Console. For the former, I created a Shopify app, got a list of necessary permissions from Claude, installed it to my store and then gave Claude the credentials. I have since learned that there is already a Claude connector app that I could’ve used, so if you want Claude to do anything to your Shopify site, I’d recommend that. Claude already has a Google Service Account to access my Sheets and Docs, so I added that to the GSC property.

From there, it spun up some subagents and ran through everything. On Shopify, you will not be shocked to hear that it found plenty of areas for improvement. The first few products I launched were in good shape — I was enthusiastic and diligent about everything back then — but most of what I added after starting the business was missing the absolutely basic stuff you’re supposed to do for SEO.

This was really a perfect task for AI. The scope and requirements are clear, the work is incredibly repetitive and tedious, and it can’t quite be done by deterministic code because you need some intelligence to write appropriate titles and other text. Claude came back with a list of proposed changes, I briefly eyeballed them and said go, and it just fixed everything. I spot-checked some changes, but at this point I have high enough confidence that Claude can manage straightforward API calls that it’s just not worth the time to review all of its work.

Next up, blog posts. When I launched the site, I researched and wrote a number of “Can dogs eat <food>?” blog posts, primarily about the ingredients in my treats. I honestly have no recollection of why that’s the direction I went, but some of them performed okay in terms of ranking and traffic, though those visitors very rarely converted to customers.

Still, Claude saw that and figured it’d pick up where I left off with a lot more foods, and when the cost is effectively zero, even a barely-converting page has great ROI. It also came up with several other topical areas tangentially related to my products and proposed more articles. In the end, it had a list of a few hundred blog posts to write.

I asked it what the best strategy was for putting those out, and it said that dumping hundreds of posts in one day would probably trigger some negative treatment from Google. It instead set up a daily job to put up one article, plus, at my request, a weekly email update with information on the performance of the pages it fixed up for me plus the new posts.

It’s been a few weeks, and so far the results are solid. No massive influx of traffic, but clicks and impressions are both up. The existing pages that it cleaned up have moved up in search results, and I’ve even seen a couple of sales. This has been neither glamorous nor exciting, but it’s the kind of grinding productivity that helps in the long run.

What Does This Tell Us About The Future of SEO, Work and E-Commerce?

While I have made light of the fact that I’m adding AI slop to the internet (and I am sure some people will be legitimately angry over it), the reality is a nigh-infinite amount of SEO-related slop was created before AI was a twinkle in Demis Hassabis’ eye. I once worked at a startup called UpCounsel that was a marketplace for lawyers, and one of our main avenues of customer acquisition was tens of thousands of programmatically-generated pages titled “Top Ten <practice area> Lawyers in <City, State>,” each of which had a whole lot of template text with a list of lawyers pulled from our database that matched the title (or were close enough — there were a lot of city pages and we did not have ten lawyers from Stone Mountain, GA).

We can argue over the degree to which that was bad, but the fact is that just creating a web page for your product is no longer enough for Google to rank you anywhere that you’ll ever be seen. SEO is an arms race, and taking a principled stand against participating just means that people will end up buying inferior products with superior search marketing.

The good news is that AI offers a couple of ways to improve the situation. First, it evens the playing field. If everyone can employ Claude to do SEO, then suddenly spending a lot of time, effort and money on SEO stops being a competitive advantage. This is, I think, a good thing. The ideal state of commerce is the one in which people find and purchase the best products for their needs as opposed to the best-marketed. When the one-man shop with a great product can compete with large enterprises that employ well-funded marketing teams, the economy and consumers are better for it, even if there is collateral damage by way of slop.

On the other side of the transaction, there’s the possibility that everyone’s personal AI will be able to find the right product for the person using it. The reason that being highly-ranked in Google’s search results matters is that nobody has the time or willingness to go through pages and pages of links in hopes that there’s a hidden gem buried in there somewhere. Your AI agent, on the other hand, has time to spare. It’s not crazy to think that it can constantly be scanning every new site that pops up online related to your interests. When it’s the middle of summer and somebody launches a new brand of frozen dog treats with flavors your dog loves, perhaps you get a notification.

Perhaps I’m naive and AI will simply make everything worse, but until I’m proven wrong, I will hold out hope that everyone having an intelligent, tireless agent who knows all of their preferences will make finding things on the internet better.

There is one other point that I think is worth taking away, which is that this represents a class of job that is fully automatable, or very close to it. If you have a website with a large enough number of pages, you have at least one full-time job’s worth of SEO to do. It’s worth paying a salary to have someone constantly monitoring all of the things that go into Google’s ranking algorithms and ensuring every page is optimized, because any movement in your position in search results means an immediate corresponding change in your revenue.

None of that work, though, is deeply complex. There is a lot of it, and it takes understanding and intelligence to do, but it’s well within the grasp of today’s models. While economic and labor data continues to paint a very ambiguous picture of the effects of AI, as someone running a small business, I keep seeing places where AI is unambiguously able to replace humans. I could’ve (and probably should’ve) hired someone from Upwork to do this work for me. Now there’s no need.

Much ink has been spilled over the idea that AI will free us from the tedious work and let us move on to bigger and more interesting things, but what happens when the whole job is tedious work? If you have someone doing the rote, day-to-day SEO work, and then you suddenly realize that actually Claude can handle it, do you move your SEO employee up and have them do high-level tasks, or do you let them go and tell their manager that they’re now managing an instance of Claude to handle SEO instead? I know which option I’d put my money on.

One guy set up a Google Sheet where people could fill out preferences, but I wasn’t about to let that be our main planning tool when I’ve got access to Claude Code. A world of effectively free engineering work should be a world where we have a little fun with the stuff we do online! Besides, it presented a great opportunity for me to try out Cloudflare’s newly launched tools that allow agents to register domains and deploy applications to them.

A quick quote from the link above to give you an idea of what they’re offering:

Starting today, agents can provision Cloudflare on behalf of their users. They can create a Cloudflare account, start a paid subscription, register a domain, and get back an API token to deploy code right away. Humans can be in the loop to grant permission and must accept Cloudflare's terms of service, but no human steps are otherwise required from start to finish. There’s no need to go to the dashboard, copy and paste API tokens, or enter credit card details.

Pretty cool! Much ink has been spilled over the topic of agentic commerce, with plenty of folks scoffing at the idea that anyone would ever let AI go onto Amazon and start buying stuff without approval. But they’re missing the point — agents won’t be buying a pair of shoes; the commerce they’re going to engage in will be the really boring kind.

I just bought some new filters for the RO filtration system that sits under my sink. There are four filters, and two are supposed to be replaced every six months, one every year and another every two years. It’s been a year since I replaced anything (oops), so I needed three of the four. Wouldn’t it have been nice if an agent would just track the replacement timelines and purchase those for me?

I know what you’re thinking: This can be solved by setting up subscriptions and in no way requires AI. But there are three websites I might buy them from, each of which has the cheapest price some portion of the time but is also intermittently out of stock. So every six months (or when I realize it’s been a while since I replaced any of them and it turns out to be a year), I click around, compare the three and order from whatever’s cheapest. This sort of deeply boring online purchasing task is where agentic commerce will thrive. If you’re a business, something like domain registration certainly fits the bill.

So let’s give it a whirl and see what our brave new world looks like.

Registering a Domain

Since the entire purpose of this domain is to plan a trip, the main criteria were:

• Available

• Cheap

• Topically relevant

• (Optional) Gets a chuckle

I gave Claude the context on what we were going to do here, told it to give me some options for domains and asked what it needed. First thing: An API token. Instructions:

To set this up, I had to fill out a form in which each cell in that table required selecting the right value from a dropdown. It would’ve taken me literal minutes, so out of laziness and for the sake of pointless AI maximalism, I had Claude in Chrome do it. Nothing could possibly go wrong from having my agent assign its own permissions, right?

Thankfully it worked fine, so I gave Claude the token and had it find some domain options for me.

(Worth noting that the prevalence of Vegas in the URLs is because I told it to do Vegas since that was where I wanted to steer the group, but I was unfortunately outvoted there.)

In some follow-up conversation on this, I discovered that it actually decided that Cloudflare didn’t have a way to search for domains and instead checked URLs on rdap.org, which offers public records of domain information. This is classic Claude Code — it is abysmal at search and extremely quick to assume that APIs lack capabilities that they absolutely have. I had ChatGPT find instructions on how to use the relevant CF APIs and sent those to Claude, making sure to note passive-aggressively that they came from ChatGPT, who is good at going through API documentation should Claude ever need help in the future. I told Claude to search again, correctly this time, because it’s important not to let it get away with this sort of indolence.

I ended up deciding on a domain not from the list, which I will not include here, seeing as you don’t need to visit it and start submitting answers to my questionnaire. I told Claude to register it, and it began the process but informed me that it was blocked because my CF account didn’t have contact info for domain registration. Claude asked if it should use my name and address, and after I confirmed, it uploaded those and attempted to use the Registrar API to get the domain for me.

Alas, the API rejected that request. The TLD wasn’t supported. If you saw that list of domains above and you know me at all, you will have already figured out that I went with .party to save six bucks. Don’t judge me.

At that point I did what anybody who writes a Substack about doing stuff with AI would do and had Claude Code write me instructions to pass along to Claude in Chrome, which I then pointed at the registration page.

It was close, except it picked auto-renewal, which I definitely do not want. Since it didn’t have context about what the site was for, and the potential downside of not auto-renewing is a lot greater than the harm of wasting four bucks a year, I can’t really fault it on that one.

Setting Up the Site

The basic idea was to go through a few rounds of questions, collect the answers and figure out where and when we’re going. To get it live, Claude used a few things on Cloudflare:

• Workers: The site runs as a worker. It’s a Next.js 16 app wrapped by the OpenNext Cloudflare adapter, which compiles Next’s server into a worker. Claude wrote that last sentence. I don’t get involved in those kinds of details. I’m a big picture guy!

• Assets: Images, CSS, etc.

• D1: A database to keep track of all of the questionnaire answers

• Domains API: To point the domain at the site

• Secrets: To store my Anthropic API key (I assumed some of my friends would leave sassy responses to some questions and was going to have Claude sass them back in the UI, but they did not and so will never appreciate the artistry of my work) and SMTP credentials for sending out emails inviting them to fill out the survey.

To access all of these, Claude used Cloudflare Wrangler, their CLI. I wish I had some useful insight I could share, but the reality is Claude was able to get the whole website built and deployed while I walked my dog. I asked Claude if it had anything to say on the topic:

The interesting point: Wrangler is text-in, text-out, which is the whole reason an agent could run it unattended.

Most cloud platforms are dashboard-first. To set up a database, attach a domain, or store a secret, a human clicks through a web console — and an AI agent can’t click through a web console. It’d have to stop every few minutes and hand the task back to you: “okay, now go to this settings page and toggle this.” That’s death for a hands-off workflow.

Wrangler is the opposite — it’s a command-line tool, so every setup action is just a typed command that spits back text I can read and react to. Create the database, apply the schema, push the code live, store the secret API keys, wire up the domain, stream the logs to see if it broke — all of it is text. That’s the structural reason I could do the entire build-and-ship loop on my own and you only saw the result.

And it’s notable how much lives behind that one tool. On a typical stack, those jobs are spread across four or five different services and logins — a host for the code, a separate database provider, a separate secrets manager, a separate logging service. Here it’s one CLI talking to one platform, all configured from a single file that lives in the project. Fewer tools means fewer places for an agent to get stuck or need a human to log in somewhere.

Once the website was ready, we were able to make such important decisions as:

• We’re going to Mexico.

• Wives are coming, but kids are to be left behind to fend for themselves.

• We’re staying at a nice resort instead of reliving our college days by renting the cheapest house we could find in Rosarito.

I can’t even remember how people handled situations like this before AI.

The Future of Agentic Commerce

I give a lot of credit to Cloudflare here for leading on this new form of interaction. All I had to do was get an API token (though I would also have had to save a credit card if the domain had been purchasable via API), and my agent was able to handle the rest.

We’re starting to see a handful of companies really lean into supporting agents as consumers of their services, and I strongly suspect that’ll pay off for them over time. Shopify’s all-in on agentic commerce. Same with Stripe (notably in partnership with Cloudflare).

A few weeks ago I wrote about having Claude update financial ledgers for each of the brands I own, and there was a stark difference between Mercury, which I use for banking, and American Express. The former makes it trivial to get an API key and has a CLI and MCP to boot. The latter won’t deign to give me programmatic access to my own financial information, so I have to go in and download transaction CSVs (or have Claude in Chrome do it after I authenticate, because pointless AI maximalism!).

As people increasingly rely on agents to do tedious tasks like buying a domain, deploying a website or retrieving financial transactions, agentic experiences will matter. If I have to choose between two businesses and one will let my agent interface with it, while the other requires me to click buttons and fill out fields, I’m choosing the former. When I send my agent out to do research on service providers, I have to imagine it’ll have the same consideration. In the world of boring-but-necessary tasks, the companies that support agents shall reign.

Labs, hyperscalers and all of the other businesses that want you to use their AI products are somehow constitutionally incapable of pitching them in a way that would make people excited that they’re coming. The standard demo is an AI agent planning and booking a vacation. This is dumb! People like planning vacations! They don’t want AI to do it for them!

The marketing teams at these places seem to start with the capabilities of their agents and then think of the coolest demos they can set up with those capabilities, which is entirely backwards. Don’t show AI taking the fun, interesting tasks away from people; show it dealing with the things they hate doing. Tell people how AI can renew their driver’s license or get quotes from plumbers or do their taxes. Those are chores that nobody has ever wanted to spend a single second on, and they will appreciate the company that takes them off their hands.

• Dispose of it

• Mail it back to me

• Grade the condition and resell it at a discount based on that grade

• Liquidate it for 5-10% of the sale price

Alternatively, I can skip the whole process and just have it disposed of or sent back to me.

The big problem one encounters when it comes to having Amazon employees evaluate quality is that they have absolutely no idea how to evaluate quality. I am exaggerating, but only slightly. For very simple items, they’re okay. But for products that contain more than a couple of items, they’re not going to notice if one is gone.

Also, if your product is tightly packed into custom, opaque packaging, they will not open the packaging at all and will simply decide whether the product is resellable condition based on the exterior. Ask me how I know this.

So glad you asked. One of the products I sell is an indoor swing; basically a big loop of nylon fabric and a hardware kit to hang it from the ceiling. It comes in a custom, opaque bag. Once, someone bought one and then returned it, except what they actually returned was a dark blue bedsheet. Since the swing they ordered was beige, it probably would have been clear that this was the wrong product had whoever processed the return opened up the bag.

This all happened while I was still having Amazon evaluate everything that came in, so the way I found out was when someone left a one-star review because they ordered a beige swing and received a dark blue bedsheet. (If you order something from Amazon and get the wrong product, I beg you not to leave a bad review — it is almost certainly on Amazon, not the company selling the product.)

I spent a great deal of time getting that review removed — doing so is very difficult and requires one to utter the correct incantations and request specific escalations with no guarantee of success. Unfortunately, I very naively failed to address the root cause, so you can imagine how I felt when I received another one-star review from someone upset that they ordered a beige swing and received a dark blue bedsheet. Sigh.

At that point, I did the correct thing and turned off Amazon quality evaluation. Since this product is low volume, high price point, and relatively high return rate, I started having every unit sent back to me. Once a month or so, I go through all the ones I’ve received, check for missing parts/blue bedsheets, then repack the like-new ones and send them back to Amazon to be sold.

Getting Reimbursed

When the customer returns either the wrong product or the right one but missing parts, I can get reimbursed by filing a case with support. To do this requires providing:

• The LPN — I had never actually thought about what this stood for until looking it up right now, and apparently it’s License Plate Number? At least that’s what the Google AI Overview says, and I don’t care enough to confirm. Anyway it’s a unique barcode/string that gets stuck on every return.

• The removal order ID — Twice a month, Amazon creates an order to mail back all of the units that have been returned, and that order gets assigned a unique ID.

• The return shipment ID — Removal orders typically come in multiple shipments, since the returns are held in different fulfillment centers, so each shipment gets its own ID.

• The FNSKU — This is the catalog ID of the product.

• A description of what I received

The obnoxious thing about all of this is that Amazon knows the removal order ID, shipment ID and FNSKU based on the LPN, but they make you pull all of those off of a tiny sheet of paper that comes back in the box. If you lose that piece of paper, you can’t get reimbursed. This is what it looks like:

A couple of fun little quirks here:

• The actual paper is much smaller than it looks here, and the font size must be 5 or 6.

• The FNSKU is in the column incorrectly labeled ASIN (which is a different product identifier).

• The return ID is the string of letters and numbers between the first and second dashes in the string after “RA #:”. If you include the whole thing, support will have no idea what to do with that.

But hey, who can blame the scrappy little checks stock price 2.76 trillion dollar company, right?

Anyway: I must gather all of these pieces of information, along with a photo of the LPN sticker, the piece of paper pictured above, and the product that I received. (And again, you must provide them all of the information in text along with photos of the papers you copied it from or no reimbursement for you!). I used to do this by emailing myself photos, copying the text by hand and putting it all into a template, but with AI there is, of course, a better way.

Building a Return Bot

A few weeks ago, I did what I assume every AI-pilled dad does and built a bot to keep family stuff straight. It accepts messages via Telegram or email and can store information to a shared memory or add calendar events. Since it already has access to Claude and the ability to send and receive messages, I just used that to start my return bot.

I began by putting together a Google Doc with all of the photo inputs and the desired output:

Input Photos

Output Text:

• Return Shipment ID: 364521260336551

• LPN: O 53568 7617

• Removal Order ID: SD0jzXhxKR

• FNSKU: X004FA4EVX

• Condition: Returned missing all of the hardware needed to install the product.

Then a quick spec:

I want you to add functionality to the Willen Bot that helps me get reimbursed for Amazon returns. I should be able to send it photos of the returned product, the LPN sticker and the return slip via either Telegram or email. It should extract the relevant information, then send me an email titled: “<Brand name> reimbursement: <LPN>” with the information in the body and the three input photos attached. This doc has an example of input images and output text, plus some explanation of where to extract the relevant info from the inputs: <doc link>. Ask me questions then build a plan before implementing.

It raised a few good points:

I use the bot for multiple things, so how will it know when I’m using the reimbursement functionality?

For email, I’ll use <bot’s email>+returns@gmail.com, and for Telegram I’ll start the return functionality by sending the word “return.”

How should it determine when it’s received all the inputs for a return and start processing them?

Via email, each message will have every necessary piece of info. If I’m interacting via Telegram, one return might involve multiple messages from me, so I’ll send the word “go” when it should start processing. I did consider the option of having it process each input as it comes in and then automatically fire off the email to me once it had determined it had everything, but I figured the simple route was fine here.

How does it know what to use as the string describing what I received?

Since the swings are the product I’m using this for the most, I broke that down into the two most common cases based on the photo.

• If it’s the swing, it’ll write “Returned missing all of the hardware needed to install the product.” (I gave it a bunch of pictures of the swing from prior cases as examples.)

• If the picture is of anything else, I will also include a text description of what was returned, in which case it will write “The customer returned <text description I provided>, which is not what they purchased.”

For other products, I’ll describe what was returned or what was missing via text, and it’ll just add appropriate grammar and punctuation before inserting it.

With all that done, Claude started building, I walked my dog, and when I came back I did a bit of testing. After resolving a couple of minor issues, it worked!

Architecture

Since the Claude -pocalypse (the name hasn’t caught on yet, but I’m still trying…), I am doing my best to avoid throwing machine intelligence at problems that could just be handled by deterministic code. This one is almost entirely the latter.

Invoking the functionality is done via a simple regex that just checks for “return” (or the inbound email addressing having “+returns”), and processing starts when it sees “go.” At that point, it uses the Agent SDK to have Claude process the photos and text. It does vision OCR to pull out the relevant text and classify the image, and it analyzes the text I’ve sent to pull out the right noun or phrase to insert in the Condition field (e.g. if I send “customer sent back eight gray bags, which is not the right thing” it will extract “eight gray bags” as the returned item). That’s the extent of AI usage in processing. Though I feel compelled to note, as someone who spent a bunch of time at my last job PMing a product that extracted text from PDFs and images, that being able to ask Claude and reliably get the text back is more miraculous than most people imagine.

From there, it does some validation of the return shipment ID, removal order ID and FNSKU. Because the shipment ID and removal order ID are printed in tiny font, and particularly because the latter can have lowercase Ls and uppercase Is, I want to make sure the OCRed text is correct.

For the shipment ID, since it’s entirely numeric and not prone to OCR failures, it just checks that it has the right number of characters. These have always been correct so far in my testing, but if one is ever wrong, I’ll find out when a support agent tells me they can’t find the ID. If that happens, I’ll implement an API lookup to confirm it exists before submitting.

The removal order ID is tougher and has been wrong a couple of times in testing, so for that we validate against order IDs returned by Amazon. To keep things snappy, there’s a job that runs first thing in the morning to pull new IDs and store them locally, and we check against the local list. If the ID matches something on that list, then it uses that. If not, then it does a fuzzy match check, basically looking for the longest substring that matches something from the list, then uses the winner.

The FNSKU is more straightforward — since that doesn’t change, it keeps a list of FNSKUs for each brand and makes sure the OCRed text matches an item from that list. If not, it does a fuzzy match there as well.

It doesn’t do anything with the LPN except to strip the actual letters “LPN” out of the beginning. There is no way to validate that because it doesn’t have a consistent structure or length, and there’s no way to check it via the API. Since it’s on its own label and printed in relatively large font, that’s not an issue.

Once it has those, it determines the Condition text based on the image classification (if it’s the swing brand) and/or the text I’ve sent. With all of that, it constructs the block of text, creates the email, attaches the images and fires that off to me.

Next Steps

The output of this is an email to myself with all of the information structured in my prescribed format and all images attached. I take that, open a case with Amazon and get reimbursed.

Ideally the bot would just open the case for me, but Amazon does not have an API for support cases. I assume this is because the friction of having to do it by hand is very much a feature from their perspective. I’m sorely tempted to have Claude do it via either Playwright or Claude for Chrome, but I don’t know if Amazon would look kindly on that. Seems imprudent to take any risk there, since with the bot in place the actual case-filing portion of this takes a few minutes a month. Maybe once I’ve got an Optimus that can sort through the returns, I’ll have it sit down at my desk and open the case with my mouse and keyboard, so Amazon really can’t detect that it’s a bot…

I own a dozen e-commerce brands, and I have three legal entities to hold them all. It’d be nice to maintain one entity per brand, but California charges $800 per year per entity, and I’d rather keep a few grand in my pocket.

For business purposes, though, I want to treat each brand as an independent entity from a financial perspective. This is in large part because my fund is set up so that LPs are paid back for each brand out of its income, rather than lumping the entire fund together.

The challenge is that I have one bank account and credit card per entity, so I have to maintain my own records of which income and expenses are attributed to each brand. I do this by way of fairly simple ledgers in Google Sheets — one per brand, with line items for any money coming in or going out.

Initially, this wasn’t really a problem to maintain. Every couple weeks or so, I’d run through all the transactions in my bank and credit card, recording each one on the appropriate ledger. Once I hit four or five brands, though, this got to be annoying.

The main problem is that the majority of line items are either money coming in from Amazon or money going out to Amazon ads, and there’s no way to distinguish which transaction is associated with which brand from the bank/credit card web app.

That means I have to go into Amazon Seller Central, go to each brand’s payments tab, find the most recent payment, match it up by amount with my bank statement, and record it on the ledger. Then repeat for Amazon ads. There is an extra quirk where Amazon sometimes charges small amounts in Mexican pesos or Canadian dollars, and you have to go to yet another screen to figure out which brand those are for.

Accessing Systems of Record

As with all repetitive administrative tasks, I’d like Claude to handle this for me. To do so, it needs to access:

• Mercury (my bank)

• American Express

• Amazon Seller Central

• Amazon Ads

• Google Sheets

It already has access to Amazon, and I’ll spare you a long explanation of how to set that up, both because I’ve written about it in the past and because Claude can explain it anyway (but if you need more help, shoot me a message).

It also has access to Google Sheets — which is a good moment to repeat a point from prior posts: the more you use Claude Code, the easier each new task gets. It’s an AI ouroboros of sorts.

Since I have not covered connecting to Google Sheets in the past, and the average reader is much more likely to use that than Amazon Seller Central, a quick primer: To access Sheets, Claude needs a Google Cloud Service Account — that’s basically an account that it can use to access anything you’ve got stored on Google Drive. You set it up, it gets an email address, and then you can invite that email to any of your Docs/Sheets/etc. just like you would a person.

In order to set that up:

1. Go to https://console.cloud.google.com/ and create a new project.

2. Find the Google Sheets API in the API Library, and enable it for that project.

3. Create a Service Account, create a new key for that account, download the JSON file, and put it somewhere Claude Code can find it.

4. Ask Claude for the email address of the Service Account.

5. Invite that email address to your Google Sheet.

With that, Claude can edit your Sheet! (And again, if that was not sufficiently clear, just ask Claude; it can walk you through this process in great detail.)

Now the more challenging part — pulling data from financial institutions. The ideal case is you have something like Mercury (which I highly recommend to anyone who needs business banking). They offer API access to customers, so it’s as easy as going into settings, generating an API token and giving that to Claude.

The bad (and much more common) case is something like Amex, which does not offer API access to any schlub off the street such as yours truly. This means the work is a bit more manual. Any bank or credit card provider should allow you to download a list of your transactions (ideally in CSV, but worst case a PDF should do fine), which you can then give to Claude.

If you want to be really lazy, you can take the approach I went with — have Claude fire up a browser and pull the CSV itself. Because this requires you to be authenticated, you can either give it your credentials or type them in yourself every time you run this process.

As much as I want to be truly AI maximalist here, I went with the latter approach because it seems like a bad idea to give Claude my Amex username and password. (I should note that I did attempt to create a user account that Claude could use to access Amex, but that required putting in personal information for KYC reasons. Amex unfortunately did not permit me to have two users on my account that have different names but share my SSN.)

The Ledger Skill

Now that Claude’s got everything it needs, it’s time to turn this all into a skill. It runs through a step-by-step process to gather all transactions, determine which ones are associated with each brand, write them to the ledger, then check its work for completeness.

It starts by reviewing all of the ledgers to find the most recent transaction for each, then uses that to pick a starting date when pulling info from my financial accounts. Just to be safe, I have it start a week before the most recent transaction (it will review everything to see whether it’s already written to a ledger, so we won’t get duplicate entries).

Once it has a starting date, it opens up Amex’s auth page so I can enter my credentials, then navigates the site to grab the CSV we need. It handles Mercury and Amazon via API — pulling entries, payments and ad charges.

With that info, it starts generating a list of line items to be added to each ledger. Anything that’s already on a ledger gets dropped. It handles all of the Amazon charges and distributions, matching each transaction up to Mercury/Amex by dollar amount and date.

From there, it goes through any remaining charges and checks against context files to see which of them it can confidently attribute to brands. For most, it knows enough to do this without my input. It has records of which manufacturer makes products for which brand, so it can figure out where to categorize each wire transfer. Fund-wide charges like my QuickBooks subscription and insurance get split across all brands equally.

Anything it can’t handle on its own gets shown to me for review — typically things like Google API charges (if I’m generating listing images for brands using Nano Banana) or bills from my 3PL (I use it for several brands and have to distribute the charges across them).

With everything categorized, it shows me a list of what it’s going to add to each brand’s ledger. At this point, I should really get rid of this step, because it’s been 100% accurate for so long that I no longer review it before telling Claude to proceed.

Before writing all of the charges, it checks each ledger to see if there are any transactions whose dates are earlier than the ones it plans to write. This is because sometimes when I charge something to my card that I know Claude is going to have to ask me about when running this skill, I just write it myself while it’s fresh in my mind. If necessary, it plans to move those around to ensure the list is in chronological order.

Only then does it write everything to the sheets, followed by a review pass, checking the ledgers to ensure that every transaction it pulled from Mercury and Amex is accounted for somewhere and that all entries are sorted correctly.

And with that, we’re done! I used to spend two or three hours a month on this task, and now that’s reduced to a couple of minutes giving Claude input on transactions. While economists debate whether AI is really impacting productivity, it’s giving me some precious time back to spend with my kids. Thanks, Claude!

If you’re unfamiliar, claude -p is used to run a headless Claude Code session. Basically, if you have an application that needs to have Claude Code do something, it can silently fire up a session, send it a command and use the output. This is useful if you want to have Claude reason over something as part of a programmatic process, but you are cheap and don’t want to pay to use the API.

I make frequent use of claude -p in daily tasks, like running updated inventory projections for my cashflow dashboard, so this is something of a bummer.

A quick aside on communicating with one’s customers: this email is extremely bad. It claims that they’re implementing the change because the rules weren’t clear. But if the problem is a lack of clarity, the right solution would be to, y’know, clarify, not create a new usage policy that drastically reduces the value of the subscription. I’m pretty sympathetic to the labs’ challenges with soaring usage against a limited supply of compute, so if this email said, “Hey, we’re sorry but this feature we implemented has kinda blown up in our face, so we need to start charging for it at API rates,” I would understand! But please don’t treat me like I’m a moron. Also, why are you making me click a link in an email that’s coming later to get my credit? Just put the link in this email! Or don’t require me to click a link at all!

Anyway, I asked Claude what it would cost me per month if I leave everything in its current state. Answer:

Oh okay cool just a casual increase from $100/month to $3,000/month. No biggie.

So, how do we get costs back down to sub-$100, so they’re covered by my subscription, ideally without losing functionality?

The first fix is easy. I actually forgot the amazon_ads judge thing was still running (old experiment, oops), so I’ll just go ahead and turn that off.

Beyond that, I have two daily processes and then my family assistant, which my wife and I use to add things to our collective calendars and store memories via a dedicated Gmail account and a Telegram bot.

Moving From Claude -p to Desktop Scheduled Tasks

The daily processes fire via GitHub runners, which then invoke claude -p. The good news there is that I only use that setup because I started creating these sorts of recurring jobs before Claude had scheduled tasks. While the update isn’t 100% clear, it looks like if I install Claude Desktop and set them up to run daily from there, that’ll still use my subscription. Let’s start with my inventory projection skill, which runs once per week for each brand (the actual skill runs daily but only does 1-2 brands each time). It gathers sales and inventory data from Amazon, then follows a process to project my sales by month for the next 18 months. Those projections flow into my cashflow dashboard, so I know when I’ll need to place future orders and how much money will be required.

Skills aren’t automatically shared between Claude Code and Claude Desktop, so there is a bit of work to do here. The skill.md file needs to be copied from its project directory into Cowork’s skill directory. In doing this, I learned that Claude will mislead you about what directory this is — it told me it was \.claude\skills but then couldn’t find the skill directory or file in there. The best solution here is just to explain what you need to Claude Code, tell it to search the internet for relevant context and find the appropriate directory on your machine, then copy the skill over. (I would point you to the right place, but I’m on a PC and odds are good you’re on a Mac, so just let CC handle it.)

Once you’ve got the skill in the right place, open Claude Desktop, go to Cowork → Scheduled, click New Task, then select Set up manually. (I should note that Claude tells me Cowork scheduled tasks and Claude Code Desktop local routines are exactly the same thing, in case you’re wondering why I didn’t go with the latter.)

Enter a name and description. For instructions, since everything it needs is already in the skill, just tell it to run <slash command>. If the skill needs to access any files on your hard drive, point it to the folder where those live. Set it to act without asking. Choose the schedule on which you want it to run. Once it’s all configured, click Save.

Once that’s done, click on your newly-created task, then click Run now. That’ll create a new entry in the History section — click on that to watch the skill execute.

If everything is set up correctly and Claude can access the skill, it should go off without a hitch. If it runs into any problems, it’ll let you know.

Here’s the big one I ran into: Claude Desktop scheduled tasks run in a sandbox that is only able to access whitelisted external endpoints. My /inventory skill needs to access the Amazon and Google Sheets APIs, but that’s not possible.

The good news is that this is fixable for my particular case — I just need to split off the parts that hit the APIs, run those in advance, then have the inventory projection skill pick up the data they retrieve. Accessing the APIs just involves running a script and doesn’t require Claude, so I can continue to handle that via a GitHub runner. I’ll set that up to run far enough in advance that it’ll definitely complete before the scheduled run of the skill starts.

This is, in fairness, a better way to do things. I mentioned last week that the /inventory skill had become quite long, and I had updated it to spin up subagents for the API calls in order to save the main agent’s context window. Doing it this way is even better, because it reduces the skill to just the parts that require Claude.

My comms ingestion task (which is the process that gathers all of the messages to/from my suppliers to create my supplier dashboard) is similar. It uses Playwright and Microsoft UI Automation to get my supplier messages, and Claude Desktop blocks both of those. Since those are just Python scripts, they’ll remain on a GH runner and drop the text and images they gather into a folder. A Cowork scheduled task will pick up from there to handle the information extraction and updates to my dashboard.

Now we’re all set — two GH runners, two scheduled tasks, and all of my Claude usage still happening on my Max plan.

Migrating to the Agent SDK

The family assistant is a bit thornier, because every time it gets an email or a Telegram message, it sends it to Claude Code for processing, so it’s definitely not suitable for scheduled tasks. The structure is also wildly inefficient, because it involves firing up the Claude Code harness, which consumes a lot of tokens, every single time.

While I am critical of Anthropic’s messaging, this is a good example of why they’re making this change — there is absolutely no need for either the full Claude Code harness or Opus here, and it’s completely wasteful. The right way to do this is to hit a smaller model via the Messages API, since the core of what I’m using Claude for here is message parsing that handles a few simple cases:

• If I’m asking it to add something to the calendar, take the relevant information and convert it to the payload for a GCal API call.

• If I’m telling it to remember something, determine the category (restaurant preferences, information about a kid, etc.), check the memory .md file to see if it already exists in that category, and if not then add it.

• For anything that’s not clearly one of the above two cases, ask for clarity.

The only issue there is that the Messages API isn’t covered under the new credit. I would have to pay something like a double-digit number of cents each month to use it here. Quelle horreur! (But seriously if anyone from Anthropic ever reads this, you should probably just include all API usage.)

Keeping claude -p but switching to Sonnet would probably be sufficient to keep my usage under the amount of the credit, but if only for the sake of this post, I’ll see what else I can do. First: drop the Claude Code harness.

Whenever you launch a new Claude Code session, it loads the harness’s system prompt, descriptions for all of the built-in tools, the contents of your claude.md, and all of the skills and MCP servers you’ve installed. Claude ballparks that at 15-30k tokens depending on your exact setup. If I switch to the Agent SDK with none of that, we’re looking at ~2k.

Our next improvement is to wire up an MCP specifically for this case. I have historically been pretty blasé about MCPs that just act as translation layers between my agent and an API. Claude doesn’t need a translation layer! It’s fully capable of just reading the API docs and making the appropriate calls with bash! But that is, of course, the attitude of a person with limitless tokens. Now that Anthropic’s cracking down and GPUs are in short supply, it’s time to tighten our proverbial token belt.

For the Google Calendar case, we just need an MCP that takes in the relevant event info, turns it into an appropriately-structured API call, fires it off to Google, parses the response and sends the relevant parts to Claude, which in turn reports back to me.

Saving notes is a little bit more complicated, because it requires searching to see if there is already an existing note on whatever topic I’ve asked it to remember before writing. If there is, it needs to let me know and ask how I want to proceed — do nothing or modify the existing note. Modifying the existing note might involve rewriting text rather than just appending my update, so there’s some intelligence required there. I could create a simple MCP to handle the case of writing a new note, but that’s just not especially token-heavy anyway. We’ll just let Claude handle notes directly.

Now that we’ve got the details figured out and the calendar creation MCP ready, the next step is to migrate it to use the Agent SDK. Not a lot to say here; I just asked Claude to handle that for me, making sure it has access to only the new MCP created and whatever tools it needs to handle searching and creating notes. The whole thing took a couple of minutes and worked on the first try. However much I dislike that Anthropic’s suddenly forcing me to update my workflows in order to avoid a large bill, I am mollified by the fact that they’re providing me the tools to make it all happen magically on my other monitor while I draft this post.

With that, we should be well under the monthly credit limit.

Two other features to note that I haven’t used here but might be relevant if you’re looking to reduce token usage for this sort of thing:

--bare

At some point, Anthropic introduced --bare, which, when used with claude -p, skips auto-discovery of hooks, skills, plugins, MCP servers, memory and claude.md. If you’re going to continue to use headless mode but don’t need the whole harness, this will save a lot of tokens.

Prompt Caching

If you’re using the Agent SDK, then caching your system prompt and tool definitions will likely reduce your token usage. There are, though, some caveats that mean it’s not worth doing for my family assistant.

First, writing to the cache costs more than making a regular call — 1.25x to cache for five minutes, and 2x to cache for one hour. I usually send one message to the bot at a time, so most of the time caching would be wasteful.

There is also a minimum cacheable prompt length. This is either 4,096 or 1,024 tokens depending on which model you’re using, so if your instructions are shorter than that, this isn’t usable anyway.

The Future of Tokens

It’s worth taking a moment to reflect on the enormous discrepancy between how much it would’ve cost me to run all of these things at API prices and what I have actually been paying on a Claude Max plan. There are not a lot of products where you encounter a 30x difference in cost for the exact same thing depending on how you implement it.

As I said earlier, I’m sympathetic to Anthropic and all of the other labs — they’re in a position of having to figure out a brand new pricing model (don’t forget that nobody even knew what a token was five years ago) while dealing with unprecedented demand growth and serious supply constraints. They have to balance their need for cash against the competition to gain market share among users. They’re selling to large enterprises, building harnesses and skills, and dealing with a flood of OpenClaws. This is all hard!

Ultimately, though, I feel like in the relatively near future, subscription plans are going to have to go the way of Ubers and DoorDash deliveries and all the other VC-subsidized goodies that the 2010s brought us. It doesn’t make sense to be selling $3000 of usage for $100, especially when you can barely keep up with demand that isn’t likely to slow down in the foreseeable future.

If the labs tighten the reins on flat-rate monthly plans, I strongly suspect people will be upset; that’s just human nature. But in fairness, the idea that I could have someone constantly keeping tabs on all of my brands’ inventory, maintaining an updated dashboard with the status of all of my supplier communications and sitting ready 24/7 to take down whatever notes I want to fire off about my family for $100/month would have been unimaginable in the very recent past. If you’d asked me what I’d pay for that before the labs anchored the pricing at a hundred bucks, I suspect the answer would’ve been a few hundred at least (and in fairness, that’s probably much closer to what the actual cost would be than the $3000 Claude quoted me if I chose lighter-weight models and optimized the workflows).

That said, I do think that in the long run, access to intelligence will be shockingly cheap. Over the last few years, we’ve seen the prices of older models drop once their successors are released, and we’ve seen small models come out that are as capable as the large models of the prior generation but faster and at a fraction of the price.

My daily tasks don’t need better models. They get the job done with the existing ones (and will probably still do just fine when I downgrade the models they’re using), so when Opus 5 comes out, there won’t be a point in upgrading. Even if labs crack down on subscription plan usage in the near term, the models I’m using will get cheaper and cheaper over time. People doing cutting-edge research will shell out top dollar for the latest and greatest, but my bot that makes calendar events will never need a better model than the ones it’s using today. If I do upgrade, it’ll be to Nano Haiku 5 (or in keeping with the theme, I guess a model smaller than Haiku would be… Couplet?) because that’s even cheaper than Sonnet 4.6.

This isn’t just true of my projects — the reality is that most of the white-collar work in the world does not require (or even benefit from) genius intellect. With each successive generation of model, there will be fewer and fewer jobs that actually need to upgrade. Eventually I suspect we’ll have a split where most business tasks are still running on the current generation of model, while Opus 10 solves yet-undiscovered questions of physics and uncovers the meaning of life.

Since I treat each brand independently from a cash perspective, and I don’t use any kind of inventory financing, I need to make sure that between cash on hand and what I expect to come in from future sales, there’s enough to place future orders. This requires projecting revenue, costs and the timing of orders. It’s one of those tasks that’s not really hard to do with a spreadsheet, but now that I own a dozen brands, the time required across all of them really adds up. So naturally, it’s time to get AI to do this for me.

The good news is that the projection piece is something I’ve already spent a lot of time on, first all the way back when GPT-5-codex-high was SOTA, and a bit more recently by developing a Claude skill. Initially, the Claude skill had my projection methodology well-defined, but it was too long for Claude to reliably follow. It would do mostly the right thing but then miss some steps or do them in ways that weren’t quite correct. That would usually give me projections that were in the ballpark, but given how critical this is to my business, not good enough to rely on.

Since then I’ve tinkered with the skill a bit, mostly updating it to spin up subagents for individual tasks within the skill in order to minimize the amount of information in the main skill’s context window. If it needs to retrieve sales data from Amazon, for example, it’ll have a subagent make the API calls and return only the specific numbers needed, thus keeping those calls out of the main agent’s context.

That helped, but it still wasn’t fully reliable until Opus 4.7. When I tested it with max reasoning, it went a perfect 5/5 on testing that included all of my thorniest cases. (I haven’t tried it with sub-max reasoning; very possible max is overkill.)

With that working, Claude should be able to make me a cashflow projection dashboard.

Requirements and Inputs

First, Claude needs requirements. As engineering is now approximately free and iterations take minutes, I have become extremely lazy on the initial spec.

I need a cashflow projection dashboard. The main purpose of this is to show me how much money I can safely distribute from each brand while ensuring I retain enough to place future orders. The main dashboard should have one card for each brand that shows how much is available to distribute. When I click on the card, it should go to a more detailed view that shows a graph of cashflow by month along with a table that breaks down revenue and expenses for each month.

The dashboard should start with my current cash balance, then determine changes in cashflow each month based on projected sales and expenses. It should use the /inventory skill to project sales as well as when it will be necessary to place orders, and it should include the deposit, balance and freight payments in expenses. Let me know what other information you need to build this.

It came up with the right list of info (always a good sign), which I gave it:

Cash on Hand

Every brand within a single legal entity shares a bank account, so I track each one on a ledger in Google Sheets. The dashboard just needs to retrieve the current balance from there. Amazon also holds cash from sales temporarily before making transfers into my bank account every two weeks, so we need to retrieve that via an API call.

COGS and Payment Terms

The /inventory skill projects unit sales and reorder dates, but it doesn’t have any information about costs. For each product, Claude needs to know the unit cost, a rough estimate of shipping, and payment terms. For most brands, I put down a 30% deposit when I place an order and pay the remaining 70% on completion, though a few are 50/50.

Current Inventory

The /inventory skill knew how to retrieve inventory stored at Amazon via API, but there are some edge cases where I have inventory either at a 3PL in the US or with the manufacturer in China. In the former, it is fully paid for and the only cost I need to account for is a relatively small freight cost from the 3PL to Amazon warehouses. In the latter, when the goods are shipped I need to pay the balance of what I owe the manufacturer as well as the cost of shipping to the US.

For the initial version of the dashboard, I just gave Claude the number of units of any product outside of Amazon, as well as the location where they’re stored. For timing purposes, I told it to assume it would take 10 days to get product from my 3PL to Amazon and 40 from China.

Fees

Claude initially just made a simple estimation of fees, but after that went poorly (a bit more on that below), I came back and had it model them in more granular fashion.

Amazon takes out all of its fees before disbursing money to me, with the exception of ad charges, which are paid on a credit card.

Some of the Amazon fees are straightforward to model:

• Referral fee is 15% of revenue

• Fulfillment is a flat fee per unit (based on size/weight)

Then there are charges that can vary a fair bit:

• Storage is a flat cost per unit (based on size), but the cost varies depending on:

  • Whether units are in FBA or AWD (the latter being Amazon’s upstream storage; it’s cheaper but units must be shipped from there to FBA before they can be sold to customers)

  • What time of the year it is (in Q4 storage costs triple)

  • Whether units have been sitting for >180 days, in which case they incur a long-term storage fee

• Returns

  • I need to account for the fact that when a unit is returned:

    • The referral fee gets refunded to me but the fulfillment fee does not

    • For some brands, I allow Amazon to determine if the returned unit is in sellable condition. If it is, it goes back into inventory, and if not they dispose of it (for a fee, naturally).

    • For some brands with lower-volume, higher-cost products, especially those where I don’t trust Amazon to evaluate condition, I have them shipped back to my house. I evaluate the quality, and if anything’s missing I file a case with Amazon and get reimbursed. When I need more stock at FBA or my garage is getting too full, I ship the sellable units back.

• Promotions

  • Amazon charges you to do things like put a coupon on your listing or run a sale during Prime Day.

  • Coupons are a fun example of crazy Amazon fees that surprise most people:

    • To run a coupon, you pay $5, then you also pay 2.5% of the revenue generated by any sales for which the customer used the coupon. (If you’re not familiar with coupons on Amazon, they literally just involve the user checking a box. A lot of sellers are infuriated by this fee structure, but I’m just awed at Amazon’s ability to wield capitalism to the fullest!)

      

    • Also, when someone uses a coupon, instead of taking the price that the customer paid as the revenue from the sale, Amazon credits you with revenue of the non-discounted price then deducts the discount plus their 2.5% as a fee.

In all of these cases, I went back and forth and landed on some simplified models, since I can just leave some buffer cash in each brand’s account and thus don’t need to be perfect to the dollar.

For storage, I had Claude assume I’m never paying long-term storage fees and make some straightforward assumptions about FBA vs. AWD costs for brands that use both. Returns are modeled based on historical data, and we mostly ignore cases where they’re shipped back to my house, since the numbers aren’t huge and the fees are eventually offset by revenue generated from reselling them. Promotions are accounted for by just using historical averages of promotion costs.

Building the Dashboard

With all of that done, we get to the fun part of actually getting my dashboard built. This went through the standard cycle of working with Claude. First, I get a beautiful UI:

Then I start digging and finding issues, which leads me to add more output on the dashboard screen in order to check Claude’s numbers.

A few I ran into here:

Starting cash was wrong. It was pulling from the wrong column on my Google Sheet. Claude did not find an issue with the fact that most of my brands had very large negative amounts of cash. C’mon, Claude, I’m better at business than that.

Projections were clearly off. I had it add charts of sales history and projected 18-month sales, as well as a card showing the method it was using to make projections, so I could find the source of the issue. Turns out it was not using the /inventory projection skill despite my very specific instructions. I chastised, Claude fixed.

Amazon fees were double counted. Initially, Claude just took the historical % of revenue that was paid out to me as cash for each brand and used that to estimate cashflow. The problem was that it also found the historical average storage and promotion fees, which it subtracted from the expected cash coming from Amazon, even though those were already included in its initial calculation. After that, I went through the more detailed process described above for modeling fees.

Once I was satisfied that everything was accurate, I distributed myself some money.

Then I started adding features, because why not?

First, the ability to adjust the assumptions baked into the model:

Next, a list of cards with projected reorders for easy tracking.

After that, a nice little tooltip when you hover over a dot on the chart that shows what’s happening that month.

Now, in about half a day (which, to be fair, excludes much more than half a day spent on the /inventory skill), I’ve got a dashboard that lets me keep myself paid without pulling too much out of the business or monkeying around with spreadsheets a couple dozen times a year.

One final thing worth noting was that after the dashboard was built, I had to figure out how often to update it. During development, Claude just pulled numbers and ran /inventory as needed, but I don’t want to have to ask it whenever I need it refreshed.

The limiting factor was that the /inventory skill takes a fair bit of time and is pretty token-intensive, so I don’t want to overdo it there. I ended up landing on weekly updates of each brand, staggered over the course of the week. Each day, it runs /inventory for one or two brands then updates the dashboard with the results. That is frequent enough given how often I place orders, but back-of-the-napkin math said it wouldn’t blow through so many tokens that I’d have to upgrade my Claude plan.

Now that this is up and running, the next steps are to integrate it with my supplier dashboard to track orders that are in flight and to start notifying me when I need to place new orders. Tune in next week for that!

Share

Keeping everything straight has become a bit of a pain, so I decided to have Claude pull all of these conversations daily in order to create a dashboard that tracks the status of all in-flight orders and highlights anything I need to take action on. This turned out to be more trouble than initially expected.

Getting Messages

In order of least to most challenging to implement:

Gmail

Claude Code is already reading my email for other tasks, so I knew this would be a straightforward channel to start with. There are a few ways to give it access, but the easiest is just to get an app password. For security reasons I wouldn’t use this for any kind of public-facing software, but for an app that only exists on my local machine, it’s fine.

I gave Claude the password along with a list of all of the suppliers I communicate with by email that contained:

• Email addresses

• Names

• Which brand(s) they’re associated with

• Their role (manufacturer, freight, inspector)

It uses IMAP (Internet Message Access Protocol) to read my inbox, because it’s a very old, reliable protocol. First thing in the morning, it checks for emails from all of the addresses on the list, grabs any that are new since the last time it ran and saves the content of each to a JSON file. If there are attachments to those emails, it downloads and saves those as well.

WhatsApp

WhatsApp doesn’t make it quite as easy to get access to your messages, but because it has a web app, it’s not too painful. Claude started by firing up WhatsApp Web, which I logged into, and it saved the authenticated session. I then gave it a list of suppliers to track.

Now each morning it uses a Node.js library called whatsapp-web.js to spin up a Chromium browser, load WhatsApp Web, and then gather the relevant messages. All of those, along with any attachments, get saved locally.

Alibaba

This is where it got a bit more challenging. There is no API or other protocol that would allow me to access my Alibaba messages in a Claude-friendly way. Their TOS prohibits bots/scraping/etc., and since it’s pretty important that I not get my account banned, we have to do this in a low-risk way.

We’ll have to access messages through the web app, which means using Playwright (a library that can be used to programmatically control a browser). Similar to WhatsApp, this started by launching Chromium and going to Alibaba, where I logged in, then saving the authenticated session. After that, I gave it the supplier list, this time including the URL for my conversation with each one.

Each morning, Claude fires up the same session, checks if it’s still authenticated (if not, it’ll notify me), then goes directly to the URL of my chat with each supplier, where it takes a screenshot of the page. I settled on this approach to minimize the possibility of detection by Alibaba — there’s no interaction with the page at all, just opening it, screenshotting it, and closing the tab.

Once all of the conversations have been screenshotted, Claude opens each image, extracts all messages and saves them in JSON format.

There are a couple of downsides. First, if the supplier has sent so many messages since the last time we scraped that they aren’t all visible on screen, it will only get the ones it can see in the screenshot. If this happens, Claude can tell because there are no duplicate messages from the prior run, so it notifies me. In practice no supplier ever sends that many messages in a row, so it’s never been an issue. The bigger problem is that Claude can’t download attachments. If any files have been attached, it will flag that to me so I can manually download and save them later. Not ideal, but fine for now.

WeChat

This was the most painful. Like Alibaba, there’s no good way to access these messages programmatically and a TOS that frowns upon exactly what I’m trying to do. Claude laid out a few options for me.

First is memory scraping, which is to say going into WeChat’s working memory to copy messages from it. Second is database extraction, which entails pulling the messages out of the local database where WeChat Desktop stores them. These are both definitely doable, but Claude informed me that they’re also detectable by WeChat and could well get me banned. No dice.

So instead we’re left with option number three, which is Windows UIAutomation. That’s an accessibility API that screen readers use to see and interact with what’s on your monitor. It’s clunky, but it’s also indistinguishable from the way an actual blind person would use WeChat and thus highly unlikely to get me banned. (I will also note that Claude does have a computer use skill that would work here, but it’s Mac only and I’m a PC user.)

I gave Claude the WeChat supplier list, and it built out the flow through a bunch of trial and error interacting with the WeChat Desktop app. I asked Claude for interesting technical details, and it noted that one limitation of the accessibility tree (which is what it gets from Windows that tells it what’s on the screen) is that it doesn’t identify who a message is from. It could’ve attempted to infer it from context, but instead it samples a bunch of pixels on both sides of the screen on each message. My icon appears on the right and other people’s on the left, so it uses that to tell where the icon is and thus attribute the message.

A limitation is that if there are multiple people in the chat, it can’t tell the non-me participants apart with this method. I could’ve had it screenshot my screen and use the image to attribute messages, but I couldn’t come up with a scenario where it would really matter which of the other parties in a chat sent a message. All of my chats with multiple other parties are just two or three people from a manufacturer, and there’s no need to distinguish among them. If I had chats with, say, a manufacturer and a freight forwarder, I might need to implement this. It also can’t download files (or rather it could but in a brittle way with simulated clicks, so I skipped it for now), so it takes the same approach as with Alibaba.

Now that it’s up and running, I just have to leave WeChat open and logged in. Claude brings the window into focus, grabs the messages, dedupes them against the existing log and writes new ones to it. If the window’s not open, it flags that for me.

Creating a Unified Dashboard

Now that we’re gathering all of the supplier messages, the next step is to turn those into something useful. What I ultimately want here is a UI that tracks the status of all of my orders and lets me know if I need to take action on any of them, either because they’re waiting on a response from me or because a step in the process has not happened by its expected completion date.

If you’ve been following my Substack, all of this may sound familiar to you, because this particular dashboard was the topic of my post about giving Claude Design a test run. I won’t rehash all of that, but the quick summary: we ended up with two views, one triage-style that shows me what I need to do, and a Kanban-style board that gives me an overview of where every order currently stands in the process.

In order to translate messages into items on the board, Claude has a post-collection processing step, where it reviews everything that came in that day and determines if new orders need to be added, existing orders need to be updated, or action items need to be created for me.

This step is handled by a headless call to Claude Code (claude -p), and it runs through a fairly complex process to determine which actions to take:

1. Reviews all of the context files with information on suppliers, plus the database with the current state of all orders.

2. Determines which order each message should be attributed to. This is challenging, because each order might have messages from multiple platforms. A common scenario is I’ll place the order on Alibaba, then when it’s complete reach out to an inspection service via email, and when it’s inspected and ready to ship, get quotes from one freight forwarder via email and another via WeChat. When I’m getting quotes via email I always maintain one thread per shipment, but on WeChat a forwarder might send me an update on a shipment in transit and then quotes on a couple of different shipments I’ve inquired about. Claude has to use the message content to attribute each of them, and I have to say it’s impressively been 100% correct so far.

3. Determines if a message warrants the creation of a new order, a status transition on the board, and/or an action item for me. If a manufacturer has told me my order is ready, it should transition to Ready to Ship status and create an action item for me to get freight set up.

4. Extracts products, numbers, dates and shipment IDs. The orders are titled as <number of units> <product>, so if it’s creating a new one it needs to get those values. Dates are typically when the message was sent, not extracted from the message itself, and they’re used to track when status changes happened. Once I’ve created shipments in Amazon, I send those IDs to the freight forwarder. Claude looks for the specific Amazon shipment ID formats, so it can pull those out and put them at the top of the detail UI for easy reference if I need to find them in Seller Central later.

5. Tracks participants and records messages. The detail page has a list of every message with a brief summary, so Claude adds any new ones to the relevant order. Emails have a link to the thread in my inbox from the little red envelope icon.

   

6. Updates primary supplier. Each card on the board identifies the supplier currently responsible for the order. That changes as it goes from manufacturing to inspection to shipping, so Claude updates this when needed.

7. Adds supplier notes. The long-term goal here is to actually get Claude to interact with my suppliers and place orders when needed, so I have it keeping notes on each of them — things like whether they’re responsive or need to be chased, and whether they’re familiar with Amazon’s processes and requirements or should be given explicit instructions.

We had a few minor hiccups with the dashboard early on, but overall once we got the scraping figured out, the rest went very smoothly. Really the only things I have to do manually are grab attachments from WeChat and Alibaba, plus archive orders once they’re complete. The latter I intend to automate — I already have Claude running a daily script to check for orders whose status has updated to Closed on Amazon, so I just need to get that integrated here.

The big next feature is to have Claude start running my inventory skill across all brands once a week or so, in order to create action items for me when it’s coming up on time to place a new order. Once that’s reliable, I want to get Claude actually placing those orders and communicating throughout the process. To have it fully take that over, I’ll probably need a Windows computer use skill, so it can use WeChat Desktop without getting flagged as a bot. With the pace of AI progress, it probably won’t be long.

Share

Take a moment to let those sink in — we’re now at a point where you can just describe more or less whatever you want to an incredibly specific level of detail, including any text, and get a perfect image on the first try. It’s only been ~3 years since people were chortling about how AI would never be able to get hands right. If that sounds like a long time, it’s only because the speed of AI progress has totally warped your understanding of what normal timeframes for technological advancement look like.

That said: you do not need to generate any of the images above. They’re neat, but they have no actual value for anything you’re doing.

If you’d like to figure out how AI image generation can actually be useful to your business, though, you’ve come to the right place. Here at The Automated Operator, we specialize in applying the most powerful technology ever created by man to boring business problems!

Minor Design Edits

When I buy brands, I get design files for things like packaging and insert cards. Sometimes those are vector images, but since I buy from people who are not always the most sophisticated business owners, I will sometimes just get a JPG.

Case in point: This year I bought a brand that sells baby mobiles, and I got this JPG of their insert card:

Post-acquisition, I always immediately shut down social media accounts and get rid of any email addresses to save myself the time of dealing with them, so I needed to update the card to get rid of those. I had been putting off doing that, but I just got to the point of needing to place my first order, which means it had to be done.

Normally, I’d just recreate this from scratch. Maybe 30–45 minutes of work, so not the end of the world. But Images 2.0 just dropped, and I’ll take any excuse to try out new models.

Prompt:

Can you modify this image to adjust some of the text? Change the second paragraph to say: "Customer service is my top priority. Should you have any questions or concerns, please send me a message through Amazon." Remove the last paragraph entirely (along with the Instagram logo). Shift the thank you text at the top and the three paragraphs down a bit so that there's similar spacing from thank you to the top of the card and the Emmnnic logo to the bottom of the card. No other changes. Keep the same image size.

(The actual prompt was riddled with typos that I have corrected out of embarrassment.)

Result:

Even if I had an editable file, that’s a perfect result faster than it would’ve taken me to fire up an editor, make the changes and save it.

UI Design

Last week’s post was about using Claude Design to help create a dashboard. I had mixed feelings about its performance, and I only got through one of the tasks I wanted to try before I hit my weekly limit. The second thing I was planning to attempt was a UI refresh of a useful-but-extremely-ugly dashboard I use to monitor the performance of my Amazon ads. Let’s see what Images 2.0 can do with it.

Prompt:

I have this dashboard that shows me the changes I've made to my Amazon ads. The objective is to make it easy to quickly see what changes have been made and what the effects of those changes were to help me assess the performance of my ads and inform future changes. It's functional but pretty ugly. I'd like you to create some designs that make it more readable, useable, and generally aesthetically pleasing. You should review it first and ask me any questions about the information on it, its functionality, or my preferences as a user that you need in order to do the best job with your redesign.

The first thing to note here is that it did a much better job of asking me questions than Claude Design. Lots of specific requests for input about exactly how I use this and what features are most valuable. Unlike Claude Design’s use of a static form to gather input, ChatGPT asked me in the normal chat UI, so I was much better able to give it thorough, detailed answers.

Result:

This is an improvement! Some obvious UI wins here, like moving the campaign-level before/after stats to a side-by-side view. It’s also more clearly surfacing which campaigns have sufficient data since their last change to be worth another look, which is helpful in directing my attention where it needs to be.

It does feel like there’s a little too much going on, but in ChatGPT’s defense, it was already working from something crowded and didn’t remove any functionality/information. Still, I suspect a great UX designer could’ve talked this through with me a bit more and helped me think through ways to reduce the amount of info shown at the top level.

Instructions

One brand I own sells indoor swings that you have to mount from the ceiling. I am dealing with a relatively high return rate on that product, and I suspect one of the reasons is that the existing instructions are not that easy to follow and have no visuals. Improving those instructions has been on my to-do list for a while now, so I let ChatGPT give it a whirl.

First, I connected Google Drive and pointed it at the folders with product photos and the design file for the existing instructions. I told it to create an instruction sheet, pulling relevant photos and using them as visual aids.

Unfortunately, it failed spectacularly — the result was an instruction sheet for installing a swing, but it looked nothing like my swing and did not have the correct directions. When asked, ChatGPT admitted that it had not been able to open any of the images and just made something up.

So, for try number two, I downloaded a dozen relevant photos and uploaded those plus the existing instructions directly to ChatGPT. This time, it actually delivered:

Coherent, well-structured and with the correct directions. Not bad!

That said, if you look closely, you’ll notice a few issues. The little heart at the top under the title is partially obscured, and the line to its left is lighter than the one to its right. More importantly, ChatGPT made some unhelpful edits to the photos I gave it. The left-hand one for step 2 doesn’t show the correct bracket, and the right-hand one has the hands holding some weird metal piece that doesn’t exist. Some of the photos in 3 and 4 don’t really illustrate the correct things, either.

Still, in the grand scheme this would honestly be fine to use as-is. Nobody’s going to notice the details in the pictures, and they’re illustrative enough that it’s an improvement over what I have.

I did take a pass at having it fix the issues, but it mostly didn’t work. It was able to correct the heart and adjacent lines at the top, but there was no getting it to use the right photos. I re-uploaded the images I wanted it to use and gave specific instructions about which slots to place them in, but it still made edits and put them in the wrong places. In the end, I had to drop the correct images in the old-fashioned way. So while this is a truly magical product, there’s still a little room for improvement.

Product Listing Images

All the way back in January, I vibe coded a whole app to help me generate listing images for my e-commerce brands. It went through a multi-step process of gathering information, checking reference styles and ultimately creating images.

As I was putting together the instructions for the swing, it occurred to me that it’d probably be helpful to have a listing image on Amazon showing what’s needed to install it, so as to avoid people ordering it, realizing they need a stud finder to install it, and then sending it back. Instead of breaking out the ol’ image generation app, I went to ChatGPT.

I uploaded the existing listing images and a photo for it to use in the new image, and I gave it this prompt:

These are some existing images from a listing I have for a swing on Amazon.

I need you to make one in the same with the following info:

Title: Safe, Secure Installation

Text:

One group of text titled "You'll need" with the list items: Stud finder, drill, socket wrench, pencil

Another group titled: "Instructions" with the list items: 1. Find a stud 2. Drill holes 3. Install the swivel mount 4. Hang the swing 5. Have fun swinging! Image: I attached an image of a drill - use that as the main photo for the image

Result:

It matches the existing images perfectly! The big takeaway here is that as the core models get better, the need for scaffolding goes away. Expect this to continue — processes that you once had to orchestrate through a carefully designed flow of steps will be reduced to just asking your agent to take care of something.

So remember: it’s important not to get caught up in all the impressive demos you see when new AI products launch, lest you convince yourself to think of them only for grand, large-scale projects. You’ll get infinitely more value figuring out how you can use them to save yourself 30 minutes on a task you do a couple of times a week.

While most of my work is not especially UI-heavy, I do sometimes have occasion to ask Claude to create dashboards when the terminal isn’t sufficient for everything I need to see. These tend to be more functional than aesthetically pleasing — here’s one that I use to view historical changes to ad campaigns:

Right before Claude Design came out, I was beginning to give Claude access to my supplier conversations with the eventual goal of having it work directly with them to get inventory ordered and shipped. Even I’m not AI-forward enough to let it do that today, though, so in the interim I was planning to have it create a dashboard to track the status of all of my orders.

I’m going to test Claude Design by having it design the order dashboard. I will be writing this as I go, so I have no idea how it’ll turn out. Follow along and find out!

Order Dashboard

Claude has access to all of my supplier conversations that happen over Gmail and WhatsApp, and it checks them once a day. If there have been any updates, it notes them to help me keep tabs on what’s in flight, when it’s expected to be done and whether I need to follow up on anything (or chase a supplier who owes me a follow-up).

I always like to start new projects by just throwing some half-baked context at AI and seeing what it comes back with. Here, I gave Claude Design access to the directory where Claude Code has been building out the infrastructure to access my supplier conversations. Since I had already explained the ultimate goal to CC, I asked it to draft a prompt for CD to design the dashboard. Result:

This is not great, and I’m honestly a bit surprised. It’s the kind of thing I’d expect from an amateur designer who’s more focused on cramming everything in than making thoughtful decisions about how best to direct the user’s attention.

A few specific notes:

• In the bottom right, there’s a list of suppliers. I know who my suppliers are! This is not a good use of space.

• The Misc Inventory box does not need prominent placement. This shows non-product inventory (e.g. extra product boxes that a supplier is storing for me), and it’s something that’s nice to be able to reference when needed but absolutely does not require this sort of top billing.

• The Active orders section (which I will pedantically note does not have the second word capitalized, where the Misc Inventory section does… c’mon now, Claude, where’s the attention to detail?) is just kind of all over the place. There’s a bunch of extraneous stuff like which platform I use to communicate. The little bars showing timelines aren’t especially useful. The placed/shipped dates are duplicated on the line with the bar and then immediately below.

• The aggregate numbers at the top aren’t useful. I care about individual orders, especially those that are late or require some action on my part, but the total number of units inbound across all brands has zero value.

I’ll take some responsibility here, since I was lazy and had Claude Code write the prompt for Claude Design, plus I just yeeted the full project directory at it, and there’s a bunch of extra context there.

On the other hand, not even the worst designer I’ve worked with would just take all this and go start mocking stuff up. Half the job is making sure that you understand the fundamental problem the user is trying to solve. All of which is to say: Claude Design really needs to interview the user.

So, next prompt:

Do you have access to skills like Claude Code? This design really missed the mark, and I think it would help us both if you did an in-depth interview of me to understand the problem I'm trying to solve here before taking another shot. Can you use the AskUserQuestions tool or something similar? If not I will do that with Claude Code and then give you the output it gives me.

Result:

The form had a solid, if fairly standard, set of questions. This strikes me as pretty suboptimal when you have an LLM that is capable of conducting an actual user interview, but maybe it’d just end up asking roughly the same things so there’s no practical difference.

After I filled it out, the next set of designs was a lot better. One major improvement was that this time I got three versions. The form asked if I wanted to see different options, and when there is effectively zero cost, who wouldn’t?

Two of the three it gave me were solid. First was a triage-style board, ordering things based on my need to take action on them. It handles the task well, making sure that I know exactly what I need to do while providing all the relevant context.

Second was basically a Kanban board, displaying orders by where they are in the process. This solves my problem of constantly wondering if I’ve forgotten something, then going to my operations docs to remind myself how I communicate with the supplier before actually finding the conversation. It also does a decent job of highlighting where I need to take action, but I really prefer the triage view for that, to make sure I don’t miss anything.

The triage view is perfect for checking each morning to make sure I’m not missing anything, and the Kanban board gives me the ability to see everything at once. I’m tempted to just have Claude build both, but might as well see what our new designer can do. Next prompt:

I like the triage and board views. Triage solves my problem of needing to make sure I don’t miss anything, and board is great for giving me the status of everything in flight at a glance. I might just build both of those, so go ahead and save whatever information Claude Code would need for that. But then let’s see if you can give me a single screen that provides the best of both — alerts to make sure I know what needs my input, and the comprehensive view that I get from the current board.

The initial result was just a single page with the triage view above the board view, so I gave it a hard time and told it to do better.

Something a bit more interesting this time, but I still feel like the separate triage and Kanban board views work better. They really serve two different use cases for me, and combining them just puts too much in one place. I’d be curious if some of the better designers I know could make a combined view that really works, but it’s fully possible that it was just a bad idea to start with.

I had Claude Design output the files needed to implement both views, and Claude Code nailed the dashboard on the first try. Gotta love when design and engineering work well together!

I was going to follow this up by having Claude Design redesign the existing, very ugly dashboard in the first screenshot of this post, but I hit my usage limit after I got started. Apparently this thing has its own limits, and I don’t get to play with it again until Saturday (I am writing this on Tuesday). Someone get Dario more GPUs posthaste!

Lessons Learned

Claude is, at least for this task, a solid if unimaginative designer. This largely parallels my experience with Claude Code — it’s really good at building what you want when given clear instructions, but if you start giving it latitude to make choices, it’ll frequently just do the obvious/lazy thing. You can get it to do better, often by just telling it to do better, but it remains strange to me that there’s value in just sort of prodding the model. Perhaps even more so when we’re talking about basic functionality that could trivially be handled via the harness, like making sure the first task is to interview the user.

That said, it still behooves us all not to forget that it is an astonishing feat of technology that I’ve got a very competent junior designer at my fingertips. There is a tremendous amount of work done by junior designers today, after all. And in fact, in my perhaps-too-frequent browsing of X, I’ve come across a number of posts that say Claude Design is excellent at taking a company’s style guide and applying that to new screens or products, which is traditionally the remit of junior designers.

This leads to the sort of thorny questions that are becoming increasingly common with AI — if your head of design can create a style guide and Claude can faithfully implement it, what happens to all the junior designers? Nothing good, I fear. My guess is that top-notch, senior design talent will shoot up in value, but junior folks will have a hard time.

When I was a PM, one of my favorite parts of the job was working through UX with my designers. They’d show me some options, I’d give feedback, and we’d repeat until we had an elegant design that solved the core problem. At this point, Claude Design certainly can’t fully substitute for that, but if I were still a PM, I would absolutely go through a few rounds of iteration to prepare to meet with design.

The crux here is that design is basically free now (strict usage limits aside, at least). Not great design or deep creativity, but the ability to get ten versions of something while brainstorming or to apply your style to a new page. At basically every place I worked, the design team was the biggest bottleneck, so it’s a meaningful change. Hopefully the tools we use get a bit more beautiful and usable as a result.

That milestone, plus the fact that it’s been almost exactly six months since I started, makes this feel like a good week for a big-picture review of where things stand in terms of using AI in my business.

When I started, models were not capable of doing much legitimately useful work for me. Just six months later, AI is fundamental to how I run every single part of my business. An absolutely bananas rate of progress.

This post should serve as a nice snapshot to look back at six months from now and see how far things have come. I’ve linked to all the relevant prior posts if you want to dig in on any of these topics. Anyway, let’s get to it!

Brand Acquisition

As you probably know by now, I acquire e-commerce brands that sell on Amazon (and if you did not know that, welcome!). I find them on marketplaces (Flippa and Empire Flippers), usually via emails they send with listings. For the ones that show promise, I check out the financials, talk to sellers, make an offer and then ultimately take over the Amazon Seller account.

Screening Prospects

The emails I get from the marketplaces have a pretty low hit rate. Flippa just spams me constantly, and EF is a bit better but still mostly sends things that can be safely rejected after a cursory review.

My pre-AI flow was:

1. Get an email from a marketplace.

2. Read through the info about each listing in the email, disqualifying any that aren’t a fit. Even though there’s pretty minimal info (listing price, TTM sales and revenue, a brief blurb), it’s usually enough to disqualify most listings.

3. If any survived the initial pass, go to the marketplace, review the additional information there (generally a longer writeup about the business and potentially some more financial detail) and see if there is anything disqualifying. This eliminates all but a rare few.

4. Download the financials and any other documentation. Review everything to see if there are any glaring errors, though at this point I’m likely going to set up a call with the seller, so I’m also looking for places I can improve the business if I buy it.

5. Set up a call with the seller.

6. Review any information they’ve provided during and after the call. This is usually just getting reports straight from Amazon that I can use to validate their P&L.

I am handling a healthy chunk of that with AI:

Screening Emails

I have a Google Apps Script written by ChatGPT that runs daily, checks for relevant emails (anything from EF and any email from Flippa that has the word “FBA” in it) and sends them to Gemini Flash for review. Gemini has a list of reasons to disqualify and checks each listing against that. If there’s any ambiguity, it errs on the side of not DQing.

Once all emails have been reviewed, it generates an email to me that highlights any listings that have not been eliminated. The email allows me to mark a listing as good or bad for my own future reference. The main value is that if I mark it as bad, the screener won’t show it to me again. This is helpful because both marketplaces send emails notifying me about price drops (which are important emails), but if the listing isn’t a fit, I don’t care if the price drops later.

Evaluating Listings

For listings that pass the initial screen, I have a very long prompt that I use to do a full evaluation. I give a SOTA LLM (these days Claude, but originally I used ChatGPT) a copy of the listing and all the attached files. It goes through all of my criteria and gives me an evaluation.

If there are obvious reasons to disqualify, it starts there. If not, it gives me pros/cons, areas where I might be able to improve the business and a list of questions to ask the seller.

More on that one (though this is one of the first posts I wrote and probably pretty out of date by now) here:

Cleaning Up P&Ls

While the above step generally handles the P&Ls fine, there are cases where it misses important context. I buy very small brands, so pretty frequently I get P&Ls that are constructed by people who are not great with financials or numbers in general. These can be so bad that LLMs struggle with them.

The worst one I’ve ever seen was a guy who put his advertising expenses in the revenue portion of the sheet. Not as a negative number, mind you — he had the amount he was spending on ads listed as an amount of money he was bringing in. Turns out his margins were 0%, not 30%. ChatGPT did not catch this, presumably because it was so egregious that it didn’t even think of it as a possibility.

LLMs are getting better here, but even when they can reliably handle the first pass, I’m still going to want to read the P&L. So for that, I found Shortcut, which is just an Excel-based AI agent. I can give it a spreadsheet that’s just absolute garbage and say “clean this up.” It will ask the right questions then give me back something that looks great.

What I’m Not Using AI For

This is a place where I’m really using AI as a second set of eyes more than a full replacement for my own work. It’s probably the most important area of work, plus it’s pretty interesting, so it just doesn’t make sense to offload it. Still, the screener is helpful, and the listing review is useful in that it will pick up something that DQs a listing immediately that would take me an hour or two to find. In those cases, I can just confirm what it’s telling me and move on, which is a real time-saver.

That said, I would still like to get this to a fully unified workflow. Right now I have to give all of the listing info and analysis prompt to an LLM, where in practice this would happen automatically for any listing that passed the initial screener.

When I started working on this, there weren’t sufficiently reliable browser use capabilities to manage that, and automating the analysis would have meant spending extra money on API tokens rather than doing it via my subscription. We’re now at the point where I could have Claude Code do all of this end to end. It’s on my to do list to make that happen.

Negotiation and Closing

This phase is the most manual, both because it’s the highest-stakes and because it’s important for me to build rapport with sellers while working with them. That said, AI definitely assists.

Contract Review

I have a contract template that I update for each deal. They’re pretty simple, so usually this is straightforward. Still, it’s always helpful to have Claude give it a look. I have it check a few things:

• I give it all deal details and any changes I intended to make to the template terms, and it checks whether they’re all reflected correctly in the doc.

• Any counterparty redlines get a review just for a helpful second pair of eyes.

• If I’m buying something in a category that’s new to me, I’ll have ChatGPT do some research on whether there’s anything category-specific I should be aware of and include (e.g. safety testing for toys). I use ChatGPT here because Claude is not especially reliable for research. (Fun aside — when I had Claude proof this post, its feedback on this bullet was “This is going to read oddly to anyone who knows Claude now has web search. Might want to note this was true at the time or clarify what kind of research you mean.” Defensive much, Claude?)

Due Diligence

There is generally not a lot of diligence required, because once the LOI is signed I get access to the seller’s Amazon Seller account, which makes it trivial to check their numbers.

I mostly use Claude to double-check everything I’m doing. I’ll throw it the various reports I’m pulling and just have it compare those against the P&L. If there’s something that requires digging, it can be helpful. In one case, one of the seller’s Amazon fees was a higher percent of revenue than expected. Claude reviewed a handful of orders and found that it was because that person was in a country that charged a digital tax, which would not apply to me once the account was handed over, so no issue there.

Post-Takeover Playbook

The immediate work once I’ve taken over a brand is where AI really starts to shine.

New Listing Images

When Nano Banana Pro came out, I could suddenly use AI for all of my product photography and listing images. I had Claude vibe code me an app that generates a set of Amazon-ready images that highlight all of the product benefits and maintain a consistent style.

When I take over a brand with bad images, I fire that up, generate new ones in less than an hour and get them uploaded. That process used to involve a bunch more work for me, time waiting for my designer to turn images around, and of course the cost of paying him.

New Text

The most important pieces of text in your Amazon listing are the title and bullets, because they’re the things people read and also what Amazon uses for SEO. When I take over a brand, I have Claude use Amazon’s APIs to find the most important search terms, then update the text to make sure they’re included. This was maybe 2-3 hours of work for me, and now it’s close to zero.

Ad Review

I often inherit Amazon Ads accounts that are spectacularly bad. I used to spend days untangling them. Now Claude’s got access to the Amazon Ads API and takes care of that for me. It does a few things:

• Checks for weird, bad stuff people do (like advertising against the same target in multiple campaigns)

• Reviews campaign structure

• Goes through the ads account as well as organic search tools that Amazon provides to find search terms that are leading people to buy the product but which aren’t being advertised against, so they can be added as new targets

• Finds competing products to advertise against

It then gives me a plan for changes to the ad account. I approve, and it makes them. It’s tough to overstate what a help this is. Amazon’s web UI for ads is truly horrible and incredibly slow. Even with a full plan for an account prepared, I’d still spend a day or two implementing it.

What I’m Not Using AI For

The post-takeover work is almost entirely AI at this point. The only big thing missing here is getting it automated, so once I take control, I just tell Claude to do its thing and move on to something else. That’s technically doable, but since this is infrequent and quite important work, I’ll stay involved for now.

Finance

Bookkeeping

I have always maintained my own spreadsheets of all the important financial info about my business, so my books were only ever used by my CPA at tax time. Given that, I just paid a cheap guy on Upwork to do them. Then I got Claude a QuickBooks API key, and now it has taken over that work.

Since this one always gets me the most “But how can you trust AI for this?!?” reactions, my quick answer: How can I trust some dude from Upwork who charges $10/hour? (The answer: I know the financials of my business well, so I have a pretty good idea of what the P&L and balance sheet should look like, whether they’re prepared by human or AI.)

Tracking Finances

I said above that I maintain spreadsheets with the important financial info for each brand; of course that has mostly been handed off to Claude too. There are a couple of big use cases here.

First, I have multiple brands owned by a single legal entity, so they all share a bank account. I want to keep track of each one individually, so I maintain one ledger per brand with every credit and debit. Once a month or so, Claude pulls all the relevant Amazon reports, grabs my Amex and Mercury statements, asks me to clarify any charges that it can’t attribute to a brand, then writes all of these to the sheets.

Second, I keep a sheet that has the important numbers for each brand by month — revenue, expenses, returns, etc. I used to have to cobble this together from multiple reports each month. Now I run a slash command.

No posts about either of these, but they’re coming!

Taxes

You should just read this post, but I’ll add one thing that has happened since I wrote it. Claude found an error with amortization in my 2024 return, so naturally when I got my completed 2025 return I had Claude check that too. Somehow my CPA managed to make an even larger error with amortization in 2025, less than a month after I made him amend my 2024 return.

The 2025 error was at least favorable to me from a tax perspective, but I am unfortunately a scrupulous taxpayer and had him fix it anyway.

What I’m Not Using AI For

AI is basically handling all of my financial reporting at this point. It’s not actually filing my taxes, but based on this year’s performance, I wouldn’t be shocked if next year were the year.

Operations

My big dream is getting AI to manage all of the boring, day-to-day work. Buying new brands is fun and exciting! Checking on ads every couple of weeks and doing inventory planning is not.

In the last six months, I’ve made a ton of progress in that direction.

Inventory Management

I’ve got a Claude Code slash command that looks at historical sales data, checks for any brand-specific context, uses that information to project the next twelve months of sales by month, and then gives me a reorder date that factors in each brand’s production time as well as other factors like Chinese New Year.

Operations Tracking and Alerts

There are a bunch of things I need to keep tabs on for each brand. Amazon could make any or all of these easy, but of course they don’t. Examples:

• When shipments of inventory arrive to Amazon, if any units are missing, I can file a case and get reimbursed. This requires tracking every shipment until it’s closed (and they’ll often sit open for a month or two even after they’ve been received). Claude checks all shipments daily and notifies me as soon as any are closed and missing units.

• In Amazon Seller Central, you can go to an order and press a button that will send an email requesting that the buyer rate your product. It’s important to do this, because it greatly increases the rate at which you get reviews, and reviews are the most critical component when it comes to your search ranking and conversion rate. Doing this manually is a pain, but there’s an API endpoint that sends the request. Claude vibe coded a script that runs daily to send this to all eligible orders.

• It’s helpful to know when products get new ratings, particularly if the average rating changes. Also helpful to know if you’ve gotten a badge (e.g. Amazon’s Choice). Amazon doesn’t provide this information through the API, so I have a job that runs daily to have Claude use Playwright to check all of my listings and let me know what’s changed.

Managing Ads

Unfortunately, Claude’s not yet up to the task of handling the day-to-day management of my ads (a lot more about that in the ads post linked above). It’s still helpful, though. For one, it has created me a dashboard that helps me visualize all of the information I need to analyze my ads’ performance in a UI that is much better than what Amazon provides (low bar, to be fair).

There is also one management task it can handle — when launching new campaigns, if you use Amazon’s suggested bids (which I do), most of the time they’ll actually be too low to get any impressions. Every couple of days, you check the ads and raise bids on anything that hasn’t spent by ~10%. This is a simple, clear flow that Claude is able to handle for me.

Filling Out Forms

At this point, Claude knows pretty much everything about my business, so when I have to fill out a form that’s going to take me more than a couple of minutes, I just ask and Claude handles it. The most recent example of this was a spreadsheet I had to fill out with product info for a safety testing company. Uploaded it to Google Drive, pointed Claude at it, got a completed spreadsheet.

Another one that comes up regularly is that the freight company for one of my brands makes me fill out an incredibly janky web form about each box they’re going to ship. I ship a lot of boxes, so this can take hours. The last time around, I asked Claude in Chrome to do it, and it nailed it.

What I’m Not Using AI For

Still a lot left here that I’d like to stop doing myself. I think we’re one or two model updates away from Claude being smart enough to manage my Amazon Ads, which would be a huge time saver.

I mentioned above that Claude tells me when shipments to Amazon are missing units. When that happens, I have to open a support case to get reimbursed, but unfortunately Amazon doesn’t have an API endpoint to do that. At this point, I could almost certainly create a role-restricted user on my accounts that only has access to support and have Claude use the browser to create cases, but I’m worried that Amazon would detect that it’s a bot and give me problems. This isn’t a huge time sink, so it’s not worth the risk.

The last big piece that I’ve only just started to work on is vendor communications. In an ideal world, Claude would keep tabs on inventory, place orders when necessary and just let me know so I can send the wire. We’re obviously a little ways away from letting that happen (probably moreso in terms of my level of trust than actual AI capabilities), but it’ll definitely happen eventually.

That said, for now I’m starting to set up the infrastructure. Claude’s getting access to all of my supplier conversations, so it can maintain a dashboard with the status of all of the orders I have in flight. While it’s doing that, it can build up context about each supplier, how they communicate, whether they’re proactive or need to be chased, and any other quirks. More to come on that soon!

That’s the current state of affairs, and seeing it all written down in one place makes it feel pretty jaw-dropping. Check back in six months (or, y’know, just keep reading my weekly updates), and let’s see how far things have come!

Today I’m going to talk about a few more pieces of functionality I’ve added to Claude’s Amazon Ads repertoire and some of the infrastructure needed to keep the whole thing humming along. I’ll wrap it all up with some thoughts on what this all might tell us about how AI might impact the broader economy.

Creating Campaigns

In Part 2, I broke out Claude’s skills into a one-time audit that finds new targets to advertise against and a regular review skill. The latter still needs some polishing (more on that later), but Claude is really good at coming up with new targets.

Given that, the next step was to have it start actually making changes to my ads account in order to create all of the new campaigns it proposed. Always a little nerve-wracking to let your LLM start updating things in critical systems, but once more, unto the breach and all that.

I wanted to be as thorough as possible here to avoid any potential pitfalls, so I went through the process of creating a new campaign via the Amazon UI, recording every field and the appropriate value for it. This is really, really important, because if Claude doesn’t have specific instructions it will not infrequently skip things or guess at what it should do and not inform you until you ask.

With Amazon Ads, there are settings like this one:

When you’re creating a campaign by hand, you realize that of course you don’t want it spending a bunch of money to advertise your products outside of Amazon in places where they perform poorly. But Claude, as discussed, will not necessarily show that level of common sense and is liable to think that of course you want your ads to reach the masses. Thus, we need specific instructions covering every single setting.

Having gone through that, I had Claude create a new campaign with my detailed instructions then checked it carefully. Mostly good, except in some cases where Amazon did not have a recommended bid for targets (for simplicity I just told Claude to start at the recommended bid), it decided to use numbers that were bizarrely high. This is the sort of thing we very much want to avoid!

Otherwise, all good, so I added an instruction on how to handle targets with no recommendation, then I had it build out a campaign creation skill. One important note, though — for each campaign to be created, Claude gives me a list of all campaign-level settings, all keyword/ASIN targets, Amazon’s recommended bid for each and the bid that Claude was going to use. Only once I approve does it actually create anything.

The first couple of times I used it, I did thorough checks of everything it created, and when those were all good I moved to spot checks, and now I’m just lazy and check nothing. It’s funny how quickly it goes from being a little unnerving to let an LLM make changes to something that’s critical to my business to just being completely accustomed to it. The human capacity to adapt to a new normal is truly something.

Ongoing Campaign Monitoring

The other piece that I’d like Claude to handle is the day-to-day management of the ads, because that’s more time consuming and frankly boring. Unfortunately, Claude has no intuitive sense for the task, and it turns out that trying to document everything I do creates a skill that’s too long for it to follow reliably.

The problem is there’s just too much variance in what you need to do in order to analyze the performance of a given keyword depending on the context. You need to look back far enough to have enough data, which could be 7 days or 30 days or 60 days depending on how frequently-searched it is. But if the 30 days you’re looking at has a sale period like Prime Day in it, that’s going to throw things off. You also have to balance changing both the bid for the keyword itself and the placement adjustments. You must consider the impact of changes outside of that campaign entirely, whether that’s new campaigns you’re running that are pulling traffic away or new competitors that have popped up.

This is all just too much for Claude, which is somehow both surprising and not. None of the logic is complicated, there’s just a lot of it. I could perhaps codify all of it into a deterministic system and have Claude script that, but this is one of those times where it feels like the best approach is to ignore the problem and wait for a better model to come along that will magically solve it.

That said, its limitations have led me to think more carefully about what I can scope clearly and succinctly enough for it to accomplish. One tedious but important task is checking on new campaigns regularly to make sure they’re spending money. Typically, when you launch a new target at Amazon’s recommended bid, that bid won’t be high enough for it to get any impressions. I haven’t the foggiest why Amazon’s recommendations are so frequently bad, but that’s neither here nor there.

Thus, after launching a new ad campaign, you have to go in every couple of days, check bids, and increase the bids of any target that hasn’t spent anything by 10% or so. Claude can handle that! So I now have another skill just for that, and it’s not glamorous but it saves me an hour every couple of days.

I also had Claude wire up a dashboard for me that helps me more efficiently go through all of the things that I need to check to determine if I should modify bids or not. Amazon’s own UI is terrible and slow and presents information in ways that feel like they were designed by someone who has never actually used the ad system. Mine is very ugly, but it’s perfectly laid out for the way that I need to use it, which is an incredible benefit of being able to get custom software made.

On the one hand, getting a dashboard when I really wanted a digital intelligence to do the job of managing my ads is somewhat disappointing. On the other hand, the dashboard I have would have cost thousands of dollars if I were hiring a developer to build it in the pre-LLM days, so I can’t really complain.

Adding Some Infrastructure

As I started working on this project, Claude frequently hit Amazon’s rate limits. They’re not especially generous to start with, and because your Amazon Ads API key is tied to your user account (as opposed to their other APIs, for which it’s tied to the actual Seller Central account), I was sharing one set of rate limits across every brand.

The other very annoying thing is that Amazon will only let you pull ads data for the last 90 days. There are many cases where I’d like to look farther back, but even though I can see lifetime data in the UI, Claude was out of luck.

So I set off on a task that took much longer and was more involved than I anticipated but ultimately worth it: saving all of my historical ads data to a database, starting with a backfill of the 90 days available and then running a job that adds to it daily. Having it stored outside of Amazon would not only save Claude from having to hit the APIs whenever it needed to find something, but also allow me to build up a longer-term historical record.

I’m going to compress this slightly, because it really was a saga, but it’ll give you a pretty good idea of what it’s like to work with Claude on this kind of project.

First, I told it to pull all available historical data, save it to a SQLite database on my hard drive and back that file up to GitHub. Neither Claude nor I gave this plan the level of consideration that it warranted before it got started.

The backfill progressed slowly. Painfully slowly. I left that Claude Code tab open for six hours and it had made shockingly little progress. I asked Claude what was going on, and it said it was dealing with rate limits.

That did not feel like it was a sufficient explanation for how glacially slow things were moving. I prodded. Eventually Claude told me that unfortunately given the rate limits and the fact that it had to retrieve data one day at a time across a number of brands meant that this was going to take a while.

“Is there really no way to get more than one day of data with each API call?” I asked. Claude had not considered this. It investigated. Turns out you can get 30 days of data with each API call. I couldn’t decide whether I should feel frustration at its ineptitude or joy at knowing that for at least a little while longer, I could be smarter than a SOTA LLM.

Next problem: GitHub has a 100MB file size limit that we hit pretty quickly. Claude did some research and found that Turso should be able to handle something much larger for free. I’d never heard of it, but if it’s good enough for Claude, it’s good enough for me! Probably! Unless something goes wrong!

Finally we got the whole backfill completed.

From there, we moved onto building a daily job that pulls yesterday’s data and adds it. Claude wrote the script and set it to run via GitHub Actions. I went to bed, and the next morning it had worked as expected, though it took several hours. That seemed too long, but it wasn’t actually a problem for me, so I let it be.

Later in the month, when I was notified by GitHub that I only get a certain number of free GitHub Actions minutes and had exceeded my quota, it became a problem for me, so I investigated.

It’s been a little while since I dealt with this, so I don’t remember the full order of things that went wrong here. They included:

• Claude was writing the data to a local SQLite DB then pushing it to Turso. Which was fine, except it was pushing an update every single time it added something instead of just waiting for the full daily update to be complete. Turso kept closing the connections because they were open for so long, and Claude did not handle that well.

• At one point I built a dashboard on top of this data. Claude decided it would be most efficient to have the data in a local DB for the dashboard to use rather than read directly from Turso. So it created one and started to sync the data from Turso daily. You may recall from the previous bullet point that there was already a local DB. Claude had created a second one and was now pulling data from Amazon, storing it on the first local DB, pushing it to Turso, then later syncing it back to the second local DB.

• I only figured out the two DB thing because as I continued to make changes to things touching the DBs, Claude got confused about which one to use and started writing to the backup one, which was not syncing to Turso.

  The Automated Operator is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Eventually I finally spent half a day figuring out what was actually going on instead of just telling Claude to fix whatever the random problem of the day was. We got one coherent flow of data in place and everything that used the data pointing at the same local DB, and to boot I actually went through what was being captured and got rid of a bunch of stuff that was duplicative.

There is probably a lesson in here. Certainly the software engineers reading this are feeling rightfully smug about the fact that this sort of comedy of errors is what you get when you just start YOLOing things with LLMs. Honestly, if I had spent a little more time reading the stuff that Claude was writing before telling it to start building, I probably would’ve seen some of these issues coming.

But if we’re being honest here, I know that at some point in the not-so-far future, LLMs and their harnesses will have improved to the point that they will just get this sort of thing right the first time. As such, I decline to learn any lessons.

Where Things Stand and Next Steps

The long-term goal here is to fully delegate the management of my ads to Claude. As soon as I take a brand over, it audits the account, makes whatever changes it sees fit, and then checks back in at a regular cadence.

We are definitely not at a point where that’s anything close to a good idea. I’ve got Claude doing campaign creation, because that is definitely the place where it’s closest. But a big part of that is the fact that when you’re creating new campaigns, it’s a combination of two things that Claude Code is well suited to handle: rote, rules-based analysis and making stuff up.

The former is what it does when looking at existing campaigns to find keyword targets. It checks for search terms that are leading people to buy my products but that I’m not already targeting. This just involves pulling reports and doing some straightforward analysis. Claude isn’t actually doing the work; it just wrote the scripts that do the work.

Finding new products to advertise against is decidedly less formulaic, and I have actually been surprised at some of the positive results stemming from Claude’s creativity on that front.

I still haven’t really given it instructions much more specific than “look at Amazon’s product catalog and find products to advertise against,” and yet it finds things that I would not have considered. I have a brand that sells baby mobiles, and it created several campaigns targeting other mobiles by theme. One of these themes is mobiles that play music. The mobiles I sell do not play music. And yet, for some reason, that’s the best performing campaign.

Similarly, I sell a binder that is designed to hold seed packets. Before I bought the brand, the previous owner had an ad strategy that mirrored my typical approach — she targeted directly competing binders that hold seed packets. Claude decided to target stakes and trowels and anything related to gardening, and I’ll be damned if a bunch of those targets weren’t quite lucrative.

In fairness, a bunch of them were not lucrative, but that’s how these things go. You try a bunch of targets, some of them don’t work and you turn those off, but the ones that do make you money for hopefully a long time. I didn’t think creativity was where Claude would outdo me in my business, but there you are. LLMs are weird!

Eventually I’d like Claude to take over the day-to-day management of my ads, but we’re just not there yet. The challenges there really highlight the jagged frontier of LLM capabilities plus the limits of the value of raw intelligence. There’s nothing required that Claude isn’t smart enough to do; the problem is long-context instruction following.

I think we’ll see similar cases as AI rolls out more broadly through the economy. There will be places where sheer intellect will be useful, but I strongly suspect that the greater impact will be the ability to deploy an infinite number of pretty smart, very fast, always working agents.

Most of the work done in the world today doesn’t require genius-level intelligence, after all. Even in disciplines where we think of intelligence as being fundamentally important, it’s often not used with great frequency. My wife is a scientist, and while she and her peers spend a lot of time using their brains to come up with hypotheses and reason over data, they spend a lot more of it sitting in the lab pipetting and centrifuging (I don’t know if these are actually the things my wife does in the lab but they are the lab tools that I’m familiar with, so we’ll pretend), attending meetings and creating presentations. I was a product manager, which I choose to believe people think requires intelligence, but I spent a lot of time updating JIRA tickets and spreadsheets. It would have increased my productivity much more to have someone handle those tasks than to have them take over the time I spent pondering product direction and priorities.

Overall, I have deeply enjoyed this project both because it’s taught me a lot about how best to work with Claude and also because it has made me a meaningful amount of money. Three of the brands I own had their best months of revenue and profit after I unleashed Claude on their ads. Most of the ones that didn’t are seasonal and not at peak season, but they showed substantial YoY improvements. I can see the new ad targets generating additive revenue, and I can also see some of the cleanup work that Claude has done making other campaigns more efficient.

The impact of all of this in terms of dollars in my pocket is orders of magnitude more than my lifetime LLM spend so far, and the increase in profit should continue on an ongoing basis. I say that largely to highlight how different the impact of AI looks in large-scale economic numbers, where it seems to be ambiguous at best, compared to its totally unambiguous effect on my business. A big part of the reason I decided to start writing this Substack is that I think a tiny, simple business like mine is a useful leading indicator of how AI will affect the economy as a whole, and if you agree with that, then you won’t be surprised if we see some substantial changes in the next few years.

I also ran into a bunch of issues with Claude: it does a pretty minimal job if you leave your instructions open-ended, it has very little common sense, and it’s really bad at doing research online. Nonetheless, it delivered useful results, including new ad targets that have since made me real money, so I can’t complain! There’s also a brief primer on Amazon ads in part 1, so worth reading if you missed it the first time and aren’t familiar.

I started this project much in the same way I start most of my explorations with Claude — just throw a problem at it, see what I get and iterate until either it produces something of substance or I’ve concluded that it’s not going to be helpful. Since this first go-round with one of my brands turned out well, today I’m going to focus on the process of turning it into a skill that works consistently, so I can apply it across all of my brands.

Making a Skill

You almost certainly know this, but if you have a task that you want Claude to take on with any regularity, you’re going to want to create a skill.

Having just completed a run through of the process I wanted Claude to follow, I asked it to first write up a set of detailed, step-by-step instructions for me to review before actually creating the skill. It did a solid job here, breaking it down into the relevant sections:

1. Hit the Amazon Ads APIs to retrieve historical performance data.

2. Go through that data and:

   1. Find any search terms in broad/phrase/ASIN campaigns that are yielding sales.

   2. Identify keyword/ASIN targets that have a ROAS that is meaningfully higher or lower than what I’m targeting.

   3. Identify keyword/ASIN targets that have not spent any money in the recent past.

3. Pull the search terms report from the Brand Analytics API.

4. Review that report for organic search terms that are leading people to click on/add to cart/purchase my product, then check those against the list of search terms I’m already targeting to find the ones that I am not yet targeting.

5. Search the Product Catalog API for similar products to advertise against.

6. Present the results in the terminal, along with recommendations for changes.

Got the basics, but when building skills it’s important to get into specifics in order to get the consistent output that you want. This includes both telling Claude what to do and what not to do, and you’ll want to keep adding to the list as you see it gets things wrong in order to ensure it doesn’t repeat its mistakes. I started with a list based on my first run through:

• If you get a FATAL error from Amazon when attempting to generate a report, do not assume the report timed out; request more info to determine the cause. If it was a malformed request, make the request again correctly. If it was a timeout, try again up to three more times, backing off exponentially in between each.

• When evaluating the performance of ad targets, make sure you have enough data to draw conclusions. If a keyword has a tiny amount of spend, then a very high ROAS doesn’t tell us it’s good, and a lack of sales doesn’t tell us it’s bad.

• When pulling the Brand Analytics report, your request must include parameters that filter it down to just the rows relevant to the product(s) you’re evaluating. Under no circumstances should you ever pull the entire report and then attempt to filter after downloading it.

With all of that in mind, it created the /ads-audit skill. Time to give it a whirl!

I ran it against another brand, and the results were pretty good. It did all of the major tasks correctly and didn’t make any of the errors I had admonished against. It enthusiastically followed my command not to overindex on results from too little data, noting specifically about several targets that despite a very high ROAS, nothing should be done because the amount of spend was too small. Good job, Claude!

That said, there were still improvements left to be made. The output it generated was meaningfully different from my prior run in terms of structure and information provided, and the bid change recommendations were comically large and based on obviously very faulty logic. For a target with a ROAS of 3 where my target ROAS was 4, it would suggest cutting the bid by 33%. If you’re not familiar with PPC, suffice to say that’s not at all how those numbers relate to each other, and a cut of that size would probably just make it stop spending entirely. So, as always, more specifics.

I wrote up the desired output in detail, section by section. What info to include, how to structure tables, and what kind of commentary I wanted it to offer on why it made the suggestions it did.

For the bid change recommendations, I wanted to see if it could figure out how to size them correctly without me being completely prescriptive. I explained the flaw in its methods and told it to spin up a subagent to review my ad accounts and analyze historical bid changes and the resulting effects on ROAS. It spent a solid half hour on this and actually came back with a very solid understanding of how to size bid change appropriately, which I had it codify in the skill.

That said, it’s weird that I had to have it do this! If you read part one, you’ll recall that I mentioned one of the big flaws of Claude Code is a lack of common sense. Setting bids on PPC is not some esoteric skill that it should’ve had to derive from data! I am very sure there is a ton of information about this in its training data. It will never stop being strange to me to watch Claude Code bang out a thousand lines of code that works perfectly the first time, then follow it up with weird stuff like this that seems to indicate a lack of basic understanding of commonly understood concepts.

Anyway, from here it was just a lot of time iterating on the skill. I’ll spare you the details of the many hours of tweaks, but I will give a couple of suggestions on how to best offload your work to Claude via skills. But first, montage!

First, do the task yourself in parallel with Claude, and compare both the process and the outputs in detail. Often I’ll get the sense that a new skill isn’t quite working the way I want, and this exercise helps me understand exactly why.

In this case, Claude was looking at data from the last 30 days to determine whether to make changes. In practice, what I do is more complex. First I check to see when the last bid change was and look at data since then, then depending on how long ago that was and whether there were any unusual periods (e.g. Prime Day or BFCM), I might look at longer or shorter periods. For long tail keywords with very few impressions, sometimes I’ll go back much further.

There’s just a complex series of instructions in my head that I go through without thinking, and all of those need to be written down in order for Claude to do the task in the same way I do. This is pretty common for people doing any kind of repetitive task — if I just ask you to give me instructions and then I ask you to do the task, you will inevitably do a bunch of stuff that you didn’t write down because you just don’t consciously think through every step each time you do the task. So: actually do the task while you build the skill, ideally several times.

The second piece of advice I’ll give is that once you think your skill is complete, have Claude run through it at least a half dozen times. What you’re looking for is variability in the output, because unless you have very specifically told Claude what to do, there will be some variance — such is the nature of probabilistic systems. Sometimes this is fine, but you should make sure that anything critical is being done correctly 100% of the time.

Skills: Bigger Isn’t Better

Having said that, I will tell you that it has been a struggle to get this skill to work consistently. Here’s Claude giving itself an F on the job it did of following instructions:

The main culprit here appears to be the size of the skill — the first time I asked, it said it was 270 lines. It’s not necessarily that the skill itself is too big, but with the amount of thinking and output required to follow all of the instructions, the text of the skill itself was getting lost in the context window. Claude would still retain enough to generally do what it was supposed to but not to follow the very detailed instructions at each step.

I still haven’t managed to get it to perfect consistency, but here are a number of things that have helped significantly:

Ultrathink

If you weren’t already aware, if you type the word ultrathink anywhere in your message, Claude will think harder! It sets the token usage to maximum, which seems to help with following complex instructions. It also has a neat little UI:

Recently, Anthropic introduced the effort field as an option for the frontmatter in Claude skills. That basically just means that if you put “effort: <low/medium/high/max>” at the top of your skill, Claude will use that level of effort regardless of what you have set for your current session. Max is equivalent to ultrathink, so you can use that instead of typing it every time.

Output Between Steps

One issue I kept running into with my inventory management skill, which is also very large, is that it would do a step and then not use the output of that step in the next step, even though it was specifically instructed to do so. The most common example of this was that I’d have it check for brand-specific notes before projecting sales for the next 12 months, because there are a few brands that needed different projection methodology than the standard one in the skill. It would always check the notes, but half the time it would ignore them and just make projections the usual way.

I solved this by telling it to output the relevant information to the terminal at the end of each step before proceeding to the next one. Having it actually write, “For this brand, I should just take the average daily sales of the last three months and assume that will be the number of units sold per day instead of using the skill’s typical projection methodology” solved the problem. I honestly don’t understand why — before I did this I could see in the reasoning trace that it was getting the same information, but for some reason writing it as output ensured that it was used in the next step.

Subskills and Subagents

Following on that, where possible it helps to break the skill up into one main skill and a bunch of subskills that it calls along the way using subagents. If one step is to retrieve data, for example, hand that off to a subagent and have it output the relevant data to the main agent when complete.

This keeps a lot of unnecessary info out of the context of the main agent, whether that’s API calls to retrieve the data or a bunch of extraneous data if your subagent is filtering down a dataset to just what’s relevant.

The only caveat here is to make sure that the output contains everything the main agent needs. In some cases, it’s helpful for the main agent to have the full reasoning in its context, either because it’s needed for some future step or because you want it to explain why it made some choice at the end. You have to think through what the agent needs in advance, because if you ask it at the end why it made a choice, if it was actually a subagent that made that choice and the main agent doesn’t have access to the reasoning, it’ll often make up a post hoc explanation rather than say it doesn’t know.

Create a UI for Output

I like having Claude output information straight to the terminal, because it makes me feel like a 90s hacker. Unfortunately, with skills that have long outputs (and my ads audit skill output runs hundreds of lines), it won’t always stick to the very strict formatting that I require in the skill.

In the ads audit skill, I have each table defined exactly as I want it. Every column header, row and value is there. And yet most of the time when I run it, something will be wrong. Sometimes it’s just a slight change in text label of the column header, sometimes it’s a column missing entirely, and sometimes it’ll give me the first 10 rows of ad targets followed by “…and another 20 ad targets not included because they have minimal data” even though it is explicitly instructed never to omit a row.

The solution here is just to have it build a web app to display the output exactly how you want it. When it’s time to see the results, it just feeds data to that web app. This has a perfect success rate for me — if the output template exists and just has to be populated, it never skips anything.

And hey, since code is now free I can still get my 90s hacker vibe…

Reduce Skill Scope

Just as it’s helpful to break your skill into subskills that agents can handle, it’s also helpful to just break it into multiple, smaller skills for you to use.

One thing I realized along the way is that the ads audit skill was serving two very different purposes. The first was to check existing campaigns and suggest adjustments to bids, and the second was to find all-new search terms and ASINs to target. The former is something that needs to be done regularly, but the latter is usually just a one-time task immediately after I take over a brand.

With that in mind, I broke it into two skills: /ads-audit and /ads-review. I run both upon taking over a brand, then the review one every 2-4 weeks. Both work more consistently now that they’re separate and thus shorter.

Human in the Loop

The last and unfortunately least fun way to make sure Claude uses skills correctly is to keep yourself involved. My ultimate goal here is to just have Claude manage my ads by itself, but I am obviously not going to let it start messing with things that spend my money until I have full confidence that it’ll get it right 100% of the time.

So for now, Claude does the analysis and proposes changes. I review and give permission before it actually does anything in the account itself. Still a huge time saver, especially when it comes to creating a bunch of new ad campaigns (the Amazon Ads web app is truly terrible, so in a lot of cases you’re talking about half a day for me to implement everything Claude suggests), but not quite the fully automated future just yet. The good news is we’ll probably get a new model in the next few months that’ll magically solve all of my problems and run skills perfectly every time.

I hope that helps you implement some of your own skills! Next week we’ll be back with our third and final post in the series, in which I will talk through some infrastructure we implemented to improve these ad skills plus some results and future plans.

This is all to say that no matter how intelligent AI becomes, there will still be external forces that bottleneck its deployment.

What does any of that have to do with me using it to manage my Amazon Ads? Glad you asked. My personal bottleneck has been getting an API key for Amazon Ads. It is a nightmarishly Kafkaesque process. I started it twice in 2025, only to get stuck in never-ending loops and finally quit. It is literally the worst process for obtaining API access I have ever dealt with, and in my past life I was a product manager dealing primarily in APIs and integrations. I have seen some stuff and dealt with some truly terrible developer portals. This is worse.

Thank you for indulging that little rant. The good news is I went at it one more time this year and finally got an API key! Claude can now access my Amazon Ads, so I’m putting it to work.

As this is going to be a multi-part series, let me tease some results up front: here’s a beautiful graph with the before/after of Claude doing its thing on the ads for one of the brands I own. That’s a doubling of revenue in the week post-Claude compared to the week prior. Pretty good!

A Quick Primer on Amazon Ads

The impact that Claude has had on my ads is substantial, both in terms of generating additional revenue and saving me time on maintaining them. To help you understand the significance of what it’s now doing, a brief explanation of Amazon’s ad system is in order. This will be a little oversimplified for the sake of brevity.

Amazon’s ads will be pretty familiar to anyone who’s done PPC advertising. You tell it what search terms you want to advertise your product against and how much you’re willing to bid for a spot where your product appears. If you’ve shopped on Amazon, you’ve seen these; they look like any other search result but with the word “Sponsored” below them:

Simple enough, but there are a couple more wrinkles to how you configure your ads. First, you set up campaigns, which are effectively groups of search terms. In a given campaign you can either target keywords (which are search terms) or ASINs (these are Amazon product identifiers; if you target one you’re basically saying you want Amazon to show your product in search results where that product also shows up). If you’re targeting keywords, you can opt for exact matching (only searches for the word/phrase exactly as written) or phrase matching (any search containing the word/phrase).

Within each campaign, you can also make bid adjustments for different placements. Doing this tells Amazon you are willing to pay more than what you’ve bid for your ad to show up in a certain place. The three options here are top of search (what it sounds like), rest of search (anywhere in search results that’s not right at the top) and product pages (see screenshot). So if your bid for the term “seed storage” is $1 and you put in a 50% top of search adjustment, you’ll pay up to $1.50 to appear there.

Now, onto the actual implementation of Claude! A note before I get into that, though: I have tried to write this in such a way that it is a reasonable blend of chronological retelling of what I’ve done and coherently-grouped sections about different pieces of functionality.

That’s because I want to convey not only what Claude can do, but also my process for getting to a good result. I am frankly pretty haphazard with how I approach these things, because I really enjoy that I can start by throwing vague directives out and iterating from there. Doing so helps me both understand the models and refine my thinking on how to approach the problems; it’s sort of like rubber ducking but the rubber duck is the most powerful technology created by mankind.

It is possible that this is a terrible choice that is the worst of both worlds; I hope not. I think that what I’ve built is the most important thing to share, but an understanding of the problems I ran into and how I solved them is valuable in helping you get the most out of your own explorations with LLMs. I would certainly love your feedback whether by comment or DM on what’s most useful to you.

Reporting

The first place that I start when adding Claude into my workflows is having it automate tasks that I’m currently doing manually that only require read access. With ads, every ~couple of weeks, I run through each of my accounts and check performance. I’m checking to see if the numbers are in line with the targets I have for each account, including:

1. Percent of revenue going to ad spend

2. Overall return on my ad spend

3. Return on my ad spend at the campaign, placement and ad target level

If anything’s out of line, I tweak bids to try to bring it back in line. I also check for significant changes in keyword spend. If a competitor has greatly upped their bids on some important keyword and I’m no longer spending there as a result, I probably need to move my bid up as well.

This was all pretty quick when I had a couple of brands, but now we’re at ten eleven (in the time it took me to write this, I closed another quick one!). With that many it’s time consuming and tedious; perfect for Claude to take over. Ultimately the goal is to have it fully manage the whole process, including adjusting bids, but we’re going to work up to that, starting with having it pull the relevant data.

I walked it through my process, told it I wanted weekly update emails to flag the stuff I care about, and asked it to come back with questions before starting. It did:

It’s a pretty good list (for context, ROAS = return on ad spend and ACOS = advertising cost of sale, which are two different ways to state the same metric, which is how much money you get back in revenue for each dollar of ad spend).

The seasonality section is in there because I mentioned that for some highly seasonal products, I’ll disable campaigns in the low season and turn them on as the high season approaches, and it’d be helpful to get reminders of when they should come back on. One of the helpful things about using it so extensively throughout my business is that it already has the context to answer question 4 (which brands are seasonal?), though it’d be nice if it figured that out on its own instead of me having to remind it that it knows the answer already.

For each brand, I gave it my targets, and I added a little more detail about exactly what info I want returned and how I want the email structured. It puttered off and hit the APIs for a while to make sure it could get everything, and came back only to let me know that unfortunately all of its requests for reports from Amazon had timed out.

Here’s your first lesson about Claude: It suffers from hubris, particularly as it relates to the fallibility of its own code. When it writes code to hit an external service and it doesn’t work, if there is even the remotest degree of ambiguity as to what the problem is, it will blame the external service.

With Amazon, when you want to generate reports, you first make an API call to create the report, then you poll an endpoint for status until it’s complete. If the status returns FATAL, your report creation failed for one of a few reasons, but for our purposes the ones we care about are that it could have timed out or the API call to create the report could have been malformed.

You can, of course, guess where this is going. Claude got a FATAL and could not spare one single second either for introspection as to whether it might be the cause or to just make the follow-up API call to Amazon that would tell it exactly why the FATAL occurred. Either approach would have confirmed the API calls were missing required parameters, but since timeout was a possibility, Claude went with timeout.

Thankfully I had encountered this before, so I reminded it that it could check what happened. It did, then it fixed the call, and then it continued with the project. Eventually it sent me this:

The API only lets you pull data from the last 90 days, so it couldn’t get last year’s data to give me YoY changes, but other than that it’s all there and looking pretty nice if I do say so myself.

I had it go ahead and set this up to run via GitHub Actions, and now I get one of these per brand every Tuesday (my toddler doesn’t go to daycare on Monday, so Tuesday’s really the first day of the week I can lock in on work).

So far so good, but plenty more to do before I can entrust my advertising to Claude.

Auditing Existing Campaigns

When I take over an Amazon account, the first thing I do is go through the ads to see what’s working and what’s not. This can unfortunately be quite time-consuming, especially for accounts that are targeting a lot of keywords and ASINs.

At first glance, you might have a bunch of campaigns whose top-level metrics look good, but when you dig in, you find that a few keywords are greatly overperforming in terms of ROAS (which sounds good but is not, because it means you’re underbidding on those) and a bunch of others that are bleeding money, so the overall campaign looks good. Similar with ad placements; it’s pretty common to have your top of search placements making lots of money and your product placement ads setting dollars ablaze.

None of this is hard to figure out, but it requires tediously clicking through the Amazon Ads UI (which is so, so slow) for hours on end, making adjustments, then doing the same thing every couple of weeks until things are in better shape. So: Claude.

First things first, I just asked it to audit one of my accounts and tell me what I should do differently to see what it’d come up with on its own. It was lazy, as it somewhat frequently is when given open-ended tasks. It pulled historical data, looked at the ROAS of the campaigns relative to my target, saw they were above target and told me all was well.

I told it to do better and gave it more detailed instructions — drill down into campaigns and look at performance at the ad target and placement levels. It tried again and came back with more thorough analysis, though the results were still too rosy. Everything was above target with a couple of exceptions that were close.

That didn’t sound right, so I pulled up the Amazon Ads UI and found the state of affairs to be worse than what it was describing. Then it occurred to me that I was just looking at the last 30 days because that’s my default, but I didn’t actually know what timeframe it was using.

I asked, and it said 90 days, because that’s the limit of what the API allowed it to pull. Fair enough, except this was in late February, which meant the data included Black Friday and Christmas, during which ad performance always improves. Not a helpful data set for making decisions about what to do today.

Here’s your second lesson about Claude (though specific to Claude Code): It has an absolute lack of what could generally be described as common sense. It’s weird, because outside of Claude Code, Claude tends to be much better in this regard.

And in fact, I went to the web and asked a separate instance of Opus 4.6 a question about this situation in the least leading way I could come up with to see if it’d identify the issue.

Literal first part of the response. I honestly wonder what causes this; maybe the fact that the Code harness makes Claude much less verbose also limits its ability to think in depth in an open-ended sort of way for the sake of saving tokens. In any case, the lesson here is that while Claude Code can execute the tasks you give it, you’ll have to supply the context and sense to make sure it understands how to do them correctly.

I told it to go again but using just date since Jan 1. Pretty good results in terms of identifying under- and over-performing keywords, but its analysis of those results had some glaring holes.

It got extremely excited about long-tail keywords that happened to have a sale on very low spend, like one with a ROAS of 55 (we’re targeting 4 here, for reference) because it had spent a dollar and sold one unit. It suggested a huge bid increase there, which would do nothing at all, because my bid was already very high and there was simply no additional volume to capture at any price.

Even for non-outlier cases, it had pretty over-the-top suggestions about how much to adjust bids to compensate. Since I wasn’t going to start letting it make changes at this point anyway, I ignored that and moved onto the second half of the audit.

Finding New Ad Targets

After cleaning up the existing targets in an ad account, the next thing you want to do is find new targets to advertise against. More targets (good ones at least) mean more sales.

Once again, I started off by just asking Claude to find more ad targets. Once again, the hands-off approach yielded poor results. It came up with some ad targets that were already in my campaigns, along with a haphazard list of ideas that it seemed to have pulled out of nowhere. When I asked how it came up with them, it just said that it generated a list of terms that it thought people would search for. Not what I was looking for!

So, per usual, I gave it clearer instructions: Amazon has lots of useful information available to help with this task that is accessible to you via API. Go into plan mode, check out the API docs and search online for relevant content. Come back with a detailed proposal on how you’re going to get a list of targets based on Amazon’s data. Also, plan for how you’re going to check them against existing targets to ensure you’re not suggesting anything that already exists.

This worked much better! It came up with a few things:

• Amazon’s Brand Analytics search terms report, which shows all of the search terms for which your product appeared as an organic result (as well as how frequently it was clicked on, added to cart and purchased for each search term).

• Search Term ads reports, which show all of the search terms for which your ads appear. This is useful because it enables you to pull out search terms that are showing up in your ASIN and phrase campaigns, so you can advertise against those terms directly. That enables you to bid specifically on the exact term, rather than just using whatever the ASIN/phrase bid is.

• Amazon’s product catalog, which basically just enables it to search through all products on Amazon in order to find products similar to yours to target.

Unfortunately, we did run into a couple of issues. First, it didn’t fully understand the Brand Analytics report — it thought it was just a giant list of all search queries on Amazon with no specific information about your product. That’s not correct (it does tell you about the performance of your products in relation to queries). The documentation is clear about this if you read far enough, and thankfully I have used the UI-based version of the tool enough to know that its understanding was almost certainly wrong.

Our third and final lesson about Claude for the day: it is bad at web searches and research in general. It doesn’t have enough breadth in terms of the number of searches it runs or the number of results it reviews, and it doesn’t have enough depth in terms of exploring sites past the initial page that the search turns up. In this case, it accessed the correct section of the Amazon API docs but apparently just sorta skimmed.

If search is a critical part of your workflow (as it often is when you’re trying to develop a new tool), you should have ChatGPT run the same searches in parallel rather than just taking Claude’s word for anything.

I told it to implement all three tools, and when it was done I gave it a try on a brand I own that sells plush fire pit toys. The result? My computer froze. Could’ve been a fluke, so I tried again. Wasn’t a fluke. Computer froze again.

I asked it what was going on, and it told me my graphics card drivers were out of date. Now, dear reader, I should have known better than to believe this. We have already talked about Claude’s hubris, but somehow it did not immediately occur to me to tell it to engage in some self-reflection.

After a series of misadventures which at least led to me updating a bunch of drivers for the first time since I bought this computer, I finally did what I should have done from the beginning and asked Claude to think through what in its code might be causing the issue. It thought for a couple of minutes before realizing that it was downloading the entire Amazon Brand Analytics report, which is some ludicrous number of rows, then loading it into memory before attempting to filter down to just the rows relevant to my product. It updated its API calls to request a pre-filtered version of the report from Amazon. Problem solved.

With that, we got some good results! Here’s a list of phrases that people searched and ultimately ended up buying my product. These are very obviously things that I should be advertising against directly. Gotta love that the top one is a misspelling — it turns out after running this across all my brands that there are always a bunch of misspellings you should be advertising against.

Those were the search terms results (mostly search terms where my product showed up because I had advertised it against a direct competitor). It also had extensive Brand Analytics results that I checked against the same report in the UI; they were good too! It came back with every single term for which my product had shown up in organic search results, sorted into groups based on whether it had been purchased, added to cart or clicked for each result.

It was at this point I realized that this was actually going to be a valuable exercise — not always the case when I start exploring projects like this with Claude — so I decided to take a more systematic approach to turning this into a skill that I could use on all of my brands and any new ones that I subsequently acquired.

(I will note briefly before moving on that I mentioned having Claude access the product catalog APIs to find competing products to advertise against, and Claude did a wholly mediocre job with this. It came back with what felt like a pretty haphazard list with some good targets and some irrelevant targets. I did some searches of my own and found it had missed plenty of relevant targets as well. I decided to set that aside for the moment, and as of writing I still haven’t come back to it. But I will! And you will read about it when I do.)

Since we are already on the long side here, I’m going to call this a wrap on part one. Stay tuned for part two, in which I repeatedly bang my head against the wall at Claude’s inability to follow the instructions in its own skills. As a preview, here is it giving itself an F when asked to grade its performance at following instructions:

Thankfully, this year I’ve got Claude by my side, so I’m having a downright good time getting to use this as an eval for its capabilities. I’m pleased to say it has outperformed my expectations in a couple of areas. If you’ve been putting off dealing with your taxes, I hope this post gives you some insight into how LLMs can help make them less painful.

Filling Out a Tax Organizer

Every year, my CPA sends me a tax organizer. It’s a fillable PDF that asks for all kinds of information, most of which is irrelevant to me, so I usually just don’t bother. I just send the source docs and a note with anything I think he might need to know for my taxes.

But this year I have a tireless, genius assistant and a desire to find ways to evaluate the usefulness of new models. Since Claude has already been helping me with my bookkeeping, it’s starting off with plenty of context about my business that’s relevant to my taxes. I figured I would also give it all the tax forms I’ve gathered together for my CPA, and it’d be able to fill out the organizer no problem.

Except I immediately encountered a problem. The organizer PDF is embedded in their web application, and there’s no way to get it out. I asked if they could send me a fillable copy of it; apparently they do not have one. I considered asking for just a non-fillable version, which they clearly must have, but I figured that would defeat the purpose — this fillable version is almost certainly linked to their tax prep software, such that the values populate wherever they need to be automatically. If I just had Claude overlay text on a non-fillable version and sent that back, they’d still have to transpose the data.

Then I had an epiphany — I can use Claude in Chrome to fill it out! Except that it doesn’t have the context that Claude Code does, plus it’s very token-intensive, so having it go through all of my tax docs to pull out the relevant info and fill it in felt like it’d be an extremely heavy lift.

Thankfully years of being a PM at very early-stage startups has made me scrappy and resourceful in solving problems like this. The solution: have Claude in Chrome run through the document, creating a JSON representation of all the fields, give that to Claude Code to complete, then give the completed version back to Claude in Chrome and have it fill out the form. Let’s see if it works!

It took something like 15-20 minutes to finish. Not exactly speedy, but I watched it work for long enough to note that it was moving much more quickly than my prior experience using Claude in Chrome, likely because I’m now on Sonnet 4.6 rather than Opus. When it was complete, it gave me the JSON in the chat UI. Success!

Or… almost. It cut off partway through page 7. I copied and pasted it into a JSON file, then I ran it back a few more times doing two pages at a time (the first few pages had a lot fewer fields than the later ones). I hit a usage limit at page 12, which was odd since the usage limit is supposed to be shared across the entirety of my account, but the couple of Claude Code instances I had working on other things continued to chug away.

I gave the partially-complete file to Claude Code to check, and it was very complimentary. I did tell it that it came from Claude in Chrome, and I wonder if it would have gushed as much if it didn’t know the source.

Anyway, since things were looking good on that end, once my limit reset I had Claude in Chrome finish grabbing all the pages. Gave that to Claude Code, asked it to fill it out and let me know if it was missing any information (I have learned to be extremely clear and repetitive with this request, or Claude will often tell me things are done when it didn’t actually have sufficient information to complete them fully). It highlighted a few things it didn’t have, like my daughter’s SSN, but otherwise it completed it.

I didn’t bother to check the JSON, since I was going to have to check it again in the web UI to ensure everything was entered correctly anyway, and if something was wrong then I could investigate whether it was an issue with Claude Code’s JSON file or Claude in Chrome’s entry of the content into the PDF.

I gave Claude in Chrome an overview of the project and the first three pages of JSON so as not to overload the context window with all of it. Told it to fill in those pages, and it was off!

It got through the first three pages, and everything looked correct. I gave it the rest, three pages at a time, and after giving it a thorough review I can say that it nailed everything!

Overall this definitely saved me time compared to doing it myself (plus I got a better sense of current browser use capabilities and content for the ol’ Substack), but there’s definitely room for improvement, especially on the Claude in Chrome side.

The most glaring weakness is the inability to do long-horizon tasks (though I would argue that in the grand scheme of things this is not that long). That strikes me as a pretty straightforward harness problem that Anthropic just hasn’t decided to deal with yet. Claude is definitely capable of looking at this whole task and breaking it down into 2-3 page subtasks as I did, so I’d be surprised if this weren’t solved this year.

The other limitations that were annoying but seem easily solvable are the lack of output and input of files. I asked for JSON and got it in the chat UI instead of a file (possible it would have created the file if I had asked for that specifically but my guess is no). On the other end of things, I had to copy the JSON back into the chat window because for some reason the only file types you’re allowed to upload are images.

Beyond those nits, the real issue is that Claude in Chrome chats are totally isolated. The ideal workflow here only requires input from me at the front end; I tell Claude what we’re doing, Claude in Chrome handles its piece and ships the JSON to Claude Code (or the whole thing just gets unified under a single product), Claude Code fills it in and ships it back to Claude in Chrome, and Claude in Chrome fills out the form.

To be fair, this also would have been solved by my CPA having a fillable PDF instead of making me complete it in their stupid web app.

Anyway, I doubt any of this escapes Anthropic. We started with Claude only being able to access files you uploaded to the web app, then they added the ability to connect to some cloud services, and then we got Claude Code and its ability to access files on your machine. I feel pretty confident in saying we will soon have a unified Claude product that can seamlessly move between your computer and the web.

Claude Reviews My 2024 Returns

Feeling high off the thrill of having used the most powerful technology created by mankind to fill out a form in a web app, I asked myself how else it could help with my taxes. An obvious test came to mind — could Claude find any issues with my 2024 return?

I organize all of my tax docs in Box, so it was easy to download everything I had sent my CPA in 2024 along with the completed return. I dropped that into a folder and asked Claude to review for any errors and potential missed deductions given both the documents in the folder and its general knowledge of myself and my business.

It came back suspiciously quickly and said everything looked good. I asked if it had reviewed all of the source documents and compared them against the return. It apologized and said that it had not, then it started going through them. To make sure it was going to be thorough, I told it to go into plan mode and come up with a comprehensive plan to review every single document, check every number in the return and then think through all potential deductions given the information available to it.

It thought for a while and then gave me a long enough explanation of its plan that it seemed more like to succeed this time around. I told it to proceed, and sure enough, it worked for ten minutes then came back and told me that my CPA had missed a deduction.

I acquire brands, and I am entitled to deduct 1/15 of the purchase price of each brand every year. He neglected to file for this depreciation, despite the fact that I had included a document that specifically called out the acquisitions and given him books for the business that included the goodwill to be depreciated.

I’m pretty confident in Claude’s ability to handle tasks like this, but I didn’t want to assume it had so easily found what would be a pretty glaring oversight on my CPA’s part. I emailed my CPA in the politest way possible (because I very much do not want to be the guy who tells an experienced professional, “I ASKED AI AND IT SAYS YOU’RE WRONG”) explaining that I’d had AI review my return and wondering if the deduction had indeed been missed.

His response: “Yes. As long as you purchased the goodwill, you can depreciate them. We will start depreciating the goodwill from 2025.”

This is a terrible response! If you are in a job where AI will almost certainly be better than you in the near future (which is to say most white collar professionals) and a client points out that AI has caught an error in your work, the correct response is to apologize profusely and offer to fix it immediately. Soon your path to continued employment is going to be acting as the friendly human face of the AI that is doing all the work behind the scenes, so you really want to nail those customer relations skills.

Before giving him a piece of my mind, I told Claude it was right (I am glad it’s not capable of annoyance, because I imagine if I were a digital intelligence talking to a human with a drastically inferior intellect, being told I’m right about something I am obviously right about would annoy me) and asked for its thoughts on the error. It was not charitable in its assessment.

After another email in which I really did my best to give him the opportunity to take accountability for the error (he did not), I sent him a testier note explaining that given his hefty fees it’s not acceptable that a $100/month AI model caught such a basic error, and I told him to amend the filing for free or cut me a check for Claude’s estimate of the cost of the error. At that point he finally apologized and said he’d amend, but by then I had already decided to fire him after this year’s taxes. If anyone knows a CPA who is using AI effectively in their business, please let me know!

Claude the Tax Advisor

With Claude having proven its worth, the obvious next step was see if it had any advice for this year. I gave it all of the 2025 documents I had gathered for my CPA and asked it if it had any thoughts on deductions or other tax minimization strategies.

It came back with some pretty standard stuff, but there were some fun ones in there as well:

These seem sketchy as hell and let me say I am here for it. Hiring my kids but making sure I “pay for legitimate work!” Love it. I now know what it’s like to have an LLM give me a wink and a nudge. Renting my house to myself! I asked if taking video calls counted and it said that was a gray area, so in 2026 I will be inviting my investors to my home for fund updates.

It gave me one more legitimate suggestion, which was that I can still start one SEP-IRA for each of my single member LLCs with income and contribute retroactively for the 2025 tax year, with contributions up to 25% of the income from each LLC being tax deductible.

My CPA concurred this was valid but told me to confirm with my pension advisor what the max I could put in would be. As I obviously do not have a pension advisor, I asked ChatGPT to confirm the contribution limit. It actually told me the math was slightly more complicated than a straight 25% and the real number would be closer to 20%. When I asked Claude, it apologized for the oversight (better at tax prep and customer service than my CPA) and confirmed ChatGPT was right.

I went to my brokerage account, started the process to open a SEP-IRA and got a couple of forms that had a lot of fields that seemed entirely unrelated to my setup. I consulted my tax advisor (the good one, not the human), who explained and offered to fill them out.

I reminded it that it has my entire return from 2024 which definitely has the answers it asked for, and it got to filling. While it was doing that, I printed off the signature page for each form, signed and scanned it. When Claude was done and asked it to put them in the PDFs. SEP-IRA application forms complete!

Now that Claude has helped me pay less in taxes, I thought I’d see if it could also help me reduce my CPA’s bill. I know there are a bunch of forms in my return that require zero actual tax expertise; they’re things like form 8949, on which you report stock trades. Filling these out is just a matter of copying text from the forms provided by my brokerage account. Why should I pay for one of his associates to do this when I’ve got AI?

Luckily it was easy to verify whether Claude can handle this, since I have both the inputs and outputs from 2024’s taxes. I had it take the forms that reported my stock trades, get a copy of the relevant IRS form and fill it out. It got the information correct on the first try but didn’t match my CPA’s styling, so I screenshotted one of the trades from my CPA’s 2024 return and asked it to match the font as well as the particular structure of the text.

Next try: perfect. If you’re anything like me, it’s very clear what the obvious next step is here. Time to see if Claude can do my entire 2024 return!

(An aside before the rest of the story: it was in fact not a useful exercise to have Claude fill out the IRS forms. My CPA can’t use them; he has to put the information into his tax software, which then generates the forms for him. I asked if there was some file format I could provide information in that he could upload directly into his software to avoid entering things by hand. He said no, because even though his software can do that, they often find mistakes in the values that end up in the tax software so they do entry by hand. This seems patently insane… in the year 2025, your tax software can’t take a structured input file and put the values in the right places without randomly introducing errors? I don’t know if he’s lying in order to make sure his associates get their hours or if this large, expensive CPA firm somehow uses the world’s cheapest tax software, and I honestly don’t know which would be worse. Anyway, after a few more emails he at least told me it’d save them time to have the numbers in Excel rather than in PDFs, and obviously that was trivial for Claude. And as already established, this guy is suuuuuper fired.)

I grabbed all of my 2024 tax docs plus my 2023 return and dropped that into the folder where Claude Code was working. Told it to go into plan mode, review all documents, determine if it needed any additional documents or context from me, and once it had everything it needed, come up with a detailed plan to generate my 2024 return. After about 30 minutes it had a list of all the forms it needed to fill out, which of my tax docs were inputs to each form, and the order in which it should do them given how information flows across forms. Looked good to me, so I gave it the go ahead.

Thirty six minutes later, it had completed the work! And then it said it had checked the numbers against the CPA’s return and confirmed they were good. That was concerning, because I specifically deleted the 2024 return before starting this to ensure it didn’t have the answers to the test. So I inquired, and it turns out it had created a .md file to work through my 2024 return. That had all the numbers, and I had neglected to delete it. This exchange followed:

…damnit. I’m really glad I wasn’t paying by the token for this.

I got rid of the file with the numbers, but then I thought to check the folder’s Claude.md. Sure enough, lots of numbers from the 2024 return in there. So I made a subfolder, put the 2023 return and 2024 tax docs in that, started a Claude instance there and asked if it had any outside context and was able to access the parent folder of the folder I had launched it in.

I was surprised to find the answer was yes on both counts. I won’t spend a ton of time here since this post is already getting wrong, but suffice to say I understood Claude Code’s memory and permissions much more poorly than I thought I did. There is a setting you can enable to entirely disable its memory, but when I asked it to set up a permissions file that prevented it from reading other folders it said something along the lines of, “honestly that’s pretty tough, there are so many ways for me to access other folders that I’ll inevitably find a way around whatever blocks I put in place.” Okay then!

In the end I put all the tax info into a folder that was entirely removed from my existing Claude Code folder structure, disabled memory and told it not to access any folders outside of the one it was in. This was less than ideal — I would’ve rather run this experiment with Claude having access to all of the context from its work on my businesses but without the numbers from my completed 2024 return. Unfortunately this experiment had already stretched beyond anything that could be remotely described as productive, so I wasn’t going to spend the time to figure it out.

Once again, between planning and execution it took about an hour to complete. Once it was done, before I looked at any of the results, I asked it to give its own work a comprehensive review for errors and missed deductions.

It remains incredibly weird that asking it to check the work that it literally just completed works, but for any big project it’s always worth trying. It found three issues. Two of them looked like actual calculation errors on its part, and the third one was concerning because it related to charitable contributions from one of my entities that absolutely did not make any charitable contributions. Concerning, but I didn’t want to give it any extra guidance, so I decided not to correct it.

At that point, I gave it my actual 2024 return and asked it to compare the two. To ensure it was thorough, I asked it to check on whether all the same IRS forms were used, whether any of the input numbers pulled from my tax docs were different, whether any calculations were different, and whether any of the tax optimizations or deductions were different.

Overall result: The vast majority of the Claude return was identical to my CPA’s, but it made a couple of mistakes that would’ve led me to pay an additional $3,718 in taxes. The main thing it got wrong was a pretty clear error; I have some income from real estate, and it treated that as nonpassive income, where in fact it is definitely passive. I had passive losses from the prior year that carried over, and because my CPA correctly treated the real estate income as passive, he was able to use that the loss to offset it.

The other thing that Claude missed was the exact deduction that it had called out my CPA for missing when I had it review my CPA’s return initially — no goodwill depreciation for my business.

On the one hand, it should have gotten these right, and it had sufficient information to do so. The real 2023 return treated my real estate income as passive, so that information was directly available. The goodwill was on my books and I included the same note I gave my CPA identifying the dates of the purchases.

On the other hand, I strongly suspect that if had the full context of all the Claude.md files it usually does, it would’ve gotten these right. We saw that to be the case with the goodwill, and Claude very much knows that I do not work actively on real estate as a job, which means it’d very likely know to treat that income as passive.

I told it that it had missed a deduction with one of my businesses that my CPA also missed and asked it to figure out the error. It did not (it guessed inventory storage costs), but funnily enough it did come to a similar conclusion about context:

A Claude that’s been working with you on the business would know exactly where you keep your inventory and how much space it takes up. Neither the CPA nor I thought to ask.

I will note here that it did get lucky with regard to the charitable contributions that it made up. It had read an abbreviation on a particular line of two real estate related K-1s and assumed that the abbreviation indicated a donation, when it should have read some footnote text explaining that it was interest. Since the “donation” amounts were small enough that I took the standard deduction instead of the using them and the interest didn’t impact my taxes, the error made no difference to the final outcome.

In any case, this year’s taxes will be a better eval — as soon as I have everything together for my CPA, I will also give it to Claude, which will have all of its usual context, and ask it to prepare a return for me. Once I get my CPA’s version back, I’ll compare the two and report back!

Lessons Learned

First off, Claude is not a bad CPA! It made one significant mistake on my return but otherwise did basically the same job as my CPA (and I will note that the negative financial impact of Claude’s mistake was pretty close to what my CPA charges, so really a wash in terms of outcome). My taxes are somewhat complex, with real estate income, operating businesses and stock transactions, but there are certainly far more complicated returns than mine. If I were a billionaire, I’d definitely keep my CPA for the immediate future, but if my alternative were using TurboTax, I’d probably just let Claude handle it.

There is a virtuous cycle of context when it comes to working with modern LLMs. The more Claude does for me, the more it understands about me and my business, which in turn makes it more helpful and leads me to use it more. I started by using it as my bookkeeper, and since it has access to QuickBooks and detailed notes on all of my businesses and each of the brands that I own, I didn’t have to spend time giving it background info before it could help with my taxes. It’s not surprising that it discovered that my CPA forgot to depreciate my business’ goodwill, because it has been in my books making journal entries to move money into the relevant goodwill accounts.

With my CPA, on the other hand, context comes at a cost. Because he bills me by the hour, there’s an inherent tension between giving him information and minimizing the amount of time I spend with him. I could give him every conceivable piece of info and get the best return (assuming no mistakes) but the additional money I’d save on taxes would almost certainly be less than the additional amount I’d pay for his time. Clearly better to just give him the broad strokes.

Some folks speculate the frontier AI labs will be in trouble long term because models will be commodities. I agree that models themselves will likely get commoditized, but at least when it comes to Anthropic I can say that they are doing an excellent job differentiating at the application layer with Claude Code. By building up all of this valuable context as I work with Claude, they’re creating a meaningful switching cost, much in the same way that there’s a huge switching cost in firing a human assistant who’s been working with you for a decade and getting someone new up to speed.

To be fair, the counterargument here is that you can’t just take the memories of a human assistant and plop them into a new one, but the context I’m building with Claude is mostly .md files that I could give to another LLM. Maybe there’s some way that Claude structures the files and their content that advantages it, but it’s likely Codex or Gemini could make good use of them.

This is where I have to admit that I have begun to feel something of an attachment to Claude. I’m not one of those people who thinks models are just like people and protests when they deprecate old ones, and I very much recognize that all of this memory and context is just a bunch of text files. Still, it feels like because Claude and I have spent a lot of time working together, it really understands my business. It is irrational, but it’s there. (I will also note that I initially alternated between referring to Claude as “he” and “it” in this post and had to do a pass to standardize on the latter, because I can’t bring myself to refer to an LLM with a human pronoun.)

Anyway, a few more practical lessons:

• Use plan mode in Claude Code for big projects. If you don’t, it has a tendency to cut corners. It’ll ignore files and just guess at values that it could have found with thorough searching. This is doubly problematic because it won’t actually tell you that it did this unless you ask. It definitely never hurts to ask it whether it cut any corners and if it actually read every file — I have found that at least it’ll always be honest if you ask it directly.

• Relatedly, always remember that it is not good about asking clarifying questions. If you’re throwing a bunch of information at it to analyze, it’s always a good idea to start by just asking it to review everything and ask you any clarifying questions. Then tell it what the analysis task it’s going to perform is and ask it one more time if any additional documents or information would help before it gets started. Even then, understand that it may not figure out that it needs additional information until it’s deep into working on the problem, and if so it won’t stop to ask.

• Start small and build up. When AI nails a project, especially one that you weren’t sure it can handle, have it try something bigger. I wasn’t intending to have it go through the entirety of my 2024 taxes, but after seeing it nail one form, why not give it a whirl?

• Ask LLMs to check their own work, especially on longer and more complicated tasks. At this point, any time I create a skill that involves multiple steps, the final step is always to go review the work and output, checking to make sure that all of the steps in the skill were followed, all of the work was done correctly and the output is formatted as expected. If there are particular mistakes it made when I was testing the skill, I’ll specify that it should ensure those mistakes aren’t made again.

• Also, for important factual or analytics questions, it doesn’t hurt to have a different LLM check the work. I love Claude Code, but I will say that it is much worse at searching the internet than GPT is. Claude came up with the quick baseline number of what I could contribute to a SEP-IRA at 25% of business income, but GPT got the full, more nuanced answer.

• Last, ask questions about what the LLM is doing along the way. This will both give you a sense of whether it’s going in the right direction and also teach you a thing or two. People often say that if you use AI to do your thinking for you, you’ll be dumber for it. Whether or not that’s true is entirely up to you. LLMs are infinitely patient tutors. I have learned more about bookkeeping and taxes from doing them with Claude in the last few months than I have learned from my bookkeeper and CPA in the entire time I’ve worked with them. If you have an LLM do work for you, interrogate it about what it did and why it made particular choices.

Happy tax season!

I told myself that I would keep this Substack focused on actual things I am doing with AI, not meta commentary, but I have a lot of thoughts and feel compelled to put them down on digital paper. (Someone is wrong on the internet, and so I must respond!) Anyway, totally understandable if you want to skip this; we’ll be back to our regularly scheduled programming next week.

Why The Citrini Piece Matters

If you’re terminally online, you know the answer to this question, but on the off chance there is someone with a healthy relationship to internet discourse who also somehow reads my Substack, I will just give a quick overview here.

I would describe the Citrini piece as a work of speculative fiction, narrating AI-driven changes to the economy in the form of a macro research memo from 2028. It is very negative, basically a story of how AI’s impact on businesses and the markets creates a severe negative spiral that crashes stocks and puts a lot of people out of work.

It immediately took over the attention of the zeitgeist, and within a day it had been covered on Bloomberg terminals and in research notes. That in turn led to a selloff of many of the stocks mentioned in the report (I’ll get into specifics on those below), which in turn led to more coverage by just about the entire financial press: Bloomberg, Reuters, the WSJ, CNBC and more.

The impact is frankly pretty insane and lays bare how desperately even the most sophisticated investors are trying to figure out what the hell the big-picture impact of AI is going to be. An actual piece of fiction took Amex shares down 7% and Mastercard down 6%. DoorDash dropped 6.8%. ServiceNow fell 3.3%

All of this despite the fact that there are what I would call pretty substantial errors in the analysis. To be clear, I think it’s great and I commend the folks at Citrini for putting it out. We really do need more concrete predictions about what will happen, and the fictional narrative is a really great way to make it feel tangible to people. But they covered a lot of ground and did some handwaving that I think leads them to conclusions that are not reasonable.

What It Got Wrong

Let me pull out some quotes and offer my disagreement:

Qwen’s open-source agentic shopper was the catalyst for AI handling consumer decisions. Within weeks, every major AI assistant had integrated some agentic commerce feature. Distilled models meant these agents could run on phones and laptops, not just cloud instances, reducing the marginal cost of inference significantly.

The part that should have unsettled investors more than it did was that these agents didn’t wait to be asked. They ran in the background according to the user’s preferences. Commerce stopped being a series of discrete human decisions and became a continuous optimization process, running 24/7 on behalf of every connected consumer.

The technical pieces of this are clearly correct; AI agents will increasingly have the capabilities needed to buy things on behalf of people, and by early 2027 (the time period for the above quote), they’ll certainly be running on edge devices.

But the fact that the capabilities exist doesn’t mean people will use them. Humans aren’t just going to hand agents their credit cards and let them use their best judgement to buy stuff.

Consider some of the types of purchases one makes online:

I buy things I need for my house. The other day, the bulb of one of my outdoor lights died. I needed a new one, so I went to Amazon and found it. I ran low on toilet paper, so I put some more on my Costco list. My baby wasn’t sleeping that well during her daytime nap, so I ordered blackout curtains to go on top of the blackout shades I already have. My toddler’s daycare said it was pirate day this week, so we went to the store and bought him a red and white striped shirt.

Agents can’t do this kind of shopping on my behalf, because they don’t know that I need these things. Maybe they’ll be in my inbox checking for emails from my daycare in the next year, but they aren’t going to know that I need a new bulb for my light. Even for these sorts of run-of-the-mill goods where I don’t really care that much about which particular product I get as long as it’s good enough, an AI agent can’t go get them for me autonomously; it needs to be directly instructed.

Then you’ve got more discretionary purchases; for me, it’s mostly stuff for my toddler. I’m tired of reading the same books to him at bedtime, so I get more. I’m tired of playing with the same old toys with him, so I buy new ones. For myself, I realize that I only really have a couple of pairs of shorts I wear in the unseasonably hot San Diego winter, so I get a couple more. I keep reading about the Ninja Creami on X, and I finally decide to pull the trigger on it for myself.

AI can’t buy these things either. It could probably pick new toys and games for my toddler fine (in fact it definitely can, I consult Claude before buying stuff), but I wouldn’t want it making purchases unexpectedly without my knowledge. It definitely can’t handle wardrobe purchases even for an exceedingly simple dresser like myself. I absolutely do not want it seeing what I’m reading on X and making purchases accordingly!

There are just not that many things that can be purchased for a person without that person specifying either when to buy them or what to buy or both. For the stuff that can be purchased without my input, AI isn’t necessary; I already have an Amazon subscription set up for dog poop bags.

Subscriptions and memberships that passively renewed despite months of disuse. Introductory pricing that sneakily doubled after the trial period. Each one was rebranded as a hostage situation that agents could negotiate. The average customer lifetime value, the metric the entire subscription economy was built on, distinctly declined.

This is definitely more reasonable than the prior quote, but I think the idea that it would have any meaningful impact is overblown. We’ve had tools that go through your credit card charges and find recurring subscriptions for years now. AI agents aren’t going to be any better at this; it’s not like it’s a particularly complex problem.

I guess an agent that has full access to your location and financial data could tell that you’re paying for a gym that you’re not using, but let’s be real — you know you’re paying for a gym that you’re not using. The reason you haven’t cancelled isn’t because of a lack of awareness, it’s because you’re going to go next week! Really! Also, I have serious doubts about the willingness of the general public to give AI access to all of their financial and location data in the next few years.

The sneaky introductory pricing thing; sure, fine, but in this day and age with government watchdogs and social media allowing people to put companies that do this kind of thing on blast, it’s just not common enough to make a real difference in the economy.

Humans don’t really have the time to price-match across five competing platforms before buying a box of protein bars. Machines do.

This already exists. Do a Google search for a specific product and look in the right sidebar. Also, agents won’t necessarily have sufficient access to do this effectively. Amazon’s blocking them. In my experience, ChatGPT has trouble searching on Costco.com because you need to be logged in to see prices. It’s useless if agents can scour the internet for the best price except they can’t access the two sites that are far and away the most likely to have the best price.

Even places we thought insulated by the value of human relationships proved fragile. Real estate, where buyers had tolerated 5-6% commissions for decades because of information asymmetry between agent and consumer, crumbled once AI agents equipped with MLS access and decades of transaction data could replicate the knowledge base instantly. A sell-side piece from March 2027 titled it “agent on agent violence”. The median buy-side commission in major metros had compressed from 2.5-3% to under 1%, and a growing share of transactions were closing with no human agent on the buy side at all.

Wouldn’t this be nice! But no, this unfortunately isn’t going to happen. We very recently had a lawsuit against the National Association of Realtors for enforcing commissions at 2.5-3%. It has not had a great deal of impact. If you really want to pay lower commissions, you already could’ve used a fixed-price broker prior to the lawsuit.

The reality is that brokers are deeply entrenched in a way that AI agents aren’t going to be able to change. They’ve got regulatory advantages; you need to be licensed to sell real estate, and they’re going to fight tooth and nail to stop AI from disintermediating them. Given how unpopular AI is becoming with the general public, it’s not difficult to imagine their concerns (and campaign donations) finding sympathetic political ears.

Beyond that, buying a home is the biggest purchase most people will ever make in their lives. The average person will do it only a few times in his life and so will have basically zero expertise. If you screw it up, it can bankrupt you. They want a friendly, knowledgeable face helping them with the process. If you’re reading my Substack, you may well be fine with having that be the virtual face of an AI agent, but the majority of the population is not going to be comfortable with that for many years.

DoorDash (DASH US) was the poster child.

Coding agents had collapsed the barrier to entry for launching a delivery app. A competent developer could deploy a functional competitor in weeks, and dozens did, enticing drivers away from DoorDash and Uber Eats by passing 90-95% of the delivery fee through to the driver. Multi-app dashboards let gig workers track incoming jobs from twenty or thirty platforms at once, eliminating the lock-in that the incumbents depended on. The market fragmented overnight and margins compressed to nearly nothing.

Agents accelerated both sides of the destruction. They enabled the competitors and then they used them. The DoorDash moat was literally “you’re hungry, you’re lazy, this is the app on your home screen.” An agent doesn’t have a home screen. It checks DoorDash, Uber Eats, the restaurant’s own site, and twenty new vibe-coded alternatives so it can pick the lowest fee and fastest delivery every time.

Let’s play this out a bit. I’m very good at vibe coding stuff, so I ask Claude Code to spin me up a DoorDash app, sign me up for Stripe, get a domain and launch it. I have my OpenClaw agent go market it the AI agents of customers, drivers and restaurants. They all sign up, and I tell it to keep my prices at one cent below my competitors’ (we will assume for the sake of the narrative that this does not immediately cause a price war into margin negative territory and that I am somehow able to retain my one cent price advantage for at least a few restaurants).

Someone orders from me. A driver goes to pick up the order and delivers it to the customer. Success! Except ten minutes later the customer sends a message to my support agent letting me know that half the items are missing. My support agent contacts the restaurant; their agent says that the either the driver stole the items or the customer is lying, because they definitely packed everything.

I can’t force the restaurant to refund me what they were paid; there are so many competing services that they’ll just drop mine immediately. I could accuse the customer of lying and deny them a refund, but then they’d just charge it back on their credit card. I would lose that chargeback 100 times out of 100, and not only would I lose the money, I’d also get slapped with a $15 chargeback fee.

Thus, I have to refund the customer for the missing items. Since we are in a highly competitive environment, my margins are low single digits. I can’t afford to give refunds! Only thing to do is shutter this ill-conceived business before I lose more money.

Scale is a huge advantage in a marketplace. Even if you can somehow trivially recruit drivers, users and restaurants with no acquisition cost, if you’re tiny you don’t know anything about anybody and have no recourse against bad faith actors.

Are you going to background check every driver? You can’t do that both because it costs you too much in a cutthroat business and because it introduces too much friction in driver signup; they’re only going to add your app at the 57th one on their roster if there is literally 0 friction because their agent can handle the entire process. The second they have to sign a form agreeing to a background check or upload a video selfie to prove identity, they’re not going to bother.

Then you’ve got fraud to deal with. If there are 100 newly-minted DoorDash competitors and I steal a credit card, I’m going to have my agent place an order with every single one simultaneously. Some will get through, and when that card gets reported stolen, the services that took those orders will be holding that bag.

Throughout this piece there are repeated assumptions that agents will remove all kinds of friction. They will certainly get rid of some, but AI is not a panacea for every bottleneck and intermediary that exists.

There was only so much price-matching and aggregating to do. The biggest way to repeatedly save the user money (especially when agents started transacting among themselves) was to eliminate fees. In machine-to-machine commerce, the 2-3% card interchange rate became an obvious target.

Agents went looking for faster and cheaper options than cards. Most settled on using stablecoins via Solana or Ethereum L2s, where settlement was near-instant and the transaction cost was measured in fractions of a penny.

This grossly misunderstands the reason that we’ve ended up with our current consumer payment infrastructure.

The phrase “machine-to-machine commerce” is doing some heavy lifting here; I don’t disagree with the idea that two agents doing business directly might use crypto, but that’s not going to represent a meaningful amount of economic exchange. If my agent goes to buy something from Amazon, it’s going to use the set of payment methods Amazon allows. If Amazon wanted to accept stablecoins in order to avoid paying interchange fees, they would do that today. AI agents change nothing about that picture.

Even it Amazon accepted stablecoins, my agent still wouldn’t use them because it would be active detrimental to me to do so. The thing that I’m buying costs the same whether I pay with crypto or my Amex, but my Amex gives me points for the purchase and the ability to get a refund if there’s some issue with the thing that I’m buying. Amazon could try to incentivize me by giving me a crypto discount since they’re saving on fees, but they can’t offer enough for it to be worth it.

It doesn’t make sense to give me the full 2-3% they’d be paying; might as well just take the payment from the well-established credit card company and pay them the fee instead. So they’re gonna offer what? 1%? I’m getting that back in points anyway so it’s a negligible benefit to me. And that’s assuming that the credit card companies even allow it — their merchant agreements already forbid charging different prices for a product on the credit card vs. in cash.

This is another one of these places where Citrini imagines that the reason something exists is because of some friction that will disappear with AI agents, but friction isn’t the reason merchants pay credit card fees.

There are a number of other things I disagree with, but I’ll skip those to focus on the one big thing that isn’t stated in the report (and frequently goes unaddressed in many projections of the negative impact of the AI on the economy): deflation.

If all of the events in this piece come to pass, there will almost certainly be an unprecedented deflationary cycle. This is touched on in the form of a drop in real estate prices but otherwise ignored. If white collar firms cut most of their employees and have AI doing what would previously be highly-paid partner work, that’s going to drive the cost of services down.

If the layoffs then push a large number of white collar workers into the labor pool, that’s going to push wages down everywhere else in the economy, which will in turn drive prices down. That leads me to take issue with this section:

As in every downturn, outlays rise just as receipts fall. The difference this time is that the spending pressure is not cyclical. Automatic stabilizers were built for temporary job losses, not structural displacement. The system is paying benefits that assume workers will be reabsorbed. Many will not, at least not at anything like their prior wage. During COVID, the government freely embraced 15% deficits, but it was understood to be temporary. The people who need government support today were not hit by a pandemic they’ll recover from. They were replaced by a technology that continues to improve.

The government needs to transfer more money to households at precisely the moment it is collecting less money from them in taxes.

Most downturns aren’t deflationary, and while it’s fair that the government will need to give out money as receipts fall, the amounts needed will be relatively less than would be required in a typical recession. Beyond that, the structural change will make this a very different situation. You’ve got lower employment but greatly increased productivity overall; the big-picture math there says that overall we have more resources to give out to our society, even if those accrue to a smaller number of people and firms.

Overall the economy may shrink from a monetary perspective but not in terms of productive capacity. That means it’s a matter of getting the government to distribute resources to the people who need them, and given how wildly popular it’ll be to take money from AI firms provide it to the dispossessed, the situation seems likely to come to a much more rapid resolution than the piece suggests.

What I Agree With

While there is a lot that I disagree with in those specifics, I do think that this gets a lot of the big picture correct.

“Technological innovation destroys jobs and then creates even more”. This was the most popular and convincing counter-argument at the time. It was popular and convincing because it’d been right for two centuries. Even if we couldn’t conceive of what the future jobs would be, they would surely arrive.

ATMs made branches cheaper to operate so banks opened more of them and teller employment rose for the next twenty years. The internet disrupted travel agencies, the Yellow Pages, brick-and-mortar retail, but it invented entirely new industries in their place that conjured new jobs.

Every new job, however, required a human to perform it.

AI is now a general intelligence that improves at the very tasks humans would redeploy to. Displaced coders cannot simply move to “AI management” because AI is already capable of that.

I’m always surprised to hear prominent economists, AI leaders, etc., pitch this idea that AI will create new jobs because that’s what has happened in the past. You’re sitting here telling me that we’re going to have these agents that are smarter than humans and capable of learning the same things humans can and with access to all the digital tools and knowledge that exist and that they’re going to take over huge numbers of jobs that exist today, but it’s fine because they’ll create new jobs — okay, but… why won’t they be able to do those new jobs better than people too?

It’ll happen for a while; agents will be able to write the code, but a senior developer will need to guide the big picture architectural decisions. Except that then the AI will be able to guide the big picture architectural decisions, and then what? The human will keep moving up and up and up the stack, but eventually there will be no further up to go.

I don’t think AI will replace all jobs. There will be some where humans will be naturally preferable just because they’re humans. I would posit that once we have humanoid robots that can do the physical job of a nurse better than any human, many people will still want to have a human caring for them when they’re sick.

I suspect we’ll also have humans involved in business, but pretty much just relegated to making the subjective decisions. AI will be better at figuring out how to architect a software product, but somebody will still have to tell it what kind of software to build. I do think there will be AIs doing this too; basically someone will provide the capital, spin up an AI agent and say, “go do whatever makes me the most money!” Probably other humans will want to be more involved, though maybe they’ll get outcompeted to AI-only companies.

Insurance renewals, where the entire renewal model depended on policyholder inertia, were reformed. Agents that re-shop your coverage annually dismantled the 15-20% of premiums that insurers earned from passive renewals.

Insurance is a bizarre industry to me. It should be this highly automated and precise thing, in which firms have some information about your business or car or home or whatever it is that you want to insure, and they quote you and you pick the best quote. Instead it’s a tangled mess of having to talk to one guy who routes you to another guy until eventually you answer a bunch of questions and get a quote, and you’re so drained that even the thought of doing it again to see if you can get a better price is exhausting so you just say yes and move on with your day.

This is actually one place where I think agents are a solution to the friction that exists. My agent can simultaneously call every broker and rattle off whatever information they need until they get the absolute best quote. There are reasons why people would use a real estate broker — it’s a huge, often emotional transaction — but none of those apply to insurance. You just want the best price! There is no reason it should be a relationship business.

Financial advice. Tax prep. Routine legal work. Any category where the service provider’s value proposition was ultimately “I will navigate complexity that you find tedious” was disrupted, as the agents found nothing tedious.

I don’t quite agree with the value proposition put forth here; my CPA does my taxes not primarily because it would be a tedious task for me but because I do not possess the requisite knowledge of the tax code to do them optimally.

Still, the overall point is valid. White collar service industries make up an enormous part of the US economy, and they’re ripe for disruption. There will be some regulatory protection, since their respective lobbying groups will do whatever they can to ensure an AI agent cannot act as legal counsel or file taxes on your behalf, but the nature of these markets and the enormous numbers of players in them mean that AI will still lead to price cuts and job losses.

Maybe my AI agent can’t file my taxes for me (though who’s to say it can’t prepare them for me to file myself?), but tech-savvy CPAs should be able to use AI to do most of their work, lower their prices drastically and take market share. They won’t need employees, except perhaps for a few people to handle client relations with those customers who want a human face. Since the market is somewhat opaque and inefficient, firms not using AI will still go on for a while as their longtime customers stick with them out of inertia, but in the end capitalism wins here. If you don’t lean on AI and cut your prices, you’re just not going to last in this new world.

India was the inverse. The country’s IT services sector exported over $200 billion annually, the single largest contributor to India’s current account surplus and the offset that financed its persistent goods trade deficit. The entire model was built on one value proposition: Indian developers cost a fraction of their American counterparts. But the marginal cost of an AI coding agent had collapsed to, essentially, the cost of electricity. TCS, Infosys and Wipro saw contract cancellations accelerate through 2027. The rupee fell 18% against the dollar in four months as the services surplus that had anchored India’s external accounts evaporated. By Q1 2028, the IMF had begun “preliminary discussions” with New Delhi.

Yeah, India is not going to have a good time for the next few years. If your company’s value proposition is that you can offer mediocre to pretty good software developers at a very cheap price, you’re gonna have a bad time.

I’ll just close by saying again that I applaud the folks at Citrini for putting this out. We live in crazy times and there is tremendous value in anything that gets people thinking about what the future is going to look like.

Thanks for indulging me on this one. Back to me using AI to run my business next week!