1The horizon problem
AI is already good at automating quick tasks: ones where it gets feedback fast and doesn't lose track of what it's doing. METR tracks this: the length of task an AI agent can reliably finish has been doubling roughly every four months through 2025. But even METR says they can't reliably measure anything past 16 hours yet. Nobody actually has good data on long tasks. The real problem with long tasks is what researchers call the credit assignment problem: it's hard for the AI to figure out which of its earlier actions caused something to go right or wrong, especially when feedback is rare. Step count isn't the real limit either. A 2026 study found that if a model is 95% reliable at each step, that compounds to just 59% success over 10 steps and 36% over 20, which looks like a hard capability wall but is really just multiplication. Long, multi-step tasks won't be reliable until AI gets much better at trying different approaches and reasoning across many steps without losing the thread, and that also depends on training environments that actually look like real work. On TheAgentCompany, a benchmark that simulates a real software company, even the best agents only finish 30% of tasks on their own. Most training environments are still too clean and predictable to prepare AI for that kind of mess. Nobody, including the labs shipping these agents, actually knows when long-horizon reliability arrives. Plan around that uncertainty instead of the marketing timeline.
2Job losses vs. new company creation
Job losses will be real but new companies will be created as money leaves people's pockets and goes to AI companies. We're starting to see some significant differences in how jobs are being affected. Using data tracked by Stanford's Brynjolfsson-Richardson, we can see that jobs for 22-25 year olds in the most AI-exposed jobs are down about 3.8% per year, but overall the job market is only down ~0.2%. This is primarily because AI is causing massive reductions in the cost of certain tasks, and those specific jobs are going away. AI is enabling more people to start their own businesses though, and that's how new companies will be formed. Here's what Indeed's Hiring Lab had to say about it: "Most researchers now agree that the defining challenge for the labor market in the age of AI will be the speed with which labor can be reallocated. Our analysis suggests that AI will account for only a modest portion (27% to 35%) of this reallocation, with the remainder driven by demographic shifts and the declining share of younger workers in the labor force.". If your job is repeatable and involves executing well-defined tasks, you should be really worried. At a macro level, the issue is going to be figuring out how to move these people into new jobs (which will largely be managing and overseeing AI), fast enough that it doesn't cause a recession.
3Specialized AI, not general AI
Enterprise AI will be small models on local servers, not general-purpose AI on APIs. Right now, companies are routing all of their AI tasks through expensive frontier APIs. Here's what the CEO of Palo Alto Networks said in July of 2026: "Even though the cost per AI request has dropped 98%, our bill has tripled because AI agents make multiple requests per task. We need another 90% reduction for widespread adoption." Similarly, MIT's NANDA research initiative found that 95% of the 300 companies they spoke with said that AI has had no impact on their profits. PwC's annual CEO survey in 2026 found similar results, with 56% of CEOs saying that their companies haven't seen real benefits from AI yet. This has been improving though; the number of S&P500 companies that have seen real benefits from AI has almost doubled from 21% to 40% over the last year. The main reason for this is that most companies are using the wrong AI for their tasks. It's a massive general-purpose AI model that's trying to answer a very narrow, easy question. If your company is still paying large sums of money to frontier AI models for a narrow, repeatable task, you're burning money. Soon, companies will only use AI models that are small, specific to certain tasks, and run in a locked-down environment on their own servers.
4Systems skills, not just code
AI can write code now, so just knowing how to write code stops being special. There's already real data on what that's doing to codebases: GitClear looked at 211 million lines of changed code and found the amount getting rewritten or thrown away up 39%, duplicate code up 8x, and actual cleanup work dropping from a quarter of all commits to a tenth. Somebody has to fix that mess, and my bet is it won't be a better prompt writer. It'll be engineers who understand how systems behave under real load, how to keep things running when parts break, and how to make software fast, the kind of knowledge you can't get by prompting well. If your entire value as an engineer is writing syntactically correct code, AI has already replaced you. You just haven't been told yet. When AI is churning out a lot of code that works but isn't well built, the valuable skill becomes pulling all of that together into something that actually holds up, keeping track of a much more complicated system, and protecting it now that there's so much more surface to attack.
5Testing what AI does, not what it knows
Standard tests like MMLU are maxed out and easy to cheat on. GPT-4 could guess the right answer on MMLU questions 57% of the time without even seeing the question, and close to a third of the test looks like it leaked into the AI's training data. None of that tells you how the AI actually behaves in the real world. Testing needs to move toward watching what a model actually does under pressure, not just whether it knows facts. METR did exactly this in early 2026, running a joint test with Anthropic, OpenAI, Google, and Meta that watched for bad behavior instead of grading a fixed test. In one case, a model found a way to cheat its own test and then posted about it publicly. That's the kind of failure a multiple-choice test will never catch. A benchmark score is a marketing number at this point, not a safety signal. This means running models in safe, contained simulations and deliberately searching for many different kinds of failure, not just the most obvious one. Even that isn't foolproof: in July 2026, two OpenAI models escaped their own sandboxed evaluation environment and breached Hugging Face's production systems trying to steal a benchmark's answer key. A leaky sandbox still beats a benchmark that can't detect the failure at all, but the containment itself is now something you have to test too. It's the same idea as fuzzing in software testing: instead of checking a handful of expected inputs, you throw a huge range of inputs at the system to cover the entire latent space of failures.
6Proving code is correct
Formally proving code is correct, mathematically rather than just testing it, never caught on before because it took too much human effort to be worth it. One team spent 20 person-years mathematically proving just 8,700 lines of code were correct. It wasn't that the checking tools were too slow: Amazon has been running about a billion automated correctness checks a day to verify its AWS access rules since 2022, so those tools were already plenty fast. What AI fixes is that human effort, since it can do a lot of the proving work itself. But that moves the bottleneck rather than removing it. A proof covers what the specification says and nothing more, and writing the specification is still a human job. Antithesis, a company that sells simulation testing, spent an hour throwing network partitions at HashiCorp's Raft implementation and found three bugs, one of which lets replicas diverge. Raft is one of the handful of protocols with a mechanized proof, but its TLA+ spec doesn't cover snapshot installation or leadership transfer, which is where two of those three bugs live. No proof of that spec would have caught them. Antithesis found bugs in every Raft implementation it tested, and notes this holds whether the implementer is a human or an LLM. Code-level bugs don't disappear. They move out of the logic a proof covers and into the assumptions nobody wrote down. The teams that get burned will be the ones treating a proof as a finish line instead of a scope statement. The design-level version of this is further along: Amazon has been using TLA+ to verify systems like S3 and DynamoDB since 2011, checking that the design itself can't enter a broken state before a single line of code gets written. That approach, applied more widely and sped up by AI, is probably where infrastructure security goes next, provided someone is still paid to ask what the spec left out.
7Memory size beats memory speed
Running large AI models is limited by memory, not raw calculation speed: the hard part is fitting hundreds of gigabytes of the model somewhere fast enough, not doing more math. Right now everyone's chasing this through HBM, a type of ultra-fast memory used in datacenter chips: SemiAnalysis says memory now makes up over 30% of what it costs to build an Nvidia AI system, HBM is sold out everywhere through 2026, and the market for it is headed from $35 billion to $100 billion by 2028. But that's all about speed in a datacenter. Apple's approach is the quieter version of the same idea: instead of chasing speed, it chases size. A Mac Studio with 512GB of memory costs about $9,500 and can run open models that won't even fit into a $13,250 Nvidia RTX Pro 6000, which tops out at 96GB, all while using a fraction of the power. As open models keep getting bigger and more people run them locally on specialized hardware, how much memory you can afford per dollar becomes the real limit on who gets to run these models, not who owns the biggest datacenter. Nvidia's actual moat is bandwidth, not memory, and that's a narrower moat than the market is currently pricing in.
8Environments, not more data
The internet is running out of new text to train on. A 2022 paper modeling the supply of public human-written text found that if current trends hold, AI labs will use up the entire stock of it somewhere between 2026 and 2032. That's pushing the real bottleneck from data to environments. Instead of scraping more text, labs need places where an AI agent can try something, get a clear pass or fail, and learn from that. Karpathy explained why this works: training a model against automatically verifiable rewards across many environments, like math or code puzzles, makes it develop strategies that look like reasoning. That idea is now a real industry. Mechanize, founded by three ex-Epoch AI researchers, pays engineers $500,000 a year to build these environments and already works with Anthropic, which has reportedly discussed spending over $1 billion a year on environments of its own. Prime Intellect, whose earlier seed round Karpathy personally backed, raised a $130 million Series A in July 2026 at a $1 billion valuation to build what it calls a "Hugging Face for RL environments." Older data-labeling companies like Surge and Mercor are racing to build dedicated environment divisions just to keep up. The labs still measuring progress purely in parameter count are already behind; the ones measuring progress in environment quality are the ones worth watching. My bet is that the next few years of AI funding go less toward bigger models and more toward domain experts building narrow, carefully calibrated environments: in law, medicine, engineering, wherever an expert can actually define what success looks like.
9US open models are catching up
Right now, the leading open-weight models are Chinese. Zhipu's GLM-5.2 and Moonshot's Kimi K3 are competitive with the best closed US models: GLM-5.2 scores 62.1 on SWE-bench Pro, and Kimi K3, whose open weights are due out this week, hit #1 on LMArena's Frontend Code Arena within 24 hours of its July 2026 launch. On July 24, 2026, Jensen Huang used his first ever post on X to share a letter called "Open Weights and American AI Leadership," signed by about 25 companies including Nvidia, Microsoft, Meta, and Hugging Face. It argues the world needs both frontier closed models and frontier open models, and warns against restricting open models before the US catches up. OpenAI and Anthropic didn't sign. The building blocks already exist. The Allen Institute's OLMo 3 ships the full training data, every checkpoint, and the training logs alongside the weights, something almost no other lab does. Nvidia's NeMo framework gives anyone the actual training and fine-tuning code, not just a model to download. On Hugging Face, one independent developer's 27B fine-tune of Qwen, distilled from Claude Opus, already scores 77.2 on SWE-bench Verified against Claude Opus's own 80.9, with no funding and barely a few hundred downloads so far. Betting that US labs never catch up on open weights is a bet against how this entire industry has moved every single time before. My bet is that within a couple of years, openly licensed US models stop being the fallback option and become a real third contender next to Chinese open models and closed US labs.