In AI, a lot of progress still comes down to trial and error — and sometimes, plain old luck. We can’t even predict how many images you might need to train a cat classifier to 95% accuracy.
When researchers train giant neural networks, the outcome can swing wildly depending on small, random details. Change the initialization seed, shuffle the data differently, or even let the GPU run in a slightly different order, and you might end up with a model that either crushes benchmarks… or flops.
Big labs try to beat this randomness by brute force — running thousands of experiments in parallel until something works. Smaller teams don’t have that luxury, which is why AI breakthroughs often come from places with deep pockets.
Scaling laws, optimization tricks, and theory give us islands of predictability. But we don’t yet have the “physics of deep learning” — the equations that would let us design a network and know it’ll hit 95% accuracy without a thousand failed runs.
Until then, success in AI will keep feeling less like engineering and more like informed gambling with increasingly sophisticated strategies.
