What if linear improvements require exponential resources?

Despite all the noise around GenAI, here’s a paper that makes more realistic expectations.

In short, for “zero-shot” generalization of multimodal models, linear improvements will require exponentially more data.

When engineers hear “exponential,” we get skeptical. That’s especially true when someone claims performance will keep improving linearly, yet it requires exponentially more resources.

This also means GenAI as a field is nearing (or at) a point where bold claims of “at least” linear performance gains long-term will trigger more skepticism. Claims like this don’t work when a market’s complexity is approaching a point of criticality.

Link to paper: https://arxiv.org/abs/2404.04125

—

Found this paper useful? Share it with others.