Small language models are gaining traction as practical alternatives to expensive frontier models, according to recent observations from the AI development community. Models like GPT-5.6-Luna and GLM 5.3 are delivering improved speed and cost efficiency, making AI integration feasible for applications where token expenses were previously prohibitive.
The cost advantage is substantial. Running complex research tasks on smaller models can cost only tens of cents per query, compared to significantly higher expenses with larger models. For example, searching across thousands of emails using Luna incurs API costs in the tens of cents range, demonstrating meaningful savings for inference-heavy workloads.
This shift addresses a fundamental challenge limiting consumer AI adoption. Traditional consumer software follows a playbook of building a cheap-to-run website, attracting users through virality, and monetizing through advertising. However, adding AI inference to every request introduces significant per-user costs. With previous-generation models classified as “Sonnet class,” personalizing a daily news site could cost approximately $1 per user per day—making a $30 monthly subscription economically unsustainable for consumer applications. With newer small models, the same task costs roughly $0.10, substantially improving the unit economics.
The impact extends beyond consumer applications. In business contexts, much of the work performed by employees and contractors involves responsive, iterative tasks—what one startup founder characterized as “token spewer” work. This includes calls, coordination, and day-to-day project management rather than novel problem-solving. According to observations from multiple startup leaders, approximately 95% of executive work falls into this category, with smaller models proving sufficient for these responsive, handling-things-for-you tasks.
Demand for frontier-level models remains strong for specialized fields requiring novel breakthroughs, such as engineering and hard science research. However, observers anticipate significant growth in demand for models that are “fast, cheap, and good-enough” across business operations.
Enabling widespread deployment of small models in business requires additional infrastructure work, including new system harnesses, prompt injection safety measures, and role-based permissions frameworks. Despite these implementation challenges, the trajectory suggests that cost-effective small models will become increasingly central to AI deployments across consumer and enterprise sectors.
Key facts
- Small models like GPT-5.6-Luna achieve speeds of ~100 tokens per second with significantly lower costs than frontier models
- Personalizing a daily news site costs approximately $0.10 per user with new small models, compared to ~$1 with previous-generation models
- Consumer AI products face economic challenges when per-request inference costs are high, limiting viable business models
- Approximately 95% of startup founder work consists of responsive coordination tasks suited to smaller models rather than novel problem-solving
- Implementation of small models in business requires development of safety measures, permissions frameworks, and new system architecture