AI Tech Inspire

Are AI Models Getting Worse? The Economics Behind the Perception

Over the last few months, developers keep asking the same unsettling question in forums, Slack groups, and sprint retros: why do some AI models feel worse even as version numbers tick up? At AI Tech Inspire, that sentiment kept surfacing often enough to warrant a deeper look at the economic incentives, product strategies, and measurement pitfalls that might be shaping these experiences.

At a glance: the key claims driving the debate

  • Many users report a perceived decline in large language model (LLM) quality despite newer version numbers.
  • Anticipation of high-profile GenAI IPOs creates pressure to show stronger revenue and improved unit economics.
  • Training and inference are expensive; larger models typically mean higher costs per token served.
  • After years of VC-subsidized losses, providers face pressure to prioritize profitability, which may change product choices.
  • Hypothesis: gradual “enshittification” once users are locked in—free tiers become less useful; paid tiers shift toward cost efficiency, potentially at the expense of capability.
  • Specific claim in circulation: large investments from NVIDIA into leading labs (e.g., figures like $30B and $10B) align incentives toward GPU utilization and revenue growth. These amounts are not independently verified here.
  • There are few independent, longitudinal audits to detect post-release capability drift; benchmarks can be over-optimized and may miss real-world regressions.
  • The original argument invites scrutiny for logical gaps or alternative explanations.

Why the finance lens matters to developers

The costs behind modern LLMs are nontrivial: compute for pretraining and finetuning, evaluation, inference serving (including multi-tenant scaling), safety tooling, and storage for ever-larger context windows. Even a small shift in tokens per answer or time spent on tool use can move the cost needle. If a provider is aiming to improve gross margin heading into an IPO or major fundraising, there are obvious levers:

  • Route more queries to smaller or distilled models by default.
  • Adjust sampling defaults (e.g., lower temperature) to produce shorter answers.
  • Tighten safety or refusal policies that reduce long-form outputs.
  • Quantize models or swap kernels to cheaper precision at some quality cost.
  • Gate expensive capabilities (advanced tools, longer context) behind higher tiers.

None of these choices require public announcements. A single hidden change to a system prompt, router threshold, or sampling default can alter the user experience overnight.

Alternative explanations besides “enshittification”

It’s tempting to explain every frustration through a business-first lens, but there are credible technical reasons why a model might feel different or worse in practice:

  • Safety tuning drift: Periodic updates to content policies can increase refusals or genericize outputs that used to be detailed.
  • Prompt distribution shift: Changes in your own prompts, longer contexts, or heavier tool use can degrade reliability—especially if the model wasn’t tuned for those patterns.
  • Context dilution: As you push windows to 128K+ tokens, relevant passages become harder to attend to consistently. Retrieval quality or ranker regressions can look like model decline.
  • Sampling defaults: Silent tweaks to top_p, temperature, or penalties can affect creativity and correctness. Even determinism at temperature=0 differs by implementation.
  • Inference optimizations: Quantization, speculative decoding, or lower-precision kernels may trim latency and cost with subtle quality trade-offs.
  • Data and training mix: Shifts toward synthetic data or different decontamination strategies can help on benchmarks but harm unmeasured behaviors.

Key takeaway: perceived quality is multi-causal. Without pinned versions, locked parameters, and repeatable tests, it’s easy to mistake routing or policy changes for capability decay.

What evidence would actually settle this?

If the goal is to detect real capability regression, treat it like an engineering problem. Build a minimal, provider-agnostic evaluation harness and run it on a schedule:

  • Pin everything you can: exact model IDs, API versions, system prompts, sampling parameters, tool availability, and context window.
  • Use temperature 0 for deterministic tasks and fix top_p and penalties.
  • Collect token-level telemetry: prompt tokens, completion tokens, and any available logprobs.
  • Adopt an arena approach (pairwise comparisons with hidden references) for subjective tasks and compute an ELO-style rating over time.
  • Separate routing from capability: force the same backbone model vs. allowing a provider’s router to pick.
  • Cryptographically timestamp results and archive transcripts so you can detect drifts and share evidence.

Open evaluations help, but many are still static. For broader context, keep an eye on community efforts (e.g., MLCommons or academic benchmarks) and supplement with your domain-specific evals—especially for coding, data extraction, and multi-hop reasoning.

Where the original argument is strong—and where it has gaps

  • Strong: The economic incentive to manage inference cost is very real. Even minor changes that reduce average tokens per response can materially improve margins. Free-tier downscoping is also consistent with common SaaS patterns.
  • Strong: The lack of independent, longitudinal audits after release is a genuine gap. Many teams rely on marketing benchmarks that don’t reflect their workloads.
  • Gap: Causation isn’t established. Profit pressure doesn’t require degrading quality; providers can monetize via enterprise features, dedicated capacity, or premium tiers while still improving baseline models.
  • Gap: Specific investment figures (e.g., tens of billions from a single vendor) should be treated as unverified unless cited. Agreements can be a mix of equity, credits, or reseller deals—and they have different implications.
  • Gap: Benchmarks aren’t entirely toothless. While they can be gamed, multi-suite evaluation and community replication reduce the incentive to overfit too narrowly.

Healthy skepticism is warranted—but so is rigorous measurement. Blending both is how engineering teams avoid cargo-cult narratives.

Practical playbook for teams feeling the squeeze

  • Build an abstraction layer: Keep your app vendor-agnostic. Libraries and SDKs make it easier to swap between providers or even fall back to local models.
  • Run A/B across providers and sizes: Rout a percentage of traffic to a smaller or alternative model. Measure tokens per dollar, latency, and quality per task, not vibes.
  • Consider local or hybrid: For predictable workloads, hosting a capable open model can be cost-stable. Tooling like vLLM or llama.cpp on CUDA-enabled GPUs gives you tighter control over routing and quantization trade-offs.
  • Use retrieval aggressively: Well-tuned RAG can reduce dependence on ever-larger backbones and curb context bloat.
  • Guardrails and workflows: Decompose prompts into checkable steps; shallow agents with retries often outperform single-shot monolith prompts.

For model experimentation and deployment, teams typically lean on ecosystems such as PyTorch and TensorFlow for training, while production inferencing and model distribution often run through platforms like Hugging Face. If you’re building content or vision pipelines, keep a parallel eye on diffusion models (e.g., Stable Diffusion) and multimodal LLMs—different cost curves, different failure modes.

Why this matters now

Perception drives adoption. If developers feel their go-to GPT-class API has become less reliable for coding, analysis, or data extraction, they will route around it—either via multi-provider brokering, model ensembling, or open-source deployments. Providers know this. The long game favors those who can improve capability and predictability while being transparent about changes. Until independent audits are standard, the best defense for engineering teams is to observe, measure, and adapt.

In the meantime, a few small habits go a long way:

  • Pin versions, lock parameters, and version your system prompts like code.
  • Track tokens_in, tokens_out, latency, and retries as first-class product metrics.
  • When quality dips, check for silent routing or policy updates before reworking prompts.
  • Document your evals; make it easy for teammates to reproduce and compare runs with a single make or npm command.

The bottom line

The argument that economic pressure could nudge LLMs toward cost-optimized behaviors is plausible—and worth watching. But attributing every quality wobble to deliberate degradation skips over a dense stack of technical variables that can change outcome without malice. Treat this as a systems problem: establish baselines, test continuously, and keep options open. The teams that operationalize their intuition with data will see through the noise—and ship more reliable AI features faster.

Recommended Resources

As an Amazon Associate, I earn from qualifying purchases.