AI Tech Inspire

Chasing Rank 1 in Teamfight Tactics with 100M Simulated Games and Self-Play

What happens when a single-player simulator runs a massively multi-agent strategy game at 5,000+ games per second on a consumer laptop? At AI Tech Inspire, this question caught our attention because it compresses months of reinforcement learning into hours—and it does it without touching a live game server.

Key facts at a glance

  • Goal: Train an RL agent to reach superhuman, rank-1 play in Teamfight Tactics (TFT) via self-play.
  • Scale: Roughly 100M simulated games estimated to reach top-level performance.
  • Constraint: Training in the real game would be costly, slow, require many accounts, and violate Riot’s ToS.
  • Solution: A from-scratch TFT simulator that runs 5,000+ games/sec on a single M4 Mac.
  • Simulator scope: Recreates combat, economy, and RNG systems; target fidelity of 70–80% versus the real game.
  • Observations and actions: ~2,500 state features; actions include roll/level/buy/position/item-combine.
  • Learning setup: Policy + value networks; initial algorithm is PPO; 8 agents train together to mimic real lobbies.
  • Why TFT: Huge state space (~10^2000), imperfect information, heavy RNG but skill-based, multi-agent, and long episodes with 30–40 decisions.
  • Next: Human testing interface to validate simulator accuracy, initial runs on simplified rules, and iterative reward shaping for stability.

Why this approach matters

TFT is a perfect storm for RL: partial observability, high variance, multi-agent dynamics, and a deep economy layer. It’s not just about positioning; it’s also about opportunity cost, reroll timing, and meta adaptation—decisions developers might analogize to resource allocation in distributed systems. The scale target—100M games—may sound unrealistic in a live setting, but a fast simulator turns this into a tractable experiment.

“A simulator running thousands of games per second is a practical superpower. It turns exploration into iteration.”

For engineers who have trained large models like GPT variants or diffusion models such as Stable Diffusion, the lesson is familiar: data quantity and controllability often beat data purity. RL simply trades internet-scale corpora for experience-scale trajectories. The more you can generate—and the more faithfully it approximates the real task—the faster you learn.

Inside the simulator: fidelity vs. speed

Building a complete TFT simulator without source code is the hard part. The project aims to reproduce:

  • Combat: Abilities, items, positioning, damage calculations, and proc logic.
  • Economy: Gold generation, streaks, interest, and leveling costs.
  • RNG: Shop pools, crits, item rolls, and carousel selection.

The fidelity target—70–80%—is a pragmatic call. For RL, perfect emulation isn’t always necessary; consistency and learnability can matter more. If the agent can internalize stable cause-effect relationships (e.g., when to press R to reroll), it can discover strategies that transfer.

Running at 5,000+ games/sec on an M4 Mac suggests careful engineering around vectorized rollouts and batched evaluations. If you’re thinking about porting to GPUs, a CUDA-accelerated version could push rates even higher—though CPU-first design keeps it portable and cost-effective.

Learning setup: self-play at lobby scale

TFT is an 8-player lobby game. Training 8 agents simultaneously matches the environment’s structure and avoids mismatched opponents. The model uses a standard RL recipe—policy and value networks—starting with PPO, a battle-tested baseline popularized by OpenAI Five.

With roughly ~2,500 input features spanning board state, economy, items, and opponent summaries, the architecture choice is still open. In practice, most teams reach for PyTorch for flexible experimentation; some prefer TensorFlow for production serving. Given the combinatorics of itemization and positioning, attention mechanisms or entity-centric encoders often outperform flat MLPs, and action masking can keep sampling efficient when only certain actions are legal.

Another detail engineers will appreciate: long episodes (30–40 decisions) mean credit assignment is tricky. Expect a focus on reward shaping, auxiliary value heads (e.g., economy stability or health preservation), and curriculum-like simplifications early on.

Why TFT challenges typical RL instincts

Most RL tutorials showcase clean Markovian tasks and single-agent control. TFT shatters those assumptions. The environment has:

  • Imperfect information: You rarely see full opponent inventories or exact economy.
  • Stochastic transitions: Shop outcomes and combat crits add variance.
  • Strategic depth: Economic snowballs make early choices echo late-game.
  • Multi-agent dynamics: An action’s value depends on shifting opponent lineups.

These ingredients stress-test algorithms built for stationary environments. Techniques like opponent modeling, population-based training, and meta-learning become not just interesting but necessary. Experience here can transfer to real-world multi-agent problems such as bidding systems, routing, or resource allocation where other actors adapt in parallel.

Developer takeaways and playbook ideas

  • Test harnesses for simulators: Differential tests against known TFT interactions help pin fidelity. Build tiny unit scenarios—1v1 units, single-item edge cases—and lock in expected outcomes.
  • Reward shaping strategies: Mix sparse rewards (placement, win) with dense proxies (interest thresholds hit, health deltas) to stabilize early training.
  • Curriculum design: Start with simplified rules (fewer traits, fewer units), then scale complexity to reduce state-space shock.
  • Instrumentation: Log roll decisions, bench composition distributions, and item combine rates. If behavior doesn’t look human-like, your reward or action space likely needs tuning.
  • Data tooling: If public replays or logs are available, pretrain with imitation learning before switching to RL. Hosting trajectories on Hugging Face can make collaboration easier.

On the systems side, consider whether to keep everything on CPU for simplicity or step up to GPU-based inference with vectorized batches. Even without GPUs, smart batching can yield large wins. If you do go GPU, keeping the rollout loop close to model inference (minimizing host-device syncs) will matter.

Open questions (and fun rabbit holes)

  • Transfer: Will a 70–80% accurate simulator produce policies that transfer to the live game’s meta shifts?
  • Opponent diversity: Does a small agent population overfit to itself? Should a “league” of frozen policies be maintained to preserve diversity?
  • Action abstraction: Are macro-actions (e.g., “fast-8” curve) better than atomic actions? Could a hierarchical policy help?
  • Generalization: Can a policy trained on one set of units/traits adapt when the set rotates—akin to domain adaptation problems in supervised learning?

These questions echo broader ML themes. Just as scaling laws shaped how teams build GPT-style models, similar principles may apply here: more diverse opponents, longer training, and better simulators often win over clever tricks.

Practical cautions

The project exists precisely because training inside the live game would be slow, expensive, and against the Terms of Service. A standalone simulator avoids those issues and encourages responsible experimentation. Still, it’s worth keeping ethics in mind: simulator R&D should not spill into automation of the live service.

How engineers can try this angle

If this sparks the “I need to try it” impulse, a minimal plan could look like:

  • Prototype a micro-simulator that models just economy and basic combat with a few units.
  • Implement PPO with a compact encoder and add action masking for legality.
  • Run self-play with 4–8 agents and watch for degenerate strategies (e.g., infinite rolls).
  • Iterate with scenario tests and slowly add complexity (items, traits, positioning grids).

For deployment considerations, frameworks like PyTorch or TensorFlow will feel familiar, and GPU acceleration via CUDA can help once model inference dominates runtime. If the goal is reproducibility and collaboration, packaging models and datasets via Hugging Face hubs is a straightforward step.

There’s an ongoing public thread documenting progress and pitfalls, including plans for a human testing interface and early training on a simplified ruleset. Expect candid notes on sample efficiency, reward shaping, and stability—all the sharp edges developers appreciate learning from.

For RL enthusiasts and systems-minded engineers alike, the takeaway is simple: a fast-enough simulator plus disciplined iteration can turn an “impossible” training run into a weekend experiment. The TFT angle is timely, but the blueprint—simulate, instrument, iterate—applies widely, from ops strategy games to marketplace simulations. And when someone says you need millions of episodes, remember: sometimes the real unlock is building the world where those episodes are cheap.

Recommended Resources

As an Amazon Associate, I earn from qualifying purchases.