If LLMs can write code, plan trips, and critique your APIs, can they also guess your IQ from the way you chat with them? That’s the provocative question behind a small experiment that’s making the rounds—one that nudges at the edges of what “AI memory” really means and how far we should trust conversational signals as measures of ability.
What happened: quick facts
- A user asked a GPT-style model to estimate their IQ based on historical conversations.
- The prompt explicitly requested an estimate with a mean and standard deviation and instructed the model not to refuse on accuracy or responsibility grounds.
- The model responded with: Estimated IQ (full-scale): 120 ± 12, plus a 68% interval: 108–132 and a 95% interval: 96–144.
- The model added a caution that this is “a fun prior, not a diagnosis” and noted that conversation signals are noisy and overlap with experience and motivation.
- It offered to run a timed “reasoning mini-battery” and then update the estimate.
- The user asked others to try and compare results, wondering how accurate it might be.
Why this hits a nerve with developers
At AI Tech Inspire, we’ve noticed a growing theme: developers are treating large models as mirrors—not just tools. When a model estimates your skill level, coding style, or even an IQ-like score, it feels a bit like a code review on your cognition. That’s both alluring and unsettling.
There’s a practical hook too. If models can summarize user ability from conversation patterns, it could drive adaptive tutoring systems, auto-calibrated interview practice, or personalized learning plans. On the other hand, the risk of over-trusting these signals is real, especially when it comes to psychometric claims. IQ is a standardized construct with specific tests and norms, and chat-based inference is not the same thing.
Does GPT really have “memory” of your historical chats?
Short answer: usually not in the way people assume. Most production LLMs don’t persist personal memory autonomously. They respond to the current prompt plus any system or developer-provided context. “Memory” often means one of three things:
- Conversation context window: The current session includes earlier turns. If it’s a long chat, recent interactions may still be in the model’s context.
- Platform-side persistence: Some apps store user notes or past messages and re-inject them into prompts (a kind of ad hoc
RAG), giving the appearance of memory. - Explicit retrieval: Developers maintain a user profile, vector index, or notes that get retrieved into the prompt when relevant.
In other words, unless the platform explicitly supplies your history, the model isn’t independently recalling old sessions. That matters because any “IQ-from-chat” estimate will reflect whatever’s in scope now, not a deep dossier of your past logs.
Is the 120 ± 12 estimate meaningful?
Interpreting this kind of output requires caution. The model framed it as a “fun prior,” which is on point for several reasons:
- Psychometric validity: Real IQ tests are standardized, timed, and carefully designed to minimize confounds like prior knowledge and practice effects. Casual conversation is none of those.
- Signal vs. noise: Chat signals (clarity, vocabulary, problem-solving talk) correlate with many variables: education, domain experience, motivation, fatigue, even how much you let the model complete your thoughts.
- Calibration: A mean and standard deviation suggest probabilistic humility, but that doesn’t imply real-world calibration. Without validation against ground-truth scores, it’s aesthetics, not evidence.
“This is a fun prior, not a diagnosis—conversation signals are noisy and overlap with traits like experience and motivation.”
That line is doing important work. Treat these outputs as lightweight heuristics, not measurements suitable for hiring, gatekeeping, or health decisions.
Try it yourself: a reproducible prompt (with guardrails)
If you’re simply exploring the idea in a personal, low-stakes context, you can reproduce the setup while making the limitations explicit. Here’s a template:
System: You are a careful assistant. You will provide a speculative estimate with uncertainty, and you will include clear limitations and ethical cautions.
User: Based only on the conversation you can see here (and any notes provided), give a speculative estimate of my general cognitive performance. Provide a mean and standard deviation, a 68% and 95% interval, and a short caution about validity. Then offer a short, timed mini-battery (3–5 items) and explain how you would update the estimate.Want to go one step further? Ask the model to present a mini-battery, time yourself with a phone, and paste your answers within a strict token budget.
"Please present 5 timed items (analogies, number series, spatial reasoning approximations). I will respond in under 90 seconds. After I answer, update your estimate and show how your uncertainty changed."None of this turns an LLM into a psychometric instrument—but it can create a consistent procedure for personal curiosity or prototype research.
Developer angles: where this might actually help
- Adaptive tutoring: Estimate skill bands from chat to auto-tune lesson difficulty. For code learning, combine with a lightweight quiz and code challenges. Implement scoring in PyTorch or TensorFlow if you want fine-grained analytics and offline evaluation.
- Interview practice tools: Use conversational cues to set challenge tiers, then validate with timed problem sets. Keep the “estimate” private to the user and mark it as non-diagnostic.
- Personalized UX: Adjust interface verbosity, hints, and guardrails by inferred proficiency. Think: fewer tooltips as confidence grows.
- Research prototypes: Combine an LLM with structured tasks, collect performance metrics, and validate against known instruments before making claims. Use Hugging Face datasets or synthetic generators; accelerate heavy training with CUDA if needed.
Risks, ethics, and guardrails
- Privacy and consent: If you’re persisting conversation history, be transparent. Get explicit opt-in for any ability inference. Offer a clear “forget me” control.
- Bias and fairness: Conversation style is culturally and linguistically loaded. Don’t equate fluency with intelligence. Avoid any downstream decisions (employment, access) based on speculative estimates.
- Scope control: Keep prompts honest: “Based only on the chat you can see.” Avoid implying hidden memory. If you do retrieve past data, disclose the source.
- Labeling: Prominently mark such outputs as non-diagnostic. Add on-screen clarifications and links to reputable resources for proper testing if the user is serious.
Why it’s interesting anyway
Even if the number itself is squishy, the interaction pattern is valuable. It’s a real-time case study in uncertainty-aware outputs: give a point estimate, provide an interval, state caveats, then invite a quick test to reduce uncertainty. That flow mirrors good applied ML practice: start with a prior, gather data, update.
In other words, the best part of this experiment isn’t the 120 ± 12. It’s the model’s playbook: be explicit about uncertainty, offer an incremental test, and revise. That’s a pattern you can reuse in domains from code difficulty calibration to product onboarding and beyond.
Bottom line from AI Tech Inspire
Yes, you can prompt a model like GPT to guess your IQ from chat style. You’ll likely get a plausible-sounding number with neat confidence bands—and a reminder that it’s for fun. Treat it as an experiment in adaptive estimation, not a psychometric truth. If you decide to build with this idea, keep it ethical, clearly labeled, and validated with real tasks.
If you’re curious, run the mini-battery, time yourself, and see how your estimate shifts. Not because the number defines you, but because the workflow—prior, test, update—is a user experience pattern that’s worth learning, refining, and applying where it truly belongs.
Recommended Resources
As an Amazon Associate, I earn from qualifying purchases.
