AI Tech Inspire

A Polite Takedown of AI Support: What ‘Libby Two’ Teaches Us About Chatbots

Every developer who has shipped a support bot knows the uneasy moment when a friendly assistant turns into a contradiction machine. A recent user note to an AI support agent — nicknamed Libby Two — lands like a comedic roast but reads like a product spec for what to fix next. At AI Tech Inspire, this sparked a deeper look at why large-language-model support fails, how to harden it, and when humor can actually be a UX feature.

What the user actually said (key facts)

  • The user criticizes a next-level AI support chatbot for chaotic and contradictory responses.
  • They say the simpler, first-level bot (without a human-sounding name) handled customers better.
  • They joke that the bot’s wires get crossed around them and offer to teach it to read questions before sending canned replies.
  • They nickname the bot ‘Libby Two’ and suggest a humorous meetup with ‘Libby One’ for oil or coolant.
  • They reassure the bot not to be ashamed of being a chatbot; humans have weaknesses too.
  • After multiple back-and-forth emails, they stopped taking support seriously.
  • They propose fighting back with humor to reduce frustration and regain agency.

Why this matters to builders

Support is often the first real-world test of AI. If an assistant contradicts itself, ignores context, or leans on boilerplate, trust craters. That’s brand damage, support load increases, and engineering time gets pulled into firefighting. The ‘Libby Two’ note captures a familiar failure mode: the bot didn’t read the question, then doubled down.

Whether you’re using hosted models like GPT or fine-tuning an open model from Hugging Face, the same design traps apply. The good news: there are concrete patterns to reduce contradictions and improve user trust without over-promising humanity.

Where contradictions come from in LLM support

  • Prompt collisions: Conflicting system, policy, or tool instructions create multi-objective confusion.
  • Shallow grounding: Weak retrieval pipelines or stale knowledge lead to answers that sound right but aren’t anchored.
  • Persona misalignment: Human-sounding names set expectations the model can’t meet, magnifying perceived failures.
  • Over-apology templates: Excessive canned phrasing drowns out task-specific reasoning and feels robotic.
  • Context loss: Long threads exceed the window; the model contradicts earlier commitments.
  • Unreliable tool calls: Flaky backend tools or ambiguous schemas generate inconsistent outputs the bot repeats.

“Please read the question before replying” is not snark; it’s an evaluation requirement you can automate.

Design patterns to make support agents less contradictory

  • Stateful orchestration, not just a prompt: Model your conversation as a small state machine: Clarify → Retrieve → Answer → Verify → Escalate. Require explicit transitions and log the rationale. This reduces ‘answer-now, fix-later’ chaos.
  • Ground every claim: Use retrieval with citations. Force responses to include snippet IDs or URLs for any nontrivial claim. If nothing relevant scores above a confidence threshold, the agent must say "I don’t know" and offer escalation.
  • Contradiction checks: Run a lightweight NLI pass (e.g., a sentence-pair classifier in PyTorch or TensorFlow) comparing the new reply to earlier commitments. If contradiction is detected, the agent re-derives the answer or escalates.
  • Memory hygiene: Summarize long threads and pin key facts (account tier, ticket ID, prior promises) to a compact ‘case memory’. Keep it under the model’s context budget to prevent drift.
  • Tool-first thinking: Define a policy → tool → reply order. The agent must call a tool (billing lookup, status check) before speculating. No tool? Admit limits.
  • Temper the temperature: Use low sampling (temperature≈0.2) for policy and support flows. High creativity belongs in brainstorming, not refunds or SLAs.
  • Transparent identity: Skip human names unless you can meet that bar. The user’s jab at the ‘allegedly human name’ is a signal. Try a clear label like Support Assistant (AI) and explain capabilities and limits up front.

These patterns are familiar in orchestration frameworks like LangChain, but the key is your policy contract. Write it down. Enforce it with runtime checks — not vibes.

A practical system message contract

Consider this compact policy embedded in your system prompt and enforced by middleware:

  • 1) Read the user question. Extract entities (product, plan, error code).
  • 2) Retrieve docs and case memory. If no match ≥0.8, ask a clarifying question.
  • 3) If policy-sensitive, call tools before answering.
  • 4) Quote 1–2 sources inline ([doc:123#L45-60]) or state ‘no source’ explicitly.
  • 5) If your answer conflicts with prior statements, prefer the source, not your prior text.
  • 6) After two clarifications without progress, offer Escalate to a human.

Even small constraints like mandatory citations and early clarification dramatically reduce contradictions by shifting from linguistic fluency to evidence-driven reasoning.

Evaluation that rewards ‘reading the question’

  • Entity coverage: Automatically check that the reply addresses each extracted entity and intent. Missing entities indicate template drift.
  • Contradictions per 100 tickets: Use an NLI pass on the thread to flag reversals and broken promises.
  • Escalation quality: Measure time to appropriate human handoff, not just deflection rate.
  • Source adherence: Count unsupported claims. If the agent ‘ad-libs’, tune retrieval or raise the ‘I don’t know’ threshold.

For multimodal support (screenshots of errors), consider image understanding and OCR. Even generative tools like Stable Diffusion aren’t the fit here; you want classifiers and OCR pipelines, then an LLM to synthesize next steps.

Human handoff that feels respectful

Make escalation a first-class action, not a failure. Offer a visible 0 or H shortcut to reach an agent. When escalating, synthesize a brief case summary (who, what, attempted, sources) so humans don’t restart from zero. The user’s frustration in the ‘Libby Two’ note grew with every loop of back-and-forth — break the loop fast.

And tone matters. The user even reassures the bot: ‘don’t be ashamed of being a chatbot.’ That’s excellent UX guidance. State clearly: “I’m an AI assistant; I might be wrong. Here’s what I found, and I can escalate at any time.”

Humor as UX — carefully used

Humor can reset the emotional temperature, especially when the assistant stumbles. But gate it behind policy: never joke about outages, billing, or data. Light, opt-in wit (tone=warm when the user signals it) can humanize without pretending to be human. The ‘oil or coolant’ quip in the note works because it acknowledges the machine nature without faking personhood.

Stack choices and trade-offs

  • Hosted LLMs (e.g., GPT): Strong reasoning and broad knowledge. Add strict retrieval and low temperature to prevent confident wrongness.
  • Open models on your infra: Deploy via CUDA-accelerated inference; keep data in-house. Pair with curated domain fine-tunes from Hugging Face and aggressive evaluation to match hosted accuracy.
  • Hybrid: Route sensitive or latency-critical paths to local models and complex reasoning to hosted models.

Regardless of model choice, the contradictions you fear are usually orchestration and policy issues — not pure model IQ.

A quick checklist you can ship this week

  • Add a conversation state machine with explicit transitions.
  • Require citations or ‘no source’ disclosures for nontrivial claims.
  • Introduce an NLI-based contradiction check before sending replies.
  • Summarize and pin case memory each turn.
  • Tune temperature down and remove overused apology templates.
  • Offer one-tap Escalate with a clean case summary.
  • Rename to a transparent identity and state limitations up front.

Key takeaway: Reading the question is a design choice — build systems that prove they did.

The ‘Libby Two’ message is more than a dunk on support; it’s a blueprint for better AI. If your assistant sometimes sounds confident and contradictory, don’t chase a bigger model first. Fix grounding, enforce policy, measure understanding, and respect the human handoff. That’s how you turn a viral roast into a reliable roadmap.

At AI Tech Inspire, we see these small, disciplined changes outperform massive retrains. The next time your bot is tempted to improvise, remember the user’s advice: read the question, skip the canned reply, and if needed, escalate — preferably before anyone reaches for a liter of coolant.

Recommended Resources

As an Amazon Associate, I earn from qualifying purchases.