Skip to content

Case study · 05 / 06

AppleSupport AI

AI Customer Support Intelligence

Stack
Python · Jupyter Notebook · RAG · Embeddings · TF-IDF · LLM Evaluation
Links
GitHub

Fig. 05 — AppleSupport AI · conceptual render

Overview

An AI support-agent project that reconstructs customer-support conversations, retrieves relevant responses, classifies requests, and flags cases needing human escalation — with LLM evaluation and human validation.

Problem

Support teams drown in conversations with no systematic way to know what was asked, what worked, and what needs a human.

How it works

  1. 01

    Conversation

  2. 02

    Classification

  3. 03

    Retrieval

  4. 04

    AI response

  5. 05

    Escalation

A retrieval pipeline over reconstructed conversations — TF-IDF baseline vs. embeddings, RAG retrieval, classification, and an explicit escalation policy, evaluated by LLMs and validated by humans.

Key features

  • Conversation reconstruction + classification
  • TF-IDF baseline vs. embeddings
  • RAG retrieval + escalation policy
  • LLM evaluation + human validation

Lessons learned

  • Lesson 1: A TF-IDF baseline keeps embedding claims honest.
  • Lesson 2: An explicit escalation policy beats a confident wrong answer.
  • Lesson 3: Human validation belongs inside the pipeline, not after it.

My contribution — Built the conversation-reconstruction and retrieval pipeline, classification and escalation logic, and the evaluation harness.