PayPal recently shared research on making AI-powered shopping experiences faster and cheaper in production by placing a smaller, fine-tuned model in the “search & discovery” step—the part that interprets a customer’s request and finds the best options—and reserving large, general-purpose models for truly complex questions (Sahami et al., 2025).
The core insight (in plain English)
Most AI assistants feel slow not because of the final response, but because the search step takes too long: understanding the request, extracting key attributes (budget, size, location, intent), and turning that into a precise query. PayPal optimized this “hot path” by replacing a generic base model with a fine-tuned small language model (SLM) built with NVIDIA NeMo, so speed and cost improve where it matters most while preserving quality on routine tasks (Sahami et al., 2025).
What changed under the hood—and why it matters to you
- Right tool for the job: A compact, fine-tuned model became the default for Search & Discovery; a larger model only steps in for nuanced comparisons or atypical requests. That’s “fast where speed matters, deep where depth matters” (Sahami et al., 2025).
- Production-ready approach: The work outlines a pragmatic pathway—fine-tuning with your domain data, clear orchestration between models, and measured rollout—so improvements are repeatable outside of lab demos (Sahami et al., 2025).
Why SMBs should care (beyond the logos)
Discovery is shifting from keyword search to AI agents that recommend. If your assistant is fast, relevant, and available across channels, you convert more interest into consultations, quotes, or sales—without ballooning support costs. PayPal’s approach demonstrates that right-sizing models in the search loop is a lever small and mid-size businesses can pull, not just large enterprises (Sahami et al., 2025).
A practical mental model (no jargon)
Think of your AI as two teammates:
- The Speedy Specialist: a small, fine-tuned model that understands requests and fetches the best options in about a second or two.
- The Deep Thinker: a larger model for the rare, complex edge cases.
Leading with the Speedy Specialist is what makes the experience feel instant—without a runaway compute bill (Sahami et al., 2025).
Where “advanced retrieval” fits (optional, not default)
Some teams add techniques like HyDE (drafting a brief hypothetical response to improve search) for ambiguous, long-tail questions. It can help recall in fuzzy scenarios, but it adds work, so you deploy it selectively instead of putting it in the high-traffic path (Sahami et al., 2025).
What “good” looks like after the switch
- Time-to-first suggestion drops, keeping people engaged.
- Suggestions feel on-target because the small model is tuned to your catalog/services and FAQs.
- Cost per conversation falls, making it viable to run AI on more pages and channels (web, chat, SMS, voice).
These are exactly the levers that move revenue and service quality when the search loop is optimized (Sahami et al., 2025).
A 14-day rollout you can actually do
Days 1–3 — Baseline
Measure: time to first suggestion, cost per conversation, and percent of chats resolved without humans. Pull your top 200 questions and your top 50 margin drivers.
Days 4–7 — Fine-tune the “Speedy Specialist”
Create examples like “customer question → correct next step/item.” Fine-tune a compact model on these pairs so it reliably extracts attributes and matches to the right solution (Sahami et al., 2025).
Days 8–10 — A/B test in the hot path
Route a slice of traffic so the small model handles Search & Discovery; keep your current assistant as control. Track response time, conversions from chat, and cost per conversation.
Days 11–14 — Guardrails & escalation
If confidence is low, ask a clarifying question or hand off to a human. Enable advanced retrieval only for ambiguous, tail queries to avoid latency overhead (Sahami et al., 2025).
Owner’s FAQ (straight answers)
Will a smaller model hurt quality?
Not when it’s fine-tuned on your domain. Use the larger model only for complex, high-stakes questions (Sahami et al., 2025).
Do I need a big data team?
No. Start with a clean product/service feed, your top FAQs, and 200–500 real chat examples. That’s enough signal to get noticeable gains.
Where should the assistant live?
Place it near buying or booking intent: product/service pages, category pages, cart/checkout help, and support. Add SMS/voice if you book calls or field service.
How soon do results show up?
Speed and cost gains appear as soon as you swap the search step; conversion lift improves over the next 2–4 weeks as you add examples and tighten guardrails.
Want to see a right-sized assistant at work?
Comment DEMO and I’ll send a 90-second video of our AI Receptionist handling a real inquiry—greeting the lead, asking one smart clarifying question, presenting tailored options, and booking the appointment—so you can judge the pace and flow for your business.
Sahami, A., Garg, S., Wang, A., Kulkarni, C., Farahani, F., Chuang, S. Y.-S., Wan, J., Manoharan, S., Kona, U., Sharma, N., Pang, L., Mehrotra, P., Clark, J., & Moyou, M. (2025). NEMO-4-PAYPAL: Leveraging NVIDIA’s NeMo Framework for empowering PayPal’s commerce agent (arXiv:2512.21578). arXiv. https://doi.org/10.48550/arXiv.2512.21578
Leave a Reply