AI Assistant for Employee Onboarding and Knowledge Retrieval

Introduction

To streamline onboarding and reduce pressure on experienced employees, we created an AI Assistant that empowers new hires to independently find answers to typical onboarding queries. Tailored for the bank’s call center operations, the Assistant enables newcomers to engage in realistic question scenarios and receive trusted, reference-backed answers. This self-service approach helps accelerate ramp-up time while freeing senior staff from repetitive training duties.

Expertise

LLMs, embeddings, reasoning, AI agents, data ingestion, human in the loop

Problem

Frequent employee turnover led to a constant cycle of onboarding and training. Senior staff were overwhelmed by the ongoing need to guide new hires, which reduced their availability for operational tasks. Additionally, company knowledge was fragmented across various internal documents, often lengthy, inconsistently written, and difficult to navigate. This created inefficiencies for newcomers trying to learn on the job and locate precise information.

Specific Pain Points:

• High training burden on experienced employees due to regular onboarding cycles
• Scattered information across multiple internal documents, some outdated or conflicting
• Mismatch in language between how users ask questions and how documentation is phrased
• No existing feedback loop to identify outdated or unclear documents
  

Solution

The Assistant application processed all available internal documentation using semantic segmentation and enrichment to make content more searchable and usable. New employees could interact with the system via a simple text-based interface, receiving AI-generated answers along with the relevant source references and a confidence score. A human-in-the-loop setup allowed experienced staff to review AI answers, give feedback, and flag documentation issues, gradually refining the system and content over time.

To ensure reliability, additional mechanisms were implemented to reduce hallucinations and improve consistency minimizing the chances of the model giving slightly different answers to the same question when asked multiple times. This helped build trust with users and aligned output with documentation truths.

Specific Steps Taken:

• Developed a web-based Assistant with a simple, intuitive interface using FastAPI and React
• Built optimized retrieval infrastructure using vector databases and re-ranking
  Applied domain-specific query transformations to bridge the gap between natural language and documentation phrasing
• Implemented a human-in-the-loop system for continuous learning and QA validation
• Introduced hallucination mitigation strategies and prompt stabilization to improve answer reliability
  Conducted extensive testing and evaluation using realistic onboarding scenarios
  

Unique Value Proposition:

• Delivered a complete, production-ready solution combining LLMs, RAG, and semantic search tailored to real-world needs
• Enabled real-time human feedback and iteration, empowering domain experts to directly improve both the Assistant and the documentation
• Implemented safeguards for answer consistency and hallucination control, fostering trust and accuracy in everyday usage
• Created a living knowledge base that evolved with each user interaction, increasing long-term value
• Integrate MS Teams and development of Azure AI Assistant to enable easier usage over existing communication application within organization
  

Results

Key Metrics:

• Achieved 87% accuracy on a curated evaluation set with strict scoring criteria (zero-or-nothing standard for correctness), before incorporating human review
• Cut down onboarding time for new employees by reducing manual document navigation
• Lowered dependency on senior staff for answering repetitive onboarding questions

Smart Tip

When deploying AI tools for internal use, make documentation improvement part of the workflow. Human-in-the-loop systems don’t just correct AI, they help improve your content and processes over time.

Smart Fact

Over 70% of new hires reported feeling more confident navigating internal knowledge within their first week thanks to the AI Assistant.

About the Clients

Big multinational bank that has footprint in various countries. Being European based made scaling of this solution to multiple countries and therefore languages much harder.

Technologies Used

• Agentic AI
• Retrieval-Augmented Generation (RAG)
• Semantic Search
• Re-ranking Pipelines
• Large Language Models (LLMs)
• Vector Databases
• FastAPI
• React