Implementation of an LLM-based Chatbot with
RAG Structure
Somyajit Chakraborty
University College Cork
Research Assistant
October 22, 2024
Abstract
This paper presents the structure and implementation of a Large Lan-
guage Model (LLM)-based chatbot that utilizes Retrieval-Augmented Gen-
eration (RAG) agents for geospatial tasks and emotional interaction. The
chatbot is designed to guide users along scenic or optimal walking paths
while providing personalized and emotionally supportive responses based
on real-time input. We also explain how to integrate this LLM system
with an Android application. The design is inspired by recent advance-
ments in geospatial LLMs, including the GeoGPT framework [1].
1 Introduction
The integration of Large Language Models (LLMs) into mobile applications
for real-time geospatial tasks is a rapidly advancing field. Retrieval-Augmented
Generation (RAG) models have recently gained popularity for optimizing search-
based operations, improving the performance of LLMs in handling user queries
[2]. This paper outlines the structure of an LLM-based chatbot that uses RAG
agents to provide route guidance, retrieve points of interest (POIs), and offer
emotionally supportive interaction with users.
We aim to demonstrate how this system can be implemented in an Android
application by leveraging Docker to host the LLM backend. The design closely
follows the GeoGPT structure [1] and builds upon recent developments in LLMs
[4].
2 System Structure
The system comprises three main components:
• RAG Agents: Responsible for retrieving relevant information from map
databases and POIs.
1
• LLM-based Emotional Interaction: Handles real-time emotionally
supportive responses to user inputs.
• Android Application: The front-end for users to interact with the chat-
bot, request routes, and receive POI or emotional responses.
The system’s core workflow involves classifying the user’s input (e.g., route
requests, POI queries, or emotional statements) and using RAG agents to re-
trieve relevant data efficiently. The Android application connects to the backend
server, where the LLM processes the data and generates responses.
User Input
Input Classification
Routing Agent POI Agent
Emotional Agent
Response
System Structure of the LLM-based Chatbot
3 RAG Structure and Agents
The RAG structure enhances the efficiency of LLMs by retrieving the most
relevant data from external sources before generating responses. In this chatbot,
three RAG agents are employed:
• Routing Agent: Uses user preferences (e.g., shortest or scenic paths) to
retrieve optimal routes from a geospatial database.
• POI Agent: Fetches information about points of interest located along
the user’s route.
2
• Emotional Agent: Classifies emotional inputs and generates empathetic
responses using LLM models.
By using these RAG agents, the chatbot can reduce the search space and
retrieve relevant responses more efficiently.
User Query
Search Space
RAG Agents
LLM Generation
Output Response
RAG Structure for Route and Emotional Interaction
4 Backend Implementation
The backend server for the chatbot is implemented using Python, FastAPI, and
Docker. The LLM models, such as LLaMA or Mistral7 [3], are hosted within
Docker containers, allowing for easy deployment and scalability.
4.1 Backend Components
• LLM Model: Hugging Face models like LLaMA or Mistral7 are used for
generating responses to user inputs.
• API Server: FastAPI is used to expose endpoints for retrieving routes,
POIs, and emotional interactions.
3
• Docker: The entire backend is containerized for easy deployment across
different environments.
The Android application interacts with this backend by sending HTTP re-
quests, which are processed by the API server. The backend retrieves the nec-
essary data using RAG agents and LLM models, and returns a response to the
Android app.
Android App
HTTP Request
API Server (FastAPI)
Dockerized LLM
HTTP Response
Backend Connectivity with Android Application
5 Conclusion
This paper presented the design and implementation of an LLM-based chatbot
integrated with RAG agents for geospatial and emotional interaction tasks. By
utilizing Docker and FastAPI, the backend can be easily deployed and scaled
across various platforms. The chatbot provides users with personalized walking
routes, POI information, and emotionally supportive responses, making it a
robust tool for real-time interaction. Future work will focus on optimizing model
performance for mobile environments and integrating additional features such
as voice interaction and weather-based route suggestions.
4
References
[1] Zhang, Y., et al. (2023). GeoGPT: Understanding and Processing Geospatial
Tasks through An Autonomous GPT. arXiv preprint arXiv:2307.07930.
[2] Lewis, P., et al. (2020). Retrieval-Augmented Generation for Knowledge-
Intensive NLP Tasks. Advances in Neural Information Processing Systems.
[3] Hugging Face. (2023). LLaMA and Mistral Models. Retrieved from https:
//huggingface.co/models.
[4] Touvron, H., et al. (2023). LLaMA: Open and Efficient Foundation Language
Models. arXiv preprint arXiv:2302.13971.
