AI in Practice: Boost Your Business with AI-Powered Chatbot
“AI is not going to take your job. The person who uses AI is going to take your job.” – Jensen Huang, CEO, NVIDIA.
If we extrapolate this, will companies that integrate AI into their workflow gain a significant advantage over those that don’t?
By automating routine tasks, AI-powered chatbots save time and allow employees to focus on more strategic work. They provide quick access to important data, making decision-making faster and more accurate. Chatbots use powerful AI to offer human-like interactions, improving customer service and internal communications.
However, companies need to ensure data and system security by limiting the chatbot’s access to necessary data and functions, and conducting regular checks to prevent misuse. One major benefit is that users do not need to be technically skilled, such as writing SQL queries, as the chatbot can understand and process natural language requests.
These customizable and scalable solutions are designed to grow with your business, making them valuable assets for any company looking to enhance efficiency and effectiveness.
Contents
Transformative Journey: AI-Powered Chatbot
We embarked on a transformative journey to enhance productivity by incorporating chatbots into our workflows. Every team member now benefits from a ChatGPT Teams subscription and, soon, access to our internally built chatbot. Our chatbot is built on a robust technological foundation, including:
Chainlit with Azure Single-Sign-On (SSO) enabled
Overview: An open-source framework designed to assist developers in building conversational applications using large language models (LLMs).
Features: Simplifies the process of creating and deploying conversational agents, allowing for seamless integration of LLMs into various applications.
LangChain with Tools and Agents
Overview: An advanced framework aimed at helping developers create applications that leverage large language models.
Focus: Simplifies the integration of LLMs into various workflows, providing tools and components to enhance the development process.
Tools Demonstration:
- Python: Leverages the bot’s capability to write Python code, enabling automation and complex data processing within conversations.
- SQL Agent / Toolkit: Utilizes the bot to interact with SQL databases, execute queries, and manage database operations, streamlining data handling and analysis.
- Retrieval: Enhances the AI-powered chatbot’s ability to fetch and retrieve relevant information from various sources, improving the accuracy and relevance of responses.
PGVector
Overview: An open-source extension for PostgreSQL that facilitates the storage and querying of vector data.
Use Case: Particularly valuable in machine learning and artificial intelligence applications, where embeddings or vector representations of data are commonly used.
LLM Models and Servers
- OpenAI API: Provides access to powerful language models for generating human-like text, enabling advanced conversational capabilities, specifically using OpenAI’s GPT-4o (“Omni”)
- Ollama LLM Server and Models: Offers an alternative LLM hosting solution, ensuring flexibility and scalability in deploying language models. Notably, we are currently testing Llama 3 70b Q6.
- HuggingFace Models: A repository of pre-trained models that can be fine-tuned and integrated into various applications, enhancing the chatbot’s performance and adaptability.
Retrieval Augmented Generation (RAG): Text Embeddings examples
Retrieval-Augmented Generation (RAG) is a framework that leverages external knowledge sources to enhance the performance of language models. It integrates retrieval-based methods, which fetch relevant documents or data from a predefined repository, and creates natural language responses. In our examples, we embed documents into a PGVector database (our repository) and utilize similarity search as our retrieval-based method.
RAG: Simple Document
In this example, we’d embedded a simple txt file depicting a company’s password policy. You aren’t restricted to text files. You can embed documents from SharePoint and other document management systems.
Chatbot: Simple RAG with a Shakespearean twist.
Chatbot: Simple Document Retrieval
RAG: Improving the efficiency of the AI-Powered Chatbot when retrieving information from a database
We deployed LangChain as the framework and a LangChain SQL agent to retrieve information from databases. The agent is designed to query the database structure and preview the data for context. However, to streamline this process (using text-embeddings), we provided the database table structure (DDL) and sample SQL queries to optimize query generation. Thus, eliminating the chatbot’s need to perform database and data discovery.
We also leveraged the LLM’s built-in trained knowledge of Jira data as we utilize Jira Cloud to manage projects. In order for the bot to perform more effectively, we had replicated the Jira Cloud data into a on-premise Postgres database via Jira API.
An unintentional plus: for users who aren’t SQL savvy, they can copy the SQL queries generated by the chatbot for use in other tools too!
Chatbot: SQL Toolkit with RAG to optimise SQL query generation
AI-Powered Chatbot: SQL Toolkit
RAG: Integrating with a Qlik Cloud App
Integrating Qlik with a AI-Powered chatbot not only enhances data accessibility and usability but also provides robust security advantages. The use of Qlik’s Section Access Feature allows for precise control over data visibility, ensuring users only see what they are permitted to see. This centralized and secure approach surpasses the traditional database SQL Agent setup, offering a more streamlined, secure, and user-friendly data management solution.
Data Security:
- Qlik Section Access: This feature allows for row-level security, ensuring users only access the data they are authorized to see. Access control is granular and can be based on user roles, making it highly secure.
- SQL: While it can restrict access to tables and views, achieving the same level of row-level security requires complex configurations and can be less robust.
Centralized Data Management:
- Qlik: Centralizes data access and security management, reducing the risk of unauthorized access. It simplifies auditing and compliance tracking.
- SQL: Typically involves managing security at the database and application levels separately, increasing the potential for configuration errors.
User Authentication and Authorization:
- Qlik: Integrates seamlessly with enterprise identity management systems, supporting Single Sign-On (SSO) and multi-factor authentication (MFA).
- SQL: Supports similar integrations, but managing these across multiple systems can be more cumbersome and less streamlined.
Data Governance:
- Qlik: Facilitates better data governance through centralized policies and access controls, ensuring data integrity and compliance.
- SQL: While capable, achieving comprehensive data governance requires more effort and coordination across different systems.
The following illustrations depict how we integrated the chatbot with a Qlik App created to visualise data from our uninterruptible power supply. We will continue to build on this capability and take advantage of Qlik’s engine to visualise data instead of the chatbot drawing them using python.
Did you know that OpenAI’s GPT 4o is competent in Qlik?
Qlik Sense Uninterruptible Power Supply Monitoring App
QDI Workshop Dashboard
Chatbot: Qlik Integration
Chatbot Security: Understanding the Risks of Giving Your AI-Powered Chatbot Too Much Access
In the example provided, the chatbot has significant access to system commands, which can lead to potential security risks. The conversation shows the bot executing commands like ls -l, touch, and even attempts to install packages using brew. While this level of control can make the chatbot highly functional and versatile, it also opens up possibilities for misuse and unintended actions.
Here’s why giving a chatbot too much access can be dangerous:
- Unintended Deletion or Modification: The bot can delete files, as seen with the delete abc.txt command. This could lead to accidental loss of important data.
- Installation of Software: Allowing the bot to install software (brew install) can expose the system to unverified software, potentially compromising security.
- Network Commands: Running network-related commands like ping can be harmless in some contexts but might be exploited for network reconnaissance or denial-of-service attacks.
Key Takeaways:
- Limit Access: Ensure the bot has access only to the necessary commands to perform its tasks.
- Strict Permissions: Implement strict permission settings and monitor the bot’s activities.
- Security Audits: Regularly audit the bot’s interactions to detect and prevent any unauthorized actions.
In summary, while an AI chatbot with extensive access can be incredibly useful, it’s crucial to implement strong security measures to prevent any potential misuse. The screenshots below are some examples that can be classified as misuse.
Chatbot: File Security
Chatbot: System Security
Data Integration and Replication: Chatbot Alignment with SQL Implementation
For AI to be effective with SQL agents, a central database is essential. This ensures that all data is accessible, consistent, and up-to-date for accurate analysis and decision-making. SQL agents can efficiently query and retrieve information from a unified database, providing actionable insights. Data replication and integration pipelines are crucial here, seamlessly integrating and synchronizing data from various sources. Without this centralization and integration, data fragmentation and inconsistency can lead to erroneous outputs, reducing the reliability and efficacy of AI-driven processes.
