TruLens + Google Cloud Vertex AI Tutorial: Improve the customers support

🌐 TruLens + Google Cloud Vertex AI Tutorial: Crafting an Advanced Customer Support Chatbot

🚀 Introduction: Crafting Your Own AI Chatbot

Welcome, aspiring creators and curious minds! This tutorial is your gateway into the world of artificial intelligence as we embark on a project to build a customer support chatbot that's a cut above the rest. By tapping into the advanced technologies of TruLens and Google Vertex AI, we'll be piecing together a chatbot that's smart enough to not just answer questions, but also learn from its interactions to get better over time.

If you're new to AI or looking to hone your skills, you're in the right place. We'll cover everything from the basics of setting up your development environment to deploying a fully functional chatbot that can handle real-world customer queries. Get ready to roll up your sleeves and create an AI assistant that could redefine customer support as we know it.

🧠 Unlocking the Potential of TruLens

At the core of our chatbot is TruLens, a tool that brings transparency to the often opaque world of AI decision-making. It's like having a window into the brain of your chatbot, watching the cogs turn and understanding the thought process behind every decision it makes.

Dive into TruLens:

• Insightful Interpretability: 🕵️‍♂️ With TruLens, you'll learn why your chatbot says what it does, helping you trust and refine its judgments.

• Performance Insights: 📊 TruLens doesn't just tell you when your chatbot gets it right or wrong; it shows you metrics that matter, helping you pinpoint exactly where improvements are needed.

• Iterative Enhancement: 🔁 Improvement is a journey, not a destination. TruLens equips you with the feedback needed to make your chatbot better with every interaction.

As we step through this tutorial, remember that you're not just following instructions—you're learning to weave the fabric of intelligence into your chatbot, giving it the ability to engage, understand, and assist with a human touch. Let's get started on this exciting path to building your AI-powered chatbot! 🌟

🛠 Setting Up the Environment

Before we dive in, let's prepare our development environment. This involves installing Python, setting up a virtual environment, and installing necessary packages.

import os
import time
import streamlit as st
from dotenv import load_dotenv
from langchain.chains import LLMChain
from langchain.chat_models import ChatOpenAI, ChatVertexAI
from langchain.memory import ConversationBufferMemory
from langchain.prompts import PromptTemplate

Here, we're importing essential libraries. Streamlit creates our web app, while dotenv manages environment variables. LangChain integrates our chatbot with AI models.

🔑 Initializing the Chatbot: Setting Up Environment Variables

Acquiring the Necessary API Keys and Credentials

To set up your chatbot, you will need API keys and credentials from Google Cloud, OpenAI, and Huggingface. Here’s how to obtain them:

Google Cloud Credentials:

1. Create a Google Cloud Account: If you don't have one already, go to Google Cloud Platform and sign up.

2. Create a New Project: In the Google Cloud Console, create a new project for your chatbot.

3. Enable APIs: In your project, navigate to the API & Services dashboard and enable the APIs you plan to use (e.g., Google Cloud Vertex AI).

4. Create Credentials:

   • Go to the ‘Credentials’ page in the API & Services dashboard.
   • Click on ‘Create Credentials’ and select ‘Service account’.
   • Follow the steps to create a service account. Assign it the necessary roles (like ‘Editor’ or specific roles if you know what you need).
   • Once created, click on the service account and navigate to the ‘Keys’ tab.
   • Click on ‘Add Key’ and choose ‘Create new key’. Select JSON as the key type and click ‘Create’. This will download a JSON file containing your credentials.

5. Set Up Environment Variable:

   • Rename the JSON file for ease of reference (e.g., google-credentials.json).
   • Place it in a secure and accessible location in your project directory.
   • In your .env file, set the GOOGLE_APPLICATION_CREDENTIALS variable to the path of this JSON file.

   GOOGLE_APPLICATION_CREDENTIALS="path/to/your/google-credentials.json"

OpenAI API Key:

1. Register on OpenAI: Visit OpenAI's website and sign up or log in.

2. Access API Keys: Navigate to the API section in your account settings.

3. Generate a New API Key: Create a new key and copy it.

4. Update Your .env File: Add the OpenAI API key to your .env file.

   OPENAI_API_KEY="your-openai-api-key"

Huggingface API Key:

1. Sign Up/Log In on Huggingface: Go to the Huggingface website and create an account or log in.

2. Access Your Profile: In your profile, look for a section dedicated to API keys.

3. Generate/Retrieve Your API Key: Copy the API key provided.

4. Update Your .env File: Add this key to your .env file.

   HUGGINGFACE_API_KEY="your-huggingface-api-key"

📝 Code Explanation:

from trulens_eval import Feedback, Huggingface, Tru, TruChain
from trulens_eval.feedback.provider.hugs import Huggingface

# Load environment variables
load_dotenv()

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = os.getenv("GOOGLE_APPLICATION CREDENTIALS")
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")
os.environ["HUGGINGFACE_API_KEY"] = os.getenv("HUGGINGFACE_API_KEY")

In this section, we're loading and setting environment variables. These variables help securely manage API keys and credentials needed for interacting with Google Cloud, OpenAI, and Huggingface services. By setting these, we ensure our chatbot can authenticate and gain access to necessary AI services and tools.

By following these steps, you’ll have all the necessary credentials to initialize your chatbot environment and start leveraging the power of these advanced AI services! 🌐🔐

🤖 Chatbot Core Initialization: Deep Dive

In this crucial segment, we are laying the groundwork for our chatbot's brain, focusing on language processing and response generation.

Understanding the Components

Huggingface & ChatVertexAI

• Huggingface: This tool is essential for accessing advanced language processing functionalities. It provides access to pre-trained models capable of understanding and generating human-like text, a core feature for our chatbot.
• ChatVertexAI: Part of Google Cloud's Vertex AI offerings, this component is leveraged for its powerful machine learning capabilities. It efficiently processes natural language queries, making it ideal for handling complex customer support interactions.

Crafting the Chatbot's Prompt

• PromptTemplate: This is where we define the structure of the chatbot's conversation. The template guides how the chatbot interprets user inputs and formulates its responses.

  template = """
  You are a professional customer support specialist chatbot, dedicated to providing helpful, accurate, and polite responses. 
  Your goal is to assist users with their queries to the best of your ability. 
  If a user asks something outside of your knowledge, politely inform them that you 
  don't have the information they need and, if possible, suggest where they might find it. 
  Remember to always maintain a courteous and supportive tone.
  
  {chat_history}
  Human: {human_input}
  Chatbot:
  """
  

  • Defining the Chatbot's Persona: The template sets a professional and polite tone, defining the chatbot's interaction style.
  • Chat History Inclusion: The {chat_history} variable is crucial. It ensures the chatbot remembers the context of the conversation. This history allows the chatbot to provide coherent and relevant responses, crucial in customer support scenarios.
  • User Input Processing: {human_input} is where the chatbot receives the latest message from the user, forming the basis for its next response.

Constructing the Conversational Logic

• LLMChain: This is the backbone of our chatbot's conversational logic.

  memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)
  conversation = LLMChain(llm=llm, prompt=prompt, verbose=True, memory=memory)
  

  • Conversation Memory: ConversationBufferMemory retains the chat history. This memory ensures the chatbot can access and utilize previous parts of the conversation for contextually rich responses.
  • Chain Creation: The LLMChain binds everything together. It uses the language model (llm), the crafted prompt (prompt), and the conversation memory to create a fluid, interactive chatbot. The verbose flag is set for detailed logging, which can be helpful during development and debugging.

🎨 Crafting a Conversational Masterpiece

By integrating these elements, we are effectively sculpting the brain of our chatbot. It is not just programmed to respond but to understand, remember, and engage in a manner that mimics human conversation, making it an invaluable tool in customer support.

With this setup, our chatbot is primed to deliver a nuanced and context-aware user experience, a key to effective and satisfying customer interactions. 🌟🗨

📈 Integrating TruLens for Feedback and Monitoring

In this section, we integrate TruLens into our chatbot to enable advanced feedback and monitoring capabilities. Let's break down each component and its role in enhancing the chatbot's functionality.

Key Components of TruLens Integration

1. Huggingface Integration:

   • hugs = Huggingface(): This initializes an instance of Huggingface, a powerful tool that provides various natural language processing (NLP) functionalities. Huggingface models are known for their efficiency in language understanding and generation, making them ideal for enhancing chatbot interactions.

2. Feedback Mechanisms:

   • f_lang_match = Feedback(hugs.language_match).on_input_output(): This feedback mechanism utilizes the Huggingface provider to check the language match between the user's input and the chatbot's output. Ensuring language consistency is key to maintaining a coherent conversation flow.
   • feedback_nontoxic = Feedback(huggingface_provider.not_toxic).on_output(): This ensures that the chatbot's responses are free from toxic content. Maintaining a safe and respectful communication environment is crucial, especially in customer support scenarios.
   • f_pii_detection = Feedback(hugs.pii_detection).on_input(): Protecting user privacy is paramount. This feedback mechanism detects any Personally Identifiable Information (PII) in the user's input, helping to prevent the chatbot from inadvertently storing or processing sensitive data.
   • feedback_positive = Feedback(huggingface_provider.positive_sentiment).on_output(): This monitors the chatbot's output for positive sentiment, which is beneficial for maintaining a friendly and positive interaction with users.

3. TruLens Chain Recorder:

   • chain_recorder = TruChain(...): The TruChain here is a crucial component. It records the entire interaction along with the feedback received from the above mechanisms. By logging these interactions, TruChain provides valuable insights into how the chatbot is performing and where it can be improved. This continuous recording and analysis are vital for iterative development, allowing developers to fine-tune the chatbot based on real-world interactions and feedback.

hugs = Huggingface()
tru = Tru()

f_lang_match = Feedback(hugs.language_match).on_input_output()
feedback_nontoxic = Feedback(huggingface_provider.not_toxic).on_output()
f_pii_detection = Feedback(hugs.pii_detection).on_input()
feedback_positive = Feedback(huggingface_provider.positive_sentiment).on_output()

# TruLens chain recorder
chain_recorder = TruChain(
    conversation,
    app_id="contextual-chatbot",
    feedbacks=[f_lang_match, feedback_nontoxic, f_pii_detection, feedback_positive],
)

The Impact of TruLens on Chatbot Development

Integrating TruLens significantly elevates the chatbot's capabilities. It transforms the chatbot from a simple question-answering tool to a sophisticated interactive system capable of understanding context, maintaining a respectful and safe conversation environment, and continuously improving based on real-world interactions.

This integration is particularly important for customer support scenarios where understanding the user's needs, maintaining a positive tone, and ensuring user data privacy are key to delivering high-quality service.

Through TruLens, our chatbot becomes more than just a responder; it becomes an intelligent, adaptive, and sensitive communicator, ready to handle the complexities of real-world customer interactions. 🌐👥🔍

💻 Creating the Streamlit Interface

st.title("Contextual Chatbot")

if "messages" not in st.session_state:
    st.session_state.messages = []

# Display chat messages
for message in st.session_state.messages:
    with st.chat_message(message["role"]):
        st.markdown(message["content"])

user_prompt = st.chat_input("What is up?")
if user_prompt:
    # Add user message to chat history
    st.session_state.messages.append({"role": "user", "content": user_prompt})
    with st.chat_message("user"):
        st.markdown(user_prompt)

    # Assistant response
    with st.chat_message("assistant"):
        message_placeholder = st.empty()
        full_response = ""

        # Construct the input for the conversation
        input_dict = {
            "chat_history": "\n".join([f'{msg["role"]}: {msg["content"]}' for msg in st.session_state.messages]),
            "human_input": user_prompt
        }

        # Record conversation with TruLens
        try:
            with chain_recorder as recording:
                response = conversation(input_dict)
                assistant_response = response["chat_history"][1].content.strip()
        except Exception as e:
            assistant_response = f"An error occurred: {e}"
            st.error("Error in generating response. Please try again.")

        # Simulate stream of response
        for chunk in assistant_response.split():
            full_response += chunk + " "
            time.sleep(0.05)
            message_placeholder.markdown(full_response + "▌")
        message_placeholder.markdown(full_response)

        # Add assistant response to chat history
        st.session_state.messages.append(
            {"role": "assistant", "content": full_response}
        )

This code block utilizes Streamlit to create a user-friendly interface. It displays chat messages and manages user interactions with the chatbot.

Run the streamlit app

streamlit run app.py

Certainly! Here's how the section incorporating the image and its caption will look in the Markdown format:

💻 Creating the Streamlit Interface

Our chatbot's interface is powered by Streamlit, which enables the creation of an engaging and interactive UI. Below is a glimpse of the chatbot in action, demonstrating its capability to handle user queries with contextually aware responses.

🚀 Running the TruLens Dashboard

tru.run_dashboard()

Lastly, we launch the TruLens dashboard. It provides a visual interface to monitor and analyze the chatbot's interactions and performance.

🚀 Running the TruLens Dashboard

Post-interaction, the chatbot's performance can be monitored and analyzed through the TruLens dashboard. This dashboard provides a suite of analytics that allows us to track the effectiveness of the chatbot and make data-driven improvements.

1. App Leaderboard

2. Evaluations

2. View the record hash

3. View the FeedBack progress

🎉 Congratulations! You've now built an AI-driven customer support chatbot equipped with advanced capabilities from TruLens and Google Vertex AI. This bot is ready to transform customer interactions with its learning and adaptive abilities. Enjoy your journey into AI! 🌐

🏁 Conclusion

Congratulations! 🎉 You have now built a sophisticated AI-driven customer support chatbot, equipped with the power of Google Vertex AI and TruLens. This bot is ready to transform customer interactions with its advanced capabilities and continuous learning.

This tutorial is crafted to guide you through each step in detail, ensuring a thorough understanding of building and deploying a cutting-edge AI chatbot. Enjoy your journey into the world of AI! 🌟