How to Use Llama 3 with PandasAI and Ollama Locally

Summary

This video script introduces viewers to the integration of Llama 3, a large language model, with PandasAI and Ollama for local data analysis. It guides through setting up a virtual environment, installing necessary open-source tools, and building an interactive app with Streamlit. The app allows users to explore and analyze the Titanic dataset using natural language prompts, demonstrating the power of generative AI in data manipulation and visualization.

Takeaways

• 🔍 Large models have huge potential for data analysis, and this tutorial covers using Llama 3 with PandasAI and Ollama.
• 🛠️ PandasAI is a Python tool that allows you to explore, clean, and analyze data using generative AI, making data interaction easier.
• 💻 Ollama helps you run large language models (LLMs) like Llama 3 locally, without needing an API key.
• 📂 The app created in this tutorial uses the Titanic dataset to demonstrate how to interact with data using PandasAI.
• 🛠️ The tutorial walks through setting up a virtual environment with conda, installing necessary tools like pandas, pandasai, and streamlit, and initializing Llama 3 with Ollama.
• 🔗 The app is built using Streamlit, a popular tool for creating web apps with Python.
• 📊 The app allows users to upload a CSV file, view the first rows of data, and interact with the dataset using natural language prompts.
• 🤖 The SmartDataframe class in PandasAI is used to convert the dataset into a format that can be queried with natural language prompts.
• 📈 The tutorial demonstrates various data queries and visualizations, including bar charts, pie charts, histograms, and heatmaps, generated by interacting with the dataset using Llama 3.
• 🔍 The key message is that good prompts lead to good outputs, and the combination of PandasAI, Llama 3, and Streamlit makes data exploration intuitive and powerful.

Q & A

What are the tools mentioned in the script for data analysis with large models?

The tools mentioned are PandasAI, Ollama, and Streamlit. PandasAI is a smart version of pandas for data exploration, cleaning, and analysis using generative AI. Ollama helps run large models like Llama 3 locally. Streamlit is used for building the app interface.

Why is Pandas AI considered a smart version of pandas?

Pandas AI is considered a smart version of pandas because it allows users to explore, clean, and analyze data using generative AI, enabling conversational interaction with the data.

How does Ollama assist in working with large models locally?

Ollama assists by allowing users to run open-source Large Language Models (LLMs) like Llama 3 locally on their computers, which would otherwise be difficult to manage.

What is the purpose of the app created in the script?

The app is created to demonstrate the power of Llama-3 by allowing users to chat with a dataset, specifically the Titanic dataset, and get responses based on their prompts.

What is the first step in setting up the environment for the app as described in the script?

The first step is to create a virtual environment using conda and naming it 'genai', followed by activating this environment.

How are the required tools for the app installed in the script?

The required tools are installed by creating a 'requirements.txt' file listing the necessary libraries such as pandas, pandasai, and streamlit, and then using pip to install them with the command 'pip install -r requirements.txt'.

What is the role of the LocalLLM class in the script?

The LocalLLM class is used to instantiate an LLM object that connects to Ollama and specifies the model to be used, facilitating the interaction with the Llama 3 model for data analysis.

How is the user input for chatting with the dataset collected in the app?

User input is collected through a text area created in the Streamlit app, where users can enter their prompts to interact with the dataset.

What is the significance of the 'spinner' method used in the app?

The 'spinner' method is used to display a loading animation with the message 'Generating response...' to indicate that the app is processing the user's prompt before displaying the results.

How can users visualize data using the app created in the script?

Users can visualize data by entering prompts to plot various types of charts such as bar charts, pie charts, histograms, and heatmaps based on the dataset's columns.

What is the importance of a good prompt when using the app?

A good prompt is crucial because it directly affects the quality of the output. 'Garbage in, garbage out' applies here; clear and specific prompts will yield more accurate and useful responses.

Outlines

00:00

🤖 Introduction to Generative AI Tools

This paragraph introduces the potential of large models in data analysis and the tools that will be used in the video: Llama 3, PandasAI, and Ollama. The video aims to demonstrate the use of these open-source tools for local data analysis without the need for an API key. PandasAI is described as an intelligent version of the popular pandas library for data manipulation, allowing users to interact with their data using generative AI. Ollama is highlighted as a tool that facilitates the local running of large models, making it easier to work with them without cloud-based services. The video will guide viewers through creating an app that uses these tools to interact with the Titanic dataset, showcasing the power of Llama-3.

05:03

🛠️ Setting Up the Development Environment

The second paragraph details the process of setting up the development environment for the app. It starts with creating a virtual environment using conda and installing the necessary tools: pandas, pandasai, and streamlit, which are specified in a requirements.txt file. The tools are installed using pip. The paragraph also covers the initialization of Llama 3 with Ollama, which involves downloading and installing Ollama, starting the Ollama server, and installing the Llama 3 model. The video script then proceeds to show how to initialize the model in the app using the LocalLLM class from pandasai, connecting it to the Ollama server and specifying the model to be used.

10:05

📊 Building the Data Analysis App with Streamlit

In this paragraph, the focus shifts to building the actual data analysis app using Streamlit. The app is initialized with a title, and a file uploader widget is created to load CSV datasets. The uploaded file is then converted into a pandas dataframe, and the first few rows are displayed to verify the upload. The dataframe is further converted into a SmartDataframe using the SmartDataframe class from pandasai, which is configured to use the previously initialized Llama 3 model. The app includes a text area for user prompts and a 'Generate' button that triggers the chat method of the SmartDataframe to interact with the dataset based on the user's input. The paragraph concludes with examples of how to use the app to ask questions about the dataset and generate responses, demonstrating the ease of exploring data with the help of generative AI.

📈 Visualizing Data and Making Inferences with Llama-3

The final paragraph demonstrates how to use the app to make inferences and visualize data from the Titanic dataset. It shows how to ask the app questions about the dataset, such as the number of rows and columns, average age, and the number of people with more than three siblings. The paragraph also covers how to calculate the percentage of surviving passengers by gender. Furthermore, it illustrates the process of plotting various charts, including bar charts, pie charts, histograms, and heatmaps, to gain insights into the dataset's distribution and relationships. The video emphasizes the importance of using clear and effective prompts to achieve accurate and meaningful results, concluding with an invitation to subscribe, like, and comment on the video.

Keywords

💡Data Analysis

Data analysis is the process of inspecting, cleaning, transforming, and modeling data to extract useful information, inform decision-making, and generate new knowledge. In the context of the video, data analysis is the main theme, focusing on how to utilize tools like PandasAI and Ollama to perform this process with large datasets like the Titanic dataset, showcasing the power of AI in data exploration and manipulation.

💡Llama 3

Llama 3 is a large language model that is part of the video's discussion. It represents the third version of a generative AI model capable of understanding and generating human-like text. The video script mentions using Llama 3 with Ollama to run the model locally, emphasizing the ease of working with large models without the need for external APIs like OpenAI.

💡PandasAI

PandasAI is introduced as a smart version of the popular Python library pandas, which is widely used for data manipulation and analysis. The script describes PandasAI as a tool that allows users to explore, clean, and analyze data using generative AI, essentially enabling conversational interaction with the data, which is a key aspect of the video's demonstration.

💡Ollama

Ollama is a tool highlighted in the video that facilitates the local running of large language models (LLMs). It is presented as a solution for the difficulty of working with large models locally, making it easier for users to leverage the capabilities of models like Llama 3 directly from their computers, as illustrated by the video's step-by-step guide.

💡Virtual Environment

A virtual environment in the context of the video refers to an isolated space where developers can install and use specific versions of libraries and tools without affecting the system's global state. The script details the creation and activation of a virtual environment named 'genai' using conda, which is essential for setting up the project's dependencies.

💡Requirements.txt

In software development, a 'requirements.txt' file lists the dependencies required for a Python project. The video script includes creating this file to specify libraries such as pandas, pandasai, and streamlit, which are necessary for the app's functionality, and then installing them using pip, a package installer for Python.

💡Streamlit

Streamlit is an open-source Python library used for quickly creating and sharing custom web applications for data science. The video describes using Streamlit to build the app interface, allowing users to upload datasets, interact with the data through prompts, and visualize the results, demonstrating the ease of creating interactive apps with this tool.

💡SmartDataframe

SmartDataframe, as mentioned in the script, is a class from the PandasAI library that enhances the functionality of a traditional pandas DataFrame by integrating AI capabilities. It allows the dataset to be interacted with using natural language prompts, making complex data analysis tasks more accessible, as shown when the video's app uses it to chat with the dataset.

💡LocalLLM

LocalLLM is a class from the PandasAI library that represents a locally running large language model. The script explains how to instantiate a LocalLLM object and connect it to Ollama, specifying the model name and API base to ensure compatibility with the locally served Llama 3 model, which is crucial for the app's operation.

💡Data Visualization

Data visualization involves creating visual representations of data to analyze and communicate patterns and insights effectively. The video script includes using the app to generate various charts and plots, such as bar charts, pie charts, histograms, and heatmaps, to illustrate different aspects of the Titanic dataset, demonstrating the power of combining AI with visualization for data exploration.

💡Garbage In, Garbage Out (GIGO)

GIGO is a principle in data processing that states the output quality is dependent on the input quality. The script mentions this principle to emphasize the importance of crafting good prompts when using AI tools like PandasAI and Ollama, as poor input can lead to uninformative or incorrect outputs, highlighting the need for clear and thoughtful interaction with AI systems.

Highlights

Large models have significant potential for data analysis.

Introduction to using Llama 3 with PandasAI and Ollama for local model deployment without the need for an API key.

Pandas AI is a smart version of pandas that allows for generative AI-based data exploration, cleaning, and analysis.

Ollama is a tool that facilitates the local running of large models, making it easier to work with them without cloud-based services.

Demonstration of an app creation process using Llama-3, showcasing its capabilities in data interaction and analysis.

Step-by-step guide on setting up a virtual environment for the project using conda.

Instructions on installing necessary tools like pandas, pandasai, and streamlit using a requirements.txt file.

Explanation of how to initialize Llama 3 with Ollama for local model usage.

Creating a Python file for the app and setting up the environment for coding with VS Code.

Using Streamlit to build and run the app, with automatic browser opening for easy access.

Creating a file uploader widget in Streamlit for dataset loading.

Conversion of uploaded CSV file into a pandas dataframe for data manipulation.

Utilization of the SmartDataframe class from PandasAI for advanced data interaction.

Incorporation of user input through a text area and a generate button for dynamic data queries.

Implementation of a spinner for user feedback during data processing.

Interactive chat with the dataset using the chat method of the SmartDataframe class.

Inference examples demonstrating how to ask the model questions about the dataset and receive answers.

Visualization capabilities of the app, including bar charts, pie charts, histograms, and heatmaps for data analysis.

Emphasis on the importance of good prompts for accurate and meaningful data analysis outcomes.

Conclusion summarizing the tutorial on using PandasAI with Llama-3 and building an interactive data analysis app with Streamlit.

Transcripts

Large models have huge potential for data analysis.

Today, we'll cover how to use Llama 3 with PandasAI and Ollama.

We'll run this model locally, so we won't leverage any api key like OpenAI.

The tools we'll use in this video are completely open-source.

Let me briefly explain these tools.

When it comes to data manipulation, as you know, pandas is king.

You can think of Pandas AI as a smart version of pandas.

Pandas AI is nothing but a Python tool that allows you to explore, clean,

and analyze your data using generative AI.

This means that you can talk to your data using this tool.

Cool, right?

Another amazing tool in generative AI is Ollama.

As you know, it is difficult to work with large models locally.

This is where Ollama comes into play.

Ollama helps you run LLMs locally.

Awesome, right?

Here is the app we'll create.

First, we'll load our dataset.

The dataset we'll use is the Titanic dataset.

Here, we'll enter our prompt to chat with our dataset.

And then the app will return a response.

This is a simple app but great to see the power of Llama-3.

Let's take a look at the topics we'll handle.

First, we'll create a virtual environment and then install the tools we'll use.

Next, we'll initialize Llama 3 with Ollama.

After that, we'll load the dataset.

And then we'll build the app.

Lastly, we'll chat with the dataset using PandasAI.

Nice, we've seen the topics we'll cover.

Let's go ahead and start with the setup.

To write our codes, we're going to use the vs code editor.

Now, let's create a virtual environment.

To do this, we're going to use conda.

Let's write,

conda create -n let's name genai

Let me press enter.

I already created this.

After that, we're going to activate this environment.

conda activate genai

Here you go.

Our environment is ready to use.

The next step we're going to do is install the tools we'll use.

To do this, let's create a requirements.txt file.

We're going to click on the new file and then name it.

requirements.txt

Okay.

Let's write the libraries we'll use here.

To work with PandasAI, let's write, pandas, and pandasai.

Next, to build the app, we'll use streamlit.

Let's write streamlit.

Okay, these tools are enough to build the app.

We're going to now install these.

It's simple to do this with pip.

Let's type,

pip install -r requirements.txt

Let me press enter.

Here you go.

Our tools are ready to use.

To write our codes, we're going to create a Python file.

Let me click on the new file and then give a name.

Let's say, app.py

Yeah, our file is ready.

So far, we created a virtual environment and installed the tools.

Let's go ahead and initialize the model with Ollama.

When it comes to working with llms locally, Ollama is king.

This tool allows you to run open-source LLMs, such as Llama 3 and Mistral.

Trust me, it's very easy to use Ollama.

All you need to do is download it from the website and then install it on your computer.

After installing you can use Ollama in the terminal.

First, let's start it. Let's go to the terminal and then

Write, ollama serve

I already started ollama.

Ollama is ready to use.

You can use many large models with Ollama.

Let me show you these models.

Let's click the model.

There you go.

Here, you can choose any model you want.

It is a piece of cake to load a model with ollama.

To install a model, we're going to use the terminal.

Let me show you.

Let's say, we want to install llama 3.

ollama pull llama3

Let me press enter.

There you go.

As you can see, llama 3 is loaded.

You can see the installed models with the list command.

ollama list

There you go.

As you can see I loaded many models.

The model we'll use is the Llama 3 8B version.

Awesome, Ollama is ready to go.

What we're going to do is initialize the model with Ollama.

To do this, we're going to use the LocalLLM.

First, let's import this class.

from pandasai.llm.local_llm import LocalLLM

Okay.

What we're going to do now is instantiate an LLM object by specifying the model name.

To do this, we're going to use compatibility.

This helps the app to connect to Ollama.

Let write, model = LocalLLM()

Let's connect Ollama.

api_base="http://localhost:11434/v1"

Okay, next, let's specify the model.

model="llama3"

Nice, our model is ready.

Let's go ahead and initialize the app with streamlit.

First, let's import this tool.

import streamlit as st

Next, let's give the app a title.

st.title("Data analysis with PandasAI")

After that, let's run this app.

To do this, in the terminal let's write,

streamlit run app.py

Let me press enter.

Yeah, the app opens automatically in the browser.

This is simple, right?

You can see the title here.

Let's go ahead and create a widget to load the dataset.

To do this, we can use the file_uploader method.

Let's say, uploaded_file = st.file_uploader()

Let's give a text, "Upload a CSV file",

Next, let's specify the file type, type=['csv'])

Okay.

Nice, we created a widget.

Let's test this widget.

Go to the app and click rerun.

There you go.

To load the dataset, let's click over here.

And then select our dataset.

Yeah, the file is uploaded.

Easy peasy, right?

Let's go ahead and take a look at the first rows of the dataset.

To do this, let's use the if statement.

if uploaded_file is not None:

After that, let's convert data into the pandas dataframe.

First, we're going to import pandas.

import pandas as pd

Next, let's read the dataset.

data = pd.read_csv(uploaded_file)

Now, let's write the first rows.

st.write(data.head(3))

Let's test the app.

To do this, let's go to the app and click rerun.

There you go.

You can see the first rows of the Titanic dataset.

What we need to do now is convert the dataset into SmartDataframe.

To do this, we're going to use the SmartDataframe class.

Let's go back and write,

from pandasai import SmartDataframe

Next, let's get an object from this class.

df = SmartDataframe()

Let's pass our data,

Let's set the config parameter, let's pass {"llm": model}

Nice, our smart dataframe is ready.

Next, to get the input from the user, let's create a text area.

prompt = st.text_area("Enter your prompt:")

Nice, our text area is ready.

What we're going to do now is create a button.

For this, let's use the button method with the if statement.

if st.button("Generate"):

When pressing this button, we want to run the prompt.

if prompt:

If the prompt is true, let's display a message while running the code.

To do this, let's use the spinner method using the with keyword.

with st.spinner("Generating response..."):

Now all we need to do is return the response.

To do this, let's write, st.write()

Let's invoke the chat method for a response.

df.chat(prompt))

Awesome, our app is ready.

Now it's time to make inferences.

Let me go to the app and then click rerun.

There you go.

All we need to do is write a prompt to chat with the dataset.

Let's write,

How many rows and columns are in the dataset?

Let me click on the generate button.

Yeah, the app is up and running.

Voila!

We got the output by talking to the dataset.

Amazing, right?

Now, we can explore our dataset using this app.

Let's want to learn the average age.

What is the average age?

Let me click generate.

There you go.

The average age is about twenty-nine.

Let's go ahead and find out How many people have more than 3 siblings?

How many people have more than 3 siblings?

Let me press generate.

There you go.

There are 30 people with more than 3 siblings.

Let's move on and want to see how many people died and how many survived.

How many people died and how many survived?

Let me click generate.

There you go.

Here are the numbers of people who survived and died.

This was very easy to find out with Llama 3 right?

Let's go ahead and want to calculate the percentage of surviving passengers by gender.

Return the percentage of passengers by gender

Let me click generate.

Yeah, it's done.

The percentage of male passengers is 64.76% and

and for female passengers, the percentage is 35.24%.

As you can see, it's easy peasy to explore the dataset with Llama-3.

Let's go ahead and plot charts to understand the dataset.

Let's write,

Draw a bar chart of the sex column

Let me click generate.

There you go.

As you can see, men more than women. Okay.

Let's go ahead and want to plot a pie plot.

Plot a pie chart of the pclass column

Let me click generate.

It's done.

Here is the pie chart.

You can see percentages here.

Let's go ahead and want to visualize the distribution of the fare column.

Visualize the distribution of the fare column

Let me click generate.

There you go.

Here is the distribution of the fare column.

Now, we want to plot the histogram of the age column.

Draw the histogram of the age column.

Let me click generate.

There you go.

Here is the distribution of the age column.

Let's go ahead and plot a histogram of the fare by sex.

Draw the histogram of the fare column separated by the sex column.

Let me click generate.

There you go.

Here is the distribution of the fare column by sex.

You can plot any chart you want.

Lastly, let's want to plot the heatmap chart of numerical variables

Draw the heatmap of numerical variables.

Let me click generate.

There you go.

Here is the heatmap chart.

Note that garbage in, garbage out.

So good prompt means good output.

You may need to try a few prompts to find a good one.

Yeah, that's it.

In this video, we covered how to use PandasAI with Llama-3.

To show this, we built an app step-by-step with Streamlit.

Hope you enjoyed it.

Thanks for watching.

Don't forget to subscribe, like the video, and leave a comment.

See you in the next video.

Bye for now.