How Copilot for Microsoft 365 can work with your external data
Summary
TLDRThe script discusses two methods to safely integrate external data into Microsoft 365 Copilot for AI-generated responses. The first is using Microsoft Graph connectors to pre-index data with Semantic Index for Copilot, enhancing the relevance of AI responses. The second involves managed plugins for real-time API data retrieval and optional data writing back. Both methods ensure security and privacy, and can be used together. An example demonstrates resolving a DNS issue by combining a Jira plugin and an intranet Graph connector. The script also explains how Semantic Index works on SharePoint and individual user data to improve search and response accuracy. Finally, it guides on setting up connectors and plugins for external data integration.
Takeaways
- 🔌 Microsoft 365 Copilot can integrate external data sources to enhance AI-generated responses through two primary methods.
- 📚 The first method uses Microsoft Graph connectors to pre-index read-only data for retrieval with prompts, ensuring relevant AI responses.
- 🔧 The second method employs managed and trusted plugins for real-time API-based data retrieval, with optional data writing capabilities.
- 🛡️ Both options adhere to Microsoft's security, compliance, and privacy standards, respecting organizational data boundaries.
- 🔄 Demonstrated in the script, connectors and plugins can work together to respond to a single prompt, as shown with a help desk scenario.
- 🔎 Semantic Index for Copilot works by default with internal Microsoft 365 data, correlating activity signals to retrieve relevant information.
- 📈 Semantic Index operates on two levels: indexing information within SharePoint and creating a separate index for individual user documents and emails.
- 📝 The dual approach of Semantic Index increases the relevancy of AI-generated results, tailoring them to the user's approved access to information.
- 🔍 An example is given to compare SharePoint search results with and without Semantic Index, highlighting the improved search experience.
- 🔗 To utilize Semantic Index with external data, Graph connectors can be used without changes, or new connections can be set up through the Microsoft 365 Admin Center.
- 📜 Plugins offer a flexible way to retrieve data and perform actions without pre-indexing, with three types available: OpenAI plugins, Microsoft Teams message extensions, and Power Platform connectors.
- 🛠️ Plugins are developed using a shared code base with similar packaging and components, and can be managed through the Microsoft 365 Admin Center for access control.
Q & A
What are the two primary options to safely bring external information into Microsoft 365 Copilot experiences?
-The two primary options are using Microsoft Graph connectors to integrate data sources with Semantic Index for Copilot, which pre-indexes defined read-only information for retrieval, and using managed and trusted plugins such as OpenAI plugins, Microsoft Teams message extensions, and Power Platform connectors, which optimize for real-time API based data retrieval and can optionally write back data.
How does Semantic Index for Copilot work with Microsoft 365 Copilot?
-Semantic Index for Copilot works by correlating activity signals with the Microsoft Graph to retrieve indexed information uniquely relevant to the user. It presents a combination of the user's prompt and the retrieved information to the large language model to generate an informed response.
What is the purpose of using Microsoft Graph connectors?
-Microsoft Graph connectors are used to integrate external data sources with Semantic Index for Copilot, allowing for the pre-indexing of defined read-only information that can be used to drive the most relevant AI-generated responses.
What types of plugins can be used with Microsoft 365 Copilot?
-The types of plugins that can be used with Microsoft 365 Copilot include OpenAI plugins, Microsoft Teams message extensions, and Power Platform connectors.
How do plugins differ from Microsoft Graph connectors in terms of data retrieval?
-Plugins optimize for real-time API-based data retrieval without pre-indexing information, and they can also perform additional actions such as writing back data, depending on user permissions.
Can both Microsoft Graph connectors and plugins work together when responding to a single prompt?
-Yes, both Microsoft Graph connectors and plugins can work together when responding to a single prompt, allowing for a more comprehensive and integrated response.
What is an example of how a help desk technician might use Microsoft 365 Copilot?
-A help desk technician might use Microsoft 365 Copilot to respond to users facing an issue with DNS by submitting a prompt to find who's impacted and to write an email with instructions that reference an internal knowledge base article.
How does the Semantic Index improve the search experience across Microsoft 365?
-Semantic Index improves the search experience by mapping semantic signals such as people relationships and recent activities surrounding the content the user has access to, helping to determine the intent of the search and formulate a more relevant answer.
What is the process for setting up a connection to external data using Microsoft Graph connectors?
-To set up a connection to external data using Microsoft Graph connectors, you need to enable the service or start a trial from the Microsoft 365 Admin Center, and then add a connection. You can define the site, who can search the data source, and whether connector results are shown in search.
How can developers make plugins available in the Microsoft 365 environment?
-Developers can make plugins available in the Microsoft 365 environment by choosing from existing plugins in the store or by uploading their own plugin packages directly in the Teams Admin Center from the Microsoft 365 Admin Center.
What is the role of the YAML file in the context of plugins?
-The YAML file defines the API for connecting to the external data or service, as well as any actions it can take on behalf of the user. It also includes natural language descriptions of the API operations and responses, which are used by the LLM to assess what the plugin can do and help respond to a prompt.
Outlines
🔗 Integrating External Data with Microsoft 365 Copilot
The paragraph discusses two primary methods for safely incorporating external data into Microsoft 365 Copilot to enhance AI-generated responses. The first method involves using Microsoft Graph connectors to integrate data sources with Semantic Index for Copilot, which pre-indexes information for retrieval. The second method utilizes managed and trusted plugins, such as OpenAI plugins, Microsoft Teams message extensions, and Power Platform connectors, which optimize for real-time API-based data retrieval and can optionally write back data. Both options adhere to Microsoft's security, compliance, and privacy standards, and can work together in response to a single prompt. An example is provided where a help desk technician uses both a connector and a plugin to address a DNS issue, demonstrating how Copilot generates a response by pulling information from these sources and how plugins can present adaptive cards for more information and allow for actions like editing a Jira issue directly within Teams.
🛠️ Setting Up External Data Integration with Microsoft 365 Admin Center
This paragraph provides a step-by-step guide on setting up external data integration for Microsoft 365 Copilot. It starts by explaining that existing Graph connectors will work without changes, but if not set up, they need to be enabled from the Microsoft 365 Admin Center. The process involves adding a connection to external data sources, such as enterprise websites, with options to define the site, search permissions, and visibility in search results. Advanced settings allow for more granular configurations like meta tags and semantic property labels. Once configured, integrated results can be seen using SharePoint search. The paragraph then moves on to discuss plugins, explaining how they work by retrieving information in real-time without pre-indexing. It demonstrates a plugin's use by showing how Copilot uses it to find VPN-related tickets in a Jira system. The paragraph also covers the technical aspects of plugins, including their code structure, the use of YAML files for API definitions, and how they can be made available in the Microsoft 365 environment through the Admin Center. The summary concludes with a mention of where to learn more about these integration options.
Mindmap
Keywords
💡Microsoft 365
💡Microsoft 365 Copilot
💡Microsoft Graph connectors
💡Semantic Index
💡Plugins
💡OpenAI plugins
💡Microsoft Teams message extensions
💡Power Platform connectors
💡Jira
💡Adaptive card
Highlights
Microsoft 365 Copilot can integrate external data sources for AI-generated responses.
Two primary options for integrating external data: Microsoft Graph connectors and managed plugins.
Microsoft Graph connectors pre-index information for retrieval with prompts.
Plugins optimize for real-time API-based data retrieval and can write back data.
Both options adhere to Microsoft's security compliance and privacy standards.
Demonstration of using a connector and a plugin simultaneously in Microsoft Teams.
Semantic Index for Copilot correlates activity signals to retrieve relevant information.
Semantic Index works on SharePoint and individual user data for increased relevancy.
Comparison of SharePoint search results with and without Semantic Index.
How to use Semantic Index for external data with Graph connectors in Microsoft 365.
Plugins allow for direct real-time data retrieval and user permission-based actions.
Types of plugins: OpenAI plugins, Microsoft Teams message extensions, and Power Platform connectors.
Plugins use a shared code base and open API spec for cross-platform functionality.
Plugins can be made available in the Microsoft 365 environment through the Admin Center.
Control over who can access plugins and their underlying data is maintained by the admin.
Resource for more information on Microsoft 365 Copilot extensibility.
Transcripts
(bright sound)
- A lot of the information you work
with today might sit outside of Microsoft 365 locations.
So how can you safely bring that information
into your Microsoft 365 Copilot experiences
to ground generative AI results?
There are two primary options.
First, you can use Microsoft Graph connectors
to integrate the data sources you want
with Semantic Index for Copilot.
This option pre-indexes defined read-only information used
for retrieval with your prompts
to drive the most relevant AI generated responses.
The second option uses managed and trusted plugins
which can be a few different types,
including OpenAI plugins,
Microsoft Teams message extensions,
and Power Platform connectors.
And with plugins instead of pre-indexing information,
they optimize for real-time API based data retrieval
and they can optionally even write back data.
Both options follow Microsoft's
comprehensive security compliance and privacy approach
while respecting the boundaries that you define
within your organization's data services.
And both types can even work together when responding
to a single prompt.
In fact, let me show you an example using a connector
and a plugin simultaneously.
Let's say I'm a help desk technician
in Microsoft Teams and I want to respond
to all users facing an issue with DNS.
I'll start by submitting a prompt
to help me find who's impacted
and to write an email
for me that I can send them with instructions
that reference our internal knowledge base article.
Copilot has taken my prompt
and is generating a response, in this case,
by pulling in information from both a plugin configured
for Jira which we're using for issue tracking,
as well as a Graph connector for an intranet site
which we're using as a knowledge base for our help desk.
And it responds with a matching DNS related ticket
along with the steps for flushing DNS
and even an email it authored that I can use to respond
to our user with the matching open ticket.
You'll also see at the bottom
of the response that both the plugin's Jira ticket
and the connector's intranet KB article are referenced.
Additionally, if I scroll up
and click on the Jira ticket details, you'll see
that my plugin also presents an adaptive card
to show more information
and it even allows me to take action.
I'll hit "Edit issue" and right from within Teams,
I can change the record in Jira, for example,
if I sent the email
to the user already and resolved the issue,
I can update the ticket status from "To Do," to "Done."
And by hitting "Save," it'll record that change
to our Jira system.
So we just saw how you can extend Microsoft 365 Copilot
to external data sources coming
into the Microsoft Graph using connectors
and two-way integration using plugins.
Next, let me show you what was going
on behind the scenes and how each one works.
The first option where we referenced our intranet article,
leveraged Semantic Index for Copilot
which works with Microsoft 365 Copilot
by default over internal Microsoft 365 data.
When you submit a prompt,
Copilot brings in the Microsoft Graph
with a Semantic Index to correlate activity signals,
retrieving indexed information uniquely relevant to you.
It presents the combination
of your prompt with what was retrieved
to the large language model
to generate an informed response.
Importantly, Semantic Index works on two levels.
By default, it indexes information
within SharePoint inside your organization's tenant.
And second, each user also has their own separate index
for email and their individual important documents
like files shared with them, where they've commented
or were at mentioned, and more.
This two-pronged approach is what increases relevancy,
allowing Microsoft 365 Copilot
to orchestrate AI generated results that are unique
to you as a user, again,
based on your approved access to information.
And a great way to demonstrate Semantic Index for real is
by showing two sets of search results in SharePoint,
one without leveraging the Semantic Index and one with it.
Here we're searching on Project Melbourne Shiproom.
Notice we have a typo.
On the left, without the Semantic Index,
you can see results are retrieved based
on keywords and ranking as well as last modified date
for relevance and a file summary for context
but it's not the specific information we're looking for.
Conversely, on the right with the Semantic Index,
instead of a list of links,
notice the informative answer right at the top.
You can see that the index has mapped semantic signals such
as people relationships
and recent activities surrounding the content
the user has access to,
to help determine the intent
of your search and formulate an answer.
So you can see how Semantic Index
improves the search experience across Microsoft 365
and that includes Microsoft 365 Copilot.
So how do you take advantage
of Semantic Index for your external data,
like we saw in our example?
First, if you already use Graph connectors in Microsoft 365,
they will work without any changes.
If not, before you connect to external data,
from the Microsoft 365 Admin Center,
you do need to enable the service
or start a trial and from there you can add a connection.
There are options for enterprise websites
and a few other common data sources listed here as well.
For example, if I open ServiceNow,
you'll see that there are three options
for ticketing, knowledge base, and catalog, but in my case,
I'll set up an enterprise website
and this will show me a few simple options
to define the site, who can search the data source,
and whether connector results are shown in search.
Then for even more options, I can go
into advanced settings for more granular configuration
with options to define meta tags, specific exclusions,
semantic property labels, and customizations to the schema.
And once you have a connection up and running,
you'll be able to see integrated results
using SharePoint search with the connected site.
In this case, I'll just perform a simple keyword search
for troubleshooting instructions and results are found
in SharePoint locations as well as from the connected site.
And here is the DNS article that was used as part
of the retrieval process to respond
to my prompt from before.
Beyond Microsoft Graph connectors with Semantic Index,
let's look at our second option
for data retrieval with Copilot, using plugins.
Again, starting with how they work.
So with a plugin configured, when we submitted the prompt,
Microsoft 365 Copilot leveraged that plugin
to retrieve information directly from its source,
again, in real time without pre-indexing the data.
It was also able to perform additional actions on behalf
of the user and with respect to their permissions,
like changing the status of the ticket.
This time, I'll demonstrate the plugin by itself
without connection to our knowledge base.
In Microsoft Teams, I'll write a new prompt
to look up VPN related tickets in our Jira system
and submit it.
Then Copilot responds using that information
and finds two tickets in the system related to VPN issues.
And as I mentioned, plugins work without pre-indexing
and there are three types.
These can be OpenAI plugins,
Microsoft Teams message extensions,
and Power Platform connectors.
Now I'll explain how plugins work
with a short tour of their code.
All three of these types of plugins are similar
with similar packaging and components.
This is an example of a Teams app manifest file
that uses NuGet Package Manager to find app packages
and their details.
You can see that it refers to a YAML file.
And in that file, is where the API is defined
for connecting to the external data or service
as well as any actions it can take on behalf of the user.
Also note the natural language descriptions
of the API get operation and its responses.
These are used by the LLM
to assess what the plugin can do
to help respond to a prompt.
And because these are portable components and can be used
for OpenAI, Teams, and Power Platform, as a developer,
you'll be able to maintain a single shared code base.
Each will leverage the same underlying open API spec.
Then to make plugins available
in your Microsoft 365 environment,
you'll do this from the Microsoft 365 Admin Center
in the Integrated Apps page.
Here you can choose from existing plugins
from the store, just like you would get other apps today.
You can also upload your own plugin packages directly
in the Teams Admin Center.
Either way you add the plugin, you're in control
of who can access the plugin and its underlying data.
So those are the two main options available to you
for safely bringing in data external to Microsoft 365
into your Microsoft 365 Copilot experiences
to ground generative AI results.
To learn more, check out aka.ms/CopilotExtensibility
and keep following Microsoft Mechanics
for the latest in tech updates, thanks for watching.
(bright sound)
Voir Plus de Vidéos Connexes
Transform productivity with AI experiences in Microsoft Fabric | OD24
Creating custom copilot with Copilot Studio based on your files in SharePoint
Apple's Surprise "AI" Punch!
How Microsoft Copilot for Security works
Realtime Powerful RAG Pipeline using Neo4j(Knowledge Graph Db) and Langchain #rag
Microsoft Academic- An Introduction
5.0 / 5 (0 votes)