The easiest way to get data from ANY site in minutes
Summary
TLDRIn this tutorial, Mike introduces 'Browse AI', a user-friendly tool that simplifies web scraping without coding. It features robots for crawling pages and a low-code UI, along with integrations for seamless data transfer to CRMs or databases. Browse AI offers pre-built templates for popular sites like YouTube, Yelp, and LinkedIn, and also allows custom scraper creation. Mike demonstrates how to extract government contracts from sam.gov and set up integrations with Google Sheets, showcasing the tool's efficiency in gathering and organizing data.
Takeaways
- π Browse AI is an AI-powered tool designed for easy data extraction and monitoring from any website.
- π€ The platform uses robots that can crawl web pages through a user-friendly, low-code UI.
- π Browse AI offers numerous integrations with CRMs and databases, allowing seamless data transfer.
- π Pre-built templates are available for popular websites like YouTube, Yelp, Google, LinkedIn, and more, simplifying data scraping.
- π οΈ Users can also create custom robot scrapers to extract specific information and organize it into CSV or Excel files.
- π The tool includes a feature to navigate to the next page and capture more data beyond the initial page view.
- π Browse AI can be integrated with Google Sheets, automatically updating the spreadsheet with scraped data.
- π A bulk run feature allows users to run multiple URLs at once, significantly speeding up the scraping process.
- π The monitor tab enables users to set up automatic, recurring scraping tasks at specified intervals.
- π Users can choose from a wide range of pre-built robots for various services, making web scraping accessible to non-technical users.
- π Data can be downloaded as CSV or JSON, and can also be sent directly to Google Sheets for ongoing updates.
Q & A
What is the main purpose of Browse AI as described in the script?
-Browse AI is a tool designed to scrape and monitor data from any website easily and quickly, using a low-code UI and pre-built templates for popular websites.
Who is the speaker in the video, and what is his focus?
-The speaker is Mike, who focuses on discussing AI and automation to help save time and make more money in businesses.
What are the key features of Browse AI that the speaker highlights?
-The key features highlighted are the use of robots for web crawling, a user-friendly low-code UI, integrations with various CRMs and databases, and pre-built templates for scraping data from popular websites.
How does Browse AI make web scraping accessible to non-technical users?
-Browse AI makes web scraping accessible by providing a simple interface where users can click on elements they want to scrape without needing to write any code.
What types of integrations does Browse AI support?
-Browse AI supports integrations with various systems such as CRMs, databases, and services like Google Sheets, allowing users to pass the scraped data through their preferred platforms.
Can Browse AI be used to scrape data from government contracting websites?
-Yes, the script demonstrates using Browse AI to scrape government contracts from sam.gov, a U.S. Government Contracting site.
How does Browse AI handle pagination to scrape more than one page of results?
-Browse AI allows users to set up navigation to the next page, enabling it to scrape data from multiple pages and compile it into a single dataset.
What is the process of creating a custom robot scraper in Browse AI?
-The process involves installing the Browse AI Chrome extension, granting permissions, recording actions on the website, selecting elements to scrape, naming variables, and configuring the robot with a name and search parameters.
How can users utilize Browse AI to scrape data from job listing websites like Indeed?
-Users can use pre-built templates in Browse AI to input job titles, locations, and the number of job listings they want to scrape, and the tool will extract the relevant data.
What is the 'Bulk Run' feature in Browse AI, and how does it save time?
-The 'Bulk Run' feature allows users to run a robot on multiple URLs at once by inputting a list of URLs and limits in a CSV file, which significantly speeds up the scraping process for multiple pages or websites.
How can users stay updated with the most recent data from websites they are interested in?
-Users can set up a monitor in Browse AI to automatically run the scraper at set intervals, ensuring they receive the most up-to-date information by receiving emails with the latest results.
Outlines
π€ Introduction to Browse AI for Web Scraping
The video introduces a powerful AI tool called Browse AI, which simplifies the process of extracting data from websites. Mike, the presenter, highlights the tool's ability to transform a website's data into a more organized format. He emphasizes the ease of use, especially for non-coders, and mentions the availability of pre-built templates for popular websites like YouTube, Yelp, and LinkedIn. The video also covers the integration capabilities of Browse AI with CRMs and databases, and encourages viewers to follow along by clicking a link in the description.
π Demonstrating Browse AI's Web Scraping Capabilities
Mike demonstrates how to use Browse AI to scrape data from sam.gov, a US Government Contracting site. He guides viewers through the process of setting up a scraping task, including installing the Browse AI Chrome extension and granting necessary permissions. The tutorial covers capturing lists and text from web pages, and how to save and name variables for the scraped data. Mike also shows how to navigate through pagination to extract more data and configure the scraping robot. The results are then displayed, showing how Browse AI can automatically organize the scraped data into a structured format.
π Advanced Features of Browse AI: Bulk Scraping and Integrations
The video continues with Mike showcasing the advanced features of Browse AI, such as bulk scraping and integrations. He explains how to use a CSV file to input multiple URLs and limits for scraping, which can significantly save time. Mike also demonstrates how to set up a workflow with Google Sheets to automatically update the spreadsheet with the scraped data. Additionally, he introduces the 'monitor' feature, which allows for automatic and recurring scraping at set intervals, ensuring up-to-date information.
π Exploring Browse AI's Pre-built Templates and Future Scraping Plans
Mike explores Browse AI's pre-built templates for various websites, such as Expedia and Indeed, showing how easy it is to extract specific data like hotel listings or job postings. He emphasizes the tool's user-friendly interface and the value it provides, especially for those without coding skills. The video concludes with a teaser for another tool, 'scrape table', which is designed for scraping Google Maps data. Mike invites viewers to check out that video and encourages them to share their scraping experiences with Browse AI in the comments.
Mindmap
Keywords
π‘Browse AI
π‘Web Scraping
π‘Templates
π‘Low-code UI
π‘Integration
π‘Sam.gov
π‘Custom Robots
π‘CSV/Excel
π‘Pagination
π‘Automation
Highlights
Introduction to Browse AI, an AI-powered tool for easy data extraction from websites.
Browse AI's user-friendly low-code UI and robot feature for web scraping.
Integration options with various CRMs and databases for data management post-extraction.
Pre-built templates available for popular websites like YouTube, Yelp, and LinkedIn for quick data scraping.
Demonstration of building custom robot scrapers for specific data needs.
Tutorial on scraping sam.gov, a US Government Contracting site, using Browse AI.
Explanation of how to install and use the Browse AI Chrome extension for web scraping.
Step-by-step guide on capturing lists and text from a webpage using Browse AI.
How to navigate and scrape multiple pages for extended data collection.
Setting up a workflow with Google Sheets to automatically update with scraped data.
Utilizing Browse AI's bulk feature for running multiple URLs at once.
Accessing and using pre-built robots for various websites without coding.
Example of extracting hotel listings from Expedia using a pre-built Browse AI template.
Setting up a monitor for automatic reoccurring scraping at set intervals.
Extraction of job listings from Indeed using Browse AI's pre-built template.
Downloading and exporting scraped data in CSV or JSON formats.
Final thoughts on the simplicity and effectiveness of Browse AI for non-coders.
Transcripts
this is by far one of the easiest ways
to extract any data from any website in
minutes if you're looking to turn a
website from this to this well you've
come to the right place I'm going to
show you a AI power tool to do just that
what's going on guys I'm Mike I talk
about all things Ai and automation to
help you save time and make more money
inside of your business let's get into
this AI tool to allow us to scrape any
website in a matter of minutes so the
tool that I want to showcase today is
called browse Ai and just as it says in
the landing page it is one of the
easiest ways to extract and monitor data
from any website and the basis of browse
AI is they basically have these robots
that allow us to crawl any page in a
very easy to use low code UI which I
love you can see they have a ton of
different Integrations which we can use
so that when we scrape our data from our
website we can then pass it through
whatever kind of CRM or database that
we're working with and also one of the
reasons why I love browse AI is because
they have a bunch of pre-built templates
that we're going to be checking out to
get all kinds of really cool data from a
bunch of different popular websites we
have everything from YouTube to Yelp
zappier Google LinkedIn here glass store
I know indeed around here somewhere Tik
Tok eBay there's a ton of great
pre-built templates in here for us to
extract information in a matter of
seconds but I'm also going to show you
how we can build custom robot scrapers
to get any kind of information and have
it in a NE organized CSV or Excel file
if you want to follow along in this
tutorial you can click the link in the
description below to take you to browse
AI you sign up with that link it does
help out the channel this is not
sponsored by browse AI I just really
like the tool and I think for a lot of
you non-coders out there you're really
going to appreciate how simple this is
so you can click the link down below to
get started and let's hop right into
browse AI so I've logged into browse Ai
and right at the start ready to start
putting in our URL to extract all of the
data that we're looking for you see we
have our little robot guy in the middle
here basically the robot that's going to
control our web scraping and allow us to
get whatever kind of information that
we're looking for so let's find a site
to scrape and today we're going to be
scraping sam.gov if you don't know what
sam.gov is it's basically a US
Government Contracting site which hosts
a ton of free government contracts in a
database for people to bid and put
quotes on there's a ton of really cool
and interesting information on here so
you can take some time to poke around
with it but what we're going to do is
head on over to the search tab here and
we're just going to search for any
specific keyword here to get some
results I'm going to type in the word
buildings and you can see here we
already got a list of some contracts
here for expanding a parking structure a
courtyard upgrade toilet Replacements
lodging spaces Civil Works there's all
kinds of really interesting stuff on
here so if you've ever been interested
in Government Contracting this is a
great place to start but this is also
going to be a great website for us to
learn how browse AI works so we can
scrape some government contracts and put
it in a nice organized CSV and Excel
file so I'm going to take this URL from
browse AI with the buildings search
query already inside of it we're going
to head back over to browse Ai and we're
going to pop this right in here if it's
your first time using browse AI it's
going to prompt you to install the
browse AI Chrome extension so make sure
you also install the browse AI Chrome
extension cuz this is what's going to
allow us to build out our Scraper on
their platform so I'm going to add this
to my browser it'll also ask us to Grant
some permissions in order to record our
actions just got to click on this and
click ow and then same thing we're going
to need to allow recording in incognito
mode so we're just going to want to open
our settings here and then I'm on Brave
so it's this allow and private tab here
but I'm pretty sure on Chrome it's allow
and incognito mode so you're just going
to switch that to be on and we should be
all good to go we have our URL in the
origin URL we just click on start
training this robot and it pops up a
brand new tab with the same URL we put
in of sam.gov I'll click on okay here
and now you'll notice at the top right
we have our browse AI guy right here who
is going to help us scrape this website
so if we give him a click you'll notice
we have a couple different options here
we can either capture a list which is
exactly how it looks in the picture
there where allows us to select items
that are similar in their structure on a
website or we can use capture text that
will allow us to just select a specific
part of the page this is going to be
good for things like product pages on
like a Shopify store there's also
capturing screenshots of a page so if
you want to capture a screenshot this is
how you do it but for today on sam.gov
we're going to be using the capture list
option right here and all we have to do
is we're going to want to hover over the
lists that we're looking for to scrape
so as I hover over the sections here
browse AI uses the different sections in
the page to determine the different
lists that we want to scrape and if I
hover kind of right here you'll notice
we actually get all of the different
list items that we're looking to scrap
so we want to get these 25 government
contracts on sam.gov and all the
information inside each of the boxes so
I'm going to kind of go over here and
give this a click and you'll notice we
get all of the different contracts on
this page so now we can just pick the
information we want from each of the
lists so I'm going to hover on over the
title of this contract here and if I get
it lined up right I can click on it and
there's two options we have here you can
save two different variables from this
one particular text here so we can
capture the visible text which I will do
and then we're also going to want to go
back over it again that didn't work
we're going to back over it one more
time and then we're going to capture the
link so this will capture two different
variables we have the text from the
title and also the link from the title
and let's also capture the notice ID and
then we'll also capture this description
here we'll capture the visible text for
the Department agency we're just going
to capture the visible text for this one
same thing with the sub tier just the
visible text we're also going to get the
office information the current date
offers are due the notice type here the
updated date we're we're just going to
grab the capture visible text and also
the publish date and now once we have
all the information from our one
contract you'll notice it grabs the same
information for all of the different
listings on this page and then we just
hit enter and after that it's going to
ask us to name each of these variables
so when it scrapes this page it'll put
them in a row according to these headers
so for this we're just going to name
this title and click on enter and this
is for the link so I'm just going to
name this link we have the notice ID so
I'm just going to name this ID we have
the description here I'll name this
department for the Department agency as
well as the sub tier office for the
office information I'll name this offer
due for the due date of the offer I'll
name this one type for the notice type
updated date for the update date and the
publish date will be the published date
there we go and now we have all of our
variables and you'll see as soon as we
get the last one entered in we have all
of our information here from our page
and what's great too is if even if it
doesn't exist it'll leave it blank which
is nice because there'll be no
formatting issues if for some reason the
listing doesn't have that information
already autop populated in and then from
here we got do a couple more things
we're going to name this list so I'm
going to name this Sam contracts we're
going to want to choose the amount of
rows that we want to extract for this
I'm going to extract a custom number of
actually what we're going to do is this
let's click on please select the page
ination type and we're going to click on
click to navigate to the next page and
what this will do is allow us to go to
the bottom here and once we get to the
last 25 we can tell it to click on this
button right here and if I move my
camera this will allow us to capture
capture more than the 25 rows of
information that are on this page so
instead of putting 25 in here let's put
like 40 cuz 40 will mean that we have to
at least go to the next page in order to
grab the additional 15 that are on the
next page because there's only 25 rows
per page and you'll notice once we have
all this populated in we get a new
button here capture list click on this
and we just click on finish recording
and it'll upload our brand new robot now
all we got to do is just configure it by
giving it a name that's fine and we'll
give it a search this is going to take a
couple minutes but it's basically going
to do the first initial run through of
our robot so I'll come back to you when
that's done awesome and a couple seconds
later we get our information so you can
see here we have Sam contracts 40
results and we have all of the
information from the pages this only
shows the first 10 but if you click on
see all 40 items we get all 40 results
and what's great too is it even goes to
the next page through that pentation
feature that we've clicked on and goes
to the next one to get all of the
information for us sweet so then from
here all we have to do just head down
here and click on yes looks good if
there's any issues with it you can
always retrain it or delete it but we'll
click on yes and now our robot is pretty
much all good to go from here we can do
a lot of different steps to integrate
this bot into whatever kind of workflow
we want we can click on the tables here
to see tables of our past searches and
also the integrate tab here is another
thing you're going to want to be looking
at there's a ton of great Integrations
here that we can use to send our data
once we run our robot inside of brows
I'm going to set up a workflow with
Google Sheets but you can obviously use
any of the ones that are listed here
I'll enable syncing with Google Sheets
I'll log into my account I'll create a
new spreadsheet and I'm going to name
this Sam contracts and then I'll just
click on create spreadsheet and activate
integration so now what's going to
happen is every time we run this
workflow it's going to put the
information we get from the contracts
and put it inside of our Google sheet so
let's give this a shot now if we head
back to run task here you'll notice we
have options to change our origin URL as
well as our cont attack limit so I'm
going to head back over to sam.gov and
instead of putting in buildings here
let's remove this keyboard and let's put
in something like food so there's a ton
of contracts here on chicken eggs HP
freezer refrigerators yeah a lot of
stuff here you can look through but
we're going to take this URL here we're
going to stick this inside of our origin
URL and then we're going to change our
contact limits to whatever we're looking
for let's say we want to scrape like 30
instead of 40 this time and then click
on run a task while this is running I
want to show you guys the Run task bulk
feature you want do more than one URL at
a time this feature is going to save you
a ton you click on this bulk run tests
here I would recommend downloading the
sample CSV input here and it will kind
of show you the format of what it's
looking for but basically if you just
make a CSV file with all your URLs and
then the limit for each of those URLs in
a list here you can then take this Excel
file and you could just pop it right
into here and it would go through and
run a robot for every single one of
those rows with a URL and a limit which
is amazing this is save me a ton of time
when I'm doing bulk scraping all right I
had to rerun it cuz it wasn't working
the first time so I got 40 you can see
here we got our brand new list of the
food contracts like I was saying earlier
and if we take a look at our Google
Sheets you'll notice we have a new Sam
contracts tab right here with all the
data we got from browse AI isn't that
awesome it just automatically gets
posted right in here once it's done
scraping but I want to show you some of
the other templates that I mentioned in
the beginning of this video that will
allow you to just hop right on here and
start playing around and scraping all
kinds of websites in minutes so if we
head back over to the dashboard here we
can choose from browse pre-built robots
right here which will allow us to select
from all the robots in the pre-built
robots page and there are a ton of good
ones in here we have all kinds of
services from Airbnb Amazon Chrome
Fiverr glass store Google indeed
LinkedIn loopet monster Reddit product
hunt Tik Tock Trip Advisor y commentator
YouTube Zillow Zoom info so let's try a
couple of these ones out personally I to
try this Expedia one out here it's kind
of peing my interest so we have the
extract hotels list from Expedia so we
can just click on use this automation so
it will basically copy this template of
a robot and put it inside of our account
so from here all we have to do is just
input our hotel list from Expedia I'm
looking to go to the Florida Keys pretty
soon so I'm going to copy this search
result just for Florida keys for the
next couple days here I'll head back
over to the robot page I'll put in my
URL and then we can change however many
hotels we want I'm going to go with 20
hotels and then we'll click on next step
it'll show us the configuration and just
start extracting in my experience I've
had a lot less hiccups and issues just
by using the pre-built templates that
browse AI has already made and it's
honestly where I see where most people
are going to get value out of browse AI
especially for people who are non Tech
and don't know how to code and don't
know how to build web scrapers is a very
friendly and intuitive way to make web
scraping accessible to pretty much
anyone and there we go we got our 20
search results right here from xedia you
can see all 20 items another thing too
that I forgot to mention while we were
scraping Sam is this monitor tab here we
can actually create a monitor for this
scraper to automatically run the Scraper
on a set interval so for instance if you
want to scrape a product page so you
always have the most upto-date price you
would use this so it constantly scrapes
the website to grab that pricing
information you can have it run once on
any day You' like in your time zone on
the specific URL with however many
results you want and you can just save
it right here and then it will run on
that interval every day and send you an
email with the results it's pretty
straightforward and intuitive once again
if you're looking to get reoccurring
information or the most up-to-date
information on any kind of website that
you're scraping this is how you would
set that schedule there before we go
let's look at one more pre-built
template if I'm looking to extract stuff
from indeed for instance I'm going to
extract job listings from indeed I'm
going to click on this one here and
let's try this one out we'll use this
Automation and this one it looks like we
don't even need a link we could just put
in our job title location and our amount
of jobs so I'm going to go for
automation expert this will be
interesting and let's put it as like
remote and let's see if we can get like
30 jobs this will be interesting I
haven't tried this out yet and here we
go looks like we get design engineer
senior full stack okay so I guess it's
just like coding which makes sense and
we get our 30 items for that search
result we have the indeed link when it's
posted little bit of a description here
location the company and the position we
can download this data if we want either
as a CSV or Json and there we go we got
all of our ND data right here or as we
saw earlier you can use the integration
here and you can set it up to a Google
sheet and you can just make a brand new
Google sheet with the indeed data so
whenever it gets ran it sends the data
that we made over to Google Sheets this
is going to be one of the simplest and
easiest ways to scrape any website in a
matter of minutes what I love about
browse AI is there's no coding at all
all it is just click on what you want
and then hit the button to go and then
you get a nice layout of all the
information that you looking for once
again I'll have a link down description
below for you to get started using
browse Ai and also let me know down in
the comments what websites are you going
to be scraping with browse AI but if
you're looking for Google Maps data
specifically how about you check out
this video here where I showcase my app
scrape table which allows you to scrape
unlimited Google Maps data for free in a
matter of seconds if you haven't seen
that video it's an absolute Banger so
make sure you go check it out and I'll
see you over there
Browse More Related Video
![](https://i.ytimg.com/vi/duMP3vNIaMQ/hq720.jpg)
LinkedIn Data Scraping Tutorial | 1-Click To Save to Sheets
![](https://i.ytimg.com/vi/h6g_3NbcQNw/hq720.jpg)
Webinar: Get Started with Browse AI (May 30, 2024)
![](https://i.ytimg.com/vi/qBi7AShxFyM/hq720.jpg)
Crea Agenti AI no code π€― - Zapier Central guida italiano
![](https://i.ytimg.com/vi/WGDhPwdErIc/hq720.jpg)
Become a Notion pro in 20min! (Full Notion Guide)
![](https://i.ytimg.com/vi/he-H-Qwes14/hq720.jpg)
Scrape website data without code using Bardeen
![](https://i.ytimg.com/vi/H9uEUKffQSo/hq720.jpg)
Steal This Ai-Powered Content System Right Now!
5.0 / 5 (0 votes)