Scrape website data without code using Bardeen
Summary
TLDRIn this informative tutorial, the speaker introduces Bardeen, a powerful and free Chrome extension designed to automate repetitive tasks, particularly useful for data scraping. The tool is highlighted for its ability to streamline workflows by copying and pasting or triggering automated processes. The demonstration showcases how Bardeen can extract data from a property search website, Rightmove, to create a spreadsheet without needing to build an integration. The step-by-step guide covers setting up a scraper, creating an auto book for right-click triggers, and finally, integrating the scraped data into Google Sheets. The video concludes by encouraging viewers to explore Bardeen's potential for market research and data analysis.
Takeaways
- 🕵️ The speaker spends a lot of time scraping data from websites and prefers using APIs or other methods when available.
- 🔧 Bardeen is a free Chrome extension that automates repetitive tasks and is particularly useful for data scraping and workflow setups.
- 🏠 The speaker uses Bardeen to scrape data from e-commerce platforms and courier services to streamline order creation and shipment tracking.
- 🔗 Bardeen can extract specific data from a webpage, such as property details, by identifying HTML elements and creating a scraper template.
- 📈 The speaker demonstrates using Bardeen on Rightmove, a property search website, to extract information from property listings.
- 📝 Bardeen allows users to create an 'auto book' with triggers and actions, enabling automated data scraping when certain conditions are met.
- 🔗 The auto book can be set to scrape data in the background when a user right-clicks on a link, without loading the page.
- 📋 Data scraped by Bardeen can be saved directly to Google Sheets, streamlining the process of data collection and analysis.
- 🛠 The speaker emphasizes the flexibility of Bardeen, suggesting it can be used for various tasks beyond the examples provided.
- 📊 Bardeen is recommended for those conducting market research or needing to pull large datasets into a spreadsheet for analysis.
Q & A
What is the main purpose of the tool Bardeen mentioned in the script?
-Bardeen is a free Chrome extension designed to automate repetitive manual tasks, particularly useful for data scraping and setting up triggered workflows on websites.
Why might someone prefer using an API over scraping data from websites?
-Using an API is often preferred because it provides a more structured and reliable way to access data from websites, avoiding the need for manual scraping which can be time-consuming and less efficient.
How does Bardeen help with scraping data from e-commerce platforms?
-Bardeen can be used to scrape data from e-commerce platforms to create orders in one's system without the need to build a custom integration, streamlining the process.
What is an example of a website where Bardeen can be used to extract information?
-The script mentions using Bardeen on Rightmove, a property search website, to extract information from standard property pages.
What specific data points can Bardeen help extract from a webpage?
-Bardeen can help extract specific data points such as address, price, property type, number of bedrooms, number of bathrooms, property size, and tenure.
How does Bardeen identify specific HTML elements to scrape?
-Bardeen identifies specific HTML elements by allowing the user to click on them and assign a title, which helps in customizing the data extraction process.
What additional feature did the speaker find useful to add to Bardeen's data extraction?
-The speaker found it useful to add the URL and timestamp to the data extraction, providing a direct link to the property and indicating when the data was scraped.
How can Bardeen's auto book feature be triggered?
-The auto book feature in Bardeen can be triggered by a user action, such as a right-click on a link, which initiates the scraping process.
What is the process of saving scraped data to Google Sheets using Bardeen?
-After setting up the scraper and auto book in Bardeen, the user can add an action to save the scraped data to a Google Sheet by specifying the sheet and how the data should be added.
How can Bardeen help in analyzing large datasets for market research?
-Bardeen can help by automating the process of pulling data into a spreadsheet, allowing for quicker analysis of large datasets, which is particularly useful for market research.
Outlines
🤖 Introduction to Bardeen Tool for Web Scraping
The speaker introduces Bardeen, a free Chrome extension that automates repetitive tasks, particularly useful for web scraping. They express their preference for APIs but acknowledge the necessity of scraping in certain situations. The tool is highlighted for its ability to streamline the process of copying and pasting data or setting up triggered workflows. The speaker shares their experience using Bardeen with partner platforms to scrape e-commerce data for order creation and to pull courier data for shipment tracking. The tutorial begins with a demonstration on the Rightmove property search website, aiming to extract property information efficiently.
🔍 Setting Up Bardeen for Property Data Scraping
The speaker demonstrates how to set up Bardeen for scraping property data from Rightmove. They guide through the process of initiating the app, creating a new scraper template, and identifying specific HTML elements to extract data such as address, price, property type, number of bedrooms and bathrooms, size, tenure, and a special field for the URL and timestamp. The focus is on creating a single page scraper tailored to gather the desired information from property listings. The setup includes an auto book feature triggered by a right-click action, which is intended to scrape data from the selected property page in the background.
📊 Automating Data Extraction and Integration with Google Sheets
The speaker continues by detailing the automation process of data extraction using Bardeen and integrating it with Google Sheets. They explain creating an action that triggers upon a right-click, which then scrapes the data from the property page in the background. The data is then saved to a spreadsheet, streamlining the process of compiling property listings. The tutorial includes connecting Bardeen with Google Sheets, setting up the action to add rows to the spreadsheet, and testing the workflow to ensure data is correctly pulled and organized. The speaker concludes by encouraging viewers to explore Bardeen's capabilities for their own data scraping and market research needs, highlighting its utility beyond the demonstrated use case.
Mindmap
Keywords
💡Scraping data
💡API
💡Bardeen
💡E-commerce platforms
💡Courier data
💡Rightmove
💡Scraper template
💡Auto bot
💡Google Sheets
💡Integration
💡Workflow automation
Highlights
Scraping data from websites can be time-consuming and frustrating without an API.
Bardeen is a free Chrome extension for automating repetitive tasks, especially for data scraping.
The speaker uses Bardeen to scrape data from e-commerce platforms for order creation.
Bardeen is also used to pull courier data to keep track of shipments.
A demonstration of using Bardeen on Rightmove, a property search website, is provided.
Bardeen allows for the creation of a new scraper template for specific tasks.
HTML elements on a webpage can be identified and customized for data extraction.
Including the URL and timestamp in the data scrape can be useful for tracking property listings.
Autobooks in Bardeen can be triggered by specific actions, like a right-click on a link.
Creating actions within Bardeen to scrape data in the background without loading the page.
Testing the Bardeen setup to ensure the data is being scraped correctly.
Integrating Bardeen with Google Sheets to automatically save scraped data.
The process of connecting Bardeen with Google Sheets for the first time requires agreeing to terms.
Using Bardeen to add rows to a Google Sheet with data from the scraper.
The speaker emphasizes the potential of Bardeen for market research and data analysis.
Bardeen can automate the process of pulling large datasets into a spreadsheet for analysis.
The speaker invites viewers to share their creative uses of Bardeen in the comments.
Transcripts
I spend an awful lot of time
scraping data from websites there I've
said it I'm not happy about it I wish
there were better ways to do it and
often when there's an API or any other
way for me to engage in a website I'm
going to use that instead but sometimes
it's not avoidable and sometimes it's
not worth the time this is where
something like bardeen the tool I'm
going to be talking about today is
extremely useful bardeen is a free
Chrome extension that packs a lot of
power it lets you automate repetitive
manual tasks and it's especially useful
for copying and pasting or setting up
triggered workflows obviously this video
isn't sponsored but I have left a link
in the description so you can check it
out for yourself now I've been using
this recently with some of our partner
platforms to scrape data from their
e-commerce platforms so that we can
create orders in our system without
having to build an integration I've also
used it to pull Courier data so that we
can keep an update on all of our
shipments that are happening at any time
today though I'm going to use this on
right move which is a property search
website I'm Keen to get some information
out of their standard property page and
a tool like bardeen is really really
great for so let's waste no more time
and I will jump into the platform and
show you how it works
okay so here we are on the right move
site you'll see there's a nice big old
search box here when I went to
University I lived in Leamington Spa so
that seems like a great place for us to
start so let's just have a really quick
look I mean Tom no I mean Tom Spa in
Warwickshire I'm gonna look at houses
for sale and I'm just going to use
whatever standard setting they have I
just want to get a list of links and
have a go so I've done my search now for
for lemington spa I've got 360 results I
don't want all of them what I want to do
is just make sure that we get the
scraping set up properly
in a way that's the most useful so the
way I'm going to start this is by
clicking one of these links and taking a
look at the page so loads of pictures
which is great we've got these key
features here which may be a useful the
property description I'm not too fast
around but there's definitely some
things that I want to make sure that we
have for our spreadsheet but yeah let's
let's have a look and see what data we
can pull out so all you need to do to
start up the the app itself is click
here on your extension list and that's
going to come up with this splash screen
for now all I want to do is play around
with the scraper and then we can worry
about linking that to Google Sheets a
little bit later so I'm going to come in
here and click on new scraper template
that's going to offer either a list
table or single page but for now we're
on a single page so I'm going to grab
this I'm going to call this right move
single page
start building so this bit down at the
bottom is going to offer me some
information from the table but what
we're going to do first off is just have
a look around the page and see what we
can click on see what information we can
actually put out so you'll see here it's
actually identifying specific HTML
elements that we want to look at all you
have to do is click on it and give it to
feel the title I'm going to call this
address we're going to add here which is
the Bliss price
property type
number of bedrooms
the number of bathrooms
and the size
tenure
pretty sure they're all free health but
I don't really know enough to say that
with any certainty Harvey from very
slightly in the future here uh one of
the things that we didn't add here which
is obviously very useful to add is the
URL you'll see down here and special
field URL time Etc generally it's quite
nice to add that so I am going to get a
page link which is the URL and I am
actually going to have a look and see
about maybe just also grabbing time
scrapes the only reason I'm going to
grab this is just because obviously
we're looking at properties and if
something's really old the chances are
that it's probably gone so I'm just
going to call that timestamp those are
now added at the bottom here and it's
just going to give us a little bit more
information and make it a little bit
easier if we did want to adjust the
scraper in the future to go back and add
some more information
foreign
[Music]
scraper set up here you can see it in
this drop down for all of our scrapers
but it's not very useful on its own so
we need to combine it with a couple of
other features within bardeen to make it
really really useful and actually do
what we need it to do so without further
Ado I'm going to start an auto boot so
when you create an auto book it's going
to ask you first for a trigger the
trigger that we're using is when
something happens because we want it to
happen as soon as I right click on one
of the links so you'll see some of the
options here that you can use which
again is pretty cool when an email is
starred that you can set a specific
series of events that might be something
that we would use to any starred emails
become projects in Asana so any anything
immediately gets taken into Osama and
then we can use that to manage our
products the things I found it most
useful for are right clicks that's why
I'm sticking with that so you can just
search here for right right even when I
right click on a website it's going to
ask you for a name so I'm going to call
this save listing so when I right click
that's what it's going to come up with
and it asks you where it should be
visible on and you can say it's only
visible on specific bits of the site so
if you only wanted images or any other
Page Body on this case we only actually
want on links because we want to use the
link as the page that should be scraped
rather than the page that we're already
on so now we've got this first piece of
that flow here what I can then do is
click on this plus button and that will
allow me to create an action to happen
as a result of what we've just done so
I'm going to click new action here and
now we are going to screen we're given
three options for what we want to scrape
here I'm going to scrape data in the
background because we don't actually
want to load the page we just want to
run through the list right click on each
one and scrape it directly from that
page and then save that to a sheet so to
create data in the background it's going
to ask us which page should be scraped
now you'll see here we have Trigger
action from our previous step so when I
right click on a website I want to take
that URL and scrape that page in
particular then it's going to ask us
which scraper template and we have the
template that we created earlier right
now single page just one can select that
it asks for the number of list items to
be extracted but actually that doesn't
really matter for what we're doing
because it's not a table scraping setup
it's actually just a single page so I'm
going to ignore that and I'm going to
click that as done so now that we've got
all the data here we can click done save
it and test that it's working so I'm
going to call this RM data extractor
I'm going to click on Save and that's
going to bring me back to this main page
I can activate this now which is exactly
what I'm going to do so I'm going to
right click on left click here even this
has now been scheduled and it will now
run automatically on the page so it can
hard bardeen and we can come back to our
property here and because this is all a
link I can right click anywhere I now
have this wonderful piece here that says
save listing if I click on that that's
going to run in the background I'm not
going to see anything happen but what I
can then do is reopen bardine and have a
look at the activity now I can see
called width here and the returns data
and I can see all of the data that I
wanted fantastic news has been pulled
out really pleased that that's working
we'll get on now to the final part which
is pulling that data from the scraper
into our spreadsheet
I'm just bringing back up the bardeen
workflow again we've got my auto book
that we created earlier which I've just
called RM now it is running it's getting
the data out and we're quite happy with
it but we want to take it a step further
and add it to this right move scraping
data spreadsheet if you've not connected
the two platforms together before you
will have to
um agree to a couple of terms just to
allow bardeen to talk with Google Sheets
it's the same with any any of the other
apps so I've got that set up already so
I'm going to come into more options
and open in Builder again we've got our
lovely two steps here and we're going to
add another action at the end here we've
got the two sections here from what
we've created already I'm going to click
on the add action once more and open up
another action I've already connected
Google Sheets so I can just type in
sheet and add rows to Google sheet I
think that option will still appear but
it will just take you through the sign
in and set up flow if you've not done it
before so I'm going to add row to Google
sheet and it's going to ask me which
Google sheet I've got my sheet ready to
go
which is a right move that's going to
bring it up fantastic and then here it's
going to ask for how we want the data to
come in
now it says from table you can add
column and or you can use commands and
if I come to use commands it's going to
give me some options including using the
data from action one to add directly to
the table so I'm just going to click on
that and that's actually all we need to
do I'm going to click on done and click
on done up here because we now have
everything we need to activate this and
start collating data
so I can close this out auto book now
entirely and I can run through here and
I can say oh this one looks great I'm
just going to grab this save listing
and then we can take a look and see if
it actually saves it
and there we go all of that data is now
coming in and we have set up the The
Columns because it was the first time
we've done it
but just to show you how this would work
then now that we've got that data in
here I can come down I can see this one
looks really nice and
don't like this one but I like this one
I'm not buying any of these houses these
are very expensive houses
and I have no money
so I've right clicked a load of times
now and I'm just going to come back into
here so here are the listings in the
sheet as I want it I have gone back
afterwards and added the URL in to the
same setup that I had before because I
want to be grabbing that as well and
keeping that every time that I pull any
any information in my link I think
there's some really cool stuff that you
can do with this way beyond what I've
done today but I would really recommend
giving it a try especially if you're
pulling data or doing market research it
can be really helpful to pull everything
into a spreadsheet and then allow you to
much more quickly analyze a large data
set I hope you found this interesting
and I can't wait to see what you decide
to do with bardeen do let me know in the
comments if you get anything cool
working I'd love to explore some more
opportunities to use that software
関連動画をさらに表示
LinkedIn Data Scraping Tutorial | 1-Click To Save to Sheets
Web Scraping Tutorial | Data Scraping from Websites to Excel | Web Scraper Chorme Extension
Automate PDF Invoices Data Transfer to Google Sheets with ChatGPT & Zapier | Tutorial
Crea Agenti AI no code 🤯 - Zapier Central guida italiano
Automate Your Real Estate Business With AI & Zapier
Twitter OSiNT (Ethical Hacking)
5.0 / 5 (0 votes)