Scrape website data without code using Bardeen

Summary

In this informative tutorial, the speaker introduces Bardeen, a powerful and free Chrome extension designed to automate repetitive tasks, particularly useful for data scraping. The tool is highlighted for its ability to streamline workflows by copying and pasting or triggering automated processes. The demonstration showcases how Bardeen can extract data from a property search website, Rightmove, to create a spreadsheet without needing to build an integration. The step-by-step guide covers setting up a scraper, creating an auto book for right-click triggers, and finally, integrating the scraped data into Google Sheets. The video concludes by encouraging viewers to explore Bardeen's potential for market research and data analysis.

Takeaways

• 🕵️ The speaker spends a lot of time scraping data from websites and prefers using APIs or other methods when available.
• 🔧 Bardeen is a free Chrome extension that automates repetitive tasks and is particularly useful for data scraping and workflow setups.
• 🏠 The speaker uses Bardeen to scrape data from e-commerce platforms and courier services to streamline order creation and shipment tracking.
• 🔗 Bardeen can extract specific data from a webpage, such as property details, by identifying HTML elements and creating a scraper template.
• 📈 The speaker demonstrates using Bardeen on Rightmove, a property search website, to extract information from property listings.
• 📝 Bardeen allows users to create an 'auto book' with triggers and actions, enabling automated data scraping when certain conditions are met.
• 🔗 The auto book can be set to scrape data in the background when a user right-clicks on a link, without loading the page.
• 📋 Data scraped by Bardeen can be saved directly to Google Sheets, streamlining the process of data collection and analysis.
• 🛠 The speaker emphasizes the flexibility of Bardeen, suggesting it can be used for various tasks beyond the examples provided.
• 📊 Bardeen is recommended for those conducting market research or needing to pull large datasets into a spreadsheet for analysis.

Q & A

What is the main purpose of the tool Bardeen mentioned in the script?

Bardeen is a free Chrome extension designed to automate repetitive manual tasks, particularly useful for data scraping and setting up triggered workflows on websites.

Why might someone prefer using an API over scraping data from websites?

Using an API is often preferred because it provides a more structured and reliable way to access data from websites, avoiding the need for manual scraping which can be time-consuming and less efficient.

How does Bardeen help with scraping data from e-commerce platforms?

Bardeen can be used to scrape data from e-commerce platforms to create orders in one's system without the need to build a custom integration, streamlining the process.

What is an example of a website where Bardeen can be used to extract information?

The script mentions using Bardeen on Rightmove, a property search website, to extract information from standard property pages.

What specific data points can Bardeen help extract from a webpage?

Bardeen can help extract specific data points such as address, price, property type, number of bedrooms, number of bathrooms, property size, and tenure.

How does Bardeen identify specific HTML elements to scrape?

Bardeen identifies specific HTML elements by allowing the user to click on them and assign a title, which helps in customizing the data extraction process.

What additional feature did the speaker find useful to add to Bardeen's data extraction?

The speaker found it useful to add the URL and timestamp to the data extraction, providing a direct link to the property and indicating when the data was scraped.

How can Bardeen's auto book feature be triggered?

The auto book feature in Bardeen can be triggered by a user action, such as a right-click on a link, which initiates the scraping process.

What is the process of saving scraped data to Google Sheets using Bardeen?

After setting up the scraper and auto book in Bardeen, the user can add an action to save the scraped data to a Google Sheet by specifying the sheet and how the data should be added.

How can Bardeen help in analyzing large datasets for market research?

Bardeen can help by automating the process of pulling data into a spreadsheet, allowing for quicker analysis of large datasets, which is particularly useful for market research.

Outlines

00:00

🤖 Introduction to Bardeen Tool for Web Scraping

The speaker introduces Bardeen, a free Chrome extension that automates repetitive tasks, particularly useful for web scraping. They express their preference for APIs but acknowledge the necessity of scraping in certain situations. The tool is highlighted for its ability to streamline the process of copying and pasting data or setting up triggered workflows. The speaker shares their experience using Bardeen with partner platforms to scrape e-commerce data for order creation and to pull courier data for shipment tracking. The tutorial begins with a demonstration on the Rightmove property search website, aiming to extract property information efficiently.

05:02

🔍 Setting Up Bardeen for Property Data Scraping

The speaker demonstrates how to set up Bardeen for scraping property data from Rightmove. They guide through the process of initiating the app, creating a new scraper template, and identifying specific HTML elements to extract data such as address, price, property type, number of bedrooms and bathrooms, size, tenure, and a special field for the URL and timestamp. The focus is on creating a single page scraper tailored to gather the desired information from property listings. The setup includes an auto book feature triggered by a right-click action, which is intended to scrape data from the selected property page in the background.

10:03

📊 Automating Data Extraction and Integration with Google Sheets

The speaker continues by detailing the automation process of data extraction using Bardeen and integrating it with Google Sheets. They explain creating an action that triggers upon a right-click, which then scrapes the data from the property page in the background. The data is then saved to a spreadsheet, streamlining the process of compiling property listings. The tutorial includes connecting Bardeen with Google Sheets, setting up the action to add rows to the spreadsheet, and testing the workflow to ensure data is correctly pulled and organized. The speaker concludes by encouraging viewers to explore Bardeen's capabilities for their own data scraping and market research needs, highlighting its utility beyond the demonstrated use case.

Keywords

💡Scraping data

Scraping data refers to the process of extracting information from websites programmatically. In the context of the video, the speaker is not satisfied with the manual process of scraping data from websites and is seeking more efficient methods. The term is central to the video's theme as it introduces the need for automation tools like Bardeen to streamline the data extraction process.

💡API

API stands for Application Programming Interface, which is a set of rules and protocols that allows different software applications to communicate with each other. The video mentions APIs as a preferred method for engaging with websites over manual scraping, indicating that APIs are more efficient and often provide a structured way to access data.

💡Bardeen

Bardeen is a free Chrome extension mentioned in the video that automates repetitive tasks, particularly useful for data scraping and workflow automation. It is highlighted as a solution to the problem of inefficient manual data scraping, showcasing its power in automating tasks on websites like Rightmove for property data extraction.

💡E-commerce platforms

E-commerce platforms are online systems that allow for the buying and selling of goods and services. The script refers to using Bardeen to scrape data from e-commerce platforms to create orders in their system, which exemplifies a practical application of Bardeen in automating data transfer for business processes.

💡Courier data

Courier data pertains to information related to the shipment and delivery of goods by courier services. The video script mentions using Bardeen to pull courier data to keep track of shipments, illustrating how the tool can be used to automate and update logistics information.

💡Rightmove

Rightmove is a UK-based property search website. The video script uses Rightmove as an example to demonstrate how Bardeen can be used to scrape data from property listings. This keyword is significant as it provides a specific context for showcasing Bardeen's capabilities.

💡Scraper template

A scraper template in the context of Bardeen is a pre-configured set of instructions that dictate how data should be extracted from a webpage. The video describes creating a 'Rightmove single page' scraper template to define the structure and data points to be scraped from Rightmove's property listings.

💡Auto bot

An auto bot in Bardeen is a feature that allows for the automation of tasks based on triggers. The video script discusses setting up an auto bot with a trigger for right-click actions on webpage links, which automates the process of scraping data from selected properties.

💡Google Sheets

Google Sheets is a web-based spreadsheet program that is part of Google's office suite. The video script explains how to use Bardeen to export scraped data directly into a Google Sheet, which is a key feature for organizing and analyzing the extracted data.

💡Integration

Integration in the context of the video refers to the process of combining different systems or applications to work together. The speaker mentions avoiding the need to build an integration by using Bardeen to scrape data from partner platforms, which simplifies the process of data transfer between systems.

💡Workflow automation

Workflow automation is the process of making a sequence of tasks or jobs within a business or system to be automated. The video demonstrates how Bardeen can be used to automate workflows, such as scraping property data and saving it to a spreadsheet, to increase efficiency and reduce manual labor.

Highlights

Scraping data from websites can be time-consuming and frustrating without an API.

Bardeen is a free Chrome extension for automating repetitive tasks, especially for data scraping.

The speaker uses Bardeen to scrape data from e-commerce platforms for order creation.

Bardeen is also used to pull courier data to keep track of shipments.

A demonstration of using Bardeen on Rightmove, a property search website, is provided.

Bardeen allows for the creation of a new scraper template for specific tasks.

HTML elements on a webpage can be identified and customized for data extraction.

Including the URL and timestamp in the data scrape can be useful for tracking property listings.

Autobooks in Bardeen can be triggered by specific actions, like a right-click on a link.

Creating actions within Bardeen to scrape data in the background without loading the page.

Testing the Bardeen setup to ensure the data is being scraped correctly.

Integrating Bardeen with Google Sheets to automatically save scraped data.

The process of connecting Bardeen with Google Sheets for the first time requires agreeing to terms.

Using Bardeen to add rows to a Google Sheet with data from the scraper.

The speaker emphasizes the potential of Bardeen for market research and data analysis.

Bardeen can automate the process of pulling large datasets into a spreadsheet for analysis.

The speaker invites viewers to share their creative uses of Bardeen in the comments.

Transcripts

I spend an awful lot of time

scraping data from websites there I've

said it I'm not happy about it I wish

there were better ways to do it and

often when there's an API or any other

way for me to engage in a website I'm

going to use that instead but sometimes

it's not avoidable and sometimes it's

not worth the time this is where

something like bardeen the tool I'm

going to be talking about today is

extremely useful bardeen is a free

Chrome extension that packs a lot of

power it lets you automate repetitive

manual tasks and it's especially useful

for copying and pasting or setting up

triggered workflows obviously this video

isn't sponsored but I have left a link

in the description so you can check it

out for yourself now I've been using

this recently with some of our partner

platforms to scrape data from their

e-commerce platforms so that we can

create orders in our system without

having to build an integration I've also

used it to pull Courier data so that we

can keep an update on all of our

shipments that are happening at any time

today though I'm going to use this on

right move which is a property search

website I'm Keen to get some information

out of their standard property page and

a tool like bardeen is really really

great for so let's waste no more time

and I will jump into the platform and

show you how it works

okay so here we are on the right move

site you'll see there's a nice big old

search box here when I went to

University I lived in Leamington Spa so

that seems like a great place for us to

start so let's just have a really quick

look I mean Tom no I mean Tom Spa in

Warwickshire I'm gonna look at houses

for sale and I'm just going to use

whatever standard setting they have I

just want to get a list of links and

have a go so I've done my search now for

for lemington spa I've got 360 results I

don't want all of them what I want to do

is just make sure that we get the

scraping set up properly

in a way that's the most useful so the

way I'm going to start this is by

clicking one of these links and taking a

look at the page so loads of pictures

which is great we've got these key

features here which may be a useful the

property description I'm not too fast

around but there's definitely some

things that I want to make sure that we

have for our spreadsheet but yeah let's

let's have a look and see what data we

can pull out so all you need to do to

start up the the app itself is click

here on your extension list and that's

going to come up with this splash screen

for now all I want to do is play around

with the scraper and then we can worry

about linking that to Google Sheets a

little bit later so I'm going to come in

here and click on new scraper template

that's going to offer either a list

table or single page but for now we're

on a single page so I'm going to grab

this I'm going to call this right move

single page

start building so this bit down at the

bottom is going to offer me some

information from the table but what

we're going to do first off is just have

a look around the page and see what we

can click on see what information we can

actually put out so you'll see here it's

actually identifying specific HTML

elements that we want to look at all you

have to do is click on it and give it to

feel the title I'm going to call this

address we're going to add here which is

the Bliss price

property type

number of bedrooms

the number of bathrooms

and the size

tenure

pretty sure they're all free health but

I don't really know enough to say that

with any certainty Harvey from very

slightly in the future here uh one of

the things that we didn't add here which

is obviously very useful to add is the

URL you'll see down here and special

field URL time Etc generally it's quite

nice to add that so I am going to get a

page link which is the URL and I am

actually going to have a look and see

about maybe just also grabbing time

scrapes the only reason I'm going to

grab this is just because obviously

we're looking at properties and if

something's really old the chances are

that it's probably gone so I'm just

going to call that timestamp those are

now added at the bottom here and it's

just going to give us a little bit more

information and make it a little bit

easier if we did want to adjust the

scraper in the future to go back and add

some more information

foreign

[Music]

scraper set up here you can see it in

this drop down for all of our scrapers

but it's not very useful on its own so

we need to combine it with a couple of

other features within bardeen to make it

really really useful and actually do

what we need it to do so without further

Ado I'm going to start an auto boot so

when you create an auto book it's going

to ask you first for a trigger the

trigger that we're using is when

something happens because we want it to

happen as soon as I right click on one

of the links so you'll see some of the

options here that you can use which

again is pretty cool when an email is

starred that you can set a specific

series of events that might be something

that we would use to any starred emails

become projects in Asana so any anything

immediately gets taken into Osama and

then we can use that to manage our

products the things I found it most

useful for are right clicks that's why

I'm sticking with that so you can just

search here for right right even when I

right click on a website it's going to

ask you for a name so I'm going to call

this save listing so when I right click

that's what it's going to come up with

and it asks you where it should be

visible on and you can say it's only

visible on specific bits of the site so

if you only wanted images or any other

Page Body on this case we only actually

want on links because we want to use the

link as the page that should be scraped

rather than the page that we're already

on so now we've got this first piece of

that flow here what I can then do is

click on this plus button and that will

allow me to create an action to happen

as a result of what we've just done so

I'm going to click new action here and

now we are going to screen we're given

three options for what we want to scrape

here I'm going to scrape data in the

background because we don't actually

want to load the page we just want to

run through the list right click on each

one and scrape it directly from that

page and then save that to a sheet so to

create data in the background it's going

to ask us which page should be scraped

now you'll see here we have Trigger

action from our previous step so when I

right click on a website I want to take

that URL and scrape that page in

particular then it's going to ask us

which scraper template and we have the

template that we created earlier right

now single page just one can select that

it asks for the number of list items to

be extracted but actually that doesn't

really matter for what we're doing

because it's not a table scraping setup

it's actually just a single page so I'm

going to ignore that and I'm going to

click that as done so now that we've got

all the data here we can click done save

it and test that it's working so I'm

going to call this RM data extractor

I'm going to click on Save and that's

going to bring me back to this main page

I can activate this now which is exactly

what I'm going to do so I'm going to

right click on left click here even this

has now been scheduled and it will now

run automatically on the page so it can

hard bardeen and we can come back to our

property here and because this is all a

link I can right click anywhere I now

have this wonderful piece here that says

save listing if I click on that that's

going to run in the background I'm not

going to see anything happen but what I

can then do is reopen bardine and have a

look at the activity now I can see

called width here and the returns data

and I can see all of the data that I

wanted fantastic news has been pulled

out really pleased that that's working

we'll get on now to the final part which

is pulling that data from the scraper

into our spreadsheet

I'm just bringing back up the bardeen

workflow again we've got my auto book

that we created earlier which I've just

called RM now it is running it's getting

the data out and we're quite happy with

it but we want to take it a step further

and add it to this right move scraping

data spreadsheet if you've not connected

the two platforms together before you

will have to

um agree to a couple of terms just to

allow bardeen to talk with Google Sheets

it's the same with any any of the other

apps so I've got that set up already so

I'm going to come into more options

and open in Builder again we've got our

lovely two steps here and we're going to

add another action at the end here we've

got the two sections here from what

we've created already I'm going to click

on the add action once more and open up

another action I've already connected

Google Sheets so I can just type in

sheet and add rows to Google sheet I

think that option will still appear but

it will just take you through the sign

in and set up flow if you've not done it

before so I'm going to add row to Google

sheet and it's going to ask me which

Google sheet I've got my sheet ready to

go

which is a right move that's going to

bring it up fantastic and then here it's

going to ask for how we want the data to

come in

now it says from table you can add

column and or you can use commands and

if I come to use commands it's going to

give me some options including using the

data from action one to add directly to

the table so I'm just going to click on

that and that's actually all we need to

do I'm going to click on done and click

on done up here because we now have

everything we need to activate this and

start collating data

so I can close this out auto book now

entirely and I can run through here and

I can say oh this one looks great I'm

just going to grab this save listing

and then we can take a look and see if

it actually saves it

and there we go all of that data is now

coming in and we have set up the The

Columns because it was the first time

we've done it

but just to show you how this would work

then now that we've got that data in

here I can come down I can see this one

looks really nice and

don't like this one but I like this one

I'm not buying any of these houses these

are very expensive houses

and I have no money

so I've right clicked a load of times

now and I'm just going to come back into

here so here are the listings in the

sheet as I want it I have gone back

afterwards and added the URL in to the

same setup that I had before because I

want to be grabbing that as well and

keeping that every time that I pull any

any information in my link I think

there's some really cool stuff that you

can do with this way beyond what I've

done today but I would really recommend

giving it a try especially if you're

pulling data or doing market research it

can be really helpful to pull everything

into a spreadsheet and then allow you to

much more quickly analyze a large data

set I hope you found this interesting

and I can't wait to see what you decide

to do with bardeen do let me know in the

comments if you get anything cool

working I'd love to explore some more

opportunities to use that software