Technical SEO

Summary

This script delves into the critical aspects of technical SEO, emphasizing its impact on website visibility in search results. It outlines Google's crawling, indexing, and ranking processes, highlighting how technical SEO can either facilitate or hinder these stages. The video also introduces SEMrush's Site Audit tool to identify and rectify common SEO issues, such as crawlability, indexing problems, and site performance, ultimately aiming to enhance a website's search ranking and user experience.

Takeaways

• 🔍 Technical SEO is crucial for website visibility on search engines like Google, and a single mistake can lead to the site disappearing from search results.
• 🌐 Understanding how Google operates is fundamental to mastering technical SEO, which involves a three-step process: crawling, indexing, and serving search results.
• 🕸️ Crawlers are automated programs that find and download web page content; technical SEO issues can prevent Google from crawling certain pages, impacting their visibility.
• 📚 The indexing process involves Google determining the relevance and value of web pages, adding them to its index, and excluding low-quality or duplicate content.
• 🏆 Rankings in SEO are not universal numbers but are personalized based on user location, language, device, and behavior, and are subject to constant changes by Google's algorithms.
• 🛠️ Semrush's Site Audit tool can identify technical SEO errors and suggest fixes, which can be implemented by web developers or SEO specialists.
• 🔄 The importance of crawlability in technical SEO cannot be overstated; without it, pages cannot be indexed or ranked by Google.
• 🔄 Redirects play a significant role in SEO, with permanent (301) redirects being preferable for SEO purposes compared to temporary redirects.
• 🔄 Duplicate content can dilute ranking power; combining similar pages or using canonical tags can help focus SEO efforts on a single, strong page.
• 🔒 HTTPS is a must for websites, providing security and improving Google rankings, whereas HTTP is outdated and less secure.
• 🌐 International SEO involves using hreflang tags to help Google serve the correct language versions of web pages to users.
• 📈 Core Web Vitals measure user experience in terms of loading speed, interactivity, and layout stability, and are a direct ranking factor in Google's algorithms.
• 🖼️ Image compression and content caching are effective ways to improve website performance, with tools like TinyPNG or Cloudflare offering solutions.
• 🔗 Internal linking is vital for SEO, helping search engines understand site structure and user navigation, with a goal of making every page reachable within three clicks from the home page.
• 🏷️ Markup, such as schema.org, helps Google understand page content better and can enhance search snippets with rich information, improving click-through rates.

Q & A

What is the most important part of SEO according to the script?

Technical SEO is considered the most important part of SEO because a single detail wrong can cause the entire website to disappear from Google, even if all other aspects are done right.

What are the three main steps in Google's process for handling web pages?

The three main steps are crawling, indexing, and serving search results. Crawling is when Google finds web pages, indexing is when Google determines the content of the page and adds it to its database, and serving search results is when Google displays the most relevant pages in response to a user's search query.

Why is it a problem if Google can't crawl a web page?

If Google can't crawl a web page, it won't be able to index it, which means the page will not appear in Google search results, leading to a loss of potential traffic and visibility.

What is the purpose of the indexing process in Google's search engine?

The indexing process involves determining what each page is about, assessing if the page is worth displaying in search results, and then adding the page to the Google index, which is a database of web pages that can be displayed in search results.

Why might Google choose not to index certain pages?

Google might choose not to index pages that are of low quality, duplicates of other pages, or do not contain anything useful, as it aims to provide the most relevant and high-quality results to users.

How does personalization affect search results in Google?

Personalization in Google means that search results can change based on an individual's location, language, device, and even past search behavior, making the ranking of a web page for a keyword an average across all users rather than a single number.

What is the role of technical SEO in the process of Google serving search results?

Technical SEO plays a role in ensuring that a website is optimized for Google's algorithms, which determine the quality and relevance of web pages to be displayed in search results.

What is the purpose of Semrush's Site Audit tool?

The Site Audit tool is used to find technical SEO errors and provide recommendations on how to fix them, helping to improve a website's search engine ranking and visibility.

Why is it important to regularly run a site audit?

Regularly running a site audit helps identify and address new issues that may arise, ensuring that the website remains optimized for search engines and continues to perform well in search results.

What are some common technical SEO issues that can be found and fixed using the Site Audit tool?

Some common issues include broken internal links, non-indexable pages, crawl budget waste due to improper redirects, duplicate content, and the improper use of canonical tags.

How can a website's architecture help improve its crawl depth?

A well-structured website architecture ensures that every page is reachable within three clicks from the home page, making it easier for search engines to crawl and index all pages.

What is the significance of using HTTPS for a website?

Using HTTPS encrypts the data between the user and the site, making it more secure for users and also serving as a ranking factor in Google's search algorithms.

What is the role of internal linking in SEO?

Internal linking helps search engines understand the relationship between pages on a website, aids in crawlability, and can improve the ranking of targeted pages by directing more link equity to them.

What are Rich Snippets and how do they benefit SEO?

Rich Snippets are additional information and visuals that can be added to regular search snippets, such as star ratings or images, which can attract user attention and potentially increase click-through rates and traffic.

Outlines

00:00

🔍 The Importance of Technical SEO

The first paragraph emphasizes the critical role of technical SEO in ensuring a website's visibility on search engines like Google. It explains that even minor technical errors can lead to a website's disappearance from search results. The paragraph outlines the three-step process of how Google operates: crawling, indexing, and serving search results. It also touches on the personalization of search results and the dynamic nature of rankings. The importance of addressing technical SEO issues at each stage is highlighted, and the paragraph concludes with an introduction to using SEMrush's Site Audit tool to identify and fix these issues.

05:01

📊 Site Audit Overview and Crawlability

This paragraph delves into the Site Audit process, starting with the Site Health score that reflects the number of errors and warnings on a site. It discusses the significance of crawlability, which is essential for Google to index pages. The paragraph provides insights into the issues found during the audit of Casper.com, such as broken internal links and their impact on user experience and search engine crawling. It also explains the importance of fixing these issues and the option to automate the Site Audit process for ongoing monitoring.

10:03

🔄 Redirects, Duplicate Content, and Canonical Tags

The third paragraph focuses on the issues of redirects and duplicate content identified during the site audit. It explains the difference between permanent (301) and temporary redirects, emphasizing the SEO benefits of using 301 redirects for moved pages. The paragraph also addresses the problem of duplicate content and how it can dilute a site's ranking power. It suggests combining duplicate pages or using canonical tags to direct search engines to the preferred version of the page.

15:04

🌐 HTTPS, International SEO, and Core Web Vitals

This paragraph covers various thematic reports from the site audit, starting with the importance of using HTTPS for security and as a ranking factor. It then touches on international SEO considerations, such as hreflang tags for multilingual sites. The Core Web Vitals report is highlighted, which assesses website performance based on user experience metrics like loading speed, responsiveness, and layout stability. The paragraph also provides recommendations for improving site performance, such as compressing images and using caching techniques.

🔗 Internal Linking and Website Structure

The final paragraph discusses the importance of internal linking for SEO, explaining how it helps search engines understand the relationship between pages and can improve rankings. It suggests an ideal website structure, resembling a pyramid, where every page is reachable within three clicks from the home page. The paragraph also advises adding contextual internal links, especially in blog posts, to enhance SEO and provide additional resources to users.

📝 Markup and Rich Snippets for SEO

The last paragraph of the script introduces the concept of markup, which can enhance a page's SEO by helping search engines understand its content better. It discusses the use of Rich Snippets to make search results more attractive with additional information and visuals, potentially increasing click-through rates. The paragraph concludes by emphasizing the benefits of using markup and the Site Audit tool's ability to identify pages lacking markup for optimization.

Keywords

💡Technical SEO

Technical SEO refers to the process of optimizing a website's technical elements to make it search engine friendly. It is crucial because it ensures that search engines can crawl, access, and rank the website effectively. In the video, it is emphasized that even if other aspects of SEO are done correctly, a single technical SEO mistake can cause a website to disappear from search results.

💡Crawlers

Crawlers, also known as spiders or bots, are automated programs used by search engines like Google to discover and download content from web pages. In the script, it is mentioned that some technical SEO issues can prevent Google's crawlers from accessing web pages, which in turn can prevent those pages from being indexed and ranked.

💡Indexing

Indexing is the process by which search engines like Google add web pages to their index, a massive database that stores information about billions of web pages. The script explains that not every page found by Google's crawlers is indexed; only those deemed valuable and non-duplicate are included. Technical SEO plays a role in ensuring pages are indexable.

💡Search Results

Search results are the listings displayed by search engines in response to a user's query. The video script discusses how Google's algorithms determine which pages are most relevant and of high quality to be included in these results. Technical SEO impacts this by ensuring a site's pages are accessible and appealing to both users and search engines.

💡Personalization

Personalization in the context of SEO refers to the customization of search results based on individual user characteristics such as location, language, device, and search history. The script clarifies that rankings are not universal numbers but are influenced by personalization, affecting how different users see search results.

💡Site Audit

A site audit is a comprehensive review of a website to identify and fix technical SEO issues. The script introduces SEMrush's Site Audit tool, which can detect a range of technical SEO errors and provide recommendations for fixes, illustrating its use with the example of the mattress brand Casper.com.

💡Crawlability

Crawlability is a measure of how easily search engine crawlers can navigate and access the pages of a website. The video emphasizes crawlability as a critical technical SEO concern because if pages are not crawlable, they cannot be indexed or ranked by Google.

💡Redirects

Redirects are used to send visitors and crawlers from one URL to another, often when a page's location changes. The script discusses the importance of using the correct type of redirect (permanent or temporary) for SEO purposes, with 301 redirects being preferred for maintaining link value and rankings.

💡Duplicate Content

Duplicate content refers to pages that are largely identical or very similar to others on the web. The script mentions that duplicate content can dilute a site's ranking power and suggests combining such pages or using canonical tags to direct search engines to a preferred version.

💡Canonical Tags

Canonical tags are used to indicate the preferred version of a web page to search engines when there are duplicate or very similar pages. In the script, it is mentioned that over a thousand pages on Casper's site use canonical tags to tell Google to prioritize a different page for indexing.

💡Core Web Vitals

Core Web Vitals are a set of metrics developed by Google to measure user experience on the web, focusing on loading speed, interactivity, and visual stability. The script explains that improving core web vitals not only enhances user experience but also impacts search rankings as they are a ranking factor.

💡Markup

Markup in the context of SEO refers to additional code added to web pages to help search engines understand their content better and to enhance how they appear in search results. The script discusses how markup can lead to rich snippets, which provide extra information and visuals in search results, potentially increasing click-through rates.

Highlights

Technical SEO is crucial as a single detail wrong can cause a website to disappear from Google search results.

Technical SEO involves ensuring search engines can find, access, and rank a website.

Understanding how search engines like Google work is fundamental to learning technical SEO.

Google's process involves crawling, indexing, and serving search results.

Crawling issues can prevent pages from ever ranking on Google.

Indexing determines if a page is worth displaying in Google search results.

Google does not index every page it finds due to quality and relevance algorithms.

Search results are personalized based on user's location, language, device, and behavior.

Rankings in SEO are not fixed numbers but averages for users in specific time and place.

SEMrush's Site Audit tool can identify and suggest fixes for technical SEO errors.

SEOs often collaborate with web developers to implement technical SEO fixes.

Casper.com is used as a case study to demonstrate the use of SEMrush's Site Audit tool.

Casper's website has a low site health score indicating potential for improvement.

Broken internal links are a common technical SEO issue affecting user experience and Google crawling.

Crawlability is the most important technical SEO concern for Google indexing.

Non-indexable pages may be intentional for brands like Casper to avoid duplicate content issues.

Redirects play a crucial role in SEO, with 301 redirects being preferred for permanent URL changes.

Duplicate content can dilute ranking power and may require consolidation or canonical tags.

Core Web Vitals, measured by Google's Lighthouse, are a ranking factor impacting user experience.

Improving site performance can be achieved through image compression and content caching.

Internal linking is vital for website architecture and can enhance SEO rankings.

Markup can enhance SEO by helping Google understand page content and improve search snippet visibility.

Transcripts

thank you

[Music]

technical SEO is arguably the most

important part of SEO that's because if

you get a single detail wrong with

technical SEO your entire website can

disappear from Google

even if you do everything else right

technical SEO includes making sure that

Google and other search engines can find

access and rank your website it also

includes optimizing your website's

structure page load speeds and some

other things

let's back up a little to learn

technical SEO you need to know a little

more about how Google and other search

engines actually work it's a three-step

process

first Google crawls the internet

it sends out automated programs called

crawlers to find web pages and it

downloads content from those web pages

including text images videos and so on

some technical SEO issues can prevent

Google from crawling your web pages and

in that case those pages will never rank

on Google because they won't make it to

the next step in the process

the next step in the process is then

Google indexes those web pages or at

least some of them the indexing process

includes determining what each page is

about figuring out if the page is worth

displaying in Google search results and

then adding the page to the Google index

if so

the Google index is a giant database of

hundreds of billions of web pages that

can be displayed in Google search

results it's important for you to know

that Google doesn't index every page

that it finds if its algorithms conclude

that the pages are just copies of other

Pages or if they're low quality or don't

appear to contain anything useful Google

doesn't want to put them in its index or

show them to people there are also

technical SEO issues that can prevent

indexing from happening for a given web

page and again this would prevent those

web pages from ranking on Google

the last step in the process is that

Google serves its search results

each time a user conducts a Google

search

Google displays the results that its

algorithms have determined are most

likely to satisfy that person

which according to Google comes down to

the highest quality and most relevant

pages and information it has indexed

technical SEO plays a role in this step

too

let me also clarify something about how

rankings work in SEO we talk about how

highly a web page is ranking for a given

keyword as if it's a single Universal

number but actually Google personalizes

many search results so the search

results in individual user sees can

change based on that person's location

language device and even their behavior

like the past searches that they've

conducted

Google's also constantly shifting

rankings of different web pages around

so when we talk about a web Page's

rankings for a keyword it's really an

average for all the users in a given

place during a given time period rather

than a single clear-cut number

now you have a better understanding of

how Google works including its three

main steps or stages

and you know that technical SEO can help

or hurt in each of those stages

now let's take a look at some of the

most important technical SEO issues and

optimizations you should know about

we'll do this by using semrush's site

audit tool this tool can find all sorts

of technical SEO errors for you and even

show you how to fix them as a side note

you don't necessarily have to implement

these fixes in a website's code yourself

if you own the website you definitely

can do that but more often seos work

with web developers to get the changes

made so if you don't know how to code

that's okay to use the site audit tool

let's head over to site audit in the SEO

section of the left hand navigation

let's do this for the mattress brand

casper.com

so we'll type in casper.com and we'll

start setting up the audit

I know Casper has a decent sized website

so I'm going to set the limit to 10 000

checked Pages there are a bunch of other

settings that will just leave as

defaults but notice this last one at the

bottom which lets us set this up to

automatically run every week and receive

an email each time it's done

this is great to do because it means

you'll find out about any issues that

come up on your website rather than

having to remember to go in and check

manually

so we'll keep that setting to run each

week and hit start site audit now

remember how Google has crawlers that go

out and explore websites semrush has its

own crawler too and its crawler is now

going through the Casper website to find

any technical SEO issues to crawl

thousands of pages takes a while so

normally you'd go do something else and

check back in 20 or 30 minutes or when

you get notified via email

but let's just skip ahead here with a

quick Video Edit and boom it's magically

done

here's the site audit overview page

the first box is the site Health score

which is based on how many errors and

warnings are on the site the higher the

score the better

and you can see that Casper's score

isn't great

but in a way that's actually good news

for Casper because it means there are

probably some big opportunities to

improve its results

that said you can't expect to score 100

here eighty percent or higher is

generally pretty good

you can also see how many errors

warnings and notices there are these are

based on how severe they tend to be so

an error is more severe warnings are

less important and notices generally

aren't a big deal at all

below those are the Thematic reports

these show the different types of issues

the semris crawler found along with a

sub score for each

we'll come back to those in a second but

first I want to show you the issues tab

above

this tab shows all the issues semrush

found

so for example we can see there are a

lot of broken internal links

now an internal link is just a link that

points from a page on the website to

another page on the same website

broken links are a problem for users

because they lead to error pages and

they're a problem for Google because it

makes it harder for Google to crawl and

index the pages you can click on the why

and how to fix it link next to any of

these issues to see exactly what it

means the impact it can have and how to

fix it

and you can click on the issue text

itself to see a list of the pages that

have the problem

when we click on this we can see each

page that has the broken link

the URL of the broken link

and the HTTP status code of the page

that's linked to

for example

a 404 status code means the page

couldn't be found and was probably

deleted

so jumping back to the issues tab you

could go through each of these to

understand the overall technical SEO

issues with your site

and either fix them yourself or send the

report to any web developers to fix them

or hire someone to fix them we won't go

into detail about every one of these

because there are hundreds of potential

issues and they only matter when they

come up

back to the overview tab these thematic

reports are just another way to view the

issues

the first one you see here is

crawlability

crawlability is probably the single most

important technical SEO concern because

as we saw in the way Google works

everything starts with a crawl if Google

isn't able to crawl your web pages it

can't index them either which means you

won't be able to rank and get search

traffic

when we pop open the crawlability report

we can see that less than half of the

pages on Casper's website are actually

indexable

that means that most of Casper's web

pages can't show up in Google search

results

but that's not necessarily as bad as it

sounds Casper might have a lot of web

pages that they need for whatever reason

but that they don't want to show up on

Google so at least some of this is

probably on purpose

for example Casper might have a bunch of

pages with duplicate content or near

duplicate content

in other words pages that are basically

copies of other pages with only minor

changes they might have 10 different

landing pages that they Point Instagram

ads at and each one has a slightly

different design to match the specific

ad but they all advertise the same

mattress that Casper also has a product

page for

in that case Casper would only want the

main product page to show up in Google

search results

so they would no index the Instagram

landing pages and they would be

non-indexable

but often pages are non-indexable on

accident for example web developers like

to set up Pages they're building on the

staging site within no index tags which

is good

but then they forget to remove those no

index tags when those pages go live on

the website so those pages can't rank on

Google

or over here in the crawl budget waste

widget there are some other potential

problems first we have temporary

redirects and permanent redirects

redirects are used when you move a web

page from one URL to another

when a person or crawler visits the old

URL the redirect will automatically send

them to the new URL instead

when it comes to SEO the right type of

redirect to use is usually a permanent

one also called a 301 redirect

that tells Google to rank the new URL

instead of the old one and that it

should count all the links that are

pointing to the old URL as if they're

pointing to the new URL instead a

temporary redirect doesn't always have

the same effect there have been studies

that show that using temporary reader

routes lead to lower rankings than using

permanent redirects however if the

change in url really is only temporary

then using a temporary redirect is the

right choice so it depends on the

situation

but there might be a big opportunity

here to turn these temporary redirects

into permanent ones and improve some of

Casper's rankings and traffic as a

result

then we have duplicate content we

already talked about this a little but

basically if you have duplicate content

it might mean you're dividing your

ranking power

if you have two or three blog posts or

web pages that all basically say the

same thing you'll often get better

results if you combine them into one and

redirect the weaker pages to the

stronger one using permanent redirects

or if you need those pages to stay on

the website for users but you only one

of them to rank on Google you can use a

Rel equals canonical tag

and that's the next item here Pages

canonical to other page

this means there are over a thousand

pages on the Casper site that basically

say to Google hey don't show this page

to people show them that other page

instead so again this is potentially an

issue but it can also be used on purpose

e-commerce sites tend to have a lot of

these because they have a lot of

internal search results pages and things

like that

I'm going to skip past the rest of these

but we'll cover some of them in the

other reports

back to the overview tab let's quickly

talk about what the other thematic

reports cover

https is pretty much a pass fail here is

the site encrypted with https or does it

use HTTP using https is more secure

better for users and it's also a Google

ranking Factor so you should use it then

we have international SEO

this is mostly about href laying tags

which you should use if you have

multiple versions of web pages in

different languages these tags help

Google serve the right pages to the

people who speak those languages

the core web vitals report uses a Google

tool called Lighthouse to measure

website performance in terms of user

experience

it looks at how fast the web pages load

how fast they respond when users click

on them and how much their layout shifts

when users interact with them

core web vitals are another ranking

factor that also impacts users so it's

good to improve this score

as you can see this report gives a lot

of specific advice to improve

performance like removing unused

JavaScript and removing or optimizing

third-party code like plugins

two other common high impact ways to

improve your website performance are to

compress your images and cache your

content

you can compress your images with a

service or Plugin like tiny PNG or short

pixel to effectively reduce their file

size without hurting their quality

and lots of web hosts offer an option to

Cache your website which basically saves

your website content so it will load

faster when people visit it

cloudflare is another Super common

option for caching that you can use

the next report is site performance

which uses some of semrush's own

measurements to look at loading speed

and things like that

it's a little different from the core

web vitals report but there's a decent

amount of overlap

the internal linking report gets into

more detail about internal links which

we touched on already in the issues tab

but it's pretty important so let's dive

into it a little more

the first widget in this report is Pages

crawl depth crawl depth means how many

clicks it takes to navigate from the

website's home page to another page on

the site ideally you want every page on

the site to be reachable within three

clicks from the home page a lot of

Casper's web pages take four or more

clicks to reach clicking on the red part

of this graph would show us all those

pages but we don't need to do that right

now

the important thing is to understand how

to avoid this problem and there are a

few ways to do that but the best way is

generally to have a good website

architecture you can structure your

website even if it has a million Pages

or more so that every page is three

clicks or less from the home page

imagine the ideal structure of a website

as the home page at the top of a pyramid

and links pointing down to maybe 100

Pages below that that's one click and

then each of those pages links to

another 100 Pages that's two clicks and

we're already up to ten thousand Pages

if every one of those pages links to a

hundred others you can reach a million

Pages within three clicks you can also

add internal links contextually

this is really common in blog posts for

example it helps show Google which pages

are related by topic and it can help

Point users to additional resources also

and it can help improve your rankings if

you have a web page that you want to

rank better pointing more internal Links

at it will often help quite a bit

back to the site audit overview page

again the last thematic report is a

markup

there are different types of markup but

in general markup is additional code

that you can add to your web pages that

can help your SEO efforts in two ways

first markup helps Google understand

what your web pages are about which can

lead to them showing up more often and

in different places on Google

and markup can also help your search

Snippets stand out more via something

called Rich Snippets

Rich Snippets add additional information

and visuals to a regular search snippet

so for example these recipe Snippets

contain star ratings cook times and

images

Rich Snippets help attract users

attention which can lead to more traffic

as they click through more often there

are lots of other ways that markup can

help improve the way your web pages are

represented on Google so it's great to

use

and the markup report can help you find

pages that aren't using markup so that

you can add it

and that wraps up this lesson on

technical SEO

you may not feel like a total expert on

it but now you know about some of the

most common technical issues and how to

use semrush to find and diagnose many

more