Duplicate content: Causes and solutions

Search engines like Google have a problem – it’s called ‘duplicate content’. Duplicate content means that similar content appears at multiple locations (URLs) on the web, and as a result search engines don’t know which URL to show in the search results. This can hurt the ranking of a webpage, and the problem only gets worse when people start linking to the different versions of the same content. This article will help you to understand the various causes of duplicate content, and to find the solution to each of them.

What is duplicate content?

Duplicate content is content which is available on multiple URLs on the web. Because more than one URL shows the same content, search engines don’t know which URL to list higher in the search results. Therefore they might rank both URLs lower and give preference to other webpages.

In this article, we’ll mostly focus on the technical causes of duplicate content and their solutions. If you’d like to get a broader perspective on duplicate content and learn how it relates to copied or scraped content or even keyword cannibalization, we’d advise you to read this post: What is duplicate content.

Let’s illustrate this with an example

Duplicate content can be likened to being at a crossroads where road signs point in two different directions for the same destination: Which road should you take? To make matters worse, the final destination is different too, but only ever so slightly. As a reader, you don’t mind because you get the content you came for, but a search engine has to pick which page to show in the search results because, of course, it doesn’t want to show the same content twice.

Let’s say your article about ‘keyword x’ appears at http://www.example.com/keyword-x/ and the same content also appears at http://www.example.com/article-category/keyword-x/. This situation is not fictitious: it happens in lots of modern Content Management Systems. Then let’s say your article has been picked up by several bloggers and some of them link to the first URL, while others link to the second. This is when the search engine’s problem shows its true nature: it’s your problem. The duplicate content is your problem because those links both promote different URLs. If they were all linking to the same URL, your chances of ranking for ‘keyword x’ would be higher.

If you don’t know whether your rankings are suffering from duplicate content issues, these duplicate content discovery tools will help you find out!

Causes of duplicate content

There are dozens of reasons for duplicate content. Most of them are technical: it’s not very often that a human decides to put the same content in two different places without making clear which is the original – it feels unnatural to most of us. There are many technical reasons though and it mostly happens because developers don’t think like a browser or even a user, let alone a search engine spider – they think like a programmer. Take that article we mentioned earlier, that appears on http://www.example.com/keyword-x/ and http://www.example.com/article-category/keyword-x/. If you ask the developer, they will say it only exists once.

Misunderstanding the concept of a URL

No, that developer hasn’t gone mad, they are just speaking a different language. A CMS will probably power the website, and in that database there’s only one article, but the website’s software just allows for that same article in the database to be retrieved through several URLs. That’s because, in the eyes of the developer, the unique identifier for that article is the ID that article has in the database, not the URL. But for the search engine, the URL is the unique identifier for a piece of content. If you explain that to a developer, they will begin to get the problem. And after reading this article, you’ll even be able to provide them with a solution right away.

Session IDs

You often want to keep track of your visitors and allow them, for instance, to store items they want to buy in a shopping cart. In order to do that, you have to give them a ‘session.’ A session is a brief history of what the visitor did on your site and can contain things like the items in their shopping cart. To maintain that session as a visitor clicks from one page to another, the unique identifier for that session – called the Session ID – needs to be stored somewhere. The most common solution is to do that with cookies. However, search engines don’t usually store cookies.

At that point, some systems fall back to using Session IDs in the URL. This means that every internal link on the website gets that Session ID added to its URL, and because that Session ID is unique to that session, it creates a new URL, and therefore duplicate content.

URL parameters used for tracking and sorting

Another cause of duplicate content is using URL parameters that do not change the content of a page, for instance in tracking links. You see, to a search engine, http://www.example.com/keyword-x/ and http://www.example.com/keyword-x/?source=rss are not the same URL. The latter might allow you to track what source people came from, but it might also make it harder for you to rank well – very much an unwanted side effect!

This doesn’t just go for tracking parameters, of course. It goes for every parameter you can add to a URL that doesn’t change the vital piece of content, whether that parameter is for ‘changing the sorting on a set of products’ or for ‘showing another sidebar’: all of them cause duplicate content.

Scrapers and content syndication

Most of the reasons for duplicate content are either the ‘fault’ of you or your website. Sometimes, however, other websites use your content, with or without your consent. They don’t always link to your original article, and therefore the search engine doesn’t ‘get’ it and has to deal with yet another version of the same article. The more popular your site becomes, the more scrapers you’ll get, making this problem bigger and bigger.

Order of parameters

Another common cause is that a CMS doesn’t use nice clean URLs, but rather URLs like /?id=1&cat=2, where ID refers to the article and cat refers to the category. The URL /?cat=2&id=1 will render the same results in most website systems, but they’re completely different for a search engine.

Comment pagination

 In my beloved WordPress, but also in some other systems, there is an option to paginate your comments. This leads to the content being duplicated across the article URL, and the article URL + /comment-page-1/, /comment-page-2/ etc.

Printer-friendly pages

If your content management system creates printer-friendly pages and you link to those from your article pages, Google will usually find them, unless you specifically block them. Now, ask yourself: Which version do you want Google to show? The one with your ads and peripheral content, or the one that only shows your article?

WWW vs. non-WWW

This is one of the oldest in the book, but sometimes search engines still get it wrong: WWW vs. non-WWW duplicate content, when both versions of your site are accessible. Another, less common situation but one I’ve seen as well is HTTP vs. HTTPS duplicate content, where the same content is served out over both.

Conceptual solution: a ‘canonical’ URL

As we’ve already seen, the fact that several URLs lead to the same content is a problem, but it can be solved. One person who works at a publication will normally be able to tell you quite easily what the ‘correct’ URL for a certain article should be, but sometimes when you ask three people within the same company, you’ll get three different answers…

That’s a problem that needs addressing because, in the end, there can be only one (URL). That ‘correct’ URL for a piece of content is referred to as the Canonical URL by the search engines.

canonical_graphic_1024x630

Ironic side note

Canonical is a term stemming from the Roman Catholic tradition, where a list of sacred books was created and accepted as genuine. They were known as the canonical Gospels of the New Testament. The irony is it took the Roman Catholic church about 300 years and numerous fights to come up with that canonical list, and they eventually chose four versions of the same story

Identifying duplicate contents issues

You might not know whether you have a duplicate content issue on your site or with your content. Using Google is one of the easiest ways to spot duplicate content.

There are several search operators that are very helpful in cases like these. If you’d want to find all the URLs on your site that contain your keyword X article, you’d type the following search phrase into Google:

site:example.com intitle:"Keyword X"

Google will then show you all pages on example.com that contain that keyword. The more specific you make that intitle part of the query, the easier it is to weed out duplicate content. You can use the same method to identify duplicate content across the web. Let’s say the full title of your article was ‘Keyword X – why it is awesome’, you’d search for:

intitle:"Keyword X - why it is awesome"

And Google would give you all sites that match that title. Sometimes it’s worth even searching for one or two complete sentences from your article, as some scrapers might change the title. In some cases, when you do a search like that, Google might show a notice like this on the last page of results:

This is a sign that Google is already ‘de-duping’ the results. It’s still not good, so it’s worth clicking the link and looking at all the other results to see whether you can fix some of them.

Read more: DIY: duplicate content check »

Practical solutions for duplicate content

Once you’ve decided which URL is the canonical URL for your piece of content, you have to start a process of canonicalization (yeah I know, try saying that three times out loud fast). This means we have to tell search engines about the canonical version of a page and let them find it ASAP. There are four methods of solving the problem, in order of preference:

  1. Not creating duplicate content
  2. Redirecting duplicate content to the canonical URL
  3. Adding a canonical link element to the duplicate page
  4. Adding an HTML link from the duplicate page to the canonical page

Avoiding duplicate content

Some of the above causes for duplicate content have very simple fixes to them:

  • Are there Session ID’s in your URLs?
    These can often just be disabled in your system’s settings.
  • Have you got duplicate printer friendly pages?
    These are completely unnecessary: you should just use a print style sheet.
  • Are you using comment pagination in WordPress?
    You should just disable this feature (under settings » discussion) on 99% of sites.
  • Are your parameters in a different order?
    Tell your programmer to build a script to always put parameters in the same order (this is often referred to as a URL factory).
  • Are there tracking links issues?
    In most cases, you can use hash tag based campaign tracking instead of parameter-based campaign tracking.
  • Have you got WWW vs. non-WWW issues?
    Pick one and stick with it by redirecting the one to the other. You can also set a preference in Google Webmaster Tools, but you’ll have to claim both versions of the domain name.

If your problem isn’t that easily fixed, it might still be worth putting in the effort. The goal should be to prevent duplicate content from appearing altogether, because it’s by far the best solution to the problem.

301 Redirecting duplicate content

In some cases, it’s impossible to entirely prevent the system you’re using from creating wrong URLs for content, but sometimes it is possible to redirect them. If this isn’t logical to you (which I can understand), do keep it in mind while talking to your developers. If you do get rid of some of the duplicate content issues, make sure that you redirect all the old duplicate content URLs to the proper canonical URLs.

 Sometimes you don’t want to or can’t get rid of a duplicate version of an article, even when you know that it’s the wrong URL. To solve this particular issue, the search engines have introduced the canonical link element. It’s placed in the <head> section of your site, and it looks like this:

<link rel="canonical" href="http://example.com/wordpress/seo-plugin/" />

In the href section of the canonical link, you place the correct canonical URL for your article. When a search engine that supports canonical finds this link element, it performs a soft 301 redirect, transferring most of the link value gathered by that page to your canonical page.

This process is a bit slower than the 301 redirect though, so if you can just do a 301 redirect that would be preferable, as mentioned by Google’s John Mueller.

Keep reading: rel=canonical • What it is and how (not) to use it »

Linking back to the original content

If you can’t do any of the above, possibly because you don’t control the <head> section of the site your content appears on, adding a link back to the original article on top of or below the article is always a good idea. You might want to do this in your RSS feed by adding a link back to the article in it. Some scrapers will filter that link out, but others might leave it in. If Google encounters several links pointing to your original article, it will figure out soon enough that that’s the actual canonical version.

Conclusion: duplicate content is fixable, and should be fixed

Duplicate content happens everywhere. I have yet to encounter a site of more than 1,000 pages that hasn’t got at least a tiny duplicate content problem. It’s something you need to constantly keep an eye on, but it is fixable, and the rewards can be plentiful. Your quality content could soar in the rankings, just by getting rid of duplicate content from your site!

Read on: Rel=canonical: The ultimate guide »

The post Duplicate content: Causes and solutions appeared first on Yoast.

SEO Basics: How to start with SEO?

You’ve had this great idea. You’ve built this amazing website. And then, you want that website to attract visitors! You want to be found! What to do? How do you get started with SEO? How do you start with SEO on a brand new site? In this blog post, I’ll talk you through the 7 steps you need to take in order to get your SEO strategy up and running. 

So, you’ve started your first site and you want it to be found, so you can share your thoughts and views with the world. What to do? Let’s go through the steps of starting with SEO!

  1. Install Yoast SEO

    Provided that your website is on WordPress, installing Yoast SEO should be the first step in your SEO strategy. Our Yoast SEO plugin will help you to make sure your website is crawlable and findable. Yoast SEO will immediately take care of some technical SEO issues, just by being installed on your website. Besides that, our plugin will help you to construct your website in such a way that Google will understand and rank it. We offer a free and a premium plugin. If you’re just starting out, you’ll probably won’t need our premium version yet, although it can already save you some valuable time.

  2. Get that first link

    Google needs to know your website exists. And, in order for Google to know about your awesome new site, you need at least one external link towards your site. The reason for this: Google crawls the web. It follows links and saves all the webpages it finds in a very large database called the index. So, if you want to get into that index, you need (at least) one external link. So make sure to get that link from an external website!

  3. What do you want to rank for?

    Make sure to attract the right audience to your website. Who are your customers? For whom did you build this website? What terms do your customers use when searching on Google? Find out as much as you can about your audience.

    SEOs refer to this stage as doing your keyword research. This is a hard and important phase. There are a lot of helpful tools that make doing keyword research easier. Some of these tools are free, others are rather expensive. While these tools will make the difficult phase of keyword research easier, you should remember that you can’t outsource your keyword research to a tool. You really need to think about your audience and about the search terms they are using. Take your time for this phase. It is crucial. If you do your keyword research correctly, you’ll come up with a long list of keywords you want to rank for.

  4. Set realistic goals

    For a new site, it is rather hard to rank high in the beginning. Older sites already have a history, established their authority and a lot of links pointing towards them. That means that Google’s crawlers come by more often at older sites. For a new site to rank, you’ll always need to be a little patient. And remember: some search terms will be out of reach for a new site because there’s too much competition. Trying to rank for [WordPress SEO] will be rather hard for any new blog, because of some fierce competition on that term from Yoast.com.

    If you’re just starting with your site, try to aim at ranking for long-tail keywords. Long-tail keywords are keywords that are longer and more specific and have far less competition than the popular head keywords. After a while, when your site starts to rank for the long-tail keywords, you could try and go after the more head keywords.

  5. Internal linking

    As I already mentioned in step 2, Google follows links. Google also follows the links on your website, your internal linking structure. It crawls through your website following the internal linking structure of your site. That structure is like a guide to Google. Make sure your internal linking structure is flawless. That’ll help with your ranking. 

    If you start with a brand new website, you’ll probably don’t have much content yet. This is the perfect time to think about structure. Now it is relatively easy. It’s like having a new closet and you haven’t started buying clothes. Now is the time to think about the things you want to put on the top shelf and which items you want to hide in the back of your closet. So, decide which pages are most important to you. What are the pages you want to rank with? Make sure that these pages have the most internal links pointing towards them.

  6. Start writing

    In order to get ranked, you need to have content. A very important step in how to start with your SEO is to write amazing content for all these search terms you want to be found for. The content analysis in the Yoast SEO plugin will help you to write that content. Our analysis will help you to write a text that is both readable and SEO friendly.

    While you’re writing, make sure to use the words you want to be found for. Use them in headings and in the introduction and conclusion of your text. After writing your text, you should optimize your SEO title and your meta description. The Yoast SEO plugin will help you to do all these things.

  7. Get those links!

    External links are important to get your site in high positions in those search engines. But gathering those external links can be a hard process. Make sure to write content people want to share and link to. Original ideas and great, valuable content will make the chance that people would want to share that much bigger.

    Of course, reaching out to people and making them aware of your awesome website and product can be a good strategy to get those external links too. Read more about a successful link building strategy or find out what link building is first.

And then what?

The truth is that SEO is more than these 7 steps. This is only the very beginning, the steps you take to start with SEO. In order to get longterm high rankings in the search engines, you need to do hard work. Your content has to be amazing, your site structure has to remain flawless (and that’s challenging when your site is growing) and you’ll have to keep earning those external links. The only way to really do that, in the long run, is to make sure that your audience enjoys visiting your website. If you want to rank the highest, make sure your site is the very best. Good luck!

Read more: wordPress SEOL the definitive guide »

The post SEO Basics: How to start with SEO? appeared first on Yoast.

Yoast SEO 12.5: Behind the scenes improvements

These last couple of months here at Yoast SEO HQ have all been about building better things. Behind the scenes, we’re making good progress at getting our flagship plugins ready for the future. While we’re busy building the future, we also stick to our regular two-week release schedule, which means it’s time to introduce Yoast SEO 12.5.

Fixing bugs and behind the scenes scaffolding

Yoast SEO 12.5 is one where most of the work went on behind the scenes. We’re working on improving our codebase and will be releasing something cool pretty soon. To get that done, we need to do some cleaning up. 

Besides getting ready for future releases, we’ve fixed a number of bugs. One of those bugs happened for terms where keywords and snippet preview data would be synced across all languages in a MultilingualPress multisite environment. Another bug misplaced visually hidden text in several elements inside the Snippet Preview. We’ve also deprecated the old Search Console integration as that won’t be returning in that same shape or form.

A reminder about support for older versions of WordPress 

With yesterday’s release of WordPress 5.3, we will return to our initial position of only supporting the latest two versions of WordPress. In this case, that’s WordPress 5.2 and WordPress 5.3, and not versions before that. This means we’ll end our support for WordPress 4.9, which we’ve supported longer than usual to allow people to transition to WordPress 5.0 and ease over people to the classic editor or block editor. Luckily, the vast majority of you have probably updated to the latest versions.

In WordPress 5.2, the core team upped the minimum PHP requirements from an ancient 5.2 to the slightly less ancient 5.6. By supporting the last two versions of WordPress, we can now develop our software using PHP 5.6. This means that we can develop faster and more securely. Read Joost’s post on supporting older versions of WordPress.

Update now to Yoast SEO 12.5

Yoast SEO 12.5 is a fairly basic release with lots of stuff going on in the background. We’ve fixed a number bugs and helped Yoast SEO get ready for future improvements.

The post Yoast SEO 12.5: Behind the scenes improvements appeared first on Yoast.

Google BERT: A better understanding of complex queries

By announcing it as the “biggest change of the last five years” and “one that will impact one in ten searches”, Google sure turned some heads with an inconspicuous name: BERT. BERT is a Natural Language Processing (NLP) model that helps Google understand language better in order to serve more relevant results. There are million-and-one articles online about this news, but we wanted to update you on this nonetheless. In this article, we’ll take a quick look at what BERT is and point you to several resources that’ll give you a broader understanding of what BERT does.

To start, the most important thing to keep in mind that Google’s advice never changes when rolling out these updates to its algorithm. Keep producing quality content that fits your users’ goals and make your site as good as possible. So, we’re not going to present a silver bullet for optimizing for the BERT algorithm because there is none. 

What is BERT?

BERT is a neural network-based technique for natural language processing (NLP) pre-training. The full acronym reads Bidirectional Encoder Representations from Transformers. That’s quite the mouthful. It’s a machine-learning algorithm that should lead to a better understanding of queries and content. 

The most important thing you need to remember is that BERT uses the context and relations of all the words in a sentence, rather than one-by-one in order. So BERT can figure out the full context of a word by looking at the words that come before and after it. The bi-directional part of it makes BERT unique.

By applying this, Google can better understand the full gist of a query. Google published several example queries in the launch blog post. I won’t repeat them all but want to highlight one to give you an idea of how it works in search. For humans, the query [2019 brazil traveler to usa need a visa] obviously is about answering if a traveler from Brazil needs to have a visa for the USA in 2019. Computers have a hard time with that. Previously, Google would omit the word ‘to’ from the query, turning the meaning around. BERT takes everything in the sentence into account and thus figures out the true meaning.

As you can see from the example, BERT works best in more complex queries. It is not something that kicks in when you search from head terms, but rather the queries in the long tail. Still, Google says it will impact every one in ten searches. And even then, Google says that BERT will sometimes get it wrong. It’s not the end-all solution to language understanding.

Where does Google apply BERT?

For ranking content, BERT is currently rolled out in the USA for the English language. Google will use the learnings of BERT to improve search in other languages as well. Today, BERT is used for featured snippets in all markets where these rich results appear. According to Google, this leads to much better results in those markets.

Useful resources

We’re not going into detail into what BERT does, talking about its impact on NLP and how it’s now being incorporated into search, because we’re taking a different approach. If you want to understand how this works, you should read up on the research. Luckily, there are plenty of readable articles to be found on this subject. 

This should give you a solid understanding of what is going on in the rapidly developing world of language understanding.

Google’s latest update: BERT

The most important takeaways from this BERT news is that Google is yet again becoming closer to understanding language on a human level. For rankings, it will mean that it will present results that are a better fit to that query and that can only be a good thing. 

There’s no optimizing for BERT other than the work you are already doing: produce relevant content of excellent quality. Need help writing awesome content? We have an in-depth SEO copywriting training that shows you the ropes.

The post Google BERT: A better understanding of complex queries appeared first on Yoast.

Managing access to Yoast SEO with SEO roles

It may be one of Yoast SEO’s lesser-known features: SEO roles. A site admin can determine in the backend of WordPress who gets access to the various settings and features of Yoast SEO. This makes for a more fluid and flexible access protocol for different kinds of users on a site. It is no longer a one-size-fits-all solution, but a more tailored one. SEO roles make Yoast SEO even more powerful for every type of user. Here, we’ll explain why these roles are so awesome.

Managing user roles in Yoast SEO

It used to be quite the challenge to use Yoast SEO in a larger site environment. As an admin, you’d have to choose between offering users full access to the plugin or just access to the SEO post editor part. That means a regular user couldn’t use the redirect manager, for instance, and had to ask an admin for help every time he or she wanted to add, change or delete redirects. We’ve seen it happing here at Yoast as well. Of course, there’s a whole range of possible permissions in between. Yoast SEO provides the option for two roles that make this a lot easier to manage: the SEO manager and SEO editor, in addition to the admin who determines who gets to see what.

Roles and capabilities

Roles in Yoast SEO consist of one or more capabilities, like:

  • managing options (this gives you full access),
  • managing redirects,
  • editing advanced metadata,
  • access to the bulk editor.

The SEO editor, for instance, can now make redirects, but cannot change the settings of the plugin or access the advanced metadata editor of Yoast SEO. This way, the SEO editor has more access than a regular user, but less than the SEO manager who can manage settings as well. If you use a permission or role manager plugin for WordPress like Justin Tadlock’s excellent Members plugin, you get even more fine-grained control over the capabilities within Yoast SEO. This way, you can mix and match capabilities in any form you’d like.

In Yoast SEO Premium, we’ve also added the capability to manage redirects without having to be an administrator. By activating this, users within a specific role get full access to the redirect manager. No longer do site managers have to be swamped with redirects requests by site editors, they can manage those themselves. Personally, I like that a lot. By adding some magic code to the plugin, the redirect manager now shows up in the WordPress sidebar menu, even if your Yoast SEO menu is hidden by default. How cool is that?!

Managing your site has never been easier

The SEO roles in Yoast SEO make it incredibly easy to give more people working on your site access to the features and settings they need, without granting them full access. Does your site editor need to edit advanced metadata? No? Block it in Yoast SEO. Does he or she need to manage redirects and do large-scale SEO optimizations with the bulk editor? Great, grant him or her access to these parts of the plugin. You can do this and more – all from the admin dashboard of Yoast SEO!

Read more: Yoast SEO 5.5: Introducing SEO roles »

The post Managing access to Yoast SEO with SEO roles appeared first on Yoast.

Update or delete? Cleaning up old content on your site

Sometimes, content on your website becomes irrelevant or out of date, and you need to decide whether to update it or delete it. It’s part of your regular content maintenance activities. There are several ways to go about this and this article helps you decide what’s the best solution for your old content!

Update old content that is still valid

Let’s start with an example: On our blog, we have an article on meta descriptions that needs constant updating to keep it relevant. We just have to make sure it stays up to date with all the changes Google keeps making to the way it handles meta descriptions. Sometimes it seems they can be a bit longer and sometimes they seem to go back to the old length again.

Our post helps writers and editors to write meta descriptions, even though the advice changes over time. Although the article itself might be what we call cornerstone content, its content must be updated to keep up with the latest standards – constantly.

You can easily create new, valuable content from your old posts if you update it and make it current again: old wine in new bottles, as the saying goes. You could, for example, replace older parts of that content with updates, or you could merge three old blog posts about the same subject into one new post. If you do this, please remember to redirect the old post URLs to the new post, using a 301 Redirect. More on that later.

Read more: Keep your content fresh and up to date »

Delete irrelevant posts or pages

It’s likely that you have old posts or pages on your site that you don’t need anymore. Think along the lines of a blog post about a product you stopped selling a while ago and have no intention of ever selling again, an announcement of an event that took place a long time ago or old pages with little or no content – so-called thin content pages.

These are just some examples, but I’m sure you know which posts and/or pages I’m talking about. This old content adds no value anymore, now or for the foreseeable future. In that case, you need to either tell Google to forget about these old posts or pages or give the URL another purpose.

When I talk about deleting old content, I don’t mean just pressing “delete” and then forgetting about it. If you do that, the content might show up in Google for weeks after deletion. The URL might actually have some link value as well, which would be a shame to waste.

So, what should you do? Here are two options:

“301 Redirect” the old post to a related one

When a URL still holds value because, say, you have a number of quality links pointing to that page, you want to leverage that value by redirecting the URL to a related one. With a 301 Redirect you’ll tell search engines and visitors there’s a better or newer version of this content elsewhere on your site. The 301 redirect automatically sends people and Google to this page.

Say you have an old post on a specific dog breed. You need to delete it, so the logical next step would be to redirect that post to a newer post about this dog breed. If you don’t have that post, choose a post about the closest breed possible. If that post isn’t available, you could redirect it to the category page for these posts (e.g. “dog breeds”) and if that is also not an option, redirect to the homepage. That last one might be about “pets”, for example. It’s a bit of a last resort though, there probably are better options on your site.

Creating a 301 Redirect (for instance in WordPress) isn’t hard, but doing it with Yoast SEO Premium is easy as pie. If you don’t have it yet, find out about all the extras that are in Yoast SEO Premium here.

Tell search engines the content is intentionally gone

If there isn’t a relevant page on your site you can redirect to, it’s wise to tell Google to forget about your old post entirely by serving a “410 Deleted” status to Google. This status code will tell Google and visitors the content didn’t just disappear; you’ve deleted it with a reason.

When Google can’t find a post, the server will usually return a “404 Not Found” status to the search engine’s bot. You’ll also find a 404 crawl error in your Google Search Console for that page. Eventually, Google will work it out and the URL will gradually vanish from the search result pages. But this takes time.

The 410 is more powerful in the sense that it tells Google that the page is gone forever, never to return. You deleted it on purpose, period. Google will act on that faster than with a 404. Read up about the server status codes if this is all gibberish to you.

Keep reading: How to properly delete a page from your site »

Do you have old content to deal with?

Cleaning up old content should be part of your content maintenance routine. If you don’t go through your old posts regularly, you’re bound to run into issues sooner or later. You might show incorrect information to visitors or hurt your own rankings by having too many pages about the same topic, increasing chances of keyword cannibalization, which is a lot of work to fix later on. Therefore, go through your old posts, and decide what to do: update, merge or delete!

Good luck cleaning up your site.

Read on: Should you keep old content? »

The post Update or delete? Cleaning up old content on your site appeared first on Yoast.

Yoast SEO 12.4: Image in snippet preview

After releasing several updates to our snippet preview in previous releases, Yoast SEO 12.4 now shows an image for your post in the mobile snippet preview, just like Google would. We have several other improvements and fixes for you in store with Yoast SEO 12.4. Find out more!

Featured image in the mobile snippet preview

Not too long ago, Google made several changes in the way they present results on mobile. To mimic that, we started updating our snippet preview. In the latest iteration, we see a favicon (added in Yoast SEO 11.5, enhanced in 12.1) and new font sizes (added in 12.1).

The one thing missing from the current snippet preview in Yoast SEO is that of an image. For some search results on mobile, Google will now show the main image next to it. In Yoast SEO 12.4, we automatically use your featured image to mimic the way Google does this now. We’ll use the first image in your content if you haven’t set a featured image. Remember, this only works in the mobile snippet preview.

The mobile snippet preview now uses the featured image

Schema structured data content blocks

Our Schema structured data content blocks for the block editor have proven to be a valid way to quickly get rich results for these types of content. The two current content blocks, namely FAQ and HowTo blocks, are incredibly easy to add, update and publish. They give you valid structured data for that content and thus a great chance of getting rich results. Be sure to try them out! In Yoast SEO 12.4, we’ve improved the findability of the blocks in the block editor library to help even more people find and use them.

Find the Yoast SEO structured data content blocks in the WordPress library

Fixes and enhancements

For this release, we had several users contributions. Emily Leffler Schulman suggested to change the readability score for empty content from “Needs Improvement” with a red icon to “Not Available” with a gray icon. This makes it less confronting for users. Emily also updated the URLs used to ping Google and Bing about the location of a sitemap. Steven Franks added information to the Twitter settings to make it more clear why you should enable Open Graph. Thanks both!

We also fixed a number of bugs in this release. One of these bugs made it impossible to set Twitter and Facebook images for attachment pages. Another bug concerned the visibility of a nested paragraph in the “noindex” metabox warning. Plus, we clear up the last of the Google+ data, there was still some leftovers in the settings export.

Update to Yoast SEO 12.4

That’s Yoast SEO 12.4 for you! We’ve updated the mobile snippet preview with the latest changes by Google and we fixed several bugs. A number enhancements makes Yoast SEO a little bit easier to use. Update to the latest version at your convenience!

The post Yoast SEO 12.4: Image in snippet preview appeared first on Yoast.

The Block Editor/Gutenberg: Why you should be using it

At Yoast, we truly believe you should be using the Block Editor (formerly known as Gutenberg) in WordPress, simply because it’s a much better experience than the ‘classic editor’. Unfortunately, when we look at our statistics, we see that a large segment of our users still uses the classic editor. This makes us sad, but also makes us want to explain why you really should start switching over.

When we talk about Gutenberg/the Block Editor within the WordPress community, there are always a lot of emotions involved. A large percentage of these emotions were down to the release process of Gutenberg. I have certainly also been critical of that process. But, while that launch could undoubtedly have been handled better, it shouldn’t cloud our judgment of the product. Unfortunately, some of that negativity has spilled into the wider world without the context, and that really is a shame.

In our eyes, it’s simple: the Block Editor is now much better than when it launched. Smashing Magazine did a good post mortem. In their post, they also differentiate between the problematic launching process and its current state. And, they assess it to be very good at the moment. It’s important to note that the Block Editor is still under heavy development within the Gutenberg project. The current version of the Block Editor is much, much better than what it was like a year ago. In fact, I can say it’s very good.

Why you should switch to the Block Editor

The Gutenberg project and with it, the Block Editor is literally where all the innovation in the WordPress space is happening. Think of it this way: the only car race you’re going to win by using old technology, is a classic car race. If you want to win in SEO in the next few years, I guarantee you’ll need to be on the Block Editor. If you’re not, and if some of your competitors are, they’re going to beat you.

While the Block Editor may be very good, you may think: why would I switch? If the classic editor is working for me, so why bother? Well: the Block Editor is only step one in a longer process. More and more parts of the WordPress admin will start using blocks, and because of that, getting familiar with the Block Editor is essential.

Future versions will iterate on what the Block Editor already does, moving to site-wide editing, instead of just the content area. The first required step for that is defining content edit areas, something Matias discussed in this post on Make Core, one of the blogs of the core WordPress development team. That post by Matias prompted this post by Justin Tadlock on how the Gutenberg project is shaping the future of WordPress themes. This is getting me, and our entire team at Yoast, very excited.

The Gutenberg project aims at making WordPress easier to use. That’s a long term goal, but it’s already doing that now too. When we have site-wide editing, we won’t need to teach people how to use widgets anymore: they’ll be the same as the blocks they see in the editor. In fact, the entire distinction will be gone.

Reasons to use the Block Editor now

Besides all of these great developments, you really should use the Block Editor now and stop using the classic editor. Let me give you an overview of simple and clear reasons. With the Block Editor:

  • You will be able to build layouts that you can’t make in TinyMCE. Most of the stuff we did for our recent digital story required no coding. Plugins like Grids make it even easier to make very smooth designs.
  • You can make FAQs and HowTo’s that’ll look awesome in search results. Our Yoast SEO Schema blocks are already providing an SEO advantage that is unmatched. For instance, check out our FAQ blocks.
  • Simple things like images next to paragraphs and other things that could be painful in TinyMCE have become so much better in Gutenberg. Want multiple columns? You can have them, like that, without extra coding.
  • Speaking of things you couldn’t do without plugins before: you can now embed tables in your content, just by adding a table block. No plugins required.
  • Creating custom blocks is relatively simple, and allows people to do 90% of the custom things they would do with plugins in the past, but easier. It becomes even easier when you use a plugin like ACF Pro or Block Lab to build those custom blocks.
  • Custom blocks, or blocks you’ve added with plugins, can be easily found by users just by clicking the + sign in the editor. Shortcodes, in the classic editor, didn’t have such a discovery method.
  • Re-usable blocks allow you to easily create content you can re-use across posts or pages, see this nice tutorial on WP Beginner.

There are many more nice features; please share yours in the comments!

Finally, it’s good to know that WordPress 5.3, slated for November 12th, will have the best version of the Block Editor yet. If you want that experience now, you can! Just install Gutenberg: the plugin.

If you haven’t used the Block Editor recently: go, try it! I’m sure you’ll be happy with it.

The post The Block Editor/Gutenberg: Why you should be using it appeared first on Yoast.

User research: the ultimate guide

When you want to make some considerable improvements to your website, what’s the best place to start? At Yoast, we feel that research is always one of the most important things to do. It’ll help you find out what needs work, why it needs work, and of course, what you have to do to make things better.

Looking at our website data in Google Analytics, Google Search Console, and other SEO tools, is already part of our weekly activities. But, in order to dive deeper than just having a look at the plain data, we love to do user research. This is the part where you truly get to know your customers and where you’ll discover your blind spots when it comes to your own website. Within this ultimate guide, we’ll show you what types of user research could be valuable for your own website or company.

Table of contents

The top task survey

What kind of user research fits and complements the existing data always depends on the type of project you run. However, we believe that running a so-called ‘top task survey’ should always be the first step when you start doing user research. As you’ll able to use the outcomes of a top task survey within all future projects.

So, what is a top task survey exactly? To get to know why your customers visit your website, you’ll need to talk to your customers. And, how do you get to talk to your customers without actually having conversations with lots of customers? You could set up an online top task survey, which pops up on your visitor’s screen as soon as you like, either immediately after entering the website or after a couple of minutes. 

Questions in a top task survey

The popup is set up for one simple reason: to find out the purpose of their visit to your website. 

To make sure you’ll get valuable data out of your top task survey, it’s important to ask the right question. We recommend asking one open question: ‘What is the purpose of your visit to this website? Please be as specific as possible.’

With this open question, you give your customers the chance to truly say what they think. Closed questions make this harder, as you’ve already drawn up certain answers and then you risk missing other important thoughts or opinions your may customers have. And we know, analyzing the answers will take a lot more time, but when you do this right you’ll get the most valuable results.

Next to this one open question, it is possible to add one or two closed questions to take a closer look at your respondents. You might want to know the age or you want to know the type of customer it is. This data can be valuable to combine with the outcomes of the open question answers. In the top task surveys of Yoast, the second question is: ‘Do you have the Yoast SEO plugin?’. This is valuable information for us because we can see the difference between what free users are looking for on our website and what Premium users are looking for on our website. 

How often should you do a top task survey?

We recommend running your top task survey once a year. If you have a small website, you can choose to run the survey once every two years. The market you work in is always changing and customers always change, so every time you’ll run the survey, you’ll receive new, valuable information to work with and to improve on. 

The exit survey

The following two types of research we’ll discuss are more specific. And, which one you should perform at what time depends on the type of project you’re about to run. 

For example, you’ve noticed in your Google Analytics data that your most visited page has a very high bounce rate. This means that you need to know why visitors are leaving this fast. Couldn’t they find what they were looking for? Or did they find what they were looking for and are they already satisfied? You can get answers to these questions by running an exit survey on a specific page.

What is an exit survey?

An exit survey pops up when a visitor is about to leave the page. When a visitor moves their mouse cursor towards their browser bar, they are usually about to leave your website. So, this is the right moment to ask your visitor one or more questions. 

Questions in an exit survey

So, your visitor is about to leave, what do you want to know before they’re gone? We recommend keeping the survey short and simple: people are already leaving, so if you want them to fill out your survey, it needs to be short.

The question you ask depends on the page and the problem you want to solve. When you have a specific blog post with a high bounce rate, you might want to know if visitors have found the information they were looking for. The simple open question you could ask: ‘What information were you looking for today on our website?’. You could add a second closed question to see if the page fulfills your visitor’s needs: ‘Have you found what you’ve been looking for?’. A simple ‘yes’ or ‘no’ is enough to get this overview. 

Within our post ‘What is an exit survey and why should you use it?’ we’ve added some more examples of questions you could ask depending on the page that needs attention.

User testing

The third type of research is ‘user testing’. User testing is the type of research in which you get ‘live’ feedback from your clients because you actually see people using your website or product. At the beginning of this guide, we already mentioned ‘blind spots’ and user testing is the best way to find these blind spots. For example, you know exactly where to find what information or what product on your website, but visitors might not. Seeing testers struggle with finding the right page on your website can be embarrassing, but the good news is, when you know, you can improve!

Why should you do user testing?

User testing can give you very valuable insights during every stage of your process. When you’re creating a new product, it’s valuable to see what potential customers think of it, but it’s just as valuable to see what your customers think of your product that has already existed for over three years. Every test will give you new insights to work with!

User testing also guarantees that the test results are ‘real’. You can see for yourself how your customers use your website or product. The customer can’t ‘lie’ about things. And, that’s a big difference with survey respondents: they can say different things compared to what they really experience.

How to get started

There are three main types of user testing which you could use for your own website or product:

  • Live, moderated user testing: your testers will test with a moderator in the same room.
  • Remote, moderated user testing: your testers will test with a moderator while they’re in contact through a video call.
  • Remote user testing without a moderator: your testers will test without a moderator in their own time and space. They will record the test so you can watch it later.

Within our specific post ‘What is user testing and why should you do it?’, we explain more thoroughly what type should be used in what situation. 

After picking the method you want to use, it’s important to set up a clear plan with goals and a test scenario. Hereby you make sure the testers will follow the right path and will give you the insights you need. After that, it’s time to recruit your testers. Decide on what types of testers you’ll need to get the best test results. We recommend recruiting different types: young people, older people, experienced people, inexperienced people, etc. Think of all the types of people that might use your website or product now and in the future. 

Then it’s time to get started! Create a plan and start testing with your recruited people. Make sure you record all tests, making it easier to analyze the results. As it’s nearly impossible to remember everything that happened during the tests. 

Analyzing user research results

There is some difference in analyzing the results of surveys, such as the top task survey and the exit survey compared to the user testing results. 

When analyzing an online survey, we recommend to export all data to a sheet and to create categories for all answers. Place every answer into a specific category to get a clear overview of what the biggest problems are. After that, you can easily see what problems need to be prioritized and you can start thinking of improvements. Set up an action plan and start improving!

For the user tests, it can more difficult to create a couple of categories that fit the test results. Here, it’s easier to write a summary for every user test and to combine those at the end. Can you discover similarities? Can you combine some issues to improve more at once? It’s important to look at the bigger picture so you can make improvements that will have a big impact on the future user experience of your website or product!

User research tools

There are several tools in which you can create a top task survey or an exit survey (or other surveys!). We’re currently using Hotjar, but we’re planning to create our own design and implementing it with Google Tag Manager. Tools we know for setting up online surveys are:

  • Hotjar
  • SurveyMonkey
  • Mopinion

On their sites, they have a clear explanation of how to use these tools to perform an exit survey.

For user testing, your needs are different. Testing a website or a product, you’ll need a testing environment for your testers or a test product they are allowed to use. Besides, you’ll need recording material: for testing a website, you can easily record a screen session, but for testing a product, you’ll need to think of a recording set up. Do you have a good camera and a tripod, for example? Then you can get started! When you’re doing user tests more often, you can use an eye tracker as well to get more insights on how people are looking at your website or product, but it’s not necessary!

Are you already doing user research as well? Or have we convinced you to start doing user research? Let us know in the comments below!

Read more: Panel research for your business: Benefits and tips »

The post User research: the ultimate guide appeared first on Yoast.

Engaging your online audience: 8 practical tips

Before I started working at Yoast to develop our Yoast Academy training courses, I worked as a high school teacher. After starting my job at Yoast, I quickly realized that the educational principles I came across in my previous job were really powerful tools in the online world as well. In this article, I’ll explore three areas to help you engage your online audience: knowledge gaps, memory overload, and creating a connection. I’ll also give lots of practical tips to help you do better yourself.

The curse of knowledge

One problem that you’re likely to run into when maintaining a site or product is the so-called ‘curse of knowledge’. The curse of knowledge is a cognitive bias that suggests it’s more difficult for experts to explain things to beginners. The Wikipedia entry does a good job of rounding up some key research into the bias. Remember those times you had no idea what the teacher was trying to say and a classmate seemed way better at explaining those things? That’s most likely due to the curse of knowledge. The curse of knowledge is everywhere. And you’re very likely to suffer from it.

The problem: the more you know, the more difficult it is to create something that is clear and intuitive to your users.

Tip #1: Do user research

Before you can solve any problems, you have to identify them first. Get users
without any previous knowledge together in a room. Let them use your product or navigate your site. Find pain points and eliminate them. If getting people into a room is difficult for you, surveys can help as well. Ask people what parts of your product they found difficult to use and use that knowledge to improve it.

Tip #2: Add scaffolding

Actively providing extra context and bridging the knowledge gap between you and your users is crucial. ‘Scaffolding‘ is everything a teacher uses to help someone do something that they can’t do on their own. An essential part of scaffolding is thinking about what another person already knows and using that to help them do something new. That’s exactly what you should do as well. Some things you may consider adding:

  • Clickable question marks that clarify difficult terms; 
  • Internal links to articles that explain a concept you use in a more difficult article; 
  • Images that clarify what you’re trying to say;
  • Tutorial videos;
  • Live chat or email support;
  • Documentation / lessons / articles that your users can use to understand;
  • Step-by-step plans / flowcharts / instructions
  • An indication of the level of an article so users can make an informed choice to read or not to read an article. 

Always try to make this scaffolding as little invasive as possible. You don’t want to annoy more advanced users.

Tip #3: Audit your materials periodically

Often, it helps to look back on something you made at a later time. When you review something you wrote three months ago, you’ve already lost some of the context and perspective you wrote it with. Which, in this case, is a huge advantage!

When you audit your materials, make sure to consider:

  • Your intended message: do the materials convey it effectively?
  • The use of jargon
  • Assumptions about previous knowledge users have available to them
  • Scaffolding 
  • Readability

Minimizing cognitive load

When humans do things, they use a cognitive system called working memory. This memory saves information in our brains for a few seconds to a few minutes. It allows us to make sense of what we’re doing. Unfortunately, working memory is limited. It’s easily overstimulated. When it is, people get frustrated or distracted. This leads to them clicking away or growing tired of your product. Managing working memory is key to keeping your audience engaged. 

Tip #4: Less is more

As creators, we like to get fancy. We want what we make to be cool and fun. Sometimes, this leads to fluff features or content. Carefully consider: does what I’m adding make the whole better? If it doesn’t, remove it. Addition by subtraction is a very powerful tool for engagement. 

Tip #5: Pay a lot of attention to readability

One of the most common problems on websites is readability. Most copy is much more difficult to read than it should be. Writing shorter sentences and using fewer difficult words can help your usability tremendously. Even if your audience is smart, easy-to-read copy is very working memory friendly. It simply costs less energy to read. This energy can then be spent on more important things. One way to improve readability is by ruthlessly editing your copy. Ideally, you should spend more time editing your text than writing it.

Read more: How to use the readability analysis in Yoast SEO »

Tip #6: Break everything down into bite-size chunks

The working memory struggles with large blocks of information. The human mind needs focus, and it’s up to you to create this focus. Don’t write 30-word sentences or 20-sentence paragraphs. Don’t crowd your menu with 20 categories. Don’t stuff 20 options into one tab. It’s overwhelming. Break your materials down into bite-size chunks that are easy to oversee, so your users can focus on what really matters.

Creating a connection

I knew all the theory when I started teaching; that wasn’t the problem. But it wasn’t until I really started connecting with my students that I became a good teacher. One of the most powerful ways to engage your online audience is by creating that fuzzy feeling of comfort, familiarity and connection. And most sites and products don’t do a good enough job of this. Of course, the first requirement is a usable product or site. There are lots of extra things you can do, though, to help reinforce your relationship with your user.

Tip #7: Invest in design and branding

It’s tough to overstate the power of consistent design and branding. Our Yoast avatars are a great example. All over the WordPress community, our avatars are immediately recognized as they stand out from the crowd in e.g. lists of speakers at conferences. The same goes for the images we use in posts and presentations. Providing your users with a similarly positive experience over all the different places where they interact with you, helps you get recognized and valued.

Tip #8: Use the power of storytelling

Stories can be an incredibly powerful medium to make a connection with your audience. Most people remember one or more teachers who were always able to get them on the edge of their seat with great stories which helped them remember what the teacher was trying to explain. Stories and narrative are how people connect and communicate with each other. And storytelling isn’t necessarily about writing a large piece of fiction. You can just as easily hide little nuggets of storytelling in your blog posts or product pages. Yoast CEO Marieke has written a great series on storytelling that you should definitely check out.

Conclusion on engaging your online audience

The tips listed are a collection of insights I gained through my experience as a teacher, product owner and online writer. There are lots and lots more things you can do to make sure your online audience stays engaged. But honestly, if you get all of this right, you’re probably a fair number of steps ahead on almost everyone. Good luck! 

Keep reading: The ultimate guide to content SEO »


The post Engaging your online audience: 8 practical tips appeared first on Yoast.