
How to Verify Website in Google Search Console (2026)
Verification proves to Google that you own your site. This guide walks through the DNS record, HTML file, HTML tag, Analytics and Tag Manager methods, and what to do when verification fails.

Last updated: November 2026
To test a robots.txt file, check it before it goes live, not after. The safest way is Google Search Console’s robots.txt report, which shows you the file Google actually fetched and flags any rules it cannot read. A free online tester gives you a second opinion, and the URL Inspection tool lets you check one page at a time.
A bad robots.txt file can hide your whole site from Google in minutes, so testing is worth the five minutes it takes. This guide shows the three testing methods in order, what to check in each one, and the mistakes that cause the most damage.
Robots.txt is a small text file that tells search engines which parts of your site they may crawl. One wrong line can block your whole site, your images, or the CSS and JavaScript files Google needs to read your pages properly. Our guide to what robots.txt is explains the file and its rules in full, and our guide to how to create a robots.txt file shows how to write one.
Testing matters because the file is easy to get wrong. A missing slash, a rule that is too broad, or a typo in a folder name can block far more than you meant. Google also caches the file for about 24 hours, so a mistake stays live for a day even after you fix it. Test first, publish second.
This is the best test, because it shows the file Google actually fetched from your site, not the file you think is there.
Check the status code first. A 200 means Google found the file. A 404 means there is no file at all, so Google crawls everything, which is fine for most small sites. A 5xx error is the dangerous one: when Google cannot fetch the file because of a server error, it may stop crawling your site entirely until the error clears.

The robots.txt report tells you about the file. The URL Inspection tool tells you about one page, which is what you need when you change a single rule.
Use this method after every change to a rule. It takes a minute per page, and it catches the broad-rule mistakes that the file-level report misses.
Online testers are useful when you want to test a new file before you upload it, or when you want a second opinion. Paste your draft file and a test URL, and the tool tells you whether that URL is allowed or blocked.
Good options include the robots.txt tester at technicalseo.com and the validator at searchenginereports.net. Both are free and need no account. Type or paste your file, enter a URL from your site, and check the result. Test several URLs: your home page, a product page, an image, and a CSS file.
Do not rely on a tester alone. Testers read the file you paste, not the file on your server. They cannot tell you whether the live file has a typo or whether Google fetched an old cached copy. Use them to check a draft, then confirm with Search Console after the file is live.
Run through these checks every time you change the file:

Blocking the whole site by accident. The rule Disallow: / blocks everything. It is useful on a staging site, but it has taken live sites out of Google more than once. Never put it on a live site.
Blocking CSS and JavaScript. Old advice said to block these files. Google now renders pages like a browser and needs them. Keep them allowed.
A file over 500KB. Google ignores everything after the first 500KB of a robots.txt file. A file that big is almost always a sign of a problem, such as thousands of generated rules. Keep the file short.
Forgetting the 24-hour cache. Google caches robots.txt for about a day. A fix you upload now may not take effect until tomorrow. Do not keep changing the file and wondering why nothing changes. Fix it once, test it, and wait.
Using robots.txt to hide a page from search results. Blocking a page stops Google from crawling it, but Google may still show the URL in search results if other sites link to it. To keep a page out of search results, use a noindex tag instead.
Use the robots.txt report in Google Search Console: Settings, then robots.txt, then Open report. It shows the file Google fetched and flags any errors. The URL Inspection tool checks single pages, and free online testers let you check a draft before you upload it.
Open Search Console, pick your property, and click Settings in the left menu. The robots.txt section is there, with an Open report link. It shows the fetched file, the fetch date, and any warnings or errors.
Google caches robots.txt for about 24 hours. You can ask for a faster recrawl from the robots.txt report by clicking Request a recrawl, but the new rules usually take up to a day to take full effect.
That is fine for most small sites. When there is no file, Google crawls everything it can reach. You only need a file when you want to block something, such as an admin area, a search results page, or a staging copy of the site.
Paste the page URL into the URL Inspection tool in Search Console. The Page availability section says whether the page is blocked by robots.txt and shows the rule that blocked it. You can also test the URL in a free online robots.txt tester.
Testing a robots.txt file takes five minutes and can save you from a costly mistake. Check the file in Search Console, test one page at a time with URL Inspection, and change one rule at a time. The file is small, but it controls what Google can see, so it deserves the check.

Verification proves to Google that you own your site. This guide walks through the DNS record, HTML file, HTML tag, Analytics and Tag Manager methods, and what to do when verification fails.

A canonical tag tells Google which version of a page is the main one. Learn what it is, when to use it, and how to add and check it in WordPress.

A simple 12-step guide to writing content that ranks on Google and reads well — with free tools, a worked example, and the mistakes to avoid.