Blocked by robots.txt

Website URL

techsupportstories.com
(please specify the URL of the site on which you are experiencing the problem)

Error Message

Blocked by robots.txt These pages aren’t indexed or served on Google
(please share the FULL error message you see, if applicable)

Other Information

15 of my pages are not indexed due to being blocked, but i dont have a robots.txt file. I do not know if infinity has its own robots.txt, and if so, are there any options for me?
(other information and details relevant to your question)

list of affected URLs
|https://www.techsupportstories.com/kb/slow.html|Oct 7, 2024|
|https://www.techsupportstories.com/windows/..phones/index.html|Oct 7, 2024|
|https://www.techsupportstories.com/hardware/Uncat/index.html|Oct 7, 2024|
|https://www.techsupportstories.com/security/Uncat/index.html|Oct 7, 2024|
|https://www.techsupportstories.com/phones/Uncat/index.html|Oct 7, 2024|
|https://www.techsupportstories.com/phones/windows/index.html|Oct 7, 2024|
|https://www.techsupportstories.com/Phones/index.html|Oct 7, 2024|
|https://www.techsupportstories.com/hardware/..phones/index.html|Oct 7, 2024|
|https://www.techsupportstories.com/security/..phones/index.html|Oct 7, 2024|
|https://www.techsupportstories.com/Uncat/index.html|Oct 6, 2024|

How do you know the pages are being blocked? What do you see that tells you this?

Google decides when and how Google wants to index pages. And it can take a long time, especially for new sites with little activity, for new websites and new pages to be crawled and indexed.

So if the pages you created a few days ago are not visible yet, that’s probably completely normal and just means you need to have patience for Google to check your site.

4 Likes

I copied and pasted that straight from GSC

InfinityFree does not provide any default robots.txt file. There is a security system which blocks most non-browser connections, but Google’s crawler is sophisticated enough to bypass that.

5 Likes

@davemill8

The links you put on the forum and probably submitted to Google are not exist
because all your articles (html files) are on this path

https://www.techsupportstories.com/kb/articles/ <- HERE

example
https://www.techsupportstories.com/kb/articles/slow.html

So please pay attention that in the KB folder you have a subfolder called “articles”

and not on:


image


it is probably the same for the other listed URLs (some are not even correct)

Many of the folders listed in your list don’t even exist

Here you have the capital letter W (Windows) and others…

but it is actually lowercase https://www.techsupportstories.com/windows/index.html

and so on…

First check and correct all your code and only then deal with GSC

6 Likes

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.