Robots.txt is unreachable

[zeussss]

Website URL

basc.infinityfreeapp.com
(please specify the URL of the site on which you are experiencing the problem)

Error Message

Google Search Console throws an error Robots.txt is unreachable. but i can access it via browser. I also triad renaming it to “robots.txt” but when i tried accessing it via browser it throws a 404 error. Any help will be much appreciated. Thanks!
(please share the FULL error message you see, if applicable)

Other Information

(other information and details relevant to your question)

[KangJL]

@zeussss

Google Search Console throws an error Robots.txt is unreachable. but i can access it via browser

Likely due to this

Ensuring only web browsers can access your website

InfinityFree is a website hosting service. That means that the hosting accounts we provided are intended for hosting websites. Websites contain pages that are accessed through web browsers. InfinityFree is not intended to be used for file sharing, API hosting, database hosting or background tasks/tools. To help enforce this, free hosting enforces a security system that makes sure that anyone trying to access your website is using a normal web browser. This is done by checking whether the web br…

@zeussss

I also triad renaming it to “robots.txt” but when i tried accessing it via browser it throws a 404 error.

No issue

Take note free hosting is case sensitive.

Redirected to InfinityFree 404 error page

While working on your website, you may be redirected to the InfinityFree 404 error page. This article explains why this typically happens and what you can do to fix this. Why do I get a 404 error When you try to access a file on a website, the server tries to find the matching file to load. By default, it works like this. Suppose you have a website http://example.com which is linked to the folder htdocs in your hosting account. If you then try to load a certain URL, like http://example.com/blo…

[zeussss]

is there other way to make this site indexed on google?

[Admin]

Your site should be indexed just fine. Having a robots.txt file is not mandatory, and if you don’t have one, search engines should assume that all pages are safe to index.

No need to find alternatives for something that just works.

[anon95807532]

Hi zeussss,

Google search console throws many issues that might not even be correct sometimes, and for your case, not having a robots.txt detected isn’t a problem at all. The robots.txt file is simply used to indicate bot preferences and places where you would not want a bot to scan or crawl. However, this does not mean that the bot necessarily follows the rules therefore consider it more of a commodity than security.

The only correct naming and case for the filename is robots.txt, plural and lower-cased.

@zeussss

is there other way to make this site indexed on google?

You can submit a sitemap via GSC to let google know your site structure, one site to generating a sitemap is:

Create your Google Sitemap Online - XML Sitemaps Generator

You can then download the generated sitemap and upload to GSC for processing. Submitting a site map does not necessarily mean it will get indexed as the content quality is also considered by Google.

Cheers!