# Generating a sitemap.xml file **URL:** \ **Category:** Hosting Support\ **Created:** [February 8, 2025, 9:35pm UTC](https://forum.infinityfree.com/t/generating-a-sitemap-xml-file/105657 "2025-02-08T21:35:24Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Andyserenity](https://forum.infinityfree.com/user_avatar/forum.infinityfree.com/andyserenity/32/49317_2.png) [@Andyserenity](https://forum.infinityfree.com/u/Andyserenity)\ **Post date:** [February 8, 2025, 9:35pm UTC](https://forum.infinityfree.com/t/generating-a-sitemap-xml-file/105657/1 "2025-02-08T21:35:24Z")
### Website URL [https://anothercloudyday.com](https://anothercloudyday.com) ### Error Message The sitemap generator ([https://www.xml-sitemaps.com](https://www.xml-sitemaps.com)) reports that the crawler bot is being blocked when I submit my homepage URL. ### Other Information My site has 250 pages. Generator support reports that the bot is getting blocked by Cloudflare. I disabled Cloudflare and re-ran and the bot reports that another block is in place, with the following crawler bot response: HTTP/1.1 200 OK Server: nginx Date: Sat, 08 Feb 2025 19:06:41 GMT Content-Type: text/html Content-Length: 841 Connection: keep-alive Expires: Thu, 01 Jan 1970 00:00:01 GMT Cache-Control: no-cache ![image](https://forum-cdn.infinityfree.net/original/3X/e/3/e32784e9dda0f90cedeee043be70140f68605f45.png) My site does not directly use javascript. The error report shows one of my webpages “news.html.” I don’t know why it’s showing that. I removed the link to it from my homepage and re-ran the generator to no avail. I have no clue where the block is coming from nor the javascript / aes.js references. Thank you. Bing Webmaster Tools also reports that the site scan will not work because I am using dynamic html. I am not to my knowledge. I honestly don’t know how to tell. I’m a novice, sorry folks. ---
**Author:** ![dan3008](https://forum.infinityfree.com/user_avatar/forum.infinityfree.com/dan3008/32/68577_2.png) [@dan3008](https://forum.infinityfree.com/u/dan3008)\ **Post date:** [February 8, 2025, 9:45pm UTC](https://forum.infinityfree.com/t/generating-a-sitemap-xml-file/105657/2 "2025-02-08T21:45:07Z")
probably something to do with this: > [@Ensuring only web browsers can access your website](https://forum.infinityfree.com/t/ensuring-only-web-browsers-can-access-your-website/49353): > > InfinityFree is a website hosting service. That means that the hosting accounts we provided are intended for hosting websites. Websites contain pages that are accessed through web browsers. InfinityFree is not intended to be used for file sharing, API hosting, database hosting or background tasks/tools. To help enforce this, free hosting enforces a security system that makes sure that anyone trying to access your website is using a normal web browser. This is done by checking whether the web br… unfortunaitly its just one of the limits on free hosting ---
**Author:** ![Andyserenity](https://forum.infinityfree.com/user_avatar/forum.infinityfree.com/andyserenity/32/49317_2.png) [@Andyserenity](https://forum.infinityfree.com/u/Andyserenity)\ **Post date:** [February 8, 2025, 11:08pm UTC](https://forum.infinityfree.com/t/generating-a-sitemap-xml-file/105657/3 "2025-02-08T23:08:49Z")
Thank you Dan. I remember dealing with security limitations by incorporating Cloudflare. Does it not allow for things such as sitemap.xml generation? That was a few months ago. It seems like things are different now. ---
**Author:** ![Meishin](https://forum.infinityfree.com/user_avatar/forum.infinityfree.com/meishin/32/59902_2.png) [@Meishin](https://forum.infinityfree.com/u/Meishin)\ **Post date:** [February 8, 2025, 11:10pm UTC](https://forum.infinityfree.com/t/generating-a-sitemap-xml-file/105657/4 "2025-02-08T23:10:03Z")
Yes, things have changed. Cloudflare is no longer exempt from the security system. ---
**Author:** ![Jri-Creator](https://forum.infinityfree.com/user_avatar/forum.infinityfree.com/jri-creator/32/65654_2.png) [@Jri-Creator](https://forum.infinityfree.com/u/Jri-Creator)\ **Post date:** [February 9, 2025, 1:38am UTC](https://forum.infinityfree.com/t/generating-a-sitemap-xml-file/105657/5 "2025-02-09T01:38:36Z")
iFastNet, InfinityFree’s hosting provider, patched the AES.JS > [@My site (using Cloudflare) is showing aes.js challenge page](https://forum.infinityfree.com/t/my-site-using-cloudflare-is-showing-aes-js-challenge-page/104501/10): > > It does appear that iFastNet changed how the testcookie system interacts when using Cloudflare. IIRC they were incompatible before, so iFastNet made an exception for Cloudflare so it would bypass the testcookie check. But it appears that iFastNet has now removed this exception and instead fixed the testcookie check so it works with Cloudflare. iFastNet didn’t communicate about this, so right now we can only speculate why they did this. But I can personally attest that Cloudflare can be astonis… ---
**Author:** ![Admin](https://forum.infinityfree.com/user_avatar/forum.infinityfree.com/admin/32/36437_2.png) [@Admin](https://forum.infinityfree.com/u/Admin)\ **Post date:** [February 9, 2025, 12:55pm UTC](https://forum.infinityfree.com/t/generating-a-sitemap-xml-file/105657/6 "2025-02-09T12:55:57Z")
If I may add my two cents: sitemap generator sites are useless anyway. The point of a sitemap is that it helps search engine crawlers understand the structure of your site. But you have to understand that search engine crawlers are quite sophisticated and will probably understand you site pretty well without it. When you use an online sitemap generator, you’re basically using a rudimentary crawler to generate a structure that should give extra information to a much more sophisticated crawler. The added value of sitemaps is for things a crawler cannot easily guess. So sitemaps only make sense if they are either crafted by hand or generated by your website building software (which knows more about the structure of your site than any crawler). Your website will get crawled and indexed just fine without a sitemap. If you decide you’re going to add a sitemap anyway, make sure that it actually adds value and isn’t just junk spat out by one crawler being fed into another crawler. ---
**Author:** ![Andyserenity](https://forum.infinityfree.com/user_avatar/forum.infinityfree.com/andyserenity/32/49317_2.png) [@Andyserenity](https://forum.infinityfree.com/u/Andyserenity)\ **Post date:** [February 9, 2025, 3:05pm UTC](https://forum.infinityfree.com/t/generating-a-sitemap-xml-file/105657/7 "2025-02-09T15:05:14Z")
Thanks to all who helped with my issue. Dan replied first so he gets the checkmark. ---
**Author:** ![system](https://infinityfree-forum-uploads.s3.dualstack.eu-central-1.amazonaws.com/original/3X/b/0/b024fd3fa88bae093c9576e55c2c94e33f890206.svg) [@system](https://forum.infinityfree.com/u/system)\ **Post date:** [February 16, 2025, 3:05pm UTC](https://forum.infinityfree.com/t/generating-a-sitemap-xml-file/105657/8 "2025-02-16T15:05:53Z")
Closed on February 16, 2025, 3:05pm UTC This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.