FTP Errors, Missing Files, and Websites Showing Connection Refused
This topic is to merge the two related current outage topics from here and here.
Starting early November, several servers got overloaded with new accounts, causing a lot of stress on these servers. Because these servers were still using an older HDD-based storage cluster, instead of the newer SSD-based cluster that’s powering most websites nowadays, the storage could not keep up causing a lot of impact to all sites hosted on them.
Trying to patch up the old HDD-based system is not sustainable, so the decision was made to deploy new SSD storage capacity and migrate all accounts to them, which will provide better stability and performance when all of this is done. Unfortunately, migrating hundreds of thousands of accounts from an overloaded storage system is not easy and will take some time.
Which components are affected by this issue
The following hosting volumes and IP addresses are affected by this issue:
- Hosting Volume
vol17_1
/ Website IP185.27.134.33
[done]. - Hosting Volume
vol17_2
/ Website IP185.27.134.34
[done]. - Hosting Volume
vol18_1
/ Website IP185.27.134.55
[done]. - Hosting Volume
vol18_2
/ Website IP185.27.134.57
[done].
What issues may you see
On the IP addresses 185.27.134.55 and 185.27.134.57, you may see intermittent outages on your website, often seen as various kinds of Connection Refused errors (e.g. ERR_CONNECTION_REFUSED
, 502 Bad Gateway
, 520 Web Server Returned an Unknown Error
).
On the IP addresses 185.27.134.33 and 185.27.134.34, you may see a 404 Not Found
error on your website, and an empty file list through FTP and the file manager, or a 421 Home directory not available - aborting
error in FTP.
When will this issue be resolved
Unfortunately, migrating hundreds of thousands of accounts from an overloaded storage system is not easy and will take some time.
The migration of vol17_1
is nearly done and the final data should be copied over within the next few days.
After that, the migration of vol17_2
will continue. It’s hard to predict how long this will take, but it could be a week or more.
After this, the remaining hosting volumes will be migrated. Again, it’s unclear how long this will take, but it will take weeks.
What can you do to fix this
Which options you have depends on the current state of the hosting volume of your account.
If your hosting volume is already being migrated, then most likely you won’t have access to your website files. If you have a copy of your own website files, then you can move your domain to a different hosting account and reconstruct the website there. If you aren’t able to reconstruct the files yourself, you have no option but to wait. Your files will appear again automatically.
If your hosting volume is not being migrated yet, then you still have full access to your files, which means you can migrate your website to a new account. Please see this article for details on how to do that.
If you don’t care about migrating your current website, then of course you’re free to remove the domain from your current account, create a new account for the domain and set up your new website there. You can do this regardless
If you don’t mind waiting, then you can just wait for the migration to complete. If you don’t do anything, your website will automatically be restored after some time.
We are aware of the issue and are doing our best to get this resolved as soon as possible.
I completely understand that this may cause confusion and you will have additional questions. I am happy to provide any information I have to help you.
However, to keep the contents of this topic clean, clear and concise, I ask that you carefully read the Outage Communication Rules before posting.
In a nutshell:
- Please keep the posts about the outage in this topic, and all posts not about this outage outside of this topic.
- Please post relevant questions only that have not been answered yet. It may be compelling to ask obvious questions like “when will this be fixed”, but basic information like that will be put in this message as soon as we know it.
- Please carefully check the known symptoms and affected components.
- If you are on the affected components, you are almost certainly affected by the outage. If so, please refrain from simply stating you are affected, it does not help get your service back up faster.
- If you are not on the affected components but do experience all of the symptoms, please say so below so we can check if our list is incomplete.
- If you do not experience all listed issues and are not on the components that are known to be affected, the issue you are experiencing is most likely not related to this outage. If so, please create a new topic in the Hosting Support category and describe the issue there.
- If you say you are affected by the outage, please always include your website URL. I can’t check if you are affected by the outage if you don’t say which of your websites is affected.
Posts that do not follow these rules may be deleted without warning. This is to help ensure that all the relevant information is easily available to everyone reading this topic.