Just before pasting in documentation links and re-emphasizing the importance of having a security measure in place, I would like to state my intention to start a technical discussion on potential improvements and why the existing method needs enhancements. I’m not here to break things nor to get into capitalism discussions about why free hosting has to add something that gets in the way or whether API should be considered banned whatsoever. I have no opinions on how the rules are being set here as they have to exist for a reason.
With that being said, I spend some free time studying the code from an enthusiast’s perspective (finally after 5-6 years since this was implemented). I do notice that the JavaScript challenge is there to test JavaScript capabilities, however, the challenge does not involve browser-specific aspects and anyone who manages to answer the challenge programmatically server-side can simply bypass this. There are 2 ways of doing that as of now.
As far as I know, the challenge is set to prevent direct crawling and bot access to websites that are hosted here and use the platform for large-scale API things, but this is also at the expense of SEO and makes it super easy for others to identify the hosting location of a certain website (if that matters to some: businesses/professionals/start-ups). Meanwhile, for many vendor-originated features that require loopback, ajax might break if the ?i=1 mattered to those.
With the 2 methods available, I do not think this is going to stand long as it simply cannot serve as a permanent fool-proof security. With challenges shown in JavaScript source code (which is a must) and browser emulation being always possible, the i=1 serves as a minor blockade for less-resourceful crawlers or mindless pings.
It doesn’t mean it should be removed entirely now, but something else could serve as a replacement in the future. Having the crawler do some effort is still a way of slowing them down, or rendering the crawl useless without providing the info they want.
To honour the security profession, the said methods will not be disclosed for the time being unless requested by admin. Other professional IT persons may have already found similar/same solutions, if without other constraints on the hosting platform here, i.e. with an external host.
Having those in mind, I would like to start a brainstorming topic here on the direction of effective rate-limiting solutions, or IP intelligence-based connection approval mechanisms.
Please note that this is not a post to encourage any type of exploitation or TOS violation. The target scope of discussion should center within a technically reasonable scope, not everything has to be political.
I agree with you, the “security” system is already flawed in many ways which I will refrain from mentioning it.
Although I do not think the whole thing is related to Capitalism, I find the reference hilarious.
In addition - they internally block more than 800k IP addresses
Instead of cookies and JS
I think it would be best to have some AI firewall in the near future
which will monitor and analyze packages and automatically make decisions, and along the way learn… and apply rate limiting and other things as soon as something/someone becomes aggressive.
Will it ever come to life on the free service and is it financially worthwhile for them; that is another question.
They will probably keep the model as it was 10 years ago (aes.js)
as well as the control panel, because I assume that the free service does not bring any significant money, so they would start creating a new system from scratch that would be more modern and adequate for today’s challenges.
That’s just in case, I think most of us here won’t associate the topic, but well if you have read some other post before then you’ll know what I mean by that.
Hi Oxy,
Blocking IP addresses is something that I would do too for cloud providers as they tend to sell VPS to people who are going to abuse them, nor would those networks bring real human traffic anyway. The same struggle for all website hosts is to maintain and block as many IPs.
If InfinityFree and iFastNet can remove the nameserver checks then I guess many will benefit from Cloudflare’s free plan with more flexibility, however, that brings on the topic of managing and terminating phishing and spamming domains.
Hi Supernova3339,
This is funny, having SSH access is more or less near bare metal depending on the access level gained. That being said tho, SSH also implies FTP and possibility for executables.
The testcookie system is indeed a rather crude security measure. It’s neither the best at being seamless to visitors nor being perfect at stopping bad traffic. However, it does strike a reasonably good balance between those things I think.
To compare, with Cloudflare you can either have regular mode, in which Cloudflare does some kind of intelligent threat detection to decide who to block, who to challenge and who to let pass. This is the most visitor friendly solution, but I have personally experienced attacks where Cloudflare stopped some traffic, but still let enough traffic through to overload the backend servers. I think our testcookie system is more effective at stopping bad traffic.
If you actually see you’re under attack, you can enable “I’m under attack” mode, which will present every visitor with an interstitial page that does some Javascript challenge. This is very effective, but is also much more intrusive to visitors. And this also blocks all API traffic, hotlinking, etc. like our solution does.
And this is Cloudflare we’re talking about. A very big company which has web security as the number 1 thing they do. If Cloudflare can’t get it right, what hope is there for a relatively small company like iFastNet, let alone InfinityFree?
Building what you’re suggesting is really complicated, and it may be out of reach for iFastNet to build such a thing. And you have to remember, this security system is for free hosting only, not premium hosting, and putting a lot of effort into something that won’t help their paying customers doesn’t seem like a very good business decision.
iFastNet is of course aware that it’s not 2003 anymore and that the hosting industry is evolving. But I don’t think having a better security system is the kind of ace in the hole that’s going to help make sure the service is relevant in the year 2040.
Everyone who has a custom domain can already use Cloudflare, and people with free subdomains cannot use Cloudflare because Cloudflare doesn’t accept subdomains. I don’t see how removing the nameserver checks will allow people to use Cloudflare who can’t use Cloudflare already.
Indeed, Cloudflare isn’t always the ideal solution here as they still allow very questionable traffic through for unknown reasons (probably because of the free plan, lol). Some notorious hostnames are somehow allowed unless I explicitly add those to the WAF to block them out, that’s with an assumption that they do not change network registration or the entire network in question and use someone else’s network.
As for the interstitial page, it really depends on the website WAF configuration whether this is skipped on those routes and implemented other checks on API endpoints. Usually, those are authenticated with tokens and rate limits instead. Having said that, I do agree that there are big companies that still do not do this correctly and have mobile applications come to a standstill when they block the API endpoints with the Cloudflare challenge page thing. However there are also successful cases where the application is programmed to complete that challenge or have some kind of bypass like method 1 to get through the security for getting the job done, it’s rather on the client application side than the hosting side for this to work as intended.
I have no comment on this one, as this is not related to technical, but more about resource allocation and business directives.
Except for people with domains first registered with Cloudflare and cannot have their nameservers changed (Ooof …) As for free subdomains, it’s more a business bureaucracy thing as subdomains are also DNS records, however allowing a well-known free hosting provider to change subdomain records all the time may result in them handling a lot of administrative processes, or at least a large chunk of traffic on top of their free tier, that’s why.
Cloudflare actually has an API to edit zones for TLDN owners, including the ability to CRUD DNS records upon API hit, meaning subdomains can be automatically covered when a website or subdomain is created from the control panel or client area if implemented correctly, assuming Cloudflare lets you do it for this use case and does not manually email you about it. They do start assigning customer sales emailing you about your increasing website counts and traffic meters and try to see if you plan on upgrading to one of their paid plans.
I don’t really agree that testcookie can be good enough to prevent attacks. If the attacker puts a little effort then it is bypassable in the easiest way possible. (Considering that there are tools to execute the page js code without using a browser)
I think the admin has already admitted to this by also mentioning the business cost and relevance issue at hand. It would be better if we focus on the technical improvements possible to move this forward, this can be from different perspectives, i.e. the hosting level, or website level of things.
I can confirm that PoCs exist and I would suggest editing “attacker” to “someone”.
This is method 2, browser emulation, which is also immune to algorithm changes but it does take the client more resources to get this running, at least a separate host is required.
I speficially said attacker due to the context regarding DDoS attacks, of course anyone could try to exploit it for any reason but when you abuse it in order to launch DDoS attacks then you get that label.
I can confirm this too. Source: the fact that we had a working cron job tool.
It may be that it doesn’t have anything to do with free plans, and just with the fact that reliably determining whether traffic is legitimate or not is really hard, especially if you have to decide this while also trying to serve a page quickly.
Authentication checks and backend side rate limits are very important to have, but when the traffic really starts picking up, it’s not going to save you. If you get 100x the usual traffic you have, having to run your application code to check the authentication is still consuming server power.
Rate limits in the application code suffer from the same issue, but you can do rate limits on the web server side too. Free hosting actually has that already, but configuring rate limits to it helps anything against bad traffic while still limiting the impact to real visitors is really hard. And it’s not very useful against DDoS attacks due to the number of IPs.
Having more granular controls helps of course, I know many websites that has the under attack challenge enabled for their login pages by default to protect against brute force attacks (same reason why the client area login page has a CAPTCHA too).
I really don’t agree with the notion that “this just works, you just need to configure it correctly”. If it was simple to do at scale, Cloudflare could have done it correctly.
You’re missing the fact that resources are finite. You can’t just make the next Cloudflare killer based on motivation alone. Cloudflare has over 3000 people working there on a product that’s basically just web performance and security. I don’t know the number of employees iFastNet has, but I expect it to be somewhere in the XX range. They simply don’t have the same level of expertise and manpower to make something like that.
Yes, that’s a problem, but not the subject of this discussion. This discussion is about website security system, not servicing people who chose a reseller which provides unusually little control over the domain you purchased. And now devolved into whether people are able to use Cloudflare on our hosting, which they still can if they transfer their domain.
Offering Cloudflare by adding customer domains to our own zones is a dirty hack. Just because it works doesn’t mean it’s a good idea. The right solution allows everyone to add their subdomains as their own domains to Cloudflare so they can manage all Cloudflare settings on their own website.
And we actually had this for a while when we still had Cloudflare integrated in the control panel.
The main problem we ran into here was with abuse. Free hosting gets abused a lot. If Cloudflare sees that one of your subdomains is clearly being used for something bad, they’ll just drop your entire domain name. That’s tens of thousands of legitimate domains getting take down.
I don’t know which provider has this still. We don’t have it anymore for good reason.
Cloudflare’s current course is to abandon hosting provider partnerships. The cPanel extension hasn’t been maintained in 5 years, they don’t accept new partnerships, partial zone integration has been removed, and Railgun will be removed next year.
Because why would they? When Cloudflare started out, the partnerships were useful because it allowed them to gain traction quickly. Nowadays, their main business is big enterprises, and even for smaller customers, it’s just much more effective to have a direct relation with them. And Cloudflare is bigger and more well known than any web hosting provider out there.
I actually tried to integrate Cloudflare in the client area, but Cloudflare wouldn’t have it.
The thing to consider is that it must be done on the hosting level, and it must be plug and play. Leaving it to the website owners to secure their own websites will result in the ones who need security most to have the least of it. Many people are building their first websites here but don’t really know that much about security yet, and unknowingly leave massive holes in their site with glaring security flaws. Having something that prevents such projects from getting brutally abused is necessary.
I realize my previous post is getting a bit defensive. I don’t think the system is as a bad of an idea as it appears to be considered sometimes, but I do acknowledge its flaws.
If we can share some good idea on how to make it better, or what could be used to replace it, it could certainly lead to some great ideas that could make the service better.
Please do consider the following advantages of the testcookie system:
It’s really effective at stopping basic attackers (which most of them are), helping to protect less secure sites against crawlers that look for unsafe sites to exploit.
The challenge is run before the traffic actually hits the account of the user, meaning any traffic that’s stopped is stopped before it generates hits, CPU usage and so on on the affected site, leaving more account resources to legitimate traffic.
It’s straight forward to maintain. It’s just a module that’s installed in the server. No need to constantly check logs, analyze what traffic gets through and what doesn’t, and use that to manually tune identification rules, IP blacklists.
It’s pretty unobtrusive to regular browser traffic. The false positive rate for regular browsers is basically zero as long as the module works as intended. IP based blocklist, traffic rules and rate limits all have false positives.
It works with sites of all kinds. No need to individually tune it for specific sites and their traffic patterns. This is very important with rate limiting.
No dependency on external services to work. Especially those with unfavorable per-site billing or poor support for free subdomains.
There are probably better alternatives to what we have right now. But it would be nice if they could at least maintain some of the advantages of the current solution.
Please remember how long it took for sometimes rather basic changes, like a PHP upgrade, to go through, and how much dissatisfaction there is about account suspensions, both temporary and permanent. I personally really don’t want it to be big and complicated, because it will take long, and I don’t want something else that’s vague, fuzzy, impossible to explain or reason about but still blocking people for no apparent reason. The solution being simple and clear is a huge benefit in my perspective.
For other companies, I would say this excuse is acceptable, not for Cloudflare though as they have a much more detailed global view than any companies that involve the internet. If they are the #1, they have to be able to do this. Besides, even if they cannot, a simple API call to AbuseIPDB would easily reveal if a certain IP is involved in malicious activities in a narrow time window.
This is more an implementation problem, if you count hits before authentication, then basically you save processing power by simply counting hits per IP with a fast key-value storage, if the count is over a certain threshold, just API-call Cloudflare to block the IP, or simply block it locally. This can all be done before any authentication process takes place, especially if that involves querying MySQL.
I will rule out DDoS as that does not necessarily involve the implementation, but rather sucking up as much of the available bandwidth as possible, it can completely ignore whatever service you have and send a heck ton of TCP (worse still partial) packets and call it a day. It’s something to be handled via Cloudflare or firewalls for sure.
They will do it, but only in enterprise plans, free plans are very limited. They only block those that are very well-known malicious and protect the website from major attacks, other than that websites’ are on their own. There is a reason that their IP headers are separated into real IP, true IP and simply connecting IP. It’s because they have the ability to get those data as a global ISP, it’s whether they want to let you have that same access, i.e $$$.
I was just trying not to bring up about the manpower part to be polite and leave it as general as possible. No single company including Cloudflare has enough manpower for this. Security is something that costs but does not generate revenue, but it keeps the business surviving. Similar to water drinking, if you know what I mean, it doesn’t give you energy but you can survive longer (and hydrated). (and I’m looking at a noodle soup while I was typing this… why am I hungry at this time…)
This one is understandable from Cloudflare perspective, they simply cannot. Allowing subdomain registrations would incur inheritance issues, especially when they belong to different persons. Let’s say if A bought www.website.com and B bought website.com. Should Cloudflare take instructions from A or B if they both want to set an A record for www.website.com. In theory B owns *.website.com, but A has everything in the scope of www, which one should take precedence given they both paid the same fees. They simply cannot decide on this one.
Completely understand your points regarding Cloudflare. As for this bit: Modern business my friend.
Indeed this is a good rationale, while the i=1 is somewhat positioned in the middle of enabling genuine good possibilities and somewhat limiting the possibilities, if you get what I mean. I was just wondering if there’s a better way to know that the current system could be bypassed at somewhat resource on the client-side.
It’s ok, nothing on this planet is perfect. The points you have mentioned are actually valid and yes these should be the standards if a new solution is to be suggested or at least be in this direction. Personally, I think the last 3 are the most important ones. Given this is a free hosting platform, things can easily complicated if we have to suit everyone’s needs.
Haha true, but those words are magical and cause them to fix something immediately. ( as SSH on free hosting via JavaScript is a giant security vulnerability for any other website out there ).
You forgot to mention the biggest advantage of aes.js, for which I think that ifastnet is the most interesting it uses client resources for validation
and the biggest flaw is that it puts i=1 in the URL…except for bad looks I always remember users who said that they couldn’t log into the WP admin section and they were trying to visit example.com?i=1/wp-admin/
I definitely think that after that initial layer would be good to have a second layer as well
and that is “rate limiting”
Everything that passes the first layer and has a cookie stored has a free hand to do whatever it wants, and that’s not right.
In order to protect server bandwidth (in the end, electricity costs and other things), which is probably in the interest of hosting, it wouldn’t be bad to have some FW rules…
if some IP requests the same file/URL 10 times in 5 seconds → to put it on ice for 10 seconds…
if he tries the same thing again x3, then a daily ban (blacklist, etc.).
I would especially emphasize the PHP files here because in addition to bandwidth, they also consume CPU/RAM.
In my case, Cloudflare is very good at protection, and I configured it in detail,
it is basically my main hosting because I use “cache all” and 99% are served from their servers.
Iin most other cases when someone is suspended here on hosting or runs out of daily hits
it is due to PHP files, because even when you are using CF, they are called directly (no cache no store), so CF cannot help much like in the case of static files.
In such cases, you don’t have to be attacked by a bot at all
it is enough for someone to find a PHP file on your website that consumes the most resources and to visit it 100+ times and thus disable you for 1 day because you are suspended.
And the rate limit would help here again
or if nothing else to record the “violent IP” and send it to the user in the form of a list that would be in the control panel, so that he can see that it is not the hosting’s fault, but some IP/ASN should be blocked manually if necessary.
That’s right - it’s an additional complication, additional price and hassle for implementation,
but the ratio of the time spent for it - I believe it would be worth it again, because on the other hand, other costs in the form of bandwidth, electricity, equipment, lifetime of hardware, etc. would be reduced.
The more opportunities you have to exclude bad traffic the more resources remain for real users
and this is then reflected in the quality of service, the speed of websites, etc.
The free hosting platform already does this. Rate limiting is already in effect.
I don’t know if rate limiting is applied before or after the cookie check, but I’ve been told it exists, and that it actually blocks plenty of bad traffic.
It’s actually good that everyone is suggesting “you should to rate limiting”, because it means it’s so unobtrusive that people don’t even realize it exists. It does mean it’s less effective of course. But given that we already see some sites, notably image galleries, hit Apache process limits because there are too many requests on the page, I think reducing the limits risks having too much impact on legitimate visitors on some sites.
I really don’t understand how some code in a browser would give anyone shell access to the server. SSH access is blocked in the server firewall and running system commands is disabled in PHP. So even without any website security in place, no unauthorized user would be able to do SSH.
Checking against a third party service isn’t really a good idea for general web traffic. Doing a request like that easily adds several hundred milliseconds to a request. You can mitigate the impact with cache, but if AbuseIPDB has issues and their APIs are responding more slowly, you’d risk slowing all websites hosted here to a crawl. So in most cases, you’ll want to use such services only when doing specific requests, such as in a registration form.
Also, checking against blacklists isn’t exactly a fool proof method. All you do is check against a list of known bad IPs that have been reported to that blacklist. But many attackers easily pick up new IPs, either from VPS providers or dynamic consumer IPs. A blacklist check cannot account for new traffic very well.
A layer 3/4 DDoS attack will just saturate your network bandwidth, and this security system won’t do anything against it. But that’s not the only kind of DDoS attack.
I’ve also experienced DDoS attack where the backend application got hit so hard the number of PHP processes skyrocketed until all CPU power and RAM was used up and things broke down. I’ve also had people DDoS a completely static site, and still be able to break it by saturating the connection limit on the load balancer. Both of these attacks used up all available resources on something other than the raw network bandwidth, and be able to cause damage just the same.
In both of these cases actually, the attacks punched through Cloudflare. Firewalls and rate limits were all in effect too, but the attackers were able to avoid those by sending seemingly normal requests from a very large number of IP addresses.
Finding the real IP is easy. When a network packet comes in, it has an IP address in it, which it must have for the network connection to work.
Cloudflare does this for free too. Those “separate headers” all have the same data too, they just have different header names because different backend applications accept forwarded IP addresses in different ways. Cloudflare HTTP request headers · Cloudflare Fundamentals docs
Not necessarily. I’ve seen DNS hosting providers that will let you add subdomains to their platform with no issues. We actually had a subdomain of infinityfree.net hosted with Amazon’s Route53 DNS service. I really don’t see any reason why Cloudflare couldn’t support this too. This is just a policy decision on their end, not something with any technical necessity.
I think the rate limit should come before the bot challenge, as the challenge itself does not protect against DDoS. The resource consideration is only valid if the crawler attempts to run the JavaScript. In a real-world DDoS attack, bots do not care what the target server has to reply to, they simply keep on sending requests without waiting for the reply, therefore the implementation does not involve mitigating that. Rate limit on the other hand effectively protects against DDoS, but at the expense of certain resources to keep track of the rate on a per IP basis.
I also think that the server has to give in the same amount of effort as the server verifies the cookie header to see if it matches the expected answer. In a DDoS scenario, this involves the server attempting to issue a large number of challenges and storage prepared for the expected answer to be used later on, at least per IP level.
From my observation, the server would issue the same challenge over and over again if the header does not get returned. Since no cookie headers are involved in the process, this is not Session-based, but rather IP+time(+UA?)-based challenge generation.
As long as the i param does not get incremented when retrying, the i will always stay at 1.
Hi Admin,
So it’s an iFastNet implementation I see. It really depends on how the rate is limited, it could be traffic rate or request per time frame. From what I can see it’s somewhere around 250kbps instead of hits. To clear things up, that is the speed limit as well. I do not have information on the hits meter tho.
I do see some other rate limits available on the market by also counting the rate of failed requests only. This way, hitting too many 403s and 404s would make a certain IP get banned quickly while still enforcing a reasonable rate limit on legitimate ones.
An SSH-like terminal can be emulated with PHP and some fancy coding, but I’m not sure if there are ways to get past the existing setup and make system calls. As far as I know, it cannot be done via PHP directly, maybe sketchy glitching tricks might work but it’s all speculative and there’s no PoC or obvious viable ways of doing it yet.
Yes, highly agree. It ain’t foolproof, and not all web devs know about AbuseIPDB anyway. but it’s something that can be worked with. Calling their API on the spot is definitely not a good idea, but syncing their bad IP list can help by a bit, not god-effective but it’s something.
This is a botnet thing, and I can tell it’s a pattern of Pet**bot from your text (but I can tell it’s not the only source). As long as they originate from host companies with a whitelist or huge IP ranges, Cloudflare does not want to risk mis-banning a lot of others who share the same IP and only let those through, kinda not what we signed up Cloudflare for but still, we had no choice.
It doesn’t actually. The cookie that’s sent to the server just contains the calculated cryptographic value, and the cookie has a validity of decades. For the cookie to be valid for that long, the server can’t realistically keep track of every visitor of every domain, along with which parameters have been presented to that visitors, so it knows which visitor should respond with what value.
So the actual parameters being used are quite static. They are rotated regularly, but at any point in time the values are the same for all visitors. Meaning they only have to be set and calculated once, and then the server doesn’t need to recalculate the values.
I really mean it’s an IP-based rate limit, like you would expect. I don’t know the exact limits, time frames and so on, and I don’t think iFastNet wants those to be known, but it’s not a bandwidth limit. The point is to block bad traffic, not just slow them down.
True, you can block by failed requests, but that requires a lot of per-site tuning. Counting 404s is easy, but if one website owner makes some mistakes and has a lot of broken URLs on their page, you don’t want all visitors to be blocked because of it. It’s not a good solution to apply system wide.
There are multiple levels of protections to prevent web shells from being setup on our hosting. We don’t want those things on our servers or for them to be used by anyone. So that’s not what this security system is intended to protect against.
I think it’s something else, it is probably made to protect the server from poorly written code (loops) or websites that exceed a certain value in the sense that the server has to serve too many different files
in a short time - it basically behaves like bandwidth throttling (probably that’s what it’s for)
unlike if something requests the same file/URL several times in a short time,
the emphasis is on the reqs of the same file/URL within a short time, which would suggest that it is “bad” traffic where the bot constantly hits some URL because it is stupid or it was made on purpose and when it does that it should be rejected through some rate limiting…even returning 403 is better in this case instead of allowing it to call some PHP that then pulls another 80+ resources every time it hits the same URL.
I don’t know what to say
I think it would be good to get rid of that ?i=1 from the address,
and certainly that when someone comes to i=3 that it doesn’t take them to Google, because it’s not right that twice a year ifastnet DDoS Google servers due to problems with aes.js, besides that it confuses users who suddenly see a page from Google regarding cookies instead of a message from their hosting…
I understand how great it is to send most of the bad traffic to Google and reduce the load on your own servers, which “should” respond to that bad traffic.
(but I can tell it’s not the only source). As long as they originate from host companies with a whitelist or huge IP ranges, Cloudflare does not want to risk mis-banning a lot of others who share the same IP and only let those through, kinda not what we signed up Cloudflare for but still, we had no choice.