Server blocks requests to WordPress REST API

Hosting Support
1

Website URL

https://ahhqart.xo.je/wp-json/wp/v2/posts

Error Message

‘Connection aborted.’, RemoteDisconnected(‘Remote end closed connection without response’,)

Other Information

Server seems to mistake requests to WordPress REST API for spam, most likely due to the user agent, which was python-requests/x.x.x. Changed it to custom UA (i.e., ahhqart/1.1.1), and it worked.

This change must be new because eveyrthing was fine the last time I posted via REST API in October.

Just want to inform the Admin of this

2

Yes, this is likely from the security system.

7 Likes
3

But it is NOT guaranteed that the security system will stop bots. For example, bots can hide behind identity of real clients by capturing browser requests and injecting JavaScript payload in HTML response. This way, bots can easily access anti-bot cookies, headers assigned by the security system itself.

1 Like
4

Yes, and bots running off of headless browsers will be allowed through with no extra effort. But most bots aren’t that sophisticated because that gets expensive. Most of them are just basic automated HTTP crawlers that can’t set cookies or execute JavaScript, and that’s what the security system is supposed to keep out. It’s also supposed to prevent you from using your hosting account to run an API, and as you can see it’s quite successful at that.

5 Likes
5

“Automated attacks” are not “basic automated HTTP crawlers” and are not necessarily expensive. As you can see, I bypassed it just by changing UA. No cookies, no headers sent, yet it worked because the ?i=1 thing is NOT applied in API endpoints.

6

The security system, along with the IP blocking system does a pretty good job of blocking many automated attacks.

However, I would say that by circumventing this system you’re running the risk of being in breach of the InfinityFree TOS… But ultimately, that’s admins decision so don’t take my word for it

7 Likes
7

Sorry, my mistake. I misread your initial post. I agree that’s a pretty big opening and is definitely a flaw in the security system. I’m sure Admin will appreciate you bringing it to their attention.

5 Likes
8

I’m not trying to circumvent anything. I’m just saying the security system is not enough to stop bots. Because non-techy readers would miscomprehend the article and think CAPTCHAs are redundant.

And I don’t think posting to WordPress REST API does violate InfinityFree’s ToS, specifically this clause you must be referring to:

You agree not to circumvent, disable or otherwise interfere with the security-related features of this Site or the Services found at this Site (including without limitation those features that prevent or restrict use or copying of any InfinityFree Content or User Content) or enforce limitations on the use of this Site or the Services found at this Site, the InfinityFree Content or the User Content therein.

InfinityFree server, if I’m not mistaken, is optimized to host WordPress sites, of which REST API is a basic component. Security does NOT mean to block it.

As well, my error is not 403 Forbidden. My error is:

Connection aborted.’, RemoteDisconnected(‘Remote end closed connection without response’,)

9

No, but circumventing the security system does. Ultimately though, its admin’s decision if this constitutes deliberately circumventing the security system.

If you look at this post from Admin:

It does say that Rest API’s are generally blocked. I know admin has stated previously that automated uploads, or uploads using external software (other than FTP) are (usually) blocked by the security system. Remember the security system doesn’t know the difference between a wordpress Rest API, or a completely unrelated rest API

Its optimised for general web hosting (as I understand it), not specifically wordpress.

403 errors generally show up when you try to access a page you don’t have access to. If an API is being blocked, it can throw all sorts of strange errors, or can simply disconnect.

Its not always as clear cut with APIs

7 Likes
10

No, not specifically. You can host them on InfinityFree because WordPress is decently universal and only requires PHP and SQL. But that’s different from a host being “optimized” for WordPress. You may have noticed that trying to host an exceptionally busy WordPress site, or using demanding plugins can easily make hosting accounts reach their daily RAM and CPU limits.

It does when the API connections come from anywhere but the WordPress installation itself. This includes anything that can publish WordPress posts remotely.

Admin has said in the past that using a workaround to do something that users aren’t intended to be able to do is frowned upon.

7 Likes
11

If an API is being blocked, it can throw all sorts of strange errors, or can simply disconnect.

No. The server should respond with 403 Forbidden when normal request was blocked or the endpoint was restricted, indicating that it was security-related and was just not random bugs. Even some law requires servers to provide appropriate error responses, such as 403 Forbidden.

But I did not get 403 Forbidden.

InfinityFree terms states

The primary purpose of any script must be to produce a web page.

I was posting via REST API.

======

You may have noticed that trying to host an exceptionally busy WordPress site, or using demanding plugins can easily make hosting accounts reach their daily RAM and CPU limits.

A simple blog can run within those limits. Some hosting providers only offers 1GB free storage, and WordPress can’t be installed because they block some PHP functions. InfinityFree offers 5GB, which is enough to host a whole blog with image uploads. As I remember, I saw an option in cPanel to install WordPress (probably in site builder section). That’s why I assume InfinityFree server is optimized to host WordPress sites.

Plugins that consume so much RAM/CPU would already be blocked by the server limits like PHP max execution time and max memory. But they are not neccessary for average WordPress sites.

Anti-bot system also helps prevent high usage of RAM/CPU, but it is just auxiliary. You ought to equip CAPTCHA to completely prevent spams just like what I previously said.

=======

Admin has said in the past that using a workaround to do something that users aren’t intended to be able to do is frowned upon.

You misunderstood it. This paragraph

However, know that by doing so, it should be considered an exploit, and we may choose to implement measures against “hacking” like that, which may give you trouble.

just refers to this

We really don’t want people to be able to register new epizy.com subdomains…

…it’s not completely impossible to create new epizy.com subdomains.

A workaround would violate ToS, but it depends on what workaround you’re doing, and it’s the Admin that decides.

12

In an ideal world you’re right.

But the way the security system here works, it has some fun quirks. And doesn’t always direct to 403.

For example if you have a none js enabled bot, sometimes it’ll just show the ase.js script then disconnect.

And in the UK (where IFastNet is based) there’s no law that requires servers to respond with specific error codes. And honestly, that’s impractical anyway. When people start doing unexpected or unintended actions, there will be times that the server doesn’t know what to do

6 Likes
13

How do you or the terms defines “security-related features”?

You agree not to circumvent, disable or otherwise interfere with the security-related features…

A security system should at least expect specific cookies and headers from a request before it gets validated.

Because the system requires JavaScript and cookies

( Browser Security System - Features and Limitations )

My requests to REST API were made using Python script and did not contain any cookies—and headers—other than essential parameters. The fact that they were accepted by the server without cookies just implies the absence of such security system at those endpoints.

So I’m confused of how you concluded that I circumvented something. Like, what security-related features did I circumvent? 403 Forbidden error is security-related but not this error:

Connection aborted.’, RemoteDisconnected(‘Remote end closed connection without response’,)

14

It certainly suggests a loophole (whether intentional or not, only Admin can say) in the security system on that particular endpoint.

As stated above, The security systems are designed in such a way that only browsers should be able to access your site. And to limit automated\none browser connections.

Taking any steps (whether deliberately or not) to connect to your site with something other than a browser, is by that definition circumventing (getting around) the security system.

I only mentioned it in the first place to make you aware that its dodgy ground, and to hopefully avoid any trouble for you.

If you look around on the forum, you’ll see admin stating regularly that you can’t (may not) access API’s on your site from external websites or applications.

You mean like this that your page is reporting now :wink:
image

Just remember that when you try and do something that a system isn’t designed to handle, it may not give you the expected result

4 Likes
15

Depending on the request, the server may respond with the browser challenge, a 403 response, or simply no content at all. The empty response you saw is normal behavior.

Our hosting isn’t specifically “optimized for” WordPress, but WordPress is probably the most popular software to use on PHP/MySQL web hosting, so good WordPress compatibility is important. However, that doesn’t mean that we design our hosting so that every feature of WordPress works perfectly.

We provide web hosting. That means: hosting for websites, i.e. things that produce web pages viewed in browsers. API hosting is not supported here. We’re well aware that some website software also includes API functionality, but that functionality cannot be used on websites hosted here.

I admit that the current security features are blunt, but they are very effective at stopping malicious scanners and other kinds of low-effort bulk scanners. It’s not bullet-proof, a sufficiently skilled and motivated hacker can bypass it. But it stops a lot of attacks and helps keep server load under control, so it works well enough.

The WordPress APIs not being available to your Python script is known and intended behavior. If that’s a hard requirement for you, then our free hosting is not a good fit for your project.

Please don’t try to circumvent our security systems. If we wanted you to, we’d have given you an option to do so.

10 Likes
16

I have other issue in my site. I learned that WordPress admin panel uses the REST API. My site was suspended yesterday because of the Hits Limits. Almost only I was accessing my site as I was still developing it. So I didn’t know where the hits came from. I suspect the admin panel. Probably the hits had been created from there as it accessed the API.

I also suspect Termly (my cookie consent functionality), which automatically scans my site to detect new cookies. But this just happens once every day or every 3 days.

And Google, to which I am indexing my site.

I was notified that repeated suspension due to Hits Limits would permanently suspend my site.

17

That is me. I block GET requests to my REST API and only allow POST.

18

As far as I understand all hits even if blocked by security, or application level rules, count as hits.

1 Like
19

Yeah they count. And to exclude hits made by bots is actually the role of the system, I guess. But its flaws will get sites suspended permanently.

The last time I checked, my Daily Hits Used was 15%—just like my regular hits. After several minutes, my account was suspended for Hits Limits.

I know stats was not real-time, but going from 15% to 100% within minutes, or say, few hours was unusual, especially for my site that was of very small traffic.

20

A note with the statistics counters is that they only update in the panel every 8 hours. A jump from 15% to 100% in 15 minutes of looking at the dashboard could be an 8+ hour event, not 15 minutes.

7 Likes