Storage upload error?

I run the simple website of my radio here.
On this website, you can only see what kind of sound file is playing, and it is accompanied by an image file with a size of 300x300 pixels and an average of 90kb.
Unfortunately, every sound file of the radio has to be associated with an image file. Currently, it is 11558 small image files and 734Mb.
The entire website currently has Disk Space Used: 775 MB.
But unfortunately I can’t upload more images at the moment. What is the reason of this? If possible, please look into this error for me, because I use the storage space offered to me fairly. The my Accounts: epiz_33930051
Thank you for your patience and helpful work!

That is a ton of images! You should probably consolidate them into image maps, it will make loading your website much much faster.

The reason you can’t upload more files is because of this:

EDIT: My bad, I thought the inode limit was 11,000 not 30,000

5 Likes

Hi Norbert1973,

You might want to check if you’ve hit an inode limit if you have too many files on the site, this will limit your ability to upload more images regardless of filesize and file type.

Inode is essentially your file count, link to images if you find reducing the count difficult using common free image hostings like:

https://imgur.com/ or https://imgbb.com/

Cheers!

6 Likes

My website always only displays the covers of the currently playing music and other media content. So there is always only one approx. 100 kilobit 300x300 pixel image, which is min. 3 minutes max. it is changed once in 1 hour. So if there is a 1-hour show, the associated cover image is displayed for 1 hour. If a music track is played, the single image file corresponding to the length of the music file is scanned.
Unfortunately, there is no way to handle this differently, because an Internet radio database requires a separate cover image file for each file. They must be in a folder and it is a matter of setting the quantity limit. This is completely normal for internet radio. The display of the visual content is only important for the interface of my website. the stream is from a completely different server and the radio’s other media database is also a different server, which only sends information corresponding to the given content to the website.
In fact, I don’t use a lot of memory or data traffic, because I work with the most economical methods possible. But the amount of image files cannot be a criterion in a folder.
My Inodes Used: 39 % (11637 of 30019) !
Disk Quota: 5120 MB
Disk Space Used: 776 MB
Bandwidth used: 756 MB

Kindly share the ftp log using any ftp client

I don’t understand the point of the request…
Because I can’t even upload to the Album-Art folder via the InfinityFree Control Panel page.
Do I upload the log file here? I don’t think it’s a wise idea to post here what it contains. I’ve uploaded it to my hosting now, which you as administrators can see.
as [email protected]. In this .log file, I only opened the Album-Art folder and tried to upload it.

You can upload the log file here, it makes it a lot easier. Additionally, only Admin has access to your account files, nobody else here does.

2 Likes

Oké, but currently, I noticed that it does not list the new files on ftp when I upload them. It doesn’t even show the uploaded file for search. But the website loads the uploaded .jpg file.

“You can upload the log file here…” How? I only see a hyperlink solution here…[email protected]

Just to make sure I’m understand what the issue is: you have uploaded a folder Album-Art with a bunch of images. The files appear to have been uploaded correctly, and are visible on your website, but you cannot see them in FTP. Am I correct so far?

Your account is using over 10k inodes, which is quite a bit, but nothing problematic on it’s own. However, it seems that the bulk of these inodes are all image files placed directly in the Album-Art folder.

I think you’re just running into the issue where FTP clients will refuse to list that many files. The files exist, but many file managers will stop at a certain number of files because it’s too complicated to show them all. Not showing more than 1000 files is not uncommon, and your album art folder has quite a bit more than that.

I see your log is from WinSCP. Maybe FileZilla can do a better job at listing all files? The web based file manager definitely won’t do.

You may want to consider structuring the album art a bit in folder. For example by grouping them in folders per artist, and within the artist folder per album.

7 Likes

“Just to make sure I’m understand what the issue is: you have uploaded a folder Album-Art with a bunch of images. The files appear to have been uploaded correctly, and are visible on your website, but you cannot see them in FTP. Am I correct so far?”
Yes, it doesn’t show new files anymore after about 11,500 files. But e.g. on the WinSCP interface, the counter stopped at 673mb and 998 files several months before this. However, with a completely different provider, there are no problems with FTP and it shows the real numbers and the real file content. But that service provider unfortunately has much worse conditions and other problems.

“Your account is using over 10k inodes, which is quite a bit, but nothing problematic on it’s own. However, it seems that the bulk of these inodes are all image files placed directly in the Album-Art folder.”
Yes.

“I think you’re just running into the issue where FTP clients will refuse to list that many files. The files exist, but many file managers will stop at a certain number of files because it’s too complicated to show them all. Not showing more than 1000 files is not uncommon, and your album art folder has quite a bit more than that.”
As I wrote, there is no such problem with another hosting, but unfortunately there are other problems there.
I see your log is from WinSCP. Maybe FileZilla can do a better job at listing all files? The web based file manager definitely won’t do.
“You may want to consider structuring the album art a bit in folder. For example by grouping them in folders per artist, and within the artist folder per album.”
Unfortunately, this is not possible because the radio broadcast software saves the cover image for each file in a single folder and, moreover, not per album, but one per file name. There is a logical explanation for this, but I would not go into that now. The bottom line is that, unfortunately, it is not possible to economize on the fact that an artist’s album cover may have up to 15 music tracks. Because the official software does not allow this. Also, the name of the image file is the same as the audio file names, only the extension is not mp3, but jpg

Are you sure it’s 998 files? Not 9998?

When trying to list the Album-Art folder in FileZilla with logging enabled, I see the server saying this:

Response:	226-Options: -a -l 
Response:	226 Output truncated to 10000 matches
Status:	Directory listing of "/htdocs/Album-Art" successful

9998 files, plus . (current directory) and .. (upper directory) makes 1000 entries in the listing. So that all makes sense.

So it seems that our FTP server won’t list more than 10k files. And given how long it took for that listing to be generated, I think it makes sense to have a limit on the number of files, and than 10k is a reasonable limit.

FileZilla also struggles with the number of files, scrolling through them is quite choppy.

This appears to be a setting that can only be changed for the entire FTP server, not for individual accounts.

So I’m afraid that this is simply expected behavior. And it’s up to you to figure out how to best deal with it. If reorganizing the files is not an option, maybe you’ll just have to accept that you may need to upload images without being able to verify that their upload was successful.

5 Likes

“Are you sure it’s 998 files? Not 9998?”
Sorry, that was a typo. Of course, the counter stopped at 9998.
I don’t actually scroll. But I have it set so that I see the last uploaded things at the top.
Also, if I want to delete a file or change its name, I should be able to list it in the search engine and perform the deletion or renaming. Sometimes this happens and I can’t solve it in this way.
But, as I wrote, there are no such problems with other radio stations on other servers, even though they have twice as much content and use the same broadcast management, database, and so on method as I do.

I’m not sure what exactly you expect from me here.

The issue seems to be quite clearly caused by our FTP server that only returns a maximum of 10,000 items in a directory.

Which is a very sensible limit to have. Listing the directory is already quite slow now, both for the server and the client. And it also puts stress on the server to collect and transmit the file information.

It’s also the default limit in cPanel, on premium hosting.

Our hosting is working as intended. You appear to have an uncommon need to have a very large number of files in a single directory. Our hosting places some limitations on this. You’re still able to upload and delete files in the folder with FTP, access them on your website, or use PHP to list them. The only thing you can’t do is list them all through FTP.

6 Likes

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.