The New File Upload Algorithm of WPFileUpload Plugin

Version 2.6.0 of WordPress File Upload plugin introduces a brand new upload algorithm. The whole upload process has been redesigned from scratch so that it becomes more stable.

Stability is very important when uploading very large files. Suppose that you upload a 1GB file, that takes 2 hours to upload. Several things can happen in between that could make the upload fail (internet connection interruption, unintentional suspension of laptop, accidentally close or refresh the upload page, web server problems etc). Imagine how frustrating and annoying it would be for your customers to see that their file failed.

The plugin uploads large files in chunks. This means that a file is sliced into smaller pieces, these pieces are sent to the server one-by-one and the server reassembles them back into a single file.Though more complicated, this technique has one major advantage, it is not affected by the web server restrictions relating to size and time limitations when uploading files. This means that files of unlimited size can be uploaded.

Suppose you want to upload a 1GB file. This will be sliced into 1000 chunks, 1MB each. It is like uploading 1000 separate files. The overhead for your browser will be very high if you try to upload them all together. Especially when this is done by a smartphone. The previous versions of the plugin initiated all 1000 uploads at once, leaving the browser responsible to handle the upload traffic and prioritization. However not all browsers are able to effectively manage that load and cause bottlenecks that eventually make the upload fail. For this reason, the latest version of the plugin limits the maximum number of concurrent chunks. So, these 1000 chunks will not be initiated all at once, but sequencially, making sure that the maximum limit of concurrent chunks is always preserved.

However, 1000 chunks are too many and the probability of something going wrong is high, because of an instant connection problem, or client overhead, or server problem, or whatever. In the previous versions of the plugin, if one chunk failed, then the whole upload would fail. However, the latest version includes an auto-resume feature, that continuously monitors the chunks. If any chunk fails, or it just takes too long to upload, it will be resumed. So, there is no way of loosing chunks and the upload will continue until the file is completely uploaded. Auto-resume is an internal process, not visible to the user. No actions are required by the user to resume the file. It will be done automatically by the plugin, as long as the user keeps the window of the upload open.

The auto-resume feature has been tested under extreme conditions (intentional repetitive interruptions of internet connection, restart of web server, upload of files over 2GB, poor internet connection, very fast internet connection) and has performed well in all cases. The uploaded files have been compared to the original ones, bit-by-bit, in order to verify the integrity of the upload algorithm.

Apart from reliability, the new algorithm is also secure. Although it uses AJAX to perform the uploads, all necessary measures have been taken to prevent unauthorized users to upload files or hackers to pass malicious scripts. Multiple checks, both on client and server side, are performed, in order to verify the user, the referer and the contents of the uploads. WordPress nonces, a standard WordPress security feature, are used in order to protect from CSRF attacks, as well as sanitization of exposed strings.

Of course the plugin preserves backward compatibity with older browsers, that do not support HTML5 and AJAX. The plugin will auto-detect the capabilities of the user’s browser and will adjust its upload functionality, though some features may not work (like chunked uploads).

If you want to know more information about the upload algorithm, just contact Iptanus through the contact form or send an email to info@iptanus.com.