[ZendTo] Re: Unable to Upload large files

[Jules]

Brian,

On 08/03/2012 20:16, Brian Ott wrote:
> Hey Jules!
>
> We have done some testing on zendto and came across some issues, we
> would appreciate any help you can provide. It seems we are not able to
> upload large files around 4G. As explained below our test setup I
> should also mention we got the same results by accessing the host
> directly without any load balancers and only using one host.
Yes, the load balancers shouldn't affect this, as they are just seeing a 
packet stream.

Read on...
>
> Cheers!
>
>
>
> Here is our setup:
>
> (Names and IPs replaced from real ones)
>
> We have our zendto setup on two servers (identical) running centos5.
64-bit I hope! :-)
>   I
> tried running Debian and Cento6 but as mentioned in earlier posts I
> experienced issues while trying to modify php to large file uploads.
>
> I follow the instructions on the website, nothing was done outside of
> that.
>
> Our two servers (zendto-www1&  zendto-ww2) have zendto placed on NFS,
> and share the mount.
NFS often doesn't handle files over 4GB very well *at all*. In fact most 
NFS installations I have seen are totally incapable of handling 4GB 
files. So that is certainly a starting point, I would make up some large 
files by hand with something like
     yes | dd of=BIGFILE bs=1024000 count=5000
which should give you 5GB of the letter "y" (better test than just 
zeros) and copy that file to, from and around the NFS mount to check 
that it can actually handle 5GB files intact. Use "cksum" to quickly 
checksum them against the original.
>   We use MySQL as the database and its on another
> host.
That's fine.
>
> here are some of our php.ini settings:
>
> post_max_size = 50000M
> file_uploads = On
> upload_tmp_dir = /nfs/zendto/zendto_files/incoming
> upload_max_filesize = 50G
> default_socket_timeout = 6000
> max_execution_time = 12000     ; Maximum execution time of each
> script, in seconds
> max_input_time = 12000  ; Maximum amount of time each script may spend
> parsing request data
> memory_limit = 1000M      ; Maximum amount of memory a script may
> consume (16MB)
That looks okay. Also make sure your apc.ini has the relevant settings 
for apc.max_file_size=50G, apc.slam_defense=off and such like as 
documented on zend.to.
>
>
> The server itself has 4G of ram (each of them).
>
> Problem
> ----------
>
> Before we wanted to roll this out here into production our final step
> was to do some testing, here is what we found.
>
> - Uploading files under 2G works fine under all Browsers
Correct.
> - Upload/Download speed is a little slow, transferring from the box
>    using SCP is almost double the speed.
That's down to the browsers mostly. But it will also be caused by your 
NFS as that doubles the amount of network traffic having to happen for 
the upload.
> - Uploading 2.5G-3G files fails on IE, Safari&  Firefox.
Yes, it will, certainly on 32-bit Windows.
> - Chrome uploads 3G+ files fine in Linux but not Windows
Yes, certainly on 32-bit Windows. You don't say whether you're using 
32-bit or 64-bit Windows. IE9 64-bit on Windows 7 64-bit does work, as 
does Chrome. But IE9 is the *only* 64-bit browser, even on 64-bit 
Windows. If you look in Task Manager on Windows 64-bit you will see that 
Chrome is only a 32-bit process.

And 2^32 (max representable in an unsigned int) is 4GB. Most coders use 
ints (which are signed) through laziness, so giving a max of 2^31 = 2GB. 
That's where the limits come from.

>
> What seems to happen is the following:
>
> 90% of the time for IE/Firefox users the connection gets reset after a
> few seconds (maybe 30 seconds?) then takes them from verify.php to
> dropoff.php. The user gets prompted with the popup saying "This page
> is asking you to confirm that you want to leave etc" With two
> selections of either "Leave Page" or "Stay on Page".
>
> Leave page obviously takes them to dropoff.php with a "Connection
> Reset" page and the file upload fails. If they choose "Stay on Page"
> they stay on "verify.php" but nothing happens and the file never
> uploads.
>
> The other issue is for Chrome users on Windows, and sometimes on Linux
> where at any percentage of a large file upload the page will have a
> popup saying the page is not responding and if you want to "Kill it"
> or "wait" if you select wait sometimes it will resume the upload till
> complete. However we have had some tests where it did not on Windows
> and just ended up having to be killed.
>
> I can enable any logging required or provide any information in
> private. We would really like your help with this as we do like this
> product and want to roll it out into Production.
You need 64-bit Windows with 64-bit-IE9 or Chrome to reliably do >4GB 
uploads.
This is entirely caused by lots of bugs in lots of browsers. As it's not 
a big marketing feature, the vendors put no effort into this area of 
their code, so it gets better *very* slowly.

And unfortunately there's nothing I can do about that. You have to be 
using 64-bit Windows if you want to use Windows for any >4GB uploads to 
any website.

The Mac browsers fare far better, as OS X has been 64-bit-only for so 
much longer. I use a Mac and everything, as they say, "just works". 
32-bit apps rarely exist on OS X any more.

Linux is somewhere in between right now.

I hope that helps, even though I can't necessarily give you a "quick 
fix" setting change to apply.

Cheers,

Jules

-- 
Julian Field MEng CITP CEng
www.Zend.To

Follow me at twitter.com/JulesFM
PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654

'It's okay to live without all the answers' - Charlie Eppes, 2011
'All programs have a desire to be useful' - Tron, 1982
'That is the land of lost content,
  I see it shining plain,
  The happy highways where I went,
  And cannot come again.' - A.E. Houseman