[ZendTo] Error on File Upload

Mon Jun 25 16:36:35 BST 2018

James,

Please can you confirm that
a) as a virus scanner you are using "clamdscan" with at least the 
"--fdpass" option, and *not* "clamscan"?
b) you have "SMTPdebug" set to FALSE.

Also, try it with no checksumming or anything like that. And make sure 
your /var/zendto/dropoffs is really on local storage and in the same 
filesystem/partition/volume as /var/zendto/incoming.

If you run "top" on the server while the upload of a huge file is 
happening, what is occupying CPU time after the browser has sent the 
whole file, but before the browser generates any error or returns the 
success page?
And how long is that delay?

Cheers,
Jules.

On 25/06/2018 13:17, Mr filesender via ZendTo wrote:
> i would like to add more detail to this issue. After i have tested further, i am still consistently reproducing it. a few observations:
>
> it is present when connecting directly to VM (although file size is larger to reproduce it. i notice 4-6GB). no firewall or loadbalancers in between client and zendto VM. the full file is sent to and present on the incoming folder. the problem occurs after the full file is transmitted. its the client browser that notices a problem, the server continues with the drop and after a time it is there and email is sent as expected
>
> so i wonder, maybe it is a design issue with how the browser is waiting for a reply? i have solved the problem by adding more CPUs to the zendto VM  (but perhaps will return with larger file/drop )
>
> thank you
>   
>   
>
> Sent: Wednesday, June 20, 2018 at 4:13 PM
> From: "Jules Field" <Jules at Zend.To>
> To: "ZendTo Users" <zendto at zend.to>
> Cc: "Mr filesender" <jamesmm1 at gmx.com>
> Subject: Re: [ZendTo] Error on File Upload
> That PHP output looks fine.
>
> I will have to leave it to you to configure your load balancer.
>
> Cheers,
> Jules.
>
> On 20/06/2018 2:01 pm, Mr filesender via ZendTo wrote:
>> ah ok. i am running it behind haproxy/loadbalancer. i have tested directly to the zendto server and it works perfectly.
>>
>> i will modify my loadbalancer timeouts to very high. in any case, i notice that when this error occurs, it is just the browser that is affected. the dropoff is still being processed on the server and eventually, after data has been moved to dropoff folder, is ready and email is sent
>>
>> here are the details you asked for:
>>
>> running over https. php -v:
>>
>> PHP 7.2.6-1+0~20180611145758.22+stretch~1.gbpe20e8b (cli) (built: Jun 11 2018 14:58:01) ( NTS )
>> Copyright (c) 1997-2018 The PHP Group
>> Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
>> with Zend OPcache v7.2.6-1+0~20180611145758.22+stretch~1.gbpe20e8b, Copyright (c) 1999-2018, by Zend Technologies
>>
>> php -i | grep sodium
>>
>> sodium
>> sodium support => enabled
>> libsodium headers version => 1.0.16
>> libsodium library version => 1.0.16
>>
>> tjhank you for the prompt advice :)
>>
>>
>>
>> Sent: Wednesday, June 20, 2018 at 10:18 AM
>> From: Jules <Jules at Zend.To>
>> To: "ZendTo Users" <zendto at zend.to>
>> Cc: "Mr filesender" <jamesmm1 at gmx.com>
>> Subject: Re: [ZendTo] Error on File Upload
>>
>> Do you happen to have anything on your network which does any sort of content-scanning (e.g. virus scanning) of your network traffic?
>> A lot of modern firewalls have this sort of functionality built in.
>> In order to content-scan a file being sent over the network, it will need a complete copy of the file, and will have restrictions on the max file size it can cope with.
>>
>> And are you using ZendTo over http or https?
>>
>> Is the PC you are testing with running any anti-virus scanner that looks at network traffic?
>>
>> Also, please run these 2 commands on your ZendTo server:
>>      php -v
>>      php -i | grep sodium
>> and tell us the output from each one.
>>
>> Cheers,
>> Jules.
>>
>> On 20/06/2018 8:54 am, Mr filesender via ZendTo wrote:
>>
>> i am also experiencing this issue and it is blocking usage of the application. I was testing with v5.0.3 stable and noticed it.
>>
>> now i reinstalled a completely fresh instance of beta 5.0.9 on debian 9 with php7 but it is still there. I use default config settings.
>>
>> i notice it happens with a file ~>1GB . i get the exact same response from the system as kevin below "sorry i failed to dropoff your files.."
>>
>> after i notice that the dropoff exists in database but with 0mb size. in dropoff folder, the data is there but only about half of the filesize.
>>
>> i have tryed with mysql, sqlite3. i have tried turning clam off and on. nothing works.
>>
>> thank you
>>
>>
>> On 01/06/2018 11:48, O'Connor, Kevin wrote:
>>> Jules,>> Disabling the clamdscan made no difference.>> I've done the upgrade to 5.03 and updated the config files. There was no zendto.conf.rpmnew I assume because there were no changes.>> The issue still exists. I can view and download the uploaded file from the outbox, using the resend mail option produces a blank page.>> This is not a production system, we still have an old 4.28-2 system running, so I'm thinking a complete rebuild from the ground up might be the best bet.>> Regards>> Kevin>>> -----Original Message----->> From: Jules Field [mailto:Jules at Zend.To[http://jul.es/mailman/listinfo/zendto]]>> Sent: 01 June 2018 10:31>> To: O'Connor, Kevin <KevinO'Connor at merseyfire.gov.uk[http://jul.es/mailman/listinfo/zendto[http://jul.es/mailman/listinfo/zendto]]>; ZendTo Users>> <zendto at zend.to[http://jul.es/mailman/listinfo/zendto[http://jul.es/mailman/listinfo/zendto]]>>> Subject: Re: [ZendTo] Error on File Upload>>>> Kevin,>>>> To rule out ClamAV permissions issues, edit preferences.php and set>>     'clamdscan' => 'DISABLED',>> Make sure you keep the old value, just comment it out.>>>> Does that make any difference?>>>> I would also advise a quick "yum update" to the latest release of 5.03.>> According to my ChangeLog, the changes from 5.01-5 and 5.03-1 are all>> security improvements or bug fixes.>>>> Cheers,>> Jules.>>>> On 01/06/2018 09:35, O'Connor, Kevin wrote:>>> Jules,>>>>>> Thanks for the quick response. Please see my responses below.>>>>>>> -----Original Message----->>>> From: Jules Field [mailto:Jules at Zend.To[http://jul.es/mailman/listinfo/zendto[http://jul.es/mailman/listinfo/zendto]]]>>>> Sent: 01 June 2018 09:23>>>> To: ZendTo Users <zendto at zend.to[http://jul.es/mailman/listinfo/zendto[http://jul.es/mailman/listinfo/zendto]]>>>>> Cc: O'Connor, Kevin <KevinO'Connor at merseyfire.gov.uk[http://jul.es/mailman/listinfo/zendto[http://jul.es/mailman/listinfo/zendto]]>>>>> Subject: Re: [ZendTo] Error on File Upload>>>>>>>> Kevin,>>>>>>>> Does anything show up in /var/zendto/zendto.log when this happens?>>> No there is nothing logged. All that the file contains as the logins>>> to the system>>>>>>>>>> How long are the uploads taking before this error appears?>>> It occurs even with small files that take seconds to upload.>>>>>>> It is caused by the PHP session failing/disappearing/timing-out>>>> during the upload process. It means the server received nothing of your>> files.>>> The files are present and as far as I can determine have completed upload.>>>>>>> The comment about files/directories is because that is one thing that>>>> will cause it, as in some browsers it is still impossible for me to>>>> check if what you've added to the list of files is a dir or a real file.>>>>>>>> Cheers,>>>> Jules.>>>>>>>> On 01/06/2018 08:52, O'Connor, Kevin via ZendTo wrote:>>>>> Morning,>>>>>>>>>> I've got a 5.01 clean install running on CentOS 7 with AD authentication.>>>>>>>>>> On completion of the file upload an error message pops up "Sorry I>>>>> failed to>>>> dropoff your files! Note that you cannot dropoff directories, only>>>> files" The file is successfully uploaded but this error seems to>>>> prevent the email being sent.>>>>> I noticed someone posted the same fault back in March but I can find>>>>> no>>>> response or resolution posted so was hoping someone could point me in>>>> the direction of a fix or a setting I might have got wrong.>>>>> TIA>>>>>>>>>> Kevin> This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed.> If you have received this e-mail in error please notify the originator of the message.>> Any views expressed in this message are those of the individual sender, except where the sender specifies and with authority, states them to be the views of Merseyside Fire & Rescue Service, (MFRS).>> Incoming and outgoing emails may be monitored in line with current legislation.>> Steps have been taken to ensure that this email and attachments are free from any virus. In keeping with good computing practice the recipient should ensure they are actually virus free.>> http://www.merseyfire.gov.uk/[http://www.merseyfire.gov.uk/][http://www.merseyfire.gov.uk/[http://www.merseyfire.gov.uk/]]>
>> _______________________________________________
>> ZendTo mailing listZendTo at zend.to[mailto:ZendTo at zend.to]http://jul.es/mailman/listinfo/zendto[http://jul.es/mailman/listinfo/zendto]
>>
>> Jules
>>
> Jules
>
> --
> Julian Field MEng CEng CITP MBCS MIEEE MACM
>
> 'No more impressive warning can be given to those who would confine
> knowledge and rsearch to what is apparently useful, than the
> reflection that conic sections were studied for eighteen hundred
> years merely as an abstract science, without regard to any utility
> other than to satisfy the craving for knowledge on the part of
> mathematicians, and that then at the end of this long period of
> abstract study, they were found to be the necessary key with which
> to attain the knowledge of the most important laws of nature.'
> - Alfred North Whitehead
>
> www.Zend.To[http://www.Zend.To]
> Twitter: @JulesFM
> PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
>
>   
>
> _______________________________________________
> ZendTo mailing list
> ZendTo at zend.to
> http://jul.es/mailman/listinfo/zendto

Jules

-- 
Julian Field MEng CEng CITP MBCS MIEEE MACM

'The owls are not what they appear.' - David Lynch (indirectly)

www.Zend.To
Twitter: @JulesFM
PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654