[ZendTo] ANNOUNCE: Version 5.10-1 "production" released

[Jules Field]

John,

On 02/07/2018 19:15, John Thurston via ZendTo wrote:
> Excellent description and explanation, Jules. Thank you for taking the 
> time to document the business need, as well as the implementation.
I'm sure there are plenty of other situations which would call for the 
encryption/decryption feature. The one I documented is merely the one 
that was brought to my attention and hence got it implemented.

>
> I dug through the code last week, and observed:
> A) The Initialization Vector is built with the PHP7 function 
> 'random_bytes'.
>
> B) To permit ZendTo to virus-scan and checksum, the uploaded file is 
> written to the filesystem and subsequently replaced with the encrypted 
> file.
>
>
> With respect to (A), my quick read indicates 'random_bytes' (on linux) 
> uses the system function 'getrandom', which uses /dev/urandom. 
> /dev/urandom should be a non-blocking call and should be able to 
> deliver 16 bytes. I don't know about the CryptGenRandom used on 
> windows. Nor do I know how either of these behave on virtualization 
> platforms. Non-blocking should be non-blocking regardless of the 
> platform, and (should entropy grow scarce) I suppose occasional 
> pseudo-random numbers are good enough for the purpose for which these 
> are being used.
I read up quite a bit on this aspect. Yes, you are quite right that 
entropy can be scarce in virtualised environments particularly. Sadly, 
no one appears to have much in the way of a better solution to the 
problem, beyond using specialised hardware to give a steady "supply" of 
entropy. The random_bytes() (i.e. getrandom()) function is as good as it 
gets.


>
> With respect to (B), this is a possible leak-point for the unencrypted 
> file. Should the process die (or the machine be powered off), while 
> the virus scan or encryption is happening, the unencrypted file will 
> be left in the file system. There isn't much to do about this except 
> perform those steps in memory before committing the file to disk. That 
> would make a large memory footprint! Does ZendTo automatically clobber 
> derelict files as part of the PHP initialization? This would at least 
> limit the duration such things are left in the filesystem.
I cannot read the whole file into RAM, as you point out (anyone got 
150GB RAM to put in their little ZendTo server? That's what one of you 
folks has the max file size set to!).
And over-writing the unencrypted copy of the file just before deleting 
it would not actually help much either, as CoW (copy on write) 
filesystems are gradually taking over from journalling filesystems. With 
a CoW filesystem, you will just gain a bunch of zeroed blocks (which 
might well be compressed to nothing in a decent filestore) in addition 
to the untouched data from the unencrypted file. And with (effectively 
CoW for this aspect) devices such as SSDs, the same would happen.

ZendTo doesn't automatically remove derelict files as part of the PHP 
initialisation, because there is really no such thing as "the PHP 
initialisation". I could add a cron job that deleted any files in 
/var/zendto/incoming which were more than a few hours old, but that's 
about it. If you would like to add that yourself, you just need to run a 
command like this every 4 hours:

find /var/zendto/incoming -type f -mmin +240 -print0 | xargs -0 rm -f

That should delete all files that were last modified more than 240 
minutes (4 hours) ago. It will handle any and all nasty characters in 
the filenames, such as spaces, quotes, semi-colons or anything else that 
could be used to wreak havoc.

> Both A and B are my observations. I don't want to indicate I perceive 
> them as problems. Any time we start talking about encryption, we 
> should work hard to understand what we're getting, what it is 
> protecting, and identify the edge cases.
If anyone can come up with a better way of mitigating B, please do let 
me know. Otherwise I will add something very like this to the standard 
/etc/cron.d file that is installed.

>
> With my new understanding of what you've built for us, I'll be trying 
> to ship this new version so my customers have the option of encryption.
Thanks!

Jules

-- 
Julian Field MEng CEng CITP MBCS MIEEE MACM

'When I read Shakespeare I am struck with wonder
  That such trivial people should muse and thunder
  In such lovely language.' - D.H. Lawrence

www.Zend.To
Twitter: @JulesFM
PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654