[ZendTo] ANNOUNCE: ZendTo beta released
Jules
Jules at Zend.To
Wed Dec 21 21:41:34 GMT 2016
Ian and others,
For your delight and delectation, I have automated the upgrading of
preferences.php.
I haven't done zendto.conf yet, but that will be dead easy compared to
preferences.php (which is of course arbitrary executable PHP code).
A gzip-ped version of the script is attached. You will need to
gunzip upgrade_preferences_php
chmod +x upgrade_preferences_php
./upgrade_preferences_php
and that will show you how to use it.
Basically you give it your old preferences.php file, and the new one I
supply with the new version.
It will work out how to write your old settings into the new one
(including any other settings you might have added because you might
have extended ZendTo's features), and will output a new preferences.php
file on stdout (hence the ">" in the example command it shows you).
If it spots any obvious improvements that can be made, such as using
internaldomains.conf instead of a huge nasty regexp, it will tell you
about it. It will also set the version number and update the copyright
statement.
Do read your new file before you use it, as it will have moved any
now-obsolete settings towards the top of the file and commented them out.
I'm pretty sure it does a fairly thorough job, but I'm sure you will
tell me what you think. :-)
Happy Christmas!
Jules.
On 20/12/2016 08:03, Elston, Ian wrote:
> Success!
> Well it got to the end of the installer... I'll continue testing later..
>
> Is there any quick way to migrate configs/templates from my current existing 4.11?
>
> This is much easier than before, thanks. How will updates be handled? Similar scripts?
>
>
>
> ------------------------------------------------------------------------------
> Ian Elston
> Networks & Developments
> The University of Bolton
> 01204 903971
> http://www.bolton.ac.uk
>
> ** SUPPORT QUERIES **
>
> Please contact x3444 or itsupport at bolton.ac.uk to report new faults or request updates on existing calls.
>
>
> -----Original Message-----
> From: zendto-bounces at zend.to [mailto:zendto-bounces at zend.to] On Behalf Of Jules
> Sent: 19 December 2016 16:30
> To: ZendTo Users <zendto at zend.to>
> Subject: Re: [ZendTo] ANNOUNCE: ZendTo beta released
>
> Ian,
>
> Don't tell me, it was a Ubuntu 16 box, right?
>
> I've just put a new copy of the installer on the website.
> Please can you:
> rm -rf ~root/installZendTo*
> Download the new installer.
>
> Unpack it and run it as before.
> But when it asks you about each stage, just say "n" to the first 6 stages, until you say "y" to the bit about installing ZendTo itself. Do that, and the 8th stage (SELinux) which you haven't done yet. As you're not going to be using SELinux (even the Ubuntu wiki admits it's not maintained for Ubuntu at all), that stage won't actually do anything.
>
> If you want to just run stage 7 on its own, without having to answer "n"
> to anything else, you can just run the "7-zendto.sh" script in the Ubuntu sub-directory directly. Provided you run it from *somewhere* within the installer's tree, it should find everything it needs.
>
> Hopefully that will fix it!
>
> Cheers,
> Jules.
>
>
> On 19/12/2016 14:44, Elston, Ian wrote:
>> I have installed and updated a Ubuntu server
>>
>> Ran the install.sh
>>
>> All goes well until the "easy bit"
>>
>> Last few lines of the script output....
>>
>> About to install ZendTo version 4.19-1 Drumroll please...
>> /tmp/install.ZendTo/Ubuntu/7-zendto.sh: line 43: cd: /home/nads/zendto-build-store: No such file or directory
>> % Total % Received % Xferd Average Speed Time Time Time Current
>> Dload Upload Total Spent Left Speed
>> 100 1088k 100 1088k 0 0 2142k 0 --:--:-- --:--:-- --:--:-- 2141k
>> dpkg: error processing archive /home/nads/zendto-build-store/zendto_4.19-1.deb (--install):
>> cannot access archive: No such file or directory Errors were
>> encountered while processing:
>> /home/nads/zendto-build-store/zendto_4.19-1.deb
>> Reading package lists... Done
>> Building dependency tree
>> Reading state information... Done
>> 0 to upgrade, 0 to newly install, 0 to remove and 5 not to upgrade.
>> dpkg: error processing archive /home/nads/zendto-build-store/zendto_4.19-1.deb (--install):
>> cannot access archive: No such file or directory Errors were
>> encountered while processing:
>> /home/nads/zendto-build-store/zendto_4.19-1.deb
>> Failed to install ZendTo deb package.
>> Exiting...
>> sudo -i /opt/zendto/bin/adduser.php
>> That will show you the syntax. Use it to add a single test user.
>> Then login to the website and drop off some files.
>>
>> Now go and configure ZendTo itself in
>> /opt/zendto/config/preferences.php and /opt/zendto/config/zendto.conf
>>
>>
>> nads at zendto-test:/tmp/install.ZendTo$
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Ian Elston
>> Networks & Developments
>> The University of Bolton
>> http://www.bolton.ac.uk
>>
>> ** SUPPORT QUERIES **
>>
>> Please contact x3444 or itsupport at bolton.ac.uk to report new faults or request updates on existing calls.
>>
>>
>> -----Original Message-----
>> From: zendto-bounces at zend.to [mailto:zendto-bounces at zend.to] On Behalf Of Jules
>> Sent: 19 December 2016 10:14
>> To: ZendTo Users <zendto at zend.to>
>> Subject: Re: [ZendTo] ANNOUNCE: ZendTo beta released
>>
>> I've just posted an announcement and a new page of information on the web site too.
>>
>> This page contains the low-down:
>> http://zend.to/beta.php
>>
>> Cheers,
>> Jules.
>>
>>
>> On 18/12/2016 19:24, Jules wrote:
>>> Folks,
>>>
>>> I have been spending the past few weeks/months getting ZendTo sorted out.
>>> I have *hopefully* fixed the security vulnerabilities that anyone has
>>> told me about. So you out there with pen-testing tendencies, please do
>>> have a go at the new version and let me know what you find!
>>>
>>> The most major thing is I have written a new installer for ZendTo,
>>> instead of trying to maintain VM images.
>>> I find most people these days need to put it into an otherwise bare
>>> server image prepared by someone in their IT department. Trying to
>>> start from a VM image, and then trying to shoehorn in all your
>>> corporate setup round it was a pain.
>>> As was installing it by hand with just the rpm/deb file.
>>>
>>> So now there is a shiny new installer. It assumes a bare server system
>>> to start with, logged in as root.
>>> I have tested it on
>>> CentOS 5
>>> CentOS 6
>>> CentOS 7
>>> RedHat Enterprise Linux 5
>>> RedHat Enterprise Linux 6
>>> RedHat Enterprise Linux 7
>>> Ubuntu Server 14
>>> Ubuntu Server 16
>>>
>>> It installs all the necessary software, entirely automatically
>>> rebuilds PHP to support big uploads (where necessary), sets up ClamAV,
>>> mail delivery, SELinux, AppArmor, Apache, usage stats, pretty much everything.
>>>
>>> It talks to you throughout the process, and will ask you the odd
>>> question. If you want to pause, just Ctrl-S it. If you want to re-run
>>> bits of it, you can run any of the 8 components individually, no
>>> problem at all. Take a look round the structure, it's very
>>> straightforwards (but non-trivial to write!). When you want to update
>>> PHP, for example, just re-run the component that rebuilds PHP. It will
>>> work out the latest versions for itself.
>>>
>>> SQLite3 is now the default database to use, as it's by far the easiest
>>> and simplest and works on everything except CentOS/RedHat 5.
>>>
>>> To get started: download, unpack and run the installer with
>>> curl -O http://zend.to/files/install.ZendTo.tgz
>>> tar xzf install.ZendTo.tgz
>>> cd install.ZendTo
>>> ./install.sh
>>>
>>> It will fetch the deb/rpm file as appropriate. But if you just want
>>> the deb/rpm and nothing else, they are in
>>> http://zend.to/files/zendto-4.19-1.noarch.rpm
>>> http://zend.to/files/zendto_4.19-1.deb
>>> http://zend.to/files/ZendTo-4.19-1.tgz
>>> The file at
>>> http://zend.to/files/ZendTo-Version
>>> will tell you the latest version number (during the beta test).
>>>
>>> P.S. If you've walked through the installer the slow way, and want to
>>> run it quickly with no questions at all, add '--defaults' to the
>>> command line of either ./install.sh or any of its components.
>>>
>>> Please do tell me how you get on! Any questions/queries/problems, get
>>> in touch on this list.
>>>
>>> Cheers, and Happy Christmas!
>>>
>>> Jules
>>>
>> Jules
>>
>> --
>> Julian Field MEng MBCS CITP CEng
>>
>> 'Ever since the dawn of civilization, people have craved for an
>> understanding of the underlying order of the world: why it is as
>> it is, and why it exists at all. But even if we do find a complete
>> theory of everything, it is just a set of rules and equations. What
>> is it that breathes fire into the equations, and makes a universe
>> for them to describe?' - Stephen Hawking
>>
>> www.Zend.To
>> Twitter: @JulesFM
>> PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
>>
>> _______________________________________________
>> ZendTo mailing list
>> ZendTo at zend.to
>> http://mailman.ecs.soton.ac.uk/mailman/listinfo/zendto
>>
>> _______________________________________________
>> ZendTo mailing list
>> ZendTo at zend.to
>> http://mailman.ecs.soton.ac.uk/mailman/listinfo/zendto
> Jules
>
Jules
--
Julian Field MEng MBCS CITP CEng
'The past is supposed to be a place of reference, not a place of
residence! There is a reason why your car has a big windshield and
a small rearview mirror. You are supposed to keep your eyes on where
you are going, and just occasionally check out where you have been.'
- Willie Jolley
www.Zend.To
Twitter: @JulesFM
PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
-------------- next part --------------
#!/usr/bin/perl
#
# This script will output the contents of a new preferences.php file based
# on an old preferences.php file and a default copy of the new file.
#
use FileHandle;
use strict;
sub Usage {
print STDERR "Usage:\n";
print STDERR "$0 old-preferences.php supplied-new-preferences.php > new-preferences.php\n";
print STDERR "\n";
}
sub Afterwards {
print STDERR <<"EOAFTER";
I have written your new file. Any preferences that were in your old
file but are not used in the new one have been put right near the
top in a labelled "Obsolete" section.
Once you have checked that new-preferences.php contains what you
want, you can then save your old one and move the new one into
place.
EOAFTER
}
my $oldfname = shift;
my $newfname = shift;
unless ($oldfname && $newfname &&
-f $oldfname && -f $newfname &&
-s $oldfname && -s $newfname) {
Usage();
if ($newfname) {
# We are really trying to do the upgrade, we have a complete command-line
unless (-f $newfname && -s $newfname) {
# Either the .rpmnew doesn't exist, or it's empty
print STDERR "No new supplied file, so just copying your existing file.\n";
system("cat $oldfname"); # Copy the original .conf to the stdout redirect
}
}
exit 1;
}
# $definesorder[n] = key's name, to maintain correct order within file
# $defines{key} = key's value
# Likewise for prefs.
my(@olddefinesorder, %olddefines, @oldprefsorder, %oldprefs);
my(@newdefinesorder, %newdefines, @newprefsorder, %newprefs);
# $definescomments{key} = lines of comments before key's definition
# $definestrailingcomments{key} = Rest of line after definition of key
# $trailingcomments = Rest of file after last define/prefs line
my(%olddefinescomments, %oldprefscomments, %olddefinesep, %oldprefsep, %olddefinestrailingcomments, %oldprefstrailingcomments, $oldtrailingcomments, $oldprefsheader, $oldfinalcomments);
my(%newdefinescomments, %newprefscomments, %newdefinesep, %newprefsep, %newdefinestrailingcomments, %newprefstrailingcomments, $newtrailingcomments, $newprefsheader, $newfinalcomments);
#
# Yes, I know, this should be OO instead of passing a
# zillion refs to arrays.
# But it isn't, so there.
#
# Read in the old preferences.php file
$oldtrailingcomments = &ReadPrefsFile($oldfname, \@olddefinesorder, \%olddefines, \@oldprefsorder, \%oldprefs, \%olddefinesep, \%oldprefsep, \%olddefinescomments, \%oldprefscomments, \%olddefinestrailingcomments, \%oldprefstrailingcomments, \$oldprefsheader, \$oldfinalcomments);
# Read in the new preferences.php file
$newtrailingcomments = &ReadPrefsFile($newfname, \@newdefinesorder, \%newdefines, \@newprefsorder, \%newprefs, \%newdefinesep, \%newprefsep, \%newdefinescomments, \%newprefscomments, \%newdefinestrailingcomments, \%newprefstrailingcomments, \$newprefsheader, \$newfinalcomments);
# Force the version number to the new value
$olddefines{'ZTVERSION'} = $newdefines{'ZTVERSION'}
if defined $olddefines{'ZTVERSION'};
#
# Output hints for the user to help them make their life easier.
#
# Tell them if their clamdscan setting has been changed,
# but does not include --fdpass,
# as adding that option speeds it up a lot.
my $oldclam = $oldprefs{"'clamdscan'"};
if ($oldclam && changed($oldclam, $newprefs{"'clamdscan'"})) {
if ($oldclam !~ /--fdpass/) {
$oldprefscomments{"'clamdscan'"} .= " //\n" .
" // Note: I *strongly* advise you add '--fdpass' for extra speed.\n";
print STDERR <<'EOCLAM';
Note: Your 'clamdscan' setting does not include the command-line option
--fdpass
I strongly advise you include this, as it greatly speeds up virus
scanning of large uploaded files.
I have not added it automatically, but advise you edit it yourself.
The setting is very near the end of the preferences.php file.
EOCLAM
}
}
# Tell them if their 'emailDomainRegexp' setting is still a regexp.
# It's a lot easier for them if they just list their domains in
# localdomains.conf instead.
my $olddom = $oldprefs{"'emailDomainRegexp'"};
if ($olddom =~ m#^'/.*/i?'$#) {
print STDERR <<'EODOM';
Note: Your 'emailDomainRegexp' is currently a nasty big regular expression.
That is really hard to manage and guarantee it is correct.
Instead, set it to
'/opt/zendto/config/internaldomains.conf'
and then in that file, simply list the email domain names used by your
organisation. There are comments at the top of that file to help you.
EODOM
}
#
# Now write out the new file to stdout...
#
# The only "define"s below the prefs are the "AYAH" ones.
# So let's go through old defines until we hit an AYAH.
# Kill off the new defines as we go.
# Then output all their remaining non-AYAH defines.
# Then output all my remaining non-AYAH defines.
# Then remember where we got to in both defines lists.
# Put out their non-AYAH defines in their order
my $olddefineN = 0;
while ($olddefinesorder[$olddefineN] !~ /AYAH/) {
my $key = $olddefinesorder[$olddefineN];
# If they have changed its value, use their comments
if ($olddefines{$key} ne $newdefines{$key}) {
print $olddefinescomments{$key};
print "define(".$key.$olddefinesep{$key}.$olddefines{$key}.");".
$olddefinestrailingcomments{$key}."\n";
} else {
# They have not changed the value, so use new comments
print $newdefinescomments{$key};
print "define(".$key.$newdefinesep{$key}.$newdefines{$key}.");".
$newdefinestrailingcomments{$key}."\n";
}
delete $newdefines{$key};
delete $olddefines{$key};
$olddefineN++;
}
# Put out new remaining non-AYAH defines (in the right order)
my $newdefineN = 0;
while($newdefinesorder[$newdefineN] !~ /AYAH/) {
my $key = $newdefinesorder[$newdefineN];
if ($newdefines{$key}) {
# new define that old file didn't have
print $newdefinescomments{$key};
print "define(".$key.$newdefinesep{$key}.$newdefines{$key}.");".
$newdefinestrailingcomments{$key}."\n";
delete $newdefines{$key};
}
$newdefineN++;
}
# $olddefineN and $newdefineN now contain the index number of the 1st AYAH define
# Next output the prefs header
print $newprefsheader;
# Put all the prefs out in the *new* order.
# Any they defined that are left over will be printed out,
# but all together and commented out with a note.
# We want to put the dead ones at the top so they see them, so we just
# save what we want to output here, and print it *after* the dead ones.
my $outputprefs = "";
foreach my $key (@newprefsorder) {
if (defined($oldprefs{$key}) &&
changed($oldprefs{$key}, $newprefs{$key})) {
# They had set it in their old file,
# and they had changed it from the supplied value.
$outputprefs .= $oldprefscomments{$key}.
" ".$key.$oldprefsep{$key}.$oldprefs{$key}.",".
$oldprefstrailingcomments{$key}."\n";
} else {
# It is either new, or they hadn't changed the supplied value.
$outputprefs .= $newprefscomments{$key}.
" ".$key.$newprefsep{$key}.$newprefs{$key}.",".
$newprefstrailingcomments{$key}."\n";
}
delete $oldprefs{$key};
delete $newprefs{$key};
}
# Any remaining oldprefs are ones that don't exist any more,
# or are ones they have added themselves.
# Comment them out.
if (keys %oldprefs) {
print " //\n" .
" // ** Obsolete settings start here **\n" .
" // You can delete all of this section.\n" .
" //\n";
foreach my $key (@oldprefsorder) {
next unless $oldprefs{$key};
# Make sure all blank lines within comments start " //"
my $comment = $oldprefscomments{$key};
$comment =~ s/^ *$/ \/\//mg if $comment;
print $comment;
print " // Obsolete: ".$key.$oldprefsep{$key}.$oldprefs{$key}.",".
$oldprefstrailingcomments{$key}."\n";
}
print " //\n" .
" // ** Obsolete settings end here **\n" .
" //\n\n";
}
# Followed by all the active prefs
print $outputprefs;
# The only things left now should be the AYAH defines,
# as they are told not to touch anything down here.
foreach my $key (@newdefinesorder[$newdefineN..$#newdefinesorder]) {
print $newdefinescomments{$key};
print "define(".$key.$newdefinesep{$key}.$newdefines{$key}.");".
$newdefinestrailingcomments{$key}."\n";
delete $newdefines{$key};
}
# Followed by any random crap at the end of the file
print $newfinalcomments;
Afterwards();
exit(0);
# Have they changed the value?
# This is *almost* the not-equal operator, but not quite.
sub changed {
my($old, $new) = @_;
# Is it a number?
return ($new != $old) if $new =~ /^\d+$/;
# Is it a word, such as true or false?
return (lc($new) ne lc($old)) if $new =~ /^[a-zA-Z0-9]+$/;
# Is it the clamdscan command?
if ($old =~ /clamdscan/ && $new =~ /clamdscan/) {
# If the only difference is the presence/absence of '--fdpass'
# then consider it not to have been changed.
$old =~ s/\s*--fdpass//; # Remove any --fdpass option
$new =~ s/\s*--fdpass//;
$old =~ s/^\s*(.*?)\s*$/$1/; # Trim leading+trailing whitespace
$new =~ s/^\s*(.*?)\s*$/$1/;
return ($new ne $old);
}
# Otherwise it's just a string not-equal.
return ($new ne $old);
}
# Read a ZendTo preferences.php file and split it into lots of bits.
sub ReadPrefsFile {
my($filename, $defineorder, $defines, $prefsorder, $prefs, $definesseps, $prefsseps, $definescomments, $prefscomments, $definestrailcomments, $prefstrailcomments, $prefsheadercomments, $finalcomments) = @_;
my($key, $value, $line, $origline, $trailcomment, $sep, $comments);
my($ndefines, $nprefs);
# Find the current year so we can update the ZendTo copyright date
my ($d, $year);
($d,$d,$d,$d,$d,$year,$d,$d,$d) = localtime;
$year += 1900;
my $fh = new FileHandle;
$fh->open($filename) or die "Cannot read file $filename, $!";
$comments = "";
while(<$fh>) {
chomp;
# Force the year in the copyright line that looks vaguely like this
# // Copyright (C) 2016 Julian Field, Jules at Zend dot To
s/(Copyright\D*)(\d+)(\D*Field.*Zend)/$1$year$3/i;
$origline = $_;
s/^\s+//;
s/\s+$//;
$line = $_;
$trailcomment = "";
# It might be a define or a pref, that is totally commented out
if ($line =~ /^\/\//) {
$comments .= "$origline\n";
next;
}
# Is it a define(k,v) line? If so, we are outside the PREFS array
if ($line =~ /^define/) {
# define('AYAH_PUBLISHER_KEY', 'OBSOLETE'); // $NSSDROPBOX_PREFS['ayah_publisher_key']);
$line =~ /^define\(\s*(\'[^']+?\')(\s*,\s*)(.*?)\)\;(.*$)/;
($key, $sep, $value, $trailcomment) = ($1, $2, $3, $4);
#print STDERR "Define .$key. to .$value. then .$trailcomment.\n";
push @$defineorder, $key;
$defines->{$key} = $value;
$definescomments->{$key} = $comments;
$definestrailcomments->{$key} = $trailcomment;
$definesseps->{$key} = $sep;
$ndefines++;
$comments = "";
next;
}
# Is this the start of the prefs?
# Must be a non-commented "$NSSDROPBOX_PREFS =" line.
if ($line =~ /^[^\/]*\$NSSDROPBOX_PREFS\s*=/) {
$$prefsheadercomments = $comments . "$origline\n";
$comments = "";
next;
}
# Is it a non-commented pref line (key => value)?
if ($line =~ /^[^\/]+=\>/) {
undef $key;
undef $key;
undef $value;
$line =~ /^(.*?)(\s*=\>\s*)(.*?)$/;
($key, $sep, $value) = ($1, $2, $3);
# value is 1 of 3 things:
# 1. A sequence of characters ending with \s*, (e.g. TRUE)
# 2. A string starting with ' and ending with '\s*, (e.g. 'word word')
# 3. A string starting with " and ending with "\s*, (e.g. "word word")
# 4. An array, starting with array\s*(\s* and ending with \s*)\s*, (e.g. array('a', 'b'))
if ($value =~ /^array\s*\(/) {
# It is an array. Keep all till the end of the array.
$value =~ s/^array\s*\((.*?)\s*\)\s*,(.*$)/array($1)/;
$trailcomment = $2;
} elsif ($value =~ /^\"/) {
# It is a double-quoted string.
$value =~ s/^(\".*?\"),(.*$)/$1/;
$trailcomment = $2;
} elsif ($value =~ /^\'/) {
# It is a single-quoted string.
$value =~ s/^(\'.*?\'),(.*$)/$1/;
$trailcomment = $2;
} else {
# It's just a non-quoted value
$value =~ s/^([^,]+?),(.*$)/$1/;
$trailcomment = $2;
}
#print STDERR "Set .$key. to .$value. then .$trailcomment.\n";
push @$prefsorder, $key;
$prefs->{$key} = $value;
$prefscomments->{$key} = $comments;
$prefstrailcomments->{$key} = $trailcomment;
$prefsseps->{$key} = $sep;
$nprefs++;
$comments = "";
next;
}
# It's not a setting, so it must just be comments and stuff
$comments .= "$origline\n";
}
$fh->close();
#print STDERR "$filename has $ndefines defines and $nprefs preferences\n";
# There will be trailing comments at the end of the file
$$finalcomments = $comments;
}
More information about the ZendTo
mailing list