[ZendTo] Re: [BUG?] Upgrading ZendTo SQLite v2 database to v3 loses dropoffs' contents

[Mike Brudenell]

Dear all,

After a lot of brain-ache here we have found the problem and it seems to be
related to upgrading from SQLite v2 to v3 along with an assumption(/bug?)
within ZendTo…

This support page describes the tables and their elements used by ZendTo:

http://www.zend.to/technical.php


Note the magical element *rowId* which *"is referenced from several other
tables and gives the unique number of each complete dropoff".*
*
*
Although described here, rowId is not a field of the table created by
ZendTo, but is instead a hidden auto-incrementing value maintained by
SQLite itself. It starts off with the value 1 and auto-increments for each
insertion.

When the v2 database is dumped out the hidden rowId field is not included
for each record. So when the dumped data is re-imported into SQLite v3 to
get a database in the new format the rowId starts at 1 again.

This means each imported record gets a different rowId from what it had
before.

Unfortunately ZendTo uses the rowId to tie the data in various tables
together and, to do this, stores it within some records. But because each
record now has a brand new rowId in the v3 database the linkage is broken,
and results in dropoffs showing as containing no files.

(It's a bit like renumbering a BASIC program but the renumber command not
updating the line numbers within program's GOTO statements!)

A solution would be for ZendTo to generate its own unique key number for
each record in the tables and use this to link them together rather than
the auto-generated one.

(Or there may be some other way we've not been able to find of exporting
the data such that it includes the hidden rowId values?)

We've worked around it by updating the hidden rowId values in the new v3
database to match the ones from the old v2 database, which has made all the
files in the dropoffs spring back into existence. (We just hope these
entries will be expired/removed before the auto-incrementing counter works
its way up there again!)

Cheers,
Mike B-)

On 12 March 2013 09:44, Mike Brudenell <mike.brudenell at york.ac.uk> wrote:

> Hi, all!
>
> We're in the middle of upgrading ZendTo our live DropOff Service from
> 4.08-something to the latest 4.11-9, and from Ubuntu 10.4 LTS to 12.04 LTS.
>
> We had no problems when we tried did this same upgrade on our test server
> a few weeks back. However today we've been left with a ZendTo that has lost
> sight of the contents of the dropoffs that existed within it.
>
> Specifically, logging in as an Admin and showing all the dropoffs lists
> them with their dropoff IDs, but shows 0 bytes against each. Clicking on
> any of the dropoffs lists no files as being present.
>
> Uploading a new file into the service enters it properly, its dropoff
> shows the bytecount and the filenames are listed within the dropoff.
>
> We're confident that we went through the same upgrade procedure, including
> exporting the SQLite 2 database and importing it into SQLite 3 to
> convert/upgrade it, so are rather baffled.
>
> Can anyone shed any light/have anything to try to relink the dropoffs'
> files (which still exist on disk) into their corresponding dropoffs?
>
> Cheers,
> Mike B-)
>
> --
> IT Services, The University of York, Heslington, York YO10 5DD, UK
> Tel: +44-1904-323811
> Disclaimer: <http://www.york.ac.uk/docs/disclaimer/email.htm>
>



-- 
IT Services, The University of York, Heslington, York YO10 5DD, UK
Tel: +44-1904-323811
Disclaimer: <http://www.york.ac.uk/docs/disclaimer/email.htm>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ecs.soton.ac.uk/pipermail/zendto/attachments/20130312/c01739c8/attachment-0001.html