[ZendTo] Re: rrd fixes
ROGERS Richard
Richard.Rogers at staffs.ac.uk
Fri Sep 23 10:43:50 BST 2011
Yes, I agree, that's why I'm so baffled... I can't see that there's anything wrong with the commands that ZendTo is executing (or the ones I'm executing manually), yet I'm ending up with no valid data in the RRD.
All very strange - maybe some rrdtool bug... (I'll check for any updates/patches).
I will keep digging and post back if I find out anything useful. (Phil, is yours working correctly now?)
Thanks for your input
Regards
Richard
From: zendto-bounces at zend.to [mailto:zendto-bounces at zend.to] On Behalf Of Jules
Sent: 23 September 2011 10:13
To: ZendTo Users
Subject: [ZendTo] Re: rrd fixes
On 23/09/2011 09:55, ROGERS Richard wrote:
Hi Jules -
I stuck in an extra print statement to get the creation command; I now have
# /usr/bin/php /opt/zendto/sbin/rrdInit.php /opt/zendto/config/preferences.php
Initial creation of RRD file (start date of 2011-09-16)
/usr/bin/rrdtool create /var/zendto/rrd/zendto.rrd --start 20110916 --step 86400 DS:dropoff_count:GAUGE:90000:0:U DS:total_files:GAUGE:90000:0:U DS:total_kb:GAUGE:90000:0:U DS:files_per_dropoff:COMPUTE:total_files,dropoff_count,/ RRA:AVERAGE:0.5:1:365 RRA:AVERAGE:0.5:30:12017
/usr/bin/rrdtool update /var/zendto/rrd/zendto.rrd 1316214000:1:1:11.9
18
/usr/bin/rrdtool update /var/zendto/rrd/zendto.rrd 1316732400:1:1:7.0
That proves to me that it is doing the right thing. You have got 1 drop-off of 11.9KB (or is it MB?) from about 6.5 days ago and 1 drop-off of 7KB from about 9 or 10 hours ago.
Look in your list of all drop-offs (login as an admin) and you should see that in the table.
Is that anywhere close to the truth? If so, the ZendTo RRD code is doing the right thing.
Jules.
The odd thing about this is that if I run the rrdtool create command manually I get a much larger RRD (maybe my print statement is wrong - just crudely pasted in from the line that does the system command - PHP is not a language I speak fluently!)
-rw-r--r-- 1 root root 398200 Sep 23 09:08 zendto.rrd.manual1
-rw-r--r-- 1 root root 17496 Sep 16 14:24 zendto.rrd.old1
Adding data manually fails now though (not clear at all what is different from yesterday!) - no errors, just NaNs in the data sequence.
Anyway, reverting to a saved copy of an old RRD, I seem only to be able to get valid data to register if I increase the timestamp to something well in the future (like 1316905200).
Some issue with intervals??
Thanks and regards
Richard
From: zendto-bounces at zend.to<mailto:zendto-bounces at zend.to> [mailto:zendto-bounces at zend.to] On Behalf Of Jules
Sent: 23 September 2011 09:25
To: ZendTo Users
Subject: [ZendTo] Re: rrd fixes
Can you send me the output of rrdInit.php? I want to see all the rrdtool commands it is doing.
On 23/09/2011 09:02, ROGERS Richard wrote:
Aaagh, back to all NaN... I'll dig a bit and see if I can get anything useful.
Regards
Richard
From: zendto-bounces at zend.to<mailto:zendto-bounces at zend.to> [mailto:zendto-bounces at zend.to] On Behalf Of Jules
Sent: 22 September 2011 15:27
To: ZendTo Users
Subject: [ZendTo] Re: rrd fixes
Yes, give it a few data points to work on.
With my own production site, all the stats now match the counts in the ZendTo database (not looking at the RRD database). So the graphs say what they should say.
Jules.
On 22/09/2011 14:54, ROGERS Richard wrote:
Hmmm... initial data population still seems to have an oddity somewhere - perhaps because I had only one unexpired dropoff in the database - it still gave all "NaN" in the RRD. However a manual update with a forged time seemed to insert a proper data line. I have done a new upload, and will recreate the database now and then check the overnight RRD update in the morning.
Thanks and regards
Richard
From: zendto-bounces at zend.to<mailto:zendto-bounces at zend.to> [mailto:zendto-bounces at zend.to] On Behalf Of Jules
Sent: 22 September 2011 14:19
To: ZendTo Users
Subject: [ZendTo] Re: rrd fixes
Right. Now I've got it to generate correct data, and correct graphs.
It really does match what the "All Dropoffs" says it should say.
So please try the attached versions. You must delete your zendto.rrd file first. It will re-create all the data, so you won't lose any old stats. Then run rrdInit.php and then rrdUpdate.php, then check the graphs on your site.
Please let me know if you think this works. If it does, I'll start on a release.
Thanks for your patience,
Jules.
On 21/09/2011 09:48, ROGERS Richard wrote:
I'm sorry to say I think there are still problems with this. I don't think the data is getting into the RRD (which is why the graphs are flat!). My experience was as follows:
# /usr/bin/php /opt/zendto/sbin/rrdInit.php /opt/zendto/config/preferences.php
Initial creation of RRD file (start date of 2011-09-07)
16
/usr/bin/rrdtool update /var/zendto/rrd/zendto.rrd 1315436400:1:1:2926562.0
17
/usr/bin/rrdtool update /var/zendto/rrd/zendto.rrd 1316214000:1:1:11.9
Which looks OK, although on closer inspection I think the dates may be suspect. What's actually shown in the Outbox is:
Claim ID Sender Recipient Size Created
daWQaXD7S2bMGMfK ROGERS Richard, Staffordshire University staff (xxxxxx at staffs.ac.uk<mailto:xxxxxx at staffs.ac.uk>) RMR <richard at xxxxxxxxx.uk><mailto:richard at xxxxxxxxx.uk>
11.9 KB 16 Sep 2011 02:14:53 PM
GJYBiNngywQJjXiR ROGERS Richard, Staffordshire University staff (xxxxxx at staffs.ac.uk<mailto:xxxxxx at staffs.ac.uk>) RMR <richard at xxxxxxxxx.uk><mailto:richard at xxxxxxxxx.uk>
2.8 GB 07 Sep 2011 07:42:47 PM
Now an rrdtool dump (excerpt) shows:
<!-- 2011-08-31 01:00:00 BST / 1314748800 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2011-09-01 01:00:00 BST / 1314835200 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2011-09-02 01:00:00 BST / 1314921600 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2011-09-03 01:00:00 BST / 1315008000 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2011-09-04 01:00:00 BST / 1315094400 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2011-09-05 01:00:00 BST / 1315180800 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2011-09-06 01:00:00 BST / 1315267200 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2011-09-07 01:00:00 BST / 1315353600 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2011-09-08 01:00:00 BST / 1315440000 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2011-09-09 01:00:00 BST / 1315526400 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2011-09-10 01:00:00 BST / 1315612800 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2011-09-11 01:00:00 BST / 1315699200 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2011-09-12 01:00:00 BST / 1315785600 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2011-09-13 01:00:00 BST / 1315872000 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2011-09-14 01:00:00 BST / 1315958400 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2011-09-15 01:00:00 BST / 1316044800 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2011-09-16 01:00:00 BST / 1316131200 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
</database>
It ends after 16/9 (because the last dropoff was on the 16th? - the time stamp on the file is "now" and the time of last update at the top is "<lastupdate>1316214000</lastupdate> <!-- 2011-09-17 00:00:00 BST -->")
Hope this helps in further diagnosis!
ZendTo 4.05 on CentOS VM.
Regards
Richard
From: zendto-bounces at zend.to<mailto:zendto-bounces at zend.to> [mailto:zendto-bounces at zend.to] On Behalf Of --[ UxBoD ]--
Sent: 21 September 2011 08:24
To: ZendTo Users
Subject: [ZendTo] Re: rrd fixes
Hmm, graphs are all empty :(
--
Thanks, Phil
________________________________
From: "Jules" <Jules at zend.to><mailto:Jules at zend.to>
To: "ZendTo Users" <zendto at zend.to><mailto:zendto at zend.to>
Sent: Tuesday, 20 September, 2011 9:35:05 PM
Subject: [ZendTo] Re: rrd fixes
To save you poor busy folks a load of patching, I might just release a new version of ZendTo tomorrow. It's the first in a month or so, and it will iron out these niggles for you.
There won't be any new preferences.php settings or database changes, though there are a few *.tpl template changes. Should be a painless upgrade (he says, cursing it forever :-)
Cheers,
Jules.
On 20/09/2011 21:14, Jules wrote:
I've found and fixed the problems you've all been having with rrdInit.php and rrdUpdate.php.
Attached are new copies of
/opt/zendto/sbin/rrdInit.php
/opt/zendto/sbin/rrdUpdate.php
/opt/zendto/lib/MySQL.php
/opt/zendto/lib/SQLite.php
You will probably need to delete your RRD data file (/var/zendto/rrd/zendto.rrd by default) first, and then copy in the 4 files. Then run
/opt/zendto/sbin/rrdInit.php /opt/zendto/config/preferences.php
/opt/zendto/sbin/rrdUpdate.php /opt/zendto/config/preferences.php
The rrdInit script creates the data file if necessary and updates it with all the latest stats.
The rrdUpdate script just creates all the graphs.
Sorry it's taken me so long to work out the solution to this!
Jules
_______________________________________________
ZendTo mailing list
ZendTo at zend.to<mailto:ZendTo at zend.to>
http://mailman.ecs.soton.ac.uk/mailman/listinfo/zendto
Jules
--
Julian Field MEng CITP CEng
www.Zend.To<http://www.Zend.To>
Follow me at twitter.com/JulesFM
PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
'Teach a man to reason, and he will think for a lifetime.' - Phil Plait
'All programs have a desire to be useful' - Tron, 1982
_______________________________________________
ZendTo mailing list
ZendTo at zend.to<mailto:ZendTo at zend.to>
http://mailman.ecs.soton.ac.uk/mailman/listinfo/zendto
_______________________________________________
ZendTo mailing list
ZendTo at zend.to<mailto:ZendTo at zend.to>
http://mailman.ecs.soton.ac.uk/mailman/listinfo/zendto
Jules
--
Julian Field MEng CITP CEng
www.Zend.To<http://www.Zend.To>
Follow me at twitter.com/JulesFM
PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
'It's okay to live without all the answers' - Charlie Eppes, 2011
'All programs have a desire to be useful' - Tron, 1982
'That is the land of lost content,
I see it shining plain,
The happy highways where I went,
And cannot come again.' - A.E. Houseman
_______________________________________________
ZendTo mailing list
ZendTo at zend.to<mailto:ZendTo at zend.to>
http://mailman.ecs.soton.ac.uk/mailman/listinfo/zendto
Jules
--
Julian Field MEng CITP CEng
www.Zend.To<http://www.Zend.To>
Follow me at twitter.com/JulesFM
PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
'It's okay to live without all the answers' - Charlie Eppes, 2011
'All programs have a desire to be useful' - Tron, 1982
'That is the land of lost content,
I see it shining plain,
The happy highways where I went,
And cannot come again.' - A.E. Houseman
_______________________________________________
ZendTo mailing list
ZendTo at zend.to<mailto:ZendTo at zend.to>
http://mailman.ecs.soton.ac.uk/mailman/listinfo/zendto
Jules
--
Julian Field MEng CITP CEng
www.Zend.To<http://www.Zend.To>
Follow me at twitter.com/JulesFM
PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
'It's okay to live without all the answers' - Charlie Eppes, 2011
'All programs have a desire to be useful' - Tron, 1982
'That is the land of lost content,
I see it shining plain,
The happy highways where I went,
And cannot come again.' - A.E. Houseman
_______________________________________________
ZendTo mailing list
ZendTo at zend.to<mailto:ZendTo at zend.to>
http://mailman.ecs.soton.ac.uk/mailman/listinfo/zendto
Jules
--
Julian Field MEng CITP CEng
www.Zend.To<http://www.Zend.To>
Follow me at twitter.com/JulesFM
PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
'It's okay to live without all the answers' - Charlie Eppes, 2011
'All programs have a desire to be useful' - Tron, 1982
'That is the land of lost content,
I see it shining plain,
The happy highways where I went,
And cannot come again.' - A.E. Houseman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ecs.soton.ac.uk/pipermail/zendto/attachments/20110923/f265c3c7/attachment-0001.html
More information about the ZendTo
mailing list