feed suddenly negative

So i have almost got things down like i want... only to discover that my kwh feed suddenly turns negative

from the emontx i transfer pulses and watt from the meter.... can it really be the sketch that uses micros to calc watt that is at fault?

 

my graphs seems ok... http://energymonitor-dk.dns4e.net/emoncms/boelle/

 

but if you look at the inputs and then feeds you will note that kwh which gets it data from watt feed is now negative...(pics attached)

 

the feed was ok when i last cleared out the test data...

 

non related to emoncms it seems my ISP from time to time updates my router and reboot it... when that happens the pi loose wifi... anyone know of a small script that can check if connection and if not down the connection and establish it gain? 

boelle's picture

Re: feed suddenly negative

about the wifi problem this did the trick for me: http://harizanov.com/2013/04/auto-reconnect-wifi-on-raspberry-pi/

Robert Wall's picture

Re: feed suddenly negative

kWh feed: I am guessing that it went from a positive value to a negative in one big step, so my first thought is a number - maybe an intermediate step in the process - has overflowed and become negative. What was the last positive value and what was the first negative one? What processing is done on that to get to the feed?

boelle's picture

Re: feed suddenly negative

yep.. from one positive to a BIG negative in one step

 

last value is 80.8 recorded at 14:13:40 and then no values after that.. not even 0... i then slowly go forward and i do not see a datapoint before 14:15:49... the value was now -175477.7

what i do is take the watt number transmitted and log that... i then do the kwh processing on the wattnumber and also log that...

now the same emontx also transmit my hotwater cylinder temp and numbers from my heat meter... the hotwater messurements are also completely gone in that timetimespan...but they return normal as they are direct from input to log without any processing,...

i also checked the watt feed that the kwh builds on and that one is not transmitted either in that timespan...

so i can see the emontx did not transmit in that time... but should that not have caused just 0's ?? 

my water emontx that only counts pulses and calc flowrate came through fine in the timeperiod... and they are feed from the same power supply... i feed one with and mini usb and then via 2 short wires i feed the other...

 

first datapoint was recored 15/10-14 14:37:30

 

EDIT: both sketches are attached.... just if there was a fault here

EDIT2: http://energymonitor-dk.dns4e.net/emoncms/vis/auto?feedid=78 shows the feed...

 

Robert Wall's picture

Re: feed suddenly negative

I think this is one for Trystan. I've just taken the step in Watts and divided it by the time difference in hours and it comes close to 2^32 (close enough to raise a suspicion but not close enough to be totally convincing).

boelle's picture

Re: feed suddenly negative

oki, will keep an eye out... 

 

but would missing transmission cause this or would the kwh number just stay still until transmission gets back?

Robert Wall's picture

Re: feed suddenly negative

It's not the register value out of the Kamstrup meter then, but the power value in Watts that's being accumulated and turned into kWh in emonCMS that's the problem.

I have not got to grips with the new feed engines, so I can't give you a definitive answer on that. The sensible things to do would be either to assume nothing and interpolate between the last value and the next, or to assume zero, or to assume the last value for all the missing points. None of those would give the answer that you got, so I think the likely explanation is it saw a corrupted number and believed it.

pb66's picture

Re: feed suddenly negative

Although boelle hasn't shown his processing of that input, the input.png shows no additional processing icons, just "log" and "kwh" so I suspect the power value is just logged to a feed and then logged as kwh's. which you would expect to be simple enough to spot any anomaly in the power feed that could lead to such a deflection.

Unless the time suddenly jumped back to 1970 for a while. ~570w for minus 35years is ~ -175000Kwh and wouldn't be noticeable in the power feed as the timestamp was 1970, there would just be a hole in the data, which he says he has.

Is it possible a read-only time issue could of caused this? if rebooted without an internet connection the date could revert to 1970 if using an unmodified SD card image.

What is your software setup on the Pi, Boelle ? Are you using the read-only SD image ? Do you have a permanent internet connection? Do you have any data input for that period ?

Paul

pb66's picture

Re: feed suddenly negative

Somehow I lost a whole decade, that should be ~ 445w for minus 45years :-)

boelle's picture

Re: feed suddenly negative

this has not to do with the kamstrup meter at all... it just the simple blinker on my electricity meter...

 

i'm using the timeseries engine as that allows me to do the intervals i want.

i'm still poking my mind how it can make missing transmits so much negative... 

boelle's picture

Re: feed suddenly negative

"Although boelle hasn't shown his processing of that input, the input.png shows no additional processing icons, just "log" and "kwh""

 

thats right... i log the watt number as they come in because i want to graph my load over time

i then choose to process that number as kwh and log it also.... 

 

What is your software setup on the Pi, Boelle: Powered by openenergymonitor.org | vmaster:8.2.8, bufferedwrite:0.0.1

Are you using the read-only SD image ? no, trystan told me the best thing was to use the lowwrite image and if i wanted put data part of that image to a harddrive... since i had a 1TB drive arround i use that as data and the sd only yo boot on, but content wise the harddrive is the same as the sd image

 Do you have a permanent internet connection? yes 24 mbit down and 12 mbit up

Do you have any data input for that period ? not from that emontx no... but i did get from the other emontx that monitors my water meter, both are power from the same source, one connected to mini usb and then via 2 hookup wires to the other

pic of my processing attached

boelle's picture

Re: feed suddenly negative

just wondering here... and i cant remember...

but if i did loose connection (my ISP tends to upgrade and reboot the cable router as they please) and i then reboot the pi to get wifi connection again it would cause it to revert to 1970's...

now i did put up a script that checks for connection and ping the router... the script runs every 5 mins, if no connection it will recreate it... that at least will reduce my need to reboot the pi if the router drops the connection.

but still it does not explain why it was only one emontx that did not get its transmits through in that timespan... i know that they are both powered... if one is the other is also, no bad soldering here... 

if it was only the reboot of the pi it should have been a drop in transmit from everything but in much shorter timespan as the pi only does very little loading from the sd and the rest from the harddrive...

now the negative value is slowly climbing towards 0 again so at least that tells me that its counting right again... and if i could just edit the data i could simply add the initial negative value to all datapoint and come up with somewhat ok data....

problem is that timeseries feeds does not show up in phpmyadmin... just the feednames... the old images a year back did allowed me to edit the data direct in phpmyadmin...

 

just thoughts here.... 

 

EDIT: small video of my setup in the utility closet (1 min long) http://youtu.be/o1Am2iLlCh8 (uploading right now)

what you dont see in the video is the 3 core cable that feeds through the fusebox then a hole out to the meter that is embedded in the wall on the outside just next to my door. also you dont see the 3 core cable that goes to the top of the hotwater cylinder. (brass pocket installed at factory)

pb66's picture

Re: feed suddenly negative

Can you check your emonhub logfile for any 1970 dates using

cat /var/log/emonhub/emonhub.log

and scroll back looking at the dates down the left.

and also check if you are using a symlinked fake-hwclock file using this command

ls -la /etc/fake-hwclock.data

if it returns "/etc/fake-hwclock.data" ok but it may return something like "/etc/fake-hwclock.data -> /home/pi/data/fake-hwclock.data" and that means you may be vulnerable to the 1970 issue at boot up until a internet connection is established.

if it is this issue it would only need the clock to be out momentarily to register the time calculation in just the kwh log, the other feeds are less sensitive because it would just miss the "1970" time stamped data and look like it dropped one or more packets, but it only takes 1 "1970" timestamp to throw out the kwh log as the time since the last datapoint is used in every calculation.

Paul

boelle's picture

Re: feed suddenly negative

yep--- loads of 1970 dates

 

1970-01-01 00:00:35,494 INFO EmonHub Pre-Release Development Version (rc1.0)
1970-01-01 00:00:35,503 INFO Opening hub...
1970-01-01 00:00:38,569 WARNING 1 Discarded RX frame 'non-numerical content' : [                                                                                                                                                             '>\x00', '8b']
1970-01-01 00:00:39,202 WARNING 4 Discard RX frame 'information' : ['>', '210g']
1970-01-01 00:00:39,632 WARNING 6 Discard RX frame 'information' : ['>', '15i']
1970-01-01 00:00:40,534 WARNING Send failure: wanted 'ok' but got Can't connect                                                                                                                                                              to redis database, it may be that redis-server is not installed or started see r                                                                                                                                                             eadme for redis installation
1970-01-01 00:00:41,154 WARNING Send failure: wanted 'ok' but got Can't connect                                                                                                                                                              to redis database, it may be that redis-server is not installed or started see r                                                                                                                                                             eadme for redis installation
1970-01-01 00:00:41,734 WARNING Send failure: wanted 'ok' but got Can't connect                                                                                                                                                              to redis database, it may be that redis-server is not installed or started see r                                                                                                                                                             eadme for redis installation
1970-01-01 00:00:42,315 WARNING Send failure: wanted 'ok' but got Can't connect                                                                                                                                                              to redis database, it may be that redis-server is not installed or started see r                                                                                                                                                             eadme for redis installation

 

boelle's picture

Re: feed suddenly negative

the lines above was the complete screen output

 

lrwxrwxrwx 1 root root 31 Aug 13 08:50 /etc/fake-hwclock.data -> /home/pi/data/fake-hwclock.data

 

so i guess it was an 1970 issue after all...

2 Q's

 

1. is there anything i can do to prevent this happen to me again?

2. given that i cant access the feed data through phpmyadmin are there other tools i can use to edit the data so i can adjust the negative numbers to positive?

pb66's picture

Re: feed suddenly negative

 

Run the following command in "read & write mode" (rpi-rw) to sort the time issue

    git clone https://github.com/emonhub/ntp-backup.git ~/ntp-backup && ~/ntp-backup/install

 

You don't want to "adjust the negative numbers to positive" as that would be just as inaccurate. you need to remove the 1 minus datapoint or add on an amount equal to the minus amount to balance it.

It's along shot, but if you can pin point the actual data point that caused the issue sat pm by looking at 

http://energymonitor-dk.dns4e.net/emoncms/boelle/vis/auto?feedid=78 or clicking on the kwh icon on the input page and zooming in to get the time and date which you can convert to UNIXTIME (UTC) and try using this url

http://energymonitor-dk.dns4e.net/emoncms/boelle/feed/deletedatapoint.json?id=78&feedtime=UNIXTIME

​to remove that one datapoint, however I don't know this will work for sure so be cautious.

(you will need to be logged in for the urls to work or add " &apikey=XXXXETC ")

Paul

boelle's picture

Re: feed suddenly negative

remember feed 78 is made up on the basis of feed 77.... so all the datapoint from the time it did go negative and forwards are negative... hence my idea of adjusting them... and by adjusting i simple mean add 175477.5 to all datapoints from the drop up to now... 

as the datapoints  are working towards 0 again would make them positive by the amount i have consumed 

but again there are no way to edit datapoints... only delete them...

 

all datapoints in my watt log are all positive.... no need to mess with those

boelle's picture

Re: feed suddenly negative

a pic on how the feed is going towards 0

pb66's picture

Re: feed suddenly negative

If you've installed ntp-backup that should stop any future occurrences.

Going back and correcting historical data isn't easy as there's no current provision for that. It's not impossible but you need to work out how much effort it's worth to retain that data, it wasn't that long ago you reset the feeds.

You can either "just start over" - easy, less desirable option or wait until you find a way to "correct" all the effected data which is still being logged incorrectly. A third option could be to correct the current value on that feed now, so that from here on it is correct and if at a later date you find a way to correct the couple of days effected great, if not it's not so bad.

You will need to stop emonhub while applying the correction but it should only take a few seconds to complete. you will need to insert a new datapoint with the current unixtime and the new feed value (current feed value + correction)

1 - Stop emonhub using

sudo service emonhub stop

2 - Calculate the NEWVALUE by adding your correction value (175477.5 ?) to the value returned by

http://energymonitor-dk.dns4e.net/emoncms/boelle/feed/value.json?id=78

3 - get the current UNIXTIME from http://www.epochconverter.com/

4 - enter UNIXTIME and NEWVALUE to this url (all one line) to correct the current feed value

http://energymonitor-dk.dns4e.net/emoncms/boelle/feed/insert.json?id=78&time=UNIXTIME&value=NEWVALUE

5 - start emonhub with

sudo service emonhub start

boelle's picture

Re: feed suddenly negative

well i might as well reset the kwh feed...

 

just a shame that timeseres can not be edited from phpmyadmin :-/

boelle's picture

Re: feed suddenly negative

watt and kwh feed cleared of datapoints...

 

but for the next update i suggest a fix of some sorts that prevents this from happening again.. i might not benefit from that as i have taken measure against the need to reboot at all if just the wifi drops.

 

 

pb66's picture

Re: feed suddenly negative

Hi boelle,

Yes. you will need to delete the feed or "just correct from now" and ignore the incorrect data. The penny has just dropped, as you are using the "buffered write" emoncms you cannot edit or delete past data, that was all disabled to achieve the low write levels required. you can only append data to the feed due to buffering.

The time issue is unfortunate, ntp-backup should fix it and hopefully it will get used in future read-only images too. 

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.