RaspberryPi Emon base persistant data?

Hi guys, I am new to the OpenEnergyMonitor system but I do however work for one of the biggest integration specialists & product suppliers to the UK energy management & building controls industry.

I got myself an emonTX v3 and the wireless board for the RPi + pre loaded SD card. All is working and I still can't believe how good and straight foward this is (especially compared to the commercial offerings we use).

I have a question though, am I correct in thinking that when using the Raspberry Pi as an emon Base (pre loaded sd card), that no data is stored persistantly? Reason I ask is because when I rebooted my RPi, I can't seem to find any of my data from yesterday. I do have a few inputs logging to feed.

Also, I looked at the SQL install and can't seem to see any Tables that contain the energy data?

Is this the case? If so its not a problem since I plan on developing an external service that leverages the emon base to fetch data from the tx's and log elsewhere anyway.

Also, great job on this product guys!

Cheers

Tom

pb66's picture

Re: RaspberryPi Emon base persistant data?

Hi Tom, As you've discovered, emonCMS is enabled by default on recent emonSD cards. emonCMS always persists data although not always to an SQL table.

Because the Pi runs from a SDcard the writes need to be reduced to lower the chance of a failing card, so the data is held in low overhead, unindexed "feeds" which can be found in the "phpfina","phpfiwa" and "phptimeseries" folders in "data". these files are written to once per minute to reduce write activity.

emonHub is also included in the SDcard image and is a "gateway" to emoncms and can run totally read only and does not persist data directly, it forwards to one or multiple emoncms installs, remote or local. It buffers data in RAM if a network or server is down. It is designed to be modular in the way that it is the local base for a remote emoncms so there are many emonHubs reporting to a central emonCMS server,

The extended version of emoncms (ie not low-write for SDcard) can write data to SQL and has a lot more functionality and input processes that are not (currently) available to a Pi without installing a hdd so to get a picture of a model closer to your needs you could set up an "extended" emoncms server on another local machine and have emonHub forward the data across a network.

Paul

tomstephens89's picture

Re: RaspberryPi Emon base persistant data?

Hi Paul, thanks for the reply.

That answers my question and like I said, is not an issue since I only intend to use it as a hub to which an external application will receive data. Looking at your feed api, I will probably just make my application call the API of the low-write emon cms install on the pi and retrieve the values from each feed that way.

Where is that data folder?

Another quick question for you....

Whats the transmit power of the radio in the emonTX v3. I am trying to get an idea of maximum range that the pi can be from the tx. Obviously I can fit a big antenna to the module on the pi, but this is only going to be as good as the transmit power from the tx itself.

Any line of sight range testing results you can share?

I am currently running the TX in a cupboard where one of our 3 phase incomers resides and the pi is around 60ft away through two walls and up one floor.

 

 

pb66's picture

Re: RaspberryPi Emon base persistant data?

One of the strengths of the OEM project is the options available and the way you can do your own thing with it, however based on what you've told us the "feed api" method may not be the best way.

For a commercial remote install you want it to be as low-maintenence and reliable as possible, a forwarder only solution has less moving parts, a fixed on disc size and is extremely robust. emonHub's less robust predecessor earned the name "the rock-solid gateway" it really is a install and forget solution.

To be able to access via the feeds api you will also need to set up the server at each location with external access which will require a fixed ip and/or dns, you will then need a custom application that can systematically poll every install rather than just sitting back and watching the data arrive. it will also need to get the backlog if there is a network interruption etc

Plus, using a central emoncms you can have one global account or set up an account for each location, then if you want you can give your client read-only access to just thier data and behind the server you can access all of the tables to get group information via a custom application.

Unfortunately I have never done any RF tests, I have quite a noisy environment and lots of obstacles, There is a lot of info regarding RF performance across the forum, the number of devices is also a factor once you have a lot of traffic collisions will be a factor.

The data is on a 3rd partition as the 2nd is read-only, that is mounted to /home/pi/data usually but it location is specified in the emoncms settings file, normally found at /var/www/emoncms/settings.php

Paul

tomstephens89's picture

Re: RaspberryPi Emon base persistant data?

Hi Paul, Thanks again for the response.

With regards to shipping data off site, we typically do that in a different application, so the API call coming inward to the Pi will actually be coming from a device or server on site. The data is then posted off site via another service we have developed.

We could, long term, just use the hub aspect of openenergymonitor to get the values off the serial line, then write some python to post them outwards into an API of our own... Or like you said, if we just hosted an emoncms instance in one of our datacentres we could then configure the RPi to post its data out to that using the options in the hub config file as is. Im an IT infrastructure architect myself, so routing, static IP's etc is no problem!

Have to say, excellent looking product so far, since my first post we have just written a service to integrate data from emoncms into the Tridium Niagara AX Framework if you know what that is. (The most versatile building integration platform there is, used in 100,000's of installations).

 

tomstephens89's picture

Re: RaspberryPi Emon base persistant data?

I have just noticed that no 'hour' graphs seem to draw. 30 mins or less works fine. Is this a bug or something?

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.