export Solar-Data from Solar-Log 1200 Meter into EmonCMS

The "Solar-Log 1200" Meter can export its logged pv-data of our houseproduction and send it daily by ftp to an external server.
So i have now the data on an own hosted Debian-server with EmonCMS running.  

Now i want to get that data into our EmonCMS.

Is there already a CSV-Import-Module available for EmonCMS ?

When not:

  • do i have to write a script and poll the emoncms by web? 
  • or is there also another and faster way to write the data into EmonCMS?
    (directly to the Database)

Thanks for any help into the right direction!

octagon's picture

Re: export Solar-Data from Solar-Log 1200 Meter into EmonCMS

Hey Mich,

I don't know of an import module but it would be easy to create it. You'd need to read the CSV one line at a time and send each entry using the API calls. If you go to your feeds page top right is a link "Feed API Help" 

That page will tell you the format of the web call you need to make. 

pb66's picture

Re: export Solar-Data from Solar-Log 1200 Meter into EmonCMS

Just to add to octagon's suggestion, if you are using emonHub you can also write a small python script like the one suggested in this discussion, to post via socket. Just change the code to open socket loop through sending each record before closing the socket rather than opening a socket to post each record.

A recent mod to emonhub's " testing " branch on git, allows pre-timestamped data in the format unixtime nodeid val1 val2 val3. This is work in progress and not yet in the "released" version if you wish to try it out.

It will then except data in the format of unixtime nodeid val1 val2 val3 etc via the above mentioned socket interfacer if a " timestamped = true " setting is added to the sockets [[[runtimesettings]]] .

Depending on how much csv data you have you may also need to increase the buffer size of the emonCMS reporter in emonhub.conf as the default of 1000 frames will fill up very fast, while a separate thread will be posting the data to emoncms 100 frames at a time, add a line like " buffer_size = 10000 " to [reporters] [[emonCMS]] [[[init_settings]]], the 100 frames per post can also be increased upto 250 by adding a setting " batchsize = 250 " in  [reporters] [[emonCMS]] [[[runtimesettings]]].

mich's picture

Re: export Solar-Data from Solar-Log 1200 Meter into EmonCMS

perfect!

thank you pb66 and octagon for pointing me to the right direction and thank you for your help!

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.