Forward data to remote emoncms server

Hi all.

A few days ago, I managed to get a setup runnig with

- an emonTX in my room monitoring temperature

- a raspberry in the living room

- an emoncms installation on my remote server

The data is sent using Martin's perl gateway. For some reason, the perl scripts is stopped after a few hours. But in the meantime, it works, and I get the temperature logged : rawdata (don't pay attention to units, this was just a test).

Martin told me his gateways are obsolete and I should use an emoncms on the rasp instead.

(I liked the idea of a local simple gateway and a remote server, I don't see the need for a whole local emoncms installation, but I don't really mind.)

So I installed emoncms on the rasp as per the documentation, and I have it working fine, apparently : I get the inputs, and I created feeds just like I had done on the remote server earlier. I get the charts and all, all on the Pi.

Yet, I would like to send the data to my remote server.

There's an option for this in the raspberry module configuration menu that seems pretty self explanatory, but I can't get it to work. I entered domain.tld/emoncms and the API key. It says "Authentication successful". But I don't see anything coming.

Besides, whenever I try to launch raspberrypi_run.php myself, I get the "Already running" message.

Anything I may have forgotten ?

The emoncms installation is in /var/www/ on the Pi but in another location on the server. This doesn't matter, does it ?

How should I proceed for the debugging ? Remove the cron line and launch manually, adding "echo" statements in the script ? Can I keep the cron line and get the "echo" in some logfile ?

Thanks.

Jérôme's picture

Re: Forward data to remote emoncms server

From the source code, it seems the importlock file is never closed. My understanding is that it is released when the script dies, and before that it is here to prevent cron fom relaunching a second instance of the script while the first is still running. Is this correct ?

I commented the crontab line and after a reboot, I can launch the raspberry module script manually.

After a few tests, I could figure what goes wrong. The gethostbyname() function yields an IP address if given an input such as domain.tld, but it fails with "domain.tld/emoncms" or "domain.tld/emoncms/" (the former has another issue : when added ":80", it gives "domain.tld/emoncms:80" which does not seem to work, whereas "domain.tld/emoncms/:80 does, at least when entered into firefox URL bar."

Jérôme's picture

Re: Forward data to remote emoncms server

I managed to get it a little bit further.

"domain.tld" can be extracted from "domain.tld/emoncms/". Then gethostbyname() can get the IP address.

But then, next failure is the GET request that is made with a "malformed Host header".

I worked that around with some more hardcoding and regex trickery, but ultimately, I think there should be a new field added in the config, so as to enter :

server : domain.tld

directory / path / whatever-we-want-to-call-that : emoncms (or whatever the path actually is)

apikey :

What do you think about that ?

FWIW, here's what I did :

--------------------------------------------------------------------------------

diff --git a/raspberrypi_run.php b/raspberrypi_run.php
index 96ed29e..87f4610 100644
--- a/raspberrypi_run.php
+++ b/raspberrypi_run.php
@@ -211,8 +211,13 @@

 function getcontent($server, $port, $file)
 {
+
    $cont = "";
-   $ip = gethostbyname($server);
+
+   $serv = preg_replace("/\/.*/",'',$server);
+   $dir = '/emoncms'; // could be regexed
+
+   $ip = gethostbyname($serv);
    $fp = fsockopen($ip, $port);
    if (!$fp)
    {
@@ -220,17 +225,20 @@ function getcontent($server, $port, $file)
    }
    else
    {
-       $com = "GET $file HTTP/1.1\r\nAccept: */*\r\nAccept-Language: de-ch\r\nAccept-Encoding: gzip, deflate\r\nUser-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows
+      $com = "GET $dir$file HTTP/1.1\r\nAccept: */*\r\nAccept-Language: de-ch\r\nAccept-Encoding: gzip, deflate\r\nUser-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windo
+
+   echo "\n".$com."\n";
+
        fputs($fp, $com);
/* Don't realy need to fetch output as it slows us down
        while (!feof($fp))
--------------------------------------------------------------------------------

Care should also be taken to catch exception such as missing leading or trailing slash in the directory/path, as this can lead to failures.

Jérôme's picture

Re: Forward data to remote emoncms server

Hi again.

I think I got that done.

I added a new field to allow for a remote path to be specified.

Do you think this is useful ? Was there a simpler way to answer my need (remote server with custom path) ?

Obviously, I'd be glad to see this modest contribution merged into the project.

What is the best way to send contributions ?

Here is a diff, based on latest commit :

(I hope I didn't break anything. I did not do extensive testing, I just checked it worked in my case. I don't claim to have good enough knowledge of the emoncms code architecture.)

-----------------------------------------------------------------------------------------------

diff --git a/raspberrypi_controller.php b/raspberrypi_controller.php
index 12954ca..0857d04 100644
--- a/raspberrypi_controller.php
+++ b/raspberrypi_controller.php
@@ -43,15 +43,17 @@
       $frequency = intval(get('frequency'));
       $baseid = intval(get('baseid'));
       $remotedomain = urldecode(get('remotedomain'));
+      $remotepath = get('remotepath');
+      if($remotepath[0]!='/') {$remotepath='/'.$remotepath;} // ensure leading slash in remotepath
       $remoteapikey = db_real_escape_string(preg_replace('/[^.\/A-Za-z0-9]/', '', get('remoteapikey')));

       $remotesend = false;
-      if ($remotedomain && $remoteapikey) {
-        $result = file_get_contents("http://".$remotedomain."/time/local.json?apikey=".$remoteapikey);
+      if ($remotedomain && $remoteapikey && $remotepath) {
+        $result = file_get_contents("http://".$remotedomain.$remotepath."/time/local.json?apikey=".$remoteapikey);
         if ($result[0]=='t') $remotesend = true;
       }

-      raspberrypi_set($userid,$apikey,$sgroup,$frequency,$baseid,$remotedomain,$remoteapikey,$remotesend);
+      raspberrypi_set($userid,$apikey,$sgroup,$frequency,$baseid,$remotedomain,$remotepath,$remoteapikey,$remotesend);

       $output['message'] = "Raspberry PI settings updated";
     }
diff --git a/raspberrypi_model.php b/raspberrypi_model.php
index f4d0aef..5a798e2 100644
--- a/raspberrypi_model.php
+++ b/raspberrypi_model.php
@@ -18,16 +18,16 @@

     if (!$row)
     {
-      db_query("INSERT INTO raspberrypi ( userid, apikey, sgroup ,frequency, baseid, remotedomain, remoteapikey, remotesend) VALUES ( '0' , '' ,'1','4','15' ,'emoncms.
+      db_query("INSERT INTO raspberrypi ( userid, apikey, sgroup ,frequency, baseid, remotedomain, remotepath, remoteapikey, remotesend) VALUES ( '0' , '' ,'1','4','15
       $result = db_query("SELECT * FROM raspberrypi");
       $row = db_fetch_array($result);
     }
     return $row;
   }

-  function raspberrypi_set($userid,$apikey,$sgroup,$frequency,$baseid,$remotedomain,$remoteapikey,$remotesend)
+  function raspberrypi_set($userid,$apikey,$sgroup,$frequency,$baseid,$remotedomain,$remotepath,$remoteapikey,$remotesend)
   {
-    db_query("UPDATE raspberrypi SET `userid` = '$userid', `apikey` = '$apikey', `sgroup` = '$sgroup', `frequency` = '$frequency', `baseid` = '$baseid' ,`remotedomain`
+    db_query("UPDATE raspberrypi SET `userid` = '$userid', `apikey` = '$apikey', `sgroup` = '$sgroup', `frequency` = '$frequency', `baseid` = '$baseid' ,`remotedomain`
   }

   function raspberrypi_running()
diff --git a/raspberrypi_run.php b/raspberrypi_run.php
index 96ed29e..45a1af6 100644
--- a/raspberrypi_run.php
+++ b/raspberrypi_run.php
@@ -56,10 +56,11 @@
   $baseid = $settings['baseid'];

   $remotedomain = $settings['remotedomain'];
+  $remotepath = $settings['remotepath'];
   $remoteapikey = $settings['remoteapikey'];

   $sent_to_remote = false;
-  $result = file_get_contents("http://".$remotedomain."/time/local.json?apikey=".$remoteapikey);
+  $result = file_get_contents("http://".$remotedomain.$remotepath."/time/local.json?apikey=".$remoteapikey);
   if ($result[0]=='t') {echo "Remote upload enabled - details correct \n"; $sent_to_remote = true; }

   // Create a stream context that configures the serial port
@@ -114,9 +115,9 @@
         if ($settings['frequency'] !=$frequency) {$frequency = $settings['frequency']; fprintf($f,$frequency."b"); }
         if ($settings['baseid'] !=$baseid) {$baseid = $settings['baseid']; fprintf($f,$baseid."i");}

-        if ($settings['remotedomain'] !=$remotedomain || $settings['remoteapikey'] !=$remoteapikey)
+        if ($settings['remotedomain'] !=$remotedomain || $settings['remoteapikey'] !=$remoteapikey || $settings['remotepath'] !=$remotepath)
         {
-          $result = file_get_contents("http://".$remotedomain."/time/local.json?apikey=".$remoteapikey);
+          $result = file_get_contents("http://".$remotedomain.$remotepath."/time/local.json?apikey=".$remoteapikey);
           if ($result[0]=='t') {echo "Remote upload enabled - details correct \n"; $sent_to_remote = true; }
         }

@@ -133,7 +134,7 @@
         $remotedata .= "]";
         echo "Sending remote data";
         //echo $remotedata."\n";
-        getcontent($remotedomain,80,"/input/bulk.json?apikey=".$remoteapikey."&data=".$remotedata);
+        getcontent($remotedomain,80,$remotepath."/input/bulk.json?apikey=".$remoteapikey."&data=".$remotedata);
         $ni = 0; $remotedata = "[";
         $start_time = time();
       }
@@ -211,7 +212,9 @@

 function getcontent($server, $port, $file)
 {
-   $cont = "";
+   //$cont = "";
    $ip = gethostbyname($server);
    $fp = fsockopen($ip, $port);
    if (!$fp)
@@ -220,7 +223,10 @@ function getcontent($server, $port, $file)
    }
    else
    {
-       $com = "GET $file HTTP/1.1\r\nAccept: */*\r\nAccept-Language: de-ch\r\nAccept-Encoding: gzip, deflate\r\nUser-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows
+      $com = "GET $file HTTP/1.1\r\nAccept: */*\r\nAccept-Language: de-ch\r\nAccept-Encoding: gzip, deflate\r\nUser-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows N

        fputs($fp, $com);
 /* Don't realy need to fetch output as it slows us down
        while (!feof($fp))
diff --git a/raspberrypi_schema.php b/raspberrypi_schema.php
index 26e0355..06ab2d9 100644
--- a/raspberrypi_schema.php
+++ b/raspberrypi_schema.php
@@ -8,6 +8,7 @@ $schema['raspberrypi'] = array(
   'baseid' => array('type' => 'int(11)'),
   'running' => array('type' => 'int(11)'),
   'remotedomain' => array('type' => 'text'),
+  'remotepath' => array('type' => 'text'),
   'remoteapikey' => array('type' => 'text'),
   'remotesend' => array('type' => 'int(11)')
 );
diff --git a/raspberrypi_view.php b/raspberrypi_view.php
index 09bbde7..ce74e2c 100644
--- a/raspberrypi_view.php
+++ b/raspberrypi_view.php
@@ -37,6 +37,7 @@
 <p><b>Forward data to remote emoncms server</b></p>

 <p>Domain name<br><input type="text" name="remotedomain" value="<?php echo $settings['remotedomain']; ?>" /></p>
+<p>Path to emoncms<br><input type="text" name="remotepath" value="<?php echo $settings['remotepath']; ?>" /></p>
 <p>Write apikey<br><input type="text" name="remoteapikey" value="<?php echo $settings['remoteapikey']; ?>" /></p>
 <?php if ($settings['remotesend']) echo "<p><b>Authentication successful</b></p>"; else echo "<p><b>Incorrect remote server details</b></p>"; ?>

-----------------------------------------------------------------------------------------------

Note this adds an attribute in the database. The magical "Update and check" button in the Admin tab sorted this all out perfectly.

TrystanLea's picture

Re: Forward data to remote emoncms server

Hello Jerome

This sounds great, I would be happy to merge it into the module. Can you send me a pull request to the raspberry pi repo on github? You will need to make your own fork first, commit to that and then send a pull request.

Thanks a lot

Trystan

Jérôme's picture

Re: Forward data to remote emoncms server

Done.

Thank you !

 

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.