Archived Forum

Not sure what it implies, but if you want to start with a clone of the log to feed function, you may want to change the parameters list in there:

public function my_test($feedid, $time_now, $value)

See:

public function log_to_feed($id, $time, $value)
    {
      $this->feed->insert_data($id, $time, $time, $value);

      return $value;
    }

Thanks for the reply Jerome, however with the changes you suggest, I still get a very low value recorded in the database;

0.00813195 instead of the feed value of approx 244 (Mains voltage). The database contains just one row of data and the number increments with each update.

A quick check of the figure recorded suggest that mysql is interpreting the data as 'power to kwh', as after 15 minutes the database is showing 0.0608 (0.244kW / 4 = 0.61kWh)  - Coincidence?

How does the database determines how to process the feed value, or should this be determined in the module process_model.php?

Thank you for your patience!

Paul

Silly question, you did use your test function on a new input without, say, applying the log to feed an power to kwh/d processes before, did you ?

As it is explained in the input configuration page:

Input processes are executed sequentially with the result being passed back for further processing by the next processor in the input processing list.

Mattia, the input processes on the feed are:

x 0.01
Log to feed
My test (this process)

....and I am comparing 'Log to feed' with 'My test', which should be the same??

Paul

Hi Paul, my guess is that there's something weird with your config.

The code you originally posted shouldn't have generated any value any time, since as Jerome pointed out you were using the wrong names for the parameters.

I replicated what I think is your setup:

Change process_model.php and add

$list[26] = array(
        _("My test"),
        ProcessArg::FEEDID,
        "my_test",
        1,
        DataType::REALTIME
      );

(the latest entry in my version of process_model is 25, I guess you have more than one experiment going on ...)

Add the method my_test:

public function my_test($id, $time, $value){

      $this->feed->insert_data($id, $time, $time, $value);

      return $value;

    }

Choose an input and add the new process:

In the input list table:

100 2 TEST-CMD-2 x log test

In the Input config table:

1 x 0.01
2 Log to feed TEST-INPUT-2
3 My test TEST-TEST

In my setup feed TEST-INPUT-2 has id 27 (mysql table feed_27) and feed TEST-TEST has id 60 (table feed_60)

Update feed:

curl -n "http://myurl.com/emoncms/input/post.json?node=100&apikey=myapikey&csv=0,22000,0"

(this is a test node in my emoncms, the relevant part is that I am sending the value 22000 for node 100 id 2, the one I set up previously)

And if I check both in emoncms and the db I have the correct value (22000x0.01 =220 ) in both feeds TEST-INPUT-2 and TEST-TEST

My guess is that you are mixing up feed names/looking at the wrong feeds when checking, or you changed something else that is having a side effect on your values .... is that the only change you made to your emoncms setup ? or else, are you experimenting with a live feed that is fed directly through the rfm12php script ? If so, did you restart it after changing the code ? 

Mattia

Mattia thanks for your patience,

I've deleted out the test feed & input config, returned back to the original modules, so everything is as downloaded from Git.

Rebooted, and made a fresh start using the changes that you have detailed,  and yes, now it works, and is logging the value correctly! I don't know where I had gone wrong, but it's obviously working fine now, and I can use this as a template to try and add a few functions that seem to be missing from emoncms, such as 'maximum value' & 'minimum value', which are present in the the emonGLCD but not emoncms, and would be useful not just for temperatures, but also solar generation levels, usage etc.

Paul

Hi Paul,

next time you feel like you want to experiment some more in the software development field (pun intended :) you could try a git diff instead of deleting and reloading:

root@raspberrypi:/var/www/emoncms/Modules/input# git diff -w process_model.php
diff --git a/Modules/input/process_model.php b/Modules/input/process_model.php
index d17f0ac..619b149 100644
--- a/Modules/input/process_model.php
+++ b/Modules/input/process_model.php
@@ -216,10 +216,27 @@ class Process
         0,
         DataType::UNDEFINED
       );
+      $list[26] = array(
+        _("My test"),
+        ProcessArg::FEEDID,
+        "my_test",
+        1,
+        DataType::REALTIME
+      );
+
+

       return $list;
     }

+    public function my_test($id, $time, $value){
+
+      $this->feed->insert_data($id, $time, $time, $value);
+
+      return $value;
+
+    }
+
     public function input($time, $value, $processList)
     {
         $process_list = $this->get_process_list();

These are the changes I made to process_model.php with respect to the master version

a plus in front means you added a line, a minus means you removed it, the garbage at the beginning is an indicator of where in the file you made the change

If you need to check changes to more than one file, just omit the file name and move to the appropriate folder level to include all the directories in which you made changes:

root@raspberrypi:/var/www/emoncms/Modules# git diff -w
diff --git a/Lib/tablejs/custom-table-fields.js b/Lib/tablejs/custom-table-fields.js
index 3be2f15..5a839fe 100644
--- a/Lib/tablejs/custom-table-fields.js
+++ b/Lib/tablejs/custom-table-fields.js
@@ -113,7 +113,8 @@ var customtablefields = {
                 key = '- inp'; type = 1; break;
               case 23:
                 key = 'kwhkwhd'; type = 2; break;
-            }
+              case 26:
+                key = 'test'; type = 2; break;            }

             switch(type)
             {
diff --git a/Modules/feed/feed_model.php b/Modules/feed/feed_model.php
index 113d48e..f22bd23 100644
--- a/Modules/feed/feed_model.php
+++ b/Modules/feed/feed_model.php
@@ -276,10 +276,11 @@ class Feed
     $this->mysqli->query("UPDATE feeds SET value = '$value', time = '$updatetime' WHERE id='$feedid'");

     //Check feed event if event module is installed
-    // if (is_dir(realpath(dirname(__FILE__)).'/../event/')) {
-    //    require_once(realpath(dirname(__FILE__)).'/../event/event_model.php');
-    //    check_feed_event($feedid,$updatetime,$feedtime,$value);
-    // }
+    if (is_dir(realpath(dirname(__FILE__)).'/../event/')) {
+      require_once(realpath(dirname(__FILE__)).'/../event/event_model.php');
+      $event = new Event($this->mysqli);
+      $event->check_feed_event($feedid,$updatetime,$feedtime,$value);
+    }

     return $value;
   }
@@ -308,10 +309,11 @@ class Feed
     $this->mysqli->query("UPDATE feeds SET value = '$value', time = '$updatetime' WHERE id='$feedid'");

   //Check feed event if event module is installed
-    // if (is_dir(realpath(dirname(__FILE__)).'/../event/')) {
-    //    require_once(realpath(dirname(__FILE__)).'/../event/event_model.php');
-    //    check_feed_event($feedid,$updatetime,$feedtime,$value);
-    // }
+    if (is_dir(realpath(dirname(__FILE__)).'/../event/')) {
+      require_once(realpath(dirname(__FILE__)).'/../event/event_model.php');
+      $event = new Event($this->mysqli);
+      $event->check_feed_event($feedid,$updatetime,$feedtime,$value);
+    }

     return $value;
   }
diff --git a/Modules/input/process_model.php b/Modules/input/process_model.php
index d17f0ac..619b149 100644
--- a/Modules/input/process_model.php
+++ b/Modules/input/process_model.php
@@ -216,10 +216,27 @@ class Process
         0,
         DataType::UNDEFINED
       );
+      $list[26] = array(
+        _("My test"),
+        ProcessArg::FEEDID,
+        "my_test",
+        1,
+        DataType::REALTIME
+      );
+
+

       return $list;
     }

+    public function my_test($id, $time, $value){
+
+      $this->feed->insert_data($id, $time, $time, $value);
+
+      return $value;
+
+    }
+
     public function input($time, $value, $processList)
     {
         $process_list = $this->get_process_list();

... if it looks like it is too complicated .. trust me, once you get used to it it isn't, and it's waay better then deleting everything and restarting from scratch ...

Mattia

I totally agree with Mattia about the use of git.

If you want to develop a feature, a bugfix, whatever, here's a possible workflow:

Start from latest revision

git checkout master

Create a branch and move to that branch

git checkout -b my_branch_for_feature_x

After a development step, commit your changes.

git add modified_file

git commit

While developping, you can check what you modified:

git diff

git difftool

If you did a git add on a file, its changes are "staged", and won't be indicated by a git diff or difftool, you can do an explicit

git diff HEAD

When your changes are committed, you can checkout any other branch

git commit master

then come back

git checkout my_branch_for_feature_x

You are not supposed to change branch with uncommitted changes. But you can do it by "stashing" your work.

git stash

When back to your dev branch, unstash

git stash pop

I'm realizing you already sent pull requests so you probably know a good part of it already.

Thanks for the advice Mattia/Jerome.

I've spent tonight switching between commits for familiarity, and surprised how powerful 'Git' is, but I don't think that I possess the necessary programming skills to yet positively contribute to OEM at the standard expected.

But I appreciate the encouragement and in time, I will learn... (but don't hold your breath!)

Paul

I've now successfully added a new function to the process_model.php module - 'max value' which record the daily highest value of a particular feed. For example, to record the highest temperature that a particular feed reaches during that day, and resets at midnight. (Similar to facility offered in emonGLCD)
I've had it successfully running on a development branch (thanks Mattia/Jerome!) for a couple of days, and it appears to work well, but would be grateful if you could have a quick look at this section of the code, to ensure there are no obvious mistakes which may cause problems for people later.
Next job - to add a 'min value' to record the lowest daily feed value, which is more problematic, as on first run the database entry is 0, so the present value must be lower than 0 to meet the 'IF' condition, otherwise the new value would remain 0.... hmmm......

Thanks
Paul

--------------------------------------><--------------------------------------

public function max_value($feedid, $time_now, $value)
    {
      $new_max = 0;

      // Get last value
      $last = $this->feed->get_timevalue($feedid);
      $last_max = $last['value'];
      $last_time = strtotime($last['time']);

// Check if we have a new max
if ($last_time) {

   if ($value > $last_max)
   {
   $new_max = $value;
   }
   else
   {
   $new_max = $last_max;
   }
  }
      $feedtime = mktime(0, 0, 0, date("m",$time_now), date("d",$time_now), date("Y",$time_now));
      $this->feed->update_data($feedid, $time_now, $feedtime, $new_max);

      return $value;
    }

Hi Paul.

Glad you're going down to it !

Here are my comments. Note that I didn't try your code and I don't have an extensive knowledge of the existing code, so I may be wrong.

- You're writing in the database every time, even when the max is unchanged. Why not do the writing only when it was changed ?

- AFAIU, ['time'] is the time when the sample was recorded. When writing in the DB, you are writing the present time, not the time of the max sample. Anyway, if you write only when max changed (see comment above), then it is correct.

- I don't see the daily reset in your code. (You wouldn't notice that in your tests if the max has been increasing each day.) Can you explain me how it is meant to reset ?

- I think (but I may be totally mistaken) that if($last_max)) ensures that a sample is there already, so it could take care of the first run issue.

It could look like this (absolutely not tested !!) :

--------------------------------------><--------------------------------------

public function max_value($feedid, $time_now, $value)
    {
      // Get last value
      $last = $this->feed->get_timevalue($feedid);
      $last_max = $last['value'];
      $last_time = strtotime($last['time']);

    // Check if we don't have a higher value already

    if (! ($last_time and ($value <= $last_max))) {

      $feedtime = mktime(0, 0, 0, date("m",$time_now), date("d",$time_now), date("Y",$time_now))

      $this->feed->update_data($feedid, $time_now, $feedtime, $value);
   }
  
      return $value;
    }

--------------------------------------><--------------------------------------

This does not address the daily reset issue.

From a quick investigation in process_model.php, I'd be tempted to use something like in the average function :

$result = $this->mysqli->query("SELECT * FROM $feedname WHERE time = '$feedtime'");

if (!$result), then don't define last_time, to force new max being written.

--------------------------------------><--------------------------------------

public function max_value($feedid, $time_now, $value)
    {

      $feedname = "feed_" . trim($feedid) . "";
      $feedtime = mktime(0, 0, 0, date("m",$time_now), date("d",$time_now), date("Y",$time_now));

      // Search for a previous value for today

      $result = $this->mysqli->query("SELECT * FROM $feedname WHERE time = '$feedtime'");
      if (!$result)  {

      // No value, set variables to 0 to force new value as the max

      $last_time =0;

      $last_max = 0;

     } else {

     // There is a value for today

     // Get last value (not optimal, we're fetching the DB again for the same line, we'd better use $result, no time to investigate this right now)
      $last = $this->feed->get_timevalue($feedid);
      $last_max = $last['value'];
      $last_time = strtotime($last['time']);

}

    // Check if we don't have a higher value today already

    if (! ($last_time and ($value <= $last_max))) {

      $feedtime = mktime(0, 0, 0, date("m",$time_now), date("d",$time_now), date("Y",$time_now))

      $this->feed->update_data($feedid, $time_now, $feedtime, $value);
   }
 
      return $value;
    }

--------------------------------------><--------------------------------------

OK, this was my 2 cent feedback. Gotta go. Sorry for the crappy code indentation.

I hope I'm not confusing. Things are a bit blurry to me and it is quite possible that I imagine problems where your implementation was correct already.

Good luck !

Hi Jerome

I've used the format used by the 'power to kwh/d' function, and added;

$list[27] = array(
        _("max value"),
        ProcessArg::FEEDID,
        "max_value",
        1,
        DataType::DAILY
      );

....to the process_model.php file.
By setting it to 'DAILY', only one row is created per day in the database, and the value is updated on each cycle, with a fresh row being created at midnight., and which will contain that 'new' day's max reading. So the database looks like this, with just one row for each 24hrs period.

I thought that this would be OK, as it doesn't significantly add  to the database size, yet provides an historical daily record for later processing. Unfortunately, I don't fully understand how this works under the hood - just copied it by example & desperation!

Paul

Oh right, I missed the "DataType::DAILY" part. I had the feeling there was something I didn't understand. I don't know how this works under the hood either. I shall look into it. Someday...

It is possible that my other points apply. For instance, if value is less than max, you don't have to write anything, so I believe it is useless to do $new_max = $last_max, then write $new_max. Just a hint. I could be wrong.