Blocking Spikes

Hello, 

I had some troubles with negative spikes on odd occasions (even with checks on my sketches and such) and wanted to know if it would be a bad idea to simply change the input_controller.php file and add a check that ignores data below a set value?

I made some adjustments to the .php already (the bold section) and it seems to work fine, below are the changes i made to the if($datain!="") function. 

i don't need values below 200 anyways and i don't feel like having to remove the spikes every now and then. Also would i be possible for me to create a process for the input that only allows values between two points to make changes to logged feeds? i'm kinda new to all this stuff :P 

 

if ($datain!="")
            {
                $json = preg_replace('/[^\w\s-.:,]/','',$datain);
                $datapairs = explode(',', $json);

                $csvi = 0;
                for ($i=0; $i<count($datapairs); $i++)
                {
                    $keyvalue = explode(':', $datapairs[$i]);

                    if (isset($keyvalue[1])) {
                        if ($keyvalue[0]=='') {$valid = false; $error = "Format error, json key missing or invalid character"; }
                        if (!is_numeric($keyvalue[1])) {$valid = false; $error = "Format error, json value is not numeric"; }
                        $data[$keyvalue[0]] = (float) $keyvalue[1];
                    } else {
                        if (!is_numeric($keyvalue[0])) {$valid = false; $error = "Format error: csv value is not numeric"; }
                        $data[$csvi+1] = (float) $keyvalue[0];
                        $csvi ++;
                    }
                }

                $userid = $session['userid'];
                $dbinputs = $input->get_inputs($userid);

                $tmp = array();
                foreach ($data as $name => $value)
                {

                if ($value < -200)
                {
                 $valid = false; 
                
                 $error = "Bad Data Recieved";
                }

                else
{

 

                    if (!isset($dbinputs[$nodeid][$name])) {
                        $inputid = $input->create_input($userid, $nodeid, $name);
                        $dbinputs[$nodeid][$name] = true;
                        $dbinputs[$nodeid][$name] = array('id'=>$inputid);
                        $input->set_timevalue($dbinputs[$nodeid][$name]['id'],$time,$value);
                    } else {
                        $inputid = $dbinputs[$nodeid][$name]['id'];
                        $input->set_timevalue($dbinputs[$nodeid][$name]['id'],$time,$value);

                        if ($dbinputs[$nodeid][$name]['processList']) $tmp[] = array('value'=>$value,'processList'=>$dbinputs[$nodeid][$name]['processList']);
                    }
                }}

                foreach ($tmp as $i) $process->input($time,$i['value'],$i['processList']);
            }
            else
            {
                $valid = false; $error = "Request contains no data via csv, json or data tag";
            }

            if ($valid)
                $result = 'ok';
            else
                $result = "Error: $error\n";
        }

Paul Reed's picture

Re: Blocking Spikes

Most of us use emoncms to record different types of data, power, energy, temperature, wind speed etc, and therefore the global -200 limit would block us recording, say for example solar power export which could be -3000 or more. That's why I suggested in your last posting to impose limits in your emontx/emonth just for the DS18B20 temperature node, which would enable emoncms to record the full range of data - should you need to.

However, if you only ever intend to record temperature readings in emoncms, and it works for you, then that's fine, add the filter in emoncms.

​Paul

Mellowed's picture

Re: Blocking Spikes

Thanks Paul, 

Yeah i'm only monitoring temperatures at the moment so i figured it would be a quick fix as it is only one feed that keeps giving me the issues. In the long run i should probably figure out why it gives the spikes. 

Would it be possible to make a process i could add to the input which blocks value outside given limits and therefore not update a logged feed, similar to the "allow positive/negative"? I don't want to start reading all the code etc only to find out its not possible ^_^

Regards

Paul Reed's picture

Re: Blocking Spikes

I'm not aware of such a process already written, but you could give it a try if you wish.
I've written 2 processes, and found the emoncms architecture quite difficult to follow, as you have to add code to (if I recall) 3 different modules.
It's far easier to add a conditional statement to the emontx/th.

Paul

chaveiro's picture

Re: Blocking Spikes

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.