Filtering spikes

I occasionally get spikes from my dallas 18b20 temperature sensor on the emonTH node, this can be plus or minus 50 degrees or more, is there a code to filter out single spikes on the node or in the emon input module?

Halldor

ngbod's picture

Re: Filtering spikes

If this node is battery powered the spikes could be a sign the voltage is getting low. 

It happens on my Funky sensor nodes when batteries are low.

Bra1n's picture

Re: Filtering spikes

I use the code below for noisy analog readings (I can't remember where I found it now so can't give credit to the author). In my case I'm taking 100 readings and discarding the top and bottom 45 and returning the average of the middle 10. This may be OTT for your purposes but you could easily adapt the code to take fewer readings etc.

// read multiple values and sort them to take the mode
  int sortedValues[NUM_READS];
  for(int i=0;i<NUM_READS;i++){
    int value = analogRead(sensorpin);  // Change this for reading the DS18B20
    int j;
    if(value<sortedValues[0] || i==0){
      j=0; //insert at first position
    }
    else{
      for(j=1;j<i;j++){
        if(sortedValues[j-1]<=value && sortedValues[j]>=value){
          // j is insert position
          break;
        }
      }
    }
    for(int k=i;k>j;k--){
      // move all values higher than current reading up one position
      sortedValues[k]=sortedValues[k-1];
    }
    sortedValues[j]=value; //insert current reading
  }
  //return scaled mode of 10 values
  float returnval = 0;
  for(int i=NUM_READS/2-5;i<(NUM_READS/2+5);i++){
    returnval +=sortedValues[i];
  }
  returnval = returnval/10;

fluppie007's picture

Re: Filtering spikes

sensors.requestTemperatures(); // Send the command to get temperatures

T1 = (int) (sensors.getTempC(address_T1) * 100);
if (T1 == -12700) {
T1 = prevT1;
} else {
prevT1 = T1;
}

T2 = (int) (sensors.getTempC(address_T2) * 100);
if (T2 == -12700) {
T2 = prevT2;
} else {
prevT2 = T2;
}

T3 = (int) (sensors.getTempC(address_T3) * 100);
if (T3 == -12700) {
T3 = prevT3;
} else {
prevT3 = T3;
}

dBC's picture

Re: Filtering spikes

Apologies in advance to the OP for dragging this so far off topic, but Bra1n if you're going to that much effort you might also want to consider AVR121 (http://www.atmel.com/Images/doc8003.pdf).

If you're measuring very slow moving analog signals, noise is actually your friend.  You can use it to extract much finer detail about the current value.  In my case I have a 4-20mA current loop probe in my rainwater tank.  With the AVR's 10-bit A/D, you'd expect a resolution of about 20uA, but by using the oversampling techniques described in that application note, I've effectively turned the 10-bit A/D into a 14-bit A/D giving a current loop resolution of close to 1uA.  The results are spectacular.  If I fill a 10L bucket to wash the car, it shows up on the dials:

http://emoncms.org/dBCC/ermtankhis

[Duplicate posts deleted - RW]

boelle's picture

Re: Filtering spikes

The OP said signals from dallas sensors, not to be an smartass but as i remember they are digital and not analog

 

So it might well be he is scared off or simply confused when analog is put in the mix

Bra1n's picture

Re: Filtering spikes

The code is still applicable just the reading method need to be changed to suit the sensor which is why I added a comment at the appropriate place in the code ("// Change this for reading the DS18B20").

Hax asked for code to filter out the spikes, which my example does and he must already have the code for reading the sensor, a simple substitution I would have thought.

dBC's picture

Re: Filtering spikes

Bra1n, if you're going to that much effort with your analog signal, you might also want to check out AVR121.  It'll show you how to use that noise to turn your 10-bit A/D into a 14-bit A/D.

Hax's picture

Re: Filtering spikes

Thanks Bra1n, this is a good solution to the problem. I will place this in the input processing on my emoncms server. There i am also building the event handling to be more flexible like monitoring the battery voltage and generating complex events.

 

 

Bra1n's picture

Re: Filtering spikes

I had a quick look at AVR121 but for my purposes the existing resolution is more than enough. The issue is with interference (RFI ? especially on longer cable runs) resulting in a few ridiculous readings which would greatly skew the mean if they weren't discarded first before calculating the average. Of course if I'd paid more for digital sensors I probably wouldn't have the issue but I'm happy with the cheaper software solution.

dBC's picture

Re: Filtering spikes

Even Airbus struggle with spike filtering.  In this particular case, their filtering algorithm could be fooled if the spikes happened to occur precisely 1.2 seconds apart:

http://www.atsb.gov.au/publications/investigation_reports/2008/aair/ao-2008-070.aspx

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.