Home > electronics > Filtering a Sensor with Noise and Spikes

Filtering a Sensor with Noise and Spikes

Recently I have been working on a robot that required accurate measurement of distance (more on that robot on a later post). I noticed some unpredictable behavior of the robot and so I investigated further to find out what was wrong with it. It turns out that the Sharp GP2D120 sensors I used are pretty noisy! I didn’t do much research about it before adding it on my project. When I needed to know the distance value, I took 10 consecutive readings and averaged them. But that was not such a clever choice for filtering and you will understand why, in a while.

Let me first explain you my testing platform. First of all we have the robot that has the GP2D120 sensor on. The PIC micro (used a PIC18F4520) on the robot is able to transmit wirelessly using an XBEE module. Data is received on my PC using another XBEE module through USB. Finally, data is analysed in MATLAB. So I position my robot to be stable, so that the distance sensor should read the same value constantly. Each value read, is transmitted and captured on my PC.

Experiment 1 – No filtering

No filtering

Notice that the y axis is the ADC value and not the voltage value directly. As you can see there are some pretty nasty spikes over there! They occur quite ofter as well. Converting those values to distance, spikes reach up to 3cm ! So you have a wrong measurement by 3cm. For a project that requires accuracy that is a nightmare!

As I told you above, I used an average value to get my reading i.e. took 10 consecutive measurements and divide them by 10. Now you can understand why this kind of filtering is quite wrong. When we are averaging some values, it means that we put equal ‘trust’ that those values are correct. That is wrong, because the spikes shouldn’t be considered at all. They should be removed.

So I searched around for a quick fix and most roads lead to the median filter.

Experiment 2 – Median Filter

What is median filter? If you want head over to wikipedia to read the formal definition. I will try to explain it as I understood it. So lets take an example measurement set (this is out of my head) [4 , 6 , 50 , 7 , 6]. Let’s assume that we are trying to measure something that is 6 and I added some noise. If you take a look at those sample you will clearly see that we have 4 measurements around 6 and a spike at 50. That spike is clearly wrong, and should not be taken into consideration. Jumping a couple of paragraphs back (where I said that the averaging filter is not suitable for spikes) taking the average of those values would give 14.6. That is pretty inaccurate compared to the correct value that is 6, right?

Now let’s apply a simple median filter. To do that, firstly we need to sort the sample set, i.e. [4 , 6 , 6 , 7 , 50]. Then take the value in the middle of the set, which is 6. And that is the value we will use. And that’s it 🙂

Intuitively, the filter assumes that we have at least some correct measurements close to the real value. After sorting, the extreme positive and negative values around those measurements are moved to the right and left positions of the set, leaving the ‘good’ values in the middle.

Of course there are some limitations. For example, imagine having 3 positive spikes [4 , 6 , 45 , 47 , 50]. In this case, median would show 45 which is completely wrong (but having 3 spikes in 5 measurements!!!). On the other hand it would be able to get the right value even if we had more spikes, as long as they were separated. Imagine 2 negative and 2 positive spikes [-40 ,1 , 6 , 47 , 50]. Median will work fine as long as the concentrated spikes are not more than the 50% of the samples. Also note that you could always increase the ‘window’ of the filter, so for example take 10 values to be more accurate.

I implemented the median filter with a simple bubble sorting of 10 measurements. I am not going to include this here, but tell me so if you need help.

So let’s see what happens to our Sharp GP2D120 sensor data.

Median Filtering

Nice! It worked 🙂 It may look pretty noisy but what is important is that the spikes were removed. Without the spikes we can apply a smoothing filter that will smooth this random noise out. From my robot placement, I know that the value should be around 191. Lets see the distribution of the error

Noise Distribution

That looks like Gaussian… I believe we can smooth that out easily.

To do that I used the exponential filter.

Experiment 3 – Adding the Exponential Filter

So now we got rid of the spikes we will smooth our signal out using the exponential filter. I like this filter because it is really light on computation and it is iterative, no needing much storage memory.

y_n = \alpha x_n + (1-\alpha)y_{n-1}

where x_n is the new measurement from the sensor and y_n is the filtered value. In code this is implemented as follows:

filtered_value = 0.4*new_value + 0.6*filtered_value;

The \alpha shows how much to ‘trust’ the new value compared to all the previous values. I am using the value of 0.3 but you could play around with that value and see how it behaves on your sensor. As you can understand, an initial value is required to start with. I am using 0, so it takes few time frames to reach a value close to what I am measuring. If you critically need correct filter values from the very first milliseconds, you should initialize the filtered_value to a value close to the measurements.

Let’s see how it improves the performance of our filter:

Exponential Filter

Much better! Much lower noise. You may noticed some slight change in the values (we are now around the 178 point) that is because I accidentaly moved the robot a bit. What is important here is that the noise is less. Remember in the first experiment we had wrong measurements as big as 3cm, now the biggest error is not more that 0.4cm. That is a huge improvement. Lets see the error histogram for the distribution

Noise Distribution

Much closer to zero 🙂

So there you have it. We have seen how to filter and get useful measurements of noisy sensors using our PIC. And all the with few lines of code.

Let me know of your comments

Related Posts:

Incoming search terms:

  • median filter (22)
  • spike filter (22)
  • sensor noise filter (20)
  • how to filter spikes sensor (16)
  • median filter adc (13)
  • c code simple spike filter (10)
Categories: electronics Tags: , , ,
  1. Shanky
    May 12th, 2011 at 09:29 | #1

    Excellent tutorial ….thanks a lot for such a clear explanation

  2. psychoul
    May 14th, 2011 at 16:01 | #2

    @Shanky Thanks for the feedback!

  3. int
    July 9th, 2014 at 12:00 | #3

    very nice tutorial! and the example is very clear. But i have a question please,

    i like to implement a median filter in Matlab, to remove spikes from my data’s acceleration. I have a raw digitized signal . do you think that i should using a medfilt1 function to do this?

    Best regards,

  1. No trackbacks yet.