Interpolate in different waves

I recommend that you start experimenting with Analysis->Interpolate using linear interpolation.

For background information, execute:

If you need further help, I recommend that you attach a sample set of 3 waves here.

Hi Hrodstein,

Thank you for the reply. I tried the Interpolation. But it changed the rest of the data when interpolating. 

I attached 3 waves here, 1 time wave and 2 data wave.

I would like to know how can I interpolate the missing data with the neighbouring value. 

Cheers,

Mortoo

Hi Hrodstein,

Thank you for the reply. I tried the Interpolation. But it changed the rest of the data when interpolating. 

I attached 3 waves here, 1 time wave and 2 data wave.

I would like to know how can I interpolate the missing data with the neighbouring value. 

Cheers,

Mortoo

I examined your time wave. It is in fractional seconds. To see that, display it in a table and then choose Table->Format->Show Fractional Seconds.

I then created a table containing the time wave and the difference in seconds from each point in the time wave to the next point in the time wave. Looking at this in a table, I see something like:

Notice that the time differences (last column) are of unequal magnitude.

I then executed this to find the average time difference:

This printed 0.208185  0  2.19483

I then did a histogram of timeDiffs:

So your times are roughly 5Hz on average but not consistently close to 5Hz.

You wrote:

I now want to interpolate them all into full strict 5Hz data

I did that with this command:

This command is from a function that I wrote called Create5HzData. It is in the attached experiment. It uses linear interpolation (/T=1) with "X From Dest" (/I=3). "X From Dest" allows precise control over the X locations at which the input data is sampled. The output is a waveform with the X data represented in the wave's X scaling.

Here are graphs, also from the attached experiment, showing the original data (data0 vs timeW - I renamed your waves for simplicity):

You can see that the output data, which is sampled every 0.2 seconds, clearly differs from the input data (note the bottom around 12:00 PM). This is because your data is very noisy. Therefore sampling every 0.2 seconds will miss the positive and negative peaks. Consequently the interpolated data is not a very good representation of your input data.

Although this does achieve your goal to "interpolate them all into full strict 5Hz data", the result is likely not what you are looking for. I think this problem will affect any kind of straightforward sampling.

Because of the nature of your data, I can't think of a practical way to achieve the goal. How to proceed depends on what you intend to do with the data.

Hi Hrodstein,

Thank you for the help. Sorry for the misunderstanding. These 3 waves of data are collected directly from the instrument. So unequal magnitude is Okay for me. The "strict" I mean is just 5 data in 1 second. My goal is to interpolate the missing points by the neighbouring points. To make sure they are exactly 5 data per second, without changing other data. 

Cheers,

Mortoo

To make sure they are exactly 5 data per second, without changing other data.

I don't think there is a simple solution for this. I think it would require custom programming and even then I'm not sure it is feasible.

As a start to understanding the difficulty, think about the fact that each 1 second interval in your time wave is potentially different from the others in unpredictable ways. Try taking the first 10 seconds and manually deciding where you would insert a point in each 1 second interval.

Hi Hrodstein,

How about creating a "strict" 5Hz time wave like

2021/12/18 00:00:00.00

2021/12/18 00:00:00:02

2021/12/18 00:00:00.04

......

and using a command to filter out data from the raw data within 0.2 seconds of the "strict" data to make a new wave. Unmatched data is displayed as NaN. 

Is there a command for this filtering function in Igor pro? Can "BinarySearch" command works here?

Cheers,

Mortoo

How about creating a "strict" 5Hz time wave like

This is what I did with interpolation. It would work with smooth data but not with spikey data, especially with very spikey like you have.

If you can come up with a fairly detailed algorithm to choose the data values for those times you have identified (every 0.2 seconds), then it could be programmed. I can not think of an algorithm that preserves the meaning of the data.

using a command to filter out data from the raw data within 0.2 seconds of the "strict" data

I don't understand that.

Is there a command for this filtering function in Igor pro?

I'm not sure what you mean by "filtering function". Some possibilities are Smooth, Extract, FilterFIR, and FilterIIR, but I don't see how any of them helps.

zixuan.cheng wrote:

and using a command to filter out data from the raw data within 0.2 seconds of the "strict" data to make a new wave. Unmatched data is displayed as NaN.

assuming monotonic time waves, this function selects the nearest point within plus-or-minus 0.1 seconds

Hi Hrodstein,

Sorry for the misunderstanding. I mean to create a new datawave with the same length as the "strict" 5Hz datetimewave. The data points of the new wave are selected from the most recent data from the old datawave.

Cheers,

Mortoo

Hi Tony,

Thanks for helping me again! I tried this function and it makes a wave with some gaps, but enough data points, which is exactly what I need. Thank you so much!

Another question is how to fill these gaps with neighbouring data. 

Cheers,

Mortoo

Hi Tony,

I replaced the "newDataWave[i] = NaN" by "newDataWave[i] = oldDataWave[j]" to fill the NaNs:

Thank you so much.

Mortoo

For simply the closest point in time, I think this works, and fixes an error in the previous version: