How to set values greater than Threshold to NaN
Hi all,
I have some I-V data recorded from biological nanopores. The data is abf format, imported with Neuromatic and therefore, a single one-dimensional wave of current values. The I-V characteristics of nanopores are linear, but, there are some unspecific transient gating events in the dataset,which prevent me from averaging multiple I-V curves correctly. In order to get rid of these gating spikes, I though deriving the wave and replacing all values greater than a threshold by NaN before reintegrating the wave should do the trick. Doing it manually is very time-intense as I have multiple hundreds of datasets, however, I could not figure a way out of how to compare the values in the wave with the threshold automatically and set them to NaN if they exceed. Is there any elegant way, or an already implemented function to do so I am currently overlooking?
Many thanks in advance!
Cheers,
Tobi
mywave = mywave[p] > thresh ? NaN : mywave[p]
March 18, 2022 at 10:04 am - Permalink
•extract/INDX/O junk, indx, junk>1
•junk[indx] = nan
Might be faster, I don't know. At least the Extract operation runs internal C++ code, and using the index wave means iterating over fewer wave elements. If the waves are large and the over-threshold values sparse, this could be the way to go.
March 18, 2022 at 10:15 am - Permalink
I'd use John's first response, possibly with the Threadsafe keyword if you really need the performance. Either way I don't think this is going to be a bottleneck in your analysis.
I will caution you about introducing NaN values into data. Make sure that you pay attention to how the various Igor commands you use handle NaNs. Some of them will ignore NaNs and some will not allow them. For some operations (e.g. Smooth), NaNs in the data might give undefined results. Also make sure that you use numtype() if you want to test a value for NaN. You can't just do something like if (x == NaN)... since that sort of comparison will always be false.
March 19, 2022 at 08:04 am - Permalink
Assuming that the previous post did not convince you out of using NaNs, I'd recommend using MatrixOP.
MatrixOP/O modifiedWave=setNaNs(inputWave,greater(inputWave,threshold))
The advantage here is that it would create the appropriate output wave (something that can hold a NaN) even if your inputWave is of integer type. It is also faster than executing the command with the '?' operator.
AG
March 19, 2022 at 02:51 pm - Permalink
Note that MatrixOp is very efficient, but does not transfer any scales to the output wave. You might want to use 'CopyScales inputWave, modifiedWave' afterwards to transfer the scales.
March 20, 2022 at 12:28 am - Permalink