Selecting data points by logical indexing

Share icon

Hi,

I am trying to use 4 conditional statements to select data based on location. I want the output to contain only those data points that fall between the max and min latitude and the max and min longitude. I'm unsure if this is the BEST way to go about it, but it seems like it could work. 

Currently, the function either generates a wave identical to the input (as if every point met the conditions) or a wave of NaN's (as if none of the points met the conditions), depending on the thresholds used. This suggests to me that the function is operating over the waves as a whole instead of individual index points? Suggestions would be greatly appreciated!

Thank you,

Walt

 mask_data_test.ipf (754 bytes) mask_data_test.pxp (469.38 KB)

Hello,

I don't understand this part

if (lat[index]>= long[index]>=longmasklow && long[index]<=longmaskhigh)

But the whole loop can be written as a wave assignment using p as your index.

function maskdata(data, lat, latmasklow, latmaskhigh, long, longmasklow, longmaskhigh)

wave  data, lat, long
variable  latmaskhigh, latmasklow, longmaskhigh, longmasklow
variable  index
variable maskavg
Make/o/d/n=(dimsize(data,0)) maskeddata=1
maskeddata[] = (lat[p] >= latmasklow && lat[p] <= latmaskhigh) ? data[p] : NaN

//wavestats maskeddata
//maskavg = v_avg

end

The condition that I didn't understand can also be applied to maskeddata in the same way.

October 9, 2018 at 01:56 pm - Permalink

First of all, I think you have a mistake.

if (lat[index]>= long[index]>=longmasklow && long[index]<=longmaskhigh)

should be:

if (long[index]>=longmasklow && long[index]<=longmaskhigh)

 

I'm also confused by the waves in the test experiment you posted. Your testlat and testlong waves both have 28312 rows, but  your testdata wave has only 1365 rows. That's not necessarily wrong, but since both your testlat and testlong waves are very close to constant for the first 1300 rows or so, you're likely to get a maskeddata wave that's either all NaN's or all original data unless your *masklow and *maskhigh variables are very close together. Your test experiment doesn't contain the commands you used to test your function so I don't know what values to use for those.

 

 

I would rewrite your function like this:

// Returns dataValue if lat is within the range of latmasklow
// and latmaskhigh (inclusive) and long is within the range of longmasklow
// and longmaskhigh (inclusive). Otheriwse returns NaN.
ThreadSafe Function LatLongWithinRangeTest(dataValue, lat, latmasklow, latmaskhigh, long, longmasklow, longmaskhigh)
    Variable dataValue, lat, latmasklow, latmaskhigh
    Variable long, longmasklow, longmaskhigh
    if (lat >= latmasklow && lat <= latmaskhigh)
        if (long >= longmasklow && long <= longmaskhigh)
            return dataValue
        endif
    endif
    return NaN  // Not within range
End

function maskdata1(data, lat, latmasklow, latmaskhigh, long, longmasklow, longmaskhigh)


    wave  data, lat, long
    variable  latmaskhigh, latmasklow, longmaskhigh, longmasklow
    variable  index
    variable maskavg

    Make/o/d/n=(dimsize(data,0)) maskeddata
    MultiThread maskeddata = LatLongWithinRangeTest(data[p], lat[p], latmasklow, latmaskhigh, long[p], longmasklow, longmaskhigh)



    //wavestats maskeddata
    //maskavg = v_avg

end

 

I tested my function by executing:

maskdata1(root:'maskdata test folder':testdata, root:'maskdata test folder':testlat, 36.6761, 36.7301, root:'maskdata test folder':testlong, 121.759, 121.779)

This gave results that looked reasonable at a quick glance, but you should test yourself to make sure this is doing what you intend.

My code above allows you to loop through the wave using Igor's wave assignment, instead of your own for loop. This gives a performance improvement and simplifies the code. This also allows you to do the wave assignment using multiple threads. For more information on wave assignment, execute the following command on Igor's command line:

DisplayHelpTopic "Waveform Arithmetic and Assignment"

I'm not sure what you plan to do with the output wave after you create it. If you only need the values within the specified range, you might be better off using the built-in Extract operation. Or, you could continue as you are doing, and call WaveTransform zapNaNs on the output of your function.

October 9, 2018 at 02:17 pm - Permalink

aclight,

I think your point about the different wavelengths may have been one of my issues. I had accidentally pulled different time resolutions in an attempt to create some test data, so it is possible the maskeddata file just contained the first 1300 lat long locations, which were all NaN's. I will look into Igor's built in wave assignment, as that would certainly simplify future codes instead of building an index for loop every time. 

Thank you!

Walt

October 10, 2018 at 09:48 am - Permalink

Forum

Support

Gallery

Igor Pro 9

Learn More

Igor XOP Toolkit

Learn More

Igor NIDAQ Tools MX

Learn More