data categorization
mwpro
I have time series of measurements (9 columns) which include wind directions. I want to categorize the measurements by wind direction, [300,360), [0, 165), [165, 180), [180, 210), [210, 220), [220, 255), [255, 300).
What I used to do is to make a new wave for each wave that meets the wind direction criteria under each category and redimension them in loop, but considering there are quite a few bins and number of concurrent measurements, this approach seems to be cumbersome. Is there an efficient way to do it?
Thank you! Your help is very very much appreciated!
June 1, 2015 at 10:42 pm - Permalink
Thank you chozo! I attached my dataset and the procedure I put together. My biggest concern is this function I have to execute multiple times to be able to process all the waves like temperature, ppfd, swrad, etc, and have to rename the waves every time they are output. If there are more categories, this becomes very inconvenient, and I have a similar task that is to categorize all the data by day and night, so I was wondering about a solution to this type of task in general.
Does anyone know a snippet that would achieve this? Thank you!
June 2, 2015 at 03:16 pm - Permalink
CatWind(thewinddirectionwave)
. I have used the sort and split approach, so the data ends up in different folders for different bins, keeping the original names intact. This all may not be the most effective approach, but it works without any manual labor. In fact, there is a lot of redundant code in there, which could probably be streamlined, but I also want to keep stuff readable.Some notes:
- there is no backup functionality and the original data will be altered. Make a copy if you want to keep the data intact (could also be added in the code if needed)
- this function will act on ALL waves in the current folder, so keep stuff which you don't want to be processed separate
- after the processing the data is still sorted according to wind direction. It is possible to revert to time sorting in an additional step, if you need that (not necessary for, e.g., plotting)
- if you want to add extra bins, just edit the third line with the numbers {165,180,210,220,255,300} (note that the last bin is always 'rest', and you cannot add bins which are greater than the highest value)
I also add the code here for everyone to look at:
Wave windwave
Make/FREE binlist = {165,180,210,220,255,300} // the list of bins, i.e., 0 to 165, 165 to 180 etc. ... data above the last item will be stored in 'the rest'
// +++ first step: sort the waves after wind direction +++
String Datalist = wavelist("!"+nameofwave(windwave),";","")+nameofwave(windwave) // make a list of all waves, make sure the wind data comes last!
Variable i, pointnum
String currentname
for (i = 0; i < itemsinlist(Datalist); i +=1) // for every data wave sort after wind direction (including wind direction itself)
currentname = Stringfromlist(i,Datalist) // extract the name of the current wave from the list
Wave current = $currentname // create a wave reference to work with
Sort windwave, current // sort
endfor
// +++ second step: get rid of points with no valid wind data (i.e., nans) +++
FindValue /V=(Wavemax(windwave)) windwave // find the last point with valid data
pointnum = V_Value
for (i = 0; i < itemsinlist(Datalist); i +=1)
currentname = Stringfromlist(i,Datalist)
Wave current = $currentname
DeletePoints pointnum+1, inf, current // delete points after valid data
endfor
// +++ third step: store the data in separate folders depending on binning criteria +++
Variable currbin
for (currbin = 0; currbin < numpnts(binlist); currbin +=1)
FindLevel/Q windwave, binlist[currbin] // find the crossing point to the next bin in the data
pointnum = ceil(V_LevelX)
NewDataFolder/O $("binned_"+num2istr(binlist[currbin])) // create a data folder
DFREF folder =root:$("binned_"+num2istr(binlist[currbin])) // refer the data folder to work with
for (i = 0; i < itemsinlist(Datalist); i +=1)
currentname = Stringfromlist(i,Datalist)
Duplicate/O $currentname, folder:$(currentname) // duplicate into data folder
Wave current = $(currentname)
Wave copy = folder:$(currentname)
DeletePoints 0, pointnum, current // delete points of current bin in main waves
DeletePoints pointnum, inf, copy // delete points OUTSIDE current bin in copied wave
endfor
endfor
NewDataFolder/O $("binned_rest") // the data above the last bin will be stored in 'rest'
DFREF folder =root:$("binned_rest")
for (i = 0; i < itemsinlist(Datalist); i +=1)
currentname = Stringfromlist(i,Datalist)
Duplicate/O $currentname, folder:$(currentname)
Wave current = $(currentname)
KillWaves/Z current // kill the main waves if possible (keeps stuff tidy)
endfor
End
June 2, 2015 at 08:39 pm - Permalink
June 4, 2015 at 01:50 pm - Permalink