n00b - Averages, skip values, etc.

Share icon

So I am new to using IGOR Pro. I have gone through the guided tours and read a lot of the other stuff, but I don't really know much about programming.

I am working at an atmospheric lab where I need to analyze data from a CPC and a CCN counter. These instruments give me huge files of data that I need to mess with. So each second it takes a measurement and gives me a count, at a certain setting. I need to be able to average the counts for just those time intervals where they are on a certain setting and skip the ones that aren't that setting.

I attached a simplified version of the data.


So how would I write a macro or procedure to be able to only average, or graph, the counts over a certain time interval for a certain setting?

Any help is much appreciated, I have been racking my brains for the past 7 hours to try and figure out how to write these commands etc, I am having no luck whatsoever CCNDATA.txt (1.49 MB)
Your problem can be solved in more than one way. Some hints to how I might work this are ...

1) Any value in a wave that is set as NaN will NOT show in a graph
2) Any value that is set as NaN will NOT be included in WaveStats calculations
3) The following construction (found in the Programming part of the Command Help menu) will set values in wave ywave below and at a Cutoff to NaN

      ywave = ywave[p] > Cutoff ? ywave[p] : NaN


Assume your data wave is called myDataOriginal, you want to cut off below 35, and you want to do this in the range from point 10 to point 350 in your data file. Here then is a function ...

Function CutMyData(mywave,CutOff)
   wave mywave
   variable Cutoff

   mywave = mywave[p] > Cutoff ? mywave[p] : NaN
   WaveStats/Q mywave
   return V_avg
end


... with which to exercise the following on the command line:

duplicate/O/R=[10,350] myDataOriginal, myDataCopy
display myDataCopy
print CutMyData(myDataCopy,35)


HTH

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH

May 13, 2008 at 11:20 am - Permalink

I really appreciate your help. This really did help me out. I was having troubles making up or down with any of this but the way you set it up makes sense.


So I set my cutoff for my SS setting wave to < 0.3. That is how I want to specify which points are good, now I want it to graph the values that correspond to those points in my Count wave (another wave). How is that done?

And how can I make it so I can cut out everything not between 0.3 and 0.5? Can I just type

Function CutMyData(mywave,CutOff,CutOff2)
wave mywave
variable Cutoff
variable Cutoff2

mywave = mywave[p] > Cutoff ? mywave[p] : NaN
mywave = mywave[p] < Cutoff2 ? mywave[p] : NaN
WaveStats/Q mywave
return V_avg
end


Thanks so much for your help!

May 13, 2008 at 12:30 pm - Permalink

I am not sure I fully understand what you need to do but if your different settings are always grouped together (as they are in your example) then the main task will be to find the starting and ending index of a desisred group. Once you have that you can create graphs using subranges and can calculate the average using subranges.

For example, if you have the three columns of data from your sample in waves named wt, ss and ccn, you can see by inspection that the group defined by ss == 0.2 ranges from point 0 to 8. Thus you can make a graph via:

Display ccn[0,8] vs wt[0,8]


and you can calculate the average via:

print mean(ccn,0,8)


On the other hand, if your measurments are scattered in the file, you will need to extract subsets. Here is one way:

Extract ccn, tempccn, ss == 0.2
Extract wt, tempwt, ss == 0.2

Display tempccn vs tempwt
print mean(tempccn)


Note that is generally not a good idea to test for equality between floating point numbers but from the looks of your ss numbers, it may be ok in this case.

Later: I see you are now talking about a range for ss, for example between 0.3 and 0.5 (although you have no data in this range,) so the extract lines would be:

Extract ccn, tempccn, ss > 0.3 && ss < 0.5
Extract wt, tempwt, ss > 0.3 && ss < 0.5

May 13, 2008 at 02:12 pm - Permalink

excellent! that extract method is just what I was looking for.

I changed the attached file of my original post to the actual data I am working with too make it easier to understand what I am talking about


So if you Load this file as Delimited Text and don't change any of the names, and then use the code you gave me (altered a bit) of

Extract CCN_Number_Conc, tempccn, Current_SS == 0.2
Extract TimeW, tempwt, Current_SS == 0.2
 
Display tempccn
print mean(tempccn)


you get a graph that shows a large jump from one set to the next, and then a cool down moment before it settles back to appropriate values. Other than manually selecting the intervals to remove, is there a way I can have it ignore the first 120 seconds (or points) of each Current_SS == 0.2 set that it finds?

May 13, 2008 at 03:45 pm - Permalink

astrotool wrote:
you get a graph that shows a large jump from one set to the next, and then a cool down moment before it settles back to appropriate values. Other than manually selecting the intervals to remove, is there a way I can have it ignore the first 120 seconds (or points) of each Current_SS == 0.2 set that it finds?


I didn't see anything special about the first 120 points. By also extracting the TimeW and plotting the extracted ccn vs extracted time, I see there are two groups but don't see what you described above. For reference, here are the commands I used on your data file:

LoadWave/J/D/W/E=1/K=0/V={","," $",0,0}/L={4,6,0,0,0} "hd:working:CCNDATA.txt"
Display tempccn

Extract TimeW, temptw, Current_SS == 0.2
Display tempccn vs temptw


If you want to find the start of each group, one method might be to differentiate the extracted TimeW and then search (FindLevels) for jumps.

May 14, 2008 at 07:25 am - Permalink