Sorry for not clearly explaining my questions. Here in the attachment, you can find the data and procedure. If you look at plot of tvalues, you'll see some sparks (not wanted) which only on the one side of the baseline, those appears as pairs (wanted) on both sides are what need to be located. My purpose is to get rid of those unwanted sparks without effect on those wanted ones. Median filter won't work in this case. Any suggestions are appreciated.Ext_Total_filtered.ibw(6.87 MB)TwoStateFittingTTestMar24 tvalues.ipf(8.81 KB)
If I understand correctly, your goal is to scan through the data with two adjacent moving windows, identify the t-value for comparison of the data in each window, and when a step in the data occurs, you will see an extreme t-value. I get the impression from your post that you expect to see extremes above and below the baseline? Why should this occur? I would think that as the window boundary crosses over a step, you would see the t-value go from near zero, to an extremely positive or negative value (depending on the direction of the step) and then fall back to near zero. What am I missing?
It is also not clear from your code what temp1_s or temp_s is supposed to represent. Are they the signals in which you are trying to find the step? What are you trying to remove the "sparks" from? The signal or the wave of t-values?
Thanks a lot for taking your time to go through the codes. I am glad you can understand. I am not a good programmer since I am just a beginner.
RGerkin wrote:
If I understand correctly, your goal is to scan through the data with two adjacent moving windows, identify the t-value for comparison of the data in each window, and when a step in the data occurs, you will see an extreme t-value. I get the impression from your post that you expect to see extremes above and below the baseline? Why should this occur? I would think that as the window boundary crosses over a step, you would see the t-value go from near zero, to an extremely positive or negative value (depending on the direction of the step) and then fall back to near zero. What am I missing? ........
It is also not clear from your code what temp1_s or temp_s is supposed to represent. Are they the signals in which you are trying to find the step? What are you trying to remove the "sparks" from? The signal or the wave of t-values?
Your understanding is correct and nothing is missing.
My problem is that by using threshold, I couldn't locate each step correctly because some "sparks" are comparable to those fairly but not extremely positive or negative values (t-values). If you look at the table of Step locations and step sizes, you'll see that some results with too small stepsizes are apparently not wanted. I was also playing with threshold but couldn't solve this problem. Maybe I should be more carefully to choose threshold. I'll try this again.
Temp1_S is just median filtered temp1 which was duplicated from inwave, because I don't want to mess up the original expt data.
Temp_S is the same as temp1_S but is the return of function medianSmooth. Temp1_S is just duplicated from temp_S.
Temp1_S is the signals in which I am trying to find more information of the steps by fitting into heaviside function. t-values is the signals used for robust determination of step location which will be used for later heaviside function fitting. And finally all the steps will be ranked. (not done yet)
In order to determine whether median filtering (or any kind of filtering) can be successful in eliminating the sparks before you do your sliding t-test, I would like to see the original data. Can you post the raw version of ext_total_filtered.ibw?
is this what you wanted?
I hope it's okay that I included some of your code in my snippet ... I included your name in the doc.
Lovely data by the way. If you paste the snippet in your procedure window and then execute AnalyzeTLS(Ext_Total, 2, 4, 2), you will get three waves EventStart, LifeTime, EventLevel as described in the documentation.
It is also not clear from your code what temp1_s or temp_s is supposed to represent. Are they the signals in which you are trying to find the step? What are you trying to remove the "sparks" from? The signal or the wave of t-values?
March 24, 2010 at 01:33 pm - Permalink
Thanks a lot for taking your time to go through the codes. I am glad you can understand. I am not a good programmer since I am just a beginner.
Your understanding is correct and nothing is missing.
My problem is that by using threshold, I couldn't locate each step correctly because some "sparks" are comparable to those fairly but not extremely positive or negative values (t-values). If you look at the table of Step locations and step sizes, you'll see that some results with too small stepsizes are apparently not wanted. I was also playing with threshold but couldn't solve this problem. Maybe I should be more carefully to choose threshold. I'll try this again.
Temp1_S is just median filtered temp1 which was duplicated from inwave, because I don't want to mess up the original expt data.
Temp_S is the same as temp1_S but is the return of function medianSmooth. Temp1_S is just duplicated from temp_S.
Temp1_S is the signals in which I am trying to find more information of the steps by fitting into heaviside function. t-values is the signals used for robust determination of step location which will be used for later heaviside function fitting. And finally all the steps will be ranked. (not done yet)
Thanks again for your help.
Xia
March 25, 2010 at 08:07 am - Permalink
March 25, 2010 at 08:11 am - Permalink
March 25, 2010 at 11:54 am - Permalink
is this what you wanted?
I hope it's okay that I included some of your code in my snippet ... I included your name in the doc.
Lovely data by the way. If you paste the snippet in your procedure window and then execute
AnalyzeTLS(Ext_Total, 2, 4, 2), you will get three wavesEventStart, LifeTime, EventLevelas described in the documentation.Happy experimenting,
Wolfgang Harneit
March 25, 2010 at 06:22 pm - Permalink