Suggestions for Multipeakfit2
chozo
Will this package be updated with the new Igor 6.2? I'm not using the new beta yet, so I don't know if there are already improvements over 6.12.
I usually write procedures myself to cover tasks of my field and also try to write additional functions to simplify some Igor tasks, but Multipeakfit is huge and still hard to read for me.
At first there are some "bugs" I want to address:
if the scaling delta of a wave is negative, after adding the first peaks the function generates an error (Make error: number of points must be between 0 and 2147 million) when attempting to generate and display the fit wave (the sum wave in blue). I think somehow the calculation of the needed wavepoints will be infinite for negative deltas, but I haven't had the time to look deeper into the code to find it.
a cubic spline background lose its values when resuming the fit
Now some annoyances I have constantly:
Maybe I should first tell a bit about the tasks I do with this package to clarify some things. I have a spectrum with, say, 10 peaks in it from which I know some of the widths and some of the positions (actually different things too, but more on this later). I do the fit until I have something I like and then extract and display the results together with a graph of the set for discussion. Then I want to fit another spectrum where some features are a little different (that’s the information I want to get) with the same number of peaks in similar positions and with the same widths. The following things would speed up things greatly (maybe they are already possible, but I haven't found out yet?):
the values which parameters are on hold should be saved (maybe in a separate wave for easy readout) and set on resume
For now, I have to go through all peaks after resume and reset the various hold checkboxes (if I still remember them all correctly and don't forget one). If I want to extract the information which parameters are on hold (maybe to show it to someone or just have an overview) I have to get this indirect. Currently I have a procedure to generate a table with the information I need (see next point why) and label the fields with error info +/-0 as hold.
results should be displayed differently (in a table at best with selectable digit count for display)
The current result output is suboptimal for my purposes. The notebook output option is complete but bloated and useless for reference (I have to scroll constantly) as well as a overview because of this. The tab-delimited view is okay, but the fit parameters are missing (I need them for the fit of other spectra… see later why). Moreover, it’s impossible to display these outputs in a layout to get a quick discussion basis (graph + fit data) out of the printer. For now I let my procedure do the work of generating a table out of the available data and the cut and compress the table to fit on A4.
have more than one revert or have a checkpoint button
When I’m fitting the data I often try different things to get it right, for example adding a peak or varying some of the parameters. Sometimes the fit will fail then or some peaks will be unreasonable large or negative. This will often destroy my peak set, even if I had an almost good solution at some point. Now the ‘revert to guesses’ button only reverts one step which is seldom enough recover the fit. It would really help to have more ‘undo’ to play around. I understand, that this may be difficult to implement (where to save all the info), but then maybe a checkpoint button helps? Even if it’s only one checkpoint, I can then decide what is a good point to revert to and what not.
possibility to constrain to only negative or only positive peaks
Ok, easy to demand and maybe complicate to implement, but I never want to have negative peaks, because for my fits they are simply wrong. Sometimes the fit procedure seeks an easy way out by producing negative and positive peaks adding up to get the work done. Then I often have a hard time rearranging the peaks to prevent this from happening. How easy would it be with a simple checkbox that disallow negative (or positive for other purposes) peaks, even if the fit won’t be as good.
possibility to load peaks from one set into another
This is really annoying for me, because I want to fit similar spectra. I want to have the same peaks for a different set to begin the fit and work out the differences, but how do I get them in? I have to manually add the peaks and copy/paste or dial in all the values. Lucky me, that I don’t do fits with 50 peaks. To achieve this I even can’t use the standard result output formats (notepad -> bloated, tab-delimited -> data missing), so I copy the parameters form my generated table.
Ok, that was long, but I’m not finished yet. I have even some cool features to suggest to you:
have area, fwhm etc. as parameters
It would be nice to keep the area constrained for example. Ok, fwhm aside (I just recalculate to the width every time by hand, but that’s ok), it would expand the set of assumptions I can make for a peak, if I could specify also other parameters a peak can have.
link parameters of different peaks together
Actually, what I really know about some (or even most of) the peaks are such things as distance to a reference peak or height ration of two peaks or width constrains. For this, I need to link parameters of different peaks such as position of p1 is the position of p2 minus c1. While it is only a substitution of parameters in the fit process (y2 -> x1-c1 or A2 -> A1*C1 or w2 -> w1*c1), it’s surely a nightmare to implement in the GUI. There are fit procedures capable of this, but just in table form. Maybe it’s possible to present an n x n table for the peaks with link constants to dial in.
Thanks for reading this wall of text.
I started a response to your posting, but got interrupted. I still have it in mind!
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
June 1, 2010 at 12:59 pm - Permalink
A bug, if a MultiPeakFit graph is open with the attached external control panel containing many peaks:
- When the focus is on another window (another graph, table, etc.) clicking & dragging the scroll bar of the external panel containing the peak list will result in a additional list generated in the graph window and scrolling this list instead of the one in the external panel
- It would make things quicker if its possible to easily get rid of all (or most of) the peaks to start over. Maybe a button ' delete all peaks' would be the easiest way. Ok, maybe the reply to this is to just start a new set, but I hate to have my experiment cluttered with useless stuff, so I use exactly one set per wave.
July 7, 2010 at 03:27 am - Permalink
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com The Wish List is never dead! Thanks for the report. I will look into it. That could be a useful enhancement. Right now, clicking the Auto-locate Peaks Now button will clear out the peak list and replace it with a new guess. If you don't want the auto guess, of course, that isn't helpful...
July 7, 2010 at 08:50 am - Permalink
Meanwhile I think I have found something regarding this issue:
In the function MPF2_AddFitCurveToGraph is this written:
if (WaveExists(xDataWave))
xleft = xDataWave[XPointRangeBegin]
xright = xDataWave[XPointRangeEnd]
else
xleft = pnt2x(yDataWave, XPointRangeBegin)
xright = pnt2x(yDataWave, XPointRangeEnd)
endif
Variable dx = MPF2_MinPeakWidth(wfi)/10
Variable npnts = 200
if (dx > 0)
npnts = (xright-xleft)/dx
endif
Variable/G MPF2_FitCurvePoints = npnts
String fitName = "fit_"+NameOfWave(yDataWave)
Make/O/N=(npnts) $fitName
...
The xleft and xright values are grabbed from the beginning and end of the wave, but with negative scaling delta xleft will be bigger than xright. This means the following npnts calculation will come out negative, thus breaking the make command. Ok, an
abs()
will solve this, but there may be other cases where the normal direction of the wave scaling is assumed. When I find the time I may change this and see if I find other errors.July 8, 2010 at 01:51 am - Permalink
In fact, in my development copy of Multipeak Fit 2, I have this:
npnts = abs(xright-xleft)/dx
endif
I think it's great that you're willing to help me debug the package, but I suggest that you get the latest version of Igor 6.20B03 so that you can avoid fixing things that are already fixed.
I still haven't forgotten about your great list of comments above!
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
July 8, 2010 at 04:25 pm - Permalink
Now, moving on to my other 'bug' (now with Igor 6.20):
I think, the cause is
Make/O/D/N=1 $(DFPath+":'Baseline Coefs'")
in the BuildMultiPeak2Panel function which munches away all existing coefs but the first while generating the panel. This destroys all baselines with more than one coef.July 9, 2010 at 03:27 am - Permalink
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
July 13, 2010 at 09:39 am - Permalink
Understood. But if you make changes to code, it's best to do it with the latest version. The extra complication makes life more interesting :)
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
July 13, 2010 at 10:19 am - Permalink
July 14, 2010 at 05:46 am - Permalink
Fixed.
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
July 14, 2010 at 11:37 am - Permalink
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
July 14, 2010 at 05:14 pm - Permalink
Now the usage report for the latest version (in the order I notice it / get it reproduced):
a) When starting fresh the manual adding of a peak seems broken.
a1) Manual add of a peak before doing anything results in 'While executing a wave read, the following error occurred: Index out of range for wave "EditPeakList"', but the peak still gets added.
a2) After one click on Auto-Locate and then trying to add a peak results in 'While executing a wave read, the following error occurred: Index out of range for wave "HoldStrings"', but the peak gets added.
b) Ok, thats nothing new but I keep wondering why:
b1) A wave 'SavedCoefWave' is created and changed when clicking the hold checkboxes and contains the coefs of the clicked peak. Does this wave have any purpose? And if yes, then why isn't it generated in the set folder?
b2) A following error message (e.g. before a fit was made) after clicking on 'Peak Results' changes the current folder to the MPFset folder. The info message probably prevents the correct reset to 'root'. There may be other situations, when a different function exit causes this. I keep looking.
c) I still can't get the hold values remembered correctly. I guess, when the hold bits are written in the 'HoldStrings' wave everything gets loaded fine again. I try to watch the 'HoldStrings' values while playing around with the checkboxes, but so far the only option to actually write something there is to use the right-click menu and choose one of the two options. Simply clicking the checkbox or every other button does't update the values at all.
d) The initialization feature is cool, even gives a new way to get rid of all peaks to start over.
d1) Currently, when a set is started with the peaks panel open clicking on continue in the Start Multi-Peak Fit panel just gives focus to the graph. Wouldn't it be even more useful now (with the new initialize feature) to rebuild the panel with the new settings (i.e. automatically close the peak panel and rebuild / update everything with the new settings)? Just a tiny thing to get working more smooth.
d2) Loading a set for another wave currently doesn't work, because there is a name check in place, giving the error 'The Y wave you selected (...) is not the same as the one previously used with this graph. ...'. I think the intention was to copy over the the data to the new set associated with this graph an use the information as a starting point, but at the moment merely a set folder and a graph gets created.
July 15, 2010 at 03:54 am - Permalink
Thank you for the comments. I'm sure you've figured out that you're the guinea pig for the new features :)
Some of the comments you made are a result of incomplete implementation- you are absolutely right, of course, that the option to initialize from another set should ignore the fact that the waves are different. I confess that I tested by making two graphs of the same data. My only excuse was that I was trying to work fast and get on to my next big project :{
I will probably not get back to the fixes for this stuff until Monday.
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
July 15, 2010 at 01:35 pm - Permalink
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
July 19, 2010 at 12:01 pm - Permalink
It is used in other parts of the code. I have added a KillWaves to get rid of it after the code executes. That code is triggered, but doesn't actually have anything to do in this case.
Fixed.
Heh. I fixed part of the hold problem but not all of it. Should be fixed now.
Yes, you're right. It now checks to see if you selected "Previous set for this graph" or if you selected the same set as the graph is using. if so, it simply brings the graph forward. Otherwise it asks if you want to close the panel and re-create it.
fixed.
Wish granted with a new Checkpoint menu. It has only one checkpoint level available. It should save each checkpoint separately, then let you go back one at a time (or select the checkpoint you want) just like modern undo/redo. Maybe for the future.
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com Fixed. The errors are real but don't cause a problem. I have added code to prevent the errors.
July 19, 2010 at 04:34 pm - Permalink
I try to write a new report, when I have more time. It seems that there are some edge cases where the checkpoint system fails, but good thing it's possible to always reload it via the main menu.
Hopefully you will find the time to work a little more on this project.
EDIT:
ok, some things I have found:
Checkpoint Feature:
- When I choose 'restore checkpoint' the checkpoint sometimes fails to load. The cause is in the function MPF2_CheckpointMenuProc(s). The
SVAR
named 'gname' is referenced as the graph name, but later on the variable gets killed by theKillDataFolder $DFpath
and giving a null string to the following MPF2_StartNewMPFit- sometimes it works somehow (maybe the folder removal fails at some point)
Initialization Feature:
- I was misled by the 'start fresh' option, which currently doesn't get rid of the peaks as I thought but generates a completely new set. Currently, for my taste it's too easy to end up generating new sets and too difficult to resume present sets (at the moment at least the graph has to remain open or a recreation macro has to be prepared). I have no cool suggestions for this at the moment. I try to come up with a idea to simplify the start menu while ease up things. Maybe reversing the order of options is better? Currently its like
{specify wave} -> graph:{old}/{new} -> init: {empty}/{old}/{set1}/{set2}...
but working from the view of sets would be like:
choose set: {new}/{set1}/{set2}
-> if previous set -> init: {delete peaks}/{old peaks}
-> if new set -> {specify wave} -> init: {empty}/{set1}/{set2}
Peak Adding Error:
Some more situations for the infamous 'HoldStrings'-issue:
In the function MPF2_SortAutoPeakWave after adding the first peak, this fails:
Duplicate/O/T holdwave, MPF2_TempHoldWave
holdwave = ""
holdwave[1,] = MPF2_TempHoldWave[MPF2_indexwave[p-1]+1]
endif
because holdwave (= HoldStrings) has only one point and the 4. line generates the error. And more occasions, where HoldStrings is referenced with too high counters:
if (i < DimSize(EditPeakList, 0))
sscanf EditPeakList[i], "EditPeak %d", peakNumber
HoldStrings[numPeaksBefore, ] = ""
endif
in function MPF2_EditOrAddDoneButtonProc(s) and
in function MPF2_RefreshHoldStrings(PanelWin)
July 20, 2010 at 07:47 am - Permalink
I have copied your comments to a notes file, and will address them soon.
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
July 20, 2010 at 04:34 pm - Permalink
Yuck. Easy mistake to make! It didn't fail on my Macintosh. I tried stepping through it in the debugger- when you step over the KillDataFolder command, the debugger shows the gname string as null, but when you step into MPF2_StartNewMPFit function, it magically gets the right contents! I suspect that Igor is not actually killing the variables in the data folder, but just moving them to a "graveyard". We use that technique frequently to protect against crashes caused by stale pointers to data structures. It would be really nice if that caused a run-time error!
Anyway, it's fixed now.
Hm. "Start fresh" should be just like starting from a completely new data set. It should come up with a blank list, no peak traces, and the annoying message that covers the Do Fit button. I see that I failed to remove the peak traces from the graph, so there are peak traces that don't reflect any actual peaks. I have fixed that.
Well, I think a new user is going to think about his problem from the point of view of data set waves, so I think the present method is the right one- choose data sets, then a graph, then a previous fit set if you want. Using a previous fit set to initialize a new fit is a somewhat advanced feature...
I agree that it is a bit cumbersome.
These errors may yet kill me...
I've fixed those. I didn't run into any more myself, but it's become evident that my testing isn't realistic!
Thank you for all your efforts!
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
July 21, 2010 at 11:42 am - Permalink
Works very well so far. I will update if I find something.
EDIT:
I tried to use the init feature today. When I have a set (e.g. set 1), I try to set this as init for another wave. Everything gets loaded, but when I try to do a fit, the procedure fits the wave from the initial set (=1) and not the wave in the graph I want to fit (= set 2). In the new set (2) folder are fit_, Res_, and Bkg_ for both the old and the new wave and both versions are loaded in the graph. So the transfer feature is broken, since the new wave gets ignored by the fitting process.
July 27, 2010 at 01:44 am - Permalink
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
August 2, 2010 at 04:35 pm - Permalink
This one should be perfect...
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
August 9, 2010 at 04:44 pm - Permalink
Now, I can imagine you have other things to do, especially since the betas are rolling out, but I'd like to bug you a bit more with ideas for a changed results export.
I think the results table is a nice overview and one can sort the data too. How about a button to get a table generated (maybe in the root folder or whatever) with the contents and the sorting of the overview?
I even wonder how many people would then go for the tab-delimited output, since the table export from Igor itself gets this done too (and more).
Thanks a lot!
August 11, 2010 at 03:55 am - Permalink
Er, I just came across this and downloaded to join the game --- the new package looks very nice!
I hope it's okay to report errors here. If you prefer e-mails, please tell me. The first thing happening were two "index out of range" errors (thanks to #pragma rtglobals=3 ? I use IP6.2.0.3 on Win7).
1) Upon invoking "Start New Multi-peak Fit" from the Analysis/Multi-peak Fit> submenu "a wave read error" occurred for
wsmFact
inFunction/C EstPeakNoiseAndSmfact(w,pBegin,pEnd)
in PeakAutoFind.ipf (using version 5.06).
2) I then set up my wave
dum1
as the y wave and pressed the "Auto-locate peaks now" button. I think this is when I first got the error message** a wave write gave error: Index out of range for wave "Res_dum1".
but I could not reproduce this the second time round when I started writing this comment. Mayhap it was due to the way I tried to follow up on the first error.
3) When trying to restore a previously saved Checkpoint, I always get an error
** KillDataFolder gave error: can't kill a wave that is part of a graph, table or user function or is in use by an XOP
Cheers,
Wolfgang
PS I absolutely love the improved result reporting in its various forms. Do you presage a mechanism to "customize" it (eg user-defined functions or some such?)
Wolfgang Harneit
August 11, 2010 at 12:07 pm - Permalink
So you get this error immediately upon selecting the menu item, while the Start Multipeak Fit panel is being built?
To try to reproduce the rest of your errors, I used the waves 'my y data' and 'my x data' from the Multipeak Fit 2 Demo experiment.
It must be something uninitialized. Try starting from a fresh experiment or even directly after starting up Igor. I don't see this error; maybe there's something particular about the order in which you do things.
I also can't reproduce this error. If you could provide data and a recipe, that would be helpful.
So far the only customizable aspects are the ability to add custom peak shapes and background functions.
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com Thank you. Reporting errors here is fine, although I'm not quite as responsive here as I am to
August 11, 2010 at 12:40 pm - Permalink
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
August 12, 2010 at 10:04 am - Permalink
Wolfgang Harneit
August 17, 2010 at 12:57 am - Permalink
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
August 17, 2010 at 09:09 am - Permalink
For now, I stick with the posted version and report bugs as I find them.
The fitting process can produce peaks (peakwaves) with negative delta, e.g. when a peak grows unreasonable big and grows outside to the left of the fitting area.
Then the tag-adding function in MPF2_AddPeaksToGraph gets a problem:
if (w(attachPoint) < 0)
anchorCode = "MT"
endif
calculates the attachpoint wrong so that the value isn't inside the peak range. The Cause: When center is below XStart, the first line gives XEnd instead XStart for some reason (bug in chain conditionals?).
September 7, 2010 at 02:29 am - Permalink
if (w(attachPoint) < 0)
anchorCode = "MT"
endif
This fix will be included with the 6.20 release, which is coming Any Day Now.
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
September 7, 2010 at 12:18 pm - Permalink
Was there a solution to this? I can't find an option for it.
Thanks for Multipeakfit, it rocks!
August 15, 2012 at 06:22 pm - Permalink
1) back up your experiment and close Igor.
2) open your Igor Pro Folder (If Igor were open you could go to menu item Help->Show Igor Pro Folder - maybe do 2) before 1)...)
3) navigate to [Igor Pro Folder]->WaveMetrics Procedures->Analysis->Peak Fitting
4) make a backup of the old Multi-peak Fitting 2.0.ipf and .ihf - copy them into a folder outside the Igor Pro Folder (and outside the Igor Pro User Files folder)
5) copy the new Multi-peak Fitting 2.0 files into the Peak Fitting folder
6) open Igor and launch your experiment
The new MPF 2.0 will be compiled into your experiment. Click the "Apply Constraints" checkbox, select one of your peaks and set the min value of its height to be 0. Then right click on the min value box and choose to apply the same value to all common peaks. This will apply the minimum value to the height coefficient of all peaks of the same type.
Good Luck!
Nate
Nate Hyde
WaveMetrics New Guy
Nate Hyde
August 16, 2012 at 10:38 am - Permalink
:)
August 16, 2012 at 01:04 pm - Permalink
Too late. We already did :)
August 16, 2012 at 01:39 pm - Permalink
Do I understand correctly that *all* of the tasty features discussed here are already in 6.3?
(I'm downloading 6.3 now... by the way, the example graphs on WaveMetrics.com do not do Igor 6 justice...)
August 15, 2013 at 02:24 am - Permalink
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
August 15, 2013 at 11:13 am - Permalink