Bring Igor Pro into the 21st century

Share icon

I know, the subject line sounds really pretentious, but as much as I enjoy working with Igor, it does feel outdated in many ways.

A few points that Igor could really benefit from:

  • Give the editor (procedure windows) a make over:

    • code folding

    • code completion

    • call tips

    • line numbers (I know this has already been discussed)


    Features like these are standard in many programming and data analysis environments.


  • Tab completion for the command line.

  • The question of having a 64-bit version available under *both* Windows and Mac OS X has already been discussed, but is important enough to point out again and again

  • long wave names – also something that has been discussed again and again, but would be really useful

  • integrating multidimensionality more thoroughly than is the case at the moment would also be very helpful



Just to repeat myself: I enjoy using Igor. it offers a lot of easy and straightforward solutions to many everyday problems and I am grateful to that. But sometimes I really long for a better handling, especially when programming.
chris wrote:
I know, the subject line sounds really pretentious, but as much as I enjoy working with Igor, it does feel outdated in many ways.

Yeah, we know that. We have code in there that was written before 1989...

We are in the process of what one of the owners has termed a "nuke and repave" of the source code. Some time in the (possibly distant) future, things may be improved.
Quote:


  • Give the editor (procedure windows) a make over:

    • code folding

    • code completion

    • call tips

    • line numbers (I know this has already been discussed)


    Features like these are standard in many programming and data analysis environments.


We generally take the attitude that Igor is fundamentally a graphing and analysis program, and not an IDE. But the programming part is important, and making it easier should also be important. These things have been suggested before, and they are on the Wish List.

Quote:

  • The question of having a 64-bit version available under *both* Windows and Mac OS X has already been discussed, but is important enough to point out again and again

    • Right- this is a consequence of Apple's having pulled the rug out from under us by suddenly and unexpectedly ditching 64-bit Carbon. We don't have a lot of desire to re-write Igor's UI in Objective C/Cocoa. We are actually taking a slightly different approach as part of the "nuke and repave" project.
    Quote:

  • integrating multidimensionality more thoroughly than is the case at the moment would also be very helpful

    • It would be a huge job to make everything totally multidimensionally aware, but I agree that things could be improved. We would be interested in what things you feel are most important.

    John Weeks
    WaveMetrics, Inc.
    support@wavemetrics.com

    January 28, 2011 at 09:21 am - Permalink

    I have a real love-hate relationship with Igor. At this point I've got several thousand lines of (occasionally) useful utility code that makes some pieces of my work incredibly easy.

    As I use Igor, it's much more of an IDE with fantastic graphing and analysis built in. If I have to do something more than once, shouldn't I be scripting it?

    I really like how easy it is to generate code from the menu commands. The problems I have stem from more than just the failings as an IDE, however.

    It's normal for me to bounce between three or four languages in a week (sometimes in a day), so anything that makes me work too hard to remember things makes me grumpy.


    • The inconsistencies in the language drive me crazy. I constantly have to remember if I call something with or without parentheses. Why do I "GetDataFolder(1)" but then "SetDataFolder foo" ? Consistency maybe be the hobgoblin of small minds, but needless inconsistency makes me into a goblin.

    • Passing in options with "option=3" is much nicer than "/g=3" - A lot of the time the options aren't even mnemonics for what they do.

    • There are a hints of a naming convention for functions, but then there are all sorts of surprises. I set the units with "SetScale" why don't I get the units with "GetUnits" ?

    • I'm tired of writing "do; if(bar == 0) break; ... ;while(1)" when I mean "while(bar != 0) ..."


    On one hand I can express some ideas in Igor very easily - I really like writing things like "foo = q + 3*p" - on the other, I feel like I'm too often battling the language to do trivial things.

    I've built some pretty sweet analysis tools in Igor - the ability to create right click menus with the window hooks totally made my day when I found it. But my inclination to stick with Igor lessens every time I switch to Igor after writing some python in vim (why isn't it easier to play nicely with an external editor? If I didn't have to leave vim to work with my code, I'd be a lot happier).

    Maybe it's not typical, but I do most of my work in Igor from the command prompt or scripts. It would be nice if those features didn't feel like they were a second thought.

    When I look at Igor, I see something that is almost awesome. At this point I'm leaning towards switching to working with something like Spyder or R - I'd like to help make Igor more awesome, but since I can't touch the guts, I'm inclined to take my ball and play elsewhere. We've got some instrumentation that is run with Igor (it's the reason my advisor started using Igor, which is why I use Igor), but right now none of my data is captured in Igor, but I still use it for my data analysis - which would be a ringing endorsement, but at this I feel like I'm on my way out of the relationship.

    February 2, 2011 at 10:26 pm - Permalink

    Just my 2cents (also as a Mac user):

    What would help for me (as a programmer):
    - improved IDE
    - the possibility to make a bit like IDL some stand alone version of your code
    - 64 bit
    - long names anywhere
    - code completion & folding

    For most of my co-workers:
    - as much as possible code should history compliant (some action don't show up in history)
    - more word like features in the report (copy paste & scale graphs with mouse etc.)

    best regards

    Stefan

    February 3, 2011 at 05:03 am - Permalink

    skf wrote:
    The inconsistencies in the language drive me crazy. I constantly have to remember if I call something with or without parentheses. Why do I "GetDataFolder(1)" but then "SetDataFolder foo" ? Consistency maybe be the hobgoblin of small minds, but needless inconsistency makes me into a goblin.

    When I first started using Igor, the difference between operations and functions was something that confused me as well. But after using it for a while the difference started to make more sense. Built-in functions always return a value. Operations sometimes return a value via one or more local or global variables that are created and set by the operation, but they always do something. If you're typing in a procedure window, functions are brown and operations are blue green. So as soon as you type the name of the function/operation you should get a hint as to whether it needs parenthesis or not. You can also right click on a function/operation name in a procedure window and choose Insert Template.

    skf wrote:
    Passing in options with "option=3" is much nicer than "/g=3" - A lot of the time the options aren't even mnemonics for what they do.

    For operations with only a few flags, that might work OK. But there is a limit on the length of any single line of code or command line of 400 characters, and for very complicated operations such as MatrixOp or ImageTransform, it would be very easy to hit that limit if each option had to be passed as optionName=value. I don't know why the length is limited to 400 characters, but it is. Also, until not that long ago, flags could be only one character. This is why some flags use letters that don't have much relation to what the flag is for. Flags can now be 4 characters in length, but most operations written before 4 characters were possible have not been rewritten to use new 4 character flags (except for new flags that we've added since then).

    skf wrote:
    There are a hints of a naming convention for functions, but then there are all sorts of surprises. I set the units with "SetScale" why don't I get the units with "GetUnits" ?

    Because you use WaveUnits(), DimSize(), DimDelta(), and DimOffset(). Because functions can only return one value, there wouldn't be a good way to return up to 4 different values with a single function such as GetUnits(). Of course it could be an operation that creates up to 4 variables, but as I mentioned earlier operations do things (not just get information). With that said, there is probably at least one example of this rule being broken.

    skf wrote:
    I'm tired of writing "do; if(bar == 0) break; ... ;while(1)" when I mean "while(bar != 0) ..."

    I believe that this is equivalent (and also happens to be the same number of characters):
    for(;bar != 0;)
        // do something
    endfor


    skf wrote:
    Maybe it's not typical, but I do most of my work in Igor from the command prompt or scripts. It would be nice if those features didn't feel like they were a second thought.

    It may feel like that to you, but in reality I would say that they are not a second thought. I think all of us here get new operations working from the command line or user-defined functions first, and only then are dialogs created (if at all).

    skf wrote:
    When I look at Igor, I see something that is almost awesome. At this point I'm leaning towards switching to working with something like Spyder or R - I'd like to help make Igor more awesome, but since I can't touch the guts, I'm inclined to take my ball and play elsewhere. We've got some instrumentation that is run with Igor (it's the reason my advisor started using Igor, which is why I use Igor), but right now none of my data is captured in Igor, but I still use it for my data analysis - which would be a ringing endorsement, but at this I feel like I'm on my way out of the relationship.

    We encourage you to use the best tool for your application. We hope that it is Igor, but that isn't always the case and we accept that.

    A lot of the quirks in Igor today stem from the fact that, when we release new versions of Igor, we try very hard to maintain backwards compatibility. To new Igor users the end result is that some things seem inconsistant and perhaps even nonsensical. For example, Igor originally only supported 1D waves. At some point this was extended to 4D wave support. Many operations were upgraded to support multidimensional waves, but in a lot of cases it is not obvious how an operation should treat multidimensional data, so we still have a lot of operations that are not multidimensional aware.

    I know that these explanations don't do much to make learning Igor easier for a new user, but perhaps you can at least understand that a lot of the inconsistencies are historical and that we are not intentionally making Igor difficult to use :)

    We always love to get feedback from our users about things they like, things they don't like, and changes/new features they'd like to see. As John mentioned earlier, we are currently overhauling a lot of Igor's parts, and when that's done I expect that a lot of the complaints people have with the IDE and editing procedure files will be addressed.

    February 3, 2011 at 08:12 am - Permalink

    While I understand that Wavemetrics sees Igor mainly as a graphing and data analysis program, I think it is important to consider that much of that analysis and graphing is done using self written macros and functions. Thus, the text editor component is of central importance to many (possibly the majority of) users. And things like call tips and tab completion are useful from the command line as well.

    Concerning the shortcomings of multidimensionality, I can't name a specific grievance off the top of my head. As aclight wrote, most of the multidimensionality is squashed into the matrixop and imagetransform operations -- not always the most straightforward manner of solving things. I will drop you a line the next time I hit a wall.

    No matter what, I am more than curious to see what the "nuke and repave" project results in!

    Chris

    February 7, 2011 at 07:45 am - Permalink

    Just curious:

    How is the nuking and paving going?

    Waiting with baited breath.

    November 14, 2011 at 10:57 am - Permalink

    chris wrote:

  • integrating multidimensionality more thoroughly than is the case at the moment would also be very helpful


    • You are not telling us exactly what kind of MD integration you have in mind. IP7 has (already) a new built-in Integrate2D. As the help says:
    "The Integrate2D operation calculates a two-dimensional numeric integral of a user function or a wave. The result of the operation is stored in the variable V_value and the variable V_Flag is set to zero if there are no errors."

    A.G.
    WaveMetrics, Inc.

    November 14, 2011 at 11:35 am - Permalink

    Quote:
    Just curious:
    How is the nuking and paving going?
    Waiting with baited breath.


    I'm guessing you might see a beta in a year. So you might want to consider breathing before then :)

    November 14, 2011 at 11:53 am - Permalink

    Sigh

    or is it

    sigh * sigh














    Pronounced (Psi Star Psi) nerd joke from MIT undergrad days

    November 14, 2011 at 02:20 pm - Permalink

    Just curious. Any chance that the nuke and repave is going to be using a cross-platform library like Qt? It'd be a lot of work initially, but would result in it being much easier to maintain versions for different operating systems - we could even have a Linux version.

    November 27, 2011 at 10:12 am - Permalink

    Quote:
    Just curious. Any chance that the nuke and repave is going to be using a cross-platform library like Qt? It'd be a lot of work initially, but would result in it being much easier to maintain versions for different operating systems - we could even have a Linux version.


    We are using Qt as the base framework so this should, at least in theory, provide a possible path to Linux.

    And yes, it is a lot of work initially.

    Just to put things in perspective, I estimate it will take us at least another year before even an early beta, so this project is a long way from bearing any fruit.

    We're happy to hear your suggestions but it's too early for us to say what features will be in Igor7.

    November 27, 2011 at 12:07 pm - Permalink

    It would be good to get a heads up if this is not going to be backwards compatible for some things, mainly GUI's in XOP's.

    I take it OS X and Windows are then going to be 64 bit? Will 64 bit be the default version? Since it's on QT, would we be able to run on Linux? If so, what's going to happen with XOP's?

    November 27, 2011 at 03:05 pm - Permalink

    We won't know the answer to those questions for a long time.

    I expect that we will be beta testing Igor7 for a year so you will have plenty of time to prepare.

    November 27, 2011 at 06:30 pm - Permalink

    Igor wrote:

    IP7 has (already) a new built-in Integrate2D. As the help says:
    "The Integrate2D operation calculates a two-dimensional numeric integral of a user function or a wave. The result of the operation is stored in the variable V_value and the variable V_Flag is set to zero if there are no errors."


    Given that IP7 doesn't appear to be out yet, I can't comment on this.

    But this is actually a good example of what I mean: Integrate1D does a 1D integral, Integrate2D does a 2D integral, will Integrate3D do a 3D and Integrate4D do a 4D? Why not a single command? And it's not as if the special command names for each dimensionality follow a stringent system. Depending on what you want to do you need to use the ImageTransform operation for anything above 1D, in other cases (i.e. interp) each dimensionality has a command of its own.

    When I discuss the advantages and disadvantages of different data analysis environments with colleagues, the one thing Igor scores really badly on is the clarity of syntax. Multidimensionality is definitely a good example of this (on the other hand, what Igor always wins with is the interactivity of plots/images and the ease of generating GUIs).

    Again, I'm a big fan, but there are some grievances.

    Cheers, Chris

    December 1, 2011 at 10:28 am - Permalink

    chris wrote:

    But this is actually a good example of what I mean: Integrate1D does a 1D integral, Integrate2D does a 2D integral, will Integrate3D do a 3D and Integrate4D do a 4D?


    The short answer is: "I don't know".

    The long answer is: you need to understand that we have finite development resources so we add functionality based on our estimate of what our customers might need/use and what they request. When dealing with data of more than 1D you will find that the number of customers using higher dimensionality falls exponentially with increasing dimensions so in as much as I would love to support integration in 15-dimensional space I really don't have the time to implement it in IGOR.

    The next issue to consider is that when we add functionality we do our best to preserve backward compatibility. When I added Integrate1D (March 2003) there was already a built-in operation called Integrate which did something different. Now I could have lumped the new functionality into the old operation and used some complicated set of flags to differentiate (no pun) between the two but I thought it would be cleaner if it used a completely different name. At this point, I suppose I could have used "IntegrateND" as a descriptive name while stating unambiguously in the documentation that it only supports 1D... Instead I chose Integrate1D which tells you exactly what it does.

    Over the years it became apparent that many people use this operation to perform 2D or 3D integrations in a manner described in the documentation. In some applications the extra overhead is prohibitive so I added the 2D version. At this point you could argue that the 2D version should be implemented under Integrate1D, but you need to consider that at any point in time there are new IGOR users who would never even dream of finding 2D integration under Integrate1D. Also, when you have access to IP7 you will find that the operation is significantly different to justify its own set of flags and keywords. After years of working with Integrate1D we (would like to) think that we learned some lessons so it is designed differently.


    Quote:

    And it's not as if the special command names for each dimensionality follow a stringent system. Depending on what you want to do you need to use the ImageTransform operation for anything above 1D, in other cases (i.e. interp) each dimensionality has a command of its own.


    There was an internal historic reason for ImageTransform. Another reason for the InterpXXX operations. Much of the obscure ImageTransform functionality is now taken over by MatrixOP. I hope that we can clean up the Interp situation -- I'd happy to hear some suggestions.

    Quote:

    Again, I'm a big fan, but there are some grievances.


    We appreciate your support and feedback.

    A.G.
    WaveMetrics, Inc.

    December 1, 2011 at 11:25 am - Permalink

    Purely out of wild speculation and naive curiosity I must ask ...

    Can IP7 be created with a "modern", consistent, forward-thinking language set AND still maintain solid, backward-compatibility? How mutually exclusive are these two requests? Could one satisfy the two apparently divergent issues via a "composite" framework of sorts -- for example, using a "compatibility XOP" that allows users to transition from the old to the new by plugging-in an "#include IP6LanguageSet" or "#pragma IP6" command line in their code as needed?

    While I understand much of the current language set in IP and would need to readjust to learn a new one, I would be more than happy to put in the effort if I could for example write something like ...

    variable cMax3 = wave.max.column[3](myMatrix)


    ... or an equally "modern" equivalent that makes the coding cleaner and more understandable all around.

    --
    J. J. Weimer
    Chemistry / Chemical & Materials Engineering, UAHuntsville

    December 1, 2011 at 01:31 pm - Permalink

    I also have a "feature request" for IP7. Since I heavily use the DataBrowser I would really welcome if it would be possible to add an option to print in the history when one duplicates a wave, or copies waves from another IGOR experiment (through Browse Experiment), etc. (basically for any of the features accessible through the DataBrowser). This way the history would be more consistent and thus more helpful - now all this info is missing and it is really difficult to follow the steps taken when the DataBrowser was used.

    Thanks for any replies!

    Gregor

    January 14, 2012 at 02:40 am - Permalink

    hrodstein wrote:
    We are using Qt as the base framework so this should, at least in theory, provide a possible path to Linux.


    So, here's a question. I have an XOP with a couple of windows that I draw myself. (Partly for historical reasons, partly because I do things like changing text style and color to indicate status.) The XOP is limited to MacOS for numerous reasons, not the least of which is that all of us using it prefer Macs.

    These windows currently use Carbon QuickDraw calls. Stop laughing, the code is 15 years old. QuickDraw is long deprecated, and is no longer documented in the standard Xcode documentation (unless I'm missing it). So I probably need to update it soon. When I update, which of these approaches is likely to continue to work under Igor 7?

    1. Stick with Carbon, and just change the calls to Quartz.

    2. Convert it to a Cocoa window and assume it will get -display messages in the main thread as needed.

    3. Give up entirely on rolling my own and just make an Igor Panel with lots of ValDisplay controls



    I would most prefer the middle way, since then I get all the control handling and resizing for free.

    September 18, 2012 at 11:56 am - Permalink

    KevinRBoyce wrote:
    When I update, which of these approaches is likely to continue to work under Igor 7?


    I don't know how XOP windows will work in Igor Pro 7. Only number 3 has a high likelihood of working. Number 1 and number 2 may work but I won't really know for a while, possibly a long while.

    September 18, 2012 at 12:02 pm - Permalink

    KevinRBoyce][quote=hrodstein wrote:


    I would most prefer the middle way, since then I get all the control handling and resizing for free.


    Iff WM are going the Qt route then you may be missing option 4. Option 4 being make the GUI in Qt. I have recently started writing apps with the Python/numpy/Pyside/Qt combo. It is simple to design a GUI with the Designer program (has all the layout managing, etc). The same code can then be used cross platform on OSX/Windows/Linux, assuming that you have a compiler. It does require that you know C++, but I guess you already have that.

    It may require that WM provide special functionality in the XOP toolkit.

    A.

    September 18, 2012 at 06:15 pm - Permalink

    Can you say anything at this point about how the "nuke and repave" is going? Also, are any major updates to the scripting language part of of the nuke-and-repave effort?

    Adam

    October 16, 2013 at 12:08 pm - Permalink

    Quote:
    Can you say anything at this point about how the "nuke and repave" is going?


    It's going steadily but slowly and we have no ETA.

    Quote:
    Also, are any major updates to the scripting language part of of the nuke-and-repave effort?


    No.

    October 17, 2013 at 07:34 am - Permalink

    Anything on the horizon yet? Really feeling the need for 64-bits on Mac OS X.

    October 14, 2014 at 08:55 am - Permalink

    We do have a Technology Preview program available to adventurous Igor users. We restrict this to longstanding users for whom we have a history of tech support interaction. Contact support if you fit this category (and other than the adventurous bit, it appears Chris does.)

    Larry Hutchinson
    WaveMetrics
    support@WaveMetrics.com

    November 1, 2014 at 03:34 pm - Permalink