Igor Package Installer
hrodstein
This sounds like a great idea. Something like Synaptic Package Manager on Debian/Ubuntu but for Igor. Howard, could you specify what sort of problems you think would arise if this was not implemented directly in the Igor source? If they are not insurmountable maybe it is worth trying it as a normal Igor user project since it seems like the interest is there for development.
I started to write a simple Igor package loader a couple of years ago. The goal was just to create shortcuts from a package folder for procedure and help files. The first problem I ran into is how to make the package available to the end user without actually loading the package. This is desirable because the user may have dozens of packages on his hard drive but does not want all of their procedure files loaded all of the time.
For WaveMetrics packages this is solved via the WMMenus procedure file (which is now an independent module so you must execute "SetIgorOption IndependentModuleDev=1" to see it). WMMMenus.ipf is installed in the Igor Procedures folder all the time. It adds menu items to various menus which load packages. The packages are not loaded until the user chooses the menu item. Thus with just one procedure file, the user has menu access to many packages.
There is no equivalent to WMMenus.ipf for user procedures. In order to add a menu, a user procedure must put a procedure file in Igor Procedures. If the user has dozens of packages available that means dozens of procedure files in Igor Procedures.
Thus what is required is a way for a package to add a menu item without being loaded. This would require that Igor look in a well known place for some configuration file that would specify the package loader menu item. This requires a new feature inside Igor.
Digression - Igor Pro 6.1 adds the "Igor Pro 6 User Files" idea which provides the well-known place. Choose Help->Show Igor Pro 6 User Files. You can access this location using SpecialDirPath. - End Digression
Another problem is that you have to quit and restart Igor to load XOPs. I have had on my to-do list to eliminate this requirement but it has never reached the top of the list.
Another problem is closing procedure files in order to replace them with a newer version. Since you can't close a procedure file while procedures are running, this can be done only via Execute/P. This therefore means that you have to run some procedure that calls Execute/P, let the procedure finish so that the Execute/P gets serviced and then run some other procedure to replace the package procedure files, let the second procedure finish so procedures can be compiled, and start a third procedure to launch the package. This is too kludgy and fragile for my taste.
This is just what comes to mind with five minutes of thought. I suspect if we were to implement this internal to Igor it would lead to many changes within Igor itself.
I suspect that you would want changes to Igor to accomodate the package installer. This would drag us little-by-little into the project and we might wind up doing it ourselves anyway or wind up with a hybrid that is kludgier than a built-in package installer would be. So rather than add pieces of the puzzle little-by-little, I prefer to wait until we have the time to think it out in advance.
Having said that, if you want to take a crack at it, more power to you!
Something like Synaptic Package Manager on Debian/Ubuntu but for Igor.
If I were to do this, I would study Synaptic Package Manager to see what can be learned from or leveraged off of it.
I'm not sure what happens on either platform if one tries to overwrite xops or procedures that are currently open because Igor is open.
You get an error.
This project would be best done as a collaboration as we could develop a more rigourous method together.
You could set up a code snippet on IgorExchange which would be not code but a description of how the installer would work. This could be revised by multiple people like a wiki.
However, I do get Howards point. Something built into Igor would have less dependencies (easthttp, zip) and would therefore be less likely to break. Plus the rules of how packages would be created, etc, would be better adhered to by the community.
I agree. But if you do try to build an installer, I think it is important to think about the structure of a package and try to define something that covers most of the bases and is flexible from the start.
What Howard talks about here is a dynamic load of XOP's and procedure files whilst IGOR is still running. Originally what I had in mind was a way of installing packages so that the next time IGOR was run those procedures would be available. Thus the install would write files to e.g. /User Procedures, create necessary shortcuts in /Igor Procedures and /Igor Extensions, before quitting IGOR so the user could then restart Igor. This install could also be an update. I hadn't totally envisaged the dynamic loading. The reason I want to do this is a lot of packages have a) a lot of files and b) it is quite hard for all those files to go in the right place, with the right shortcuts, etc. It is best to take as much strain off non-expert users as possible. You wouldn't believe (well ok you probably do) how difficult it is for some people to create shortcuts.
The installer package could create a menu procedure file as part of the installation, with the last step being an INSERTINCLUDE directive. One could include that list of commands from code enclosed in the package manifest. (one would probably want to check those commands for baddies e.g. deletefile *.*).
I don't think it's much of a problem quitting IGOR as part of the install (after asking the user if they want to save the experiment). It's two seconds to restart it.
I agree, that that is quite kludgy. However, I've just been experimenting with overwriting live procedure files with my zip xop. The file is changed even though IGOR doesn't think so. If you then quit and reload IGOR the new file will be used. Of course one would have to warn the user that procedures are about to be overwritten.
A repository function would be quite cool because one can take care of all dependency packages quite simply. For example, lots of people may use the multifile loader package. However, I install in with my code because I need it to do something. Someone else installs it when they need it. You end up with a lot of repeat files and a lot of collisions. With a bit of disicpline, and the aid of a repository, people install things only once. An installer package could glean information from a manifest file what the other dependencies were, so it could also download and install them as well.
THe difficulty there is that the installer would need to know whether those dependencies were already installed or not, that's the main problem.
March 14, 2009 at 01:25 am - Permalink
What about one file in Igor Procedures witb #include directives that would be updated when the list of loaded packages is updated? You could use CloseProc to close it, then write to it like a text file, then OpenProc to open it again.
Didn't know about this. Previously (on Windows) I was using shortcuts containing the environment variable %USERPROFILE% so that a shortcut in the conventional User Procedures folder would map to one folder when user "Smith" was logged in and another when user "Jones" was logged in.
I think that should be a higher priority for another reason -- XOP development time would be much faster if you didn't have to quit Igor and restart it around every compile of an XOP.
I am already doing this as part of built-in Subversion support I added to Igor. It closes all procedures (save one), updates them from the server, and opens them up again. Jim Prouty fixed a CloseProc bug associated with this usage, which will be in the next public beta.
March 14, 2009 at 04:36 am - Permalink
I agree that restarting Igor is not a big deal (though it takes a lot more than 2 seconds to restart it, at least on my computer). For example if you install or update a Firefox plugin you have to do the same thing. What would be nice is if Igor provided a way to restart itself, just as the Firefox plugin installer does. Ideally you would just click the "Ok to restart" button after running the installer/updater and Igor restarts and then reloads the same experiment you were already in.
March 14, 2009 at 05:18 am - Permalink
That would simplify things. What would simplify them even more is to quit Igor BEFORE starting the install. This would mean that the installer was a standalone program. This could be written in QT which is platform-independent, although there is a learning curve.
It would work if the installer created the moral equivalent of WMMenus.ipf in Igor Procedures, for example, "User Menus.ipf". Menu items for all user packages would be created in this file. However I would store the file in "Igor Pro 6 User Files" not in the Igor Pro Folder. VISTA does nasty things when you install files in the Program Files folder.
I'm a bit surprised at that. Is that the case on Macintosh and Windows?
Yes, this would work. However I would store the file in "Igor Pro 6 User Files" not in the Igor Pro Folder. VISTA does nasty things when you install files in the Program Files folder.
You would still have to quit to compile because you're overwriting the XOP executable file.
This might be fragile. That is, we could change something internally regarding how Igor compiles procedures and inadvertently break your procedures.
March 14, 2009 at 12:29 pm - Permalink
SavePackagePreferencesmode. THis is a big issue, as procedures in a package can be stored anywhere.March 14, 2009 at 08:21 pm - Permalink
The user should need only to specify the folder containing the package. Igor should then make available any procedure files, XOPs and help files in the package folder. There should be no need to create shortcuts in various folders (Igor Procedures, Igor Extensions, Igor Help Files).
This will require changes to Igor to allow the user to activate the package by pointing to the folder and to scan the folder contents when Igor is launched.
March 15, 2009 at 04:59 pm - Permalink
I actually created an installer for NIDAQ Tools MX. For initial install, it works moderately well, unless you lack administrator privileges. On Vista it works less well, because it writes to the Program Files folder. Use of the new features of 6.1 would help with that, but I can't do that yet.
It is shipped as an Igor experiment file so that it can be easily run and to keep it all in one piece.
It keeps a log notebook so that you can see the progress and check errors later.
It even shuts down Igor after the installation and re-starts.
If it is installing an update, it has big problems because it can't overwrite files that are already open. This includes the XOP and help files. Procedure files are not a problem because the installer is an experiment file, and it does not have the procedure files open.
I made a decision a while back to put my NIDAQ Tools procedure files into the WaveMetrics Procedures folder. I did that because it is, after all, a WaveMetrics product. I don't want them in the User Procedures folder because that is a place for your own files. But now (unless I'm mistaken) that means I can't use the new 6.1 features that look for files in user-controlled folders. That may have to change, or maybe we need yet another place for procedure files- an "add-on package procedures" folder.
I have attached a copy of the installer experiment so that you can all have a good laugh!
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
March 16, 2009 at 09:35 am - Permalink
Yes, I think we need something like that or the moral equivalent which would be a preference remembered by Igor and managed via a Manage Packages menu item/dialog.
(John, I suspect this message will travel a long way to get from my office to yours :)
March 16, 2009 at 02:33 pm - Permalink
Here Jan is having problems distributing easyHttp, because he needs to have it for his package. However other people may already have installed it in other places. (maintenance and other issues). If the package was available from a repository Jan could install it during the installation process.
March 16, 2009 at 05:22 pm - Permalink
1) How should Igor packages be managed (stored and/or loaded)?
2) How should an Igor package be installed or updated?
3) How should a user know when to update a package?
First, I suggest that distinguishing between procedures (.ipf) and XOPs (.xop) is and will only continue to be confusing to a new user. One terminology is needed for both ... I suggest it be "package". Note, this terminology excludes consideration of Experiments (.pxp) and Help Files (.ihf) that are standalone (not associated with a given "package").
As for the first issue, if my wishes could be granted, I would want to manage Igor packages in a different way than currently done.
* Storage
Rather than hunting for package files in at least six folders (IFDL Procedures, Igor Extensions, Igor Procedures, More Extensions, User Procedures, Wavemetrics Procedures), I would like to find only three folders for storing them ....
---> Startup Packages - a place for packages that a user wants always to be loaded at start up
---> User Packages - a place for packages that a user prefers to load by #include "" or #include <>
---> Optional Packages - a place for other packages that can only be loaded by #include <>
* Loading
I would like to have the #include directive be re-defined based on the above package storage paradigm.
---> #include "MyPackage" - search the User Packages folder only (ie, this is a package the user desires to include)
---> #include < MyPackage > - search the User Package folder, followed by the Optional Packages folder (followed by the Startup Packages folder when issued from an IndependentModule) (ie, this is an optional package the user may or may not want to include themselves)
The above set up would solve at least one major package management issue. When a user develops a package on Igor Exchange that relies on OtherPackage, he/she would only need have the #include < OtherPackage > directive at the top to be certain the required file is properly included. It would also provide a step toward cleaner management at the install/update side of things.
As for installing or updating packages, I suggest for overall peace of mind, the installer needs to force Igor Pro to quit any current instance, re-start Igor Pro in an "Installer Only" mode, run an installer script, quit the installer only mode, and restart Igor Pro. I see an "Installer Only" mode of Igor Pro as a minimal application that essentially only needs to do file manipulations (delete and copy) at the directory level. I see the package installer script as a combination of XSL/XML. The XSL is a hard-coded, user immutable part. It translates the package developer's XML into a sequence of Igor Pro file delete and copy directives.
Finally, as to informing a user about a package update, I tend first to take the LaTeX/CTAN approach. When I download a package that requires a newer version of one of my existing packages, I get an error/warning on compile. The second step after this is to have each package include the equivalent of a "Check for Updates" menu selection specific to itself only. This could be run from a common package to Igor Pro and could be developed separately from the universal package installer.
So, in summary, it seems to me the needs are for ...
A) A universal CheckForPackageUpdate(PackageName,CurrentVersion,[URL, protocol]) type function
B) An "Installer Only" mode for Igor Pro
C) The syntax of the XML portion of a package installer script
D) The syntax for the corresponding XSL --> Igor Pro portion of the installer script
E) The Igor Pro installer package
--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH
March 18, 2009 at 03:19 pm - Permalink
Igor wouldn't actually NEED to do any file manipulations other than altering the contents of its preferences file. It would remember which packages were enabled. This wouldn't require an "installer mode", either.
"If the only tool you have is a hammer, everything looks like a nail."
XSL/XML is a useful tool, but here the contents of a package are obvious and distinguishable through the file extension. If an .ipf is in a package folder, then it's opened as a procedure file when the package is enabled. Same with .xop, .ihf. Even .pxps could be put in the Example Experiments submenu. And .ipfs have an #pragma igorversion=[required igor version].
XSL/XML is overkill, in my opinion. Well, except for the "requires [other package]" and "update from [here] via http" stuff.
Software Engineer, WaveMetrics, Inc.
March 19, 2009 at 10:53 am - Permalink
1) what dependencies were required to be downloaded
2) What to with ipfs
3) What to do with XOP's (which platform)
4) What shortcuts to make.
5) What to delete.
All this can be done in a well defined text file, that isn't XML.
March 19, 2009 at 04:32 pm - Permalink
The arguments against XML/XSL are rather hard for me to swallow for a number of reasons. I see it less as a "sledgehammer to a finishing nail" approach. Rather, I see it as a step toward using a well-documented standard with foreseeable long term stability, expandability, and flexibility. I almost think that developing anything else as an installer script in this day and age is akin to re-inventing the wheel. Also, the suggestion that XML/XSL is overkill EXCEPT ... leads me to ask .... then why not just take XML/XSL all the way? Why ask someone who is to write an installer script for their package to write that script in two different formats?
Ultimately, I see the distribution of future updates to an Igor Pro installer as being the distribution of updated XSL scripts (and corresponding XML schema) to run under a nearly immutable Igor Pro installer package rather than ongoing distributions of new Igor Pro installer packages.
Be all this as it may, having something that I can actually visualize at this point would help. Are you proposing something along the lines of having a minimal installer "script" ....
Package Name: SLIM<br />
Version: 1.0<br />
Igor Version: 6.10<br />
Platform: Both<br />
... along with a defined folder/file architecture ...
root (folder)<br />
-- AboutMe.txt<br />
-- InstallerScript.txt<br />
-- XOPs (folder)<br />
---- Windows (folder)<br />
------ Slim.xop<br />
---- Macintosh (folder)<br />
------ Slim.xop<br />
-- Procedures (folder)<br />
------ Slim.ipf<br />
... as all that would be needed by an "Igor Pro" installer? Note, I am separating the "installer" part of things from the "download it" part of things - I presume above that "root" folder is resident in an "install" directory on the user's local machine.
Actually, as I review Jim's comments, it seems the suggestion might even be to eliminate the need for any type of "installer script" -- the installer should parse all required information (igor version, procedure file version, etc) directly from the files in the root folder???
--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH
March 19, 2009 at 05:31 pm - Permalink
1) I would also like to see the creation of an Igor archive format. In my view this is a document that looks like a single file to the end user, but is actually an encoded (zipped?) folder that has a custom Igor extension. Inside this folder are the various files required for the package, such as xop, ipf, dll, or whatever other files are required. The core part of the folder is a 'script' file that lets Igor know what it should do with the files (copy them to some folder) and also includes metadata like a description, version number or language. This metadata should be required and should make the package identifiable to a reasonable degree of (but not absolute) confidence, and also allow verification of a its version.
I follow jjweimer's suggestion to use an xml format for the 'script' file. It's standardized yet flexible, and a lot of tools are available to process xml data. Let's not define another completely custom format.
The core idea is that the user can just doubleclick the archive and let Igor handle the installation automatically. Internally Igor should also create a database of all installed packages, making it easy for the user to see what files are installed by each package, activate or deactivate a given package, and uninstall a given package. This would mean adding a 'package viewer' or something as a menu item to Igor. The WaveMetrics procedures could also be made to use this. For security reasons Igor should only allow the installation process to alter files inside directories belonging to Igor, and even then make sure that it doesn't touch a set of 'core files' defined by WaveMetrics.
It's entirely possible of course that a given package requires another package to use, or that two packages share the same file names. Taking this into account, however, would require a full-blown package manager with an enforced centralized and authenticated repository. I believe that this is overkill for 99% of the (casual) Igor programmers out there, and therefore a full-blown package manager would be unmaintainable. The only exception may be a cryptographically-authenticated repository for WaveMetrics packages only.
2) In addition to installation a second issue is that of updates. I continuously update my packages, but distributing these updates to the users is hard. Therefore I propose that the packages are able to optionally specify a central repository. The Igor 'package viewer' would then automatically check these for updates, and offer the user the option to apply a given update. Note that this is analogous to a source control repository, not to a package manager, since it doesn't track external requirements but only cares about a single package! It would therefore be similar to a normal package file on disk, except that it automatically checks for updates.
I think that this central repository should be IgorExchange. However, a programmer should be given the choice to make the package public or private. Therefore I propose that each registered user of IgorExchange would be able to create private or public projects. Given a custom ID, Igor could then add this package to its 'package viewer' and install, update, or remove as required.
March 22, 2009 at 06:17 am - Permalink
You probably know this already, but IgorExchange does allow hosting of projects using Subversion. For more information on how to do this I'd recommend starting at http://www.igorexchange.com/documentation/subversion and working from there.
To be honest, I don't see providing private hosting in the future of IgorExchange. This would require us to have multiple Subversion repositories instead of the single repository currently used for all hosted projects. Administering these separate repositories, and the hook scripts that are required to get Subversion and the web content management system to work together, would be quite a task. From a more philosophical perspective, one of the goals of IgorExchange is to foster collaborative efforts on projects, similar to SourceForge and the other open source collaboration sites, and allowing private projects would hurt that goal. We don't require that all hosted projects be open source (for example project owners can upload only compiled XOPs without being required to also upload the source code), but I like the idea of everything being available to anyone. There also is the issue that the software currently running IgorExchange is not designed with the idea of private use. It is getting closer to supporting this, but not that close.
I do realize the utility of providing private access, and I realize that especially if this package manager/repository idea gets implemented it would be useful to have a centralized place for the Igor package manager to look for updates. I'm not aware of any services similar to IgorExchange that allow private access to projects and do not charge for this ability. If you know of some please post them here so I can take a look. One possibility is that IgorExchange could charge a fee for private access to offset the additional costs associated with implementing private access, but I'm not sure that's something we want to get into, and charging would bring its own set of hassles.
Just out of curiosity, what are your reasons for wanting private access? I can think of many, but wonder what yours are.
March 22, 2009 at 06:50 am - Permalink
I can understand that the private hosting would come with all kinds of technical complications. I don't think that you should sacrifice your free time for this, but I rather imagined it as becoming somewhat of a selling point for Igor: develop your package, deploy it, and keep it updated without effort! In that sense I imagined that the work on this would be picked up by WaveMetrics, since they could turn it into a feature.
I don't think that allowing private repositories would hurt the goal of collaboration. Every project is private until active effort is taken to make it public. You can't make projects 'more private'.
I'm sad to admit that they're pretty small-minded: I've put a lot of effort into developing a specialized analysis package that I consider to be pretty good compared to other solutions that I know of. But, having made the effort, I would also like the appropriate academic credit for it (i.e. paper co-authorship), especially as I'm not going to write a paper about the package itself (so there is nothing to cite directly).
I realize that this is completely against the spirit of openness advocated by IgorExchange. But for now I want to keep the software within the lab, though I might change my mind on this once some results become available. It's also specialized enough that I doubt that it is of general interest to anyone outside the research field.
The main code is in an XOP, and like you suggest I could just submit binaries. But the way I see it that's the same as not submitting the project at all.
Another option is doing the hosting myself. However, I don't want to deal with setting up subversion on an internet-accessible computer, and then having to train users on how to use it properly. So an ideal solution would be implemented entirely within Igor. If that implementation occurs through subversion then that's fine with me, but it doesn't even need to have source control. Just a directory with files and a version number is enough. Private hosting on IgorExchange (or WaveMetrics) would just make it a no-brainer to use.
As I said, I imagine this as being an Igor feature that WaveMetrics can use as a selling point.
– edit: typing
March 22, 2009 at 03:28 pm - Permalink
That's true to a point, but I suspect that for some people the services currently provided on IgorExchange convince them to publicly release their project when they otherwise wouldn't. If there was an equally easy (and free) private option they might have just gone with the private option instead.
Source code is always nice, but especially for XOPs the utility of the source code for many people is limited by their ability to understand and modify it. But in most circumstances just having the XOP is the most important thing. After all, the source code for only 1 or 2 of the official Igor XOPs is available, but I use many of them on a daily basis. Collaboration is not so easy if others can't modify the source code, but often something is better than nothing.
I understand what you mean. I have written software that everyone in my lab uses, and it can be challenging to keep everyone's copy up to date. My stuff is purely Igor procedure files, and I only require XOPs that ship with Igor, so it's a little different than your situation. I have a lab server that I also use to host a Subversion repository, and I have everybody get the software by using Subversion. For them to update, it's as simple as running "svn up" on their working copy. Getting them set up originally is a bit more difficult, but Subversion is not that complicated, especially if you're only checking out code and updating and not dealing with branches or tags.
If you want to use Subversion but don't want to go to the trouble of maintaining the server and repository, which can be a drag, I suggest checking out www.devguard.com. They provide very good service for a pretty good price. I use their service for several things unrelated to Igor (like, for example, source code control of my dissertation) and I've had no problems. It's definitely a lot easier to pay a few dollars per month than to set up apache and the svn bindings if you aren't already familiar with how to do that.
I will keep these ideas in mind as IgorExchange progresses. Work is being done on a next-generation system that, when finished, will allow a lot more flexibility than our current Subversion integration system has. Once that system is ready and running, it may be a lot more feasible to support additional options.
March 23, 2009 at 04:32 am - Permalink
I've been experimenting with package installation, with the eventual aim of constructing a package repository. I still feel that this would be best done inside IGOR, but this is a users version.
You need the easyHttp and XMLutils XOP's installed.
How it works.
The xml_package_installer.ipf file installs packages into ~/Wavemetrics/User Procedures/, based on the package structure contained in the XML manifest, motofit.xml. It uses easyHttp to download files from a URL. At the moment it just installs stuff into User Procedures, XOP's will happen soon (some issues with distributing XOP bundles on OSX). To get it to run use
packageXMLinstallerIt then saves a copy of the manifest in /User Procedures/packages, for future use.
To update the package you run
update_package. The code looks for existing manifests in /User Procedures/packages. It looks at a version number, and a URL. It attempts to download a new manifest from the URL, and if the version number is greater than the existing version it will update the new package.It also has scope to install dependencies, but that's not implemented yet.
The eventual idea is that there will be a webpage somewhere, which will host all these package manifests.
I don't know how to do uninstallation though, as IGOR won't let you delete procedure files and XOP's that are in use.
April 22, 2010 at 06:45 pm - Permalink
More power to you. I just started using a professional installer-maker for NIDAQ Tools MX. The things that finally pushed me over the edge were the issues involved with installing help files. Since you can't have Igor running while help files are changed, it's really tough to update help files. On Vista and Windows 7, permissions issues are likely to kill you if you need to do anything beyond the usual (I put NIDAQ Tools MX procedure files into the WaveMetrics Procedures folder, something users are *not* encouraged to do).
The professional installer does a number of nice things:
Handles the differences between different versions of Windows with respect the proper path to Program FIles (and Program Files x86, etc.).
Handles permissions checking before installing.
Allows symbolic access to user-accessible folders so that you don't have to find it yourself.
Puts a shortcut in the appropriate Start menu place if you want it.
Accesses the registry to find keys that tell me if Igor is installed, and if so what version. Also to check of the NI-DAQmx installer (a National Instruments product) is installed, and if it is a recent enough version.
Now, a caveat- thanks to NI's intransigence with respect to Macintosh drivers, I don't have to install that product on Macintosh. The Tarma installer maker is just $89 and when I e-mail for support it feels a lot like WaveMetrics:
http://www.tarma.com/
Now, I admit that there are lots of less heavy-duty applications that would be easier to handle. But you still need to handle the help file issue.
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
April 23, 2010 at 04:49 pm - Permalink
I currently use the NULLSOFT installer, and that does all the sorts of things that you describe, although I find it quite hard to work with. The main problem I've been having is how to find the user files, i.e. ~/My Documents/Wavemetrics/..... There is quite a lot of logic involved.
For OS X it's a different issue. PackageMaker is fine, but you only get the choice to install in one place, instead of user locatable folders.
One beef is that I have plenty of pre 6.1 users, so ~/My Documents/Wavemetrics/ doesn't necessarily exist.
I spend most of my support time for these packages helping people with installation issues. e.g. installer doesn't work, they have multiple copies of files, outdated IGOR versions, etc.
In the best of all possible worlds IGOR would have a repository system, and that's what I'm trying to emulate with an 'in IGOR approach'. Package installation should be made so much easier. At the moment users each have their own way of distributing and installing things, which leads to fragmentation and duplication of effort.
But as you point out XOPS/procedures/help files can't be deleted/moved,etc, if IGOR is running.
April 24, 2010 at 12:56 am - Permalink
You mention that you store information in the folder /Wavemetrics .... User Procedures. Would any better benefits arise at this point to justify starting the convention for a new Igor Pro "Special Folder" (example shown here based on the MacOS path syntax)?
Igor Pro 6 User Files
Igor Extensions
Igor Help Files
...
Igor Pro 6 User Settings
Repository Files
Preference Files
...
--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAHuntsville
April 24, 2010 at 01:47 pm - Permalink
I agree that Igor needs an installer and a repository would be very nice. Hopefully some day we will support this in Igor.
Because of the problem of handling open files and the fact that you can't readily change procedures while procedures are running, I think a user-defined 'in IGOR approach' is going to be frustrating for the programmer and the end-user and I don't recommend it. I would either rely on the user being able to follow instructions or on an external installer that runs when Igor is not running.
April 24, 2010 at 07:51 pm - Permalink
Yeah, I looked at the NULLSOFT installer, which has the advantage of being free. But I couldn't figure out how to use it from their appallingly bad documentation. The $89 for Tarma was worth the time saved by not having to figure out a complicated system from disassembling their code!
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
April 26, 2010 at 10:35 am - Permalink
I like this, and lets put a place for Packages there, too. The fact that the SpecialDirPath for Packages is in a hidden folder is a bit inconvenient when I try to explain to people how they can copy package settings from e.g. one user to another.
April 26, 2010 at 08:38 pm - Permalink
April 26, 2010 at 08:43 pm - Permalink
Then they say I have IGOR < 6.1, which means that ~/Documents/Wavemetrics doesn't exist (they have to upgrade).
THen you have windows, which doubles the effort.
Given these issues, installing tortoiseSVN is beyond the capability of the average user. If they can't get it working in a minute then your chance to get them using your program disappears.
April 26, 2010 at 11:12 pm - Permalink
Subversion on the MacOS side is not as well developed as TortoiseSVN (although progress is being made), and I must agree anyway with Andy that installing an SVN tool would be beyond the basics for all but the more advanced and computer literate users.
I would like to see an installer that is standard on both platforms. Unfortunately, my understanding of installation tools is near to zero. From what I see just as I bounce between Windows and Mac systems, very few applications follow a "standard install" path, and installations vary even from one application to the next on the same platform. So, with Igor packages, I basically say ... put this there ... and hope for the best.
The one exception that I have recently run across is the Adobe AIR platform (I am testing out a budget software application that runs on it). The framework is supposedly platform agnostic. I wonder whether this would be reasonable for Igor.
--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAHuntsville
April 27, 2010 at 08:37 am - Permalink
OK, here are a few brainstorming ideas.
Basically, if I understand what you are saying, for package installation to occur properly while Igor is running requires that XOPs, help files, procedures files, or in fact almost anything, should not be loaded or opened or running. The "Igor-as-an-Installer" case should just start be equivalent to having an "instance" of the Igor Pro application running on the given platform.
What then would be the case if Igor had a new pragma ... #pragma InstallerMode=1 ... that will do all that is needed to put Igor in such a "ready-to-install a package" mode? Can Igor be forced to open only in such a state in such a way? Or are the pragma directives of a procedure window in an experiment read AFTER the XOPs, help files, and so forth have already been loaded? If having such a pragma method is not possible, could a stripped down version of Igor be made that is designed as the "Igor Installer Core" and that is callable ONLY from a User Procedure with a pragma InstallerMode=1 directive at its head (in other words, when a user opens an "Installer Package" procedure file, it forces the IgorInstallerCore application to run rather than the Igor Pro application)?
This approach would be akin to how develop of LaTeX "installers" are handled ... the LaTeX kernel provides for a platform agnostic way to initiate a core installer to unpackage install files, and developers rely on this to distribute their packages.
I think this approach could open the door for us (the users) to develop a consistent, reliable "in-Igor" package installer approach, freeing up your (WaveMetrics) efforts to continue on the application itself.
--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAHuntsville
April 27, 2010 at 09:15 am - Permalink
Sorry ... what do you mean by "Packages"? I am perhaps confused between this statement and a "Package" that is a set of XOPs, procedure files and so forth. The latter are already installed in the Igor Pro 6 User Files folders.
--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAHuntsville
April 27, 2010 at 09:04 am - Permalink
Sorry, I meant specifically the directory returned by SpecialDirPath("Packages",0,0,0), which is the place where we are supposed to put things like user settings. I use this folder for storing structures (and whole .pxp files) containing settings information using SavePackagePreferences and LoadPackagePreferences. I guess I could utilize the /P flag in those operations and just store such "packages" in the Igor Pro User Files folder given by SpecialDirPath("Igor Pro User Files",0,0,0), which is where we are now storing procedure files and extensions. It seems that the use of a user-specific packages directory predates Igor 6, but with the advent of Igor 6.1, perhaps every user-specific thing should be consolidated in the "Igor Pro User Files" location.
Rick
May 3, 2010 at 05:10 pm - Permalink
It is possible to programatically close all procedure files (except the one controlling the installing) by iterating over WinList("*",";","WIN:128,INDEPENDENTMODULE:1") with CloseProc, followed by a Silent 101. So that leaves XOPs. There is no documented way to list loaded XOPs, and the undocumented but useful in a pinch KillXOP probably isn't stable.
Maybe an alternative to a #pragma would be command line flag (let's say /THIN for Igor.exe (and the equivalent on Mac) to start Igor with no extensions and no procedure files. The installer could then be a .pxp file, let's say install.pxp, that the user opens, which then notifies him via an AfterExperimentOpenHook about what will be installed, and whether or not to proceed. If they click OK, "Igor.exe /THIN/I/S install.pxp" is executed via ExecuteScriptText, which causes the current version of Igor to close and a new version to open the same .pxp file, only this time with no external procedure files or XOPs loaded. After verifying that this is so, the AfterExperimentOpenHook proceeds to call the installation routines.
Rick
May 3, 2010 at 05:30 pm - Permalink
I knew there was something like Closeproc, I just couldn't see it in the help file. Lots of potential for things going wrong with that operation. killXOP is nice and nasty. It does seem to unload the DLL but it doesn't remove it from an internal list. If you call a function from the unloaded XOP IGOR crashes immediately and I can't say I'm surprised. Using killXOP would be fraught with danger, as you never know what people are doing inside them.
Andy
May 3, 2010 at 06:48 pm - Permalink
I found that there is a loadXOP operation but it didn't work.
May 3, 2010 at 09:48 pm - Permalink
John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
May 5, 2010 at 10:01 am - Permalink
Use FetchURL to get the recent SVN commit number of an IgorExchange package (quickly cobbled together example below returns SVN for AnnotationTools) ...
Function getit()
String webPageText = FetchURL("http://www.igorexchange.com/project/cvs/1575")
string theText
if (numtype(strlen(webPageText)) == 2)
Print "There was an error while downloading the web page."
endif
Variable start, stop
start = strsearch(webPageText, "SVN messages for", 0)
start = strsearch(webPageText,"Commit ",start)
stop = strsearch(webPageText,"by",start)
theText = webPageText[start,stop-2]
SplitString/E=gExp theText
print S_value
return 0
End
Compare this with $Revision: $ tag information in SVN filed procedure. When newer, assume commit at IgorExchange is an update and flag user.
Within PackageTools, I've now run a function immediately after a new package is loaded into an experiment (using the recent AfterCompiledHook function -- thanks WaveMetrics!!!!). So, when a user installs a package, that function code can do the FetchURL check automatically (or not -- preference setting might be nice).
Comments?
--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAHuntsville
December 3, 2010 at 05:59 pm - Permalink
CloseHelp, so together with the killxop magic and the not anymore locked procedure files, it should now be possible to build an completely igor based installer. I'm no finished with mine though.April 1, 2016 at 12:21 pm - Permalink