Hook after loading pxp but before compiling main procedure

Advanced users might be asked to provide a sanitized version of their experiment without the #includes.

For less experienced users, I wonder how the #include statements arrive in the main procedure. My users have two tools that create includes: I provide a 'default includes' file which has a selection of #include statement that mostly mirror the includes in my own default includes procedure file. Those 'default' includes therefore don't appear in the main procedure. Also in the default includes procedure file I provide a lot of menu items for users to include other code from my collection - these follow the same format as the menu items for WM packages found in the WMMenus procedure file:

My packages appear 'below the line' following the WM packages. After they have been included they usually add their own menu items to the higher level menu. Notice that I use #include, optional, so even though include statements are added to the main procedure window they won't prevent compilation when the file is missing.

The other tool that creates includes for my users is the procedure loader project, which creates 'normal' include statements in the main procedure window, rather than optional includes. For me this is not usually a problem because users are typically using procedure files that I have provided as a package and that are therefore present in my own User Procedures folder. It would be easy to provide an option in the procedure loader to create optional includes instead of normal ones.

A third way that my users might end up with an include that they haven't typed themselves is if they use the package installer included in the updater project to install a project, since the installer offers to include a project after successful installation. A determined user could end up including a bunch of procedure files that way.

I have to wonder though how your colleagues end up with a list of include statements in the main procedure window.

tony wrote:

Advanced users might be asked to provide a sanitized version of their experiment without the #includes.

Or to create a copy of the experiment will all files adopted:

DisplayHelpTopic "Adopt All"

I get data files from experiments which uses Igor to control said experiments. Unfortunately, neither are these control procedures properly set up to prevent this from happening nor have the users enough knowledge and/or patience to deal with this. I am in contact with the people in charge, but it has been slow to get through at best. Yes, this would be a complete non-issue from the start if procedures were properly placed to avoid the need of any include statements in the first place. In any case, there are also quite a lot of data files which already have been saved in this 'broken' state. I hoped for a way to ease the problem a bit by providing an automated solution for 'fixing' these problems on the fly. But short of asking everybody to have a bunch of dummy procedure files on their PCs I came up empty so far.

I think that technical note PTN003 should have enough information about the file structure to allow you to excise the procedure record from a packed experiment before loading. But, unless you are dealing with a huge number of experiments, it's probably not worth the time investment to figure it out, and in any case it's only a solution if you're willing to preprocess the files.

Here's a dirty hack (which may have unintended consequences):

load pxp into string, find and replace #include with //nclude (same number of bytes), write back to file and load.

Interesting idea to change the line without changing the no. of byte, which can be pulled off with a text editor in the worst case. While loading the file into a string might be difficult with 2 GB files, it might be pulled off with some cleanup script in c or something.

If you turn off auto compile, you can open the main procedure window, comment out or remove the offending lines, and recompile. Yes, each time. But perhaps this is also where a scripting language such AppleScript or the equivalent on Windows may help.

My only other thought depends on your desire to continue allowing your patience to be worn out by the limits in knowledge from your user base. Consider creating a Clean Up for Distribution procedure. The procedure would have a menu "Save for Distribution". The function run by this menu asks for a new file name, sanitizes the main procedure by removing all #include statements (except ones that you accept), and creates the cleaned up pxp. Distribute the Clean Up for Distribution procedure file to your team. Insist that it be installed in the Igor Procedures folder so that the "Save for Distribution" menu option is always resident (e.g. as a Macro option). After you distribute the tool, eventually stop accepting experiments from your team unless they are Cleaned Up for Distribution.

Some time in the near future, I might even eventually need to do the same thing to avoid getting the equivalent of "dirty" experiments from my graduate students.

JJ, thanks for your comment. Yes, I would certainly do that if I was in control of the work flow. Or rather I would never work with include statements in such an environment. I wonder, is there actually a flag which can disable autocompile upon stating Igor? But I assume this then covers all procedures, including independent modules. I guess the easiest solution is to push forward to get the source fixed, with the second best option to have an external script or manual intervention fixing the file before their use in Igor (such as with the tip from Tony). Still, it would be nice to have a way to hook into the starting process to fix all kinds of issues with the main procedure if the need arises (another such case would be local function names clashing with loaded procedures or function calls to old, non-existing procedures).

edited to comment out the offending lines without replacing text

chozo wrote:

 I would never work with include statements in such an environment.

IMO the better solution is to avoid having the pxp as the only way to store the data. From an end user perspective having to export a data file from the control software (or having data files written during data collection) makes sense.

Tony, thanks a lot of this ready-made solution. As expected for you, works flawlessly. It only takes about 5 seconds to work through an one GB file and is way less tedious. I will use it and adapt this to a more targeted script which deletes only includes of unavailable files. Thanks again.

Our approach in tech support is simply to ask people to send us an experiment with files adopted. You tend to get a lot of irrelevant code, but at least it compiles. Hold down shift, pull down File and select Adopt All...

Thanks John, yes this would be another good option. Unfortunately, I have absolutely no vote in how the data is dished to me. Tony's approach works quite well for me, though. So I would consider this case closed. Of course, I would be more than happy if Igor would get another hook function as proposed above. There can not be too many hooks in all kinds of places to clean up after users in the background. ;)

@Tony: I though it would be neat if above code was part of your Updater project. There you have already all the functionality in place to scan for procedures. So, you could also easily offer to clean up experiments with invalid include statements upon request. By the way, I added this piece of code as well, after replacing invalid "#include" with "//delete", which removes any traces from the procedure window:

Hmm, an include for a non-existent file is just one of many potential ways that the code in a procedure file could fail to work for an end-user. A more likely scenario is a missing #include for a WM procedure; that's a mistake I have made more than once in files that I've distributed. It's easy to forget to add the include if the file is always open in the copy of Igor used for development! Ultimately, though, it's not the job of an installer to correct code. It would be nice, however, if the PrepareProjectRelease() code were smart enough to detect and flag potential include problems, either files missing from the distribution or missing include statements. I'll have to think about how to do that. It looks like the code already checks includes for user files and tries to make sure the files are included in the package. Maybe I should also prepare a list of all functions in currently included WM procedures and check to see whether any of these have been used in the files to be distributed without the needed #include.

Note that I revised the code in my earlier post to comment-out rather than replace text.

Tony, OK I see. The revised code is nice as well and less intrusive. I personally use the previous version because I am fine with eradicating all the invalid statements. Interesting idea to check for missing includes. While I personally don't use WM code in my projects at all, I guess this would be an useful addition. This sounds not too difficult to do, if I am not wrong. If the project complied on the PC, you basically would just need to accumulate a list of all open procedures using WinList(), then for each of these check FunctionList() against the contents of the project and, if matches are found, check if there is an include statement for the procedure in question somewhere.

chozo wrote:

I personally don't use WM code in my projects at all

Then you have not yet discovered the delights of Resize Controls :)

I wrote the code to do the check for missing includes for WM procedures, tested it on a random procedure, and sure enough I had forgotten to add the line

#include <Resize Controls>

Thanks, I will keep it in mind if I ever build a more complicated panel which also needs to change in size. :) Do you have any other WM tools which you use frequently? I guess WaveSelector is another candidate. I might upgrade my tools to this wave selection method at some point.

Interesting discussions, especially as I anticipate facing comparable problems in trouble-shooting issues that arise after I distribute targeted analysis packages to users who are less savvy with Igor Pro. I'll direct follow ups elsewhere.

chozo wrote:

any other WM tools which you use frequently?

Mainly Resize Controls and WaveSelectorWidget. Some of my older code uses Axis Utilities and ReadbackModifyString.