Calling Python Code from Igor Pro

We've considered this, but there hasn't been a lot of demand for it and it would require a lot of work to implement.

"Matlab has such functionality" is true, but I believe that Mathworks also has thousands of employees. You can count the number of WM employees on two hands, even if you are missing some fingers :)

I could use that functionality as well.

This may seem a bit like using a grenade to kill a fly, but couldn't one compile a python interpreter into an Igor Pro XOP using open source code from the latest python distribution?

Embedding python is what this 2010 topic was about: https://www.wavemetrics.com/forum/general/embedded-python

It would seem that it might be possible to just call a standard python distro via ExecuteScriptText using files for passing data back a forth. You would need to write some code on both the Igor and python side to deal with the data.

I would be helpful to know what kind of processing and data you need via python.

Be aware that, according to https://developer.apple.com/documentation/xcode_release_notes/xcode_11_beta_release_notes?language=objc:

• Scripting language runtimes such as Python, Ruby, and Perl are included in macOS for compatibility with legacy software. In future versions of macOS, scripting language runtimes won’t be available by default, and may require you to install an additional package. If your software depends on scripting languages, it’s recommended that you bundle the runtime within the app. (49764202)

• Use of Python 2.7 isn’t recommended. This version is included in macOS for compatibility with legacy software. Future versions of macOS won’t include Python 2.7. Instead, it’s recommended that you run python3 in Terminal. (51097165)

I imagine that those folks on macOS who want Python are not wanting the macOS standard 2.7 Python libraries anyway. So, it is a good call from Apple to just dump this in favor of letting folks pip or homebrew their "third party" installations.

10.15 ships with python3 also.

I would say anyone who is like really using python is using a custom installation. The one shipping with the OS/distro is usually outdated and incomplete.

thomas_braun wrote:

I would say anyone who is like really using python is using a custom installation. The one shipping with the OS/distro is usually outdated and incomplete.

Isn't that an argument for NOT linking in a standard install of python? 

Also, speaking as a python novice, it seem that python seems to make a special effort to break programs developed under previous versions. 

[My only experience with python was in translating a Neural Net tutorial into Igor. A useful project where I made some changes to Igor syntax and programming in order to match speed of learning the NIST handwriting examples.]

Larry Hutchinson wrote:

Isn't that an argument for NOT linking in a standard install of python? 

Also, speaking as a python novice, it seem that python seems to make a special effort to break programs developed under previous versions. 

I would say that folks who currently use python *in a serious way* do not link to a standard installation, at least under macOS. And, perhaps being tongue-in-cheek or factious or cynical ... Python is totally free. You only need to invest your time to fix what was broken during the recent update.

Full Disclosure: I started about a year ago at python3 and I only dabble at the edges of it using JupyterLabs to make interactive demos. I have no dog in the fight as to whether python should or should be linked with Igor Pro.

I asked this before with no response so again: What kind of processing and data transfer do you want to do with Python?

I ask this because I am looking into using the pickle format to exchange data between Igor and Python and I'd like to know if the effort would be worthwhile.

The pickle format is not designed to be used by programs other than Python but a subset could be read and written using Igor code.

I've been able to read an array of images from a sample file (mnist.pkl from http://neuralnetworksanddeeplearning.com/chap1.html) but only after manually inspecting the output of pickletools.dis(). Making this automatic and general is the hard part.
 

@Larry: I'm not very fond of pickled data. What would you do with that in igor? I would say usually it is about array data and that is much easier exported from python via numpy.save, https://www.numpy.org/devdocs/reference/generated/numpy.lib.format.html. So support for reading/writing that is in my eyes more worthwhile than partial pickle support. Of course you could save in hdf5 from python and just load it into igor as well.

What I would like to see in IP is the ability to call python code from within Igor and exchange data with it. https://github.com/pybind/pybind11 is used by a project I know for writing python code on top of C++. Of course that is easier suggested than done.

I was just thinking the pickle format was easy for python users. The one file I have contains multiple numpy arrays.

Any reason you cant just call python using ExecuteScriptText with disk files to pass data?

If all you want to read numpy arrays from file then I wouldn't use pickles, the format spec is available: https://www.numpy.org/devdocs/reference/generated/numpy.lib.format.html.

Reading Pickle files would be a fragile way to go.

To call Python from C/C++: https://docs.python.org/3.7/extending/embedding.html

Otherwise it could be possible via ExecuteScriptText, but you have to know the correct path to the Python command.

Otherwise it could be done via sockets, by making a Python webapp.

I agree with Larry about pkl in terms of usability and familiarity to python users, although something like feather or HDF5 (which I think I have seen in Igor) could work too -- I believe Andy is correct from a 'python that lasts' perspective though. pkl can behave strangely depending on how the user is working with python. Yes, this contradicts my earlier statement about usability, but my experience with pkl is that it is easy until it is not, at which point you actually need to understand how python works at a level many people do not.

I would love to be able to [call Igor]/[pass data to an awaiting Igor] just for the sake of plotting moderately sized datasets with snappy interactivity (ie. 1000s of FTIR spectra with ~4000 variables each). This is still something that is difficult to achieve without . . . writing the plotting code in C or C++. To me, just the snappy plots are well worth the cost of Igor Pro, and I am just outputting my data to a .csv and loading it into Igor with the GUI. If I could call Igor for plotting from Python, with the ability to specify some characteristics of my desired plot, that would incredible.

kelleyjbrady wrote:

If I could call Igor for plotting from Python, with the ability to specify some characteristics of my desired plot, that would incredible.

If you simply want to use Igor as a plotting engine from python, some combination of subprocess on the python side, igor scripting (displayhelptopic "Calling Igor from Scripts"), and perhaps an itx file as a container for data and commands might get you there. Not sure that it would work well for the volume of data you're dealing with, though.