Hyperspectral Browser

OK. This looks so cool that I want to run out and collect hyperspectra just to use it!!!

As you may have surmised, my three recent project releases were motivated by working with infrared hyperspectra generated by a focal plane array detector. I hope though that parts of this work will be more generally useful for Igor Pro users who work with spatially-resolved spectroscopic data. In version 1.01 I've added a demo function for testing the interface: execute hyperspec#demo() from the command line to generate some fake data.

That looks very nice. I wish this project had been around when I was still working with hyperspectral data. The selection of regions is very elegant.

Perhaps it would also be good if the exported spectra could retain the original wave and range as a note. Otherwise there is a risk that this might be forgotten.

And is there a reason why the exported wave is Single Float 32 bit (and not Double Float 64 bit)? Information may be lost here.

These issues are addressed in version 1.03. The precision of exported waves is now the same as that of the input data. When an output wave represents a pixel selected with the crosshair cursor or a rectangular region selected with the marquee, the range is saved in the wave's note. If you select a collection of pixels, draw a ROI using the UI, or use drawing layer shapes to select a region of interest, the output wave's note contains the name of the input wave + 'masked region'.

If you downloaded this project prior to version 1.30 to test under Windows, I suggest trying version 1.30 or later, in which Windows compatibility is significantly improved.

This continues to be one of the most eye-catching projects in Igor. Amazing!

Hi Tony, very nice. I finally got around playing with this. While I probably have not (yet) any use for something like this, it is always interesting to see your approach. A few things I have noticed:

- The plots have issues when the data has units in all three dimensions. You might want to override units in the plots or otherwise incorporate them into the label (\u#2 or \U, respectively)

- Setting a marquee / ROI or other live actions can take very long if the data is 'big' (and I am talking only about data which is about 50 MB). This makes the tool unresponsive (in fact, it took up to several minutes in my case to generate the quick-roi). You might want to include a progress bar and abort button. I did not investigate what takes so long, but there might be room for optimization.

- Automatically set a minimum and maximum >0 if log colors are selected (to be fair, I also do not have this in my tools)?

- The package data is not deleted upon closing the tool? Is this intended to work with the data further? Otherwise it creates quite some overhead.

- Consider having the possibility for more than one browser open at the same time.

- Also, choosing another data set from the top bar resets the window position in a strange way. I guess you forgot to incorporate the proper scaling factors for the Windows version (I feel really sorry for Mac developers, but I have to deal with this crazy stuff as well ;-).

Hi Chozo,

Thanks for the useful feedback. I'll reply to each in turn:

You might want to override units in the plots

That was indeed the easiest solution.

Setting a marquee / ROI or other live actions can take very long if the data is 'big'

I've been working with datasets of around 30 MB, wave dimensions are 64 x 64 x ~2000, 32 bit. If the option to plot all of the pixel traces in the ROI is selected, updates are a bit slow for me - typically the code runs in less than 0.5s, but the graph update takes much longer. I'm not sure if there's anything I can do to improve efficiency here. If you choose not to plot every trace and just display the computed average the response should be better. What are the dimensions of the data that caused the long delays? I would like to investigate further.

Automatically set a minimum and maximum >0 if log colors are selected

Well, the auto min and max refer to the data limits, so I don't want to cause confusion by fudging the values. It would be great if selecting log would generate logarithmically spaced colours even when the limits are negative... the axis for color scale would need to be redesigned, though. The current behaviour of giving up when data are negative is OK for me. The user can easily adjust the limit to a non-negative value.

The package data is not deleted upon closing the tool

Fixed in next release

choosing another data set from the top bar resets the window position in a strange way

Yeah, to get the size of the existing panel I used GetWindow wsizeRM which, according to the manual, returns points, even for a panel. Seems that isn't the case. Hopefully fixed for the next release.

Missed one!

Consider having the possibility for more than one browser open at the same time

My intention was to create a tool for efficiently exploring 3D datasets, but not to produce polished graphics or do a lot of data reduction, hence 'browser' in the name. I don't expect to relax that restriction in the near future.

I've been working with datasets of around 30 MB, wave dimensions are 64 x 64 x ~2000, 32 bit. If the option to plot all of the pixel traces in the ROI is selected, updates are a bit slow for me - typically the code runs in less than 0.5s, but the graph update takes much longer.

OK, now I see what is going on. I was testing this with one of my data sets which was 500 x 1000 x 30 points. Naturally, choosing even a tiny ROI will result in an enormous amount of traces. So the solution would be to limit the number of traces generated for the single-pixel option. Alternatively, you could make this feature scalable into "pseudo-pixels", i.e., groups of pixels whose size scales with the x and y dimensions. Anyway, you don't need to adapt to my type of data. But I pretty much have to kill the Igor task if I, even accidentally, drag with the mouse while using your tool. :-) The cheapest option would be to outright disable the single-pixel option if the data grows beyond, e.g., 5000 points in a single layer.

I think I found a reasonable solution for version 1.63. Let me know if this makes working with your large datasets less painful.

Tony, thanks. This works great. Another small thing I found: In line mode the profile gets 'spikey' when the profile line has a certain direction across my data. I found that 'distanceCTline' can have stepped structures which change in size while dragging the line across the data causing this. I don't know if this is an artifact or just a result of the large layer I am working with. See the attachment.

When you use the cursors to define a line, the code computes a 'Bresenham' line between the end points. The profile should then have one point for every pixel along the line, and the distances and values for the profile should match the data, unless I have made some mistake. Using the dialog to save an ROI you can see which pixels are selected on the screen, or you can export the profile to a 2D wave and investigate the coordinates and values. The distances are calculated from pixel coordinates, so they do not increase with equal step size.

If you instead draw a linear path ("Draw ROI Linear"), the profile is calculated using ImageLineProfile, which will give different results because ImageLineProfile does some interpolation.

FYI, if you draw a curved path, the profile is calculated using ImageLineProfile based on xy coordinates that divide the path between each node into 50 sets of coordinates. I could expose that parameter and make it user-adjustable.

OK, thank you for the explanation. I did not look too deeply into it. This might simply be an artifact from sweeping over many pixels (>800) in one direction vs. very few in the other. Draw ROI Linear works as expected.

If you place the mouse cursor over a spike in the profile, the corresponding pixel in the map should be marked by a cross cursor. If the map is zoomed in you may be able to tell if the spikes are real noise in the data or if something is wrong with the profile.

In 1.65 response times should be further improved for large datasets.