Four: Rug Plot, Ternary Plot, Polar Plot, Kernel Density Estimator

I mentioned two of these in private e-mails during debugging of v7 support, but thought for sorta completeness I'd mention here.

1) Rug plots. Very useful if you have a lot of data, especially binned data. There are work-arounds to do this in Igor, but they're not that easy and a native rug plot option would be nice.

2) Ternary plots or diagram. Perhaps niche, but these are very useful when you have three axes of something, especially when the sum of all adds to 100%. Often used in mineralogy.

3) Polar plot. I know there's some script written (I think in one of the example files?), but again, an actual *native* option to do this would be nice.

4) Kernel density estimator. I've recently started to use these in 1D for some work where we're trying to revise how stats are treated in my field, and while I've coded it fine myself, a native option with a variety of functions would be nice (and goes hand-in-hand with the rug plot). I also need to do this in 2D, which I have not yet started to code because 2D Gaussians give me a headache, so again, a native support option for this would be nice.

As a fifth "would be cool, you're unlikely to ever do it" option that someone else mentioned I think in the general help forum, the ability to make an Igor RTE file with code that *other* people could run who don't have Igor would be awesome. As in, I could do all my development and debugging, and then produce a package that I could deliver to colleagues and they could just import their data and get the groups out they need based on my code. IDL (shudder) does this, and there's a tool based on it that everyone in my field who doesn't know what they're doing uses. I don't use it because it's WRONG, but because of the incredibly ease of the blackbox and IDL letting you run just that package as an RTE without a license has made it propagate like crazy.
astrostu wrote:
... the ability to make an Igor RTE file with code that *other* people could run who don't have Igor would be awesome.


I can appreciate the interest to be able to develop Igor Pro packages that are "stand alone". I can also imaging that Igor Pro is a rather complicated beast to slice out to make just an RTE from it.

astrostu wrote:
I don't use it because it's WRONG, but because of the incredibly ease of the blackbox and IDL letting you run just that package as an RTE without a license has made it propagate like crazy.


So ... What would happen if you would make a comparison demo for IDL versus Igor Pro to cement home your statement? In the demo, run something in the IDL package. Run the same thing in Igor Pro. Show, side by side where the IDL code is wrong (and gives ugly results) compared to the Igor Pro package. Post what you have created as a YouTube video, a LinkedIn announcement, a ResearchGate question, an OpenSource publication, and so forth.

The return investment could be that folks will start asking you (and paying you) to collaborate with them. Or, they will buy Igor Pro and ask you how to get your package.

As to the high propagation factor of free apps, that will not be controlled for or fought against by having an RTE for Igor Pro. Incredibly sloppy and horribly wrong will still happen regardless (which is the polite version of sh** happens in this case).

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH
astrostu wrote:

2) Ternary plots or diagram. Perhaps niche, but these are very useful when you have three axes of something, especially when the sum of all adds to 100%. Often used in mineralogy.


Not native but still works well: Windows->New->Packages->Ternary Diagram
astrostu wrote:

4) Kernel density estimator. I've recently started to use these in 1D for some work where we're trying to revise how stats are treated in my field, and while I've coded it fine myself, a native option with a variety of functions would be nice (and goes hand-in-hand with the rug plot). I also need to do this in 2D, which I have not yet started to code because 2D Gaussians give me a headache, so again, a native support option for this would be nice.


Since you mentioned V7 you must be familiar with StatsKDE. If that is the case, I'd really appreciate a more specific feedback.

A.G.
WaveMetrics, Inc.
jjweimer wrote:
astrostu wrote:
I don't use it because it's WRONG, but because of the incredibly ease of the blackbox and IDL letting you run just that package as an RTE without a license has made it propagate like crazy.


So ... What would happen if you would make a comparison demo for IDL versus Igor Pro to cement home your statement? In the demo, run something in the IDL package. Run the same thing in Igor Pro. Show, side by side where the IDL code is wrong (and gives ugly results) compared to the Igor Pro package. Post what you have created as a YouTube video, a LinkedIn announcement, a ResearchGate question, an OpenSource publication, and so forth.

The return investment could be that folks will start asking you (and paying you) to collaborate with them. Or, they will buy Igor Pro and ask you how to get your package.

As to the high propagation factor of free apps, that will not be controlled for or fought against by having an RTE for Igor Pro. Incredibly sloppy and horribly wrong will still happen regardless (which is the polite version of sh** happens in this case).

There are several of us in the community who have pointed out that the standard in the field does stuff wrong. Problem is, my sub-sub-sub field is like a hammer. It's usually thought of as a tool for more interesting things. So people think they can do more interesting things and "obviously" pick up a hammer and know how to use it, despite this being false, but it gives them something that they think looks right.

My long-term plans are a Python-driven back-end to a web interface that'll do it right.
ChrLie wrote:
astrostu wrote:

2) Ternary plots or diagram. Perhaps niche, but these are very useful when you have three axes of something, especially when the sum of all adds to 100%. Often used in mineralogy.


Not native but still works well: Windows->New->Packages->Ternary Diagram

Yup, hoping for something native. I actually didn't know about that one, but looks like there are some issues, like can't resize.
Igor wrote:
astrostu wrote:

4) Kernel density estimator. I've recently started to use these in 1D for some work where we're trying to revise how stats are treated in my field, and while I've coded it fine myself, a native option with a variety of functions would be nice (and goes hand-in-hand with the rug plot). I also need to do this in 2D, which I have not yet started to code because 2D Gaussians give me a headache, so again, a native support option for this would be nice.

Since you mentioned V7 you must be familiar with StatsKDE. If that is the case, I'd really appreciate a more specific feedback.

I started on this before I knew about this feature. Which was a few minutes ago. Looks nice! You can cross that off my wish list ...

... though unfortunately, my specific needs require a customized implementation that I have to code myself at the moment. In particular, the bandwidth for *each* value has to be different ... I know that's not a standard KDE thing, but my data is crater populations on solar system bodies which follows generally a power-law distribution, and the smoothing parameter (bandwidth) has to be set by the individual crater's size rather than a global thing for each feature in the population. E.g., it makes sense that the uncertainty in the diameter of a 1 km crater is ±0.1 km, but then doesn't make sense that's the same uncertainty on a 100 km crater. So I wrote code that lets me not only set the destination wave spacing (via a diameter (x) wave I create) to be even on a log-plot, but also such that the bandwidth of each crater is 10% (or whatever%) of the diameter going in, as opposed to a fixed value.
astrostu wrote:
Yup, hoping for something native. I actually didn't know about that one, but looks like there are some issues, like can't resize.

Can't resize what? The graph window certainly can be resized. It's just a graph with transformed data and fake axes drawn using the graph's drawing tools.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
astrostu wrote:

... though unfortunately, my specific needs require a customized implementation that I have to code myself at the moment. In particular, the bandwidth for *each* value has to be different ... I know that's not a standard KDE thing, but my data is crater populations on solar system bodies which follows generally a power-law distribution, and the smoothing parameter (bandwidth) has to be set by the individual crater's size rather than a global thing for each feature in the population. E.g., it makes sense that the uncertainty in the diameter of a 1 km crater is ±0.1 km, but then doesn't make sense that's the same uncertainty on a 100 km crater. So I wrote code that lets me not only set the destination wave spacing (via a diameter (x) wave I create) to be even on a log-plot, but also such that the bandwidth of each crater is 10% (or whatever%) of the diameter going in, as opposed to a fixed value.


You might also want to check out the implementation (http://www.igorexchange.com/node/4749) which is reasonably efficient.

A.G.

astrostu wrote:
My long-term plans are a Python-driven back-end to a web interface that'll do it right.


I know of one place that was doing (or planning) a Web interface as a front-end to Igor Pro (running on a server). I _think_ others lurking here have developed this approach too. I _think_ all it requires is the right attention to foreground / background operations, a well-connected server running Igor Pro, the interface code to collect the data input and dump the data in to Igor Pro and spit the results back, and of course the Igor Pro package to do the processing.

IOW, when you have the IP code, rather than redo that entire code in Python, you may benefit more to find someone to help you put it up as a Web-server instead.

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH
johnweeks wrote:
astrostu wrote:
Yup, hoping for something native. I actually didn't know about that one, but looks like there are some issues, like can't resize.

Can't resize what? The graph window certainly can be resized. It's just a graph with transformed data and fake axes drawn using the graph's drawing tools.


I tried resizing by clicking on the lower-right corner and dragging. Didn't work at all except I could pull down, but not right to make the thing bigger. Maybe Igor7 bug? I didn't test it in 6, I just randomly played with it when mentioned here.
Igor wrote:
astrostu wrote:

... though unfortunately, my specific needs require a customized implementation that I have to code myself at the moment. In particular, the bandwidth for *each* value has to be different ... I know that's not a standard KDE thing, but my data is crater populations on solar system bodies which follows generally a power-law distribution, and the smoothing parameter (bandwidth) has to be set by the individual crater's size rather than a global thing for each feature in the population. E.g., it makes sense that the uncertainty in the diameter of a 1 km crater is ±0.1 km, but then doesn't make sense that's the same uncertainty on a 100 km crater. So I wrote code that lets me not only set the destination wave spacing (via a diameter (x) wave I create) to be even on a log-plot, but also such that the bandwidth of each crater is 10% (or whatever%) of the diameter going in, as opposed to a fixed value.


You might also want to check out the implementation (http://www.igorexchange.com/node/4749) which is reasonably efficient.

A.G.

Thanks. I have a direct implementation that works for now, though it takes ~20 minutes on 100s of thousands of points. Runs in a few seconds on ~5k.
jjweimer wrote:
astrostu wrote:
My long-term plans are a Python-driven back-end to a web interface that'll do it right.


IOW, when you have the IP code, rather than redo that entire code in Python, you may benefit more to find someone to help you put it up as a Web-server instead.

Interesting ... I'll look into that. Though I like the idea of client-side processing. Last time I did server-side stuff (2003) it was a royal pain, and I can't guarantee a dedicated server as opposed to a client with a web browser. The code isn't hugely complicated, it's more tricks that Igor has that Python doesn't, and tricks that Python has that Igor doesn't, for running it faster.
Quote:
I tried resizing by clicking on the lower-right corner and dragging. Didn't work at all except I could pull down, but not right to make the thing bigger. Maybe Igor7 bug? I didn't test it in 6, I just randomly played with it when mentioned here.

Ah- the Igor 7 restricted resizing mode... A ternary plot made with that package is in Plan Mode (see the Modify Graph dialog). To avoid lots of really annoying flashing and artifacts, any constraint on the window size or shape is implemented with a constraint on what direction you can resize. So you can only grab the upper or lower edge to drag. It is unfortunate that on Windows the mouse cursor still indicates that you can resize from any edge or corner.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
johnweeks wrote:
Quote:
I tried resizing by clicking on the lower-right corner and dragging. Didn't work at all except I could pull down, but not right to make the thing bigger. Maybe Igor7 bug? I didn't test it in 6, I just randomly played with it when mentioned here.

Ah- the Igor 7 restricted resizing mode... A ternary plot made with that package is in Plan Mode (see the Modify Graph dialog). To avoid lots of really annoying flashing and artifacts, any constraint on the window size or shape is implemented with a constraint on what direction you can resize. So you can only grab the upper or lower edge to drag. It is unfortunate that on Windows the mouse cursor still indicates that you can resize from any edge or corner.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

I'm on a Mac. :)
Quote:
I tried resizing by clicking on the lower-right corner and dragging.

Quote:
I'm on a Mac. :)

Right- so you won't get any resize arrow cursor in the corner; you have to use the lower edge.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com