Neural network and crashes
A warm hello to everybody!
Can you please help me on this one: I am using neural networks in Igor, I understand it would be better to go for something like Keras but I decided to give it a try given my experience in Igor and my love for it.
I have 21 experimental curves with ~5000 Points, and I am trying to predict one single parameter, coming from a different kind of test, from each curve. I know 21 is not really a lot but this is what I have and I guess it is good enough to start learning. So my input wave has 21 rows and ~5000 columns. My output wave has only 1 column and 21 rows.
Since I am not familiar with learning rate, momentum etc., in particular for this small data set, I went brute force with 3 nested loops changing the nodes and these parameters, recording the best RMS error. Very cool, I leave the PC running and the day after I find out that a simple network can apparently predict that one single parameter with the same accuracy as applying a physical model and tuning the fitting parameters (done separately). The RMS error is kind of high, 0.05, but as said it is as good as it gets with old school directly programming.
Now the disappointing part: when I try to run the neural network on a test wave (1 row, ~5000 columns. Actually the first curve/row of the input wave just to see how it works), Igor crashes. I'm shocked, never seen Igor crashing :) I tried several times, on two computers (both Windows 10), with 2 Igor versions (I think 6.37 and 8.02), continuously crashing on me.
instruction:
NeuralNetworkRun Input=test, WeightsWave1=M_weights1, WeightsWave2=M_weights2
I tried to kill the weights waves, re-run the training and re-run the run -> crash
I'm thinking that if the PC can train on 21 curves, it shouldn't be a matter of resources to apply the model to 1 curve. Besides my Igor has done more demanding things than that, without crashing. From the Crash Report I see that USEDPHYSMEM is 1/40 of PHYSMEM, and FREEMEM is much, much higher. Exception Code: 0xC0000005
Any feedback from anyone, please?
Since I am at it, any chance to do something like deep learning too? I know there's tons of possibilities already but since it seems to be a very important subject, now and in the future, it would be very cool to have it (ready for Igor 9!).
Thanks a lot!
Only Wavemetrics can help to fix a crash.
They are very responsive if you use "Help --> Contact Support" and describe a reproducible way to cause the crash. I suggest you go that way.
June 4, 2019 at 07:23 am - Permalink
If you can reliably reproduce the crash using the latest version of Igor (8.03), please use Help->Contact Support to send us steps we can follow to reproduce the crash. If you can reproduce the crash with 8.03 but not reliably, please send the minidump that is usually created after a crash and as much information as you can about what you were doing at the time of the crash. Sometimes we can get some clues from the minidump.
You mentioned that you're using 8.02, so I do suggest that you update to 8.03 first to make sure it's not something we've already fixed.
June 4, 2019 at 07:53 am - Permalink