LoadData from a series of experiments

Share icon

Hi,

I am using loaddata to import waves from one pack experiment to the other.
I use it 2 ways:
LoadData/O/L=1 path+file // when I want all the waves that were in the experiment
LoadData/O/J=objectlist path+file // when I want some waves whose name was defined

Now I would like to load the waves that for example only start with "TOP". So What I have down is that I use LoadData/O/L=1 path+file and then I kill all the waves that do not start with "TOP". The problem is that loading all the experiment for each file is long and I get pretty quickly out of memory.

*So maybe you could tell me if there is a way to load the list of wave in the experiment, simply a string that would contain the wave names as datafolderdir(2) but for the second experiment.
Then I could select the waves using this string and load only the one I need.
If you have any other suggestion I am more than happy to hear them!
Thank you in advance,
Celine
Quote:

So What I have down is that I use LoadData/O/L=1 path+file and then I kill all the waves that do not start with "TOP". The problem is that loading all the experiment for each file is long and I get pretty quickly out of memory.


Igor's memory management was improved in Igor Pro 6.00. If you are using an earlier version, upgrade to Igor Pro 6.1 and that should solve the memory problem.

You can download the Igor Pro 6.1 demo from the WaveMetrics web site. It is fully function for 30 days.

July 29, 2010 at 10:14 am - Permalink

Is this still the state of the art? I would like to load waves from another experiment that begin with some prefix, but I am not coming up with a nice way to do that. This would be straightforward if LoadData had wildcard capability, or WaveList could operate on a different experiment, which I think neither can. Any suggestions for strategy?

-Matthew

January 11, 2018 at 01:08 pm - Permalink

Usually I parse the file header using freadline and load the waves of interest via an
LoadWave /O /A /B=TempString /J /L={0,NumberOfSkippedLines,0,0,0} /Q S_FileName
command.
'Tempstring' contains a pattern of skipped (e.g., "C=1,N='_skip_'; ") and parsed (e.g., "C=1,F=0,T=2,N=Frequency; ") parameters for each column.

However, this was on IP6.3x. (If you still use IP6.1, you might want to update...)

HJ

EDIT: I Just realized that the question was on LoadData and not on LoadWave.
Maybe programming the data browser might help?
displayhelptopic "Programming the Data Browser"

January 11, 2018 at 03:30 pm - Permalink

Quote:
Is this still the state of the art?


Yes for Igor7.

I am investigating the feasibility of providing more flexibility for Igor8. I should know more in a day or two.

January 11, 2018 at 06:11 pm - Permalink

For Igor Pro 8, I have added to the LoadData operation the ability to apply regular expressions to control which objects are loaded.

If you are interested in testing this feature, you first need to become an Igor8 beta tester - see https://www.igorpro.net/beta-signup. You will need a valid activation key for an Igor Pro 7 license with to become a beta tester.

Once you have signed up and installed the Igor8 beta, choose Help->Igor Pro Nightly Builds and install the latest nightly build.

Finally, to get documentation for the new LoadData feature, in Igor8 choose Help->Contact Support to create a new email addressed to WaveMetrics support. Enter "LoadData with Regular Expressions" for the subject, enter a brief explanation of what you are trying to do in the body, and send the request to support.

January 17, 2018 at 10:03 am - Permalink

I knew there was a reason I have been using Igor for decades and decades. Thanks for implementing that. I am certain it will be very useful. However, I'm afraid this is a production routine, so I have to stick with non-beta.
I figured out a workaround. I made people name their waves intelligently (i.e. with a sequential index, i.e. wave0, wave1, wave2, ...), so I could stop loading when LoadData failed to find the file.
function TryLoad (mywavename, sourceFile)
    string mywavename, sourceFile
   
    if (strlen (sourceFile) == 0 || strlen (mywavename) == 0)
        return 0
    endif
    if (exists (mywavename))
        return 1
    endif
    loaddata /q/j=mywavename sourcefile
    return (exists (mywavename))
end
function /t LoadMatches (prefixstr, sourceFile)
    string prefixstr, sourceFile
    variable i
    string resultstr = ""
   
    i = 0
    do
        if (tryload (prefixstr + num2str (i), sourceFile))
            resultstr = AddListItem (prefixstr + num2str (i), resultstr, ";", inf)
        else
            break
        endif
        i += 1
    while (1)
    return resultstr
end


That solution may only work for my particular situation, of course.

-mxf

January 19, 2018 at 06:40 am - Permalink