null

 

 

Looking at your code and your description I think you need to change your concatenate commands:
                        Concatenate /NP {ProtonEnergyFor_SpecAmp}, ProtonEnergyFor_SpecAmpNew
                        Concatenate /NP {ProtonTime_SpecAmp}, ProtonTime_SpecAmpNew
                        Concatenate /NP {ProtonEnergyFor_PreAmp}, ProtonEnergyFor_PreAmpNew
                        Concatenate /NP {ProtonTime_PreAmp}, ProtonTime_PreAmpNew


The second part is the destwave.
sjr51 wrote:
Looking at your code and your description I think you need to change your concatenate commands:
                        Concatenate /NP {ProtonEnergyFor_SpecAmp}, ProtonEnergyFor_SpecAmpNew
                        Concatenate /NP {ProtonTime_SpecAmp}, ProtonTime_SpecAmpNew
                        Concatenate /NP {ProtonEnergyFor_PreAmp}, ProtonEnergyFor_PreAmpNew
                        Concatenate /NP {ProtonTime_PreAmp}, ProtonTime_PreAmpNew


The second part is the destwave.


Thank you very much for your reply. I switched the source and destination wave position as you said:

              Concatenate /NP {ProtonEnergyFor_SpecAmp}, ProtonEnergyFor_SpecAmpNew
                        Concatenate /NP {ProtonTime_SpecAmp}, ProtonTime_SpecAmpNew
                        Concatenate /NP {ProtonEnergyFor_PreAmp}, ProtonEnergyFor_PreAmpNew
                        Concatenate /NP {ProtonTime_PreAmp}, ProtonTime_PreAmpNew


But the problem is not fixed. Now the code is running well but I am getting the concatenated waves without any values. I am attaching a screenshot of it. Could you have a look of this and let me know if you have any ideas? Thanks.
I believe the suggestion is to write this ...

Concatenate /NP {ProtonEnergyFor_SpecAmp},  theConcatenatedWave
Concatenate /NP {ProtonTime_SpecAmp},  theConcatenatedWave
Concatenate /NP {ProtonEnergyFor_PreAmp}, theConcatenatedWave
Concatenate /NP {ProtonTime_PreAmp},  theConcatenatedWave


... where theConcatenatedWave is the wave that should contain the concatenated results.

Alternatively, you need to specify a different designation like this ...


Concatenate /NP {ProtonEnergyFor_SpecAmp},  ProtonEnergyFor_SpecAmpConcatenated
Concatenate /NP {ProtonTime_SpecAmp},  ProtonTime_SpecAmpConcatenated
Concatenate /NP {ProtonEnergyFor_PreAmp}, ProtonEnergyFor_PreAmpConcatenated
Concatenate /NP {ProtonTime_PreAmp},  ProtonTime_PreAmpConcatenated



--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH
jjweimer wrote:
I believe the suggestion is to write this ...

Concatenate /NP {ProtonEnergyFor_SpecAmp},  theConcatenatedWave
Concatenate /NP {ProtonTime_SpecAmp},  theConcatenatedWave
Concatenate /NP {ProtonEnergyFor_PreAmp}, theConcatenatedWave
Concatenate /NP {ProtonTime_PreAmp},  theConcatenatedWave


... where theConcatenatedWave is the wave that should contain the concatenated results.

Alternatively, you need to specify a different designation like this ...


Concatenate /NP {ProtonEnergyFor_SpecAmp},  ProtonEnergyFor_SpecAmpConcatenated
Concatenate /NP {ProtonTime_SpecAmp},  ProtonTime_SpecAmpConcatenated
Concatenate /NP {ProtonEnergyFor_PreAmp}, ProtonEnergyFor_PreAmpConcatenated
Concatenate /NP {ProtonTime_PreAmp},  ProtonTime_PreAmpConcatenated



--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH


[quote]

Well, I used a different designation but the problem is not fixed. If I change like that, the code again doesn't run. I am working on it for hours to get the expected concatenated waves with correct length but each time having some issues. Anyways, thanks a lot.
I just looked again at the code and how it works. Forget my earlier answer. I see now how it is supposed to run (I think).

When I run your code, it does the first two iterations (loads the 1st and 2nd dat files) and does the concatenation, on the next iteration it says that it loads the file but no waves have been created in the temporary data folder. This means that the wave references to ProtonEnergyFor_SpecAmpNew etc are NULL, when the concatenate command runs it gives Error: Expected Wave Reference for this reason. I didn't investigate why no waves are loaded on subsequent iterations. I did check if it was something to do with the file, but the code just fails on the next file if I take that particular file out of the directory. Sorry I haven't fixed the problem, but hopefully this brings you closer to a solution.

Switch the debugger on and check out the state of the tempdatafolder when it throws the error. Note that in the history window it says:
path: "Macintosh HD:Users:xxx:Downloads:ONeDatFile:"
Reading from file "s_0340_r_000001_1.dat". Number of Event is: 326
Loaded file 0: "s_0340_r_000001_1.dat"
Reading from file "s_0340_r_000003_1.dat". Number of Event is: 270
Loaded file 1: "s_0340_r_000003_1.dat"
Loaded file 2: "s_0340_r_000001_7.dat"

It looks like it doesn't read from s_0340_r_000001_7.dat so my guess is that the bug is with LoadOneNeutronDataFile
sjr51 wrote:
I just looked again at the code and how it works. Forget my earlier answer. I see now how it is supposed to run (I think). When I run your code, it does the first two iterations (loads the 1st and 2nd dat files) and does the concatenation, on the next iteration it says that it loads the file but no waves have been created in the temporary data folder. This means that the wave references to ProtonEnergyFor_SpecAmpNew etc are NULL, when the concatenate command runs it gives Error: Expected Wave Reference for this reason. I didn't investigate why no waves are loaded on subsequent iterations. I did check if it was something to do with the file, but the code just fails on the next file if I take that particular file out of the directory. Sorry I haven't fixed the problem, but hopefully this brings you closer to a solution. Switch the debugger on and check out the state of the tempdatafolder when it throws the error. Note that in the history window it says: path: "Macintosh HD:Users:xxx:Downloads:ONeDatFile:" Reading from file "s_0340_r_000001_1.dat". Number of Event is: 326 Loaded file 0: "s_0340_r_000001_1.dat" Reading from file "s_0340_r_000003_1.dat". Number of Event is: 270 Loaded file 1: "s_0340_r_000003_1.dat" Loaded file 2: "s_0340_r_000001_7.dat" It looks like it doesn't read from s_0340_r_000001_7.dat so my guess is that the bug is with LoadOneNeutronDataFile
Hi, Thank you very much for your explanation. I got your point. I tried to modify the code in a different way. For your attention, my target is to read only _1.dat files of series 340. Since Each series has almost 2500 files, so I switched from "Concatenate [/DL /KILL /NP[=dim ] /O] [typeFlags ] {wave1, wave2, wave3,...}, destWave" command to "Concatenate [ /DL /KILL /NP[=dim ] /O] [typeFlags ] waveListStr, destWave" command, since according to Igor manual, the first one is limited to concatenate 100 waves or less, but there is no limit when using waveListStr. Now the code is looping over all files but I am facing another problem, it is not giving the correct length of the concatenated wave. For example, If I concatenate wave1 with N points to wave2 with M points, then I should have a concatenated wave with points (N+M). To check it, I took one file in a different folder (file "s_0340_r_000000_1.dat") and then ran the code. It gave me the correct number of points (274) which is equal to the number of events in that file. Then I add another file (file "s_0340_r_000001_1.dat") to that folder and ran the code again over these two files. Before that, I ran the code for the second file "s_0340_r_000001_1.dat" separately and got the correct number of data points and values in the table which is 326, the number of events in that file. So after running the code over these two files, I must have (274+326)=600 points in my concatenated wave. But instead of that, I am getting 652 points. I guess something wrong is happening when it is overwriting the existing waves, or something else, though not sure. I faced the same problem before also and tried to figure out the issue but couldn't. I hope now you will understand what is going on with the code. Any help will save my so many evening hours working on it. Again thanks for your help.
#pragma TextEncoding = "MacRoman"
#pragma rtGlobals=3     // Use modern global access method and strict wave access.


//At first read a single file and then concatenate the data from each file
//Use the following command in the commmand window to read all the files
//LoadAndConcateNateAllFiles("")

//**************************
//READING A SINGLE FILE//
//*************************

Menu "Load Waves"
    "Load One File.....", LoadOneNeutronDatFile("","")
    "Load and Concatenate All Files in Folder.....", LoadAndConcateNateAllFiles("")
    "Make Histogram for Proton Energy and Proton Time...", MakeHist()
    "Make Two Dim Hisotgram....", MakeTwoDimHist()
End

static strConstant FileNameExtension=".dat"

//LoadOneNeutronDatFile(pathName, fileName)
//Creates the following wave: TimeStampForSpecAmp, TimeStampForPreAmp, ProtonEnergyFor_SpecAmp, ProtonTime_Spec, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp    


Function LoadOneNeutronDatFile(pathName, fileName)
    String pathName     //Name of an Igor symbolic path or ""
    String fileName     //Name of file or full path to file
         
    //First we have to get a valid reference to a file
    if (strlen(pathName) == 0 ||strlen(fileName) == 0)
        //Display dialog looking for file
        variable refNum
        Open/D=2/R/F=FileNameExtension/P=$pathName refNum as fileName
        filename=S_fileName                   //S_filename is set by Open/D
        If (strlen(fileName)==0)              //User Cancelled?
            Print "LoadOneNeutronDatFile was Cancelled"
            return -1
        endif
    endif
         
         
    //String fullpath = S_filename    
         
    Variable i=0, j=0, k
    Variable Vnam1, Vnam2, NumEvent, SpecMax, PreAmpMax, SpecTiMax, PreAmpTiMax, SpecAmpAvg, PreAmpAvg, Numerator, Denominator, ProtonEnergy
         
         
         
    //String fullpath = S_filename                          
    Open/P=$pathName/R refNum as fileName // fullpath
    Printf "Reading from file \"%s\". Number of Event is: ", fileName//fullpath            // full path is basically the s_filename      
                 
                 
    FStatus refNum              // Used to read the file size
    NumEvent = V_logEOF/8208    // Gives the total Number of Events where 8208 is the total byte of a single event
    Print  NumEvent
         
         
    //Making Waves
         
    Make/O/N=(NumEvent) TimeStampForSpecAmp
    Make/O/N=(NumEvent) TimeStampForPreAmp
         
    Make/O/N=(NumEvent) ProtonEnergyFor_SpecAmp
    Make/O/N=(Numevent) ProtonTime_SpecAmp
         
    Make/O/N=(NumEvent) ProtonEnergyFor_PreAmp
    Make/O/N=(NumEvent) ProtonTime_PreAmp
         
         
    for (k=0; k<NumEvent; k+=1)    // Nested loop for reading all events in a single file  
        Numerator = 0; Denominator = 0
                 
        //*******START OF ONE EVENT*******            
        SpecMax = -inf; PreAmpMax = -inf  //Assuming the maximum value of SpecMax and PreAmpMax at the beginning to run the loop    
                 
                 
        //Reading 8 byte time stamp for SpecAmp start
        Fbinread/B=3/F=5 refNum, Vnam1
        TimeStampForSpecAmp[k] = Vnam1
                 
        //Reading 2 byte for Spec Amp
        for (j=0; j<2048; j+=1)
            Fbinread/B=3/F=2 refNum, Vnam2
            if (Vnam2 > Specmax)
                Specmax = Vnam2
                SpecTiMax = j*10^-7
            endif
            Numerator = Numerator + Vnam2                    
        endfor
        //SpecAmpAvg = Numerator / Denominator
        SpecAmpAvg = Numerator / 2048                  
        ProtonEnergy = Specmax - SpecAmpAvg
        ProtonEnergyFor_SpecAmp[k] = ProtonEnergy
        ProtonTime_SpecAmp[k] = SpecTimax
                 
                 
                 
        //Reading 8 byte time stamp for PreAmp start
        Fbinread/B=3/F=5 refNum, Vnam1
        TimeStampForPreAmp[k] = Vnam1
                 
        //Reading 2 byte for PreAmp
        for (j = 0; j < 2048; j+=1)
            Fbinread/B=3/F=2 refNum, Vnam2
            if (Vnam2 > PreAmpmax)
                PreAmpmax = Vnam2
                PreAmpTimax = j*10^-7
            endif
            Numerator = Numerator + Vnam2
        endfor
        PreAmpAvg = Numerator / 2048
        ProtonEnergy = PreAmpmax - PreAmpAvg
        ProtonenergyFor_PreAmp[k] = ProtonEnergy
        ProtonTime_PreAmp[k] = PreAmpTiMax
                   
        //*********End OF ONE EVENT**********  
    endfor
    Close refNum
    return 0  //Signifies Success
End
                 
   
//*************************************//
//READING ALL FILES IN SERIES 340//
//************************************//    

Function LoadAndConCateNateAllFiles(pathName)
    String pathName   // Name of symbolic path or " " to get dialog
    String fileName
    Variable index=0


    if (strlen(pathName) == 0)         // If no path specified, create one
        NewPath/O temporaryPath // This will put up a dialog. Also NewPath sets the variable V_flag to zero if the operation succeeded or to nonzero if it failed
        If (V_flag != 0)
            return -1        // User Cancelled
        endif
        pathName = "temporaryPath"
    endif
         
    String ProtonEnerFor_SpecAmpWaveList = ""
    String ProtonTimeFor_SpecAmpWaveList = ""
         
    Variable result
    do             // Loop through each file in the folder
        fileName = IndexedFile($pathName, index, FileNameExtension) // If index  is greater than or equal to zero, IndexedFile returns a STRING containing the name of the index th file in the folder specified by pathName  which matches the file type or extension specified by fileTypeOrExtStr
        If (strlen(fileName) == 0)  // No more files?
            break           //Break out of the loop
        endif
                   
                 
        // Create data folder from file Name
        String dfName = ParseFilePath(0, fileName, ":", 1, 0)   //  Provides the ability to manipulate file paths & to extract sections of file paths and gives the file name eg s_0340_r_000000_1.dat
        //Print dfName // Prints the file name
        dfName = RemoveEnding(dfName, ".dat")
        //print dfName // Prints the file name without extension i.e. s_0340_r_000000_1
                   
                   
        Variable LengthOfFileNameString = strlen(dfName)  // Determines the length of the file name string
                   
                   
        String FirstPart = dfName[3,5]   // For the purpose of reading series 340 only
        //Print FirstPart = dfName[3,5] i.e. 340
        String LastPart = dfName[LengthOfFileNameString - 1, LengthOfFileNameString - 1]   // To split the file name for the purpose of reading only _1.dat files
        //Print LastPart =  dfName[LengthOfFileNameString - 1, LengthOfFileNameString - 1] i.e. 1
                   
                   
        if (stringmatch(FirstPart, "*340*") && stringmatch(LastPart, "1"))
            result = LoadOneNeutronDatFile(pathName, fileName)
                               
            if (result == 0)
                               
                // Create wave reference for the wave named "ProtonEnergyFor_SpecAmp" & "ProtonTime_SpecAmp
                Wave/Z TargetWavProEngFor_SpecAmp = ProtonEnergyFor_SpecAmp
                if (WaveExists(TargetWavProEngFor_SpecAmp))    // The WaveExists function returns one if wave reference is valid or zero if the wave reference is null
                    ProtonEnerFor_SpecAmpWaveList = AddListItem(GetWavesDataFolder(TargetWavProEngFor_SpecAmp, 2), ProtonEnerFor_SpecAmpWaveList)   // AddListItem function Returns listStr after adding itemStr. GetWavesDataFolder function is used when we want to create a wave in the data folder containing a wave passed as a parameter
                else
                    Print "Couldn't find a wave named \"ProtonEnergyFor_SpecAmp\" in the current data folder."
                endif
                                         
                                         
                                         
                                         
                Wave/Z TargetWavProTimFor_SpecAmp =  ProtonTime_SpecAmp
                if (WaveExists(TargetWavProTimFor_SpecAmp))
                    ProtonTimeFor_SpecAmpWaveList = AddListItem(GetWavesDataFolder(TargetWavProTimFor_SpecAmp, 2), ProtonTimeFor_SpecAmpWaveList)
                else
                    Print "Couldn't find a wave named \"ProtonTime_SpecAmp\" in the current data folder"
                endif

                if (ItemsInList(ProtonEnerFor_SpecAmpWaveList) >= 0)
                    Concatenate/O/NP ProtonEnerFor_SpecAmpWaveList, ProtonEnergyFor_SpecAmpConcat   // Concatenates source waves to the destinated waves
                endif
                     
                     
                if (ItemsInList(ProtonTimeFor_SpecAmpWaveList) >= 0)
                    Concatenate/O/NP ProtonTimeFor_SpecAmpWaveList, ProtonTime_SpecAmpConcat
                endif

            endif
        endif  

        //      if (DataFolderExists(dfName))
        //          String prompt
        //          Sprintf prompt, "Data folder name '%s' exists. Click Yes to overwrite the existing data, No to cancel.", dfName
        //          DoAlert 1, prompt
        //          if (V_flag == 2)
        //              Print "Load cancelled. You can rename the existing data folder and try again."
        //              return -1 // User Cancelled
        //          endif
        //          SetDataFolder $dfName
        //      else
        //          NewDataFolder/O/S $dfName
        //      endif
                     
                                   

                             //SetDataFolder ::      //Back to the parent data folder
                                                               
                             //KilldataFolder $dfName
        Printf "Loaded file %d: \"%s\"\r", index, fileName
        index += 1
                     
    while (1)
                                                                                                                                                                                                                                                                     
    if (Exists("temporaryPath"))
        Killpath temporaryPath
    endif
                     
                     
    Wave ProtonEnergyFor_SpecAmp, ProtonTime_Spec, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp
    //                     Sort ProtonEnergyFor_SpecAmp, ProtonTime_Spec, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp, ProtonEnergyFor_SpecAmp
                     
    return 0       // Signifies success
                   
End
                                     



//***************************************************************//
//MAKING HISTOGRAM FOR PROTON ENERGY AND PROTON TIME FOR SPECAMP//
//***************************************************************//

Function MakeHistogram()
   
   
   
    WAVE ProtonEnergyFor_SpecAmpConCat, ProtonTime_SpecAmpConcat
         
    // Histogram for proton energy for SpecAmp        
    Make/N=100/O ProtEnrgyFor_SpecAmpConCat_His; DelayUpdate
    Histogram/C/B={0,300,100} ProtonEnergyFor_SpecAmpConCat, ProtEnrgyFor_SpecAmpConcat_His; DelayUpdate  // binstatrt, binwidth, num of bins
    Display ProtEnrgyFor_SpecAmpConcat_His
    Modifygraph mode=5
    ModifyGraph standoff=0
    Label bottom "proton energy (a.u.)"
    Label left "frequency of proton energ"
    TextBox/C/N=text0/A=MC "proton energy histogram"
         
         
    // Histogram for proton time for SpecAmp
    Make/N=100/O ProtTime_SpecAmpConcat_His; DelayUpdate
    Histogram/C/B={0,1.2e-6,100} ProtonTime_SpecAmpConcat, ProtTime_SpecAmpConcat_His; DelayUpdate        
    Display ProtTime_SpecAmpConcat_His
    Modifygraph mode=5
    ModifyGraph standoff=0 
    Label left "frequency of proton time"
    Label bottom "proton time (s)"
    TextBox/C/N=text0/A=MC "histogram for proton time"

End



                                               
//********************************
//MAKING TWO DIM JOINT HISTOGRAM
//********************************


Function TwoDimJointHist()
    Wave ProtonEnergyFor_SpecAmpConcat, ProtonTime_SpecAmpConcat        
    JointHistogram/DEST=M_JointHistogram/BINS={100,100 } ProtonEnergyFor_SpecAmpConcat, ProtonTime_SpecAmpConcat
    NewImage M_JointHistogram

End
I had another look at this. The original code as you have it works, I think the reason it fails is to do with reading from the file. So, I think it works because if you take the dat files ending 1_1 or 3_1 and make copies giving them other names e.g. 2_1, 4_1, and just have those in a directory. The code works. The concatenated waves are the right length and all is good.

So why doesn't it work on the whole folder? If you run LoadOneNeutronDatFile("","") and point it at 1_7, it fails to make the waves (I think), hence the load from directory option is failing.

I don't know enough about the files or the data structure to help further. The error is because of this part:
TimeStampForSpecAmp[k] = Vnam1

k is out of range. This is because:
NumEvent = V_logEOF/8208

which sets the size of the waves, is not a whole integer for this particular file.
Setting it to ceil(V_logEOF/8208) causes the read to crash because it reaches the end of file unexpectedly and throws a different error (ceil takes the integer above).
So, if you can fix this error I think the whole code will run on all files in your directory.
sjr51 wrote:
I had another look at this. The original code as you have it works, I think the reason it fails is to do with reading from the file. So, I think it works because if you take the dat files ending 1_1 or 3_1 and make copies giving them other names e.g. 2_1, 4_1, and just have those in a directory. The code works. The concatenated waves are the right length and all is good.

So why doesn't it work on the whole folder? If you run LoadOneNeutronDatFile("","") and point it at 1_7, it fails to make the waves (I think), hence the load from directory option is failing.

I don't know enough about the files or the data structure to help further. The error is because of this part:
TimeStampForSpecAmp[k] = Vnam1

k is out of range. This is because:
NumEvent = V_logEOF/8208

which sets the size of the waves, is not a whole integer for this particular file.
Setting it to ceil(V_logEOF/8208) causes the read to crash because it reaches the end of file unexpectedly and throws a different error (ceil takes the integer above).
So, if you can fix this error I think the whole code will run on all files in your directory.

[quote=Ann]

Hi,

Thank you very much for your explanations and suggestions. I have figured out the problem with the code. Now it is running well and giving the concatenated wave with the correct length.
Again thanks for your time.