But the problem is not fixed. Now the code is running well but I am getting the concatenated waves without any values. I am attaching a screenshot of it. Could you have a look of this and let me know if you have any ideas? Thanks.
--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH
[quote]
Well, I used a different designation but the problem is not fixed. If I change like that, the code again doesn't run. I am working on it for hours to get the expected concatenated waves with correct length but each time having some issues. Anyways, thanks a lot.
I just looked again at the code and how it works. Forget my earlier answer. I see now how it is supposed to run (I think).
When I run your code, it does the first two iterations (loads the 1st and 2nd dat files) and does the concatenation, on the next iteration it says that it loads the file but no waves have been created in the temporary data folder. This means that the wave references to ProtonEnergyFor_SpecAmpNew etc are NULL, when the concatenate command runs it gives Error: Expected Wave Reference for this reason. I didn't investigate why no waves are loaded on subsequent iterations. I did check if it was something to do with the file, but the code just fails on the next file if I take that particular file out of the directory. Sorry I haven't fixed the problem, but hopefully this brings you closer to a solution.
Switch the debugger on and check out the state of the tempdatafolder when it throws the error. Note that in the history window it says:
path: "Macintosh HD:Users:xxx:Downloads:ONeDatFile:"
Reading from file "s_0340_r_000001_1.dat". Number of Event is: 326
Loaded file 0: "s_0340_r_000001_1.dat"
Reading from file "s_0340_r_000003_1.dat". Number of Event is: 270
Loaded file 1: "s_0340_r_000003_1.dat"
Loaded file 2: "s_0340_r_000001_7.dat"
It looks like it doesn't read from s_0340_r_000001_7.dat so my guess is that the bug is with LoadOneNeutronDataFile
I just looked again at the code and how it works. Forget my earlier answer. I see now how it is supposed to run (I think).
When I run your code, it does the first two iterations (loads the 1st and 2nd dat files) and does the concatenation, on the next iteration it says that it loads the file but no waves have been created in the temporary data folder. This means that the wave references to ProtonEnergyFor_SpecAmpNew etc are NULL, when the concatenate command runs it gives Error: Expected Wave Reference for this reason. I didn't investigate why no waves are loaded on subsequent iterations. I did check if it was something to do with the file, but the code just fails on the next file if I take that particular file out of the directory. Sorry I haven't fixed the problem, but hopefully this brings you closer to a solution.
Switch the debugger on and check out the state of the tempdatafolder when it throws the error. Note that in the history window it says:
path: "Macintosh HD:Users:xxx:Downloads:ONeDatFile:"
Reading from file "s_0340_r_000001_1.dat". Number of Event is: 326
Loaded file 0: "s_0340_r_000001_1.dat"
Reading from file "s_0340_r_000003_1.dat". Number of Event is: 270
Loaded file 1: "s_0340_r_000003_1.dat"
Loaded file 2: "s_0340_r_000001_7.dat"
It looks like it doesn't read from s_0340_r_000001_7.dat so my guess is that the bug is with LoadOneNeutronDataFile
Hi,
Thank you very much for your explanation.
I got your point. I tried to modify the code in a different way. For your attention, my target is to read only _1.dat files of series 340. Since Each series has almost 2500 files, so I switched from "Concatenate [/DL /KILL /NP[=dim ] /O] [typeFlags ] {wave1, wave2, wave3,...}, destWave" command to "Concatenate [ /DL /KILL /NP[=dim ] /O] [typeFlags ] waveListStr, destWave" command, since according to Igor manual, the first one is limited to concatenate 100 waves or less, but there is no limit when using waveListStr. Now the code is looping over all files but I am facing another problem, it is not giving the correct length of the concatenated wave. For example, If I concatenate wave1 with N points to wave2 with M points, then I should have a concatenated wave with points (N+M). To check it, I took one file in a different folder (file "s_0340_r_000000_1.dat") and then ran the code. It gave me the correct number of points (274) which is equal to the number of events in that file. Then I add another file (file "s_0340_r_000001_1.dat") to that folder and ran the code again over these two files. Before that, I ran the code for the second file "s_0340_r_000001_1.dat" separately and got the correct number of data points and values in the table which is 326, the number of events in that file. So after running the code over these two files, I must have (274+326)=600 points in my concatenated wave. But instead of that, I am getting 652 points. I guess something wrong is happening when it is overwriting the existing waves, or something else, though not sure. I faced the same problem before also and tried to figure out the issue but couldn't. I hope now you will understand what is going on with the code. Any help will save my so many evening hours working on it.
Again thanks for your help.
#pragma TextEncoding = "MacRoman"#pragma rtGlobals=3// Use modern global access method and strict wave access.//At first read a single file and then concatenate the data from each file//Use the following command in the commmand window to read all the files//LoadAndConcateNateAllFiles("")//**************************//READING A SINGLE FILE////*************************Menu"Load Waves""Load One File.....", LoadOneNeutronDatFile("","")"Load and Concatenate All Files in Folder.....", LoadAndConcateNateAllFiles("")"Make Histogram for Proton Energy and Proton Time...", MakeHist()"Make Two Dim Hisotgram....", MakeTwoDimHist()EndstaticstrConstant FileNameExtension=".dat"//LoadOneNeutronDatFile(pathName, fileName)//Creates the following wave: TimeStampForSpecAmp, TimeStampForPreAmp, ProtonEnergyFor_SpecAmp, ProtonTime_Spec, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp Function LoadOneNeutronDatFile(pathName, fileName)String pathName //Name of an Igor symbolic path or ""String fileName //Name of file or full path to file//First we have to get a valid reference to a file if(strlen(pathName) == 0||strlen(fileName) == 0)//Display dialog looking for filevariable refNum
Open/D=2/R/F=FileNameExtension/P=$pathName refNum as fileName
filename=S_fileName //S_filename is set by Open/D If(strlen(fileName)==0)//User Cancelled?Print"LoadOneNeutronDatFile was Cancelled"return -1endifendif//String fullpath = S_filename Variablei=0, j=0, k
Variable Vnam1, Vnam2, NumEvent, SpecMax, PreAmpMax, SpecTiMax, PreAmpTiMax, SpecAmpAvg, PreAmpAvg, Numerator, Denominator, ProtonEnergy
//String fullpath = S_filename Open/P=$pathName/R refNum as fileName // fullpathPrintf"Reading from file \"%s\". Number of Event is: ", fileName//fullpath // full path is basically the s_filename FStatus refNum // Used to read the file size
NumEvent = V_logEOF/8208// Gives the total Number of Events where 8208 is the total byte of a single eventPrint NumEvent
//Making WavesMake/O/N=(NumEvent) TimeStampForSpecAmp
Make/O/N=(NumEvent) TimeStampForPreAmp
Make/O/N=(NumEvent) ProtonEnergyFor_SpecAmp
Make/O/N=(Numevent) ProtonTime_SpecAmp
Make/O/N=(NumEvent) ProtonEnergyFor_PreAmp
Make/O/N=(NumEvent) ProtonTime_PreAmp
for(k=0; k<NumEvent; k+=1)// Nested loop for reading all events in a single file
Numerator = 0; Denominator = 0//*******START OF ONE EVENT*******
SpecMax = -inf; PreAmpMax = -inf//Assuming the maximum value of SpecMax and PreAmpMax at the beginning to run the loop //Reading 8 byte time stamp for SpecAmp startFbinread/B=3/F=5 refNum, Vnam1
TimeStampForSpecAmp[k] = Vnam1
//Reading 2 byte for Spec Ampfor(j=0; j<2048; j+=1)Fbinread/B=3/F=2 refNum, Vnam2
if(Vnam2 > Specmax)
Specmax = Vnam2
SpecTiMax = j*10^-7endif
Numerator = Numerator + Vnam2
endfor//SpecAmpAvg = Numerator / Denominator
SpecAmpAvg = Numerator /2048
ProtonEnergy = Specmax - SpecAmpAvg
ProtonEnergyFor_SpecAmp[k] = ProtonEnergy
ProtonTime_SpecAmp[k] = SpecTimax
//Reading 8 byte time stamp for PreAmp startFbinread/B=3/F=5 refNum, Vnam1
TimeStampForPreAmp[k] = Vnam1
//Reading 2 byte for PreAmpfor(j = 0; j<2048; j+=1)Fbinread/B=3/F=2 refNum, Vnam2
if(Vnam2 > PreAmpmax)
PreAmpmax = Vnam2
PreAmpTimax = j*10^-7endif
Numerator = Numerator + Vnam2
endfor
PreAmpAvg = Numerator /2048
ProtonEnergy = PreAmpmax - PreAmpAvg
ProtonenergyFor_PreAmp[k] = ProtonEnergy
ProtonTime_PreAmp[k] = PreAmpTiMax
//*********End OF ONE EVENT********** endforClose refNum
return0//Signifies SuccessEnd//*************************************////READING ALL FILES IN SERIES 340////************************************// Function LoadAndConCateNateAllFiles(pathName)String pathName // Name of symbolic path or " " to get dialogString fileName
Variable index=0if(strlen(pathName) == 0)// If no path specified, create oneNewPath/O temporaryPath // This will put up a dialog. Also NewPath sets the variable V_flag to zero if the operation succeeded or to nonzero if it failedIf(V_flag != 0)return -1// User Cancelledendif
pathName = "temporaryPath"endifString ProtonEnerFor_SpecAmpWaveList = ""String ProtonTimeFor_SpecAmpWaveList = ""Variable result
do// Loop through each file in the folder
fileName = IndexedFile($pathName, index, FileNameExtension)// If index is greater than or equal to zero, IndexedFile returns a STRING containing the name of the index th file in the folder specified by pathName which matches the file type or extension specified by fileTypeOrExtStrIf(strlen(fileName) == 0)// No more files?break//Break out of the loopendif// Create data folder from file NameString dfName = ParseFilePath(0, fileName, ":", 1, 0)// Provides the ability to manipulate file paths & to extract sections of file paths and gives the file name eg s_0340_r_000000_1.dat//Print dfName // Prints the file name
dfName = RemoveEnding(dfName, ".dat")//print dfName // Prints the file name without extension i.e. s_0340_r_000000_1 Variable LengthOfFileNameString = strlen(dfName)// Determines the length of the file name stringString FirstPart = dfName[3,5]// For the purpose of reading series 340 only//Print FirstPart = dfName[3,5] i.e. 340String LastPart = dfName[LengthOfFileNameString - 1, LengthOfFileNameString - 1]// To split the file name for the purpose of reading only _1.dat files//Print LastPart = dfName[LengthOfFileNameString - 1, LengthOfFileNameString - 1] i.e. 1if(stringmatch(FirstPart, "*340*")&&stringmatch(LastPart, "1"))
result = LoadOneNeutronDatFile(pathName, fileName)if(result == 0)// Create wave reference for the wave named "ProtonEnergyFor_SpecAmp" & "ProtonTime_SpecAmpWave/Z TargetWavProEngFor_SpecAmp = ProtonEnergyFor_SpecAmp
if(WaveExists(TargetWavProEngFor_SpecAmp))// The WaveExists function returns one if wave reference is valid or zero if the wave reference is null
ProtonEnerFor_SpecAmpWaveList = AddListItem(GetWavesDataFolder(TargetWavProEngFor_SpecAmp, 2), ProtonEnerFor_SpecAmpWaveList)// AddListItem function Returns listStr after adding itemStr. GetWavesDataFolder function is used when we want to create a wave in the data folder containing a wave passed as a parameterelsePrint"Couldn't find a wave named \"ProtonEnergyFor_SpecAmp\" in the current data folder."endifWave/Z TargetWavProTimFor_SpecAmp = ProtonTime_SpecAmp
if(WaveExists(TargetWavProTimFor_SpecAmp))
ProtonTimeFor_SpecAmpWaveList = AddListItem(GetWavesDataFolder(TargetWavProTimFor_SpecAmp, 2), ProtonTimeFor_SpecAmpWaveList)elsePrint"Couldn't find a wave named \"ProtonTime_SpecAmp\" in the current data folder"endifif(ItemsInList(ProtonEnerFor_SpecAmpWaveList)>= 0)Concatenate/O/NP ProtonEnerFor_SpecAmpWaveList, ProtonEnergyFor_SpecAmpConcat // Concatenates source waves to the destinated wavesendifif(ItemsInList(ProtonTimeFor_SpecAmpWaveList)>= 0)Concatenate/O/NP ProtonTimeFor_SpecAmpWaveList, ProtonTime_SpecAmpConcat
endifendifendif// if (DataFolderExists(dfName))// String prompt// Sprintf prompt, "Data folder name '%s' exists. Click Yes to overwrite the existing data, No to cancel.", dfName// DoAlert 1, prompt// if (V_flag == 2)// Print "Load cancelled. You can rename the existing data folder and try again."// return -1 // User Cancelled// endif// SetDataFolder $dfName// else// NewDataFolder/O/S $dfName// endif//SetDataFolder :: //Back to the parent data folder//KilldataFolder $dfNamePrintf"Loaded file %d: \"%s\"\r", index, fileName
index += 1while(1)if(Exists("temporaryPath"))Killpath temporaryPath
endifWave ProtonEnergyFor_SpecAmp, ProtonTime_Spec, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp
// Sort ProtonEnergyFor_SpecAmp, ProtonTime_Spec, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp, ProtonEnergyFor_SpecAmpreturn0// Signifies successEnd//***************************************************************////MAKING HISTOGRAM FOR PROTON ENERGY AND PROTON TIME FOR SPECAMP////***************************************************************//Function MakeHistogram()WAVE ProtonEnergyFor_SpecAmpConCat, ProtonTime_SpecAmpConcat
// Histogram for proton energy for SpecAmp Make/N=100/O ProtEnrgyFor_SpecAmpConCat_His; DelayUpdateHistogram/C/B={0,300,100} ProtonEnergyFor_SpecAmpConCat, ProtEnrgyFor_SpecAmpConcat_His; DelayUpdate// binstatrt, binwidth, num of bins Display ProtEnrgyFor_SpecAmpConcat_His
Modifygraph mode=5ModifyGraph standoff=0Label bottom "proton energy (a.u.)"Label left "frequency of proton energ"TextBox/C/N=text0/A=MC "proton energy histogram"// Histogram for proton time for SpecAmpMake/N=100/O ProtTime_SpecAmpConcat_His; DelayUpdateHistogram/C/B={0,1.2e-6,100} ProtonTime_SpecAmpConcat, ProtTime_SpecAmpConcat_His; DelayUpdateDisplay ProtTime_SpecAmpConcat_His
Modifygraph mode=5ModifyGraph standoff=0Label left "frequency of proton time"Label bottom "proton time (s)"TextBox/C/N=text0/A=MC "histogram for proton time"End//********************************//MAKING TWO DIM JOINT HISTOGRAM//********************************Function TwoDimJointHist()Wave ProtonEnergyFor_SpecAmpConcat, ProtonTime_SpecAmpConcat
JointHistogram/DEST=M_JointHistogram/BINS={100,100} ProtonEnergyFor_SpecAmpConcat, ProtonTime_SpecAmpConcat
NewImage M_JointHistogram
End
I had another look at this. The original code as you have it works, I think the reason it fails is to do with reading from the file. So, I think it works because if you take the dat files ending 1_1 or 3_1 and make copies giving them other names e.g. 2_1, 4_1, and just have those in a directory. The code works. The concatenated waves are the right length and all is good.
So why doesn't it work on the whole folder? If you run LoadOneNeutronDatFile("","") and point it at 1_7, it fails to make the waves (I think), hence the load from directory option is failing.
I don't know enough about the files or the data structure to help further. The error is because of this part:
TimeStampForSpecAmp[k] = Vnam1
k is out of range. This is because:
NumEvent = V_logEOF/8208
which sets the size of the waves, is not a whole integer for this particular file.
Setting it to ceil(V_logEOF/8208) causes the read to crash because it reaches the end of file unexpectedly and throws a different error (ceil takes the integer above).
So, if you can fix this error I think the whole code will run on all files in your directory.
I had another look at this. The original code as you have it works, I think the reason it fails is to do with reading from the file. So, I think it works because if you take the dat files ending 1_1 or 3_1 and make copies giving them other names e.g. 2_1, 4_1, and just have those in a directory. The code works. The concatenated waves are the right length and all is good.
So why doesn't it work on the whole folder? If you run LoadOneNeutronDatFile("","") and point it at 1_7, it fails to make the waves (I think), hence the load from directory option is failing.
I don't know enough about the files or the data structure to help further. The error is because of this part:
TimeStampForSpecAmp[k] = Vnam1
k is out of range. This is because:
NumEvent = V_logEOF/8208
which sets the size of the waves, is not a whole integer for this particular file.
Setting it to ceil(V_logEOF/8208) causes the read to crash because it reaches the end of file unexpectedly and throws a different error (ceil takes the integer above).
So, if you can fix this error I think the whole code will run on all files in your directory.
[quote=Ann]
Hi,
Thank you very much for your explanations and suggestions. I have figured out the problem with the code. Now it is running well and giving the concatenated wave with the correct length.
Again thanks for your time.
The second part is the destwave.
January 9, 2018 at 01:40 pm - Permalink
Thank you very much for your reply. I switched the source and destination wave position as you said:
But the problem is not fixed. Now the code is running well but I am getting the concatenated waves without any values. I am attaching a screenshot of it. Could you have a look of this and let me know if you have any ideas? Thanks.
January 9, 2018 at 02:33 pm - Permalink
... where theConcatenatedWave is the wave that should contain the concatenated results.
Alternatively, you need to specify a different designation like this ...
--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH
January 9, 2018 at 02:41 pm - Permalink
[quote]
Well, I used a different designation but the problem is not fixed. If I change like that, the code again doesn't run. I am working on it for hours to get the expected concatenated waves with correct length but each time having some issues. Anyways, thanks a lot.
January 9, 2018 at 03:07 pm - Permalink
When I run your code, it does the first two iterations (loads the 1st and 2nd dat files) and does the concatenation, on the next iteration it says that it loads the file but no waves have been created in the temporary data folder. This means that the wave references to ProtonEnergyFor_SpecAmpNew etc are NULL, when the concatenate command runs it gives Error: Expected Wave Reference for this reason. I didn't investigate why no waves are loaded on subsequent iterations. I did check if it was something to do with the file, but the code just fails on the next file if I take that particular file out of the directory. Sorry I haven't fixed the problem, but hopefully this brings you closer to a solution.
Switch the debugger on and check out the state of the tempdatafolder when it throws the error. Note that in the history window it says:
path: "Macintosh HD:Users:xxx:Downloads:ONeDatFile:"
Reading from file "s_0340_r_000001_1.dat". Number of Event is: 326
Loaded file 0: "s_0340_r_000001_1.dat"
Reading from file "s_0340_r_000003_1.dat". Number of Event is: 270
Loaded file 1: "s_0340_r_000003_1.dat"
Loaded file 2: "s_0340_r_000001_7.dat"
It looks like it doesn't read from s_0340_r_000001_7.dat so my guess is that the bug is with
LoadOneNeutronDataFileJanuary 10, 2018 at 12:08 am - Permalink
January 10, 2018 at 01:44 pm - Permalink
So why doesn't it work on the whole folder? If you run LoadOneNeutronDatFile("","") and point it at 1_7, it fails to make the waves (I think), hence the load from directory option is failing.
I don't know enough about the files or the data structure to help further. The error is because of this part:
k is out of range. This is because:
which sets the size of the waves, is not a whole integer for this particular file.
Setting it to ceil(V_logEOF/8208) causes the read to crash because it reaches the end of file unexpectedly and throws a different error (ceil takes the integer above).
So, if you can fix this error I think the whole code will run on all files in your directory.
January 10, 2018 at 11:39 pm - Permalink
[quote=Ann]
Hi,
Thank you very much for your explanations and suggestions. I have figured out the problem with the code. Now it is running well and giving the concatenated wave with the correct length.
Again thanks for your time.
January 11, 2018 at 09:03 am - Permalink