Dear all:
I have to convert an ascii data into root file. Its size is 8GB. I used syntax similar to basic2.C in the tutorial. It takes so much time that I lose patience by pressing CTRL+C. I have a 4core machine with 4GB memory.
Please help
Regards,
Navds
Hi Navds,
it is hard to say without any information about the ascii file.
You could easily estimate the total time parsing a portion of the file and scaling the elapsed time to the total number of entries.
If the disk is fast enough you can always split the file in 4 parts and use 4 independent processes to translate the ascii input in root format.
Cheers,
Danilo
Thanks for immediate response. Well, it’s a csv file containing 29 columns. This is the code I used:
TString branch_desc ="type"; //branch descriptor
Int_t nfeat = 28 ;
for(Int_t i=1 ; i<=nfeat ; i++){
branch_desc += ":f"+TString::Itoa(i,10);
}
Long64_t nlines = ptreeData->myReadFile(csvInput,branch_desc);
Is it possible to increase buffer/cache size ?
The file splitting is a good idea but I don’t know how to do that.
Hi,
for the splitting, any script would do. If you are on linux/mac you can use split (man split for the doc).
What I was trying to say is that if you process a sub sample of of N lines and you know how many lines you have in total, you’ll be able to know how much time it takes to read the whole file.
Cheers,
Danilo
The split works! And for “use 4 independent processes to translate the ascii input in root format” ? Can you give me syntax ?
Hi,
the syntax very much depends on your environment.
Just pack a bash script launching 4 processes simultaneously doing the transformation to root, use & at the end of the invocation.