Too big files

ROOT
1

Dear root experts

I am having the problem that my files become too big (2G) and then root tries to split them and it fails, it finishes the job but the outputs are missing events (~15000 - 30000 events missing). In these files I am trying to write TTrees

Is there a clever way to round this problem?

thank you in advance!
Zlatka

2

Hi Zlatka,

You're going to have to provide more details.  But here are a couple of comments:

  • Root usually works very well at splitting files. What do you mean by “fails”?

  • You can set the size at which Root splits files (most modern operating systems don’t mind files bigger than 2 Gb although I personally do. :smiley: )

    Cheers,
    Charles

3

Hello

thank you for your reply!

only now it comes to my mind that it could be a problem of my OS. If this is the case please give me an advice how could I split the job on two files!

The jobs finish without processing the full statistics, the error messages I recieve are:

Fill: Switching to new file: output_1.root
Using host libthread_db library "/lib/tls/libthread_db.so.1".
Attaching to program: /proc/17534/exe, process 17534
[Thread debugging using libthread_db enabled]
[New Thread -1208755456 (LWP 17534)]
0x002647a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
#1  0x01d877e3 in __waitpid_nocancel () from /lib/tls/libc.so.6
#2  0x01d31649 in do_system () from /lib/tls/libc.so.6
#3  0x004ae8bd in system () from /lib/tls/libpthread.so.0
#4  0x010ed633 in TUnixSystem::Exec ()
   from /opt/products/root/5.18.00/lib/libCore.so.5.18
#5  0x010f2f75 in TUnixSystem::StackTrace ()
   from /opt/products/root/5.18.00/lib/libCore.so.5.18
#6  0x010efce6 in TUnixSystem::DispatchSignals ()
   from /opt/products/root/5.18.00/lib/libCore.so.5.18
#7  0x010efd74 in SigHandler ()
   from /opt/products/root/5.18.00/lib/libCore.so.5.18
#8  0x010eefd9 in sighandler ()
   from /opt/products/root/5.18.00/lib/libCore.so.5.18
#9  <signal handler called>
#10 0x00000000 in ?? ()
#11 0x0804d6fc in main (argc=3, argv=0xbfeb13c4) at i586_rhel40/../myMacro.C:444

The line 444 is

tree.Write();
file->Close();

and the size of the file is 1.8G

The events I should have inside are 406943 and in fact I have 400063 in my output.root

please let me know if I should provide more information!

4

ROOT can write files as large as your OS supports.
When writing Trees, the default behaviour is to close a Tree when its size reaches 1.9GB.
You can chnge this max value by calling TTree::TetMaxTreeSize.
Example;

//Authorize Trees up to 2 Terabytes (if the system can do it) TTree::SetMaxTreeSize(1000*Long64_t(2000000000));

Rene

5

Hi,

Note that you need to either default the default max size as explain by Rene or use tree.Write(); file = tree.GetCurrentFile(); //to get the pointer to the current file file->Write(); file->Close();

Cheers,
Philippe.

6

Dear Rene, dear all

thank you very much for your useful tips, the suggestion from Rene worked!

Thank you very much, it is all fine now!

have a good weekend!
Zlata