Multithread RDataFrame.Snapshot with big files and lots of columns

[StephanH]

Hello again!

I believe your test with hadd ran into this issue:

Root 6.04.14 hadd 100Gb and rootlogon

Dear experts, I am currently trying to use hadd to make an output file with a tree with size > 100 Gb. I read: https://root-forum.cern.ch/t/root-6-04-14-hadd-100gb-tfilemerger-recursiveremove/24117/4 and: It seems that in my case the easiest solution is to create in the directory I execute “hadd” a rootlogon.C file with: { TTree::SetMaxTreeSize( 1000000000000LL ); // 1 TB } However, it seems that when I run “hadd” this new option is not taken into account. I tried two commands: hadd…

Unfortunately, RDataFrame does not react to this setting, so I believe that the snapshot crash you encountered is different from the hadd 100G limit that your test revealed.

Is there a way of using ROOT.RDataFrame(<largeNumber>), i.e. an RDF that’s just counting up? You could attach a dummy computation graph and we try to snapshot it, no input data required.