TFileMerger and the grid

ROOT
1

Hello,

I am having problems merging files that are on the grid with the latest versions of root. This problem doesn’t exist with root v5-17-06 (but I cannot use it because then I get two copies of each NTuple in the merged file)

If locally I run the attached macro with the trunk (or with version v5-18-00a) I get the following error message (for two files):

[quote]Info in TAlienFile::Open: Accessing image 1 of alien:///alice/cern.ch/user/m/mercedes/muonAnalysis/output/Bntkine/200007/no_se/root_archive.zip?filetype=raw&cachesz=2000000&readaheadsz=1000000&rmpolicy=1#bNtKineAnalysis_singleM.root in SE ALICE::GSI::SE
[TFile::Cp] Total 0.87 MB |====================| 100.00 % [2.6 MB/s]
Error in TFile::Cp: read and written bytes differ (922647 != 912536)
Error in TFileMerger::AddFile: cannot get a local copy of file alien:///alice/cern.ch/user/m/mercedes/muonAnalysis/output/Bntkine/200007/no_se/root_archive.zip#bNtKineAnalysis_singleM.root

Info in TAlienFile::Open: Accessing image 1 of alien:///alice/cern.ch/user/m/mercedes/muonAnalysis/output/Bntkine/200007/no_se_1/root_archive.zip?filetype=raw&cachesz=2000000&readaheadsz=1000000&rmpolicy=1#bNtKineAnalysis_singleM.root in SE ALICE::GSI::SE
[TFile::Cp] Total 0.87 MB |====================| 100.00 % [3.3 MB/s]
Error in TFile::Cp: read and written bytes differ (925697 != 915586)
Error in TFileMerger::AddFile: cannot get a local copy of file alien:///alice/cern.ch/user/m/mercedes/muonAnalysis/output/Bntkine/200007/no_se_1/root_archive.zip#bNtKineAnalysis_singleM.root

[/quote]

If now I copy those files locally (I can do that with no problem), their sizes are: 912536 and 915586 respectively, and I can merge them with no problem.

Any suggestion?
Thanks a lot,
Mercedes
mergeFiles.C (703 Bytes)

2

Mercedes,

does the problem also exist when you first copy the file to some other location and then use TFile::Cp() to copy the file to another location, i.e. without using alien:// in the filename, but root://?

Cheers, Fons.

3

Hi Fons,

I am not sure I understand what you suggest.

I think the problem could be related to the fact that the file I want to add is inside a zip archive.

If I do:

[quote]TFile::Cp(“alien:///alice/cern.ch/user/m/mercedes/muonAnalysis/output/Bntkine/200007/no_se/root_archive.zip#bNtKineAnalysis_singleM.root”,“bNtKineAnalysis_singleM.root”)
Info in TAlienFile::Open: Accessing image 1 of alien:///alice/cern.ch/user/m/mercedes/muonAnalysis/output/Bntkine/200007/no_se/root_archive.zip?filetype=raw&cachesz=2000000&readaheadsz=1000000&rmpolicy=1#bNtKineAnalysis_singleM.root in SE ALICE::GSI::SE
[TFile::Cp] Total 0.87 MB |====================| 100.00 % [2.0 MB/s]
Error in TFile::Cp: read and written bytes differ (922647 != 912536)
(Bool_t)(0)[/quote]

But if I remove the reference to the zip:

[quote]TFile::Cp(“alien:///alice/cern.ch/user/m/mercedes/muonAnalysis/output/Bntkine/200007/no_se/bNtKineAnalysis_singleM.root”,“bNtKineAnalysis_singleM.root”)
Info in TAlienFile::Open: Accessing image 1 of alien:///alice/cern.ch/user/m/mercedes/muonAnalysis/output/Bntkine/200007/no_se/bNtKineAnalysis_singleM.root?filetype=raw&cachesz=2000000&readaheadsz=1000000&rmpolicy=1 in SE ALICE::GSI::SE
[TFile::Cp] Total 1.01 MB |====================| 100.00 % [2.6 MB/s]
(Bool_t)(1)[/quote]

However if I try to open the local file that I just copied, I get the following message:

[quote]Error in TFile::Init: bNtKineAnalysis_singleM.root not a ROOT file
[/quote]
If I do what you suggest:

[quote]TFile::Cp(“root:///alice/cern.ch/user/m/mercedes/muonAnalysis/output/Bntkine/200007/no_se/root_archive.zip#bNtKineAnalysis_singleM.root”,“bNtKineAnalysis_singleM.root”)
Error in TXNetFile::CreateXClient: open attempt failed on root:///alice/cern.ch/user/m/mercedes/muonAnalysis/output/Bntkine/200007/no_se/root_archive.zip?filetype=raw&cachesz=2000000&readaheadsz=1000000&rmpolicy=1#bNtKineAnalysis_singleM.root
Error in TFile::Cp: cannot open source file root:///alice/cern.ch/user/m/mercedes/muonAnalysis/output/Bntkine/200007/no_se/root_archive.zip#bNtKineAnalysis_singleM.root
(Bool_t)(0) [/quote]

I have also observed when I used the macro that the local file that it creates in my /tmp/ directory is OK and it’s exactly the same file I have on the grid.
-rw-r–r-- 1 lopez ei 912536 2008-05-19 16:45 ROOTMERGE-0031efde-9286-1831-9717-065d9e86beef.root

If I try to copy the file to another grid directory:

[quote]TFile::Cp(“alien:///alice/cern.ch/user/m/mercedes/muonAnalysis/output/Bntkine/200007/no_se/bNtKineAnalysis_singleM.root”,“alien:///alice/cern.ch/user/m/mercedes/bNtKineAnalysis_singleM.root”)
Info in TAlienFile::Open: Accessing image 1 of alien:///alice/cern.ch/user/m/mercedes/muonAnalysis/output/Bntkine/200007/no_se/bNtKineAnalysis_singleM.root?filetype=raw&cachesz=2000000&readaheadsz=1000000&rmpolicy=1 in SE ALICE::GSI::SE
[TFile::Cp] Total 1.01 MB |====================| 100.00 % [1.1 MB/s]
Error in TAlienFile::GetSize: cannot stat the file alien:///alice/cern.ch/user/m/mercedes/bNtKineAnalysis_singleM.root
Error in TAlienFile::GetSize: cannot stat the file alien:///alice/cern.ch/user/m/mercedes/bNtKineAnalysis_singleM.root
(Bool_t)(1)
[/quote]

And the file is not there.

Hope this helps.
Mercedes

4

Hi Fons,

I know this solution is not the best one but I have commented out:

if (sfile->GetBytesRead() != dfile->GetBytesWritten()) { ::Error("TFile::Cp", "read and written bytes differ (%lld != %lld)", sfile->GetBytesRead(), dfile->GetBytesWritten()); success = kFALSE; }

in io/io/src/TFile.cxx and everything works fine.

Best,
Mercedes