CACHEREAD with given local file name to overwrite local file on many file reads

ROOT
1

Hello,

I have a batch job which reads sequentially many remote files. It would be convenient to be able to tell ROOT to fetch each file to the local disk - e.g. TFile::Open("…", “CACHEREAD”) - but storing it not only at a given directory (TFile::SetCacheFileDir()), but also with a given name, so that the file is overwritten for each new remote file read. That would prevent filling up disk space of the node. Is this possible somehow?

Cheers,
Antoni

2

Hi,

I invite @pcanal to have a look at this.

3

You can use TFile::ShrinkCacheFileDir. to control the amount of space used.

4

Thank you. Looks like I don’t understand how to use it properly. What I do is the following:

void
do()
{
  const TString files[] = { // each file is about 30 MB
    "root://eosna61.cern.ch//eos/experiment/na61/data/Simulation/Ar_Sc_150_15/15_011_v14e_v1r2p0_pA_slc6_phys/EPOS/MiniSHOE/ArSc150_EPOS_x06047.minishoe.root",
    "root://eosna61.cern.ch//eos/experiment/na61/data/Simulation/Ar_Sc_150_15/15_011_v14e_v1r2p0_pA_slc6_phys/EPOS/MiniSHOE/ArSc150_EPOS_x06048.minishoe.root",
    "root://eosna61.cern.ch//eos/experiment/na61/data/Simulation/Ar_Sc_150_15/15_011_v14e_v1r2p0_pA_slc6_phys/EPOS/MiniSHOE/ArSc150_EPOS_x06049.minishoe.root",
    "root://eosna61.cern.ch//eos/experiment/na61/data/Simulation/Ar_Sc_150_15/15_011_v14e_v1r2p0_pA_slc6_phys/EPOS/MiniSHOE/ArSc150_EPOS_x06050.minishoe.root",
    "root://eosna61.cern.ch//eos/experiment/na61/data/Simulation/Ar_Sc_150_15/15_011_v14e_v1r2p0_pA_slc6_phys/EPOS/MiniSHOE/ArSc150_EPOS_x06051.minishoe.root",
    "root://eosna61.cern.ch//eos/experiment/na61/data/Simulation/Ar_Sc_150_15/15_011_v14e_v1r2p0_pA_slc6_phys/EPOS/MiniSHOE/ArSc150_EPOS_x06052.minishoe.root",
    "root://eosna61.cern.ch//eos/experiment/na61/data/Simulation/Ar_Sc_150_15/15_011_v14e_v1r2p0_pA_slc6_phys/EPOS/MiniSHOE/ArSc150_EPOS_x06053.minishoe.root",
    "root://eosna61.cern.ch//eos/experiment/na61/data/Simulation/Ar_Sc_150_15/15_011_v14e_v1r2p0_pA_slc6_phys/EPOS/MiniSHOE/ArSc150_EPOS_x06054.minishoe.root",
    "root://eosna61.cern.ch//eos/experiment/na61/data/Simulation/Ar_Sc_150_15/15_011_v14e_v1r2p0_pA_slc6_phys/EPOS/MiniSHOE/ArSc150_EPOS_x06055.minishoe.root",
    "root://eosna61.cern.ch//eos/experiment/na61/data/Simulation/Ar_Sc_150_15/15_011_v14e_v1r2p0_pA_slc6_phys/EPOS/MiniSHOE/ArSc150_EPOS_x06056.minishoe.root",
    "root://eosna61.cern.ch//eos/experiment/na61/data/Simulation/Ar_Sc_150_15/15_011_v14e_v1r2p0_pA_slc6_phys/EPOS/MiniSHOE/ArSc150_EPOS_x06057.minishoe.root",
    "root://eosna61.cern.ch//eos/experiment/na61/data/Simulation/Ar_Sc_150_15/15_011_v14e_v1r2p0_pA_slc6_phys/EPOS/MiniSHOE/ArSc150_EPOS_x06058.minishoe.root",
    "root://eosna61.cern.ch//eos/experiment/na61/data/Simulation/Ar_Sc_150_15/15_011_v14e_v1r2p0_pA_slc6_phys/EPOS/MiniSHOE/ArSc150_EPOS_x06059.minishoe.root",
    "root://eosna61.cern.ch//eos/experiment/na61/data/Simulation/Ar_Sc_150_15/15_011_v14e_v1r2p0_pA_slc6_phys/EPOS/MiniSHOE/ArSc150_EPOS_x06060.minishoe.root",
    "root://eosna61.cern.ch//eos/experiment/na61/data/Simulation/Ar_Sc_150_15/15_011_v14e_v1r2p0_pA_slc6_phys/EPOS/MiniSHOE/ArSc150_EPOS_x06061.minishoe.root"
  }
  for (int i = 0; i < 15; ++i) {
    TFile::SetCacheFileDir("tmp");
    TFile::ShrinkCacheFileDir(70000000); // I expect to limit tmp/ to 70MB, i.e. to 2 files at a given time
    TFile::Open(files[i], "CACHEREAD");
  }
}

Instead of having tmp/ with 2 files and < 70MB, I have all 15 files and 450MB. What did I misunderstood?

5

You understood correctly. Let me check if that code still work ;(

6

https://github.com/root-project/root/pull/5317 repairs TFile::ShrinkCacheFile.

Note that it is probably ‘bad’ to be deleting the cache file when open. so I believe your may have meant:

  TFile::SetCacheFileDir("tmp");
  for (int i = 0; i < 15; ++i) {
    TFile::ShrinkCacheFileDir(70000000); // I expect to limit tmp/ to 70MB, i.e. to 2 files at a given time
    auto f = TFile::Open(files[i], "CACHEREAD");
     // do stuff
    delete f;
  }
7

@pcanal Thanks! Indeed now it works as expected. In which ROOT version should I expect this to land?

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.