Most efficient way to graph many ASCII files

Newbie
1

Hi there I am relatively new to ROOT, and I primarily do my coding in python, but my current job is all root and c++, so I would greatly appreciate some help.
I have a folder with about 123 .txt files that have many different columns of data. I am trying to graph these files, but I have heard that just using TGraph every time I want to parse the columns of data is not a very good way to do this. Does anybody have any good suggestions? I can provide the code I currently have, just know that it is currently a work in progress and is unfinished

ROOT Version: 5.34/38
Platform: Linux (Debian 64-bit
Compiler: Not Provided

2

Hi and welcome!

You might try instead with TTree::ReadFile
https://root.cern.ch/doc/master/classTTree.html#a9c8da1fbc68221b31c21e55bddf72ce7

or just do the parsing yourself with C++:

ifstream f("myfile.txt");
if(!f.good()) return;
double x,y,z;
vector<double> xs, ys, zs;
while(f >> x >> y >> z)//assuming ascii file columns separated by spaces
{
    xs.push_back(x);
    ys.push_back(y);
    zs.push_back(z);
}
1 Like
3

This looks like it could work, but is there a way to iterate over the files I have in the directory without needing to hardcode each file name I am reading? I had seen that trees could fill plots with all of certain variables, but trees just looked more difficult to work with.

4

If your 123 .txt files have a kind of naming convention let’s say foo1.txt, foo2.txt … foo123.txt you can iterate over them using a loop, read them and fill a TTree. In the loop you will need to encode the file names using Form:

for (int i=1; i<=123; i++ {
   ifstream f(Form("foo$d.txt",i);
   ...
}
5

Yeah that would have been my first choice as well, but as you can see in the image, the only naming convention is that they all end in .txt. Any other thoughts?

Screenshot (139)
Screenshot (139)1920×1080 322 KB

6

They seems encoded somehow like: ped.22xxxxxx.txt . Do you know the logic behind the six digits xxxxxx ? a run number ? a time ? … something you can get from elsewhere… ?

7

Maybe something like this:

   TSystemDirectory dir("~/fcs/qafiles", "~/fcs/qafiles");
   // get the list of files in the given directory
   TList *files = dir.GetListOfFiles();
   if (files) {
      TSystemFile *file;
      TString fname;
      TIter next(files);
      // now loop over the files
      while ((file=(TSystemFile*)next())) {
         fname = file->GetName();
         if (!file->IsDirectory() && fname.EndsWith(".txt") && 
              fname.BeginsWith("ped.")) {
            // this is a .txt file and its name begins with "ped."
            fname.Prepend(dirname);
            cout << "Processing: " << fname.Data() << endl;
            // so lets import its data
            ifstream f(fname.Data());
            ...
         }
      }
   }

This allows to add/remove files in ~/fcs/qafiles without having to change the code.

1 Like
8

If you can use C++17, you can also try:
https://en.cppreference.com/w/cpp/filesystem/directory_iterator

and combine the ‘if-check’ it with std::starts_with()

9

Sorry for the late reply but thanks! This in conjunction with adding the files to an array solved my problem!

1 Like
10

This was very helpful for parsing the files, thank you.

11

Actually, I have one quick follow up. I just assumed honestly that this code work, but I have been having some issues with the fname.Prepend(dirname) line. I do not see where dirname is being defined, am I missing something? Thanks

12

Also after doing some bug hunting, it seems that I am not entering the first if statement, if (files). If sending my current code would help I would be more than happy to

13

Right, replace dirname by the name of the directory containing the files (e.g. ~/fcs/qafiles). and note that you might replace the ~ with the full path (your $(HOME}) everywhere (also in TSystemDirectory dir("~/fcs/qafiles", "~/fcs/qafiles");)