Converting csv to root

[Danny_Lee]

Hi, I know this has been asked before and the basic.c and basic2.c are example of how to do it. However the file I have is much more complicated and with the additional problem the csv file has Japanese characters for some of the parameters(I would like to keep the parameter name as it is so my colleague can use root files). I think would give a try asking for help.

The example csv file I have contain the first 20 lines about information about the daq settings which i would want to put in a txt file so I can refer later.

from line 21 onwards, the csv file would look like this

スキャン,時刻,101 <DigitalGauge1> (VDC),アラーム 101,102 <DigitalGauge2> (VDC),アラーム 102,103 <DigitalGauge3> (VDC),アラーム 103,104 <DigitalGauge4> (VDC),アラーム 104,105 <DigitalGauge5> (VDC),アラーム 105,106 <PiraniGauge> (VDC),アラーム 106,107 (VDC),アラーム 107,108 (VDC),アラーム 108,109 (VDC),アラーム 109,110 (VDC),アラーム 110,111 (VDC),アラーム 111,112 (VDC),アラーム 112,113 <1KPot> (VDC),アラーム 113,114 <MixingChamber> (VDC),アラーム 114,115 <HeatExchanger> (OHM),アラーム 115,116 <Still> (OHM),アラーム 116
1,2017/03/22 21:08:21:256,3.0216646,0,2.996026,0,2.999162,0,2.9948355,0,3.021831,0,0.009563304,0,0.005924331,0,-0.011202323,0,-0.000468457,0,-0.062406373,0,-0.039981009,0,-0.017782287,0,0.1360781,0,0.1006473,0,2165.986,0,6376.459,0
2,2017/03/22 21:08:31:239,3.0215494,0,2.996026,0,2.9992004,0,2.9948483,0,3.021895,0,0.009562658,0,-0.019853747,0,-0.001696982,0,0.035609786,0,-0.006360003,0,0.011028187,0,-0.022275683,0,0.1358348,0,0.1007241,0,2165.857,0,6382.348,0
........

where the rest are values recorded by the daq as the 2 lines above in repeating fashion.

I would like to convert the data from line 21 onwards into root format with each tree corresponds to the parameter name in the first line of line21 onwards.

There is a bit too much things I am trying to include here. To start off I modified the basic.c example together with the answer i found here http://stackoverflow.com/questions/31420191/converting-900-mb-csv-into-root-cern-ttree. But I know with that the code would not work since there are additional stuff i am trying incoperate here.

Apologies if this looks like I am asking too much.

Danny

[couet]

It seems you already have done a small prototype. Can you post it here ?

[Danny_Lee]

Sorry that i forgot to include some of the code i wrote. I only manage wrote the part that convert to root. (note: from TBrowser() I only see the name of the tree so i figure i must have done something wrong)

void convertCSV() {
   TString dir = gSystem->UnixPathName(__FILE__);
   dir.ReplaceAll("convertCSV.C","");
   dir.ReplaceAll("/./","/");
   ifstream in;
   in.open(Form("%stestdata.csv",dir.Data()));

   Int_t nlines = 20;
   Int_t n,Alarm101,Alarm102,Alarm103,Alarm104,Alarm105,Alarm106,Alarm107,Alarm108,Alarm109,Alarm110,Alarm111,Alarm112,Alarm113,Alarm114,Alarm115,Alarm116;
   Float_t Time,DigitalGauge1,DigitalGauge2,DigitalGauge3,DigitalGauge4,DigitalGauge5,PiraniGauge,VDC107,VDC108,VDC109,VDC110,VDC111,VDC112,OneKPot,MixingChamber,HeatExchanger,Still;

   TFile *f = new TFile("testdata.root","RECREATE"); 
   TTree *tree = new TTree("ntuple","data from csv file");
   
   tree->Branch("スキャン",&n,"スキャン/I");
   tree->Branch("時刻",&Time,"時刻/C");
   tree->Branch("101 <DigitalGauge1> (VDC)",&DigitalGauge1,"101 <DigitalGauge1> (VDC)/F");
   tree->Branch("アラーム 101",&Alarm101,"アラーム 101/I");
   tree->Branch("102 <DigitalGauge2> (VDC)",&DigitalGauge2,"102 <DigitalGauge2> (VDC)/F");
   tree->Branch("アラーム 102",&Alarm102,"アラーム 102/I");
   tree->Branch("103 <DigitalGauge3> (VDC)",&DigitalGauge3,"103 <DigitalGauge3> (VDC)/F");
   tree->Branch("アラーム 103",&Alarm103,"アラーム 103/I");
   tree->Branch("104 <DigitalGauge4> (VDC)",&DigitalGauge4,"104 <DigitalGauge4> (VDC)/F");
   tree->Branch("アラーム 104",&Alarm104,"アラーム 104/I");
   tree->Branch("105 <DigitalGauge5> (VDC)",&DigitalGauge5,"105 <DigitalGauge5> (VDC)/F");
   tree->Branch("アラーム 105",&Alarm105,"アラーム 105/I");
   tree->Branch("106 <PiraniGauge> (VDC)",&PiraniGauge,"106 <PiraniGauge> (VDC)/F");
   tree->Branch("アラーム 106",&Alarm106,"アラーム 106/I");
   tree->Branch("107 (VDC)",&VDC107,"107 (VDC)/F");
   tree->Branch("アラーム 107",&Alarm107,"アラーム 107/I");
   tree->Branch("108 (VDC)",&VDC108,"108 (VDC)/F");
   tree->Branch("アラーム 108",&Alarm108,"アラーム 108/I");
   tree->Branch("109 (VDC)",&VDC109,"109 (VDC)/F");
   tree->Branch("アラーム 109",&Alarm109,"アラーム 109/I");
   tree->Branch("110 (VDC)",&VDC110,"110 (VDC)/F");
   tree->Branch("アラーム 110",&Alarm110,"アラーム 110/I");
   tree->Branch("111 (VDC)",&VDC111,"111 (VDC)/F");
   tree->Branch("アラーム 111",&Alarm111,"アラーム 111/I");
   tree->Branch("112 (VDC)",&VDC112,"112 (VDC)/F");
   tree->Branch("アラーム 112",&Alarm112,"アラーム 112/I");
   tree->Branch("113 <1KPot> (VDC)",&OneKPot,"113 <1KPot> (VDC)/F");
   tree->Branch("アラーム 113",&Alarm113,"アラーム 113/I");
   tree->Branch("114 <MixingChamber> (VDC)",&MixingChamber,"114 <MixingChamber> (VDC)/F");
   tree->Branch("アラーム 114",&Alarm114,"アラーム 114/I");
   tree->Branch("115 <HeatExchanger> (OHM)",&HeatExchanger,"115 <HeatExchanger> (OHM)/F");
   tree->Branch("アラーム 115",&Alarm115,"アラーム 115/I");
   tree->Branch("116 <Still> (OHM)",&Still,"116 <Still> (OHM)/F");
   tree->Branch("アラーム 116",&Alarm116,"アラーム 116/I");

   while (1) {
      in >> n >> Time >> DigitalGauge1 >> Alarm101 >> DigitalGauge2 >> Alarm102 >> DigitalGauge3 >> Alarm103 >> DigitalGauge4 >> Alarm104 >> DigitalGauge5 >> Alarm105 >> PiraniGauge >> Alarm106 >> VDC107 >> Alarm107 >> VDC108 >> Alarm108 >> VDC109 >> Alarm109 >> VDC110 >> Alarm110 >> VDC111 >> Alarm111 >> VDC112 >> Alarm112 >> OneKPot >> Alarm113 >> MixingChamber >> Alarm114 >> HeatExchanger >> Alarm115 >> Still >> Alarm116;
      if (!in.good()) break;
      tree->Fill();
      nlines++;
   }
   
   in.close();

   f->Write();
}

[couet]

You have declared Time as a Float_t whereas it is a string. It looks like:

2017/03/22 21:08:21:256

Thats not a float. You better revise the way you read the file. The format might be more complex.

[Danny_Lee]

Thanks for the correction on my mistake. Do you have any suggestion/example on how I might improve the way I read the file?

Danny

[couet]

testcsv.C (248 Bytes)
testcsv.txt (72 Bytes)

[Danny_Lee]

Thanks, would you have any clue on how I might selectively choose which line to copy in the csv file to the root file. I could not find any examples on root which demonstrate to read selective lines of any text file

[couet]

That more a C++ question that a ROOT one. I would do a read of the whole line in a large character string, check if you want to keep this line or not using your own criteria (doing some string comparisons) and then, if a line is selected, do a scanf to read your data in that character string similarly to what I sent you before.