Making separate csv files

ASHISH_SEHRAWAT · December 23, 2022, 9:48pm

Hi all,

I have a csv file with format run number, lumi section and count. Can anybody please help me with a code that can be used to make separate csv files for each run number and set names of each csv file as run_number.csv?

For example, following csv file consist of two run numbers 317080 and 317087. I need a C++ code/python script that can make two separate csv files consisting of only one run number in each file.

Also if there are suppose N run numbers in one csv file, how can I make N csv files for each of them?

317080,40,0.000000317080,41,0.000000317080,42,0.000000317080,43,0.000000317080,44,0.000000317080,45,0.000000317080,46,0.000000317080,47,0.000000317080,48,0.000000317080,49,0.000000317080,50,0.000000317080,51,0.000000317080,52,0.000000317080,53,0.000000317080,54,0.000000317080,55,0.000000317080,56,0.000000317080,57,0.000000317080,58,0.000000317080,59,0.000000317080,60,0.000000317080,61,0.000000317080,62,0.000000317080,63,0.000000317080,64,0.000000317080,65,0.000000317080,66,0.000000317080,67,0.000000317080,68,0.000000317080,69,0.000000317080,70,0.000000317080,71,0.000000317080,72,0.000000317080,73,0.000000317080,74,0.000000317080,75,0.000000317087,42,0.000000317087,43,0.000000317087,44,0.000000317087,45,0.000000317087,46,0.000000317087,47,0.000000317087,48,0.000000317087,49,0.000000317087,50,0.000000317087,51,0.000000317087,52,0.000000317087,53,0.000000317087,54,0.000000317087,55,0.000000317087,56,0.000000317087,57,0.000000317087,58,0.000000317087,59,0.000000317087,60,0.000000317087,61,0.000000317087,62,0.000000317087,63,0.000000317087,64,0.000000317087,65,0.000000317087,66,0.000000317087,67,0.000000317087,68,0.000000317087,69,0.000000317087,70,0.000000317087,71,0.000000317087,72,0.000000317087,73,0.000000317087,74,0.000000317087,75,0.000000317087,76,0.000000317087,77,0.000000317087,78,0.000000317087,79,0.000000317087,80,0.000000317087,81,0.000000317087,82,0.000000317087,83,0.000000317087,84,0.000000317087,85,0.000000317087,86,0.000000317087,87,0.000000317087,88,0.000000317087,89,0.000000317087,90,0.000000317087,91,0.000000317087,92,0.000000317087,93,0.000000317087,94,0.000000317087,95,0.000000317087,96,0.000000317087,97,0.000000317087,98,0.000000317087,99,0.000000317087,100,0.000000317087,101,0.000000317087,102,0.000000317087,103,0.000000317087,104,0.000000317087,105,0.000000317087,106,0.000000317087,107,0.000000317087,108,0.000000317087,109,0.000000317087,110,0.000000317087,111,0.000000317087,112,0.000000317087,113,0.000000317087,114,0.000000317087,115,0.000000317087,116,0.000000317087,117,0.000000317087,118,0.000000317087,119,0.000000317087,120,0.000000317087,121,0.000000317087,122,0.000000317087,123,0.000000317087,124,0.000000317087,125,0.000000317087,126,0.000000317087,127,0.000000317087,128,0.000000317087,129,0.000000317087,130,0.000000317087,131,0.000000317087,132,0.000000317087,133,0.000000317087,134,0.000000317087,135,0.000000317087,136,0.000000317087,137,0.000000317087,138,0.000000317087,139,0.000000317087,140,0.000000317087,141,0.000000317087,142,0.000000317087,143,0.000000317087,144,0.000000317087,145,0.000000317087,146,0.000000317087,147,0.000000317087,148,0.000000317087,149,0.000000317087,150,0.000000317087,151,0.000000317087,152,0.000000317087,153,0.000000317087,154,0.000000317087,155,0.000000317087,156,0.000000317087,157,0.000000317087,158,0.000000317087,159,0.000000317087,160,0.000000317087,161,0.000000317087,162,0.000000317087,163,0.000000317087,164,0.000000317087,165,0.000000317087,166,0.000000317087,167,0.000000317087,168,0.000000317087,169,0.000000317087,170,0.000000317087,171,0.000000317087,172,0.000000317087,173,0.000000317087,174,0.000000317087,175,0.000000317087,176,0.000000317087,177,0.000000317087,178,0.000000317087,212,0.000000317087,213,0.000000317087,214,0.000000317087,215,0.000000317087,216,0.000000317087,217,0.000000317087,218,0.000000317087,219,0.000000317087,220,0.000000317087,221,0.000000317087,222,0.000000317087,257,0.000000317087,258,0.000000317087,259,0.000000317087,260,0.000000317087,261,0.000000317087,262,0.000000317087,263,0.000000317087,264,0.000000317087,265,0.000000317087,266,0.000000317087,267,0.000000317087,268,0.000000317087,269,0.000000317087,270,0.000000317087,271,0.000000317087,272,0.000000317087,273,0.000000317087,274,0.000000317087,275,0.000000317087,276,0.000000317087,277,0.000000317087,278,0.000000317087,279,0.000000317087,280,0.000000317087,281,0.000000317087,282,0.000000317087,283,0.000000317087,284,0.000000317087,285,0.000000317087,286,0.000000317087,287,0.000000317087,288,0.000000317087,289,0.000000317087,290,0.000000317087,291,0.000000317087,292,0.000000317087,293,0.000000317087,294,0.000000317087,295,0.000000317087,296,0.000000317087,297,0.000000317087,298,0.000000317087,299,0.000000317087,300,0.000000317087,301,0.000000317087,302,0.000000317087,303,0.000000317087,304,0.000000317087,305,0.000000317087,306,0.000000317087,307,0.000000317087,308,0.000000317087,309,0.000000317087,310,0.000000317087,311,0.000000317087,312,0.000000317087,313,0.000000317087,314,0.000000317087,315,0.000000317087,316,0.000000317087,317,0.000000317087,318,0.000000317087,319,0.000000317087,320,0.000000317087,321,0.000000317087,322,0.000000317087,323,0.000000317087,324,0.000000317087,325,0.000000317087,326,0.000000317087,327,0.000000317087,328,0.000000317087,329,0.000000317087,330,0.000000317087,331,0.000000317087,332,0.000000317087,333,0.000000317087,334,0.000000317087,335,0.000000317087,336,0.000000317087,337,0.000000317087,338,0.000000317087,339,0.000000317087,340,0.000000317087,341,0.000000317087,342,0.000000317087,343,0.000000317087,344,0.000000317087,345,0.000000317087,346,0.000000317087,347,0.000000317087,348,0.000000317087,349,0.000000317087,350,0.000000317087,351,0.000000317087,352,0.000000317087,353,0.000000317087,354,0.000000317087,355,0.000000317087,356,0.000000317087,357,0.000000317087,358,0.000000317087,359,0.000000317087,360,0.000000317087,361,0.000000317087,362,0.000000317087,363,0.000000317087,364,0.000000317087,365,0.000000317087,366,0.000000317087,367,0.000000317087,368,0.000000317087,369,0.000000317087,370,0.000000317087,371,0.000000317087,372,0.000000317087,373,0.000000317087,374,0.000000317087,375,0.000000317087,376,0.000000317087,377,0.000000317087,378,0.000000317087,379,0.000000317087,380,0.000000317087,381,0.000000317087,382,0.000000317087,383,0.000000317087,384,0.000000317087,385,0.000000317087,386,0.000000317087,387,0.000000317087,388,0.000000317087,389,0.000000317087,390,0.000000317087,391,0.000000317087,392,0.000000317087,393,0.000000317087,394,0.000000317087,395,0.000000317087,396,0.000000317087,397,0.000000317087,398,0.000000317087,399,0.000000317087,400,0.000000317087,401,0.000000317087,402,0.000000317087,403,0.000000317087,404,0.000000317087,405,0.000000317087,406,0.000000317087,407,0.000000317087,408,0.000000317087,409,0.000000317087,410,0.000000317087,411,0.000000317087,412,0.000000317087,413,0.000000317087,414,0.000000317087,415,0.000000317087,416,0.000000317087,417,0.000000317087,418,0.000000317087,419,0.000000317087,420,0.000000317087,421,0.000000317087,422,0.000000317087,423,0.000000317087,424,0.000000317087,425,0.000000317087,426,0.000000317087,427,0.000000317087,428,0.000000317087,429,0.000000317087,430,0.000000317087,431,0.000000317087,432,0.000000317087,433,0.000000317087,434,0.000000317087,435,0.000000317087,436,0.000000317087,437,0.000000317087,438,0.000000317087,439,0.000000317087,440,0.000000317087,441,0.000000317087,442,0.000000317087,443,0.000000317087,444,0.000000317087,445,0.000000317087,446,0.000000317087,447,0.000000317087,448,0.000000317087,449,0.000000317087,450,0.000000317087,451,0.000000317087,452,0.000000317087,453,0.000000317087,454,0.000000317087,455,0.000000317087,456,0.000000317087,457,0.000000317087,458,0.000000317087,459,0.000000317087,460,0.000000317087,461,0.000000317087,462,0.000000317087,463,0.000000317087,464,0.000000317087,465,0.000000317087,466,0.000000317087,467,0.000000317087,468,0.000000317087,469,0.000000317087,470,0.000000317087,471,0.000000317087,472,0.000000317087,473,0.000000317087,474,0.000000317087,475,0.000000317087,476,0.000000317087,477,0.000000317087,478,0.000000317087,479,0.000000317087,480,0.000000317087,481,0.000000317087,482,0.000000317087,483,0.000000317087,484,0.000000317087,485,0.000000317087,486,0.000000317087,487,0.000000317087,488,0.000000317087,489,0.000000317087,490,0.000000317087,491,0.000000317087,492,0.000000317087,493,0.000000317087,494,0.000000317087,495,0.000000317087,496,0.000000317087,497,0.000000317087,498,0.000000317087,499,0.000000317087,500,0.000000317087,501,0.000000317087,502,0.000000317087,503,0.000000317087,504,0.000000317087,505,0.000000317087,506,0.000000317087,507,0.000000317087,508,0.000000317087,509,0.000000317087,510,0.000000317087,511,0.000000317087,512,0.000000317087,513,0.000000317087,514,0.000000317087,515,0.000000317087,516,0.000000317087,517,0.000000317087,518,0.000000317087,519,0.000000317087,520,0.000000317087,521,0.000000317087,522,0.000000317087,523,0.000000317087,524,0.000000317087,525,0.000000317087,526,0.000000317087,527,0.000000317087,528,0.000000317087,529,0.000000317087,530,0.000000317087,531,0.000000317087,532,0.000000317087,533,0.000000317087,534,0.000000317087,535,0.000000317087,536,0.000000317087,537,0.000000317087,538,0.000000317087,539,0.000000317087,540,0.000000317087,541,0.000000317087,542,0.000000317087,543,0.000000317087,545,0.000000317087,546,0.000000317087,550,0.000000317087,551,0.000000317087,552,0.000000317087,553,0.000000317087,554,0.000000317087,555,0.000000317087,556,0.000000317087,557,0.000000317087,558,0.000000317087,559,0.000000317087,560,0.000000317087,562,0.000000317087,563,0.000000317087,564,0.000000317087,565,0.000000317087,566,0.000000317087,567,0.000000317087,568,0.000000317087,569,0.000000317087,570,0.000000317087,571,0.000000317087,572,0.000000317087,573,0.000000317087,574,0.000000317087,577,0.000000317087,578,0.000000317087,580,0.000000317087,581,0.000000317087,582,0.000000317087,584,0.000000317087,586,0.000000317087,588,0.000000317087,589,0.000000317087,590,0.000000317087,591,0.000000317087,592,0.000000317087,594,0.000000317087,595,0.000000317087,598,0.000000317087,599,0.000000317087,602,0.000000317087,603,0.000000317087,604,0.000000317087,605,0.000000317087,606,0.000000317087,607,0.000000317087,612,0.000000317087,613,0.000000317087,615,0.000000317087,624,0.000000

Please help me with this problem. Thanks!

Best regards,
Ashish

Wile_E_Coyote · December 24, 2022, 7:58am

sed -e '{s/317080,/\n317080,/g;s/317087,/\n317087,/g;s/^\n//;s/\n\n//g}' original.csv > fixed.csv
sed -e '{/^317080,/p;d}' fixed.csv > 317080.csv
sed -e '{/^317087,/p;d}' fixed.csv > 317087.csv

ASHISH_SEHRAWAT · December 24, 2022, 9:07am

It worked. Thanks a lot. If there are many csv files in a directory consisting of many run numbers, do you know of any code to make run_number.csv files more quickly?

Wile_E_Coyote · December 24, 2022, 9:10am

Are these “original.csv” files are screwed as the one in your original post here?
Or are they already “nicely” formatted as the “fixed.csv”?

ASHISH_SEHRAWAT · December 24, 2022, 9:12am

all are original.csv files. There are 704 run numbers and around 270 original csv files.

Wile_E_Coyote · December 24, 2022, 9:23am

Well, talk to the one who produces these “original.csv” files.
They need to be formatted so that there are separate lines (see the “fixed.csv”), e.g.:

317080,40,0.000000
317080,41,0.000000
317080,42,0.000000
(...)
317087,613,0.000000
317087,615,0.000000
317087,624,0.000000

Then it will be easy to execute:

for run in 317080 317087; do sed -e '{/^'${run}',/p;d}' original.csv > ${run}.csv; done

ASHISH_SEHRAWAT · December 24, 2022, 9:45am

thanks a lot very helpful. I only produced them using a python script. I will fix it.

Wile_E_Coyote · December 24, 2022, 10:11am

So, if you produce these files yourself, why don’t you simply create different files for different runs?

ASHISH_SEHRAWAT · December 24, 2022, 10:59am

actually there are some limitations in the workflow because input root files contain more than one run number and I have to apply some corrections too for all run numbers using a shell and python script. So making csv files containing only one run number is not possible. I can only process one root file at a time to give corresponding csv file and number of run numbers in csv file is same as in the input root file.

Wile_E_Coyote · December 24, 2022, 11:06am

Attach your macro and one root file.

ASHISH_SEHRAWAT · December 24, 2022, 11:48am

do you have lxplus account?

Wile_E_Coyote · December 24, 2022, 11:56am

If you put them in your “public” area, people should be able to reach them from any machine with an “afs client”.

ASHISH_SEHRAWAT · December 24, 2022, 11:57am

I mean can you use cmsRun command?

Wile_E_Coyote · December 24, 2022, 11:59am

No, not really.

ASHISH_SEHRAWAT · December 24, 2022, 12:02pm

my python script is a CMS configuration file. It uses CMSSW. You can’t run it on a local machine. But I really appreciate the commands that you provided me. Those are good enough to make separate csv files.

system · January 7, 2023, 12:03pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.