Submit Jobs in parallel

[sanjeeda]

Dear Experts,

I want to do a simultaneous fit using 4 sets of parameters (in text files). I would like to have one job corresponding to each .txt file. I am referring to How to submit parallel jobs using TChain and it suggests that I use PROOF. but I am not sure how I can use it in my case or how I should modify the code to get it done. I intend to use my laptop to submit these jobs.

Currently, I have a version that loops over the .txt files. all I want to do is to run the jobs simultaneously and not one after the other.
Please see this folder: loop_txt.tar.gz (2.9 MB). The macro that I am using is: asymmetry_all_params.C. The runtime output is output.log. One can also see the output plots saved as .pdf files in the folder.

Please help.

[eguiraud]

My suggestion would be to write a program that runs on one .txt file and then a bash script that runs it on all .txt files at the same time. The body of the script would be something like:

for fname in *.txt; do
  ./myprogram fname & # & makes it run in the background
done

Cheers,
Enrico

[sanjeeda]

Dear @eguiraud,

Thank you very much for your response. I made the changes and I am now able to submit the jobs in parallel and I am also getting the desired results. But, there is still a minor issue that I am still not able to resolve. I am describing it below:

The fitting code gives 4 plots corresponding to one .txt file. For eg: if the name of the .txt file is kpi.txt, the names of the output plots will be like: d0pi_d01kpi.txt.pdf, kk_d0bar1kpi.txt.pdf . However, I am also getting 8 extra plots with names like d0pi_d01..pdf, d0pi_d01...pdf and so on. I am not able to understand why and from where these extras are coming and how to prevent them

Here’s the working directory: batch_sim_fit.tar.gz (2.9 MB)

The shell script is submit.sh. The directory txt_files has all the required text files.
In case you want to run it, you just need to change the paths in the following lines of asymmetry_all_params_batch.C:

char *outputpath = “/home/lenovo/Desktop/new_sim_fit/txt_files/”;
char *dirname = “/home/lenovo/Desktop/new_sim_fit/txt_files/”;

Will you please have a look?

Gratefully,
Sanjeeda

[eguiraud]

Hi Sanjeeda,
I am afraid solving the mystery of the extra plots needs some debugging on your part

The broken filenames, with a .. and a ..., would point to an logic error in your program.

Cheers,
Enrico

[sanjeeda]

Hi @eguiraud, sorry for bothering you again.

I discovered that I was looping over the files in the code because of which I was getting plots in multiples of 4 for more than one file. I did not realise this earlier because the number of saved plots was always correct because they were being over-written. I am trying to fix this now.

The following is the patch of code that is supposed to read the files. fname is now an argument.

  RooArgSet* params = model_d0.getParameters(combData);
   
  TString dir1 = gROOT->GetTutorialsDir();
  dir1.Append(fname);
  params->readFromFile(fname);

The message that I get at the very beginning is the following:
[#0] ERROR:InputArguments -- RooArgSet::readFromFile(parameters) error opening file and then the code proceeds with old parameters.

I am referring to: ROOT: tutorials/roofit/rf505_asciicfg.C File Reference . I have also tried to use the usual method using fstream but it doesn’t seem to work with this.
Here’s the updated macro: asymmetry_all_params_arg.C (40.8 KB)

Will you please have a look?

Sincerely,
Sanjeeda

[eguiraud]

Hi @sanjeeda ,
if I understand correctly the problem is now that params->readFromFile(fname) cannot open the file. That is most probably due to fname being wrong, you can try to print it out to see why.

Cheers,
Enrico

[sanjeeda]

Hi @eguiraud,

Thank you for your response.
I am not sure what is meant by filename being wrong. But, I did try to print the file name but it seems like it is not able to pick the file names and I fail to understand why.

Earlier, when I was looping over the file names, there was no problem I did fullfname.Data() as shown below:

char *outputpath = "/home/lenovo/Desktop/new_sim_fit/all_trials/loop_txt/";
  char *dirname = "/home/lenovo/Desktop/new_sim_fit/all_trials/loop_txt/";
  char *ext= ".txt";
  TSystemDirectory dir(dirname, dirname);
  TList *files = dir.GetListOfFiles();
  if (files) {
    TSystemFile *file;
    TString fname;
    TIter next(files);
    while ((file=(TSystemFile*)next())) {
      fname = file->GetName();
      if (!file->IsDirectory() && fname.EndsWith(ext)) {
	cout << fname.Data() << endl;
	
	TString fullfname = dirname + fname;
	cout << fullfname.Data() << endl;
	
	TString dir1 = gROOT->GetTutorialsDir();
	dir1.Append(fullfname.Data());
	params->readFromFile(fullfname.Data());
	//params->Print("v");
	

Now when I do cout << fname <<endl;, there is no result and the code proceeds with
the old parameters.