Compound tokens in TString

[abilandz]

Dear Root Experts,

My understanding is that TString::Tokenize(…) automatically treats multi-character argument as consecutive single-character tokens, for instance:

TString s("a_b.c")
s.Tokenize("_")->At(0)->GetName() // prints "a"
s.Tokenize("_.")->At(2)->GetName() // prints "c", since now both "_" and "." are tokens

However, what if I need a token itself to be a multi-character expression? Using the above example, I would like to have for instance “_b” as a compound token, and get the following result:

TString s("a_b.c")
s.Tokenize("_b")->At(0)->GetName() // prints "a"
s.Tokenize("_b")->At(1)->GetName() // prints ".c"

What is the best strategy to handle compound tokens in ROOT in this sort of problem?

Thanks!

Cheers,
Ante

---

Please read tips for efficient and successful posting and posting code

ROOT Version: Not Provided
Platform: Not Provided
Compiler: Not Provided

---

[dastudillo]

Use a regular experssion and the other Tokenize method.

void toke() {
  TString s = "a_b.c_d";
  TString tok;
  Ssiz_t from = 0;

  cout << " Method 1" << endl;
  cout << s.Tokenize("a_")->At(0)->GetName() << endl;
  cout << s.Tokenize("a_")->At(1)->GetName() << endl;

  cout << " Method 2 - group" << endl;
  while (s.Tokenize(tok, from, "a_")) {
    cout << tok << endl;
  }

  cout << " Method 2 - separate" << endl;
  from = 0;
  while (s.Tokenize(tok, from, "[a_]")) {
    cout << tok << endl;
  }
}

gives

root [0] .x toke.C
 Method 1
b.c
d
 Method 2 - group

b.c_d
 Method 2 - separate


b.c
d
root [1]

[abilandz]

@dastudillo

  cout << " Method 1" << endl;
  cout << s.Tokenize("a_")->At(0)->GetName() << endl;
  cout << s.Tokenize("a_")->At(1)->GetName() << endl;

  cout << " Method 2 - group" << endl;
  while (s.Tokenize(tok, from, "a_")) {
    cout << tok << endl;
  }

  cout << " Method 2 - separate" << endl;
  from = 0;
  while (s.Tokenize(tok, from, "[a_]")) {
    cout << tok << endl;
  }

Thanks a lot, this works! I overlooked there is another Tokenize(…) method!

Cheers,
Ante