Regex usage in TSystem::ListLibraries()

Changing the flag to kFALSE in GetLibraries make ListLibrarie mush easier to use.

root [2] gSystem->ListLibraries(".*");
 
Loaded shared libraries
=======================
/Users/couet/git/couet-root-bin/lib/libRint.so
/Users/couet/git/couet-root-bin/lib/libCore.so
/Users/couet/git/couet-root-bin/lib/libRIO.so
/Users/couet/git/couet-root-bin/lib/libThread.so
/Users/couet/git/couet-root-bin/lib/libtbb.dylib
/Users/couet/git/couet-root-bin/lib/libCling.so
/Users/couet/git/couet-root-bin/lib/libMathCore.so
/Users/couet/git/couet-root-bin/lib/libImt.so
/Users/couet/git/couet-root-bin/lib/libMultiProc.so
/Users/couet/git/couet-root-bin/lib/libNet.so
-----------------------
10 libraries loaded
=======================
root [3] gSystem->ListLibraries(".*RIO.*");
 
Loaded shared libraries
=======================
/Users/couet/git/couet-root-bin/lib/libRIO.so
-----------------------
1 libraries loaded
=======================
root [4] 

But I won’t do that change …

//
// Usage example:
// root -l -q 'ListLibraries.cxx(".*libCore\\..*")'
//
#include "TString.h"
#include "TRegexp.h"
#include "TSystem.h"
#include <iostream>
void ListLibraries(const char *regexp = ".*", Bool_t wildcard = kFALSE) {
  if (!gSystem) return; // just a precaution
  if (!(regexp && regexp[0])) regexp = (wildcard ? "*" : ".*");
  TRegexp pat(regexp, wildcard);
  TString libs(gSystem->GetLibraries());
  TString tok;
  Ssiz_t from = 0, ext;
  while (libs.Tokenize(tok, from, " ")) {
    if ((tok.Index(pat, &ext) != 0) || (ext != tok.Length())) continue;
    std::cout << tok << "\n";
  }
}

@couet How about something that is “backward compatible”:

void TSystem::ListLibraries(const char *regexp = "", Bool_t wildcard = kTRUE)
{
   // ...
   TString libs = GetLibraries(regexp, wildcard);
   // ...
}

Yes I was thinking of it.

I made a PR.

This PR allows the following:

root [0] gSystem->ListLibraries("Core")
 
Loaded shared libraries
=======================
-----------------------
0 libraries loaded
=======================


root [1] gSystem->ListLibraries("Core",kFALSE)
 
Loaded shared libraries
=======================
/Users/couet/git/couet-root-bin/lib/libCore.so
/Users/couet/git/couet-root-bin/lib/libMathCore.so
-----------------------
2 libraries loaded
=======================

@couet This output is terribly wrong. The regexp "Core" should return nothing (there is no library with the full file name “Core”. Only a regexp like ".*Core.*" should return both libraries.

@Wile_E_Coyote

That’s exactly the problem the regular expression like ".*Core.*" does not work as a regular expression as it is a wildcard (cf my previous post) and it returns nothing. (bug ?) … now gSystem->ListLibraries("Core") returns nothing either (as before).

But, with the PR, when you put the flag to kFALSE then regexp is considered as a string and you get the two libraries. The PR does not change anything compared to the current behavior. Seems to me it is buggy but you said we should not change it and you proposed a quite complex workaround.

Now I added the possibility to change the flag. So we can tell the method to not consider regexp as a wildcard. The logic is already encoded in GetLibraries, this flag only allows us to trigger it. By doing that (see the new help) you can display the two “Core”.

So, maybe stop calling this parameter “regexp”, because it’s misleading. Change its name to “substring”. Then, when the “wildcard = true” (the default now), it will be treated as a ROOT “wildcard expression”, and when “wildcard = false”, it will be treated as an ordinary “substring” (i.e., it’s never a “regular expression”).

I think the core of the problem now is that the TSystem::GetLibraries method names its parameter “regexp”, but it’s never a “regular expression” (again, it’s either treated as a ROOT “wildcard expression” or as an ordinary “substring”).

Well, maybe the original author of the ROOT “wildcard expression” source code knows how to force it to treat “/” as a regular character.

See my “ListLibraries.cxx” macro. It really takes a “regular expression” as a parameter.

Dear all,

let me thank you for your interest in finding a solution! From a user’s standpoint, it would be great if the name of the parameter were updated, to clarify what, by default, the parameter represents, and have the possibility (e.g., via an additional flag, as you both suggested) to pass a real regular expression. That would make the command very powerful and flexible. I believe I did try both ".*" (reg exp) and "*" (a wildcard expression?), and in both cases the command returned 0 libraries found, hence my original message. Thank you very much again for your suggestions, and proposals for a fix! Best,
Alberto

Hi Alberto,
I will update the PR today.
Cheers,
Olivier

Just remember that, also with the new PR, you will NOT get the “regular expression” functionality.

I know. The wildcard mode is kept in the PR. Seems to me that’s wrong We should go to regexp seems to me.

I have simplified the code. See the PR.

Looks like you incorporated my macro … but maybe you could leave the “wildcard” possibility, too … see my ListLibraries.cxx source code again.

That said, I have some doubts.
This is clearly some change that breaks “backward compatibility”, but maybe there are not any people who really rely on it (well, it should explicitly be described in the “release notes”, of course).
The original source code parses the returned “libs” using " \t\s" as separators, while my macro considers only " " (on the other hand, the TSystem::GetLibraries description says it returns a space separated list of loaded shared libraries, so I don’t understand why / when another separators need to be considered and what is “\s”).
The original source code thinks there can be “-lNameOfLib” (it will remove the leading “-l”), and it will completely skip “-Lpathname” (actually, anything that begins with a “-”, except “-l”), while my macro doesn’t care.

Yes, I mentioned you in the PR log.

I thought of it but as wildcards do not work because of “/”, is it not simpler to have only regular expression? The way it is done now only changes ListLibrariesinternal code which only parses the RegExp and displays the libs satisfying it. Is that not what is requested from that simple utility?

Calling sequences are not modified so it is much less intrusive. Also, I added a clear example in the method doc.

I checked in the ROOT code and in roottest: ListLibraries is not used in any tests. So there will be no issue like comparing the new output to a reference one.

Note also that I did not keep the header and footer of the listing. That did not bring any interesting information and would only make the parsing by a program more difficult.

I’ve been thinking about the “wildcard” possibility because someone might already be using such strings with an “appropriate” number of “/” characters inside (and the required changes in the macro were trivial).

BTW. I don’t really understand why you people do it. Whatever I “propose” / commit" is done anonymously (otherwise, I would have created my own pull requests, wouldn’t I). Well, short / simple pieces of code (let’s say up to something like several tens of lines) could be “CC CC0”, and longer / complicated pieces of code could be “CC BY”.

Seems to me the list libraries always contains / in libraries’ names. A wildcard with ‘/’ does not work. Or at least I did not find a simple way to make it work. Did you ?
The regular expression on the opposite works nicely as it should. So why not simply use working code instead of something which works only when you list all the libraries (which I think 99% of the 0.1% of people using this functionality are doing).

Because that’s the proper way to behave when you use (or get some inspiration from) code written by somebody else.

In the case of “wildcards”, you need to use an appropriate number of “/*” substrings (yes, it’s a pain, but maybe someone really wants to make sure that some library comes from some precisely defined directory).
For your example, in one of the previous posts, you would need:
root -l -q 'ListLibraries.cxx("/*/*/*/*/*/*Core*", kTRUE)'

Well, short / simple pieces of code (let’s say up to something like several tens of lines) could be “CC CC0”, and longer / complicated pieces of code could be “CC BY” → Wikipedia → Creative Commons license

Ok, but that’s a real pain, isn’t it ? plus this solution requires to add an extra parameter, and then the help will need to explain the diffrence between wilcard and regexp. Are we sure we want a such complexity for a such simple fonctionnality ?

Of course one other track will be to understand why ‘/’ are special in wilcard.
I’ll check.

That’s actually explained: TRegexp::MakeWildcard