Slow initialisation on locally mounted remote drive

ROOT
1

Hello,

I encountered a problem when using my pyROOT program, but it is likely to be a general ROOT problem.

It happens when I run my ROOT GUI python program in a remote directory mounted via sshfs on Linux (tested on Fedora 25 and Debian). The initialisation phase when importing different modules that use ROOT is very slow - a few seconds instead of a fraction of a second when I run the program in a local directory. I removed the remote directory from the sys.path, so it is not he python importing machine that is taking time. According to my prints, the most time was spent actually… between imports. So I used sysdig to find out what is happening. This is first time for me to use this tool, but it shows lot’s of access to the remote directory, for example:

1048 16:16:21.255648528 0 python (20165) < stat res=0 path=/home/lewhoo/opt/root/etc/plugins/TGuiFactory/P010_TRootGuiFactory.C
1049 16:16:21.255649397 0 python (20165) > stat
1262 16:16:21.261729545 0 python (20165) < stat res=-2(ENOENT) path=RtypesCint.h(/data/jem-euso-data/spb/cpu_remote/tmpdata/RtypesCint.h)
1264 16:16:21.261732598 0 python (20165) > stat
1268 16:16:21.261738951 0 python (20165) < stat res=0 path=/home/lewhoo/opt/root/etc/plugins/TGuiFactory/P010_TRootGuiFactory.C
1269 16:16:21.261739960 0 python (20165) > stat
3303 16:16:21.270365756 0 python (20165) < stat res=-2(ENOENT) path=iostream(/data/jem-euso-data/spb/cpu_remote/tmpdata/iostream)
3304 16:16:21.270369524 0 python (20165) > stat
3305 16:16:21.270376593 0 python (20165) < stat res=0 path=/home/lewhoo/opt/root/etc/plugins/TGuiFactory/P010_TRootGuiFactory.C
3306 16:16:21.270377584 0 python (20165) > stat
5135 16:16:21.274847164 0 python (20165) < stat res=-2(ENOENT) path=iostream.h(/data/jem-euso-data/spb/cpu_remote/tmpdata/iostream.h)

or

34425 16:16:21.564726971 2 python (20165) > getcwd
34426 16:16:21.564728459 2 python (20165) < getcwd res=43 path=/data/jem-euso-data/spb/cpu_remote/tmpdata
34427 16:16:21.564732466 2 python (20165) > access mode=4(R_OK)
34428 16:16:21.564737025 2 python (20165) < access res=-2(ENOENT) name=/home/lewhoo/geant/geant/lib64/libRIO.so
34429 16:16:21.564740517 2 python (20165) > access mode=4(R_OK)
34430 16:16:21.564744062 2 python (20165) < access res=-2(ENOENT) name=/home/lewhoo/jem-euso/spweuso/host/usr//euso/libRIO.so
34431 16:16:21.564747366 2 python (20165) > access mode=4(R_OK)
34432 16:16:21.564750487 2 python (20165) < access res=-2(ENOENT) name=/home/lewhoo/jem-euso/event_dump/libRIO.so
34433 16:16:21.564753390 2 python (20165) > access mode=4(R_OK)
34434 16:16:21.564757865 2 python (20165) < access res=0 name=/opt/root/lib/libRIO.so
34435 16:16:21.564759296 2 python (20165) > stat
34436 16:16:21.564762561 2 python (20165) < stat res=0 path=/opt/root/lib/libRIO.so

where cpu_remote is the remote directory mounted locally by sshfs. This happends to maaaany ROOT modules and it seems that only to them. I am using ROOT 5.34.36. Would anyone of the developers know a solution to this problem?

2

Hello,

It is not clear what would be the ROOT problem in here.
The number of file accesses are the ones required (you can replay the exercise locally and and should find the same).
Sshfs does encryption and perhaps compression which costs.
Here you have performance measurements and hints:

G Ganis

3

I think the ROOT problem is that during “import ROOT” it looks for many libraries in the current path. If I temporarily change the current path to some local one, the problem disappears.

I am not sure how ROOT performs a search for its libraries and why it does it in the local directory… Is it ROOT behaviour or some general OS thing?

4

I can add, that scipy and numpy, which also use lots of C code, do not look for files in the local directory…

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.