[Windows 10] ROOT 6.28 with Miniconda Python 3.10 slow import

ROOT
, ,
1

Continuing the discussion from [Windows 10] ROOT 6.28 build with Miniconda Python 3.10 crashed:

ROOT Version: ROOT 6.28/00 (64 bits)
Python: Miniconda3-py310_23.1.0-1-Windows-x86_64.exe
Platform: Windows 10 (20H2)
Compiler: Visual Studio 2022 (17.3.6)

Dear ROOT developers and maintainers,

I built ROOT 6.28 with Miniconda Python 3.10 on Windows 10 like in the mentioned topic. And here are the selected features:

-- Enabled support for:  asimage builtin_afterimage builtin_clang builtin_cling builtin_freetype builtin_llvm builtin_lz4 builtin_nlohmannjson builtin_pcre builtin_xxhash builtin_zstd dataframe fftw3 mlp minuit2 pyroot shared xml

However ROOT module loading time in Python seems particularly long:

(base) PS C:\URANIE\33846> Measure-Command {python -c "import ROOT"}


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 4
Milliseconds      : 435
Ticks             : 44352536
TotalDays         : 5,13339537037037E-05
TotalHours        : 0,00123201488888889
TotalMinutes      : 0,0739208933333333
TotalSeconds      : 4,4352536
TotalMilliseconds : 4435,2536



(base) PS C:\URANIE\33846>

A verbose loading seems to indicate a bottleneck at TMVA pythonization:

(base) PS C:\URANIE\33846> Measure-Command {python -vc "import ROOT"}
[...]
import 'ROOT._pythonization._tmva._crossvalidation' # <_frozen_importlib_external.SourceFileLoader object at 0x000002CFE61C8BB0>
# C:\URANIE\33846\uranie_master_win10_saclay\uranie\bin\ROOT\_pythonization\_tmva\__pycache__\_rbdt.cpython-310.pyc matches C:\URANIE\33846\uranie_master_win10_saclay\uranie\bin\ROOT\_pythonization\_tmva\_rbdt.py
# code object from 'C:\\URANIE\\33846\\uranie_master_win10_saclay\\uranie\\bin\\ROOT\\_pythonization\\_tmva\\__pycache__\\_rbdt.cpython-310.pyc'
import 'ROOT._pythonization._tmva._rbdt' # <_frozen_importlib_external.SourceFileLoader object at 0x000002CFE61C8CD0> # [Ctrl + C]
(base) PS C:\URANIE\33846> echo $?
False
(base) PS C:\URANIE\33846>

Removing the directory seems to solve loading issue:

(base) PS C:\URANIE\33846> mi C:\URANIE\33846\cache\outputs\root\bin\ROOT\_pythonization\_tmva .                        
(base) PS C:\URANIE\33846> Measure-Command {python -c "import ROOT"}


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 650
Ticks             : 6508886
TotalDays         : 7,53343287037037E-06
TotalHours        : 0,000180802388888889
TotalMinutes      : 0,0108481433333333
TotalSeconds      : 0,6508886
TotalMilliseconds : 650,8886



(base) PS C:\URANIE\33846> python -c "import ROOT"
(base) PS C:\URANIE\33846> $?
True
(base) PS C:\URANIE\33846>

Do you know how to explain this loading time issue?

Could you selectively define py2_py3_sources in root/CMakeLists.txt at v6-28-00 · root-project/root · GitHub?

In case it might be useful here is a temporary link to my ROOT installation with Miniconda: FileSender.

2

I think we’ll have to investigate

Why would you need to customize this?

3

Thank you for your answer.

Concerning the customisation of py2_py3_sources, here is the patch I used as a workaround for my special case (without TMVA): root_v6-28-00_tmva.patch.txt (1.4 KB)

The loading time issue seems to be solved then:

(base) PS C:\URANIE\33846\root_v6-28-00_b0f07d5c_py310_e3b0c442-3> gci .\outputs\root\bin\ROOT\_pythonization\_tmva
gci : Impossible de trouver le chemin d'accès «
C:\URANIE\33846\root_v6-28-00_b0f07d5c_py310_e3b0c442-3\outputs\root\bin\ROOT\_pythonization\_tmva», car il n'existe
pas.
Au caractère Ligne:1 : 1
+ gci .\outputs\root\bin\ROOT\_pythonization\_tmva
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (C:\URANIE\33846...onization\_tmva:String) [Get-ChildItem], ItemNotFound
   Exception
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetChildItemCommand

(base) PS C:\URANIE\33846\root_v6-28-00_b0f07d5c_py310_e3b0c442-3> . .\outputs\root\bin\thisroot.ps1
(base) PS C:\URANIE\33846\root_v6-28-00_b0f07d5c_py310_e3b0c442-3> Measure-Command {python -c "import ROOT"}


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 654
Ticks             : 6548666
TotalDays         : 7,57947453703704E-06
TotalHours        : 0,000181907388888889
TotalMinutes      : 0,0109144433333333
TotalSeconds      : 0,6548666
TotalMilliseconds : 654,8666



(base) PS C:\URANIE\33846\root_v6-28-00_b0f07d5c_py310_e3b0c442-3>

I suppose enabling tmva-pymva option might cause the loading time issue again.

4

OK, so that would be interesting to see if the issue exists on other platforms. Maybe @moneta or @vpadulan could be interested to give it a try?

5

And BTW, I think what you propose for modularizing the bindings/pyroot/pythonizations/CMakeLists.txt make sense. I’ll check with @vpadulan and @Axel what we can do there.

6

Thank you for your messages.

I took inspiration from dataframe section: root/CMakeLists.txt at v6-28-00 · root-project/root · GitHub.

7

Yes, I’ve seen that. I think we should do the same for TMVA (PyMVA) and roofit

8

Sorry. I didn’t pay attention to [PyROOT] Only build RooFit and TMVA pythonizations if needed · root-project/root@d759a42 · GitHub.

9

Oh you’re right! I guess we can backport this to v6-28-00-patches…

10

Hello,

In a derived project of ROOT, I investigated a class which is similar to MsgLogger.

And the root cause of Python loading time issue might be related to std::ostringstream inheritance:

  • Derived project:
14:17 $ git grep "public std::ostringstream"
dataSERVER/souRCE/UMessageLogger.h:class UMessageLogger: public std::ostringstream, public TObject
✔ /[...]/33846/souRCE [master|…5]

14:17 $ git grep "public std::ostringstream"
core/foundation/inc/ROOT/RLogger.hxx:class RLogBuilder : public std::ostringstream {
tmva/tmva/inc/TMVA/MsgLogger.h:   class MsgLogger : public std::ostringstream, public TObject {
✔ /[...]/33846/root [v6-28-00|⚑ 1]

Further investigation may support or reject this hypothesis.

11

Thanks for the investigations! Maybe @moneta would be interested

12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.