Not refreshed jsroot graphics within jupyter notebook v7 / jupyterlab

malfonsi79 · March 10, 2025, 8:04am

Dear ROOTers,

As I changed from the old Debian 10 to Debian 12, I rebuilt my software stack to have “ROOT notebooks” on the new machines, using a virtual-env to have to proper python packages under my control and compiling from sources ROOT refering to this python environment.

Every notebook that was working in the Debian 10 environment correctly works in the new environment, with a single, very annoying bug: jsroot graphics disappears (or better it becomes a grey box) as soon as, e.g. by scrolling the browser down and back up, it gets out of the “browser painted view” and back to it, see pictures:

Please note that the jsroot graphics is not lost or broken, it is still there, it is just not repainted by the browser. In fact if I find a way to force repainting, e.g. passing to full screen mode or back, the jsroot graphics block shows up again with its full interactivity. This is happening with Firefox, Chromium and also Microsoft Edge, both if I have a local browser or a ssh tunnel to connect a local browser to a remote jupyter server.

The big difference between the two environments is that on Debian 12 my python stack uses a notebook version > 7, which I learned is based on the jupyter lab engine. I searched the forum and it seems that jupyterlab was a headache initially, but then all issues were solved since at least 3 years (JsRoot in Jupyterlab - #4, jsroot and JupyterLab · Issue #166 · root-project/jsroot · GitHub, [jupyter] configure ServerApp to let use "lab" properly by linev · Pull Request #8538 · root-project/root · GitHub).

I cannot find much more information on the forum and googling is even worse because you find basically only zillions of discussions on jupyter in general and the few in which root or jsroot is included point to the topics in the ROOT forum already found.

I am wondering if it is related to how browsers cache the page contents, but my knowledge is very limited in this aspect so I do not know how to investigate more (I read that you can open a console in a browser, but I need some more hints to know what I should look for there)

Also downgrading the notebook version to something < 7 is having a bit of issues because of packages compatibilities with the Debian 12 python version. I guess that at the end I can find the right combination of python packages by trial and error, but, still, having the last python package and a notebook version not doomed to obsolescence is way way way more preferable. For this reason any hint to solve the issue with jupyterlab is very very welcome

Thanks,
Matteo

_ROOT Version: 6.32.04 and 6.34.02
Platform: Debian 12.9
Compiler: gcc (Debian 12.2.0-14) 12.2.0

Danilo · March 10, 2025, 8:10am

Hi Matteo,

Thanks for the post.
Let me start by adding @linev in the loop because of his expertise with jsroot and browsers.
In the meantime, I tried to reproduce the issue on macOS (firefox, chrome and safari with no luck).

Cheers,
D

malfonsi79 · March 10, 2025, 8:33am

Dear @linev

on the GSI lxpool, you should be able to reproduce the issue by sourcing:

source /cvmfs/sfrsbi.gsi.de/debian12/2025/python3/bin/activate
source /cvmfs/sfrsbi.gsi.de/debian12/2025/ROOT6_34_02/bin/thisroot.sh

I can also provide separately one of the ipynb with many jsroot graphics blocks on which I see this issue (just in case the issues are on my notebooks, but I cannot find a reason why, I just Draw() and they were working with the old notebook). You cannot run it, but the jsroot block are embedded in the notebook so you should be able to see them as soon as you open and trust the notebook.

Please let me know,
Matteo

linev · March 10, 2025, 9:41am

Hi Matteo,

Can you post exact code of your notebook here?
With simple histogram drawing I cannot reproduce problem - also on GSI lxpool cluster.

Since a while I using and testing ROOT with jupyter lab 4.3.5 - which is very close to installed on GSI cluster.

About TBox drawing - you can make it simpler by doing:

b = ROOT.TBox(-2, 0, 2, 50)
b.SetFillColor(5)
b.Draw()
c.Draw()

Regards,
Sergey

linev · March 10, 2025, 9:51am

Actually, playing further I can reproduce problem in chrome browser.
And switching between full screen and normal mode helps me as well.
I will try to investigate it now.

linev · March 10, 2025, 12:37pm

Hi Matteo,

It is unfortunate behavior of chrome browser.
If HTML element is not in the scroll area, width and heigh of such element cannot be evaluated properly. I provide a fix which should help to overcome such problem.

But it will take time until next patch release of ROOT 6.34.

As alternative, you can checkout 7.8 branch of JSROOT and configure JSROOTSYS environment variable like:

export JSROOTSYS=/cvmfs/sfrsbi.gsi.de/debian12/2025/jsroot_7_8/

Then ROOT and jupyter will use that JSROOT version where problem should be fixed.

Regards,
Sergey

malfonsi79 · March 10, 2025, 2:48pm

Dear Sergey,

thanks a lot for a so quick fix! I would like to ask for a bit of guidance to apply your suggested workaround:

linev:

As alternative, you can checkout 7.8 branch of JSROOT and configure JSROOTSYS environment variable like:
export JSROOTSYS=/cvmfs/sfrsbi.gsi.de/debian12/2025/jsroot_7_8/
Then ROOT and jupyter will use that JSROOT version where problem should be fixed.

because it seems that I am doing something wrong and the variable is ignored, in the screenshot below the chromium console mentions jsroot 7.7.4 and it should be 7.8.1 (actually dev or master):

This is what I am doing:

I git-cloned the jsroot master (in git-log the fix is mentioned, so I guess master is what I need, and it is the same as the ‘dev’ tag)
I cloned it in a local dir on the machine, no need I guess to pass through cvmfs transactions until I manage to have it working… a local dir must work the same
the jsroot repository is all javascript and html, so at the best of my poor JS knowledge, nothing must be compiled or built, it is used as it is. This is also what I understand from the README
I source the two scripts (the python activate and the ROOT thisroot.sh) and THEN I set JSROOTSYS environmental variable to the cloned git repository.
I erased the ~/.rootnb_sfrsbi_deb12 directory (I patched nbmanin.cc before compiling ROOT to have a different hard-coded name for the ~/.rootnb notebook settings dir)
I launch root --notebook --no-browser and I open the link in a local chromium (I have a ssh tunnel to the remote machine)

When I inspect the recreated ~/.rootnb_sfrsbi_deb12/jupyter_notebook_config.py:

import os
rootbin = '/cvmfs/sfrsbi.gsi.de/debian12/2025/ROOT6_34_02/bin'
rootlib = '/cvmfs/sfrsbi.gsi.de/debian12/2025/ROOT6_34_02/lib'
os.environ['PYTHONPATH']      = '%s' % rootlib + ':' + os.getenv('PYTHONPATH', '')
os.environ['PATH']            = '%s:%s/bin' % (rootbin,rootbin) + ':' + os.getenv('PATH', '')
os.environ['LD_LIBRARY_PATH'] = '%s' % rootlib + ':' + os.getenv('LD_LIBRARY_PATH', '')
c.NotebookApp.extra_static_paths = ['/cvmfs/sfrsbi.gsi.de/debian12/2025/ROOT6_34_02/js/']

it seems that my request was ignored for some reason. I will try (but later) to explicitly edit the last line with the new jsroot version path, and I will let you know.

linev · March 10, 2025, 2:54pm

Hi,

Normally I start jupyter lab to get jupyter running.
If you still using jupyter_notebook_config.py file, then you should change following line:

c.NotebookApp.extra_static_paths = ['/cvmfs/sfrsbi.gsi.de/debian12/2025/jsroot_7_8/']

JSROOT 7.7.4 is fallback version for ROOT 6.34 when jupyter static path is not working.

Regards,
Sergey

linev · March 10, 2025, 3:09pm

One more comment.

Please check your ROOT build directory, etc/notebook/jupyter_notebook_config.py file.
It looks different now. There JSROOTSYS variable is taken into account.
Also one probably need to configure c.ServerApp.extra_static_paths to be usable with jupyter lab.

malfonsi79 · March 10, 2025, 10:12pm

Hi Sergey,

I hard-coded in the ~/.rootnb_sfrsbi_deb12/jupyter_notebook_config.py the newest jsroot version:

[...]
#c.NotebookApp.extra_static_paths = ['/cvmfs/sfrsbi.gsi.de/debian12/2025/ROOT6_34_02/js/']
c.NotebookApp.extra_static_paths = ['/data.local1/malfonsi/Software/jsroot']

However it seems that for some reason the proposed local version is refused by the browser, and this was happening also in the previous test, see the console output in the earlier screenshot, it says:

Refused to execute script from 'http://localhost:8888/static/build/jsroot.js' because its MIME type ('text/html') is not executable, and strict MIME type checking is enabled.

VM400:43 Fail to load JSROOT locally, please check your jupyter_notebook_config.py file

And it falls back to the official version on the web https://root.cern/js/7.7.4/build/jsroot.js

Any idea why is the local version refused? What are these MIME types? If I completely clear all the output from the notebook and restart the kernel, at the first canvas painting I get also:

VM704:14  GET http://localhost:8888/static/build/jsroot.js net::ERR_ABORTED 404 (Not Found)

just before the other two messages in the console.

I need more hints to understand your comment:

maybe this is the key of the problem

Moreover, I learned also few more things.

I thought that this “user” ~/.rootnb/jupyter_notebook_config.py was a copy of the jupyter_notebook_config.py in the ROOT installation, but I did not go through main/src/nbmain.cxx with enough attention. When the “user directory” is not existing, the function InstallNbFiles(source, dest) indeed copies all files in the newly created “user directory”, but then the function CreateJupyterConfig(dest, rootbin, rootlib, rootdata) completely rewrite jupyter_notebook_config.py from scratch. In other words, when starting with root --notebook the JSROOTSYS variable seems to play no role because the jupyter_notebook_config.py in the ROOT installation is not used.
I cannot launch jupyter notebook or jupyter lab in place of root --notebook, because it would try to write in the read-only cvmfs ROOT dirs.

$ jupyter notebook --no-browser
Traceback (most recent call last):
  File "/cvmfs/sfrsbi.gsi.de/debian12/2025/python3/bin/jupyter-notebook", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/cvmfs/sfrsbi.gsi.de/debian12/2025/python3/lib/python3.11/site-packages/jupyter_server/extension/application.py", line 616, in launch_instance
    serverapp = cls.initialize_server(argv=args)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cvmfs/sfrsbi.gsi.de/debian12/2025/python3/lib/python3.11/site-packages/jupyter_server/extension/application.py", line 586, in initialize_server
    serverapp.initialize(
  File "/cvmfs/sfrsbi.gsi.de/debian12/2025/python3/lib/python3.11/site-packages/traitlets/config/application.py", line 118, in inner
    return method(app, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cvmfs/sfrsbi.gsi.de/debian12/2025/python3/lib/python3.11/site-packages/jupyter_server/serverapp.py", line 2784, in initialize
    super().initialize(argv=argv)
  File "/cvmfs/sfrsbi.gsi.de/debian12/2025/python3/lib/python3.11/site-packages/traitlets/config/application.py", line 118, in inner
    return method(app, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cvmfs/sfrsbi.gsi.de/debian12/2025/python3/lib/python3.11/site-packages/jupyter_core/application.py", line 256, in initialize
    self.migrate_config()
  File "/cvmfs/sfrsbi.gsi.de/debian12/2025/python3/lib/python3.11/site-packages/jupyter_core/application.py", line 185, in migrate_config
    migrate()
  File "/cvmfs/sfrsbi.gsi.de/debian12/2025/python3/lib/python3.11/site-packages/jupyter_core/migrate.py", line 243, in migrate
    with Path.open(Path(env["jupyter_config"], "migrated"), "w", encoding="utf-8") as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/pathlib.py", line 1045, in open
    return io.open(self, mode, buffering, encoding, errors, newline)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 30] Read-only file system: '/cvmfs/sfrsbi.gsi.de/debian12/2025/ROOT6_34_02/etc/notebook/migrated'

I would have to launch with:

JUPYTER_CONFIG_DIR=~/.rootnb_sfrsbi_deb12 JUPYTER_PATH=~/.rootnb_sfrsbi_deb12 jupyter notebook

but this is exactly what root --notebook is doing.

Finally, I attach here the jupyter notebook
AGG001lowscan0V86-Copy1.ipynb (2.0 MB)

linev · March 11, 2025, 7:08am

Hi Matteo,

Refused to execute script from ‘http://localhost:8888/static/build/jsroot.js’ because its MIME type (‘text/html’) is not executable, and strict MIME type checking is enabled.

This message caused by 404 error from http server.

Your configuration is still incomplete. To work with jupyter lab you have to configure c.ServerApp.extra_static_paths variable. Once jupyter correctly returns http://localhost:8888/static/build/jsroot.js, problem should be resolved for you.

Regards,
Sergey

malfonsi79 · March 12, 2025, 10:18am

Dear Sergey,

after testing with Chromium, Firefox and also Microsoft Egde (always my laptop local browser with ssh port forwarding from the remote server, I do not sit in front of the machine where the server is running - but I guess it does not matter), I confirm that the last jsroot solves the problem. There is still a glitch on the fonts when I reopen the notebook, which gets solved as soon as I interact with the block (e.g. zooming in and out):

But this is just cosmetics and it does not hinder the work. So many, many, many thanks!!!

What I did is to insert the mentioned lines from $ROOTSYS/etc/notebook/jupyter_notebook_config.py into the “user” ~/.rootnb/jupyter_notebook_config.py. In this way the logic with the environmental variable JSROOTSYS is respected.

I wonder why nbmain.cxx does not produce the “user” ~/.rootnb/jupyter_notebook_config.py copying from the version in $ROOTSYS/etc/notebook/. Maybe the reason is another, but still in the next days I want to try to apply this patch:

nbmain_for_jupyter_notebook_config.patch.txt (1.6 KB)

and rebuild ROOT. The idea is that calling root --notebook prepares directly the config file as we discussed here. I will write again here the result of the test.

malfonsi79 · March 25, 2025, 2:36pm

@linev @Danilo

I would like to add that the proposed patch seems to work (after 12 days testing).

It basically relies on the jupyter_notebook_config.py that is produced in the $ROOTSYS/etc/ directory during build.

I was initially afraid about the Windows version (which I cannot test) because of the backslash path separator, but I learn that python can handle the OS details as soon as you keep the normal slash in the path (and actually nbmain.cc is converting Windows backslashes to python convention).

It might well be that there are other reasons why it cannot be applied in general, but if I managed to be of any help I would be very happy.

Ciao,
Matteo