Lazy snapshots, callbacks and speed

Good evening,
First of all thanks for resolving ROOT-9468! I saw a huge speedup (x3) in my current test setup.

As I went ahead to modify my setup and draw a progress bar like in df013, I saw that it slowed down. The callback (drawing the progress bar) is not supported from the snapshot action directly. It throws a runtime error

what():  This action does not support callbacks!

So I tried a workaround solution with a dummy histogram that is filled during the event loop and on which the callback is used. The timing then got up from 10s to 37s. Is this expected? Also, is it planned to implement callbacks for snapshots?
You can find my code attached. Sorry, it’s not standalone at the moment, but I can make it if needed.
tree_trimmerRDF.cpp (5.3 KB) (line 124ff.)
Maybe I wasn’t using the lazy actions correctly in the first place?

Cheers,
Marian


ROOT Version: current nightlies
Platform: x86_64-slc6
Compiler: g++ 7.3.0


Hi Marian,
always nice to see our work appreciated! :smiley:

The timing then got up from 10s to 37s. Is this expected?

Your event loop is multi-threaded, and your callback uses a mutex, which effectively synchronizes the execution of the callback from different threads. I bet this synchronization is responsible for the slowdown you see: try to remove the mutex (which makes the callback thread-unsafe, as is) and see if that “fixes” it.

Is it planned to implement callbacks for snapshots?

Unfortunately not, because we can’t produce a partial result for a Snapshot action. We don’t have a “partially ready new RDataFrame”, although we can provide partially filled histograms.
However you can simply use a Count action to get a RResultPtr on which it’s easy to call OnPartialResult.

N.B.
The snippet you linked has the part that actually starts the event loop commented, so if I read correctly, as it is now, if you execute that code it will not run an event loop at all – but I guess when you ran it that part was uncommented :smiley:

Hope this helps!
Cheers,
Enrico

Hi Enrico,
The 10s came in fact from the code as it is, so no event loop was running and the time doesn’t make any sense :man_facepalming:
Thanks for the hint with the mutex and the Count action. I will test them tomorrow and report back.
Thanks,
Marian

Hi Enrico,
Using the Count action like you recommended (attached) works perfect. I don’t not see any difference in timing between the script with and without progress bar now. This also means that the mutex doesn’t slow the script down. The factor 3 speedup I mentioned measured the pure time for JIT compilation and is a great improvement!
Sorry for the noise, and thanks for your quick help!
Marian
tree_trimmerRDF.cpp (5.4 KB)

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.