Better definitions and illustrations for boxplots

Dear Rooters,

I wrote the below written idea to roottalk@cern.ch, but Olivier Couet ask me to start the topic over here, so I copied the original message. The original post included box.h and box-example.C, but Olivier corrected some little bugfixes to make it run in root 6 as well, the bugfixes are included here - thank you very much

Original Post:

Dear root-developers, dear users,

I learnt the advantages of root when I worked for HEP-experiments in the
past. Nowerdays I’m no longer working for HEP but still like to use root
for process-control purposes of larger datasets

The people in the company I’m working for like box-plots. So I started to
implement first classes creating boxplots on top of TH2s. Since a while
the “candle” option is available for the Draw-method, but I’m not
satisfied with it. Reasons:

  • The Definition of the candle could be improved (especially the length
    of the whiskers and the non-existance of so called outliers
  • no grouping if possible. I like to use trend-charts (some value vs the
    time. When using grouping you can compare two values vs. the time, see my
    example)

I would like to provide you a simplified version of the boxplot-class I’m
using. I have anther complilable version running under Linux, but this
here runs in CINT without compilation (I have tested it with root 5.32 and
5.34)

I would appreciate it very much if you could think about to implement
these features in a better way by using THistPainter (what I’m doing here
is a bit -poor- :slight_smile:)

In fact there are several more Features one can implement into a candle
such as:

  • Kinks in the candle to show the significance of the median
  • Show all values in the candle (underlying scatter plot)
  • Show a vertical histogram next to each candle
  • Possiblity to scale the box-width
  • Write numbers on at the side of the candle to provide the numbers for
    mean, median, iqr, etc…

Please tell me what you think about this, and let me know. If you are
interested I can provide some code also

Thanks
Georg
box.h (9.76 KB)
box-example.C (804 Bytes)

Hi Georg,

Thanks for this post, Do you think you could do the implementation in THistPainter yourself ?
The only method to modify is:

void THistPainter::PaintCandlePlot(Option_t *)

May be there will be also change in the option management but I can help …

Olivier

Hi Olivier,

I just had my very first look inside the code of THistPainter, although I do not fully understand the mechanismn (how the h1->Draw() gets into the THistPainter :slight_smile: ) I think that the changes are relatively straight ahead - I think I can handle this.

But we need to think about what we want. Do you want to overwrite the old candle-function? Do you want a candle1 candle2 …??

Much more difficult is the question how to handle the boxes in groups: as you can see from my example, the boxes are getting slimmer when adding more histograms.
I’m not sure if one can do this with the “same” option, because you then have to modifiy the former histo-illustrations as well - is is possible?
What do we expect from the user to write in that case? h1->Draw(“candle1x”); h2->Draw(“samecandle1x”); or is it enough to write same in order to let the second one appear as candle, too.

Another possibility might be to adapt a class which is using a list of histograms such as THStack, here you can omit the “same”-stuff and you have a simple draw. Or is this also running though THistPainter? I didn’t find out how, so far…

I think as important as the re-coding of the candle-definition is the GUI-Edititor, will you handle that part, when I am ready?

Georg

Hi Georg,

You do not really need to know the details how your h1 will bee seen there. You only need to know that it will be seen as “fH” inside PaintCandlePLot :slight_smile:

Yes we will very likely need several options.

The “SAME” approach is one way. We do that already with the BOX option in order to make the new BOX plot in proportion with the previous ones.

Yes it is the other way. The stacks are painted by THStack::Paint(Option_t *option).
This method can use the PaintCandlePlot method from THistPainter.

Yes once the new options are in place I will modify the GUI to handle them.

Hi Olivier,

my very first version of the a new THistPainter is ready, but I’m still not happy with the grouping. I’m searching for a way that the candles are drawn in the middle of the bin for the first call of h1->Draw(“candle”), but when calling h2->Draw(“candle same”) I want that the first histo is redrawn in the left part of the bin and the second histo occurs in the right part. Do I have a chance to redraw a former histo from THistPainter??

I had a closer look in the example of the bar-option, here this is realized in a very ugly way as well, I would prefer a better way (as written in the top), Maybe one could think about another option word like “aside”, “beside” or just “side” instead of “same”

Georg

No … Not really… Of course everything can be done but I would not suggest to do that.
I think the best way t handle this if to implement the CANDLE option for THStack.

The Bar option it is “ugly”… That’s simply not done automatically. You can draw several bars on top of each other if you want. The new plot drawn with option SAME ignored what as been drawn before. You can do that too…

But as you are looking for something automatic, the best way will be THSTack seems to me.

Hi Georg,

I thought that the mechanism of the bar option with the BarWidth and the BarOffset combine with the option same is also a good approach. It can be implemented in THistPainter. At drawing time you just need to take care of this two parameters. That’s a simple approach,

Hi Olivier,

yes that is one possible way, but I would like to implement that method using the THStack, too. I will try to figure out how it works in the afternoon. I can only run root6 at home :slight_smile:

Do I understand it correctly, that I only have to check that (fStack != 0)?? And that I can access the individual histos of the Stack by using fStack->At(0) e.g.?

Georg

Yes I think we need both. The “bar way” is nice because it is more general in the sense you can draw other things between each drawing of histogram. The stack way is good when you want to compute the whole scale including the complete set of histograms. May be start with the “bar way” it is simpler to implement.
In a THStack the histograms are stored in a TList. You access the individual histograms doing something like:

         h1 = (TH1*)fStack->At(nhists-i-1); // line 874 in THSstack.cxx

Hi Olivier,

the code is 90% ready. Unfortunately I still have problems with the stack-stuff. in the MakeChopt function I do get a pointer to fStack, but in PaintCandlePlot the pointer is always NULL. I did not found out so far where the reset occurs. I checked how this is done in PaintLego, but it seems the programmer accesses fStack here as well…??? Maybe here some magic done by TPainter3dAlgorithm??

What is wrong here? How can I access fStack from PaintCandlePlot?

Georg

What are you modifying ? THistpainter or THstack ?
May be we should do the two way one after each other. 1st the Bar-width way and then the Stack way.
I suggest we concentrate 1st on the Bar-Width way which is much easier. This will also allows us
to define which extra options we need… if any … For this technique you need to modify only THistPainter.

Hi Olivier,

I attached a changed version of THistPainter. You can use the diff-file if you want (I created it from the root6 Branch).

One can easily enable or disable features of the candle by using the bits in the Hoption.Candle parameter.
At the moment I implemented an opt(…) Parameter, so that one can set all bits indiviually (look at the example), we can change this in the future of cause. I think it will be pretty easy for you to attach a gui to this, you can simple activate or deactivate individual bits.

we should think about to predefine more standard-candles like candlex1, candlex2,… it would be possible to attach something like “candlex2+outliers” or “candlex5+scatter”

I attached a detailed documentation in the source-code. Maybe you will prefer to move the PaintOneCandle into an indiviual class (I didn’t know what you prefer here), please decide yourself

Not done so far is:

  • A solution for THStack
  • A would like to implement a combinded candle + violin plot (showing a candle and beside half of a violin)

Would be nice to get these Features for root5 as well :slight_smile:
Please let me know, if you need something, or if I can help

Georg
commit-newcandle.diff (25.2 KB)
THistPainter.cxx (378 KB)
THistPainter.h (6.87 KB)
boxplotexample.C (1.97 KB)

Thanks Georg,

I will look at it and let you know.
Cheers,
Olivier

Hi Georg,

Since you changed THistPainter a new option COL2 has been added therefore the.cxx and .h are out of date regarding this option. Can you put your mods in the very new version of THistPainter ? I will ease the process.
Thanks.

Olivier

Hi,
Thats why I send you the patch-file which was created with git, but I will try to find the time to update this night

Georg

Yes please that would be simpler.

Hi Olivier, Hi ROOTers,
took me a couple of days to explain my internet service provider “O2-Germany” that the cern homepage really exists and I am not too dumb to spell “cern.ch”. Indeed the whole cern-network was not available for a couple of days from O2-germany. “Yes, sure, the internet is working, only one page is not, yes sure…” argh!!!

…finally, I was able to pull the latest master-trunk from git. I patched the THistPainter.cxx and .h as explained before. Please use the example from the post above to check

Please tell me if you need something.

Georg
THistPainter.h (7.06 KB)
THistPainter.cxx (377 KB)
commit-candle-2016-02-29.diff (25.2 KB)

Hi Georg,

Thanks for your last post. I have been busy end of last week with other matters and bugs fixes. I have still one to complete. As soon as it is done I will be back to your code.

Cheers,
Olivier

Hi Georg,

I have executed you example it gives me this ice plot. I need now to work a bit on you code.


The new CANDLE plot mechanism is now in THistPainter. See the doc for the details.