Colored 2D histograms in pdf output

Many thanks Alberto,
What would be the comment of for the picture ?

One of the best workarounds to avoid the bad-rendering of the pdf is to save your canvas as .tex and then compile it with pdflatex.

E.g.

TCanvas* c = new TCanvas();
TF2 *f2 = new TF2("f2","xygaus + xygaus(5) + xylandau(10)",-4,4,-4,4);
Double_t params[] = {130,-1.4,1.8,1.5,1, 150,2,0.5,-2,0.5, 3600,-2,0.7,-3,0.3};
f2->SetParameters(params);
TH2F h2("h2","xygaus + xygaus(5) + xylandau(10)",100,-4,4,100,-4,4);
h2.FillRandom("f2",1000000);
h2.Draw("COLZ");
c->SaveAs("a.tex");
c->SaveAs("b.pdf");

And then in latex (see https://root.cern.ch/drupal/content/saving-canvas-tex):

\documentclass{article}
\usepackage{tikz}

\begin{document}
\begin{figure}
\begin{center}
\scalebox{0.31}{\input{a.tex}}\includegraphics[scale=0.31]{b.pdf}
\caption{Left: Image generated thanks to TTeXDump (save canvas as .tex) and tikz. Right: Generated directly through pdf export (save canvas as .pdf) and includegraphics.}
\end{center}
\end{figure}
\end{document}

See attachments for comparing both outputs. The left figure does not suffer from bad rendering and is however not bitmap-based (in contrast to matplotlib).

If you use too many histogram bins, there can be a problem with the pdftex maximum memory capacity.

If you import h2.pdf to inkscape and compare the xml-code of a given rectangle-color bin, you get:

Left (tikz good rendering): fill:#6300ff;fill-opacity:1;fill-rule:nonzero;stroke:#6300ff;stroke-opacity:1;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:10;stroke-dasharray:none
Right (pdf bad rendering) fill:#6300ff;fill-opacity:1;fill-rule:evenodd;stroke:none



h2.pdf (216 KB)

Right (pdf bad rendering)

Seems to me it is good …

Hi all,
There is a big difference to me. Well, to my computer, better said.
It is mac Preview (in all its forms, i.e., the web plugin, the printer output, etc.)
which is producing the bad rendering with white lines among the bins.
This is a very important issue to me because it is the only remaining thing to feel satisfied with the outlook of ROOT’s plots.
I am preparing a long article which is full of colored 2D histograms and they look very nice apart from this fact.
Every colleague of mine tells me about these white lines. They like the plots, but they find funny these lines.
Given the popularity of the OS X, I really think that it would be very worthy to try to get rid of this problem.
If I try e.g. png output, the white lines are gone, but I get ugly rendering for the lines, markers and text labels.
When I say ugly I mean not that nice and smooth like the scalable graphics.

The Ferhue user seems to have managed to get rid of this problem, but his workaround implies that one has to convert to .tex and then compile from there to pdf, which makes the task kind of arduous to do it systematically for all the plots.
Besides (from his example two posts away), it seems that his trick does not respect the label fonts.

I wonder what is pdflatex doing to produce a better output than root itself.
Ferhue showed by means of inkscape that the xml-code is different in both cases.
Perhaps this could be the key to implement some option in ROOT that produces the same.

What do you think? Anybody is up to try to solve this once for all?
If you at least point out what we would have in the ROOT code to fix this problem, I could try to hack it myself.

Thanks and cheers!

That’s a smoothing bug in preview… If you turn off the “Smooth text and Line art” in the Preview Preferences then the white lines are gone. If you open your file with Adobe Acrobat Reader (which is the reference because pdf comes from Adobe), there is no white lines.
When you draw an histogram with option COL the boxes are drawn next to each other. No line are drawn… These line appear in Preview when the soothing is on. That’s a bug in Preview and I cannot do anything about it.
If you print you file it is also fine…
The only people who can fix that are the Apple’s Preview developers.

Dear Oliver,
I don’t agree. It is true that the white lines are only present in Preview when the smoothing is enabled and that they are not present in Adobe Acrobat, but this does not mean that we can’t do anything.
Ferhue showed in this post [url]Test other language
that the lines are gone in Preview if one produces a pdf from the canvas saved in tex format.
He also showed that there is a difference in the xml code of both the pdf produced directly from root and the one from tex.
The last one is rendered perfectly in Preview while the first one isn’t.
That means that something can be done with the attributes of the fill areas (the bins) when creating the pdf from the canvas. Something that produces an output compatible with the Preview rendering.
I guess that we would just need to change some defaults in root code when printing colored areas in a pdf.
Perhaps, changing this will solve the problem in Preview while not affecting the behavior in other viewers.
I have been exploring in TPDF.cxx, but I couldn’t find anything…
Would you tell me which class is the responsible of actually drawing the 2D histograms with col option into a pdf?

I need to fix this because one can’t use Preview without the smoothing: Everything becomes ugly!
I have also noticed that there are no white lines when one draws the histogram with the “cont” option.
In any case I’ll be glad if you can tell me where to look in the ROOT code to try to fix it.

Cheers

I agree it is ugly. The recipe going via tex makes a complete new pdf file… therefore it is difficult to compare… I do not know what this conversation does (may be some bitmap stuff) … The COL option rendering is just boxes next to each other. You will find the code in THistPainter::PaintColorLevels …
If you find a solution it is welcome …

I have to say that this pdf bad rendering in the “pdf output” also affects all Linux pdf readers like “evince”, “atril”, “inkscape”, not only MAC’s “preview”.

In the meanwhile, I have made some improvements for .tex workaround:

  • With pdflatex, if the 2D histogram had a lot of bins, then the conversion “crashed” due to exceeded memory capacity. The trick is to use “lualatex” instead, which does not have that problem and produces equivalent plots. It is already installed by default eg in Linux. Just change in the command line the program you execute.

  • I have created a simple macro to automize the “.tex” to “.pdf” conversion, maybe delaossa you are interested for doing this in all your plots. You can adapt it to your needs.

Execute as root -l -b -q autoconvert.cpp

/**
 * \brief Save a canvas as .tex, then convert it to .pdf via pdflatex or lualatex
 * \param c pointer to the TCanvas
 * \param output the output name (.pdf)
 * \note the intermediate tex figure and skeleton are also saved (.tex) in /tmp. Adapt to your needs this function.
 */
void SaveAsTikz(TCanvas* const c, const TString output)
{
	if(!c) return;
	if(!output.EndsWith(".pdf",TString::kIgnoreCase)) return;
	
	const TString texName = "/tmp/"+TString(output).ReplaceAll(".pdf",".tex");
	const TString skelName= "/tmp/"+TString(output).ReplaceAll(".pdf","_skeleton.tex");
	
	c->SaveAs(texName);

	//Now we create the skeleton tex file
	TString s = 
	"\\documentclass[crop,tikz]{standalone}\n"
	"\\usepackage{tikz}\n"
	"\\usetikzlibrary{patterns,plotmarks}\n"
	"\\begin{document}\n"
	"\\input{";
	s+=texName+
	"}\n"
	"\\end{document}";

	cout << texName << " " << skelName << endl;
	ofstream g(skelName);//or "/tmp/" + tmpName
	g << s << endl;
	g.close();
	
	//gSystem->ChangeDirectory("/tmp");//To use sth like that if you want to change the subdirectory structure
	gSystem->Exec("lualatex -job-name="+output+" "+skelName);
}

void autoconvert()
{
	TCanvas* c = new TCanvas();
	TF2 *f2 = new TF2("f2","xygaus + xygaus(5) + xylandau(10)",-4,4,-4,4);
	Double_t params[] = {130,-1.4,1.8,1.5,1, 150,2,0.5,-2,0.5, 3600,-2,0.7,-3,0.3};
	f2->SetParameters(params);
	TH2F* h2 = new TH2F("h2","xygaus + xygaus(5) + xylandau(10)",100,-4,4,100,-4,4);
	h2->FillRandom("f2",1000000);
	h2->Draw("COLZ");
	SaveAsTikz(c,"a.pdf");
	
	//~ c->SaveAs("a.tex");
	//~ c->SaveAs("b.pdf");
	
}

Additionally, if you use a latex document for your paper, you can skip this workaorund, just call c->SaveAs(".tex") and directly insert the figures in your code as “.tex” instead of previously converting to pdf. See “tikzexternal” for not slowing down your compilation time. I have chosen the mode “list and make” and it works flawlessly.

\tikzexternalize[mode=list and make, prefix=i/]

Couet, concerning the issue of the pdf rendering, I have analysed more in depth the problem via the xml code of inkscape. (Open h2.pdf as inkscape and then open xml editor). I have chosen the same bin rectangle (the most right bottom one) in the respective figures of h2.pdf.

Left plot (tikz)

d m 477.92722,63.05365 0,2.69125 3.96855,0 0,-2.69125 -3.96855,0 z
id path690
style fill:#9200ff;fill-opacity:1;fill-rule:nonzero;stroke:#9200ff;stroke-opacity:1;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:10;stroke-dasharray:none
rectangle width x height = 1.692x1.197

Right plot (save as pdf)

d m 477.921,63.0529 3.9685,0 0,2.69128 -3.9685,0 0,-2.69128 z
id path41838
style fill:#9200ff;fill-opacity:1;fill-rule:evenodd;stroke:none
rectangle widthxheight = 1.538x1.043

So I think that the issue is clear, the rectangles in tikz are bigger in size than in saveaspdf, so that they partially overlap each other and make the white lines disappear. The question is, why are the rectangles in tikz bigger? The trick is the fill style: they have a “border stroke line” instead of just the interior fill. This makes them bigger. So, if one just changes the style fill recursively in the xml adding the same stroke as in tikz, then the new rectangle WxH = 1.692x1.197, i.e. the same as tikz.

In summary, if one could say in the TPDF class that the rectangles are not only a fill, but also a stroke of the same color, and takes an equivalent stroke-width as in TTexDump, then both figures should be equivalent.

Of course, mathematically the TPDF seems better because the rectangles do not overlap, but I guess that when coming to real world, the limited precision of floats or whatever makes that the perfect-mathematical-overlapping of rectangles leave some 0.0001 gaps that produce white lines. So maybe it is just an internal “rounding” error of most pdf viewers.

As final example, right figure:

Rectangle 1: Y = 710.338, heightY = 1.043  ---> endingY = 710.338+1.043 = 711.381
Rectangle 2: Y = 711.381, heightY = 1.043 

left tikz figure:

Rectangle 1: Y=710.339, heightY=1.197 --> endingY = 711.536
Rectangle 2: Y =711.381, heightY=1.197

Attached figure comparing overlapping structure is seen. I agree with couet that the problem is from the pdf viewers, but I suggest to implement in ROOT following provisional workaround until the viewers correct it:
The function SaveAs can take “options” given by the user:
SaveAs(const char* filename = “”, Option_t* option = “”)
If I all c->SaveAs(".pdf",“STROKE”), then stroke is added to the rectangles of the pdf output when exporting the 2D COLZ histogram . The function TPDF::DrawBox should be modified accordingly.

Or if one has contact with people developing pdf readers, maybe it is the same buggy library used in all open source pdf viewers…?






Because they all badly implement the antialiasing. Abode (the reference) is fine. It might be that all these previewers are based on the same buggy rendering engine. Turning off the antialiasing should prouve that they are wrong on that side.

Alberto proposed to look at it. I do not know if he made progresses.

Great post Ferhue,
I have learnt a lot with it.
I have to say that I also agree that ROOT does well and not the pdf viewers, but it would be convenient to have a workaround so we can adapt the plots for those popular viewers.
For instance the pdf reader in android devices (from google) renders these colorful histograms perfectly as they come from root! The best I have seen to date…

I am going to check a way to implement something like the “STROKE” option in TPdf, as you proposed.
Thanks!

Yes, i did for a while, but i wasn’t very clear to me what to change in TPdf.
I am going to take another look now.

It works!
The solution is to paint a line stroke with the same color of the bin with width 1.
In order to do this I added an option in Hoption.h:

   int Strk;        // if selected with any COL option paints a line stroke with the same color of the bin.

which becomes true if one adds the string “strk” together with “col” in the Draw function of a TH2 histogram.
Then in the THistPainter::PaintColorLevels(), I have located the line that actually paints the beam:

gPad->PaintBox(xlow, ylow, xup, yup);
And I replaced it by:

if (Hoption.System != kPOLAR) { if(Hoption.Strk) { TBox box; Int_t hcolor = fH->GetFillColor(); Int_t hstyle = fH->GetFillStyle(); box.SetFillColor(hcolor); box.SetFillStyle(hstyle); box.SetLineColor(hcolor); box.SetLineWidth(1); box.PaintBox(xlow, ylow, xup, yup,"l"); } else { gPad->PaintBox(xlow, ylow, xup, yup); }

Thus, when nothing is specified root does like always:
LotovRadius.pdf (30.6 KB)

But if ones adds the string “strk” to the draw options, we get:
LotovRadius.strk.pdf (34.4 KB)

So it works as expected.
Note that the white lines persist in the z-axis. I haven’t changed anything there, but the trick should work equally there.
Now it is matter of implementing it properly in ROOT if they think that it is convenient.
Let me know what you think.
Cheers!

Thanks delaossa for your fast solution, I checked on Linux pdf viewers and the “strk” version works perfectly for me.

+1 vote for implementing this in future ROOT patches, as a “pragmatic (provisional?) workaround” for our buggy widespread pdf viewers (most of them do not allow turning antialiasing off as sugested).

Yes, that’s a good idea ! painting the stroke around the box makes it ok. I will think where is the best place to put it. Not that the palette needs also this extra line painting. May be something to be put at the pdf level only.
Anyway it seems the right direction.
Plus the size increase is not too big … 31kb vs 35kb

This problem is now fixed in both PDF and PostScript outputs. Your idea to make the stroke around the box was the right approach. But I did it in the better way in the PDF and PS drivers themselves. It is more general, fixes all the boxes and doesn’t change the file size. So all the tricks to bypass this issue are not needed anymore.

The PDF and PS outputs now work with buggy previewers… finally !! :slight_smile:

Great, Oliver!
I would like to know what did you change there.
Maybe plotting colored rectangles always in this way is not the best idea.
At least, it should be possible to select/deselect this option.
I have done some more tests with the “strk” option enabled/disabled.

Take a look to this plot with normal root behavior (no stroke):
Snapshot1-rake-v10kA.G.SR2.RI.3D_20.pdf (320 KB)

The buggy previewers show the white lines like always, but it also provides some kind of transparency.
This is cool, and one can see the orange histogram (electrons) behind the gray histogram (plasma).

With the new option “strk” that is what happens:
Snapshot1-rake-v10kA.G.SR2.RI.3D_20.strk.pdf (562 KB)

No white lines, no transparency …

At the end i don’t know which one I like the most #-o

Would be possible to have transparency and no white lines?
This would be great!

Oh, this is interesting.
Playing with the option “strk”, I can make one of the histograms (grey) transparent (with white lines) and the other two opaque (with no lines):
Snapshot1-rake-v10kA.G.SR2.RI.3D_20.hybrid.pdf (355 KB)

This a feature only for buggy previewers! funny, but it is the option that a like the most so far…
It would be then convenient to activate the stroke as an option when drawing the histogram and not in general for every ps or pdf.
Cheers!

We do not need the strk option. I did the change at the very low level in ps and pdf classes. The changes are minors. I just told ps and pdf to always stroke filled boxes.

Thanks couet for your modifications, that’s great news. I will be now able to export my TH2 to pdf instead of png :smiley:

Concerning delaossa, I think that what one should ask for is to have the possibility of filled 1D or 2D transparent histograms also in the pdf output, rather than profiting from an anti-aliasing bug for mimicking transparency. There is this post:
[Transparent histogram fill style

But it seems that the exported pdf does not contain the transparency for solid areas, and the densely dotted fill style mimicks it but is ugly for pdf. In my case, I use full solid areas, that are not transparent in the exported pdf, and then edit by hand the pdf with inkscape and add the transparency to these fill areas (now fill areas and strokes :smiley: ). Then I export back to pdf and it renders perfectly.

If inkscape can do it, then ROOT may also be able to include this in the future. But I think we need to create another post for this “new” request :wink:

Transparency now works for pdf and i have implemented transparent palettes. I will check monday what Alberto means