Multipage PDF - less memory consumption using png or something else

[rsln]

(Hello! I have memory consumption problem when trying to output my data to pdf with several pages.
My pipeline is looking kinda like this: I make TCanvas, then in a loop I create some class instance that owns TMultiGraph, then that class gathers some data and draws it via TGraph in the TMultiGraph (mg). After one loop is done, that mg draws to canvas, canvas prints to pdf, then canvas is cleared, and then it starts over.

void create_multigraph_with_tgraph(int index,const char* pdf_filename,  bool first, bool last) {
    TCanvas* c = new TCanvas(Form("c%d", index), Form("Canvas %d", index), 800, 600);
    
    TMultiGraph* mg = new TMultiGraph();
   
    const int n_points = 1000;
    double x[n_points], y[n_points];
    
    TRandom rand;
    for (int i = 0; i < n_points; ++i) {
        x[i] = rand. Uniform(0, 10);
        y[i] = rand. Uniform(0, 10);
    }
    
    TGraph* graph = new TGraph(n_points, x, y);
    graph->SetTitle(Form("Graph %d", index));
    graph->SetMarkerStyle(20);
    graph->SetMarkerSize(0.5);
    graph->SetMarkerColor(index + 1);
    
   
    mg->Add(graph);
    mg->SetTitle(Form("MultiGraph %d;X axis;Y axis", index));
    mg->Draw("AP");
    
    c->SaveAs(Form("graph_%d. png", index));

    if (first) {
        c->Print(Form("%s(", pdf_filename), "pdf");
    } else if (last) {
        c->Print(Form("%s)", pdf_filename), "pdf");
    } else {
        c->Print(pdf_filename, "pdf");
    }
    delete c;
}

void create_image_pdf(const char* image_pdf_filename, int n_graphs) {
    TCanvas* c = new TCanvas("image_canvas", "Image Canvas", 800, 600);

    for (int i = 0; i < n_graphs; ++i) {
        TString png_filename = Form("graph_%d. png", i);
        TImage* img = TImage::Open(png_filename);
        
        if (!img) {
            printf("Could not open image file %s\n", png_filename. Data());
            continue;
        }
        
        c->Clear();
        img->Draw();
        
        if (i == 0) {
            c->Print(Form("%s(", image_pdf_filename), "pdf");
        } else if (i == n_graphs - 1) {
            c->Print(Form("%s)", image_pdf_filename), "pdf");
        } else {
            c->Print(image_pdf_filename, "pdf");
        }
        delete img;
    }

    delete c;
}

void create_pdfs(const char* pdf_filename, const char* image_pdf_filename) {

    for (int i = 0; i < 10; ++i) {
        create_multigraph_with_tgraph(i, pdf_filename, i == 0, i == 9);
    }
    

    create_image_pdf(image_pdf_filename, 10);
}

int main() {
  
     const char* pdf_filename = "output. pdf";
     const char* image_pdf_filename = "output_images. pdf";
    
    create_pdfs(pdf_filename, image_pdf_filename);
    return 0;
}

this is as rough recreation of what is going on in my program. The ROOT calls are actuall scattered across different classes, but that doesn’t affect the result. I, however, discovered that there’s a problem with PDF size. If I rack up the number of points to 100000 per plot (for example), my PDF will be huge (since it’s vector), but pngs, being raster, do not take that much space. For 10000 points per plot and 10 plots on pdf I believe the code above gives me about 3.2 mb PDF, ubt cummulative size of pngs is about 1.5 mb. And the more points - the worse (about 150kb pngs x 10 gives me once again 1.5 megs, but the PDF will be around 30+ megs). The whole point of using ROOT was to make code much better than gnuplot, because gnuplot is somewhat hard to incorporate into C++ code. But for some reason gnuplot’s PDF plots weigh times less than ROOT’s PDF (with same data).

My initial idea was to try and save plots as png and then put it on TCanvas and print to PDF, possibly reducing the final size of PDF.
But as you can see, I attempted it in my code (create_image_pdf()), and it gives “Warning in <TASImage::Paint>: PDF not implemented yet:”.

So, my question stands: is there a way to produce multipage PDF, AND reduce the size of it? I believe the problem lies in the fact that too many points for vector image leads to memory consumption by design, unlike raster pics, but I’m willing to try anything else as long as it just ROOT, no standalone apps (i can defo manually stitch with some kind of imagemagick or something else, but I want to get PDF from my app, no other apps)

Also, I’ve definetely seen the same topics on this forum (there’s one called " Saving png logo in ROOT as pdf", i cant put links as new user, it specifiaclly has the same issue with putting pngs on pdf, but only told to go vector), but maybe there are some new ideas regarding this issue?

Thank you in advance!)

[Danilo]

Hi,

Thanks for the interesting post (it’s likely to become a reference) and welcome to the ROOT Community!

I am not completely sure if the size of the pdf can be reduced, and before suggesting something misleading, I am adding in the loop 3 experts @bellenot @couet and @linev .
In the meantime, are you blocked by the issue or can you use pngs?

Cheers,
Danilo

[rsln]

Hi! Well, I am not exactly blocked by the issue – it’s just that memory consumption really bothers me, because at the moment I cannot predict how huge the data might be, but it seems that scalability is a ‘must have’ in my case – I would like to get plots with lots of points, but without spending too much space. Going PNG-only is not really an option for me, because I specifically need my plots to be in one PDF. I discussed an option of creating something like a big tpad with all the necessary plots and then exporting it to png, but i was advised that it’s not a desired approach.

I also tried an advice from (Warning in <TASImage::Paint>: PDF not implemeted yet) about going to PS, but it created an even bigger file than my pdf, (something like ~350 kilobytes cummulative pngs size vs 3.2 megabytes PS file).

I also realized that it kinda depends on points – in small cases there’s no difference, but the further I go in terms of points amount, the huger PDF is.

At the moment I will be investigating for some kind of small C++ library that would allow me to fuze PNGs to PDF and see whether the idea is at all helping with size reduction.

But I am still very much looking forward to some advice, since I would love to do everything with ‘in-house’ methods of ROOT!

[linev]

Hi,

It would be nice if you can post small reproducer here.
It can be random histograms or graphs generated directly in the macro.

Regards,
Sergey

[rsln]

Hi, Sergey!

I may not really understand, isn’t it that the code I’ve provided in my first message is already a small reproducer? Or should I export my TCanvas as .C/.root to make a macro? Will it represent my attempts at saving PNGs to PDF?

Regards,
Ruslan

[linev]

@rsln

isn’t it that the code I’ve provided in my first message is already a small reproducer?

Yes, one can use it. I did not realize this.

PDF file is vector graphics and includes as many points as you create in TGraph.
Yes - it can grow to very large size.

And looks like TImage is not supported in native ROOT graphics for PDF generation.

But since version 6.32 we have support of web-based graphics - including PDF generation.
And web graphics support image embedding into PDF.

I attach macro which demonstrates how it works.
multi_pdf.cxx (1.4 KB)

Main function looks like:

   std::vector<TPad *> vect;
   for (int i = 0; i < 10; ++i)
      vect.push_back(create_multigraph_with_tgraph(i));
   TCanvas::SaveAll(vect, "output.pdf");
   TCanvas::SaveAll(vect, "graph_.png");

   std::vector<TPad *> vect2;
   for (int i = 0; i < 10; ++i)
      vect2.push_back(create_image_canvas(i));
   TCanvas::SaveAll(vect2, "output_images.pdf");

Here I use new TCanvas::SaveAll() method.
It saves N canvases either as N images or as 1 multi-page PDF file.
It works with native and web-based graphics.

To run with web-graphics, just do:

root -b --web=chrome multi_pdf.cxx -q

For me output.pdf file is ~5MB and output_images.pdf is about 600KB.

[rsln]

It actulaly works, thank you! I launched it and got 605 kb PDF vs 5.8 mb PDF. But now I am a bit puzzled at how to turn on this web-based rendering in my C++ code. I tried to do something like

#include <TROOT.h>
//code
TROOT::SetWebDisplay("chrome");

But this doesn’t seem to work. It tells me that ‘Call to non-static member function without an object argument’. I believe I’m just missing something, maybe some CMake params for building my app?
Thank you very much!

[linev]

@rsln

But now I am a bit puzzled at how to turn on this web-based rendering in my C++ code.

gROOT->SetWebDisplay("chrome");

[rsln]

I guess, that settles all my questions regarding this topic atm. Thank you very much