Real-time analasys of data stream

the idea is to have a laptop running root to display real time measurements in the form of histograms as data is being collected and passed to storage device.

i have a server that is collecting data from an experiment. the data passes through the server briefly on its way to storage.

as the data passes through the server real-time measurements are streamed to a laptop where it is displayed as a number of root histograms. the histograms should be updated on the order of 1 second intervals.

the code is relatively straightforward, but i am having trouble understanding how the filling and updating should work in root.

overview of program structure is:

  1. tcp socket connection is made from the laptop to the server.
  2. a number of TH1D objects are created.
  3. a TCanvas is created and divided to draw a number of Histograms.
  4. thread is created to read from socket and fill the histograms.
  5. on a regular 1/2 sec interval: the Histograms are drawn on the Canvas with Canvas::cd(i) and TH1D::Draw(), as well as a call is made to Canvas::Update().

the program seems to work for the most part. the histograms are drawn and updated. you can see them fill up.

***the problem is that the program crashes when you mouse over the canvas.

i don’t understand where the crash comes from. is this the correct call sequence? is it a concurrency problem? is it not advised to fill a TH1D while updating the canvas? is there a better way to restructure the program?

do i need to fill the histograms in the event loop with the same thread as the event loop? could be done with a synchronous timer object.

thanks
Juaquin

[quote] on a regular 1/2 sec interval: the Histograms are drawn on the Canvas with Canvas::cd(i) and TH1D::Draw(), as well as a call is made to Canvas::Update().[/quote]The best way to do this is to ‘Draw’ the histogram only once. To refresh the data, you update the same histogram object with the new data (either via Fill or via calls to Reset and Add) and to update the canvas (via canvas->Modified(); canvas->Update(); ).

[quote]2) thread is created to read from socket and fill the histograms.[/quote]The implementation details of this steps is very important. From the fact that your process crash when you move your move over, I am guessing that this steps actually deletes some of the histogram objects rather than ‘updating’ them.

Cheers,
Philippe.

Since this post is not the announcement of a ROOT application useable by other but a request for help/support ; this post has been moved to the ‘Root Support’ forum.

thank you for your reply.

first to address your second suggestion…
the part of the program where thread is created to read from socket and fill the histograms, is written to only access the TH1D only through its method TH1D::Fill(x) without deleting anything. i do intend to use the TH1D::Add and TH1D::Reset when i can demonstrate the stability of this code. the structures for the histograms are created in the beginning with new and will remain in memory until the program exits.

the histograms are updated in a separate thread, but i hope this is ok since there are only calls to Fill.

otherwise, i have changed the code as you suggested…
as the program is now, the canvas is divided with Canvas::Divide(1, 4), the histograms are placed into the pads with calls to Canvas::cd(i) and TH1D::Draw() for each histogram. this is done only once in the beginning. the display looks correct as expected.

also i have changed the calls which happen on 1/2 sec intervals, as you suggested…
now the calls are Canvas::Modified() and Canvas::Update().

the result is that the display is no longer updated.

i have tried to understand the way these things are supposed to work with the root api. the calls you suggested make sense, but do not seem to work as i would expect.

with trial and error, i have found that the display is updated with calls to Canvas::Draw() and Canvas::Update() on 1/2 second intervals. …also the display is updated with calls to Canvas::Paint() and Canvas::Update() on 1/2 second intervals… but the program still crashes when the mouse moves around over the display.

the display looks a little wrong on some updates as the mouse moves around over stuff.

i wonder if this may be a problem with the sequence of calls, or concurrency with mouse interactive qualities of the canvas, or if this may be a bug in root.

i am looking for a way to make this work.
thank you
J

[quote] i have found that the display is updated with calls to Canvas::Draw() and Canvas::Update() on 1/2 second intervals. [/quote]humm … the canvas should be 'Draw()'ed only once at the beginning … and even that is usually done indirectly. For example, see tutorials/hsimple.C which does: TCanvas *c1 = new TCanvas("c1","Dynamic Filling Example",200,10,700,500); .... const Int_t kUPDATE = 1000; for (Int_t i = 0; i < 25000; i++) { if (i && (i%kUPDATE) == 0) { if (i == kUPDATE) hpx->Draw(); c1->Modified(); c1->Update(); if (gSystem->ProcessEvents()) break; } }

Cheers,
Philippe.

thanks for the reply.

that code is interesting… the call to gSystem->ProcessEvents(), seems to be part of an event loop written to explicitly cause the display update to happen. i see Draw()ing happens only once.

i used the TApplication::Run() call in main(), which i believe includes an event loop.

i expect there are many ways to accomplish the same things with the root libraries.

i am verry new to root programming. i tried to use threads for my application. my approach may be unusual, but it seems like it should work provided there is some thread safety in the root objects and methods.

in my application there are three threads. the calls used in the threads are: TPad::Modified(), TCanvas::Update(), TH1D::Fill(), and in the main() thread there is a call to TApplication::Run().

i have simplified the code to “test.c” as an example:

#include <stdio.h>
#include <pthread.h>
#include <unistd.h>


#include <TRandom1.h>

#include <TH1.h>

#include <TApplication.h>

//#include <TTimer.h>

#include <TCanvas.h>


TCanvas *theCanvas;
TH1D *theHistograms[4];


pthread_t displayupdatethread;
pthread_t histogramfillthread;



static void *
histogramfill(void *qqq)
// this is the plot handleing thread for plots processed on the local machine.
{TRandom1 theRand;

  while (1)
    { usleep(1000);
      { int i, j;
        for (i = 0; i < 4; i++)
          for (j = 0; j < 1000; j++)
            theHistograms[i]->Fill(theRand.Gaus(5, 1));
      }
    }

  pthread_exit(NULL);// (code execution never gets here)
}



static void *
displayupdate(void *qqq)
// this is the thread that should cause the display to be updated.
// here i can try various calls to see if the display is updated as expected.
{ while (1)
    { usleep(100000);// delay 1/10 second

      //*
      // this is one set of calls that cause the display to be updated:
      int i;
      for (i = 0; i < 4; i++)
        { theCanvas->cd(i + 1);
          gPad->Modified();
          //gPad->Update();
        }
      theCanvas->cd(0);
      theCanvas->Update();
      //*/

      /*
        // this is another set of calls that cause the display to be updated:
        theCanvas->Paint();
        theCanvas->Update();
      */

      /*
        // this set of calls does not cause the display to be updated:
        theCanvas->Modified();
        theCanvas->Update();
      */
    }
  pthread_exit(NULL);// (code execution never gets here)
}



int
connecthistogramfill(void *qqq)
{ if (pthread_create(&histogramfillthread, NULL, &histogramfill, qqq))
    { printf("the histogramfill thread did not start\n");
      return(-1);
    }
  else return(0);
}



int
connectdisplayupdate(void *qqq)
{ if (pthread_create(&displayupdatethread, NULL, &displayupdate, qqq))
    { printf("the displayupdate thread did not start\n");
      return(-1);
    }
  else return(0);
}



int
main(int argc, char* argv[])
{ TApplication* rootapp = new TApplication("test", &argc, argv);


  //theCanvas is allocated for the life of the program:
  { Double_t w = 600;
    Double_t h = 600;
    theCanvas = new TCanvas("theCanvas", "the Canvas", w, h);
    theCanvas->SetWindowSize(w + (w - theCanvas->GetWw()), h + (h - theCanvas->GetWh()));
  }


  // theHistograms[i] are allocated for the life of the program:
  { int i;
    char name[255], title[255];
    for (i = 0; i < 4; i++)
      { sprintf(name, "theHistogram[%d]", i);
        sprintf(title, "the Histogram #%d", i);
        theHistograms[i] = new TH1D(name, title, 200, 0.0, 10.0);
      }
  }


  // create the sub pads:
  theCanvas->Divide(1, 4);


  // draw the histograms once
  { int i;
    for (i = 0; i < 4; i++)
      { theCanvas->cd(i + 1);
        theHistograms[i]->Draw();
      }
    theCanvas->cd(0);
  }


  // create two threads for filling and displsaying theHistograms[i].
  { int i, j;
    if ((i = connecthistogramfill(NULL)) == 0)
      printf("successfully started thread to fill histogram..\n");
    if ((j = connectdisplayupdate(NULL)) == 0)
      printf("successfully started thread to update the display..\n");
  }


  /*
  //  display could be updated with TTimer instead of with histogramfillthread:
  TTimer *timer = 0;
  timer = new TTimer(100,kFALSE);   // 100 msec timeout
  timer->Connect("Timeout()", "TCanvas", theCanvas, "Paint()");
  timer->Connect("Timeout()", "TCanvas", theCanvas, "Update()");
  timer->Start(100, kFALSE);
  */

  rootapp->Run();

  return 0;
}

the above code in file “test.c” was compiled:

g++  -pthread -c `root-config --cflags`  test.c
g++ -pthread  `root-config --libs`   test.o -o test

when the resulting program “test” is run, it seems to be fine as long as the mouse is kept away from the window, but if you move the mouse over the histograms a while it shows artifacts and crashes.

could this be a bug in root?

even if it is a bug, i do need to work around the problem in order to use root for this application.

thanks
J
test.tar.gz (1.39 KB)

http://savannah.cern.ch/bugs/?73778

should try using the TThread class. see if it improves things.

i tested incorporating the class TThread into my example, and caused the application not to crash. i have uploaded a file test3.tar.gz which incorporates TThread.

it is still unclear to me the correct sequence of calls to have the display updated correctly. the proram behavior is puzzeling.

in the new version of “test.c”, the display is not updated at first, then if you pass the mouse over the screen the display is updated and updated continuously thereafter, even when the mouse moves out of the root display window.

what is the correct sequence of calls for updating the display and filling the histograms in three threads?

could the calls to TCanvas::cd(i) be causing some problems or artifacts? is it better to avoid calls to TCanvas::cd()?

can someone please clarify what the call sequence is for the three threads in my modified example?

thank you.
J
test3.tar.gz (1.45 KB)