Code coverage tools for ROOT6 macros

sstiefel19 · November 27, 2019, 3:39pm

Dear experts,
I’d like to know if there are possibilities to use code coverage tools (such as gcov for instance) on root6 macros. My use case is that I am confronted with a rather large software stack (~1M lines in ~500 files) and I’d like to find out, which functions and lines from which files are executed on specific calls to the API. I did some research but could not find much more then a CERN summer school presentation from 2009 on gcov: https://indico.cern.ch/event/65115/contributions/1226089/subcontributions/109817/attachments/1013158/1441688/code_coverage-rroska.pdf
Do you know of currently used tools which could help me in achieving my goal?

Thanks a lot already in advance!
Stephan
_ROOT Version: 6
_Platform: MacOS mojave

oshadura · November 27, 2019, 6:14pm

Hi,

Which code coverage are you trying to measure, for your framework or for ROOT?

Usually code coverage are measurements which describes the degree of which the source code of the framework has been tested (unit tests, benchmarks and etc.).

In case of ROOT we have a special flag -Dcoverage=ON which enables coverage testing for ROOT. For your framework, you need to implement something similar…

sstiefel19 · November 27, 2019, 7:46pm

Hi oshadura,
thanks for your reply. For my framework. I was hoping there would be sth ready to use - like gcov when having a pure c++ project. Implementing it myself would not be worth the effort I guess.
Putting this aside, is there an documentation on -Dcoverage for root? I could not find anything in the root documentation.

Thanks and best,
Stephan

oshadura · November 28, 2019, 7:43am

Hi,

This option will enable coverage testing for ROOT framework only (not for your framework, which is using ROOT, but for ROOT project itself).

You will need to implement coverage testing separately for your project.

sstiefel19 · November 28, 2019, 8:53am

Hi,

Sure, this I understood. Still wanted to ask if there is
documentation on the -Dcoverage flag?

Best,
Stephan

oshadura · November 28, 2019, 9:30am

To configure ROOT with coverage support you need to use:
cmake /your/build/directory -Dcoverage=ON

How it is implemented, check here:

github.com

root-project/root/blob/master/CMakeLists.txt#L143


    set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE ${CCACHE_COMMAND})
    execute_process(COMMAND ${CCACHE_COMMAND} "-V" OUTPUT_VARIABLE CCACHE_VERSION)
    string(REGEX REPLACE "ccache version ([0-9\\.]+).*" "\\1" CCACHE_VERSION ${CCACHE_VERSION})
  else()
    message(STATUS "Could NOT find ccache")
    set(ccache OFF CACHE BOOL "Use ccache (disabled since ccache was not found)" FORCE)
  endif()
endif()


#---Enable test coverage -----------------------------------------------------------------------
if(coverage)
  set(GCC_COVERAGE_COMPILE_FLAGS "-fprofile-arcs -ftest-coverage")
  set(GCC_COVERAGE_LINK_FLAGS    "-fprofile-arcs")
  set(CMAKE_CXX_FLAGS            "${CMAKE_CXX_FLAGS} ${GCC_COVERAGE_COMPILE_FLAGS}")
  set(CMAKE_EXE_LINKER_FLAGS     "${CMAKE_EXE_LINKER_FLAGS} ${GCC_COVERAGE_LINK_FLAGS}")
  set(CMAKE_SHARED_LINKER_FLAGS  "${CMAKE_SHAREDLINKER_FLAGS} ${GCC_COVERAGE_LINK_FLAGS}")
  set(CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS "${CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS} ${GCC_COVERAGE_COMPILE_FLAGS}")
endif()


#--- Enable build timing -----------------------------------------------------------------------
if (build_timing)

ahesam · November 28, 2019, 10:21am

I’d like to find out, which functions and lines from which files are executed on specific calls to the API

It seems to me like you’d me more interested in code profiling tools Checkout tools like VTune, perf and gprof.

sstiefel19 · November 28, 2019, 11:02am

Hi ahesam,
those tools u mentioned are for performance profiling of code as much as I know. Like when u want to find out in which function a lot of times is spent. I however only want to know, which lines in which files got touched when I run s.th in the code. So to my understanding this means I want code coverage tooling - what u use to find out if your unit tests cover all of your code. Am I wrong?

eguiraud · November 28, 2019, 11:14am

Hi @sstiefel19,
welcome to the ROOT forum! I’m not a tooling expert, but I don’t think there is a tool that can get you coverage of ROOT macros (as in, code interpreted through cling). Macros do not even provide debug symbols. You would have to hack cling to do this for you as it parses the code.

As you probably know ROOT macros are meant for quick scripts. For a framework of ~1M lines of code, you should use compiled C++ code and use ROOT as a usual C++ library – this way you can use standard profiling, debugging and static analysis tools on the framework, and you would also get a (possibly large) boost in performance – interpreted code is not optimized by default. I realized that this might not be feasible for you at this point.

I hope I did not misinterpret your question.
Cheers,
Enrico

ahesam · November 28, 2019, 11:37am

You can also use those tools to also discover which consecutive functions (in which files) are being called for each API invocation. But @eguiraud has a point, in that they work only (AFAIK) for compiled programs…

sstiefel19 · November 28, 2019, 11:46am

Hi @eguiraud,
sorry I think was not accurate in my post. I think I do use compiled c++ code. I mean run
root -x -q -l -b TaskV1/ExtractSignalV2.C+($1,$MODE,$USETHNSPARSE,-1,"$CORRFSETTING")
for example, which compiles the code first and then executes the binary… or? But I guess this is still not the way u mean when u say ‘use compiled C++ code and use ROOT as usual C++ library’. How would I do that?
Best,
Stephan

eguiraud · November 28, 2019, 12:07pm

Hi Stephan,
yes what I meant is really g++ -g -O2 your_main.C $(root-config --libs --cflags) or similar. If you use standard C++ workflows, you can certainly use standard C++ tooling.

As far as I know, root script.C+ does compile the script, invoking your compiler under the hood. I don’t know how that plays with debug symbols and tooling in general. Maybe @Axel or @pcanal have an idea.

Cheers,
Enrico

Axel · November 28, 2019, 7:49pm

Well, I doubt that a 1M lines framework should be interpreted - so indeed, compiling it is the way to go! As Oksana pointed out, -fprofile-arcs -ftest-coverage are the flags to be used. This isn’t really ROOT specific, so any web site introducing coverage with GCC should do.

Cheers, Axel.

system · December 12, 2019, 7:49pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.