GitHub actions and conda docker problem

ROOT
1

Hi,

I am trying to run CI on Github Actions for my projects. I am running on ubuntu-latest (20.04) but my project is using c++17. First I was trying to download precompiled image but it is compiled agains c++11. So my next step was to try with docker and conda.

But whenever I add conda container, the apt which I need to install deps, pulls them from bionic, not focal, what in the end breaks the build.

Is conda docker in any way forces changes to bionic? If yes, maybe it should be somehow documented on docker page, like more verbose description of configuration for each docker. If not, what do I do wrong?

Workflow execution: Update of SContainer and SParContainer · SiFi-CC/sifi-framework@835d6fa · GitHub (it breaks in different place, but checking the Install common deps sections shows my problem).

My workflow (MWE) is following:

name: CMake Build Matrix

on:
  push:
  pull_request:

jobs:
  build:
    name: ${{ matrix.config.name }}
    runs-on: ${{ matrix.config.os }}
    container:
      image: rootproject/root:6.22.06-conda
    strategy:
      fail-fast: false
      matrix:
        config:
        - {
            name: "Ubuntu_GCC",
            os: ubuntu-latest,
            build_type: "Release",
            cc: "gcc-9",
            cxx: "g++-9",
          }
        - {
            name: "Ubuntu_CLANG",
            os: ubuntu-latest,
            build_type: "Release",
            cc: "clang-11",
            cxx: "clang++-11",
          }

    steps:
      - name: Print env
          echo github.event.action: ${{ github.event.action }}
          echo github.event_name: ${{ github.event_name }}

      - name: Install common dependencies
        run: |
          apt update
          apt install libcppunit-dev cmake -y
          cmake --version
      - name: Install dependencies on ubuntu with gcc
        if: startsWith(matrix.config.name, 'Ubuntu_GCC')
        run: |
          gcc --version
      - name: Install dependencies on ubuntu with clang
        if: startsWith(matrix.config.name, 'Ubuntu_CLANG')
        run: |
          #apt install clang-11 -y
          clang --version

      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - name: Checkout repository
        uses: actions/checkout@v1
        with:
          submodules: recursive

and in case Worklow log is gone from github, here some snippeps from it:

2021-04-11T12:05:14.4158137Z ##[section]Starting: Request a runner to run this job
2021-04-11T12:05:14.6655370Z Can't find any online and idle self-hosted runner in current repository that matches the required labels: 'ubuntu-latest'
2021-04-11T12:05:14.6655480Z Can't find any online and idle self-hosted runner in current repository's account/organization that matches the required labels: 'ubuntu-latest'
2021-04-11T12:05:14.6655896Z Found online and idle hosted runner in current repository's account/organization that matches the required labels: 'ubuntu-latest'
2021-04-11T12:05:14.7583300Z ##[section]Finishing: Request a runner to run this job
2021-04-11T12:05:20.8435215Z Current runner version: '2.277.1'
2021-04-11T12:05:20.8459203Z ##[group]Operating System
2021-04-11T12:05:20.8460054Z Ubuntu
2021-04-11T12:05:20.8460430Z 20.04.2
2021-04-11T12:05:20.8460834Z LTS
2021-04-11T12:05:20.8461235Z ##[endgroup]
2021-04-11T12:05:20.8461769Z ##[group]Virtual Environment
2021-04-11T12:05:20.8462312Z Environment: ubuntu-20.04
2021-04-11T12:05:20.8462844Z Version: 20210405.1
2021-04-11T12:05:20.8463836Z Included Software: https://github.com/actions/virtual-environments/blob/ubuntu20/20210405.1/images/linux/Ubuntu2004-README.md
2021-04-11T12:05:20.8464915Z Image Release: https://github.com/actions/virtual-environments/releases/tag/ubuntu20%2F20210405.1
2021-04-11T12:05:20.8465583Z ##[endgroup]

2021-04-11T12:05:23.8269657Z ##[command]/usr/bin/docker pull rootproject/root:6.22.06-conda
2021-04-11T12:05:24.5322381Z 6.22.06-conda: Pulling from rootproject/root
2021-04-11T12:05:24.5323091Z f22ccc0b8772: Pulling fs layer
2021-04-11T12:05:24.5323731Z 3cf8fb62ba5f: Pulling fs layer
2021-04-11T12:05:24.5324316Z e80c964ece6a: Pulling fs layer
2021-04-11T12:05:24.5324888Z c99136bf9213: Pulling fs layer
2021-04-11T12:05:24.5325468Z 76af7e69ef6b: Pulling fs layer
2021-04-11T12:05:24.5326018Z c99136bf9213: Waiting
2021-04-11T12:05:24.5327728Z 76af7e69ef6b: Waiting
2021-04-11T12:05:24.5328324Z 3cf8fb62ba5f: Verifying Checksum
2021-04-11T12:05:24.5329008Z 3cf8fb62ba5f: Download complete
2021-04-11T12:05:24.5329638Z e80c964ece6a: Verifying Checksum
2021-04-11T12:05:24.5330280Z e80c964ece6a: Download complete
2021-04-11T12:05:24.5330918Z f22ccc0b8772: Verifying Checksum
2021-04-11T12:05:24.5331552Z f22ccc0b8772: Download complete
2021-04-11T12:05:24.9940766Z c99136bf9213: Verifying Checksum
2021-04-11T12:05:24.9945725Z c99136bf9213: Download complete
2021-04-11T12:05:25.9593833Z f22ccc0b8772: Pull complete
2021-04-11T12:05:26.0476753Z 3cf8fb62ba5f: Pull complete
2021-04-11T12:05:26.1201367Z e80c964ece6a: Pull complete
2021-04-11T12:05:30.5318619Z c99136bf9213: Pull complete
2021-04-11T12:05:32.7763730Z 76af7e69ef6b: Verifying Checksum
2021-04-11T12:05:32.7766181Z 76af7e69ef6b: Download complete
2021-04-11T12:05:54.4658838Z 76af7e69ef6b: Pull complete
2021-04-11T12:05:54.4707654Z Digest: sha256:9f91809faa39f8d982b1ebb841b2fc5f6fe32afacb9c7ad90aabc61cc5c1e920
2021-04-11T12:05:54.4743807Z Status: Downloaded newer image for rootproject/root:6.22.06-conda
2021-04-11T12:05:54.4762388Z docker.io/rootproject/root:6.22.06-conda
2021-04-11T12:05:54.4823767Z ##[command]/usr/bin/docker create --name e78ba339ae4947df91b3000a6ba2e1ff_rootprojectroot62206conda_b3045c --label 5588e4 --workdir /__w/sifi-framework/sifi-framework --network github_network_2235a4614224436ea3e564256333214b  -e "HOME=/github/home" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work":"/__w" -v "/home/runner/runners/2.277.1/externals":"/__e":ro -v "/home/runner/work/_temp":"/__w/_temp" -v "/home/runner/work/_actions":"/__w/_actions" -v "/opt/hostedtoolcache":"/__t" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" --entrypoint "tail" rootproject/root:6.22.06-conda "-f" "/dev/null"
2021-04-11T12:05:57.4397949Z 78b231459cee7867ffb4d97ce581b940f827f4a38c7046bacad4825ba4a244e1
2021-04-11T12:05:57.4411258Z ##[command]/usr/bin/docker start 78b231459cee7867ffb4d97ce581b940f827f4a38c7046bacad4825ba4a244e1
2021-04-11T12:05:58.0681149Z 78b231459cee7867ffb4d97ce581b940f827f4a38c7046bacad4825ba4a244e1
2021-04-11T12:05:58.0729323Z ##[command]/usr/bin/docker ps --all --filter id=78b231459cee7867ffb4d97ce581b940f827f4a38c7046bacad4825ba4a244e1 --filter status=running --no-trunc --format "{{.ID}} {{.Status}}"
2021-04-11T12:05:58.1118127Z 78b231459cee7867ffb4d97ce581b940f827f4a38c7046bacad4825ba4a244e1 Up Less than a second
2021-04-11T12:05:58.1151425Z ##[command]/usr/bin/docker inspect --format "{{range .Config.Env}}{{println .}}{{end}}" 78b231459cee7867ffb4d97ce581b940f827f4a38c7046bacad4825ba4a244e1
2021-04-11T12:05:58.1568744Z GITHUB_ACTIONS=true
2021-04-11T12:05:58.1569178Z CI=true
2021-04-11T12:05:58.1569502Z HOME=/github/home
2021-04-11T12:05:58.1569985Z PATH=/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2021-04-11T12:05:58.1570491Z CONDA_DIR=/opt/conda
2021-04-11T12:05:58.1570824Z LANG=C.UTF-8
2021-04-11T12:05:58.1571151Z LC_ALL=C.UTF-8
2021-04-11T12:05:58.1601424Z ##[endgroup]

2021-04-11T12:05:58.3222109Z ##[group]Run apt update
2021-04-11T12:05:58.3222604Z e[36;1mapt updatee[0m
2021-04-11T12:05:58.3223157Z e[36;1mapt install libcppunit-dev cmake -ye[0m
2021-04-11T12:05:58.3223711Z e[36;1mcmake --versione[0m
2021-04-11T12:05:58.3224226Z shell: sh -e {0}
2021-04-11T12:05:58.3224620Z ##[endgroup]
2021-04-11T12:05:58.4531948Z 
2021-04-11T12:05:58.4533709Z WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
2021-04-11T12:05:58.4534233Z 
2021-04-11T12:05:58.6777819Z Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
2021-04-11T12:05:58.7038803Z Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease
2021-04-11T12:05:58.7736980Z Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
2021-04-11T12:05:59.0725746Z Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
2021-04-11T12:05:59.0961275Z Get:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [2045 kB]
2021-04-11T12:05:59.2097131Z Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [378 kB]
2021-04-11T12:05:59.3839266Z Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [2173 kB]
2021-04-11T12:05:59.5615820Z Get:8 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [24.5 kB]
2021-04-11T12:05:59.5617881Z Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [348 kB]
2021-04-11T12:05:59.5710414Z Get:10 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1402 kB]
2021-04-11T12:05:59.5878442Z Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [31.4 kB]
2021-04-11T12:05:59.5895355Z Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [2476 kB]
2021-04-11T12:06:00.2126826Z Fetched 9130 kB in 2s (5425 kB/s)
2021-04-11T12:06:01.2570004Z Reading package lists...
2021-04-11T12:06:01.4471378Z Building dependency tree...
2021-04-11T12:06:01.4533992Z Reading state information...
2021-04-11T12:06:01.4601135Z 24 packages can be upgraded. Run 'apt list --upgradable' to see them.
2021-04-11T12:06:01.4675572Z 
2021-04-11T12:06:01.4676388Z WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
2021-04-11T12:06:01.4703744Z 
2021-04-11T12:06:02.5458961Z Reading package lists...
2021-04-11T12:06:02.7324040Z Building dependency tree...
2021-04-11T12:06:02.7326037Z Reading state information...
2021-04-11T12:06:02.8695765Z The following additional packages will be installed:
2021-04-11T12:06:02.8697791Z   binutils binutils-common binutils-x86-64-linux-gnu cmake-data cpp cpp-7 gcc
2021-04-11T12:06:02.8699421Z   gcc-7 gcc-7-base libarchive13 libasan4 libatomic1 libbinutils libc-dev-bin
2021-04-11T12:06:02.8700997Z   libc6 libc6-dev libcc1-0 libcilkrts5 libcppunit-1.14-0 libcurl4 libgcc-7-dev
2021-04-11T12:06:02.8702372Z   libgomp1 libicu60 libisl19 libitm1 libjsoncpp1 liblsan0 liblzo2-2 libmpc3
2021-04-11T12:06:02.8703365Z   libmpfr6 libmpx2 libquadmath0 librhash0 libtsan0 libubsan0 libuv1 libxml2
2021-04-11T12:06:02.8704372Z   linux-libc-dev make manpages manpages-dev
2021-04-11T12:06:02.8705077Z Suggested packages:
2021-04-11T12:06:02.8706009Z   binutils-doc cmake-doc ninja-build cpp-doc gcc-7-locales gcc-multilib
2021-04-11T12:06:02.8707566Z   autoconf automake libtool flex bison gdb gcc-doc gcc-7-multilib gcc-7-doc
2021-04-11T12:06:02.8708695Z   libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg libasan4-dbg
2021-04-11T12:06:02.8729658Z   liblsan0-dbg libtsan0-dbg libubsan0-dbg libcilkrts5-dbg libmpx2-dbg
2021-04-11T12:06:02.8730971Z   libquadmath0-dbg lrzip glibc-doc locales libcppunit-doc make-doc man-browser
2021-04-11T12:06:02.9685603Z The following NEW packages will be installed:
2021-04-11T12:06:02.9688408Z   binutils binutils-common binutils-x86-64-linux-gnu cmake cmake-data cpp
2021-04-11T12:06:02.9690637Z   cpp-7 gcc gcc-7 gcc-7-base libarchive13 libasan4 libatomic1 libbinutils
2021-04-11T12:06:02.9693289Z   libc-dev-bin libc6-dev libcc1-0 libcilkrts5 libcppunit-1.14-0 libcppunit-dev
2021-04-11T12:06:02.9695516Z   libcurl4 libgcc-7-dev libgomp1 libicu60 libisl19 libitm1 libjsoncpp1
2021-04-11T12:06:02.9697556Z   liblsan0 liblzo2-2 libmpc3 libmpfr6 libmpx2 libquadmath0 librhash0 libtsan0
2021-04-11T12:06:02.9699571Z   libubsan0 libuv1 libxml2 linux-libc-dev make manpages manpages-dev
2021-04-11T12:06:02.9701000Z The following packages will be upgraded:
2021-04-11T12:06:02.9701867Z   libc6
2021-04-11T12:06:03.1402812Z 1 upgraded, 42 newly installed, 0 to remove and 23 not upgraded.
2

Hi @rlalik ,
the rootproject/root:6.22.06-conda indeed runs bionic:

~ docker run --rm -it rootproject/root:6.22.06-conda cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS"

rootproject/root:6.22.08-conda instead runs focal, you can probably use that instead (6.22.08 instead of 6.22.06).

I see how this might be confusing, but ROOT’s conda image is not guaranteed to run any Ubuntu version (in fact it is not even guaranteed to run Ubuntu): it just uses whatever OS condaforge/miniforge3 uses, as per the Dockerfile (and the assumption is that other packages will be installed via conda too, so fixing the OS version should not be important). For fixed Ubuntu releases, we provide the Ubuntu Docker images instead (I do realize that in your case the different C++ standard they come with is too old).

You can easily build your own image starting from our own Dockerfiles or you can use rootproject/root:6.22.08-conda which happens to satisfy your requirements.

Importantly, though, as I see you install g++ and clang++ via the package manager in different elements of the CI matrix: I am not sure using ROOT with a different compiler than the one it was used to compile it is supported (due to ROOT’s C++ interpreter, cling). I will check with the cling experts and get back to you. EDIT: turns out I was wrong

Cheers,
Enrico

3

Thank you Enrico,

I am not really familiar with the docker technology. I though the image provides the ROOT installation only, not the (full) OS or any part which could interfere with the host. So I though I can use conda docker on top of the Ubuntu.

Previously I was downloading the precompiled ROOT packages but again, for Ubuntu they are providing only c++11.

Do you think you could provide for users ROOT compiled with c++17 (or also c++14 per what user needs?).

I am using clan because I run some sanitizers and some are available only on clang. But I never noticed any problem.

4

Docker images always contain a full OS. EDIT: you can “layer” Docker images on top of each other but what that means is that what you get on a new image is the contents of its base image + whatever changes were applied by the commands in the Dockerfile.

Our pre-compiled binaries are compiled with the default C++ standard of the default system compiler (excluding bugs). For the upcoming 6.24, that means C++14 on Ubuntu 20.04.

Indeed @Axel confirmed that as long as the C++ standard is the same, changing compilers should not be an issue, I was mistaken.

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.