ROOT does have a test infrastructure, but I think from your perspective you’d want to focus on things beyond that primarily, and more on the system integration and reliability side of things.
With the snap, I’ve always advertised it as a sort of ROOT-Lite. A good package for the physics university student going into higher-education, it gets you (most) things needed to get going. And it’s also true that there’s a fair bit of use beyond that in actual production workloads, but it’s not those I target because production workloads are very well covered by everyone else, whereas new users aren’t to the same extent.
The reason I bring that up is because it’s impossible to please everyone. Whether over lesser things like file size (as I say, the snap started out 150MB and is now closer to 1.3GB, that happened over about 2-3 months), the libraries you bundle (the snap forces its own versions of packages, and the UX is hostile to “sideloading”, to the point some would find it easier to just rebuild their own), or the features embedded (I can’t get tkinter in Python working without breaking ACLiC, and had to make the choice which I’d prefer, it was ACLiC, you can still use TKInter in Jupyter, but you won’t get ACLiC at all if I broke that).
Specifically then I’d encourage you to consider:
- Desktop entries to the main CLI (
root) itself
- Are all the various root commands supported and exposed? E.G
hadd is a good example, I know from other Flatpaks that systems don’t tend to add the flatpak bin directory to $PATH. Do your users have instructions recommending they should, to avoid needing e.g flatpak run cern.root.hadd.
- Importantly, I’d simulate a few updates on your own workstation, and make sure that things like filepaths don’t come into play, does ROOT keep working reliably on upgrades? To what extent?
For ACLIC in the snap for example, you get about 2 years of stability before I upgrade the runtime environment to break ABI compatibility. That doesn’t effect 99% of users, but it means there’s a specific point where I know I am breaking people with no ability to do anything about that. It’s important that they know that so they can react appropriately if needs be. (Though in the student use case, that’s basically just rerun the compiler, it’s not a big ask, but it can be a surprising ask).
And ultimately, being a Flatpak, people are going to expect that it works cross distribution. Have you tested a wide variety of distributions? With the snap 90% of users use Ubuntu, which fits with my target audience. But there are 10% who would use Fedora, CentOS, RHEL, etc, where it happens to work too.
With Flatpak, I’d imagine you’d have a much much more varied demographic to cover, so it’d be worth actively trialing a few of them to see if the expectations hold up.
In summary, there are tests yes, and you should look into using them. But if you know in advance that you’ll never hit 100% compatibility because it’s impossible even at a technical level, then my advice would be to actually set a specific goal and work towards that.
I tell people regularly to not use my own package, there’s no shame in that. It’s good at what it focuses on, and less good where it doesn’t, that’s not an attack on myself or the technology behind it, just a truth that has to be acknowledged. Ultimately, it’s had somewhere around 30,000-50,000 unique users over its lifetime; I can see people install when University starts and uninstall when University ends. And if they then move onto a different ROOT package more suited for them after that point, well I’ve done my job.