Pod-ssh for configuring a cluster of workers with pool disk on each worker

[Chinmay]

Hi,
I am trying to set up a small cluster of 4 machines each with 24 workers using pod-ssh.
Each machine has a disk storage so I want to configure each machine to work as server/data-server.
This case has been described at https://root.cern.ch/configuring-cluster-workers-pool-disk-each-worker.
I understand I can customize the proof configuration in PoD by adding ‘user_xpd.cf*’ to $HOME/.PoD. So I tried
adding the file as given at above link to $HOME/.PoD. But it doesn’t seem to work.
I am able to set up PROOF cluster using standard PoD (as described in PoD manual with PoD_ssh.cfg file.) way.

[ganis]

What exactly

@Chinmay

doesn’t seem to work

?
Are you able to start the cluster without customized additions of the form ‘user_xpd.cf*’?

This said, the information in the link needs to be reviewed. I will have a look.
Could you specify which version of ROOT you are working with?

G Ganis

[Chinmay]

Yes.
I am able to start the cluster without customized additions.
I am doing
pod-ssh --config pod-ssh.cf --debug submit
where pod-ssh.cf looks like this (testing with 1 server 8 workers)

@bash_begin@    
    # Temp dir
    export TMPDIR=/home/chinmay/tmp
    # ROOT
    . /home/chinmay/root-6.08.04/PROOF_INSTALLATION/bin/thisroot.sh
    # XROOTD
    . /home/chinmay/root-6.08.04/PROOF_INSTALLATION/bin/setxrd.sh /home/chinmay/xrootd-4.5.0
@bash_end@

r3, chinmay@10.159.63.110,-X,/home/chinmay/tmp,8

If I understand right, according to above link, for data-serving I need to have xrootd/cmsd running on all servers.
While PoD seems to be concerned only with setting up PROOF cluster. So what’s the solution ? Can I have some example script that I can run with pod-ssh to set up xrootd/cmsd ?

[Chinmay]

I tried $HOME/.PoD/etc/user_xpd.cf0 of the following form

xrd.port 1094
if named PoDServer
 all.adminpath _POD_TMP_DIR
else
 all.adminpath _G_WORKER_TMP_DIR
 xpd.sockpathdir _G_WORKER_TMP_SOCK_DIR
fi
xrd.fslib default
if named PoDServer
 ofs.forward all
fi 
all.export _POD_TMP_DIR/data
if named PoDServer
 all.role manager
else 
 all.role server
fi 
all.manager PoDServer 3121
cms.allow host PoDServer

But, after adding this file, I get

[chinmay@chinmay ~]$ pod-server start
Starting PoD server...
updating xproofd configuration file...
starting xproofd...
WARNING: Can't detect whether xproofd has started or not...
starting PoD agent...
WARNING: Can't detect whether pod-agent has started or not...
preparing PoD worker package...
selecting pre-compiled bins to be added to worker package...
Failed to create the package. There is a missing component: "/home/chinmay/.PoD/etc/server_info.cfg"
Error: Can't start PoD server. Please check pod-agent.server.log and xpd.log for more information.

xpd-log.txt (695 Bytes)
pod-agent-server-log.txt (1.9 KB)

[ganis]

Hi,
It depends of what kind of data serving you want, i.e. persistent and independent from PROOF, or for PROOF operations.
PoD is designed for on-the-service PROOF clusters, so the daemons go away when you close the sessions.
If you want generic access to the data indipendently of PROOF, so with or without PRROF, a separate XRootD setup maybe the right choice. This would also allow you some file management, i.e. to control the file population of your cluster.

G Ganis

[Chinmay]

I need data serving only for PROOF operations.
In fact above user_xpd.cf0 file is just the modification of the data-serving configuration
provided at the link.
Sorry I forgot to mention the versions. It is ROOTv6.08.04 and PoD3.17 with xrootd4.5.0