Dear mgv4ce,
PROOF can help in providing more CPU but the bottleneck can be somewhere else, especially if the files do not reside on a fast media, like SSD. Also, with respect to serial processing, there is the additional merging step in the end which can be heavy for large output objects. So the optimal number of workers may not be the maximum available.
All this said, I would start by checking what happens with 1 worker (this gives an idea of the overhead of PROOF in your setup) and 2 and 4 workers which gives hints about the scale up.
Can you also provide the following information:
- size of the files and type of device where they reside
- amount of memory available on the machine
G Ganis