Repository navigation
Address large files being given #2
Description
Currently large files being given will take a long time using cp. I'm not sure we want to encourage the reproduction of large files that are already stored, so this may be a non-issue.
We should at least check file size before "giving", maybe print an error and exit if the file is too large? Or let the user know the give is going to take too long. If we want give to allow large file sharing, we need to parallelize the copy or let the user know it could take a very long time.
Activity
dmanno commentedon Sep 28, 2017
At LANL, our givedirs reside on lustre, so an easy fix would be to check if the underlying FS is lustre (or another stripe-able FS) then based on filesize, setup an appropriate stripe count for the file being copied. It won't speed up the read, but it will speed up the write.