chirp_distribute is a quick and simple way for replicating a directory from a Chirp server to many Chirp Servers by creating a spanning tree and then transferring data concurrently from host to host using third party transfer. It is faster than manually copying data using parrot cp, chirp_put or chirp_third_put
chirp_distribute also can clean up replicated data using -X option.
-a,--auth <flag> | |
Require this authentication mode. | |
-d,--debug <flag> | |
Enable debugging for this subsystem. | |
-D, --info-transfer | Show detailed location, time, and performance of each transfer. |
-F,--failures-file <file> | |
Write matrix of failures to this file. | |
-i,--tickets <files> | |
Comma-delimited list of tickets to use for authentication. | |
-N,--copies-max <num> | |
Stop after this number of successful copies. | |
-p,--jobs <num> | |
Maximum number of processes to run at once (default=100) | |
-R, --randomize-hosts | Randomize order of target hosts given on command line. |
-t,--timeout <time> | |
Timeout for for each copy. (default is 3600s) | |
-T,--timeout-all <time> | |
Overall timeout for entire distribution. (default is 3600). | |
-v, --verbose | Show program version. |
-X, --delete-target | Delete data from all of the target hosts. |
-Y, --info-success | Show confirmation of successful placements. |
-h, --help | Show help text. |
chirp_distribute server1.somewhere.edu /mydata server2.somewhere.edu server3.somewhere.edu server4.somewhere.eduTo replicate a directory from server1 to all available Chirp server(s) in Chirp catalog using chirp_distribute and chirp_status:
chirp_distribute server1.somewhere.edu /mydata \`chirp_status -s\`To replicate a directory from server1 to all available Chirp server(s) in Chirp catalog using chirp_distribute and chirp_status. However stop when reach 100 copies with -N option:
chirp_distribute -N 100 server1.somewhere.edu /mydata \`chirp_status -s\`To clean up replicated data using chirp_distribute using -X option:
chirp_distribute -X server1.somewhere.edu /mydata \`chirp_status -s\`