chirp_stream_files copy duplicates a single file to multiple hosts. The <localfile> argument names a file on the local filesystem. The command will then open a connection to the following list of hosts, and stream the file to all simultaneously.
chirp_stream_files split divides an ASCII file up among multiple hosts. The first line of <localfile> is sent to the first host, the second line to the second, and so on, round-robin.
chirp_stream_files join collects multiple remote files into one. The argument <localfile> is opened for writing, and the remote files for reading. The remote files are read line-by-line and assembled round-robin into the local file.
In all cases, files are accessed in a streaming manner, making this particularly efficient for processing large files. A local file name of - indicates standard input or standard output, so that the command can be used in a pipeline.
-a,--auth <flag> | |
Require this authentication mode. | |
-b,--block-size <size> | |
Set transfer buffer size. (default is 1048576 bytes) | |
-d,--debug <flag> | |
Enable debugging for this subsystem. | |
-i,--tickes <files> | |
Comma-delimited list of tickets to use for authentication. | |
-t,--timeout <time> | |
Timeout for failure. (default is 3600s) | |
-v, --version | Show program version. |
-h, --help | Show help text. |
% chirp_stream_files copy mydata server1.somewhere.edu /mydata server2.somewhere.edu /mydata server2.somewhere.edu /mydataTo split the file mydata into subsets at three locations:
% chirp_stream_files split mydata server1.somewhere.edu /part1 server2.somewhere.edu /part2 server2.somewhere.edu /part3To join three remote files back into one called newdata:
% chirp_stream_files join newdata server1.somewhere.edu /part1 server2.somewhere.edu /part2 server2.somewhere.edu /part3