גאבור סבו - Gábor Szabó > Pipe > Pipe



Annotate this POD

View/Report Bugs
Module Version: 0.05   Source  


Pipe - Framework to create pipes using iterators


 use Pipe;
 my @input = Pipe->cat("t/data/file1", "t/data/file2")->run;
 my @lines = Pipe->cat("t/data/file1", "t/data/file2")->chomp->run;
 my @uniqs = Pipe->cat("t/data/file1", "t/data/file2")->chomp->uniq->run;

 my $pipe = Pipe->cat("t/data/file1", "t/data/file2")->uniq->print("t/data/out");


This is Alpha version. The user API might still change


Building an iterating pipe with prebuilt and home made tubes.




    $self->logger("log messages");

The method that actually executes the whole pipe.

my $pipe = Pipe->cat("file"); $pipe->run;


Tubes available in this distibution:


Read in the lines of one or more file.


Remove trailing newlines from each line.



Returns every file, directory, etc. under the directory tree passed to it.



Iterates over the elements of an array. Basically the same as the for or foreach loop of Perl.


Implements the Perl glob function.


Selectively pass on values.

Can be used either with a regex:

 ->grep( qr/regex/ )

Or with a sub:

 ->grep( sub { length($_[0]) > 12 } )

Very similar to the built-in grep command of Perl but instead of regex you have to pass a compiled regex using qr// and instead of a block you have to pass an anonymous sub {}


Similar to the Perl map construct, except that instead of a block you pass an anonymous function sub {}.

 ->map(  sub {  length $_[0] } );

Prints out its input. By default it prints to STDOUT but the user can supply a filename or a filehandle.

 Pipe->cat("t/data/file1", "t/data/file2")->print("out.txt");
 Pipe->cat("t/data/file1", "t/data/file2")->print(':a', "out.txt");

It is the same as print but adds a newline at the end of each line. The name is Perl6 native.


Similar to the built in sort function of Perl. As sort needs to have all the data in the memory, once you use sort in the Pipe it stops being an iterator for the rest of the pipe.

By default it sorts based on ascii table but you can provide your own sorting function. The two values to be compared are passed to this function.

 Pipe->cat("t/data/numbers1")->chomp->sort( sub { $_[0] <=> $_[1] } );

Given a regex (or a simple string), will split all the incoming strings and return an array reference for each row.

Param: string or regex using qr//

Input: string(s)

Output: array reference(s)


Given one or more array references, on every iteration it will return an n-tuple (n is the number of arrays), one value from each source array.

    my @a = qw(foo bar baz moo);
    my @b = qw(23  37  77  42);

    my @one_tuple = Pipe->tuple(\@a);
    # @one_tuple is ['foo'], ['bar'], ['baz'], ['moo']

    my @two_tuple = Pipe->tuple(\@a, \@b);
    # @two_tuple is ['foo', 23], ['bar', 37], ['baz', 77], ['moo', 42]

Input: disregards any input so it can be used as a starting element of a Pipe

Ouput: array refs of n elements


Similary to the unix uniq command eliminate duplicate consecutive values.

23, 23, 19, 23 becomes 23, 19, 23

Warning: as you can see from the example this method does not give real unique values, it only eliminates consecutive duplicates.

Building your own tube ^

If you would like to build a tube called "thing" create a module called Pipe::Tube::Thing that inherits from Pipe::Tube, our abstract Tube.

Implement one or more of these methods in your subclass as you please.


Will be called once when initializing the pipeline. It will get ($self, @args) where $self is the Pipe::Tube::Thing object and @args are the values given as parameters to the ->thing(@args) call in the pipeline.


Will be called every time the previous tube in the pipe returns one or more values. It can return a list of values that will be passed on to the next tube. If based on the current state of Thing there is nothing to do you should call return; with no parameters.


Will be called once when the Pipe Manager notices that this Thing should be finished. This happens when Thing is the first active element in the pipe (all the previous tubes have already finshed) and its run() method returns an empty list.

The finish() method should return a list of values that will be passed on to the next tube in the pipe. This is especially useful for Tubes such as sort that can to their thing only after they have received all the input.

Debugging your tube

You can call $self->logger("some message") from your tube. It will be printed to pipe.log if someone sets $Pipe::DEBUG = 1;

Examples ^

A few examples of UNIX Shell commands combined with pipelines


Probably plenty but nothing I know of. Please report them to the author.

Development ^

The Subversion repository is here: http://svn1.hostlocal.com/szabgab/trunk/Pipe/

Thanks ^

to Gaal Yahas


Gabor Szabo <gabor@pti.co.il>


Copyright 2006 by Gabor Szabo <gabor@pti.co.il>.


See http://www.perl.com/perl/misc/Artistic.html

See Also ^

Shell::Autobox and File::Tools

syntax highlighting: