NAME

App::PipeFilter::Generic - a generic pipeline filter.

SYNOPSIS

  package App::PipeFilter::JsonToYaml;
  
  use Moose;
  extends 'App::PipeFilter::Generic';
  
  with (
    "App::PipeFilter::Role::Reader::Sysread",
    "App::PipeFilter::Role::Input::Json",
    "App::PipeFilter::Role::Transform::None",
    "App::PipeFilter::Role::Output::Yaml",
  );
  
  1;

DESCRIPTION

App::PipeFilter::Generic is a generic shell pipeline filter. It is designed to be customized by subclassing and consuming roles that implement specific behaviors.

For example, the App::PipeFilter::JsonToYaml manpage extends the generic pipeline filter with a role to read data in large chunks, a role to parse that input as a stream of JSON objects, a role that doesn't alter the data that has been read, and a role to format the output as YAML.

PUBLIC METHODS

decode_input SCALAR_REF

decode_input() accepts a reference to an input buffer containing raw octets from some input stream. It deserializes as many input records as it can from the buffer. It removes the deserialized octets from the buffer, leaving any unused data for another time. decode_input() returns an array of the deserialized data structures.

This method is usually implemented by Input roles.

This is App::PipeFilter::Role::Input::Json sub decode_input().

  sub decode_input {
    my ($self, $buffer_ref) = @_;
    my (@return) = $self->_json()->incr_parse($$buffer_ref);
    $$buffer_ref = "";
    return @return;
  }

encode_output ARRAY

encode_output() accepts an array of Perl data structures to be output. It returns an array of serialized data, one element for each input data structure.

This method is usually implemented by Output roles.

This is App::PipeFilter::Role::Output::Yaml sub encode_output().

  sub encode_output {
    # Skips $self in $_[0].
    return map { Dump($_) } @_[1..$#_];
  }

filter_file IN_FILEHANDLE, OUT_FILEHANDLE

filter_file() translates one input file to output. IN_FILEHANDLE is an open filehandle to the source data. OUT_FILEHANDLE is an open filehandle to the output data sink.

filter_file() invokes read_input(), decode_input(), transform() and encode_output() until a single input file is completely filtered.

This is filter_file().

  sub filter_file {
    my ($self, $ifh, $ofh) = @_;
  
    my $buffer = "";
  
    while ($self->read_input($ifh, \$buffer)) {
      next unless my (@input) = $self->decode_input(\$buffer);
      next unless my (@output) = $self->transform(@input);
      print $ofh $_ foreach $self->encode_output(@output);
    }
  }

next_input_file

next_input_file() returns the name of the next file to read. App::PipeFilter::Generic's run() method uses it to iterate over the list of input files provided on the command line.

next_input_file() is implemented in the App::PipeFilter::Role::Flags::Standard manpage.

open_input FILENAME

open_input() opens a named file and returns the opened filehandle to it. The special file name "-" opens STDIN.

App::PipeFilter::Generic's run() method uses open_input() to open the files named by next_input_file().

open_input() is implemented in Opener roles.

This is App::PipeFilter::Role::Opener::GenericInput sub open_input().

  sub open_input {
    my ($self, $filename) = @_;
  
    if ($filename eq '-') {
      warn "reading from standard input\n" if $self->verbose();
      return \*STDIN;
    }
  
    warn "reading from $filename\n" if $self->verbose();
    open my $fh, "<", $filename or die "can't open $filename: $!";
    return $fh;
  }

open_output FILENAME

open_output() opens a named file and returns the opened filehandle to it. The special file name "-" opens STDOUT.

App::PipeFilter::Generic's run() method uses open_output() to open standard output for writing results.

open_output() is implemented in Opener roles.

This is App::PipeFilter::Role::Opener::GenericOutput sub open_output().

  sub open_output {
    my ($self, $filename) = @_;
  
    if ($filename eq '-') {
      warn "writing to standard output\n" if $self->verbose();
      return \*STDOUT;
    }
  
    warn "writing to $filename\n" if $self->verbose();
    open my $fh, ">", $filename or die "can't write $filename: $!";
    return $fh;
  }

run

run() opens standard output using open_output("-"). It then calls next_input_file() to iterate through all the input file names provided on the command line. Each input file is opened with open_input(). The resulting input and output file handles are passed to filter_file(), which filters the entire file.

run() is implemented in App::PipeFilter::Generic.

This is run().

  sub run {
    my $self = shift();
  
    my $ofh = $self->open_output('-');
  
    while (defined (my $input_filename = $self->next_input_file())) {
      my $ifh = $self->open_input($input_filename);
      $self->filter_file($ifh, $ofh);
    }
  
    # Exit value.
    0;
  }

transform ARRAY

transform() accepts an array of data references. It returns another array of data references after performing some transformative function on each one. The resulting array may have greater or fewer items than the input array. transform() may return an empty array if all input records are to be discarded.

Common data transformations may be implemented in Transform roles. Mostly however, individual utility classes like the App::PipeFilter::JsonMap manpage define their own transformations.

This is App::PipeFilter::Role::Transform::None sub transform().

  sub transform {
    # Skips $self in $_[0].
    return @_[1..$#_];
  }

BUGS

https://rt.cpan.org/Public/Dist/Display.html?Name=App-PipeFilter

REPOSITORY

TODO - Standard =repository

https://github.com/rcaputo/app-pipefilter