NAME POE::Component::Curl::Multi - a fast HTTP POE component VERSION version 0.06 SYNOPSIS use strict; use warnings; use HTTP::Request::Common qw[GET]; use POE qw[Component::Curl::Multi]; $!=1; my @urls = ( 'https://api.github.com/repos/git/git/tags', 'http://this.is.made.up.stuff/', 'http://www.cpan.org/', 'http://www.google.com/', ); my $curl = POE::Component::Curl::Multi->spawn( Alias => 'curl', FollowRedirects => 5, Max_Concurrency => 10, ); POE::Session->create( package_states => [ main => [qw(_start _response)], ], ); $poe_kernel->run(); exit 0; sub _start { $poe_kernel->post( 'curl', 'request', '_response', GET($_) ) for @urls; return; } sub _response { my ($request_packet, $response_packet) = @_[ARG0, ARG1]; use Data::Dumper; local $Data::Dumper::Indent=1; warn Dumper( $response_packet->[0] ); return; } DESCRIPTION POE::Component::Curl::Multi is an HTTP user-agent for POE. It lets other sessions run while HTTP transactions are being processed, and it lets several HTTP transactions be processed in parallel. It uses WWW::Curl internally to provide access to "libcurl" for fast performance. It strives to be API compatible(ish) with POE::Component::Client::HTTP. It is inspired by AnyEvent::Curl::Multi. CONSTRUCTOR "spawn" Starts an instance of the component. Takes a number of options: "alias" Set an alias for this instance of the component. There is no default. "timeout" Specifies a timeout, in seconds, for each request, defaults to 180. "followredirects" Specifies how many redirects (e.g. 302 Moved) to follow. If not specified defaults to 0, and thus no redirection is followed. "agent" Can either be a string or an "ARRAYREF" of strings. This will be used as "UserAgent" header in requests. If an "ARRAYREF" is provided one of them will be picked randomly to send. See "proxy" Specify a proxy to use. See "max_concurrency" Specify the maximum number of concurrent requests, a value of 0 means no limit will be imposed. The default is 0. "curl_debug" Enable "libcurl"'s verbosity. See Returns an object that accepts some methods as documented below. AVAILABLE METHODS "session_id" Takes no arguments. Returns the ID of the component's session. "shutdown" Responds to all pending requests with 408 (request timeout), and then shuts down the component and all subcomponents. "pending_requests_count" Returns the number of requests currently being processed. "cancel" Cancel a specific HTTP request. Requires a reference to the original request (blessed or stringified) so it knows which one to cancel. ACCEPTED EVENTS Sessions communicate asynchronously with the component. They post requests to it, and it posts responses back. "request" Requests are posted to the component's "request" state. They include an HTTP::Request object which defines the request. For example: $kernel->post( 'ua', 'request', # http session alias & state 'response', # my state to receive responses GET('http://poe.perl.org'), # a simple HTTP request 'unique id', # a tag to identify the request 'progress', # an event to indicate progress 'http://1.2.3.4:80/' # proxy to use for this request ); This invocation is compatible with POE::Component::Client::HTTP. You may also send either an "arrayref" or "hashref" to the "request" state with the following parameters: "request" A HTTP::Request object which defines the request. "response" Either a string specifying the state in the sender session to receive responses or, alternatively, a POE::Session "postback" that will be invoked with responses. "tag" A tag to identify the request. "progress" An optional handler, if specified the component will provide progress metrics (see sample handler below). "proxy" Specify a proxy to use. This overrides the "proxy" set with "spawn", if applicable. See "session" Specify a POE::Session object, ID or alias to send responses to instead of the sending session. If a "postback" is used for "response", this option will be ignored. "pending_requests_count" Returns the number of requests currently being processed. To receive the return value, it must be invoked with $kernel->call(). my $count = $kernel->call('ua' => 'pending_requests_count'); This is also available as a method on the object returned by "spawn" my $count = $curl->pending_requests_count(); "shutdown" Responds to all pending requests with 408 (request timeout), and then shuts down the component and all subcomponents. SENT EVENTS response handler In addition to all the usual POE parameters, HTTP responses come with two list references: my ($request_packet, $response_packet) = @_[ARG0, ARG1]; $request_packet contains a reference to the original HTTP::Request object. This is useful for matching responses back to the requests that generated them. my $http_request_object = $request_packet->[0]; my $http_request_tag = $request_packet->[1]; # from the 'request' post $response_packet contains a reference to the resulting HTTP::Response object. my $http_response_object = $response_packet->[0]; Please see the HTTP::Request and HTTP::Response manpages for more information. progress handler The example progress handler shows how to calculate a percentage of download completion. sub progress_handler { my $gen_args = $_[ARG0]; # args passed to all calls my $call_args = $_[ARG1]; # args specific to the call my $req = $gen_args->[0]; # HTTP::Request object being serviced my $tag = $gen_args->[1]; # Request ID tag from. my $got = $call_args->[0]; # Number of bytes retrieved so far. my $tot = $call_args->[1]; # Total bytes to be retrieved. my $percent = $got / $tot * 100; printf( "-- %.0f%% [%d/%d]: %s\n", $percent, $got, $tot, $req->uri() ); return; } STREAMING This component does not (yet) support POE::Component::Client::HTTP's streaming options. CLIENT HEADERS POE::Component::Curl::Multi sets its own response headers with additional information. All of its headers begin with "X-PCCH". X-PCCH-Errmsg POE::Component::Curl::Multi may fail because of an internal client error rather than an HTTP protocol error. X-PCCH-Errmsg will contain a human readable reason for client failures, should they occur. The text of X-PCCH-Errmsg may also be repeated in the response's content. X-PCCH-Peer This response header is not yet supported. ATTRIBUTION POE::Component::Curl::Multi is based on both POE::Component::Client::HTTP by Rocco Caputo, Rob Bloodgood and Martijn van Beers and AnyEvent::Curl::Multi by Michael S. Fischer SEE ALSO WWW::Curl AnyEvent::Curl::Multi POE::Component::Client::HTTP AUTHOR Chris Williams COPYRIGHT AND LICENSE This software is copyright (c) 2014 by Chris Williams, Michael S. Fischer, Rocco Caputo, Rob Bloodgood and Martijn van Beers. This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.