Net-Async-Graphite/lib/Net/Async/Graphite/API.pm
package Net::Async::Graphite::API;
our $VERSION = '0.1_1';
=head1 NAME
Net::Async::Graphite::API - Interface between perl and graphite.
=head1 SYNOPSIS
with 'Net::Async::Graphite::API';
=head1 DESCRIPTION
Don't use this module directly, use L<Net::Async::Graphite> and create
objects using its C<new> method in the normal way. Those objects will
include the functionality documented here.
This role brings the capacity to transmit HTTP requests to graphite
and asynchronously wait for and return the response.
It also defines a structure describing complex target paths in a perl
data structure.
=head1 BUGS
Assumes graphite-api and doesn't handle transport errors at all.
Using Memoize may be pointles.
=cut
use v5.14;
use strictures 2;
use Moo::Role;
use Carp;
use Memoize qw(memoize);
use PerlX::Maybe qw(maybe);
use URI;
use namespace::clean;
=head1 ROLE
Asynchronity is achieved by using L<Net::Async::HTTP> inside
L<Net::Async::Graphite::HTTPClient>.
JSON data is decoded using a L<JSON> object handled by
L<Net::Async::Graphite::JSONCodec>.
=cut
with 'Net::Async::Graphite::HTTPClient';
with 'Net::Async::Graphite::JSONCodec';
=head1 ATTRIBUTES
=over
=item endpoint (required, read-only)
The base of the graphite URL to which C</metrics> and C</render>
requests will be directed. Everything up to and B<including> the
trailing C</>. eg. C<https://monitoring.example.com:8080/graphite/>.
If the trailing C</> is not included it's assumed you know what you're
doing so only a warning will be printed.
=cut
has endpoint => (
is => 'ro',
isa => sub {
carp 'endpoint should include a trailing "/", attempting to continue'
unless $_[0] =~ /\/$/;
croak 'endpoint must include a protocol' unless $_[0] =~ /^[a-z-]+:/i;
},
required => 1,
);
=back
=head1 PUBLIC METHODS
All of these methods return a L<Future> which will complete with the
indicated result (or which will die).
=over
=item metrics ($method, $query, [%extra])
Perform C<$method> on/with the metrics which match C<$query>, which
should be a simple scalar describing the query to perform, or an
arrayref of such scalars if the method can handle multiple query
parameters. ie. this will request a URI which looks like:
C<< <graphite-endpoint>/metrics/<$method>?query=<$query>[&query=...] >>. If
arguments are supplied in C<%extra>, they are added to the generated
URI as encoded query parameters.
The three methods which graphite-api exposes at this time are supported:
=over
=item find
=item expand
=item index[.json]
C</metrics/index.json> can be requested with C<$method> of C<index> or
C<index.json> because requiring the C<.json> suffix is silly. Also I
can consider creating real methods C<find>, C<expand> and C<index>.
=back
The L<Future> will complete with the http content. If you want the
HTTP request object see the C<_download> private method, but note
I<private>.
=cut
my %allowed_metrics_params = (
expand => [qw(groupByExpr leavesOnly)],
find => [qw(wildcards from until position)],
index => [],
);
$allowed_metrics_params{'index.json'} = $allowed_metrics_params{index}; # FFS
sub metrics {
my $self = shift;
my ($method, $query, %extra) = @_;
$method = 'index.json' if $method eq 'index';
return Future->fail("Invalid metrics method: $method")
unless exists $allowed_metrics_params{$method};
my $uri = URI->new($self->endpoint . "metrics/$method");
my %query = (
maybe query => $query, # No query in index
map { maybe $_ => $extra{$_} } @{ $allowed_metrics_params{$method} },
);
# Only if .../find. Is default_* stupid?
$query{from} ||= $self->default_from if $self->has_default_from;
$query{until} ||= $self->default_until if $self->has_default_until;
$uri->query_form(%query);
$self->_download($uri)->then(sub { Future->done($_[0]->content) });
}
=item metrics_asperl ($method, $query, [%extra])
Calls C<metrics()> with the same arguments and decodes the result,
which is assumed to be JSON text, into a perl data structure.
=cut
sub metrics_asperl {
my $self = shift;
$self->metrics(@_)->then(sub { Future->done($self->_json->decode(shift)); });
}
# L<http://graphite.readthedocs.io/en/latest/render_api.html>. ie. this
=item render ($format, $target, [%extra])
Fetch the metric data for the given C<$target>, which should be a
simple scalar specifying a path identifying metrics, or an arrayref of
such scalars, as described in the
L<< graphite C</render> documentation | http://graphite.readthedocs.io/en/latest/render_api.html >>
C<render()> will then request a URI which looks like:
C<< <graphite-endpoint>/render?format=<$format>&target=<$query>[&target=...] >>.
The name of the format may also be called as a method directly,
without the first (C<$format>) argument:
=over
=item csv ($target, [%extra])
=item dygraph ($target, [%extra])
=item json ($target, [%extra])
=item pdf ($target, [%extra])
=item png ($target, [%extra])
=item raw ($target, [%extra])
=item rickshaw ($target, [%extra])
=item svg ($target, [%extra])
=back
=cut
sub render {
my $self = shift;
my ($format, $target, %extra) = @_;
my $uri = URI->new($self->endpoint . 'render');
my %query = (
format => $format,
target => $target,
map { maybe $_ => $extra{$_} } qw(graphType jsonp maxDataPoints noNullPoints), # Many more
);
$uri->query_form(%query);
$self->_download($uri)->then(sub { Future->done($_[0]->content) });
}
for my $format (qw(
csv
dygraph
json
pdf
png
raw
rickshaw
svg
)) {
no strict 'refs';
*{$format} = sub { $_[0]->render($format => @_[1..$#_]) };
}
=item render_asperl ($target, [%extra])
Calls C<render()> with the same arguments but prepended by the format
C<raw>, and decodes the result.
Graphite's raw format is documented in
L<< graphite C</render> documentation | http://graphite.readthedocs.io/en/latest/render_api.html >>
but in short it's an ASCII (or UTF-8 if you're so inclined) encoded
file consisting of one line per metric found. Each line consists of a
header and the data separated by a C<|>. Within each of these each
datum is separated by a C<,>.
I don't know or care if the line endings include C<\r>.
The data is returned as a list of hashrefs containing the data broken
up into B<all> of its components. ie. Each datum of each metric will
be turned into its own perl scalar.
Each hashref will contain five values:
=over
=item target
The path of metric.
=item start
Unix timestamp pinpointing the beginning of the data.
=item end
Unix timestamp pinpointing the end of the data.
=item step
The time interval in seconds (probably) between each datum.
=item data
The data. An arrayref of however many scalars it takes.
The data are B<not> converted from their original text form, and
graphite include the string C<None> in the list of returned values, so
not every datum will necessarily look like a number.
=back
=cut
sub render_asperl {
my $self = shift;
$self->render(raw => @_)->then(sub {
my @sets = map {
chomp;
my ($head, $data) = split /\|/;
my %set;
(@set{qw(target start end step)}) = split /,/, $head;
$set{data} = [ split /,/, $data ];
\%set;
} split /\n/, $_[0];
Future->done(@sets);
});
}
=item find_target_from_spec ($spec, ...)
Construct strings for use by the C<target> parameter of C</render>
queries from the specifications in each C<$spec>, which must be a
plain text scalar of the metric's path or an arrayref containing 2
items (second optional):
=over
=item $function
The name of a function as defined by the graphite API.
=item [ @arguments ] (optional)
An arrayref containing 0 or more arguments. Each argument may be a
scalar, which left alone, or an arrayref which must itself be another
C<$spec>-like scalar/pair.
=back
It sounds a lot more confusing than it is. These are valid:
# Scalar
'simple.metric.name'
# Becomes "simple.metric.name"
# Function
[ 'summarize' => [ 'simple.metric.name' ] ]
# Becomes "summarize(simple.metric.name)"
# Function with more arguments
[ 'summarize' => [ 'me.tr.ic', '"arg1"', 42 ] ]
# Becomes "summarize(me.tr.ic,%52arg1%52,42)"
# Recursive function
[
'summarize' => [
'randomize' => [
'pointless.metric', 'argue', 'bicker'
],
'and-fight',
]
]
# Becomes "summarize(randomize(pointless.metric,argue,bicker),and-fight)"
Any other form is not.
Simply put, where it's not just a plain string the specification is a
recursive nest of function/argument-list pairs, where each argument
can itself be another function pair.
The implementation of the target string construction uses L<Memoize>
to avoid repeatedly calling functions which return the same data. It
probably destroys any advantage gained by doing this by normalising
the function arguments with JSON first. I'll measure it some day.
=cut
sub find_target_from_spec {
my $self = shift;
return map { __construct_target_argument($_) } @_;
}
=back
=head1 PRIVATE METHODS
=over
=item __construct_target_argument ($argument)
This is not an object or class method.
Compile C<$argument> into its part of the string which will build up
the (or a) target component of a Graphite request URI. C<$argument>
must be a scalar or an arrayref with exactly one or two items in
it. If included, the second must be an arrayref of the function's
arguments.
Returns the scalar as-is or calls C<__construct_target_function> to
decode the arrayref.
This function uses L<Memoize> for which it normalises its arguments
using L<JSON>.
=cut
memoize('__construct_target_argument',
NORMALIZER => sub { ref($_[0]) ? encode_json($_[0]) : $_[0] });
sub __construct_target_argument {
my ($argument) = @_;
my $ref = ref $argument;
if (not $ref) {
$argument;
} elsif ($ref eq 'ARRAY' and scalar @$argument <= 2) {
croak 'Function arguments must be an arrayref'
unless not defined $argument->[1] or ref $argument->[1] eq 'ARRAY';
__construct_target_function($argument->[0], @{ $argument->[1] || [] });
} else {
croak "Unknown argument reference type: $ref";
}
}
=item __construct_target_function ($name, [@arguments])
This is not an object or class method.
Compile C<$name> and C<@arguments> into their part of the string which
will build up the target component of a Graphite request URI. Recurses
back into C<__construct_target_argument> to build each argument
component.
This function uses L<Memoize> for which it normalises its arguments
using L<JSON>.
=cut
memoize('__construct_target_function',
NORMALIZER => sub { encode_json({@_}) });
sub __construct_target_function {
my $name = shift;
croak 'Function name must be a scalar' if ref $name;
my @arguments = map { __construct_target_argument($_) } @_;
$name . '(' . join(',', @arguments) . ')';
}
1;
=back
=head1 SEE ALSO
Graphite's C</render> documentation L<http://graphite.readthedocs.io/en/latest/render_api.html>
L<Net::Async::Graphite>
L<Net::Async::Graphite::HTTPClient>
L<Net::Async::Graphite::JSONCodec>
L<Future>
L<Moo>
L<Net::Async::HTTP>
=head1 AUTHOR
Matthew King <matthew.king@cloudbeds.com>
=cut