Neo4j-Driver/lib/Neo4j/Driver/Net.pod
# PODNAME: Neo4j::Driver::Net
# ABSTRACT: Explains the design of the networking modules
__END__
=pod
=encoding UTF-8
=head1 NAME
Neo4j::Driver::Net - Explains the design of the networking modules
=head1 VERSION
version 1.00
=head1 OVERVIEW
I<This document will be removed from the CPAN distribution with the
next major update. It will continue to be available
L<on GitHub|https://github.com/johannessen/neo4j-driver-perl/blob/0.52/lib/Neo4j/Driver/Net.pod>.>
Each L<Neo4j::Driver::Session> has exactly one network controller
instance that is used by the session and all of its transactions
to communicate with the Neo4j server. This document discusses the
features and known limitations of the network controllers.
B<Unless you're planning to develop custom network adapters
for the driver, you probably don't need to read this document.>
The controllers don't communicate with the server directly. Instead,
they use another module that has responsibility for the actual
network transmissions. For HTTP connections, that other module can
be customised via L<Neo4j::Driver::Plugin/"http_adapter_factory">.
For Bolt, see L</"EXTENSIONS"> below.
Network responses received from the server will be parsed for Neo4j
statement results by the appropriate result handler for the response
format used by the server. A custom networking module can also
provide custom response parsers, for example implemented in XS code.
Please note that the division of labour between sessions or
transactions on the one hand and networking controllers on the other
hand is an internal implementation detail of the driver and as such
is B<subject to unannounced change.> While some of those details are
explained in this document, this is done only to help contributors
and users of I<public> APIs better understand the driver's design.
See L<Neo4j::Driver::Plugin/"USE OF INTERNAL APIS"> for more
information on this topic.
=head1 SYNPOSIS
$helper = Neo4j::Driver::Net::HTTP->new($driver);
$helper->_set_database($db_name);
$helper->{active_tx} = ...;
@results = $helper->_run($tx, @statements);
# Parsing a JSON result
die unless $helper->{http_agent}->http_header->{success};
$json_coder = $helper->{http_agent}->json_coder;
$json_coder->decode( $helper->{http_agent}->fetch_all );
B<WARNING:> Most of these calls are private APIs.
See L<Neo4j::Driver::Plugin/"USE OF INTERNAL APIS">.
=head1 FEATURES
The networking controllers primarily deal with the following tasks:
=over
=item * Establish a database connection.
=item * Provide L<Neo4j::Driver::ServerInfo>.
=item * Handle certain generic protocol requirements
(such as HTTP content negotiation).
=item * Sync state between server transactions and
driver transaction objects.
=item * Control the translation of Cypher statements
to network transmissions, and of network transmissions
to statement results.
=back
HTTP connections use proactive content negotiation
(L<RFC 7231|https://tools.ietf.org/html/rfc7231#section-3.4.1>)
to obtain a suitable response from the Neo4j server. The driver
supports both Jolt and JSON as result formats. There is also
a fallback result handler, which is used to parse error
messages out of C<text/*> responses. The HTTP result handlers
are individually queried for the media types they support.
This information is cached by the networking controller.
All result handlers inherit a common interface from
L<Neo4j::Driver::Result>. They provide methods to initialise and
bless result data records as L<Neo4j::Driver::Record> objects.
Result handlers are also responsible that all values returned
from Neo4j are provided to users in the format that is documented
for L<Neo4j::Driver::Record/"get">. For backwards compatibility,
a lot of the internal data structures currently (0.52) match the
format of Neo4j JSON responses.
The first HTTP connection to a Neo4j server is always made to the
L<Discovery API|https://neo4j.com/docs/http-api/4.2/discovery/>,
which is used to obtain L<Neo4j::Driver::ServerInfo> and the
transaction endpoint URI template. These are the only GET
requests made by the driver.
Because of a known issue with Neo4j, the C<Accept> request
header field needs to be varied by HTTP request method
(L<#12644|https://github.com/neo4j/neo4j/issues/12644>).
With HTTP being a stateless protocol, Neo4j supports multiple
concurrent transactions by using a different URL for each one
in a REST-like fashion. For requests made to such explicit
transaction endpoints, the Neo4j
L<Transactional HTTP API|https://neo4j.com/docs/http-api/4.2/actions/>
always provides transaction status information in the response.
Transactions that remain open include an expiration time. The
networking controller parses and stores this timestamp and uses it to
track which transactions are still open and which have timed out.
The origination C<Date> field is used to synchronise the clocks
of the driver and the Neo4j server
(L<RFC 7231|https://tools.ietf.org/html/rfc7231#section-7.1.1.2>).
Bolt, on the other hand, currently only supports a single open
transaction per connection. While a Bolt connection can be viewed
as a simple state machine in the backend Bolt library (see
L<Bolt Protocol Server State Spec|https://neo4j.com/docs/bolt/current/appendix/version-4/>),
L<Neo4j::Bolt> currently doesn't allow users to directly observe
state changes, so it is currently somewhat difficult to determine
the Bolt connection state. This driver infers it based
on the behaviour of L<Neo4j::Bolt>.
=head1 COMPATIBILITY
L<Neo4j::Driver> S<version 0.52> is compatible with
S<L<Neo4j::Bolt> 0.01> or later.
Note that some L<Neo4j::Bolt> and L<Neo4j::Client> versions are
broken, or simply incompatible with each other. Recommended pairs
of known-good module versions are as follows:
+-- Neo4j::Bolt
| +-- Neo4j::Client
| | +-- max recommended Neo4j server
| | |
0.5000 0.54 5.x
0.4203 0.46 4.4
0.20 0.17 3.4
0.12 - 3.4 (system libneo4j-client)
When using HTTP, S<L<Neo4j::Driver> 0.52> supports determining the
version of any Neo4j server via L<Neo4j::Driver::Session/"server">.
This even works on S<Neo4j 1.x>, but running statements on
S<Neo4j 1.x> will fail, because it lacks the transactional API.
S<L<Neo4j::Driver> 0.52> is compatible with Neo4j S<versions 2.x>,
3.x, S<4.x, and 5.x>. It supports HTTP responses in the
L<formats|https://neo4j.com/docs/http-api/4.4/actions/result-format/>
JSON and Jolt (strict and sparse, line-delimited and sequenced).
Only HTTP 1.1 is supported.
For Bolt as well as HTTP, future versions of the driver will tend
to implement new requirements in order to stay compatible with
newer versions of Neo4j. Support for old Neo4j or library versions
is likely to only be dropped with major updates to the driver
(such as S<0.x to 1.x>).
=head1 BUGS AND LIMITATIONS
Clock synchronisation using the HTTP C<Date> header does not take
into account network delays. In case of high network latency, the
driver may treat transactions as open even though they have already
expired on the server. To address this, you could either increase
the transaction idle timeout in F<neo4j.conf> or manipulate the
return value of C<date_header()> in a custom network adapter.
The metadata in HTTP JSON responses is often insufficient to fully
describe the response data. In particular:
=over
=item * Path metadata doesn't include node labels or relationship
type (L<#12613|https://github.com/neo4j/neo4j/issues/12613>).
=item * Records with fields that are maps or lists have unparsable
metadata (L<#12306|https://github.com/neo4j/neo4j/issues/12306>).
=item * Byte arrays are coded as lists of integers in JSON results.
=back
For further limitations related to data types, see
L<Neo4j::Driver::Types/"BUGS AND LIMITATIONS">.
Should you discover any other issues with the driver, please be sure
to report them on GitHub.
=head2 IPv6 / dual-stack support
The driver supports IPv4 and IPv6 addresses as well as regular
hostnames of all varieties.
However, there appears to be an issue contacting S<Neo4j 4.0>
and newer in some dual-stack environments. The issue only
occurs when using HTTP networking with dual-stack hostnames
(such as C<localhost>). The issue causes a delay upon sending
a query to the server, usually by the duration of the timeout
(as configured in the driver, or the socket's default timeout);
however, an actual timeout exception is never reported to the
driver. The issue only occurs I<sometimes> and at this point
it's not known how to reproduce it reliably.
I think this is caused in part by some slightly odd behaviour in
L<setup() in IO::Socket::IP|https://metacpan.org/release/PEVANS/IO-Socket-IP-0.41/source/lib/IO/Socket/IP.pm#L668>.
For the localhost address, the while loop is usually executed
twice, with the first call to C<< $self->connect( $addr ) >>
failing (with a C<$!> of "Connection refused"), but the second
call succeeding. It seems like the first call uses IPv6 to ::1,
but the second uses IPv4 to 127.0.0.1. For some reason, the
first call I<sometimes> blocks until the timeout. The second
call then still succeeds, which is why there is no timeout
error reported by L<LWP>. This behaviour is documented under
L<IO::Socket::IP/"Timeout =E<gt> NUM">.
Specifying C<127.0.0.1> instead of C<localhost> seems to avoid
this problem entirely. The same seems to be true for C<::1>
(not sure why).
The algorithm known as L<Happy Eyeballs
(RFC 8305)|https://datatracker.ietf.org/doc/html/rfc8305>
can prevent this kind of issue, but L<IO::Socket::IP> doesn't
currently implement it
(L<RT 78136|https://rt.cpan.org/Ticket/Display.html?id=78136>).
This issue never seemed to occur with Neo4j 3.5, which raises
the question which of the changes that came with S<version 4.0>
may have caused this. While I don't have the answer, it seems
that explicitly enabling or disabling IPv6 in the S<Neo4j 4.0>
server configuration in F<neo4j.conf> may help to avoid
triggering this issue:
# Disable IPv6 in neo4j.conf
dbms.jvm.additional=-Djava.net.preferIPv4Stack=true
# Enable IPv6 in neo4j.conf
dbms.jvm.additional=-Djava.net.preferIPv4Stack=false
dbms.jvm.additional=-Djava.net.preferIPv6Addresses=true
=head1 EXTENSIONS
The C<net_module> config option available in driver S<versions 0.21>
S<through 0.30> has been replaced with an experimental plug-in API;
see L<Neo4j::Driver::Deprecations/"Custom networking modules">.
=head2 Custom Bolt networking modules
By default, Bolt networking uses the amazing XS module
L<Neo4j::Bolt> by Mark A. Jensen (MAJENSEN), which in turn uses the
S<C library> L<libneo4j-omni|https://majensen.github.io/libneo4j-omni/>
to actually connect to the Neo4j server. Updates and improvements
are quite possibly best made directly in those libraries, so that
not only L<Neo4j::Driver>, but also other users benefit from them.
There is currently no public API for custom Bolt network adapters;
see L<Neo4j::Driver::Plugin/"Network adapter API for Bolt">.
However, the driver's test suite still uses the former C<net_module>
option internally, so it will remain available for the time being.
use Local::MyBolt;
$driver->config(uri => 'bolt://...');
$driver->{config}{net_module} = 'Local::MyBolt'; # private API
The module name provided will be used in place of
L<Neo4j::Bolt> and will have to match its API I<exactly.>
Note that there is no support for this approach. For details,
see L<Neo4j::Driver::Plugin/"USE OF INTERNAL APIS">.
=head2 Custom HTTP networking modules
L<Neo4j::Driver> includes a default HTTP network adapter.
The included adapter may change in future. As of S<version 0.52>,
the default adapter uses L<LWP> directly. Versions earlier
S<than 0.21> used L<REST::Client>.
The driver's test suite still uses the former C<net_module>
option internally. Modules to be used as C<net_module> must
implement the API for an HTTP adapter; see
L<Neo4j::Driver::Plugin/"Network adapter API for HTTP">.
Additionally, they must implement the following method:
=over
=item new
sub new {
my ($class, $driver) = @_;
...
}
Initialises the object. May or may not establish a network
connection. May access C<$driver> config options using the
method L<Neo4j::Driver/"config"> only.
=back
=head2 Custom result handlers
This section has been replaced by
L<Neo4j::Driver::Plugin/"Result handler API">.
=head1 USE OF INTERNAL APIS
This section has been replaced by
L<Neo4j::Driver::Plugin/"USE OF INTERNAL APIS">.
=head1 AUTHOR
Arne Johannessen (L<AJNN|https://metacpan.org/author/AJNN>)
=head1 COPYRIGHT AND LICENSE
This software is Copyright (c) 2016-2024 by Arne Johannessen.
This is free software; you can redistribute it and/or modify it under
the terms of the Artistic License 2.0 or (at your option) the same terms
as the Perl 5 programming language system itself.
=cut