redis.pl that implements a client for the
[redis](https://redis.io/), ” An open source (BSD licensed),
in-memory data structure store, used as a database, cache and message
broker’
Redis is an in-memory key-value store. Redis can be operated as a simple store for managing (notably) volatile persistent data. Redis can operate in serveral modes, ranging from a single server to clusters organised in several different ways to provide high availability, resilience and replicating the data close to the involved servers. In addition to being a key-value store, Redis enables additional communication between clients such as publish/subscribe to message channels, streams, etc.
These features can be used to connect micro services, both for sharing state, notifications and distributing tasks.
The connection between the redis client and server uses a stream pair. Although SWI-Prolog stream I/O is thread-safe, having multiple threads using this same connection will mixup writes and their replies.
At the moment, the following locking is in place.
If SWI-Prolog includes the ssl library, the Redis client
can connect to the server using TLS (SSL). Connecting requires the same
three files as redis-cli requires: the root certificate
file, a client certificate and the private key of the client
certificate. Below is an example call to redis_server/3:
:- redis_server(swish, localhost:6379,
[ user(bob),
password("topsecret"),
version(3),
tls(true),
cacert('ca.crt'),
key('client.key'),
cert('client.cert')
]).
Redis sentinels is one of the two options to create a high
availability service. It consists of minimally three Redis servers and
mininally three sentinel servers. The sentinel servers monitor the Redis
servers and will initiate a fail-over when the master becomes
disfunctional and certain safety constraints are satisfied. A client
needs to be aware of this setup. It is given an initial list with (a
subset of) the known sentinels. The client attempts to connect to one of
the sentinels and ask it for the current Redis master server. Details
are described in Sentinel
client spec. The SWI-Prolog client maintains the actual list of
sentinels dynamically after successful discovery of the first sentinel.
Below is an example
redis_server/3 to connect to
a sentinel network. The Address specification
sentinel(swish) tells the library we want to connect to a
sentinel network that is monitored under the name swish.
:- redis_server(swish_sentinel, sentinel(swish),
[ user(janbob),
password("topsecret"),
version(3),
sentinels([ host1:26379,
host2:26379,
host3:26379
])
]).
The current stable version of Redis is 6. Many Linux distros still ship with version 5. Both talk protocol version 2. Version 6 also supports protocol version 3. The main differences are:
Name-Value), while version 2 exchanges hashes as a list of
alternating names and values. This is visible to the user. High level
predicates such as redis_get_hash/3
deal with both representations.New projects are encouraged to use Redis version 6 with the version 3 protocol. See redis_server/3.
Starting with Redis 5, redis supports streams. A stream is a
list of messages. Streams can be used as a reliable alternative to the
older Redis PUB/SUB (Publish Subscribe) mechanism that has no memory,
i.e., if a node is down when a message arrives the message is missed. In
addition, they can be used to have each message processed by a
consumer that belongs to a consumer group. Both facilities
are supported by library(redis_streams) (section
3)
Redis streams provide all the low-level primitives to realise message brokering. Putting it all together is non-trivial though. Notably:
The directory doc/packages/examples/redis in the
installation provides an example using streams and consumer groups to
realise one or more clients connected to one or more compute nodes.
This module is based on the gpredis.pl by Sean Charles
for GNU-Prolog. This file greatly helped me understanding what had to be
done, although, eventually, not much of the original interface is left.
The main difference to the original client are:
Value as prolog, after which
they are returns as a (copy of) Value. This prefixes the value using "\u0000T\u0000".
This library is a client to Redis, a popular key value store to deal with caching and communication between micro services.
In the typical use case we register the details of one or more Redis servers using redis_server/3. Subsequenly, redis/2-3 is used to issue commands on the server. For example:
?- redis_server(default, redis:6379, [password("secret")]).
?- redis(default, set(user, "Bob")).
?- redis(default, get(user), User).
User = "Bob"
default
points at localhost:6379 with no connect options. The default
server is used for redis/1 and redis/2
and may be changed using this predicate. Options are
described with redis_connect/3.
Connections established this way are by default automatically
reconnected if the connection is lost for some reason unless a
reconnect(false) option is specified.
redis_connect(+Address, -Connection, +Options). redis_connect/1
is equivalent to
redis_connect(localhost:6379, Connection, []). Options:
true, try to reconnect to the service when the
connection seems lost. Default is true for connections
specified using
redis_server/3 and false
for explictly opened connections.version(3) and password(Password) are
specified, these are used to authenticate using the HELLO
command.3, the HELLO
command is used to upgrade the protocol.cacert, key and cert
options.sentinel(MasterName)
to enable contacting a network of Redis servers guarded by a sentinel
network.Instead of using these predicates, redis/2 and redis/3 are normally used with a server name argument registered using redis_server/3. These predicates are meant for creating a temporary paralel connection or using a connection with a blocking call.
| Address | is a term Host:Port, unix(File)
or the name of a server registered using redis_server/3.
The latter realises a new connection that is typically used for
blocking redis commands such as listening for published messages,
waiting on a list or stream. |
redis_connect(-Connection, +Host, +Port) provides
compatibility to the original GNU-Prolog interface and is equivalent to redis_connect(Host:Port, Connection, []).redis-cli), we accept the certificate
as long as it is signed, not verifying the hostname.SENTINEL SLAVES mastername
true (default false), do not raise any
errors if
Connection does not exist or closing the connection raises a
network or I/O related exception. This version is used internally if a
connection is in a broken state, either due to a protocol error or a
network issue.redis(Connection, Command, _) and
second, it can be used to exploit Redis pipelines and transactions.
The second form is acticated if Request is a list. In
that case, each element of the list is either a term Command -> Reply
or a simple
Command. Semantically this represents a sequence of redis/3
and
redis/2 calls. It differs in the
following aspects:
multi and the last exec,
the commands are executed as a Redis transaction, i.e., they are
executed atomically.Procedurally, the process takes the following steps:
Command -> Reply terms.Examples
?- redis(default,
[ lpush(li,1),
lpush(li,2),
lrange(li,0,-1) -> List
]).
List = ["2", "1"].
"A:B:...".
This is a common shorthand for representing Redis keys.\u0000T\u0000"
followed by Term in canonical form.
Reply is either a plain term (often a variable) or a term Value as Type.
In the latter form, Type dictates how the Redis bulk
reply is translated to Prolog. The default equals to auto,
i.e., as a number of the content satisfies the Prolog number syntax and
as an atom otherwise.
status(Atom) Returned if the server replies with + Status.
Atom is the textual value of Status converted to lower case,
e.g., status(ok) or status(pong).nil This atom is returned for a NIL/NULL value. Note
that if the reply is only nil, redis/3 fails.
The nil value may be embedded inside lists or maps.\u0000T\u0000"
it is supposed to be a Prolog term. Note that this intepretation means
it is not possible to read arbitrary binary blobs.nil. If Reply
as a whole would be nil the call fails.
Redis bulk replies are translated depending on the as Type
as explained above.
bytes (iso_latin_1), utf8
and text (the current locale translation).type_error(Type, String)
is raised.min_tagged_integer
and max_tagged_integer, allowing the value to be used as a
dict key.auto(atom, number)auto(atom,tagged_integer). This allows the value
to be used as a key for a SWI-Prolog dict.pairs
type can also be applied to a Redis array. In this case the array length
must be even. This notably allows fetching a Redis
hash as pairs using HGETALL using version 2 of the
Redis protocol.pairs(AsKey, AsValue), but convert the resulting
pair list into a SWI-Prolog dict. AsKey must convert to a
valid dict key, i.e., an atom or tagged integer. See dict_key.dict(dict_key, AsValue).Here are some simple examples
?- redis(default, set(a, 42), X).
X = status("OK").
?- redis(default, get(a), X).
X = "42".
?- redis(default, get(a), X as integer).
X = 42.
?- redis(default, get(a), X as float).
X = 42.0.
?- redis(default, set(swipl:version, 8)).
true.
?- redis(default, incr(swipl:version), X).
X = 9.
redis_error(Code, String)LRANGE
requests. Note that this results in O(N^2) complexity.
Using a lazy list is most useful for relatively short lists holding
possibly large items.
Note that values retrieved are strings, unless the value was
added using Term as prolog.
It seems possible for LLEN to return OK. I
don't know why. As a work-around we return the empty list rather than an
error.
[], Key
is deleted. Note that key values are always strings in Redis.
The same conversion rules as for
redis/1-3 apply.HGETALL
command. If the Redis hash is not used by other (non-Prolog)
applications one may also consider using the
Term as prolog syntax to store the Prolog dict as-is.| Tag | is the SWI-Prolog dict tag. |
SCAN, SSCAN, HSCAN
and ZSCAN‘commands into a lazy list. For redis_scan/3
and redis_sscan/4 the result
is a list of strings. For redis_hscan/4
and redis_zscan/4, the result
is a list of pairs. Options processed:
MATCH subcommand, only returning matches for
Pattern.COUNT subcommand, giving a hint to the size of the
chunks fetched.TYPE subcommand, only returning answers of the
indicated type.redis(info, String) and parses the result. As this is for
machine usage, properties names *_human are skipped.redis(Id, Channel, Data)
If redis_unsubscribe/2 removes the last subscription, the thread terminates.
To simply print the incomming messages use e.g.
?- listen(redis(_, Channel, Data),
format('Channel ~p got ~p~n', [Channel,Data])).
true.
?- redis_subscribe(default, test, Id, []).
Id = redis_pubsub_3,
?- redis(publish(test, "Hello world")).
Channel test got "Hello world"
1
true.
| Id | is the thread identifier of the listening
thread. Note that the Options alias(Name) can be
used to get a system wide name. |
| Channels | is either a single channel or a list thereof. Each channel specification is either an atom or a term‘A:B:...`, where all parts are atoms. |
A Redis stream is a set of messages consisting of key-value pairs that are identified by a time and sequence number. Streams are powerful objects that can roughly be used for three purposes:
This library abstracts the latter two scenarios. The main predicates are
MAXLEN ~ Count
option to the XADD command, capping the length of the
stream. See also
Redis as a message brokering system (section
1.5)maxlen(Count). If Id is unbound,
generating the id is left to the server and Id is unified
with the returned id. The returned id is a string consisting of the time
stamp in milliseconds and a sequence number. See Redis docs
for details.XREAD on one or more Streams on the
server Redis. For each message that arrives, call broadcast/1,
where Data is a dict representing the message.
broadcast(redis(Redis, Stream, Id, Data))
Options:
0 to start get all messages from the epoch
or $ to get messages starting with the last. Default is $.
Note that this predicate does not terminate. It is normally
executed in a thread. The following call listens to the streams
key1 and key2 on the default Redis
server. Using
reconnect(true), the client will try to re-establish a
connection if the collection got lost.
?- redis_connect(default, C, [reconnect(true)]),
thread_create(xlisten(C, [key1, key2], [start($)]),
_, [detached(true)]).
| Redis | is either a Redis server name (see redis_server/3) or an open connection. If it is a server name, a new connection is opened that is closed if xlisten/3 completes. |
XACK
is sent to the server.Options processed:
XREADGROUP to return with timeout when no messages
arrive within Seconds. On a timeout, xidle_group/5
is called which will try to handle messages to other consumers pending
longer than Seconds. Choosing the time depends on the
application. Notably:
max_deliveries(Count) is exceeded. Note that the original
receiver does not notice that the job is claimed and thus multiple
consumers may ultimately answer the message.XCLAIM) a message max Count
times. Exceeding this calls xhook/2.
Default Count is 3.10.redis(stop(Leave)), which is
caught by xlisten_group/5.XACK. From introduction
to streams:
"So once the deliveries counter reaches a given large number that you chose, it is probably wiser to put such messages in another stream and send a notification to the system administrator. This is basically the way that Redis streams implement the concept of the dead letter."