Connecting to Redis¶
Generic Client¶
This is the client used to connect directly to a standard Redis node.
- class redis.Redis(host='localhost', port=6379, db=0, password=None, socket_timeout=None, socket_connect_timeout=None, socket_keepalive=None, socket_keepalive_options=None, connection_pool=None, unix_socket_path=None, encoding='utf-8', encoding_errors='strict', charset=None, errors=None, decode_responses=False, retry_on_timeout=False, retry_on_error=None, ssl=False, ssl_keyfile=None, ssl_certfile=None, ssl_cert_reqs='required', ssl_ca_certs=None, ssl_ca_path=None, ssl_ca_data=None, ssl_check_hostname=False, ssl_password=None, ssl_validate_ocsp=False, ssl_validate_ocsp_stapled=False, ssl_ocsp_context=None, ssl_ocsp_expected_cert=None, max_connections=None, single_connection_client=False, health_check_interval=0, client_name=None, username=None, retry=None, redis_connect_func=None)[source]¶
Implementation of the Redis protocol.
This abstract class provides a Python interface to all Redis commands and an implementation of the Redis protocol.
Pipelines derive from this, implementing how the commands are sent and received to the Redis server. Based on configuration, an instance will either use a ConnectionPool, or Connection object to talk to redis.
- classmethod from_url(url, **kwargs)[source]¶
Return a Redis client object configured from the given URL
For example:
redis://[[username]:[password]]@localhost:6379/0 rediss://[[username]:[password]]@localhost:6379/0 unix://[[username]:[password]]@/path/to/socket.sock?db=0
Three URL schemes are supported:
redis:// creates a TCP socket connection. See more at: <https://www.iana.org/assignments/uri-schemes/prov/redis>
rediss:// creates a SSL wrapped TCP socket connection. See more at: <https://www.iana.org/assignments/uri-schemes/prov/rediss>
unix://
: creates a Unix Domain Socket connection.
The username, password, hostname, path and all querystring values are passed through urllib.parse.unquote in order to replace any percent-encoded values with their corresponding characters.
There are several ways to specify a database number. The first value found will be used:
A
db
querystring option, e.g. redis://localhost?db=0If using the redis:// or rediss:// schemes, the path argument of the url, e.g. redis://localhost/0
A
db
keyword argument to this function.
If none of these options are specified, the default db=0 is used.
All querystring options are cast to their appropriate Python types. Boolean arguments can be specified with string values “True”/”False” or “Yes”/”No”. Values that cannot be properly cast cause a
ValueError
to be raised. Once parsed, the querystring arguments and keyword arguments are passed to theConnectionPool
’s class initializer. In the case of conflicting arguments, querystring arguments always win.
- load_external_module(funcname, func)[source]¶
This function can be used to add externally defined redis modules, and their namespaces to the redis client.
funcname - A string containing the name of the function to create func - The function, being added to this class.
ex: Assume that one has a custom redis module named foomod that creates command named ‘foo.dothing’ and ‘foo.anotherthing’ in redis. To load function functions into this namespace:
from redis import Redis from foomodule import F r = Redis() r.load_external_module(“foo”, F) r.foo().dothing(‘your’, ‘arguments’)
For a concrete example see the reimport of the redisjson module in tests/test_connection.py::test_loading_external_modules
- lock(name, timeout=None, sleep=0.1, blocking=True, blocking_timeout=None, lock_class=None, thread_local=True)[source]¶
Return a new Lock object using key
name
that mimics the behavior of threading.Lock.If specified,
timeout
indicates a maximum life for the lock. By default, it will remain locked until release() is called.sleep
indicates the amount of time to sleep per loop iteration when the lock is in blocking mode and another client is currently holding the lock.blocking
indicates whether callingacquire
should block until the lock has been acquired or to fail immediately, causingacquire
to return False and the lock not being acquired. Defaults to True. Note this value can be overridden by passing ablocking
argument toacquire
.blocking_timeout
indicates the maximum amount of time in seconds to spend trying to acquire the lock. A value ofNone
indicates continue trying forever.blocking_timeout
can be specified as a float or integer, both representing the number of seconds to wait.lock_class
forces the specified lock implementation. Note that as of redis-py 3.0, the only lock class we implement isLock
(which is a Lua-based lock). So, it’s unlikely you’ll need this parameter, unless you have created your own custom lock class.thread_local
indicates whether the lock token is placed in thread-local storage. By default, the token is placed in thread local storage so that a thread only sees its token, not a token set by another thread. Consider the following timeline:- time: 0, thread-1 acquires my-lock, with a timeout of 5 seconds.
thread-1 sets the token to “abc”
- time: 1, thread-2 blocks trying to acquire my-lock using the
Lock instance.
- time: 5, thread-1 has not yet completed. redis expires the lock
key.
- time: 5, thread-2 acquired my-lock now that it’s available.
thread-2 sets the token to “xyz”
- time: 6, thread-1 finishes its work and calls release(). if the
token is not stored in thread local storage, then thread-1 would see the token value as “xyz” and would be able to successfully release the thread-2’s lock.
In some use cases it’s necessary to disable thread local storage. For example, if you have code where one thread acquires a lock and passes that lock instance to a worker thread to release later. If thread local storage isn’t disabled in this case, the worker thread won’t see the token set by the thread that acquired the lock. Our assumption is that these cases aren’t common and as such default to using thread local storage.
- parse_response(connection, command_name, **options)[source]¶
Parses a response from the Redis server
- pipeline(transaction=True, shard_hint=None)[source]¶
Return a new pipeline object that can queue multiple commands for later execution.
transaction
indicates whether all commands should be executed atomically. Apart from making a group of operations atomic, pipelines are useful for reducing the back-and-forth overhead between the client and server.
Sentinel Client¶
Redis Sentinel provides high availability for Redis. There are commands that can only be executed against a Redis node running in sentinel mode. Connecting to those nodes, and executing commands against them requires a Sentinel connection.
Connection example (assumes Redis exists on the ports listed below):
>>> from redis import Sentinel
>>> sentinel = Sentinel([('localhost', 26379)], socket_timeout=0.1)
>>> sentinel.discover_master('mymaster')
('127.0.0.1', 6379)
>>> sentinel.discover_slaves('mymaster')
[('127.0.0.1', 6380)]
Sentinel¶
- class redis.sentinel.Sentinel(sentinels, min_other_sentinels=0, sentinel_kwargs=None, **connection_kwargs)[source]¶
Redis Sentinel cluster client
>>> from redis.sentinel import Sentinel >>> sentinel = Sentinel([('localhost', 26379)], socket_timeout=0.1) >>> master = sentinel.master_for('mymaster', socket_timeout=0.1) >>> master.set('foo', 'bar') >>> slave = sentinel.slave_for('mymaster', socket_timeout=0.1) >>> slave.get('foo') b'bar'
sentinels
is a list of sentinel nodes. Each node is represented by a pair (hostname, port).min_other_sentinels
defined a minimum number of peers for a sentinel. When querying a sentinel, if it doesn’t meet this threshold, responses from that sentinel won’t be considered valid.sentinel_kwargs
is a dictionary of connection arguments used when connecting to sentinel instances. Any argument that can be passed to a normal Redis connection can be specified here. Ifsentinel_kwargs
is not specified, any socket_timeout and socket_keepalive options specified inconnection_kwargs
will be used.connection_kwargs
are keyword arguments that will be used when establishing a connection to a Redis server.- discover_master(service_name)[source]¶
Asks sentinel servers for the Redis master’s address corresponding to the service labeled
service_name
.Returns a pair (address, port) or raises MasterNotFoundError if no master is found.
- execute_command(*args, **kwargs)[source]¶
Execute Sentinel command in sentinel nodes. once - If set to True, then execute the resulting command on a single node at random, rather than across the entire sentinel cluster.
- master_for(service_name, redis_class=<class 'redis.client.Redis'>, connection_pool_class=<class 'redis.sentinel.SentinelConnectionPool'>, **kwargs)[source]¶
Returns a redis client instance for the
service_name
master.A
SentinelConnectionPool
class is used to retrieve the master’s address before establishing a new connection.NOTE: If the master’s address has changed, any cached connections to the old master are closed.
By default clients will be a
Redis
instance. Specify a different class to theredis_class
argument if you desire something different.The
connection_pool_class
specifies the connection pool to use. TheSentinelConnectionPool
will be used by default.All other keyword arguments are merged with any connection_kwargs passed to this class and passed to the connection pool as keyword arguments to be used to initialize Redis connections.
- slave_for(service_name, redis_class=<class 'redis.client.Redis'>, connection_pool_class=<class 'redis.sentinel.SentinelConnectionPool'>, **kwargs)[source]¶
Returns redis client instance for the
service_name
slave(s).A SentinelConnectionPool class is used to retrieve the slave’s address before establishing a new connection.
By default clients will be a
Redis
instance. Specify a different class to theredis_class
argument if you desire something different.The
connection_pool_class
specifies the connection pool to use. The SentinelConnectionPool will be used by default.All other keyword arguments are merged with any connection_kwargs passed to this class and passed to the connection pool as keyword arguments to be used to initialize Redis connections.
SentinelConnectionPool¶
Cluster Client¶
This client is used for connecting to a Redis Cluster.
RedisCluster¶
- class redis.cluster.RedisCluster(host=None, port=6379, startup_nodes=None, cluster_error_retry_attempts=3, require_full_coverage=False, reinitialize_steps=10, read_from_replicas=False, dynamic_startup_nodes=True, url=None, **kwargs)[source]¶
- determine_slot(*args)[source]¶
Figure out what slot to use based on args.
- Raises a RedisClusterException if there’s a missing key and we can’t
determine what slots to map the command to; or, if the keys don’t all map to the same key slot.
- execute_command(*args, **kwargs)[source]¶
Wrapper for ERRORS_ALLOW_RETRY error handling.
It will try the number of times specified by the config option “self.cluster_error_retry_attempts” which defaults to 3 unless manually configured.
If it reaches the number of times, the command will raise the exception
- Key argument :target_nodes: can be passed with the following types:
nodes_flag: PRIMARIES, REPLICAS, ALL_NODES, RANDOM ClusterNode list<ClusterNode> dict<Any, ClusterNode>
- classmethod from_url(url, **kwargs)[source]¶
Return a Redis client object configured from the given URL
For example:
redis://[[username]:[password]]@localhost:6379/0 rediss://[[username]:[password]]@localhost:6379/0 unix://[[username]:[password]]@/path/to/socket.sock?db=0
Three URL schemes are supported:
redis:// creates a TCP socket connection. See more at: <https://www.iana.org/assignments/uri-schemes/prov/redis>
rediss:// creates a SSL wrapped TCP socket connection. See more at: <https://www.iana.org/assignments/uri-schemes/prov/rediss>
unix://
: creates a Unix Domain Socket connection.
The username, password, hostname, path and all querystring values are passed through urllib.parse.unquote in order to replace any percent-encoded values with their corresponding characters.
There are several ways to specify a database number. The first value found will be used:
A
db
querystring option, e.g. redis://localhost?db=0If using the redis:// or rediss:// schemes, the path argument of the url, e.g. redis://localhost/0
A
db
keyword argument to this function.
If none of these options are specified, the default db=0 is used.
All querystring options are cast to their appropriate Python types. Boolean arguments can be specified with string values “True”/”False” or “Yes”/”No”. Values that cannot be properly cast cause a
ValueError
to be raised. Once parsed, the querystring arguments and keyword arguments are passed to theConnectionPool
’s class initializer. In the case of conflicting arguments, querystring arguments always win.
- get_node_from_key(key, replica=False)[source]¶
Get the node that holds the key’s slot. If replica set to True but the slot doesn’t have any replicas, None is returned.
- keyslot(key)[source]¶
Calculate keyslot for a given key. See Keys distribution model in https://redis.io/topics/cluster-spec
- load_external_module(funcname, func)[source]¶
This function can be used to add externally defined redis modules, and their namespaces to the redis client.
funcname
- A string containing the name of the function to createfunc
- The function, being added to this class.
- lock(name, timeout=None, sleep=0.1, blocking=True, blocking_timeout=None, lock_class=None, thread_local=True)[source]¶
Return a new Lock object using key
name
that mimics the behavior of threading.Lock.If specified,
timeout
indicates a maximum life for the lock. By default, it will remain locked until release() is called.sleep
indicates the amount of time to sleep per loop iteration when the lock is in blocking mode and another client is currently holding the lock.blocking
indicates whether callingacquire
should block until the lock has been acquired or to fail immediately, causingacquire
to return False and the lock not being acquired. Defaults to True. Note this value can be overridden by passing ablocking
argument toacquire
.blocking_timeout
indicates the maximum amount of time in seconds to spend trying to acquire the lock. A value ofNone
indicates continue trying forever.blocking_timeout
can be specified as a float or integer, both representing the number of seconds to wait.lock_class
forces the specified lock implementation. Note that as of redis-py 3.0, the only lock class we implement isLock
(which is a Lua-based lock). So, it’s unlikely you’ll need this parameter, unless you have created your own custom lock class.thread_local
indicates whether the lock token is placed in thread-local storage. By default, the token is placed in thread local storage so that a thread only sees its token, not a token set by another thread. Consider the following timeline:- time: 0, thread-1 acquires my-lock, with a timeout of 5 seconds.
thread-1 sets the token to “abc”
- time: 1, thread-2 blocks trying to acquire my-lock using the
Lock instance.
- time: 5, thread-1 has not yet completed. redis expires the lock
key.
- time: 5, thread-2 acquired my-lock now that it’s available.
thread-2 sets the token to “xyz”
- time: 6, thread-1 finishes its work and calls release(). if the
token is not stored in thread local storage, then thread-1 would see the token value as “xyz” and would be able to successfully release the thread-2’s lock.
In some use cases it’s necessary to disable thread local storage. For example, if you have code where one thread acquires a lock and passes that lock instance to a worker thread to release later. If thread local storage isn’t disabled in this case, the worker thread won’t see the token set by the thread that acquired the lock. Our assumption is that these cases aren’t common and as such default to using thread local storage.
- monitor(target_node=None)[source]¶
Returns a Monitor object for the specified target node. The default cluster node will be selected if no target node was specified. Monitor is useful for handling the MONITOR command to the redis server. next_command() method returns one command from monitor listen() method yields commands from monitor.
- on_connect(connection)[source]¶
- Initialize the connection, authenticate and select a database and send
READONLY if it is set during object initialization.
- pipeline(transaction=None, shard_hint=None)[source]¶
- Cluster impl:
Pipelines do not work in cluster mode the same way they do in normal mode. Create a clone of this object so that simulating pipelines will work correctly. Each command will be called directly when used and when calling execute() will only return the result stack.
- pubsub(node=None, host=None, port=None, **kwargs)[source]¶
Allows passing a ClusterNode, or host&port, to get a pubsub instance connected to the specified node
ClusterNode¶
Async Client¶
See complete example: here
This client is used for communicating with Redis, asynchronously.
Async Cluster Client¶
RedisCluster (Async)¶
ClusterNode (Async)¶
ClusterPipeline (Async)¶
Connection¶
See complete example: here
Connection¶
- class redis.connection.Connection(host='localhost', port=6379, db=0, password=None, socket_timeout=None, socket_connect_timeout=None, socket_keepalive=False, socket_keepalive_options=None, socket_type=0, retry_on_timeout=False, retry_on_error=<object object>, encoding='utf-8', encoding_errors='strict', decode_responses=False, parser_class=<class 'redis.connection.PythonParser'>, socket_read_size=65536, health_check_interval=0, client_name=None, username=None, retry=None, redis_connect_func=None)[source]¶
Manages TCP communication to and from a Redis server
Connection (Async)¶
Connection Pools¶
See complete example: here
ConnectionPool¶
- class redis.connection.ConnectionPool(connection_class=<class 'redis.connection.Connection'>, max_connections=None, **connection_kwargs)[source]¶
Create a connection pool.
If max_connections
is set, then this object raisesConnectionError
when the pool’s limit is reached.By default, TCP connections are created unless
connection_class
is specified. Use class:.UnixDomainSocketConnection for unix sockets.Any additional keyword arguments are passed to the constructor of
connection_class
.- disconnect(inuse_connections=True)[source]¶
Disconnects connections in the pool
If
inuse_connections
is True, disconnect connections that are current in use, potentially by other threads. Otherwise only disconnect connections that are idle in the pool.
- classmethod from_url(url, **kwargs)[source]¶
Return a connection pool configured from the given URL.
For example:
redis://[[username]:[password]]@localhost:6379/0 rediss://[[username]:[password]]@localhost:6379/0 unix://[[username]:[password]]@/path/to/socket.sock?db=0
Three URL schemes are supported:
redis:// creates a TCP socket connection. See more at: <https://www.iana.org/assignments/uri-schemes/prov/redis>
rediss:// creates a SSL wrapped TCP socket connection. See more at: <https://www.iana.org/assignments/uri-schemes/prov/rediss>
unix://
: creates a Unix Domain Socket connection.
The username, password, hostname, path and all querystring values are passed through urllib.parse.unquote in order to replace any percent-encoded values with their corresponding characters.
There are several ways to specify a database number. The first value found will be used:
A
db
querystring option, e.g. redis://localhost?db=0If using the redis:// or rediss:// schemes, the path argument of the url, e.g. redis://localhost/0
A
db
keyword argument to this function.
If none of these options are specified, the default db=0 is used.
All querystring options are cast to their appropriate Python types. Boolean arguments can be specified with string values “True”/”False” or “Yes”/”No”. Values that cannot be properly cast cause a
ValueError
to be raised. Once parsed, the querystring arguments and keyword arguments are passed to theConnectionPool
’s class initializer. In the case of conflicting arguments, querystring arguments always win.