List All Available Redis Keys

1. Overview

Collections are an essential building block typically seen in almost all modern applications. So, it’s no surprise that Redis offers a variety of popular data structures such as lists, sets, hashes, and sorted sets for us to use.

In this tutorial, we’ll learn how we can effectively read all available Redis keys that match a particular pattern.

2. Explore Collections

Let’s imagine that our application uses Redis to store information about balls used in different sports. We should be able to see information about each ball available from the Redis collection. For simplicity, we’ll limit our data set to only three balls:

  • Cricket ball with a weight of 160 g
  • Football with a weight of 450 g
  • Volleyball with a weight of 270 g

As usual, let’s first clear our basics by working on a naive approach to exploring Redis collections.

3. Naive Approach Using redis-cli

Before we start writing Java code to explore the collections, we should have a fair idea of how we’ll do it using the redis-cli interface. Let’s assume that our Redis instance is available at 127.0.0.1 on port 6379, for us to explore each collection type with the command-line interface.

3.1. Linked List

First, let’s store our data set in a Redis linked list named balls in the format of sports-name_ball-weight with the help of the rpush command:

% redis-cli -h 127.0.0.1 -p 6379
127.0.0.1:6379> RPUSH balls "cricket_160"
(integer) 1
127.0.0.1:6379> RPUSH balls "football_450"
(integer) 2
127.0.0.1:6379> RPUSH balls "volleyball_270"
(integer) 3

We can notice that a successful insertion into the list outputs the new length of the list. However, in most cases, we’ll be blind to the data insertion activity. As a result, we can find out the length of the linked list using the llen command:

127.0.0.1:6379> llen balls
(integer) 3

When we already know the length of the list, it’s convenient to use the lrange command to retrieve the entire data set easily:

127.0.0.1:6379> lrange balls 0 2
1) "cricket_160"
2) "football_450"
3) "volleyball_270"

3.2. Set

Next, let’s see how we can explore the data set when we decide to store it in a Redis set. To do so, we first need to populate our data set in a Redis set named balls using the sadd command:

127.0.0.1:6379> sadd balls "cricket_160" "football_450" "volleyball_270" "cricket_160"
(integer) 3

Oops! We had a duplicate value in our command. But, since we were adding values to a set, we don’t need to worry about duplicates. Of course, we can see the number of items added from the output response-value.

Now, we can leverage the smembers command to see all the set members:

127.0.0.1:6379> smembers balls
1) "volleyball_270"
2) "cricket_160"
3) "football_450"

3.3. Hash

Now, let’s use Redis’s hash data structure to store our dataset in a hash key named balls such that hash’s field is the sports name and the field value is the weight of the ball. We can do this with the help of hmset command:

127.0.0.1:6379> hmset balls cricket 160 football 450 volleyball 270
OK

To see the information stored in our hash, we can use the hgetall command:

127.0.0.1:6379> hgetall balls
1) "cricket"
2) "160"
3) "football"
4) "450"
5) "volleyball"
6) "270"

3.4. Sorted Set

In addition to a unique member-value, sorted-sets allows us to keep a score next to them. Well, in our use case, we can keep the name of the sport as the member value and the weight of the ball as the score. Let’s use the zadd command to store our dataset:

127.0.0.1:6379> zadd balls 160 cricket 450 football 270 volleyball
(integer) 3

Now, we can first use the zcard command to find the length of the sorted set, followed by the zrange command to explore the complete set:

127.0.0.1:6379> zcard balls
(integer) 3
127.0.0.1:6379> zrange balls 0 2
1) "cricket"
2) "volleyball"
3) "football"

3.5. Strings

We can also see the usual key-value strings as a superficial collection of items. Let's first populate our dataset using the mset command:

127.0.0.1:6379> mset balls:cricket 160 balls:football 450 balls:volleyball 270
OK

We must note that we added the prefix “balls:so that we can identify these keys from the rest of the keys that may be lying in our Redis database. Moreover, this naming strategy allows us to use the keys command to explore our dataset with the help of prefix pattern matching:

127.0.0.1:6379> keys balls*
1) "balls:cricket"
2) "balls:volleyball"
3) "balls:football"

4. Naive Java Implementation

Now that we have developed a basic idea of the relevant Redis commands that we can use to explore collections of different types, it's time for us to get our hands dirty with code.

4.1. Maven Dependency

In this section, we'll be using the Jedis client library for Redis in our implementation:

<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>3.2.0</version>
</dependency>

4.2. Redis Client

The Jedis library comes with the Redis-CLI name-alike methods. However, it's recommended that we create a wrapper Redis client, which will internally invoke Jedis function calls.

Whenever we're working with Jedis library, we must keep in mind that a single Jedis instance is not thread-safe. Therefore, to get a Jedis resource in our application, we can make use of JedisPool, which is a threadsafe pool of network connections.

And, since we don't want multiple instances of Redis clients floating around at any given time during the life cycle of our application, we should create our RedisClient class on the principle of the singleton design pattern.

First, let's create a private constructor for our client that'll internally initialize the JedisPool when an instance of RedisClient class is created:

private static JedisPool jedisPool;

private RedisClient(String ip, int port) {
try {
if (jedisPool == null) {
jedisPool = new JedisPool(new URI("http://" + ip + ":" + port));
}
} catch (URISyntaxException e) {
log.error("Malformed server address", e);
}
}

Next, we need a point of access to our singleton client. So, let's create a static method getInstance() for this purpose:

private static volatile RedisClient instance = null;

public static RedisClient getInstance(String ip, final int port) {
if (instance == null) {
synchronized (RedisClient.class) {
if (instance == null) {
instance = new RedisClient(ip, port);
}
}
}
return instance;
}

Finally, let's see how we can create a wrapper method on top of Jedis's lrange method:

public List lrange(final String key, final long start, final long stop) {
try (Jedis jedis = jedisPool.getResource()) {
return jedis.lrange(key, start, stop);
} catch (Exception ex) {
log.error("Exception caught in lrange", ex);
}
return new LinkedList();
}

Of course, we can follow the same strategy to create the rest of the wrapper methods such as lpush, hmset, hgetall, sadd, smembers, keys, zadd, and zrange.

4.3. Analysis

All the Redis commands that we can use to explore a collection in a single go will naturally have an O(n) time complexity in the best case.

We are perhaps a bit liberal, calling this approach as naive. In a real-life production instance of Redis, it's quite common to have thousands or millions of keys in a single collection. Further, Redis's single-threaded nature brings more misery, and our approach could catastrophically block other higher-priority operations.

So, we should make it a point that we're limiting our naive approach to be used only for debugging purposes.

5. Iterator Basics

The major flaw in our naive implementation is that we're requesting Redis to give us all of the results for our single fetch-query in one go. To overcome this issue, we can break our original fetch query into multiple sequential fetch queries that operate on smaller chunks of the entire dataset.

Let's assume that we have a 1,000-page book that we're supposed to read. If we follow our naive approach, we'll have to read this large book in a single sitting without any breaks. That'll be fatal to our well-being as it'll drain our energy and prevent us from doing any other higher-priority activity.

Of course, the right way is to finish the book over multiple reading sessions. In each session, we resume from where we left off in the previous session — we can track our progress by using a page bookmark.

Although the total reading time in both cases will be of comparable value, nonetheless, the second approach is better as it gives us room to breathe.

Let’s see how we can use an iterator-based approach for exploring Redis collections.

6. Redis Scan

Redis offers several scanning strategies to read keys from collections using a cursor-based approach, which is, in principle, similar to a page bookmark.

6.1. Scan Strategies

We can scan through the entire key-value collection store using the Scan command. However, if we want to limit our dataset by collection types, then we can use one of the variants:

  • Sscan can be used for iterating through sets
  • Hscan helps us iterate through pairs of field-value in a hash
  • Zscan allows an iteration through members stored in a sorted set

We must note that we don’t really need a server-side scan strategy specifically designed for the linked lists. That’s because we can access members of the linked list through indexes using the lindex or lrange command. Plus, we can find out the number of elements and use lrange in a simple loop to iterate the entire list in small chunks.

Let’s use the SCAN command to scan over keys of string type. To start the scan, we need to use the cursor value as “0”, matching pattern string as “ball*”:

127.0.0.1:6379> mset balls:cricket 160 balls:football 450 balls:volleyball 270
OK
127.0.0.1:6379> SCAN 0 MATCH ball* COUNT 1
1) "2"
2) 1) "balls:cricket"
127.0.0.1:6379> SCAN 2 MATCH ball* COUNT 1
1) "3"
2) 1) "balls:volleyball"
127.0.0.1:6379> SCAN 3 MATCH ball* COUNT 1
1) "0"
2) 1) "balls:football"

With each completed scan, we get the next value of cursor to be used in the subsequent iteration. Eventually, we know that we’ve scanned through the entire collection when the next cursor value is “0”.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Rakesh Tripathi

Rakesh Tripathi

Consulting Engineer, Software Developer, Infra, Quora