[redis-db] [Question]: Listing Streams / Groups

[Ben Waters]

Currently there doesn't seem to be a (documented) way of being able to list groups or streams with xinfo

Each xinfo groups/stream need a key. Is there a way to pass a wildcard? or to able to list all groups or streams?

Use case would be if a developer wants to integrate a new consumer into a stream without necessarily knowing the stream name.

[Salvatore Sanfilippo]

Hello Ben, the only way you can iterate the key space is by using SCAN
followed by TYPE in order to return all the streams.
On each stream returned you can then use XINFO to get the streams.
However your may want to reconsider your design in terms of Redis
operations:
if I want certain consumers to automatically join a list of N streams
for a specific group, then I would take a Set key that works as
service discovery, instead of trying to scan the whole space.
Such Set key basically would contain the set of keys that have a given
group, so set "discovery:some_consumer_group" will map to N key names.
However SCAN+TYPE is a viable solution in case you are using the Redis
instance only for streaming and you have a small set of keys, and you
want to avoid the complexity of taking in sync the set data type with
the list of streams and consumer groups.

Cheers,
Salvatore

Post by Ben Waters
Currently there doesn't seem to be a (documented) way of being able to list groups or streams with xinfo
Each xinfo groups/stream need a key. Is there a way to pass a wildcard? or to able to list all groups or streams?
Use case would be if a developer wants to integrate a new consumer into a stream without necessarily knowing the stream name.
--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--
Salvatore 'antirez' Sanfilippo
open source developer - Redis Labs https://redislabs.com

"If a system is to have conceptual integrity, someone must control the
concepts."
— Fred Brooks, "The Mythical Man-Month", 1975.

[Sripathi Krishnan]

Antirez,

SCAN followed by TYPE seems to be a very common pattern, and it would be
nice to have support for it within the SCAN command. Something like

SCAN 0 MATCH * TYPE STREAM
SCAN 0 MATCH * TYPE SET

I maintain a GUI client for redis, and the ability to identify streams
without having any knowledge of the key space comes in handy.

That said, I agree my use case is rather unique. I also understand that
currently there is no efficient way for redis to implement a filter for
type on top of the scan results - it will be O(n) pass anyways. But still,
it's worth a discussion I guess.



--Sri

Post by Salvatore Sanfilippo
Hello Ben, the only way you can iterate the key space is by using SCAN
followed by TYPE in order to return all the streams.
On each stream returned you can then use XINFO to get the streams.
However your may want to reconsider your design in terms of Redis
if I want certain consumers to automatically join a list of N streams
for a specific group, then I would take a Set key that works as
service discovery, instead of trying to scan the whole space.
Such Set key basically would contain the set of keys that have a given
group, so set "discovery:some_consumer_group" will map to N key names.
However SCAN+TYPE is a viable solution in case you are using the Redis
instance only for streaming and you have a small set of keys, and you
want to avoid the complexity of taking in sync the set data type with
the list of streams and consumer groups.
Cheers,
Salvatore

Post by Ben Waters
Currently there doesn't seem to be a (documented) way of being able to

list groups or streams with xinfo

Post by Ben Waters
Each xinfo groups/stream need a key. Is there a way to pass a wildcard?

or to able to list all groups or streams?

Post by Ben Waters
Use case would be if a developer wants to integrate a new consumer into

a stream without necessarily knowing the stream name.

Post by Ben Waters
--
You received this message because you are subscribed to the Google

Groups "Redis DB" group.

Post by Ben Waters
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--
Salvatore 'antirez' Sanfilippo
open source developer - Redis Labs https://redislabs.com
"If a system is to have conceptual integrity, someone must control the
concepts."
— Fred Brooks, "The Mythical Man-Month", 1975.
--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

[Itamar Haber]

@Sri - agreed, a "typed" SCAN is indeed a common pattern.

That said and as I'm sure you already know, until an extended version of
SCAN is implemented, you can Lua the call to SCAN and the filtering loop
for compartmentalization and performance.

On Tue, Nov 20, 2018 at 6:26 AM Sripathi Krishnan <

Post by Sripathi Krishnan
Antirez,
SCAN followed by TYPE seems to be a very common pattern, and it would be
nice to have support for it within the SCAN command. Something like
SCAN 0 MATCH * TYPE STREAM
SCAN 0 MATCH * TYPE SET
I maintain a GUI client for redis, and the ability to identify streams
without having any knowledge of the key space comes in handy.
That said, I agree my use case is rather unique. I also understand that
currently there is no efficient way for redis to implement a filter for
type on top of the scan results - it will be O(n) pass anyways. But still,
it's worth a discussion I guess.
--Sri

Post by Salvatore Sanfilippo
Hello Ben, the only way you can iterate the key space is by using SCAN
followed by TYPE in order to return all the streams.
On each stream returned you can then use XINFO to get the streams.
However your may want to reconsider your design in terms of Redis
if I want certain consumers to automatically join a list of N streams
for a specific group, then I would take a Set key that works as
service discovery, instead of trying to scan the whole space.
Such Set key basically would contain the set of keys that have a given
group, so set "discovery:some_consumer_group" will map to N key names.
However SCAN+TYPE is a viable solution in case you are using the Redis
instance only for streaming and you have a small set of keys, and you
want to avoid the complexity of taking in sync the set data type with
the list of streams and consumer groups.
Cheers,
Salvatore

Post by Ben Waters
Currently there doesn't seem to be a (documented) way of being able to

list groups or streams with xinfo

Post by Ben Waters
Each xinfo groups/stream need a key. Is there a way to pass a wildcard?

or to able to list all groups or streams?

Post by Ben Waters
Use case would be if a developer wants to integrate a new consumer into

a stream without necessarily knowing the stream name.

Post by Ben Waters
--
You received this message because you are subscribed to the Google

Groups "Redis DB" group.

Post by Ben Waters
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--
Salvatore 'antirez' Sanfilippo
open source developer - Redis Labs https://redislabs.com
"If a system is to have conceptual integrity, someone must control the
concepts."
— Fred Brooks, "The Mythical Man-Month", 1975.
--
You received this message because you are subscribed to the Google Groups
"Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--
*Itamar Haber*
Technicalist Evangely

Phone: [phone]

[image: Redis Labs] <https://redislabs.com/>