[redis-db] Recommended way to expire an entire hash

[Martin Perez]

Hello everyone.

I know that there is no SETEX semantics for the key of a Hash object which
means the developer needs to deal with the atomic expiration in the best
way possible to avoid race conditions. What would be the best-recommended
pattern to expire an entire hash on a system that is under a decent write
load (e.g. half million writes per day). I am considering approaches like
sending an expire on every write, write a Lua script or keep a set with
keys an expiration times and have custom delete logic in the application
for traversing the expire set. But I'm not totally confident about the
performance implications of any of these measures. An option for setting an
expiration when creating a hash would come really handy here although
perhaps this approach ends up being the same as sending a new expire with
every update approach.

Any ideas?

Best Regards,
Martin

[Itamar Haber]

Hello Martin,

I would argue that in lieu of a single atomic command, this use case can be
safely addressed by sending an expire on every write. It is simple to
implement in your code and shifts the burden of handling expiry to the
Redis server.

There is no real concern about atomicity IMO, worst case some expiry times
will be off by a millisecond, so you don't need a Lua script or even a
MULTI/EXEC (although a pipeline would be beneficial).

In terms of performance, intuitively and without actually benchmarking,
pipelining the two commands will be good enough. A Lua script may or may
not provide some gain, but not substantially. A custom Redis module may
outperform both, but I don't expect an order of magnitude difference. If
you're up to it, I recommend you do some benchmarking yourself (and
hopefully share the results with us here ;)).

Cheers,

Post by Martin Perez
Hello everyone.
I know that there is no SETEX semantics for the key of a Hash object which
means the developer needs to deal with the atomic expiration in the best
way possible to avoid race conditions. What would be the best-recommended
pattern to expire an entire hash on a system that is under a decent write
load (e.g. half million writes per day). I am considering approaches like
sending an expire on every write, write a Lua script or keep a set with
keys an expiration times and have custom delete logic in the application
for traversing the expire set. But I'm not totally confident about the
performance implications of any of these measures. An option for setting an
expiration when creating a hash would come really handy here although
perhaps this approach ends up being the same as sending a new expire with
every update approach.
Any ideas?
Best Regards,
Martin
--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--
*Itamar Haber*
Technicalist Evangely

Phone: [phone]

[image: Redis Labs] <https://redislabs.com/>

[Martin Perez]

Hi Itamar,

Thanks for the advice here. It really helps. I'll put some numbers here if
they are worth sharing but I also believe the performance impact should be
negligible. I have some concern about dealing with cache pollution when the
expire operation fails due to some network glitch. I imagine retries and
batch traverse-clean processes should deal with this issue effectively.

Regards,
Martin

Post by Itamar Haber
Hello Martin,
I would argue that in lieu of a single atomic command, this use case can
be safely addressed by sending an expire on every write. It is simple to
implement in your code and shifts the burden of handling expiry to the
Redis server.
There is no real concern about atomicity IMO, worst case some expiry times
will be off by a millisecond, so you don't need a Lua script or even a
MULTI/EXEC (although a pipeline would be beneficial).
In terms of performance, intuitively and without actually benchmarking,
pipelining the two commands will be good enough. A Lua script may or may
not provide some gain, but not substantially. A custom Redis module may
outperform both, but I don't expect an order of magnitude difference. If
you're up to it, I recommend you do some benchmarking yourself (and
hopefully share the results with us here ;)).
Cheers,

Post by Martin Perez
Hello everyone.
I know that there is no SETEX semantics for the key of a Hash object
which means the developer needs to deal with the atomic expiration in the
best way possible to avoid race conditions. What would be the
best-recommended pattern to expire an entire hash on a system that is under
a decent write load (e.g. half million writes per day). I am considering
approaches like sending an expire on every write, write a Lua script or
keep a set with keys an expiration times and have custom delete logic in
the application for traversing the expire set. But I'm not totally
confident about the performance implications of any of these measures. An
option for setting an expiration when creating a hash would come really
handy here although perhaps this approach ends up being the same as sending
a new expire with every update approach.
Any ideas?
Best Regards,
Martin
--
You received this message because you are subscribed to the Google Groups
"Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--
*Itamar Haber*
Technicalist Evangely
Phone: [phone]
[image: Redis Labs] <https://redislabs.com/>