Working with binary data using redis-cli?

[Frantisek Fuka]

Hello,

I need to use Redis on my Linux machine from my script (in fact it's Lua
script, but I don't have access to LuaSocket library or any other custom
library, just to plain standard io.open and io.popen commands). It would be
easiest to just execute "redis-cli set xxx yyy" and read the command output
but the problem is my data is binary and contains all possible byte values
(including possibly newlines, quotes etc.) so I have to operate in raw mode.

That's no problem. I construct a command sequence like this, for example
(just an example, in reality the values are long binary strings):

**3*
*$3*
*SET*
*$5*
*mykey*
*$5*
*hello*
**2*
*$3*
*GET*
*$5*
*mykey*

I put all of this in com.txt file and then I exec this:

*cat com.txt | redis-cli --pipe --raw -s /tmp/redis_socket*

And grab the output. This ALMOST works, the commands get executed.

But I have no idea how I can get the result of these commands in my file to
be processed afterwards. I only get the following output:

All data transferred. Waiting for the last reply...
Last reply received from server.
errors: 0, replies: 2

How do I get the actual result of my commands, or at least the result of
the last command (the expected string "hello")?

I found another possible way, using netcat, which works like this:

*cat com.txt | nc -q 1 -U /tmp/redis_socket*

This returns:

*+OK*
*$5*
*hello*

Which is exactly what I want! Great! However, there is one second pause
which is a problem for me because I need to execute many Redis commands in
quick succession. And I cannot put them all into one long file because the
commands are generated based on the results of the previous commands. And
without pause, the socket is closed before I get the result back.

So I did one final modification, I forked the whole command when calling it:

*cat com.txt | nc -q 1 -U /tmp/redis_socket > result.txt &*

(Note the "&" at the end)

Now it SEEMS to me that this works perfectly and does exactly what I need.
But I'd like to know if this is indeed the simplest solution and won't lead
to any troubles when called like this 100 times a second. Have I missed
something?

[Marc Gravell]

Would netcat close quicker if you added a

*1
QUIT

to the end of the command, causing the stream to close cleanly?

Post by Frantisek Fuka
Hello,
I need to use Redis on my Linux machine from my script (in fact it's Lua
script, but I don't have access to LuaSocket library or any other custom
library, just to plain standard io.open and io.popen commands). It would be
easiest to just execute "redis-cli set xxx yyy" and read the command output
but the problem is my data is binary and contains all possible byte values
(including possibly newlines, quotes etc.) so I have to operate in raw mode.
That's no problem. I construct a command sequence like this, for example
**3*
*$3*
*SET*
*$5*
*mykey*
*$5*
*hello*
**2*
*$3*
*GET*
*$5*
*mykey*
*cat com.txt | redis-cli --pipe --raw -s /tmp/redis_socket*
And grab the output. This ALMOST works, the commands get executed.
But I have no idea how I can get the result of these commands in my file
All data transferred. Waiting for the last reply...
Last reply received from server.
errors: 0, replies: 2
How do I get the actual result of my commands, or at least the result of
the last command (the expected string "hello")?
*cat com.txt | nc -q 1 -U /tmp/redis_socket*
*+OK*
*$5*
*hello*
Which is exactly what I want! Great! However, there is one second pause
which is a problem for me because I need to execute many Redis commands in
quick succession. And I cannot put them all into one long file because the
commands are generated based on the results of the previous commands. And
without pause, the socket is closed before I get the result back.
*cat com.txt | nc -q 1 -U /tmp/redis_socket > result.txt &*
(Note the "&" at the end)
Now it SEEMS to me that this works perfectly and does exactly what I need.
But I'd like to know if this is indeed the simplest solution and won't lead
to any troubles when called like this 100 times a second. Have I missed
something?
--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/redis-db/-/Fl6IhUbAybsJ.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/redis-db?hl=en.

--
Regards,

Marc

[Marc Gravell]

oops, I obviously meant

*1
$4
QUIT

Post by Marc Gravell
Would netcat close quicker if you added a
*1
QUIT
to the end of the command, causing the stream to close cleanly?

Post by Frantisek Fuka
Hello,
I need to use Redis on my Linux machine from my script (in fact it's Lua
script, but I don't have access to LuaSocket library or any other custom
library, just to plain standard io.open and io.popen commands). It would be
easiest to just execute "redis-cli set xxx yyy" and read the command output
but the problem is my data is binary and contains all possible byte values
(including possibly newlines, quotes etc.) so I have to operate in raw mode.
That's no problem. I construct a command sequence like this, for example
**3*
*$3*
*SET*
*$5*
*mykey*
*$5*
*hello*
**2*
*$3*
*GET*
*$5*
*mykey*
*cat com.txt | redis-cli --pipe --raw -s /tmp/redis_socket*
And grab the output. This ALMOST works, the commands get executed.
But I have no idea how I can get the result of these commands in my file
All data transferred. Waiting for the last reply...
Last reply received from server.
errors: 0, replies: 2
How do I get the actual result of my commands, or at least the result of
the last command (the expected string "hello")?
*cat com.txt | nc -q 1 -U /tmp/redis_socket*
*+OK*
*$5*
*hello*
Which is exactly what I want! Great! However, there is one second pause
which is a problem for me because I need to execute many Redis commands in
quick succession. And I cannot put them all into one long file because the
commands are generated based on the results of the previous commands. And
without pause, the socket is closed before I get the result back.
*cat com.txt | nc -q 1 -U /tmp/redis_socket > result.txt &*
(Note the "&" at the end)
Now it SEEMS to me that this works perfectly and does exactly what I
need. But I'd like to know if this is indeed the simplest solution and
won't lead to any troubles when called like this 100 times a second. Have I
missed something?
--
You received this message because you are subscribed to the Google Groups
"Redis DB" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/redis-db/-/Fl6IhUbAybsJ.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/redis-db?hl=en.

--
Regards,
Marc

--
Regards,

Marc

[Frantisek Fuka]

Yes, great! Adding "QUIT" at the end of request seems much cleaner than
forking. Thanks.

However, I am still puzzled that this (i.e. sending raw commands, getting
raw results back) cannot be done easier without socket library.

Post by Marc Gravell
oops, I obviously meant
*1
$4
QUIT

Post by Marc Gravell
Would netcat close quicker if you added a
*1
QUIT
to the end of the command, causing the stream to close cleanly?

Post by Frantisek Fuka
Hello,
I need to use Redis on my Linux machine from my script (in fact it's Lua
script, but I don't have access to LuaSocket library or any other custom
library, just to plain standard io.open and io.popen commands). It would be
easiest to just execute "redis-cli set xxx yyy" and read the command output
but the problem is my data is binary and contains all possible byte values
(including possibly newlines, quotes etc.) so I have to operate in raw mode.
That's no problem. I construct a command sequence like this, for example
**3*
*$3*
*SET*
*$5*
*mykey*
*$5*
*hello*
**2*
*$3*
*GET*
*$5*
*mykey*
*cat com.txt | redis-cli --pipe --raw -s /tmp/redis_socket*
And grab the output. This ALMOST works, the commands get executed.
But I have no idea how I can get the result of these commands in my file
All data transferred. Waiting for the last reply...
Last reply received from server.
errors: 0, replies: 2
How do I get the actual result of my commands, or at least the result of
the last command (the expected string "hello")?
*cat com.txt | nc -q 1 -U /tmp/redis_socket*
*+OK*
*$5*
*hello*
Which is exactly what I want! Great! However, there is one second pause
which is a problem for me because I need to execute many Redis commands in
quick succession. And I cannot put them all into one long file because the
commands are generated based on the results of the previous commands. And
without pause, the socket is closed before I get the result back.
*cat com.txt | nc -q 1 -U /tmp/redis_socket > result.txt &*
(Note the "&" at the end)
Now it SEEMS to me that this works perfectly and does exactly what I
need. But I'd like to know if this is indeed the simplest solution and
won't lead to any troubles when called like this 100 times a second. Have I
missed something?
--
You received this message because you are subscribed to the Google
Groups "Redis DB" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/redis-db/-/Fl6IhUbAybsJ.
.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/redis-db?hl=en.

--
Regards,
Marc

--
Regards,
Marc

[Matthew Palmer]

Post by Frantisek Fuka
Yes, great! Adding "QUIT" at the end of request seems much cleaner than
forking. Thanks.
However, I am still puzzled that this (i.e. sending raw commands, getting
raw results back) cannot be done easier without socket library.

Similarly, I've always wondered why it's so hard to drive screws with a
hammer.

- Matt

[Frantisek Fuka]

That's probably because hammer was never intended to drive screws.
Redis-cli, on the other hand, was built specifically for communicating with
Redis DB and yet you cannot use it to simply communicate with the Redis DB
and return raw results of this communication. It either interprets the
results (without "--pipe") or discards them (with "--pipe") and these
features cannot be turned off. Or maybe I'm mistaken and they CAN be turned
off. That's why I asked here. Better analogy would be "A screwdriver that
automatically drives screws of black and green color but cannot be used
with other screws at all, because they don't have the right color".

Post by Frantisek Fuka

Post by Frantisek Fuka
Yes, great! Adding "QUIT" at the end of request seems much cleaner than
forking. Thanks.
However, I am still puzzled that this (i.e. sending raw commands,

getting

Post by Frantisek Fuka
raw results back) cannot be done easier without socket library.

Similarly, I've always wondered why it's so hard to drive screws with a
hammer.
- Matt

[Javier Guerra Giraldez]

Post by Frantisek Fuka
Redis-cli, on the other hand, was built specifically for communicating with
Redis DB and yet you cannot use it to simply communicate with the Redis DB
and return raw results of this communication.

my understanding is that Redis-cli is a "Redis command line
interface", that is, a user interface. Not a client library.

--
Javier

[Frantisek Fuka]

Aha! I am Redis newbie so I was under impression that redis-cli (which I
thought stands for "client") is to be used for batch jobs and scripts
(which is shown in several places on the official site), while redis
command line interface is the "redis-server" to which I give the
configuration commands.

Anyway, thanks to Marc Gravell's advice, I now have my own REDIS client
written in pure Lua which does exactly what I wanted and I can use it in my
Lua scripts.

I never wanted to imply that redis-cli is defective. I was just thinking
that maybe I missed something in the documentation, because redis-cli must
communicate with redis server, must receive raw data from it and then it
parses and displays the data. So I thought that there must be some option
to simply disable the last step in this process and let redis-cli display
the raw data it received, without doing any further processing - the same
way it can do it with input data. If this is not the case, and redis-cli
cannot do this with data it receives from redis server, thanks for that
definitive answer anyway.

Post by Javier Guerra Giraldez

Post by Frantisek Fuka
Redis-cli, on the other hand, was built specifically for communicating

with

Post by Frantisek Fuka
Redis DB and yet you cannot use it to simply communicate with the Redis

DB

Post by Frantisek Fuka
and return raw results of this communication.

my understanding is that Redis-cli is a "Redis command line
interface", that is, a user interface. Not a client library.
--
Javier

[Quincy Campbell]

i agree this is strange... hex output would be nice

Post by Frantisek Fuka
Aha! I am Redis newbie so I was under impression that redis-cli (which I
thought stands for "client") is to be used for batch jobs and scripts
(which is shown in several places on the official site), while redis
command line interface is the "redis-server" to which I give the
configuration commands.
Anyway, thanks to Marc Gravell's advice, I now have my own REDIS client
written in pure Lua which does exactly what I wanted and I can use it in my
Lua scripts.
I never wanted to imply that redis-cli is defective. I was just thinking
that maybe I missed something in the documentation, because redis-cli must
communicate with redis server, must receive raw data from it and then it
parses and displays the data. So I thought that there must be some option
to simply disable the last step in this process and let redis-cli display
the raw data it received, without doing any further processing - the same
way it can do it with input data. If this is not the case, and redis-cli
cannot do this with data it receives from redis server, thanks for that
definitive answer anyway.

Post by Javier Guerra Giraldez

Post by Frantisek Fuka
Redis-cli, on the other hand, was built specifically for communicating

with

Post by Frantisek Fuka
Redis DB and yet you cannot use it to simply communicate with the Redis

DB

Post by Frantisek Fuka
and return raw results of this communication.

my understanding is that Redis-cli is a "Redis command line
interface", that is, a user interface. Not a client library.
--
Javier

[Josiah Carlson]

Resurrecting an almost 3 year old thread? Whee!

... printing output as hex is further processing beyond the minimal
processing that the (mostly for debugging and testing) redis-cli is
intended for and already does. So you want literally the opposite of what
Frantisek was implying when he was saying he didn't want that processing
step.

Now this would have been a different discussion if redis-cli was meant to
be the equivalent of psql, pgloader, etc., but it's not.

You know, if you all want different behavior in redis-cli, you can always
provide patches, or, you know, use a client library from any one of like
30+ languages instead of relying on what is basically a debugging and
testing tool. Just a thought.

- Josiah

Post by Quincy Campbell
i agree this is strange... hex output would be nice

Post by Frantisek Fuka
Aha! I am Redis newbie so I was under impression that redis-cli (which I
thought stands for "client") is to be used for batch jobs and scripts
(which is shown in several places on the official site), while redis
command line interface is the "redis-server" to which I give the
configuration commands.
Anyway, thanks to Marc Gravell's advice, I now have my own REDIS client
written in pure Lua which does exactly what I wanted and I can use it in my
Lua scripts.
I never wanted to imply that redis-cli is defective. I was just thinking
that maybe I missed something in the documentation, because redis-cli must
communicate with redis server, must receive raw data from it and then it
parses and displays the data. So I thought that there must be some option
to simply disable the last step in this process and let redis-cli display
the raw data it received, without doing any further processing - the same
way it can do it with input data. If this is not the case, and redis-cli
cannot do this with data it receives from redis server, thanks for that
definitive answer anyway.

Post by Frantisek Fuka

Post by Frantisek Fuka
Redis-cli, on the other hand, was built specifically for communicating

with

Post by Frantisek Fuka
Redis DB and yet you cannot use it to simply communicate with the

Redis DB

Post by Frantisek Fuka
and return raw results of this communication.

my understanding is that Redis-cli is a "Redis command line
interface", that is, a user interface. Not a client library.
--
Javier

--

You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at http://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.