glz-encoder: Avoid double byte swap sending image magic

Enterprise / Virtualization / SPICE - Frediano Ziglio [redhat.com] - 5 June 2018 13:13 EDT

encode_32 already deals with endian, don't swap twice. Tested with a ppc64 server machine and a x64 client.

This looks the reverse of a previous patch (59c6c82) supposed to fix big endian machine. encode_32 has been always:

static inline void encode_32(Encoder *encoder, unsigned int word) { encode(encoder, (uint8_t)(word >> 24)); encode(encoder, (uint8_t)(word >> 16) & 0x0000ff); encode(encoder, (uint8_t)(word >> 8) & 0x0000ff); encode(encoder, (uint8_t)(word & 0x0000ff)); }

while encode basically is similar to a putc on a FILE stream so is writing numbers from host endian to big endian order. The "main" endian (the one more tested since ever) is host/guest being little endian. So if you call encode_32 with a 0x01020304 you get 4 bytes in the order 1, 2, 3, 4. Before and after 59c6c82 LZ_MAGIC was defined as: #define LZ_MAGIC (*(uint32_t *)"LZ ") so on little endian this was 0x4c, 0x5a, 0x20, 0x20 that is 0x20205a4c which written through encode_32 become 0x20, 0x20, 0x5a, 0x4c so we can say that at the end on the network we must have 0x20, 0x20, 0x5a, 0x4c. On big endian however LZ_MAGIC got the value 0x4c5a2020 which written through encode_32 get 0x4c, 0x5a, 0x20, 0x20 which is the opposite expected. So patch 59c6c82 reverted the order having again 0x20, 0x20, 0x5a, 0x4c on the network. However commit 5a7e587 (spice-common), in an attempt to avoid double swapping on LZ, changed LZ_MAGIC to #define LZ_MAGIC 0x20205a4c breaking endianness again for GLZ code.

48f7188e glz-encoder: Avoid double byte swap sending image magic
server/glz-encoder.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Upstream: cgit.freedesktop.org


  • Share