fabi/Snappy
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

The result on protobuf benchmark is around 19%. Results vary by their propensity for compression. As the frequency of finding matches influences the amount of branch misspredicts and the amount of hashing.

Browse Source 
Two ideas
1) The code uses "heuristic match skipping" has a quadratic interpolation. However for the first 32 bytes it's just every byte. Special case 16 bytes. This removes a lot of code.
2) Load 64 bit integers and shift instead of reload. The hashing loop has a very long chain data = Load32(ip) -> hash = Hash(data) -> offset = table[hash] -> copy_data = Load32(base_ip + offset) followed by a compare between data and copy_data. This chain is around 20 cycles. It's unreasonable for the branch predictor to be able to predict when it's a match (that is completely driven by the content of the data). So when it's a miss this chain is on the critical path. By loading 64 bits and shifting we can effectively remove the first load.

PiperOrigin-RevId: 302893821
This commit is contained in:
Snappy Team
2020-03-25 15:24:14 +00:00
committed by Victor Costan
parent 3c77e01459
commit 0c7ed08a25
2 changed files with 61 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
snappy-stubs-internal.h
View File

@@ -330,6 +330,9 @@ class LittleEndian {
static uint32 FromHost32(uint32 x) { return bswap_32(x); }
static uint32 ToHost32(uint32 x) { return bswap_32(x); }
static uint32 FromHost64(uint64 x) { return bswap_64(x); }
static uint32 ToHost64(uint64 x) { return bswap_64(x); }
static bool IsLittleEndian() { return false; }
#else // !defined(SNAPPY_IS_BIG_ENDIAN)
@@ -340,6 +343,9 @@ class LittleEndian {
static uint32 FromHost32(uint32 x) { return x; }
static uint32 ToHost32(uint32 x) { return x; }
static uint32 FromHost64(uint64 x) { return x; }
static uint32 ToHost64(uint64 x) { return x; }
static bool IsLittleEndian() { return true; }
#endif // !defined(SNAPPY_IS_BIG_ENDIAN)
@@ -360,6 +366,14 @@ class LittleEndian {
static void Store32(void *p, uint32 v) {
UNALIGNED_STORE32(p, FromHost32(v));
}
static uint64 Load64(const void *p) {
return ToHost64(UNALIGNED_LOAD64(p));
}
static void Store64(void *p, uint64 v) {
UNALIGNED_STORE64(p, FromHost64(v));
}
};
// Some bit-manipulation functions.