fabi/Snappy
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

This cl does two things

Browse Source 
1) It shaves of a few cycles from the data dependency chain. By using "shrd" instead of a load.
2) The important loop is finding small copies (4-12) which are either "copy 1", or "copy 2" depending if the offset fits <2048. It turns out that this is a branch that is mispredicted often. Due to the long dependency chain the CPU is running with IPC~1 anyway so we can freely add instructions to instead emit copies branchfree. This reduces the branch misspredicts from 15% to 11% (for BM_ZFlat/6 txt1) and from 5.6% to 4% (for BM_ZFlat/10 or pb).

PiperOrigin-RevId: 303328967
This commit is contained in:
Snappy Team
2020-03-27 14:57:37 +00:00
committed by Victor Costan
parent 0c7ed08a25
commit 0faf56378e
3 changed files with 67 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
snappy.cc
View File

@@ -385,11 +385,18 @@ static inline char* EmitCopyAtMost64(char* op, size_t offset, size_t len) {
assert(offset < 65536);
assert(len_less_than_12 == (len < 12));
if (len_less_than_12 && SNAPPY_PREDICT_TRUE(offset < 2048)) {
// offset fits in 11 bits. The 3 highest go in the top of the first byte,
// and the rest go in the second byte.
*op++ = COPY_1_BYTE_OFFSET + ((len - 4) << 2) + ((offset >> 3) & 0xe0);
*op++ = offset & 0xff;
if (len_less_than_12) {
uint32 u = (len << 2) + (offset << 8);
uint32 copy1 = COPY_1_BYTE_OFFSET - (4 << 2) + ((offset >> 3) & 0xe0);
uint32 copy2 = COPY_2_BYTE_OFFSET - (1 << 2);
// It turns out that offset < 2048 is a difficult to predict branch.
// `perf record` shows this is the highest percentage of branch misses in
// benchmarks. This code produces branch free code, the data dependency
// chain that bottlenecks the throughput is so long that a few extra
// instructions are completely free (IPC << 6 because of data deps).
u += offset < 2048 ? copy1 : copy2;
LittleEndian::Store32(op, u);
op += offset < 2048 ? 2 : 3;
} else {
// Write 4 bytes, though we only care about 3 of them. The output buffer
// is required to have some slack, so the extra byte won't overrun it.
@@ -615,7 +622,7 @@ char* CompressFragment(const char* input,
// "literal bytes" prior to ip.
const char* base = ip;
std::pair<size_t, bool> p =
FindMatchLength(candidate + 4, ip + 4, ip_end);
FindMatchLength(candidate + 4, ip + 4, ip_end, &data);
size_t matched = 4 + p.first;
ip += matched;
size_t offset = base - candidate;
@@ -629,12 +636,12 @@ char* CompressFragment(const char* input,
if (SNAPPY_PREDICT_FALSE(ip >= ip_limit)) {
goto emit_remainder;
}
assert(LittleEndian::Load64(ip) == data);
// We are now looking for a 4-byte match again. We read
// table[Hash(ip, shift)] for that. To improve compression,
// we also update table[Hash(ip - 1, shift)] and table[Hash(ip, shift)].
data = LittleEndian::Load64(ip - 1);
table[HashBytes(data, shift)] = ip - base_ip - 1;
data >>= 8;
table[HashBytes(LittleEndian::Load32(ip - 1), shift)] =
ip - base_ip - 1;
uint32 hash = HashBytes(data, shift);
candidate = base_ip + table[hash];
table[hash] = ip - base_ip;