Cache tool #5

persiaAziz-zz · 2017-08-02T18:35:43Z

Don't merge this please. I have a lot of debug prints here

SolidWallOfCode · 2017-08-03T14:49:41Z

cmd/traffic_cache_tool/CacheTool.cc

 using ts::MemView;
 using ts::CacheDirEntry;
-
+#define VOL_HASH_TABLE_SIZE 32707


No #define unless absolutely necessary. constexpr.

SolidWallOfCode · 2017-08-03T14:50:02Z

cmd/traffic_cache_tool/CacheTool.cc

 using ts::CacheDirEntry;
-
+#define VOL_HASH_TABLE_SIZE 32707
+#define VOL_HASH_ALLOC_SIZE (8 * 1024 * 1024) // one chance per this unit


This should be a Scalar instance.

SolidWallOfCode · 2017-08-03T14:50:19Z

cmd/traffic_cache_tool/CacheTool.cc

+#define VOL_HASH_TABLE_SIZE 32707
+#define VOL_HASH_ALLOC_SIZE (8 * 1024 * 1024) // one chance per this unit
+#define VOL_HASH_EMPTY 0xFFFF
+#define STORE_BLOCK_SHIFT 13


Not needed, handled by Scalar.

SolidWallOfCode · 2017-08-03T14:50:33Z

cmd/traffic_cache_tool/CacheTool.cc

+#define VOL_HASH_ALLOC_SIZE (8 * 1024 * 1024) // one chance per this unit
+#define VOL_HASH_EMPTY 0xFFFF
+#define STORE_BLOCK_SHIFT 13
+#define STORE_BLOCK_SIZE 8192


Not needed, handled by Scalar.

SolidWallOfCode · 2017-08-03T14:51:13Z

cmd/traffic_cache_tool/CacheTool.cc


 struct Stripe;
-
+std::vector<Stripe *> globalVec_stripe;


This should be a member of Cache,not a global.

SolidWallOfCode · 2017-08-03T14:51:31Z

cmd/traffic_cache_tool/CacheTool.cc

 struct Stripe;
-
+std::vector<Stripe *> globalVec_stripe;
+std::unordered_set<std::string> URLset;


This shouldn't be a global, it should be embedded in the command function.

SolidWallOfCode · 2017-08-03T14:51:44Z

cmd/traffic_cache_tool/CacheTool.cc

-
+std::vector<Stripe *> globalVec_stripe;
+std::unordered_set<std::string> URLset;
+unsigned short *stripes_hash_table;


This should be a member, not a global.

SolidWallOfCode · 2017-08-03T14:52:04Z

cmd/traffic_cache_tool/CacheTool.cc


  int64_t _buckets;  ///< Number of buckets per segment.
  int64_t _segments; ///< Number of segments.
+  const char *hashText = nullptr;


This should be a std::string.

SolidWallOfCode · 2017-08-03T14:52:42Z

cmd/traffic_cache_tool/CacheTool.cc


 Stripe::Stripe(Span *span, Bytes start, CacheStoreBlocks len) : _span(span), _start(start), _len(len)
 {
+  const char *diskPath        = span->_path.path();


What is this used for, that _span->_path.path() can't be?

No use of this. I think I was looking for a shorter variable name

SolidWallOfCode · 2017-08-03T14:54:10Z

cmd/traffic_cache_tool/CacheTool.cc

+  char *hash_text             = static_cast<char *>(ats_malloc(hash_text_size));
+  strncpy(hash_text, diskPath, hash_text_size);
+  snprintf(hash_text + hash_seed_size, (hash_text_size - hash_seed_size), " %" PRIu64 ":%" PRIu64 "", (uint64_t)_start,
+           (uint64_t)_len.count() * STORE_BLOCK_SIZE);


This should just be _len. If not, we'll need to update the version of Scalar here.

SolidWallOfCode · 2017-08-03T14:54:39Z

cmd/traffic_cache_tool/CacheTool.cc

+  const char *diskPath        = span->_path.path();
+  const size_t hash_seed_size = strlen(diskPath);
+  const size_t hash_text_size = hash_seed_size + 32;
+  char *hash_text             = static_cast<char *>(ats_malloc(hash_text_size));


Should be std::string.

SolidWallOfCode · 2017-08-03T14:56:16Z

cmd/traffic_cache_tool/CacheTool.cc

 Cache::allocStripe(Span *span, int vol_idx, CacheStripeBlocks len)
 {
  auto rv = span->allocStripe(vol_idx, len);
+  std::cout << span->_path << ":" << vol_idx << std::endl;


Is this just temporary debugging?

SolidWallOfCode · 2017-08-03T14:57:40Z

cmd/traffic_cache_tool/CacheTool.cc

  return zret;
 }

+Errata


Why load them in a table?

SolidWallOfCode · 2017-08-03T15:11:41Z

cmd/traffic_cache_tool/CacheTool.cc

+    uint64_t x = elt->hash_id.fold();
+    // seed random number generator
+    rnd[i] = (unsigned int)x;
+    total += ((elt->_len.count() * STORE_BLOCK_SIZE) >> STORE_BLOCK_SHIFT);


Isn't this a no-op?

yah.. I calculated this to verify with the ATS implementation

Add URL stripe assignment simulation.

YTSATS-2687: Align enums for hooks

Fix the `make check` failure with the following output by changing to .gold which meets our license exclusion regex rules: Printing headers for text files without a valid license header... ===================================================== == File: ./tests/gold_tests/records/gold/renamed_records.out ===================================================== ``` ┌■ 8 Renamed records: └┬──» #1 : proxy.config.output.logfile -> proxy.config.output.logfile.name ├──» #2 : proxy.config.exec_thread.autoconfig -> proxy.config.exec_thread.autoconfig.enabled ├──» #3 : proxy.config.hostdb -> proxy.config.hostdb.enabled ├──» #4 : proxy.config.tunnel.prewarm -> proxy.config.tunnel.prewarm.enabled ├──» #5 : proxy.config.ssl.origin_session_cache -> proxy.config.ssl.origin_session_cache.enabled ├──» #6 : proxy.config.ssl.session_cache -> proxy.config.ssl.session_cache.value ├──» #7 : proxy.config.ssl.TLSv1_3 -> proxy.config.ssl.TLSv1_3.enabled └──» #8 : proxy.config.ssl.client.TLSv1_3 -> proxy.config.ssl.client.TLSv1_3.enabled ```

persiaAziz-zz force-pushed the cache-tool branch 2 times, most recently from 25a3572 to 570bbdf Compare August 2, 2017 19:28

SolidWallOfCode reviewed Aug 3, 2017

View reviewed changes

cmd/traffic_cache_tool/CacheTool.cc

return zret;

}

Errata

Copy link

Owner

SolidWallOfCode Aug 3, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why load them in a table?

SolidWallOfCode reviewed Aug 3, 2017

View reviewed changes

persiaAziz-zz force-pushed the cache-tool branch from 3106d53 to e8846f8 Compare August 3, 2017 21:52

persiaAziz-zz added 13 commits August 4, 2017 10:41

debug

666d253

print

c9d8e07

hqsh

46e8931

half done build_hash_table

2c31795

build_stripes_hashtable

d5d928a

call build_stripe_hash_table

4a8d68d

fix

4239119

global stripe table

59fe9e4

debug

4fc2d65

fix

40f9f0c

p

097e932

meh

6047084

fix

209def2

persiaAziz-zz added 6 commits August 4, 2017 10:43

total to scalar

efb3067

rtable_entries to scalar

713c724

rtable to scalar

197eab6

rtable to scalar

5bb5153

replace define with constexpr

1b29564

move methods under struct cache

962ee45

persiaAziz-zz force-pushed the cache-tool branch from a29c7dd to 962ee45 Compare August 4, 2017 15:43

SolidWallOfCode merged commit 12fbedd into SolidWallOfCode:cache-tool Aug 4, 2017

SolidWallOfCode pushed a commit that referenced this pull request Nov 11, 2017

Cache tool (#5)

9f56dc7

Add URL stripe assignment simulation.

SolidWallOfCode pushed a commit that referenced this pull request Sep 7, 2019

Merge pull request #5 from feid/ytsats-2687

ac7f5b6

YTSATS-2687: Align enums for hooks

Cache tool #5

Cache tool #5

Uh oh!

Conversation

persiaAziz-zz commented Aug 2, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SolidWallOfCode Aug 3, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SolidWallOfCode Aug 3, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

SolidWallOfCode Aug 3, 2017 •

edited

Loading

SolidWallOfCode Aug 3, 2017 •

edited

Loading