Change processor signature to expose buf_sock #152

thinkingfish · 2017-04-10T01:56:03Z

While working on a pubsub prototype, it was evident that the processing logic has to be aware of the identification of the channel to properly index all subscribers. This prompts me to change the signature of the worker thread to accommodate this use case.

Note buf_sock is not quite the final signature I want- I would like to pass in the stream object, which is the abstract type for which buf_sock is an implementation. But this is not currently necessary as we only support buf_sock.

I would like to refactor both channel and stream in ccommon to make it easier to call the right methods (handlers) against various channel types (TCP socket, Unix domain socket, pipe, etc). And I would like to come back and further refactor the core functionalities when that is done in ccommon.

727905b adding a new API to duration timer (twitter#152) 9406717 add missing "extern C" qualifiers to a few headers for C++ linking; (twitter#150) a0aafdf Add missing semicolon to ring array example code in documentation. (twitter#149) 9264bbb Zero byte (twitter#147) (emergency fix needed for pelikan) d4002d7 simplify cc_print_int64 (twitter#146) b164fcf Clean-up hash functions and introduce MurmurHash3 (twitter#145) b1babb2 change wheel's sleep timer to make it less flaky (twitter#143) ce0b9ea allow printing negative integers in cc_print (twitter#141) ab0edc8 add metrics to track buf_sock objects (twitter#138) ae02038 add travis ci (copied from pelikan) (twitter#139) 964645a Merge pull request twitter#135 from paegun/fix_cmake_install 70710c2 fixed and re-added cmake install instructions, w/ following notes: include directory made proper relative; opened pattern match b/c include directory should only contain files meant for inclusion. 5b095bc Merge pull request twitter#126 from kevyang/kevyang/120 426d56a return NULL when cc_alloc/cc_realloc is called with size == 0 ad271d4 Merge pull request twitter#133 from kevyang/132 47dbdba suppress unused parameter warning in debug_log_flush 648d19e Merge pull request twitter#127 from kevyang/56 780941a Merge pull request twitter#130 from kevyang/129 b8af6c0 Merge pull request twitter#131 from kevyang/128 6ecc318 fix duplicate symbols in cc_signal 080c41d cc_array - stop doing arithmetic on void * d526f7a add debug oriented memory management a4fb927 Update bool member rules in style guide 05c6e1e explicitly make ccommon a C project to avoid checking for CXX related variables git-subtree-dir: deps/ccommon git-subtree-split: 727905b

727905b adding a new API to duration timer (#152) 9406717 add missing "extern C" qualifiers to a few headers for C++ linking; (#150) a0aafdf Add missing semicolon to ring array example code in documentation. (#149) 9264bbb Zero byte (#147) (emergency fix needed for pelikan) d4002d7 simplify cc_print_int64 (#146) b164fcf Clean-up hash functions and introduce MurmurHash3 (#145) b1babb2 change wheel's sleep timer to make it less flaky (#143) ce0b9ea allow printing negative integers in cc_print (#141) ab0edc8 add metrics to track buf_sock objects (#138) ae02038 add travis ci (copied from pelikan) (#139) 964645a Merge pull request #135 from paegun/fix_cmake_install 70710c2 fixed and re-added cmake install instructions, w/ following notes: include directory made proper relative; opened pattern match b/c include directory should only contain files meant for inclusion. 5b095bc Merge pull request #126 from kevyang/kevyang/120 426d56a return NULL when cc_alloc/cc_realloc is called with size == 0 ad271d4 Merge pull request #133 from kevyang/132 47dbdba suppress unused parameter warning in debug_log_flush 648d19e Merge pull request #127 from kevyang/56 780941a Merge pull request #130 from kevyang/129 b8af6c0 Merge pull request #131 from kevyang/128 6ecc318 fix duplicate symbols in cc_signal 080c41d cc_array - stop doing arithmetic on void * d526f7a add debug oriented memory management a4fb927 Update bool member rules in style guide 05c6e1e explicitly make ccommon a C project to avoid checking for CXX related variables git-subtree-dir: deps/ccommon git-subtree-split: 727905b

dd56abd break up ASSERT in buf_sock_read (twitter#153) 727905b adding a new API to duration timer (twitter#152) 9406717 add missing "extern C" qualifiers to a few headers for C++ linking; (twitter#150) a0aafdf Add missing semicolon to ring array example code in documentation. (twitter#149) 9264bbb Zero byte (twitter#147) (emergency fix needed for pelikan) d4002d7 simplify cc_print_int64 (twitter#146) b164fcf Clean-up hash functions and introduce MurmurHash3 (twitter#145) b1babb2 change wheel's sleep timer to make it less flaky (twitter#143) ce0b9ea allow printing negative integers in cc_print (twitter#141) ab0edc8 add metrics to track buf_sock objects (twitter#138) ae02038 add travis ci (copied from pelikan) (twitter#139) 964645a Merge pull request twitter#135 from paegun/fix_cmake_install 70710c2 fixed and re-added cmake install instructions, w/ following notes: include directory made proper relative; opened pattern match b/c include directory should only contain files meant for inclusion. 5b095bc Merge pull request twitter#126 from kevyang/kevyang/120 426d56a return NULL when cc_alloc/cc_realloc is called with size == 0 ad271d4 Merge pull request twitter#133 from kevyang/132 47dbdba suppress unused parameter warning in debug_log_flush 648d19e Merge pull request twitter#127 from kevyang/56 780941a Merge pull request twitter#130 from kevyang/129 b8af6c0 Merge pull request twitter#131 from kevyang/128 6ecc318 fix duplicate symbols in cc_signal 080c41d cc_array - stop doing arithmetic on void * d526f7a add debug oriented memory management a4fb927 Update bool member rules in style guide 05c6e1e explicitly make ccommon a C project to avoid checking for CXX related variables git-subtree-dir: deps/ccommon git-subtree-split: dd56abd

* consolidate code a bit * Squashed 'deps/ccommon/' changes from bb298bc..dd56abd dd56abd break up ASSERT in buf_sock_read (#153) 727905b adding a new API to duration timer (#152) 9406717 add missing "extern C" qualifiers to a few headers for C++ linking; (#150) a0aafdf Add missing semicolon to ring array example code in documentation. (#149) 9264bbb Zero byte (#147) (emergency fix needed for pelikan) d4002d7 simplify cc_print_int64 (#146) b164fcf Clean-up hash functions and introduce MurmurHash3 (#145) b1babb2 change wheel's sleep timer to make it less flaky (#143) ce0b9ea allow printing negative integers in cc_print (#141) ab0edc8 add metrics to track buf_sock objects (#138) ae02038 add travis ci (copied from pelikan) (#139) 964645a Merge pull request #135 from paegun/fix_cmake_install 70710c2 fixed and re-added cmake install instructions, w/ following notes: include directory made proper relative; opened pattern match b/c include directory should only contain files meant for inclusion. 5b095bc Merge pull request #126 from kevyang/kevyang/120 426d56a return NULL when cc_alloc/cc_realloc is called with size == 0 ad271d4 Merge pull request #133 from kevyang/132 47dbdba suppress unused parameter warning in debug_log_flush 648d19e Merge pull request #127 from kevyang/56 780941a Merge pull request #130 from kevyang/129 b8af6c0 Merge pull request #131 from kevyang/128 6ecc318 fix duplicate symbols in cc_signal 080c41d cc_array - stop doing arithmetic on void * d526f7a add debug oriented memory management a4fb927 Update bool member rules in style guide 05c6e1e explicitly make ccommon a C project to avoid checking for CXX related variables git-subtree-dir: deps/ccommon git-subtree-split: dd56abd

a2d5c01 add a function that will reject all pending incoming connections (twitter#154) dd56abd break up ASSERT in buf_sock_read (twitter#153) 727905b adding a new API to duration timer (twitter#152) 9406717 add missing "extern C" qualifiers to a few headers for C++ linking; (twitter#150) a0aafdf Add missing semicolon to ring array example code in documentation. (twitter#149) 9264bbb Zero byte (twitter#147) (emergency fix needed for pelikan) d4002d7 simplify cc_print_int64 (twitter#146) b164fcf Clean-up hash functions and introduce MurmurHash3 (twitter#145) b1babb2 change wheel's sleep timer to make it less flaky (twitter#143) ce0b9ea allow printing negative integers in cc_print (twitter#141) ab0edc8 add metrics to track buf_sock objects (twitter#138) ae02038 add travis ci (copied from pelikan) (twitter#139) 964645a Merge pull request twitter#135 from paegun/fix_cmake_install 70710c2 fixed and re-added cmake install instructions, w/ following notes: include directory made proper relative; opened pattern match b/c include directory should only contain files meant for inclusion. 5b095bc Merge pull request twitter#126 from kevyang/kevyang/120 426d56a return NULL when cc_alloc/cc_realloc is called with size == 0 ad271d4 Merge pull request twitter#133 from kevyang/132 47dbdba suppress unused parameter warning in debug_log_flush 648d19e Merge pull request twitter#127 from kevyang/56 780941a Merge pull request twitter#130 from kevyang/129 b8af6c0 Merge pull request twitter#131 from kevyang/128 6ecc318 fix duplicate symbols in cc_signal 080c41d cc_array - stop doing arithmetic on void * d526f7a add debug oriented memory management a4fb927 Update bool member rules in style guide 05c6e1e explicitly make ccommon a C project to avoid checking for CXX related variables git-subtree-dir: deps/ccommon git-subtree-split: a2d5c01

6b59401 Rust build tweak (twitter#165) b39086d fix docs to new ring_array_destroy semantics (twitter#163) 95e04b0 Additional ring_array API (twitter#159) 3c994cc a single job in the travis-ci matrix for building with rust enabled (twitter#161) 1fe907e rust support - Native bstrings (twitter#160) b3de2ee add HAVE_RUST option defaulted to off (twitter#158) ffc70a8 Port travis ci changes - closes twitter#186 (twitter#155) a2d5c01 add a function that will reject all pending incoming connections (twitter#154) dd56abd break up ASSERT in buf_sock_read (twitter#153) 727905b adding a new API to duration timer (twitter#152) 9406717 add missing "extern C" qualifiers to a few headers for C++ linking; (twitter#150) a0aafdf Add missing semicolon to ring array example code in documentation. (twitter#149) 9264bbb Zero byte (twitter#147) (emergency fix needed for pelikan) d4002d7 simplify cc_print_int64 (twitter#146) b164fcf Clean-up hash functions and introduce MurmurHash3 (twitter#145) b1babb2 change wheel's sleep timer to make it less flaky (twitter#143) ce0b9ea allow printing negative integers in cc_print (twitter#141) ab0edc8 add metrics to track buf_sock objects (twitter#138) ae02038 add travis ci (copied from pelikan) (twitter#139) 964645a Merge pull request twitter#135 from paegun/fix_cmake_install 70710c2 fixed and re-added cmake install instructions, w/ following notes: include directory made proper relative; opened pattern match b/c include directory should only contain files meant for inclusion. 5b095bc Merge pull request twitter#126 from kevyang/kevyang/120 426d56a return NULL when cc_alloc/cc_realloc is called with size == 0 ad271d4 Merge pull request twitter#133 from kevyang/132 47dbdba suppress unused parameter warning in debug_log_flush 648d19e Merge pull request twitter#127 from kevyang/56 780941a Merge pull request twitter#130 from kevyang/129 b8af6c0 Merge pull request twitter#131 from kevyang/128 6ecc318 fix duplicate symbols in cc_signal 080c41d cc_array - stop doing arithmetic on void * d526f7a add debug oriented memory management a4fb927 Update bool member rules in style guide 05c6e1e explicitly make ccommon a C project to avoid checking for CXX related variables git-subtree-dir: deps/ccommon git-subtree-split: 6b59401

f5efe29 forgot to remove unused include (#167) 441934b update ring_array docs for new API, add multi threaded unit test for ring_array (#166) 67ce9c2 Adding issue and pull request templates to comply with twitter OSS policies (#151) 6b59401 Rust build tweak (#165) b39086d fix docs to new ring_array_destroy semantics (#163) 95e04b0 Additional ring_array API (#159) 3c994cc a single job in the travis-ci matrix for building with rust enabled (#161) 1fe907e rust support - Native bstrings (#160) b3de2ee add HAVE_RUST option defaulted to off (#158) ffc70a8 Port travis ci changes - closes #186 (#155) a2d5c01 add a function that will reject all pending incoming connections (#154) dd56abd break up ASSERT in buf_sock_read (#153) 727905b adding a new API to duration timer (#152) 9406717 add missing "extern C" qualifiers to a few headers for C++ linking; (#150) a0aafdf Add missing semicolon to ring array example code in documentation. (#149) 9264bbb Zero byte (#147) (emergency fix needed for pelikan) d4002d7 simplify cc_print_int64 (#146) b164fcf Clean-up hash functions and introduce MurmurHash3 (#145) b1babb2 change wheel's sleep timer to make it less flaky (#143) ce0b9ea allow printing negative integers in cc_print (#141) ab0edc8 add metrics to track buf_sock objects (#138) ae02038 add travis ci (copied from pelikan) (#139) 964645a Merge pull request #135 from paegun/fix_cmake_install 70710c2 fixed and re-added cmake install instructions, w/ following notes: include directory made proper relative; opened pattern match b/c include directory should only contain files meant for inclusion. 5b095bc Merge pull request #126 from kevyang/kevyang/120 426d56a return NULL when cc_alloc/cc_realloc is called with size == 0 ad271d4 Merge pull request #133 from kevyang/132 47dbdba suppress unused parameter warning in debug_log_flush 648d19e Merge pull request #127 from kevyang/56 780941a Merge pull request #130 from kevyang/129 b8af6c0 Merge pull request #131 from kevyang/128 6ecc318 fix duplicate symbols in cc_signal 080c41d cc_array - stop doing arithmetic on void * d526f7a add debug oriented memory management a4fb927 Update bool member rules in style guide 05c6e1e explicitly make ccommon a C project to avoid checking for CXX related variables git-subtree-dir: deps/ccommon git-subtree-split: f5efe29

* adding a new API to duration timer * adding const modifier to snapshot API

Yao Yue added 4 commits April 9, 2017 18:32

make signature changes in worker.[ch]

061a0a3

change signature of pingserver to match core/data/worker

8ac6849

change signature of slimcache to match core/data/worker

fa13f6f

change signature of twemcache to match core/data/worker

9feb41d

seppo0010 approved these changes Apr 10, 2017

View reviewed changes

thinkingfish merged commit 93fa86e into twitter:master Apr 11, 2017

thinkingfish deleted the change_processor_signature branch April 22, 2017 00:07

swlynch99 pushed a commit to swlynch99/pelikan-twitter that referenced this pull request Sep 30, 2019

adding a new API to duration timer (twitter#152)

727905b

* adding a new API to duration timer * adding const modifier to snapshot API

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Change processor signature to expose buf_sock #152

Change processor signature to expose buf_sock #152

thinkingfish commented Apr 10, 2017

Change processor signature to expose buf_sock #152

Change processor signature to expose buf_sock #152

Conversation

thinkingfish commented Apr 10, 2017