[katran]: flags for reals #56

jsajadi · 2019-10-01T14:31:40Z

Add support for real's flags. Also, add XDP_PASS_ONLY flag for reals in order to prevent encapsulation (ipip) and just return XDP_PASS. This feature is useful when vip and real are the same machine.

tehnerd

aside from inline comments seems like too much things to changed just for a single XDP_PASS. imo it would be much easier to just have addLocalAddress(address) function/helper in KatranLb and in updateRealsMap just check if real is equal to localAddress, and if yes - add PASS_ONLY flag to existing __u8 field. that would make changes of all Reals and related structs unnecessary.

tehnerd · 2019-10-01T18:39:38Z

katran/lib/BalancerStructs.h

+  __u8 flags;
+  __u64 real_flags;  //real flags
+};
+


why not reuse existing flags? (__u8; only 1 is used there to distinct v4 vs v6). also it is already defined in https://github.com/facebookincubator/katran/blob/master/katran/lib/IpHelpers.h#L28 so no need to redifine real_definition

To be real's flags as same as vip's flags (__u64), and future use of current __u8 flags, I thought it is good to have this field in the definition. Anyway, I change the code to use current flags. Thanks for the comment.

Could you please also rework the code in a way that local ip (up which if real is equal to) would be configured with a function instead of new per real flag. And in updateReal would be compared against. It would simplify the code greatly.

Sure. I am currently working on it and trying to test its functionality.

tehnerd · 2019-10-01T18:40:20Z

katran/lib/CHHelpers.h

@@ -33,6 +33,7 @@ struct Endpoint {
  uint32_t num;
  uint32_t weight;
  uint64_t hash;
+  uint64_t flags;


why this is added? i do not see any usage of flags in consistent hashing

tehnerd · 2019-10-01T18:44:56Z

katran/lib/KatranLb.cpp

  auto real_addr = IpHelpers::parseAddrToBe(real);
+  real_definition real_def = {};


all of this is not neede if __u8 flags are reused

tehnerd · 2019-10-01T18:45:27Z

katran/lib/KatranLb.cpp

  auto res = bpfAdapter_.bpfUpdateMap(
-      bpfAdapter_.getMapFdByName("reals"), &num, &real_addr);
+    bpfAdapter_.getMapFdByName("reals"), &num, &real_def);


same as above

tehnerd · 2019-10-01T18:51:54Z

katran/lib/bpf/balancer_kern.c

+  if(dst->real_flags & F_XDP_PASS_ONLY) {
+    return XDP_PASS;
+  }
+


exiting here wont bump any pckt or bytes counter.

jsajadi · 2019-10-05T12:05:00Z

I am closing this merge request and create another one that supports localAddress for vips.

Summary: After diving in all the build system I found that the first error mentioned in #219 and #220 ``` 1597 | static_assert(formattable_char, "Mixing character types is disallowed."); ``` Was basically happening while compiling folly, after compiling it by itself I noticed this didn't happened, so I found that there was an issue with the fmt dependencies, removing the one that was downloaded by katran The issue mentioned in: #221 Was because katran was configured to use C++14 by default, and some of the libraries of folly require C++17, updated our requirements. Test Plan: TEST Output: ``` Test project /home/ubuntu/ivanmorett/katran/_build/build/katran/lib/tests Start 1: IpHelpersTests.testV4ParsingBe 1/56 Test #1: IpHelpersTests.testV4ParsingBe ....................... Passed 0.01 sec Start 2: IpHelpersTests.testV4ParsingInt 2/56 Test #2: IpHelpersTests.testV4ParsingInt ...................... Passed 0.01 sec Start 3: IpHelpersTests.testV6ParsingBe 3/56 Test #3: IpHelpersTests.testV6ParsingBe ....................... Passed 0.01 sec Start 4: IpHelpersTests.testV6ParsingInt 4/56 Test #4: IpHelpersTests.testV6ParsingInt ...................... Passed 0.01 sec Start 5: IpHelpersTests.testIncorrectAddr 5/56 Test #5: IpHelpersTests.testIncorrectAddr ..................... Passed 0.01 sec Start 6: CHHelpersTest.testMaglevCHSameWeight 6/56 Test #6: CHHelpersTest.testMaglevCHSameWeight ................. Passed 0.01 sec Start 7: CHHelpersTest.testMaglevV2CHSameWeight 7/56 Test #7: CHHelpersTest.testMaglevV2CHSameWeight ............... Passed 0.01 sec Start 8: CHHelpersTest.testMaglevCHDiffWeight 8/56 Test #8: CHHelpersTest.testMaglevCHDiffWeight ................. Passed 0.01 sec Start 9: CHHelpersTest.testMaglevV2CHDiffWeight 9/56 Test #9: CHHelpersTest.testMaglevV2CHDiffWeight ............... Passed 0.01 sec Start 10: CHHelpersTest.testMaglevWeightsSumLargerThanRing 10/56 Test #10: CHHelpersTest.testMaglevWeightsSumLargerThanRing ..... Passed 0.01 sec Start 11: CHHelpersTest.testMaglevWeightsSumBelowRingSize 11/56 Test #11: CHHelpersTest.testMaglevWeightsSumBelowRingSize ...... Passed 0.01 sec Start 12: KatranLbTest.testChangeMac 12/56 Test #12: KatranLbTest.testChangeMac ........................... Passed 0.01 sec Start 13: KatranLbTest.testIfIndex 13/56 Test #13: KatranLbTest.testIfIndex ............................. Passed 0.01 sec Start 14: KatranLbTest.testVipHelpers 14/56 Test #14: KatranLbTest.testVipHelpers .......................... Passed 0.14 sec Start 15: KatranLbTest.testAddingInvalidVip 15/56 Test #15: KatranLbTest.testAddingInvalidVip .................... Passed 0.01 sec Start 16: KatranLbTest.testRealHelpers 16/56 Test #16: KatranLbTest.testRealHelpers ......................... Passed 0.01 sec Start 17: KatranLbTest.testRealFlags 17/56 Test #17: KatranLbTest.testRealFlags ........................... Passed 0.01 sec Start 18: KatranLbTest.testVipStatsHelper 18/56 Test #18: KatranLbTest.testVipStatsHelper ...................... Passed 0.01 sec Start 19: KatranLbTest.testLruStatsHelper 19/56 Test #19: KatranLbTest.testLruStatsHelper ...................... Passed 0.01 sec Start 20: KatranLbTest.testLruMissStatsHelper 20/56 Test #20: KatranLbTest.testLruMissStatsHelper .................. Passed 0.01 sec Start 21: KatranLbTest.testHcHelpers 21/56 Test #21: KatranLbTest.testHcHelpers ........................... Passed 0.01 sec Start 22: KatranLbTest.getVipFlags 22/56 Test #22: KatranLbTest.getVipFlags ............................. Passed 0.01 sec Start 23: KatranLbTest.getAllVips 23/56 Test #23: KatranLbTest.getAllVips .............................. Passed 0.01 sec Start 24: KatranLbTest.testUpdateRealsHelper 24/56 Test #24: KatranLbTest.testUpdateRealsHelper ................... Passed 0.07 sec Start 25: KatranLbTest.testUpdateQuicRealsHelper 25/56 Test #25: KatranLbTest.testUpdateQuicRealsHelper ............... Passed 0.06 sec Start 26: KatranLbTest.testUpdateQuicReal 26/56 Test #26: KatranLbTest.testUpdateQuicReal ...................... Passed 0.01 sec Start 27: KatranLbTest.getRealsForVip 27/56 Test #27: KatranLbTest.getRealsForVip .......................... Passed 0.01 sec Start 28: KatranLbTest.getHealthcheckersDst 28/56 Test #28: KatranLbTest.getHealthcheckersDst .................... Passed 0.01 sec Start 29: KatranLbTest.invalidAddressHandling 29/56 Test #29: KatranLbTest.invalidAddressHandling .................. Passed 0.01 sec Start 30: KatranLbTest.addInvalidSrcRoutingRule 30/56 Test #30: KatranLbTest.addInvalidSrcRoutingRule ................ Passed 0.01 sec Start 31: KatranLbTest.addValidSrcRoutingRuleV4 31/56 Test #31: KatranLbTest.addValidSrcRoutingRuleV4 ................ Passed 0.01 sec Start 32: KatranLbTest.addValidSrcRoutingRuleV6 32/56 Test #32: KatranLbTest.addValidSrcRoutingRuleV6 ................ Passed 0.01 sec Start 33: KatranLbTest.addMaxSrcRules 33/56 Test #33: KatranLbTest.addMaxSrcRules .......................... Passed 0.01 sec Start 34: KatranLbTest.delSrcRules 34/56 Test #34: KatranLbTest.delSrcRules ............................. Passed 0.01 sec Start 35: KatranLbTest.clearSrcRules 35/56 Test #35: KatranLbTest.clearSrcRules ........................... Passed 0.01 sec Start 36: KatranLbTest.addFewInvalidNets 36/56 Test #36: KatranLbTest.addFewInvalidNets ....................... Passed 0.01 sec Start 37: KatranLbTest.addInvalidDecapDst 37/56 Test #37: KatranLbTest.addInvalidDecapDst ...................... Passed 0.01 sec Start 38: KatranLbTest.addInvalidDecapDstNet 38/56 Test #38: KatranLbTest.addInvalidDecapDstNet ................... Passed 0.01 sec Start 39: KatranLbTest.addValidDecapDst 39/56 Test #39: KatranLbTest.addValidDecapDst ........................ Passed 0.01 sec Start 40: KatranLbTest.delValidDecapDst 40/56 Test #40: KatranLbTest.delValidDecapDst ........................ Passed 0.01 sec Start 41: KatranLbTest.delInvalidDecapDst 41/56 Test #41: KatranLbTest.delInvalidDecapDst ...................... Passed 0.01 sec Start 42: KatranLbTest.addMaxDecapDst 42/56 Test #42: KatranLbTest.addMaxDecapDst .......................... Passed 0.01 sec Start 43: VipTestF.testBatchUpdateReals 43/56 Test #43: VipTestF.testBatchUpdateReals ........................ Passed 0.04 sec Start 44: VipTestF.testBatchUpdateRealsWeight 44/56 Test #44: VipTestF.testBatchUpdateRealsWeight .................. Passed 0.05 sec Start 45: VipTestF.testGetRealsAndWeight 45/56 Test #45: VipTestF.testGetRealsAndWeight ....................... Passed 0.01 sec Start 46: VipTestF.testGetReals 46/56 Test #46: VipTestF.testGetReals ................................ Passed 0.02 sec Start 47: VipTest.testAddRemoveReal 47/56 Test #47: VipTest.testAddRemoveReal ............................ Passed 0.01 sec Start 48: EventPipeCallbackTest.SimpleCallbackTest 48/56 Test #48: EventPipeCallbackTest.SimpleCallbackTest ............. Passed 0.01 sec Start 49: EventPipeCallbackTest.LargeWriteTest 49/56 Test #49: EventPipeCallbackTest.LargeWriteTest ................. Passed 0.15 sec Start 50: TestMonitoringServiceCore.SimpleAcceptSubscription 50/56 Test #50: TestMonitoringServiceCore.SimpleAcceptSubscription ... Passed 0.01 sec Start 51: TestMonitoringServiceCore.SimpleErrors 51/56 Test #51: TestMonitoringServiceCore.SimpleErrors ............... Passed 0.01 sec Start 52: TestMonitoringServiceCore.EventIntersection 52/56 Test #52: TestMonitoringServiceCore.EventIntersection .......... Passed 0.01 sec Start 53: TestMonitoringServiceCore.RacingClients 53/56 Test #53: TestMonitoringServiceCore.RacingClients .............. Passed 0.01 sec Start 54: TestMonitoringServiceCore.SubscribeAndCancel 54/56 Test #54: TestMonitoringServiceCore.SubscribeAndCancel ......... Passed 0.01 sec Start 55: PcapWriterTest.SingleWriter 55/56 Test #55: PcapWriterTest.SingleWriter .......................... Passed 0.02 sec Start 56: PcapWriterTest.MultiWriter 56/56 Test #56: PcapWriterTest.MultiWriter ........................... Passed 0.01 sec 100% tests passed, 0 tests failed out of 56 Total Test time (real) = 1.04 sec + cd ../testing/ + ctest -v ./CMakeFiles ./CTestTestfile.cmake ./Makefile ./base64helpers-tests './base64helpers-tests[1]_include.cmake' './base64helpers-tests[1]_tests.cmake' ./cmake_install.cmake ./katran_tester ./libbase64_helpers.a ./libbpftester.a ./libkatran_test_provision.a ./libkatran_test_util.a ./libpcap_parser.a ctest: /usr/local/lib/libcurl.so.4: no version information available (required by ctest) Test project /home/ubuntu/ivanmorett/katran/_build/build/katran/lib/testing Start 1: Base64Tests.testEncode 1/2 Test #1: Base64Tests.testEncode ........... Passed 0.01 sec Start 2: Base64Tests.testDecode 2/2 Test #2: Base64Tests.testDecode ........... Passed 0.01 sec 100% tests passed, 0 tests failed out of 2 Total Test time (real) = 0.01 sec + popd ~/ivanmorett/katran/_build ``` Differential Revision: D55108012 Pulled By: lima1756

jsajadi added 4 commits October 1, 2019 17:09

add flags for reals

667a701

add support for XDP_PASS_ONLY flag for reals

4227225

add tests for real's flags

f09e434

rename field (bug fix)

c5d452f

facebook-github-bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Oct 1, 2019

bug fix

76642ff

tehnerd reviewed Oct 1, 2019

View reviewed changes

jsajadi changed the title ~~Flags for reals~~ [katran]: flags for reals Oct 2, 2019

jsajadi closed this Oct 5, 2019

jsajadi mentioned this pull request Dec 2, 2020

return xdp_pass for local reals #107

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[katran]: flags for reals #56

[katran]: flags for reals #56

jsajadi commented Oct 1, 2019

tehnerd left a comment

tehnerd Oct 1, 2019

jsajadi Oct 2, 2019

tehnerd Oct 2, 2019 •

edited

Loading

jsajadi Oct 2, 2019

tehnerd Oct 1, 2019

tehnerd Oct 1, 2019

tehnerd Oct 1, 2019

tehnerd Oct 1, 2019

jsajadi commented Oct 5, 2019

		auto real_addr = IpHelpers::parseAddrToBe(real);
		real_definition real_def = {};

[katran]: flags for reals #56

[katran]: flags for reals #56

Conversation

jsajadi commented Oct 1, 2019

tehnerd left a comment

Choose a reason for hiding this comment

tehnerd Oct 1, 2019

Choose a reason for hiding this comment

jsajadi Oct 2, 2019

Choose a reason for hiding this comment

tehnerd Oct 2, 2019 • edited Loading

Choose a reason for hiding this comment

jsajadi Oct 2, 2019

Choose a reason for hiding this comment

tehnerd Oct 1, 2019

Choose a reason for hiding this comment

tehnerd Oct 1, 2019

Choose a reason for hiding this comment

tehnerd Oct 1, 2019

Choose a reason for hiding this comment

tehnerd Oct 1, 2019

Choose a reason for hiding this comment

jsajadi commented Oct 5, 2019

tehnerd Oct 2, 2019 •

edited

Loading