Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updating mongoose_cets_discovery_rdbms #4165

Merged
merged 2 commits into from
Nov 14, 2023

Conversation

DenysGonchar
Copy link
Collaborator

this PR introduces the following changes:

  • fix for node num selection algorithm
  • avoiding of self-removal from the DB cluster, because removal is based on row selection before TS updating.
  • extension of the cets disco test suite

@mongoose-im
Copy link
Collaborator

mongoose-im commented Nov 10, 2023

elasticsearch_and_cassandra_26 / elasticsearch_and_cassandra_mnesia / 355c70f
Reports root/ big
OK: 369 / Failed: 0 / User-skipped: 38 / Auto-skipped: 0


small_tests_25 / small_tests / 355c70f
Reports root / small


small_tests_26 / small_tests / 355c70f
Reports root / small


small_tests_26_arm64 / small_tests / 355c70f
Reports root / small


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 355c70f
Reports root/ big
OK: 4277 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


ldap_mnesia_25 / ldap_mnesia / 355c70f
Reports root/ big
OK: 2310 / Failed: 0 / User-skipped: 867 / Auto-skipped: 0


dynamic_domains_mysql_redis_26 / mysql_redis / 355c70f
Reports root/ big
OK: 4245 / Failed: 0 / User-skipped: 144 / Auto-skipped: 0


ldap_mnesia_26 / ldap_mnesia / 355c70f
Reports root/ big
OK: 2310 / Failed: 0 / User-skipped: 867 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / 355c70f
Reports root/ big
OK: 4666 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


pgsql_cets_26 / pgsql_cets / 355c70f
Reports root/ big
OK: 4663 / Failed: 1 / User-skipped: 121 / Auto-skipped: 0

cets_disco_SUITE:rdbms:rdbms_backend_db_queries
{error,
  {{assertMatch,
     [{module,cets_disco_SUITE},
      {line,156},
      {expression,"select ( CN )"},
      {pattern,
        "{ selected , [ { << \"test1\" >> , 1 , TS2 } , { << \"test2\" >> , 2 , TS } ] }"},
      {value,
        {selected,
          [{<<"test2">>,2,1699625607},{<<"test1">>,1,1699625707}]}}]},
   [{cets_disco_SUITE,rdbms_backend_db_queries,1,
      [{file,
         "/home/circleci/project/big_tests/tests/cets_disco_SUITE.erl"},
       {line,156}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1302}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1234}]}]}}

Report log


internal_mnesia_26 / internal_mnesia / 355c70f
Reports root/ big
OK: 2460 / Failed: 0 / User-skipped: 717 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_26 / odbc_mssql_mnesia / 355c70f
Reports root/ big
OK: 4273 / Failed: 1 / User-skipped: 115 / Auto-skipped: 0

cets_disco_SUITE:rdbms:rdbms_backend_db_queries
{error,{{assertEqual,[{module,cets_disco_SUITE},
            {line,148},
            {expression,"insert_new ( CN , << \"test1\" >> , TS , 1 )"},
            {expected,{error,duplicate_key}},
            {value,{error,#{reason =>
                      "[FreeTDS][SQL Server]Violation of PRIMARY KEY constraint 'PK__discover__59D9EC255EDC8271'. Cannot insert duplicate key in object 'dbo.discovery_nodes'. The duplicate key value is (big_test_rdbms_backend_db_queries_e834c50ecbe41bf5, test1).[FreeTDS][SQL Server]The statement has been terminated. SQLSTATE IS: 01000",
                    odbc_query =>
                      <<"INSERT INTO discovery_nodes (cluster_name, node_name, node_num, updated_timestamp) VALUES (?, ?, ?, ?)">>,
                    odbc_params =>
                      [{{sql_wlongvarchar,50},
                      [<<98,0,105,0,103,0,95,0,116,0,
                         101,0,115,0,116,0,95,0,114,0,
                         100,0,98,0,109,0,115,0,95,0,
                         98,0,97,0,99,0,107,0,101,0,
                         110,0,100,0,95,0,100,0,98,0,
                         95,0,113,0,117,0,101,0,114,0,
                         105,0,101,0,115,0,95,0,101,0,
                         56,0,51,0,52,0,99,0,53,0,48,0,
                         101,0,99,0,98,0,101,0,52,0,49,
                         0,98,0,102,0,53,0>>]},
                       {{sql_wlongvarchar,5},
                      [<<116,0,101,0,115,0,116,0,49,0>>]},
                       {sql_integer,[1...

Report log


dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / 355c70f
Reports root/ big
OK: 4277 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


mysql_redis_26 / mysql_redis / 355c70f
Reports root/ big
OK: 4646 / Failed: 0 / User-skipped: 139 / Auto-skipped: 0


pgsql_mnesia_26 / pgsql_mnesia / 355c70f
Reports root/ big
OK: 4666 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


mssql_mnesia_26 / odbc_mssql_mnesia / 355c70f
Reports root/ big
OK: 4662 / Failed: 1 / User-skipped: 122 / Auto-skipped: 0

cets_disco_SUITE:rdbms:rdbms_backend_db_queries
{error,{{assertEqual,[{module,cets_disco_SUITE},
            {line,148},
            {expression,"insert_new ( CN , << \"test1\" >> , TS , 1 )"},
            {expected,{error,duplicate_key}},
            {value,{error,#{reason =>
                      "[FreeTDS][SQL Server]Violation of PRIMARY KEY constraint 'PK__discover__59D9EC25E6636497'. Cannot insert duplicate key in object 'dbo.discovery_nodes'. The duplicate key value is (big_test_rdbms_backend_db_queries_ffa4f1a668706488, test1).[FreeTDS][SQL Server]The statement has been terminated. SQLSTATE IS: 01000",
                    odbc_query =>
                      <<"INSERT INTO discovery_nodes (cluster_name, node_name, node_num, updated_timestamp) VALUES (?, ?, ?, ?)">>,
                    odbc_params =>
                      [{{sql_wlongvarchar,50},
                      [<<98,0,105,0,103,0,95,0,116,0,
                         101,0,115,0,116,0,95,0,114,0,
                         100,0,98,0,109,0,115,0,95,0,
                         98,0,97,0,99,0,107,0,101,0,
                         110,0,100,0,95,0,100,0,98,0,
                         95,0,113,0,117,0,101,0,114,0,
                         105,0,101,0,115,0,95,0,102,0,
                         102,0,97,0,52,0,102,0,49,0,97,
                         0,54,0,54,0,56,0,55,0,48,0,54,
                         0,52,0,56,0,56,0>>]},
                       {{sql_wlongvarchar,5},
                      [<<116,0,101,0,115,0,116,0,49,0>>]},
                       {sql_integer,[1]...

Report log

Copy link

codecov bot commented Nov 10, 2023

Codecov Report

Attention: 9 lines in your changes are missing coverage. Please review.

Comparison is base (f9ba894) 84.04% compared to head (42a2d30) 84.03%.
Report is 4 commits behind head on master.

Files Patch % Lines
src/mongoose_cets_discovery_rdbms.erl 74.28% 9 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4165      +/-   ##
==========================================
- Coverage   84.04%   84.03%   -0.01%     
==========================================
  Files         564      564              
  Lines       34067    34064       -3     
==========================================
- Hits        28630    28627       -3     
  Misses       5437     5437              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@DenysGonchar DenysGonchar force-pushed the mongoose_cets_discovery_rdbms-improvements branch from 355c70f to 35336c9 Compare November 10, 2023 15:35
@mongoose-im
Copy link
Collaborator

mongoose-im commented Nov 10, 2023

elasticsearch_and_cassandra_26 / elasticsearch_and_cassandra_mnesia / 35336c9
Reports root/ big
OK: 369 / Failed: 0 / User-skipped: 38 / Auto-skipped: 0


small_tests_25 / small_tests / 35336c9
Reports root / small


small_tests_26 / small_tests / 35336c9
Reports root / small


small_tests_26_arm64 / small_tests / 35336c9
Reports root / small


ldap_mnesia_25 / ldap_mnesia / 35336c9
Reports root/ big
OK: 2310 / Failed: 0 / User-skipped: 867 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 35336c9
Reports root/ big
OK: 4271 / Failed: 1 / User-skipped: 112 / Auto-skipped: 5

bosh_SUITE:essential_https:accept_higher_hold_value
{error,
  {{assertEqual,
     [{module,bosh_SUITE},
      {line,260},
      {expression,"get_bosh_sessions ( )"},
      {expected,[]},
      {value,
        [{bosh_session,<<"0f075a504601b3c5dc6692173d120deca51318ff">>,
           <9327.10247.0>}]}]},
   [{bosh_SUITE,accept_higher_hold_value,1,
      [{file,"/home/circleci/project/big_tests/tests/bosh_SUITE.erl"},
       {line,260}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1291}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1223}]}]}}

Report log


ldap_mnesia_26 / ldap_mnesia / 35336c9
Reports root/ big
OK: 2310 / Failed: 0 / User-skipped: 867 / Auto-skipped: 0


dynamic_domains_mysql_redis_26 / mysql_redis / 35336c9
Reports root/ big
OK: 4245 / Failed: 0 / User-skipped: 144 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_26 / odbc_mssql_mnesia / 35336c9
Reports root/ big
OK: 4273 / Failed: 1 / User-skipped: 115 / Auto-skipped: 0

cets_disco_SUITE:rdbms:rdbms_backend_db_queries
{error,{{assertEqual,[{module,cets_disco_SUITE},
            {line,148},
            {expression,"insert_new ( CN , << \"test1\" >> , TS , 1 )"},
            {expected,{error,duplicate_key}},
            {value,{error,#{reason =>
                      "[FreeTDS][SQL Server]Violation of PRIMARY KEY constraint 'PK__discover__59D9EC250FC5DB03'. Cannot insert duplicate key in object 'dbo.discovery_nodes'. The duplicate key value is (big_test_rdbms_backend_db_queries_e53beb0badead419, test1).[FreeTDS][SQL Server]The statement has been terminated. SQLSTATE IS: 01000",
                    odbc_query =>
                      <<"INSERT INTO discovery_nodes (cluster_name, node_name, node_num, updated_timestamp) VALUES (?, ?, ?, ?)">>,
                    odbc_params =>
                      [{{sql_wlongvarchar,50},
                      [<<98,0,105,0,103,0,95,0,116,0,
                         101,0,115,0,116,0,95,0,114,0,
                         100,0,98,0,109,0,115,0,95,0,
                         98,0,97,0,99,0,107,0,101,0,
                         110,0,100,0,95,0,100,0,98,0,
                         95,0,113,0,117,0,101,0,114,0,
                         105,0,101,0,115,0,95,0,101,0,
                         53,0,51,0,98,0,101,0,98,0,48,
                         0,98,0,97,0,100,0,101,0,97,0,
                         100,0,52,0,49,0,57,0>>]},
                       {{sql_wlongvarchar,5},
                      [<<116,0,101,0,115,0,116,0,49,0>>]},
                       {sql_integer,[...

Report log


dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / 35336c9
Reports root/ big
OK: 4277 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


internal_mnesia_26 / internal_mnesia / 35336c9
Reports root/ big
OK: 2460 / Failed: 0 / User-skipped: 717 / Auto-skipped: 0


pgsql_cets_26 / pgsql_cets / 35336c9
Reports root/ big
OK: 4664 / Failed: 0 / User-skipped: 121 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / 35336c9
Reports root/ big
OK: 4666 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


mysql_redis_26 / mysql_redis / 35336c9
Reports root/ big
OK: 4646 / Failed: 0 / User-skipped: 139 / Auto-skipped: 0


pgsql_mnesia_26 / pgsql_mnesia / 35336c9
Reports root/ big
OK: 4666 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


mssql_mnesia_26 / odbc_mssql_mnesia / 35336c9
Reports root/ big
OK: 4662 / Failed: 1 / User-skipped: 122 / Auto-skipped: 0

cets_disco_SUITE:rdbms:rdbms_backend_db_queries
{error,{{assertEqual,[{module,cets_disco_SUITE},
            {line,148},
            {expression,"insert_new ( CN , << \"test1\" >> , TS , 1 )"},
            {expected,{error,duplicate_key}},
            {value,{error,#{reason =>
                      "[FreeTDS][SQL Server]Violation of PRIMARY KEY constraint 'PK__discover__59D9EC254E125712'. Cannot insert duplicate key in object 'dbo.discovery_nodes'. The duplicate key value is (big_test_rdbms_backend_db_queries_38b202b6442ce891, test1).[FreeTDS][SQL Server]The statement has been terminated. SQLSTATE IS: 01000",
                    odbc_query =>
                      <<"INSERT INTO discovery_nodes (cluster_name, node_name, node_num, updated_timestamp) VALUES (?, ?, ?, ?)">>,
                    odbc_params =>
                      [{{sql_wlongvarchar,50},
                      [<<98,0,105,0,103,0,95,0,116,0,
                         101,0,115,0,116,0,95,0,114,0,
                         100,0,98,0,109,0,115,0,95,0,
                         98,0,97,0,99,0,107,0,101,0,
                         110,0,100,0,95,0,100,0,98,0,
                         95,0,113,0,117,0,101,0,114,0,
                         105,0,101,0,115,0,95,0,51,0,
                         56,0,98,0,50,0,48,0,50,0,98,0,
                         54,0,52,0,52,0,50,0,99,0,101,
                         0,56,0,57,0,49,0>>]},
                       {{sql_wlongvarchar,5},
                      [<<116,0,101,0,115,0,116,0,49,0>>]},
                       {sql_integer,[1]},...

Report log

@DenysGonchar DenysGonchar force-pushed the mongoose_cets_discovery_rdbms-improvements branch from 35336c9 to c0a840f Compare November 10, 2023 16:16
@mongoose-im
Copy link
Collaborator

mongoose-im commented Nov 10, 2023

elasticsearch_and_cassandra_26 / elasticsearch_and_cassandra_mnesia / c0a840f
Reports root/ big
OK: 369 / Failed: 0 / User-skipped: 38 / Auto-skipped: 0


small_tests_25 / small_tests / c0a840f
Reports root / small


small_tests_26 / small_tests / c0a840f
Reports root / small


small_tests_26_arm64 / small_tests / c0a840f
Reports root / small


ldap_mnesia_25 / ldap_mnesia / c0a840f
Reports root/ big
OK: 2310 / Failed: 0 / User-skipped: 867 / Auto-skipped: 0


dynamic_domains_mysql_redis_26 / mysql_redis / c0a840f
Reports root/ big
OK: 4245 / Failed: 0 / User-skipped: 144 / Auto-skipped: 0


ldap_mnesia_26 / ldap_mnesia / c0a840f
Reports root/ big
OK: 2310 / Failed: 0 / User-skipped: 867 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / c0a840f
Reports root/ big
OK: 4277 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / c0a840f
Reports root/ big
OK: 4277 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_26 / odbc_mssql_mnesia / c0a840f
Reports root/ big
OK: 4274 / Failed: 0 / User-skipped: 115 / Auto-skipped: 0


internal_mnesia_26 / internal_mnesia / c0a840f
Reports root/ big
OK: 2460 / Failed: 0 / User-skipped: 717 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / c0a840f
Reports root/ big
OK: 4666 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


pgsql_cets_26 / pgsql_cets / c0a840f
Reports root/ big
OK: 4664 / Failed: 0 / User-skipped: 121 / Auto-skipped: 0


mysql_redis_26 / mysql_redis / c0a840f
Reports root/ big
OK: 4645 / Failed: 1 / User-skipped: 139 / Auto-skipped: 0

carboncopy_SUITE:one2one:dropped_client_doesnt_create_duplicate_carbons
{error,
  {{badmatch,
     [{xmlel,<<"message">>,
        [{<<"from">>,
        <<"alice_dropped_client_doesnt_create_duplicate_carbons_591@localhost">>},
         {<<"to">>,
        <<"alice_dropped_client_doesnt_create_duplicate_carbons_591@localhost/res2">>},
         {<<"xmlns">>,<<"jabber:client">>},
         {<<"type">>,<<"chat">>}],
        [{xmlel,<<"sent">>,
           [{<<"xmlns">>,<<"urn:xmpp:carbons:2">>}],
           [{xmlel,<<"forwarded">>,
            [{<<"xmlns">>,<<"urn:xmpp:forward:0">>}],
            [{xmlel,<<"message">>,
               [{<<"from">>,
                 <<"alice_dropped_client_doesnt_create_duplicate_carbons_591@localhost/res1">>},
                {<<"type">>,<<"chat">>},
                {<<"to">>,
                 <<"bob_dropped_client_doesnt_create_duplicate_carbons_591@localhost/res1">>},
                {<<"xmlns">>,<<"jabber:client">>}],
               [{xmlel,<<"body">>,[],
                  [{xmlcdata,
                     <<"And pious action">>}]}]}]}]}]}]},
   [{carboncopy_SUITE,
      '-dropped_client_doesnt_create_duplicate_carbons/1-fun-0-',4,
      [{file,
         "/home/circleci/project/big_tests/tests/carboncopy_SUITE.erl"},
       {line,189}]},
    {escalus_story,story,4,
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
       {line,72}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_serv...

Report log


pgsql_mnesia_26 / pgsql_mnesia / c0a840f
Reports root/ big
OK: 4666 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


mssql_mnesia_26 / odbc_mssql_mnesia / c0a840f
Reports root/ big
OK: 4663 / Failed: 0 / User-skipped: 122 / Auto-skipped: 0


mysql_redis_26 / mysql_redis / c0a840f
Reports root/ big
OK: 4646 / Failed: 0 / User-skipped: 139 / Auto-skipped: 0

@DenysGonchar DenysGonchar marked this pull request as ready for review November 10, 2023 17:31
Copy link
Member

@chrzaszcz chrzaszcz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good in general, nice to see this fixed. I added minor comments.

big_tests/tests/cets_disco_SUITE.erl Show resolved Hide resolved
big_tests/tests/cets_disco_SUITE.erl Show resolved Hide resolved
Copy link
Contributor

@arcusfelis arcusfelis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a comment about ct pal

big_tests/tests/cets_disco_SUITE.erl Outdated Show resolved Hide resolved
@mongoose-im
Copy link
Collaborator

mongoose-im commented Nov 14, 2023

elasticsearch_and_cassandra_26 / elasticsearch_and_cassandra_mnesia / 42a2d30
Reports root/ big
OK: 369 / Failed: 0 / User-skipped: 38 / Auto-skipped: 0


small_tests_25 / small_tests / 42a2d30
Reports root / small


small_tests_26 / small_tests / 42a2d30
Reports root / small


small_tests_26_arm64 / small_tests / 42a2d30
Reports root / small


ldap_mnesia_25 / ldap_mnesia / 42a2d30
Reports root/ big
OK: 2310 / Failed: 0 / User-skipped: 867 / Auto-skipped: 0


ldap_mnesia_26 / ldap_mnesia / 42a2d30
Reports root/ big
OK: 2310 / Failed: 0 / User-skipped: 867 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 42a2d30
Reports root/ big
OK: 4277 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / 42a2d30
Reports root/ big
OK: 4277 / Failed: 0 / User-skipped: 112 / Auto-skipped: 0


internal_mnesia_26 / internal_mnesia / 42a2d30
Reports root/ big
OK: 2460 / Failed: 0 / User-skipped: 717 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / 42a2d30
Reports root/ big
OK: 4666 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


dynamic_domains_mysql_redis_26 / mysql_redis / 42a2d30
Reports root/ big
OK: 4245 / Failed: 0 / User-skipped: 144 / Auto-skipped: 0


pgsql_cets_26 / pgsql_cets / 42a2d30
Reports root/ big
OK: 4664 / Failed: 0 / User-skipped: 121 / Auto-skipped: 0


pgsql_mnesia_26 / pgsql_mnesia / 42a2d30
Reports root/ big
OK: 4666 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_26 / odbc_mssql_mnesia / 42a2d30
Reports root/ big
OK: 4274 / Failed: 0 / User-skipped: 115 / Auto-skipped: 0


mysql_redis_26 / mysql_redis / 42a2d30
Reports root/ big
OK: 4646 / Failed: 0 / User-skipped: 139 / Auto-skipped: 0


mssql_mnesia_26 / odbc_mssql_mnesia / 42a2d30
Reports root/ big
OK: 4663 / Failed: 0 / User-skipped: 122 / Auto-skipped: 0

Copy link
Contributor

@arcusfelis arcusfelis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good

@arcusfelis arcusfelis merged commit 42c7274 into master Nov 14, 2023
3 of 4 checks passed
@arcusfelis arcusfelis deleted the mongoose_cets_discovery_rdbms-improvements branch November 14, 2023 13:24
@chrzaszcz chrzaszcz added this to the 6.2.0 milestone Dec 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants