Gdpr retrieve personal data from mod_private #2288

aleklisi · 2019-04-26T09:24:36Z

This PR retrieves personal data from mod_private

rebase after Fix string:prefix/2 for OTP < 20 #2293 is merged
add test for retrieving more than one private xml

implement get_all_nss/2 callback for:

mnesia
riak
rdbms
mysql ??

codecov · 2019-05-06T13:41:16Z

Codecov Report

Merging #2288 into gdpr-retrieve-clean will increase coverage by 0.02%.
The diff coverage is 89.65%.

@@                   Coverage Diff                   @@
##           gdpr-retrieve-clean    #2288      +/-   ##
=======================================================
+ Coverage                78.94%   78.96%   +0.02%     
=======================================================
  Files                      334      334              
  Lines                    29067    28963     -104     
=======================================================
- Hits                     22946    22870      -76     
+ Misses                    6121     6093      -28

Impacted Files	Coverage Δ
src/mod_private_mysql.erl	`79.16% <0%> (-11.31%)`	⬇️
src/mod_private.erl	`87.71% <100%> (+3.62%)`	⬆️
src/mod_private_mnesia.erl	`96% <100%> (+0.76%)`	⬆️
src/rdbms/rdbms_queries.erl	`83.16% <100%> (+0.08%)`	⬆️
src/mod_private_riak.erl	`85.18% <100%> (+3.36%)`	⬆️
src/mod_private_rdbms.erl	`91.66% <100%> (+1.19%)`	⬆️
...bal_distrib/mod_global_distrib_hosts_refresher.erl	`82.22% <0%> (-2.23%)`	⬇️
src/mod_bosh.erl	`92.85% <0%> (-2.15%)`	⬇️
...c/global_distrib/mod_global_distrib_server_mgr.erl	`83.09% <0%> (-2.12%)`	⬇️
src/ejabberd_admin.erl	`58.76% <0%> (-1.99%)`	⬇️
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f1f4cc0...c24ba6e. Read the comment docs.

mongoose-im · 2019-05-10T12:55:31Z

6462.1 / Erlang 19.3 / small_tests / 258638d
Reports root / small

6462.4 / Erlang 19.3 / odbc_mssql_mnesia / 258638d
Reports root/ big
OK: 3093 / Failed: 0 / User-skipped: 230 / Auto-skipped: 0

6462.3 / Erlang 19.3 / mysql_redis / 258638d
Reports root/ big
OK: 3102 / Failed: 3 / User-skipped: 232 / Auto-skipped: 0

pubsub_SUITE:dag+collection:request_all_items_leaf_test

{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,<<"alicE6.91256@localhost/res1">>,escalus_tcp,
          <0.11159.3>,
          [{event_manager,<0.11111.3>},
           {server,<<"localhost">>},
           {username,<<"alicE6.91256">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.11111.3>},
            {server,<<"localhost">>},
            {username,<<"alicE6.91256">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,<<"alicE6.91256">>},
           {server,<<"localhost">>},
           {host,<<"localhost">>},
           {port,5222},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,<<"alicE6.91256">>},
           {server,<<"localhost">>},
           {password,<<"matygrysa">>},
           {stream_id,<<"87822D908F589E0E">>}]},
        5000],
       [{file,
          "/home/travis/build/esl/MongooseIM/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,138}]},
     {pubsub_tools,receive_response,3,
       [{file,"pubsub_tools.erl"},{line,457}]},
     {pubsub_tools,receive_and_check_response,4,
       [{file,"pubsub_tools.erl"},{line,448}]},
     {pubsub_SUITE,'-request_all_items_leaf_test/1-fun-0-',2,
       [{file,"pubsub_SUITE.erl"},{line,1519}]},
     {escalus_story,story,4,
       [{file,
          "/home/travis/build/esl/MongooseIM/big_tests/_build/default/lib/esc...

Report log

pubsub_SUITE:dag+collection:notify_collection_and_leaf_same_user_test

{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,<<"alicE6.85648@localhost/res1">>,escalus_tcp,
          <0.11160.3>,
          [{event_manager,<0.11082.3>},
           {server,<<"localhost">>},
           {username,<<"alicE6.85648">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.11082.3>},
            {server,<<"localhost">>},
            {username,<<"alicE6.85648">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,<<"alicE6.85648">>},
           {server,<<"localhost">>},
           {host,<<"localhost">>},
           {port,5222},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,<<"alicE6.85648">>},
           {server,<<"localhost">>},
           {password,<<"matygrysa">>},
           {stream_id,<<"9DD27D02BC104E41">>}]},
        5000],
       [{file,
          "/home/travis/build/esl/MongooseIM/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,138}]},
     {pubsub_tools,receive_response,3,
       [{file,"pubsub_tools.erl"},{line,457}]},
     {pubsub_tools,receive_and_check_response,4,
       [{file,"pubsub_tools.erl"},{line,448}]},
     {pubsub_SUITE,'-notify_collection_and_leaf_same_user_test/1-fun-0-',
       2,
       [{file,"pubsub_SUITE.erl"},{line,1343}]},
     {escalus_story,story,4,
       [{file,
          "/home/travis/build/esl/MongooseIM/big_tests/...

Report log

pubsub_SUITE:dag+collection:retrieve_subscriptions_collection_test

{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,<<"alicE6.97805@localhost/res1">>,escalus_tcp,
          <0.11163.3>,
          [{event_manager,<0.11128.3>},
           {server,<<"localhost">>},
           {username,<<"alicE6.97805">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.11128.3>},
            {server,<<"localhost">>},
            {username,<<"alicE6.97805">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,<<"alicE6.97805">>},
           {server,<<"localhost">>},
           {host,<<"localhost">>},
           {port,5222},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,<<"alicE6.97805">>},
           {server,<<"localhost">>},
           {password,<<"matygrysa">>},
           {stream_id,<<"6A43F882E4425136">>}]},
        5000],
       [{file,
          "/home/travis/build/esl/MongooseIM/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,138}]},
     {pubsub_tools,receive_response,3,
       [{file,"pubsub_tools.erl"},{line,457}]},
     {pubsub_tools,receive_and_check_response,4,
       [{file,"pubsub_tools.erl"},{line,448}]},
     {pubsub_SUITE,'-retrieve_subscriptions_collection_test/1-fun-0-',2,
       [{file,"pubsub_SUITE.erl"},{line,1431}]},
     {escalus_story,story,4,
       [{file,
          "/home/travis/build/esl/MongooseIM/big_tests/_build/defa...

Report log

6462.5 / Erlang 19.3 / ldap_mnesia / 258638d
Reports root/ big
OK: 1203 / Failed: 2 / User-skipped: 102 / Auto-skipped: 0

sm_SUITE:parallel_manual_ack_freq_1:resume_session_state_stop_c2s

{error,{{assertion_failed,assert_many,true,
              [is_presence,#Fun<sm_SUITE.16.86460923>],
              [{xmlel,<<"presence">>,
                  [{<<"from">>,
                  <<"alicE27.324727@localhost/escalus-default-resource">>},
                   {<<"to">>,
                  <<"alice27.324727@localhost/escalus-default-resource">>},
                   {<<"xml:lang">>,<<"en">>}],
                  []},
               {xmlel,<<"presence">>,
                  [{<<"from">>,
                  <<"alicE27.324727@localhost/escalus-default-resource">>},
                   {<<"to">>,
                  <<"alice27.324727@localhost/escalus-default-resource">>},
                   {<<"xml:lang">>,<<"en">>}],
                  []}],
              "   <presence from='alicE27.324727@localhost/escalus-default-resource' to='alice27.324727@localhost/escalus-default-resource' xml:lang='en'/>   <presence from='alicE27.324727@localhost/escalus-default-resource' to='alice27.324727@localhost/escalus-default-resource' xml:lang='en'/>"},
    [{escalus_new_assert,assert_true,2,
               [{file,"/home/travis/build/esl/MongooseIM/big_tests/_build/default/lib/escalus/src/escalus_new_assert.erl"},
                {line,84}]},
     {sm_SUITE,resume_session_state_stop_c2s,1,
           [{file,"sm_SUITE.erl"},{line,581}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1529}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1045}]},
     {...

Report log

mod_global_distrib_SUITE:mod_global_distrib:test_pm_with_ungraceful_reconnection_to_different_server

{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,<<"eve9.96015@localhost/res1">>,escalus_tcp,
          <0.16873.1>,
          [{event_manager,<0.16864.1>},
           {server,<<"localhost">>},
           {username,<<"eve9.96015">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.16864.1>},
            {server,<<"localhost">>},
            {username,<<"eve9.96015">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,<<"eve9.96015">>},
           {server,<<"localhost">>},
           {host,<<"localhost">>},
           {port,5222},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,<<"eve9.96015">>},
           {server,<<"localhost">>},
           {password,<<"password">>},
           {port,5222},
           {stream_management,true},
           {stream_id,<<"5978F7BF42A7B119">>}]},
        10000],
       [{file,
          "/home/travis/build/esl/MongooseIM/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,138}]},
     {mod_global_distrib_SUITE,
       '-test_pm_with_ungraceful_reconnection_to_different_server/1-fun-0-',
       4,
       [{file,"mod_global_distrib_SUITE.erl"},{line,610}]},
     {escalus_story,story,4,
       [{file,
          "/home/travis/build/esl/MongooseIM/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
        {line,72}]},
     {test_ser...

Report log

6462.6 / Erlang 19.3 / elasticsearch_and_cassandra_mnesia / 258638d
Reports root/ big
OK: 469 / Failed: 0 / User-skipped: 8 / Auto-skipped: 0

6462.8 / Erlang 20.0 / pgsql_mnesia / 258638d
Reports root/ big / small
OK: 3125 / Failed: 0 / User-skipped: 198 / Auto-skipped: 0

6462.9 / Erlang 21.0 / riak_mnesia / 258638d
Reports root/ big / small
OK: 1455 / Failed: 0 / User-skipped: 63 / Auto-skipped: 0

* Implement retreive personal data for mod_private with mnesia * Enable test for retrive data from mod_private * Implement retreive personal data for mod_private with riak * Implement retreive mod_privare for rdbms * Test retrieving multiple private xmls * Implement retreive for mod_private_mysql * Refactor retreive personal xml tests * Minor fixes in gdpr_SUITE * Fix GDPR retrieval for mod_private_riak * Move new mod_private callback to other behaviour defs * Retrieve personal private XML from all backends * Style fixes in private XML GDPR retrieval * Fix style fixes in private XML GDPR retrieval * Fix the fix of the style fixes * Fix of the fix the fix of the style fixes

aleklisi added the WIP 🚧 label Apr 26, 2019