Rebased code changes in EI-2.X to EI-3.X and added TTL indexes automatic updation. #507

Gangisetty-Alekhya · 2021-09-09T09:59:32Z

Description of the Change

Rebased the changes that were done on EI2.X to EI3.X. Modified code to handle the missed subscriptions and EI performance improvements and also handled the TTL values and updation of indexes for the collections is done automatically.

Alternate Designs

NA

Benefits

No events are dropped, and the event processing rate is improved which inturn increases the EI performance.
The indexes are updated automatically w.r.t TTL values provided.

Possible Drawbacks

NA

Sign-off

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or

(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or

(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.

(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

Signed-off-by: Raju Beemreddy raju.beemreddy@tcs.com, Gangisetty Alekhya gangisetty.alekhya@tcs.com, Kolli Suryakumari sk.surya@tcs.com

- Reduced the duplicate calls on subscription checks - Introduced ttl for event object map collection - Fixed Null pointer exceptions - Cache implemented for repeat handler collection. - Added log statements to print response time of the mongo and ER queries

…for the REST calls while triggering the notifications. Updated the code to handle the index creation only once.

"notification.httpRequest.timeout"

zburswa · 2021-09-16T04:39:15Z

src/integrationtests/java/com/ericsson/ei/integrationtests/flow/FlowStepsIT.java

@@ -88,10 +88,12 @@
    private JenkinsXmlData jenkinsXmlData;
    private SubscriptionObject subscriptionObject;
    private JSONObject jobStatusData;
+    public String AggEvent;


Please give a better name instead of AggEvent

follow java naming standards for variables

Done, changed AggEvent to aggregatedEvent

zburswa · 2021-09-16T04:43:56Z

src/main/java/com/ericsson/ei/handlers/EventHandler.java

-            LOGGER.debug(
-                    "Sent back the event to queue with un-acknowledgement: " + message.getBody());
+            LOGGER.info(
+                    "Sent back the event {} to queue with un-acknowledgement due to {}", id, mdce);


Can you explain what is this mdce?

SantoshNC68 · 2021-09-30T06:15:51Z

...unctionaltests/java/com/ericsson/ei/notifications/trigger/SubscriptionNotificationSteps.java

-                                                        EIFFEL_EVENTS_JSON_PATH, eventNamesToSend),
-                                                        0);
+        JsonNode parsedJSON = IntegrationTestBase.getJSONFromFile(EIFFEL_EVENTS_JSON_PATH);
+        for(String eventName : eventNamesToSend) {


use forEach

SantoshNC68 · 2021-09-30T06:16:55Z

src/functionaltests/java/com/ericsson/ei/notifications/ttl/TestTTLSteps.java

@@ -177,6 +183,8 @@ public void aggregated_object_is_created() throws Throwable {
        // verify that aggregated object is created and present in db
        LOGGER.debug("Checking presence of aggregated Object");
        List<String> allObjects = mongoDBHandler.getAllDocuments(database, collection);
+        String id = eventManager.getEventsIdList(EIFFEL_EVENTS_JSON_PATH, getEventNamesToSend()).get(0);


rename the method to getEventIdsList

SantoshNC68 · 2021-09-30T06:17:35Z

src/functionaltests/java/com/ericsson/ei/rules/RuleTestSteps.java

@@ -107,6 +107,10 @@ public void get_content(String contentFileName) throws Throwable {
        if (expectedResponse instanceof JSONArray) {
            JSONArray expectedArray = new JSONArray(responseBody);
            JSONArray responseArray = new JSONArray(response.getBody().toString());
+            for(int i=0;i<responseArray.length();i++) {


format the code

SantoshNC68 · 2021-09-30T06:18:38Z

src/integrationtests/java/com/ericsson/ei/integrationtests/flow/FlowStepsIT.java

@@ -88,10 +88,12 @@
    private JenkinsXmlData jenkinsXmlData;
    private SubscriptionObject subscriptionObject;
    private JSONObject jobStatusData;
+    public String AggEvent;


follow java naming standards for variables

SantoshNC68 · 2021-09-30T06:21:14Z

src/integrationtests/java/util/IntegrationTestBase.java

+        String queryString = "{\"_id\": " + aggId + "}";
+        MongoStringQuery query = new MongoStringQuery(queryString);
+        documents = mongoDBHandler.find(database, collectionName, query);
+        JSONObject json = new JSONObject(documents.get(0));


should this only be performed for the single event?

SantoshNC68 · 2021-09-30T06:21:32Z

src/integrationtests/java/util/IntegrationTestBase.java

+        JSONObject json = new JSONObject(documents.get(0));
+        JSONArray jsonArray = new JSONArray();
+        jsonArray.put(json.get("objects"));
+        String s = json.get("objects").toString();


proper name instead of s

SantoshNC68 · 2021-09-30T06:23:29Z

src/integrationtests/java/util/IntegrationTestBase.java

+                break;
+            }
+        }
+        if (content.isEmpty()) {


move this check next to "content = read...."

SantoshNC68 · 2021-09-30T06:24:21Z

src/integrationtests/java/util/IntegrationTestBase.java

@@ -95,7 +101,9 @@
    private String failedNotificationCollectionName;
    @Value("${sessions.collection.name}")
    private String sessionsCollectionName;
-
+
+    List<String> expectedAggId = Stream.of("sb6efi4n-25fb-4d77-b9fd-5f2xrrefe66de47","aacc3c87-75e0-4b6d-88f5-b1a5d4e62b43","e46ef12d-25gb-4d7y-b9fd-8763re66de47").collect(Collectors.toList());


use ArrayLIst instead of Stream.of and collect.

This ArrayList is not required, I have removed this line.

…-intelligence # Conflicts: # src/functionaltests/java/com/ericsson/ei/notifications/trigger/SubscriptionNotificationSteps.java

Raju Beemreddy and others added 25 commits July 22, 2021 18:33

Logger info add for the received events

7ae0aee

Configurable paramter is introduced to restrict the default timeouts …

0a0f732

…for the REST calls while triggering the notifications. Updated the code to handle the index creation only once.

Added loggers with additional info

bc6347a

Fixed test cases

45cd09f

Fixed the failure test cases

a6fcbfc

Dummu push to test the travis build

401c3df

Travis build testing

7708b12

Dummy push to test the travis build

da242cf

Testing failed Fts

3c71e9b

Testing failed test cases

442ba40

Failure tests

05cfc1f

Fix for waitlist queue issue

318c7e5

checking for testcase failures

70ed7ca

dummy push

dcdd6cc

checking for itegration test failures

ae3b4c9

Checking for Integration test failures

4e51740

dummy push

c0a1afa

dummy push2

53ae74a

checking for integration testcases

ad6b69f

dummy push to check for IT failures

265efb6

dummy push

1de3f93

Checking Integration tests

23a6438

check for ft failures

015e358

Fixed EventToObjectMap handling and removed the configurable paramter

57b96d1

"notification.httpRequest.timeout"

Gangisetty-Alekhya marked this pull request as ready for review September 15, 2021 05:12

final check

e1b6c4e

zburswa reviewed Sep 16, 2021

View reviewed changes

Surya-Kolli self-assigned this Sep 30, 2021

Surya-Kolli unassigned SantoshNC68 and RajuBeemreddy1 Sep 30, 2021

Surya-Kolli requested review from RajuBeemreddy1, SantoshNC68 and tobiasake September 30, 2021 05:52

SantoshNC68 requested changes Sep 30, 2021

View reviewed changes

Gangisetty-Alekhya added 18 commits September 30, 2021 14:37

fixed the comments

981a14b

checking for testcase failures

f44b4d6

dummy push

4c7bf65

checking for itegration test failures

c531d69

Checking for Integration test failures

bae597c

dummy push

c6eeb9d

dummy push2

4fba1c3

checking for integration testcases

e35eddd

dummy push to check for IT failures

210d23e

dummy push

9c4d875

Checking Integration tests

7812024

check for ft failures

5dc46f1

final check

76b12b8

fixed the comments

c1a99e6

final check

0c2ee6d

Merge branch 'master' of https://github.com/Gangisetty-Alekhya/eiffel…

144d63b

…-intelligence # Conflicts: # src/functionaltests/java/com/ericsson/ei/notifications/trigger/SubscriptionNotificationSteps.java

final push

133d408

Fixed all the comments

0064e7e

Surya-Kolli changed the title ~~EI should handle the TTL indexes updating automatically~~ Rebased code changes in EI-2.X to EI-3.X and added TTL indexes automatic updation. Oct 20, 2021

Gangisetty-Alekhya added 2 commits October 20, 2021 12:03

Updated version in pom file from 3.1.2 to 3.1.3

8cdd190

Fixed the comments and modified documentation.

0bf11ae

RajuBeemreddy1 approved these changes Oct 20, 2021

View reviewed changes

SantoshNC68 approved these changes Oct 21, 2021

View reviewed changes

Surya-Kolli merged commit a1bd6ad into eiffel-community:master Oct 21, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rebased code changes in EI-2.X to EI-3.X and added TTL indexes automatic updation. #507

Rebased code changes in EI-2.X to EI-3.X and added TTL indexes automatic updation. #507

Gangisetty-Alekhya commented Sep 9, 2021 •

edited by Surya-Kolli

Loading

zburswa Sep 16, 2021

SantoshNC68 Sep 30, 2021

Gangisetty-Alekhya Oct 20, 2021

zburswa Sep 16, 2021

SantoshNC68 Sep 30, 2021

Gangisetty-Alekhya Oct 20, 2021

SantoshNC68 Sep 30, 2021

Gangisetty-Alekhya Oct 20, 2021

SantoshNC68 Sep 30, 2021

Gangisetty-Alekhya Oct 20, 2021

SantoshNC68 Sep 30, 2021

SantoshNC68 Sep 30, 2021

Gangisetty-Alekhya Oct 20, 2021

SantoshNC68 Sep 30, 2021

Gangisetty-Alekhya Oct 20, 2021

SantoshNC68 Sep 30, 2021

Gangisetty-Alekhya Oct 20, 2021

SantoshNC68 Sep 30, 2021

Gangisetty-Alekhya Oct 20, 2021

Rebased code changes in EI-2.X to EI-3.X and added TTL indexes automatic updation. #507

Rebased code changes in EI-2.X to EI-3.X and added TTL indexes automatic updation. #507

Conversation

Gangisetty-Alekhya commented Sep 9, 2021 • edited by Surya-Kolli Loading

Description of the Change

Alternate Designs

Benefits

Possible Drawbacks

Sign-off

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Gangisetty-Alekhya commented Sep 9, 2021 •

edited by Surya-Kolli

Loading