Task/migrate esper 8.4 #176

AlvaroVega · 2021-09-10T08:33:33Z

issue #136

Clean old code
Improve logs

Add new tests:

perseo-core/perseo-main/src/test/java/com/telefonica/iot/perseo/test/Help.java

Lines 53 to 56 in 8e2420e

    
           ,"SELECT current_timestamp() as r FROM iotEvent WHERE cast(value?,int) > 10" 
        
           ,"expression twoPI alias for { java.lang.Math.PI * 2 > 5 } SELECT *, twoPI as r FROM iotEvent WHERE twoPI" 
        
           ,"expression E {(v1,v2) => max(v1,v2)} select E(1, 2) from iotEvent" 
        
           ,"expression double js:fib(num) [ fib(num); function fib(n) {  if(n <= 1) return n; return fib(n-1) + fib(n-2);}] select fib(5) from iotEvent"

Select minor version of esper (> 8.4 com.espertech.esper.compiler.client.EPCompileExceptionItem: Implicit conversion from datatype 'double' to 'Float' for property 'cast(cast(BloodPressure?,String),float)' is not allowed (strict filter type coercion)
Propagate error detail to perseo-fe

AlvaroVega · 2021-09-10T09:57:02Z

sudo docker exec iotp-pqa1 bash -c "cd /opt/iotp-pqa && behave ./tests/ --tags=onpremise --tags=-skip --tags=cb2cep"

Failing scenarios:
tests/features/TS03_cbflows/001_cb2cep.feature:91 cb001003 Entity update in CB matches a basic TYPE rule with action EMAIL -- @1.1
tests/features/TS03_cbflows/001_cb2cep.feature:92 cb001003 Entity update in CB matches a basic TYPE rule with action EMAIL -- @1.2
tests/features/TS03_cbflows/001_cb2cep.feature:261 cb001010 Entity update in CB matches a basic TYPE rule with action EMAIL and wildcards -- @1.1
tests/features/TS03_cbflows/001_cb2cep.feature:262 cb001010 Entity update in CB matches a basic TYPE rule with action EMAIL and wildcards -- @1.2
tests/features/TS03_cbflows/006_cb2cep2_conditionbasedonattrmetadata.feature:109 trigger a plain rule with a Email action -- @1.1
tests/features/TS03_cbflows/006_cb2cep2_conditionbasedonattrmetadata.feature:110 trigger a plain rule with a Email action -- @1.2
tests/features/TS03_cbflows/006_cb2cep2_conditionbasedonattrmetadata.feature:144 trigger a plain rule with a Email action and Metadata available -- @1.1
tests/features/TS03_cbflows/006_cb2cep2_conditionbasedonattrmetadata.feature:145 trigger a plain rule with a Email action and Metadata available -- @1.2
tests/features/TS03_cbflows/006_cb2cep2_conditionbasedonattrmetadata.feature:251 trigger a plain rule with a post action -- @1.2
tests/features/TS03_cbflows/008_cb2cep2_conditionbasedonlocation.feature:79 trigger a plain rule based on location using GeoJSON format and Email action
tests/features/TS03_cbflows/009_cb2cep2_conditionbasedontimestamp.feature:72 SC_2 - cb00902 - Trigger a plain rule based on timestamp if it is into the 30 last minutes and Email action

AlvaroVega · 2021-09-14T14:55:47Z

with esper 8.5+ EPL like

 "@Audit select *,\"blood_1_action\" as iotcepaction,"
                    + "ev.BloodPressure? as Pression, ev.id? as Meter from pattern "
                    + "[every ev=iotEvent(cast(cast(BloodPressure?,String),float)>1.5"
                    + " and type=\"BloodMeter\")]"

com.espertech.esper.compiler.client.EPCompileExceptionItem: Implicit conversion from datatype 'double' to 'Float' for property 'cast(cast(BloodPressure?,String),float)' is not allowed (strict filter type coercion)

AlvaroVega · 2021-09-16T15:08:24Z

Enabling query plan:
// Produce query plan output unless logging at debug-level
configuration.getCommon().getLogging().setEnableQueryPlan(true);

and trying with a rule with text:

expression double js:fib(num) [ fib(num); function fib(n) { if(n <= 1) return n; return fib(n-1) + fib(n-2);} ] expression twoPI alias for { java.lang.Math.PI * 2 } SELECT "mirule2" as ruleName, *, 3 as r FROM iotEvent

time=2021-09-16T15:05:51.852Z | lvl=DEBUG | from=::ffff:172.17.0.15 | corr=dabb741a-a074-4fdd-84eb-79f783c0da01; node=k6ah1LRew; perseocep=67 | trans=a3f9e91e-8fff-4419-bed7-806fe534b587 | srv=smartcity | subsrv=/ | op=parse | comp=perseo-core | msg=Error parsing statement [expression double js:fib(num) [ fib(num); function fib(n) { if(n <= 1) return n; return fib(n-1) + fib(n-2);} ] expression twoPI alias for { java.lang.Math.PI * 2 } context ctxt$smartcity$ SELECT "mirule2" as ruleName, *, 3 as r FROM iotEvent]
org.antlr.v4.runtime.misc.ParseCancellationException
at org.antlr.v4.runtime.BailErrorStrategy.recover(BailErrorStrategy.java:51)
at com.espertech.esper.compiler.internal.generated.EsperEPL2GrammarParser.expressionDef(EsperEPL2GrammarParser.java:1054)
at com.espertech.esper.compiler.internal.generated.EsperEPL2GrammarParser.expressionDecl(EsperEPL2GrammarParser.java:925)
at com.espertech.esper.compiler.internal.generated.EsperEPL2GrammarParser.startEPLExpressionRule(EsperEPL2GrammarParser.java:641)
at com.espertech.esper.compiler.internal.util.CompilerHelperSingleEPL$1.invokeParseRule(CompilerHelperSingleEPL.java:49)
at com.espertech.esper.compiler.internal.parse.ParseHelper.parse(ParseHelper.java:94)
at com.espertech.esper.compiler.internal.util.CompilerHelperSingleEPL.parse(CompilerHelperSingleEPL.java:155)
at com.espertech.esper.compiler.internal.util.CompilerHelperSingleEPL.parseCompileInlinedClassesWalk(CompilerHelperSingleEPL.java:62)
at com.espertech.esper.compiler.internal.util.CompilerHelperStatementProvider.compileItem(CompilerHelperStatementProvider.java:92)
at com.espertech.esper.compiler.internal.util.CompilerHelperModuleProvider.compileToBytes(CompilerHelperModuleProvider.java:127)
at com.espertech.esper.compiler.internal.util.CompilerHelperModuleProvider.compile(CompilerHelperModuleProvider.java:89)
at com.espertech.esper.compiler.internal.util.EPCompilerImpl.compile(EPCompilerImpl.java:98)
at com.telefonica.iot.perseo.Utils.compileDeploy(Utils.java:417)
at com.telefonica.iot.perseo.RulesManager.make(RulesManager.java:138)
at com.telefonica.iot.perseo.RulesServlet.doPost(RulesServlet.java:122)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:681)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:196)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:544)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:698)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:364)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:624)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:831)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1650)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.antlr.v4.runtime.NoViableAltException
at com.espertech.esper.compiler.internal.generated.EsperEPL2GrammarParser.stringconstant(EsperEPL2GrammarParser.java:22593)
at com.espertech.esper.compiler.internal.generated.EsperEPL2GrammarParser.expressionDef(EsperEPL2GrammarParser.java:1042)
... 37 more

only sencond expression works

fgalan · 2021-09-27T14:16:28Z

perseo-main/src/test/java/com/telefonica/iot/perseo/test/EventBeanMock.java

+        // To be included with using Esper 8.7
+        // @Override
+        // public EPTypeClass getUnderlyingEPType() {
+        //     throw new UnsupportedOperationException("Not supported yet."); //To change body of generated methods, choose Tools | Templates.
+        // }
+
+        // @Override
+        // public EPType getPropertyEPType(java.lang.String string) {
+        //     throw new UnsupportedOperationException("Not supported yet."); //To change body of generated methods, choose Tools | Templates.
+        // }
+        };


Leftover to be removed?

Just a comment for the future, when we try to update from 8.4 to 8.7, we will need it

fgalan · 2021-09-27T14:16:43Z

perseo-main/src/test/java/com/telefonica/iot/perseo/test/Help.java

+                 //,"inlined_class \"\"\"\n  public class MyUtility {\n    public static double fib(int n) {\n      if (n <= 1) {\n        return n;\n      }\n      return fib(n-1) + fib(n-2);\n    }\n  }\n\"\"\"\n expression FIBC alias for { MyUtility.fib(5) > 3 } SELECT *, FIBC as r FROM iotEvent WHERE FIBC"
+                 //,"expression double js:fib(num) [\nfib(num);\nfunction fib(n) {\n  if(n <= 1)\n    return n;\n  return fib(n-1) + fib(n-2);\n}\n]\nexpression FIBE alias for { fib(5) > 3 } SELECT *, FIBE as r FROM iotEvent WHERE FIBE"


Leftover commented lines?

Just a comment for the future, we expect that epl text will work with a future version

fgalan · 2021-09-27T14:17:17Z

perseo-main/src/main/java/com/telefonica/iot/perseo/RulesManager.java

                    statement.addListener(new GenericListener());
                } else {
                    logger.debug(String.format("found repeated statement: %s", name));
                    statement = prevStmnt;
                }
            }
            return new Result(HttpServletResponse.SC_OK,
-                    Utils.Statement2JSONObject(statement).toString());
+                              Utils.Statement2JSONObject(statement, epa, statement.getDeploymentId()).toString());


Over indentation here?

fixed in 24494fc

fgalan

LGTM

AlvaroVega added 12 commits September 1, 2021 16:24

migrate to epl 8

7e85540

fix compilation errors

fb5b7af

add missed exceptions

968d688

migrate tests to esper 8 api

9a231ad

initialize runtime

ab4e09d

add Utils for esper 8

9839029

fix statement

3ac3c66

update Util test

4170750

update delete server test

c26cc0c

set compiler options like in esper 7

41cdb9b

get deploymentId when st is not null

c6240ca

revert servlet test about delete

1591925

AlvaroVega changed the title ~~- [ ] Task/migrate epl 8~~ [WIP] Task/migrate esper 8.x Sep 10, 2021

AlvaroVega added 11 commits September 10, 2021 14:55

migrate to Esper 8.X

8c20c57

improve logs when fire rule

c70ed3b

set 8.7.0 esper version

53dd0d4

update CNR

2cb3810

add missed method in 8.7

02b484b

add missed method in 8.7

bbe82d0

add missed method in 8.7

329f7b0

add missed getPropertyEPType method

d9b6c53

add missed getPropertyEPType method

987b0f3

add missed getPropertyEPType method

dc79b70

fix esper version to 8.4

dcbb382

propagate error compilir epl

ed872d6

AlvaroVega changed the title ~~[WIP] Task/migrate esper 8.x~~ [WIP] Task/migrate esper 8.4 Sep 15, 2021

AlvaroVega added 2 commits September 15, 2021 12:46

loop to make all ExampleRules

4a859c8

extend to provide several notices examples

e5c39f5

fix to esper 8.7

8e2420e

AlvaroVega mentioned this pull request Sep 15, 2021

[WIP] Task/migrate epl 8.7 #177

Closed

1 task

AlvaroVega changed the title ~~[WIP] Task/migrate esper 8.4~~ [WIP] Task/migrate esper 8.7 Sep 15, 2021

add new EPL test with expression and case

075eae2

AlvaroVega added 2 commits September 17, 2021 12:24

fix to esper 8.4

c7bafcc

add WIP test

3a87986

AlvaroVega changed the title ~~[WIP] Task/migrate esper 8.7~~ Task/migrate esper 8.4 Sep 20, 2021

fgalan reviewed Sep 27, 2021

View reviewed changes

AlvaroVega added 3 commits September 28, 2021 10:56

fix identation

24494fc

Update EventBeanMock.java

63cd47b

Update Help.java

0078935

fgalan approved these changes Sep 28, 2021

View reviewed changes

fgalan merged commit dd1be3f into master Sep 28, 2021

fgalan deleted the task/migrate_epl_8 branch September 28, 2021 09:38

fgalan mentioned this pull request Sep 28, 2021

Migrate to esper 8.X.0 #136

Closed

AlvaroVega mentioned this pull request Feb 24, 2022

rule state is false and is not working telefonicaid/perseo-fe#633

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Task/migrate esper 8.4 #176

Task/migrate esper 8.4 #176

AlvaroVega commented Sep 10, 2021 •

edited

Loading

AlvaroVega commented Sep 10, 2021

AlvaroVega commented Sep 14, 2021 •

edited

Loading

AlvaroVega commented Sep 16, 2021

fgalan Sep 27, 2021

AlvaroVega Sep 28, 2021

fgalan Sep 28, 2021

fgalan Sep 27, 2021

AlvaroVega Sep 28, 2021

fgalan Sep 28, 2021

fgalan Sep 27, 2021

AlvaroVega Sep 28, 2021

fgalan left a comment

	,"SELECT current_timestamp() as r FROM iotEvent WHERE cast(value?,int) > 10"
	,"expression twoPI alias for { java.lang.Math.PI * 2 > 5 } SELECT *, twoPI as r FROM iotEvent WHERE twoPI"
	,"expression E {(v1,v2) => max(v1,v2)} select E(1, 2) from iotEvent"
	,"expression double js:fib(num) [ fib(num); function fib(n) { if(n <= 1) return n; return fib(n-1) + fib(n-2);}] select fib(5) from iotEvent"

		//,"inlined_class \"\"\"\n public class MyUtility {\n public static double fib(int n) {\n if (n <= 1) {\n return n;\n }\n return fib(n-1) + fib(n-2);\n }\n }\n\"\"\"\n expression FIBC alias for { MyUtility.fib(5) > 3 } SELECT *, FIBC as r FROM iotEvent WHERE FIBC"
		//,"expression double js:fib(num) [\nfib(num);\nfunction fib(n) {\n if(n <= 1)\n return n;\n return fib(n-1) + fib(n-2);\n}\n]\nexpression FIBE alias for { fib(5) > 3 } SELECT *, FIBE as r FROM iotEvent WHERE FIBE"

Task/migrate esper 8.4 #176

Task/migrate esper 8.4 #176

Conversation

AlvaroVega commented Sep 10, 2021 • edited Loading

AlvaroVega commented Sep 10, 2021

AlvaroVega commented Sep 14, 2021 • edited Loading

AlvaroVega commented Sep 16, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fgalan left a comment

Choose a reason for hiding this comment

AlvaroVega commented Sep 10, 2021 •

edited

Loading

AlvaroVega commented Sep 14, 2021 •

edited

Loading