Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sketches deploy JAR is missing the datasketches library #3159

Closed
GCHQDeveloper314 opened this issue Feb 14, 2024 · 0 comments · Fixed by #3160
Closed

Sketches deploy JAR is missing the datasketches library #3159

GCHQDeveloper314 opened this issue Feb 14, 2024 · 0 comments · Fixed by #3160
Assignees
@GCHQDeveloper314
Labels
bug Confirmed or suspected bug
Milestone
v2.2.0

Comments

@GCHQDeveloper314
Copy link
Member

Describe the bug
The Sketches deploy JAR sketches-library-2.1.0-deploy.jar contains only com.clearspring.analytics:stream. It's missing the datasketches library also used in the sketches-library module.
This JAR is used to bundle essential dependencies required by Gaffer Accumulo instances. The missing datasketches library results in failures if datasketches properties are used in a Gaffer graph with Accumulo.

To Reproduce
Steps to reproduce the behaviour:

  1. Run the road-traffic example with Accumulo (as done by Gaffer docker) with Gaffer 2.1.0
  2. Submit a query on the data (e.g. GetAllElements)

Expected behaviour
Should run the query as expect without errors and return results.

Stack trace and errors
Error response from REST API after submitting query:

{
  "statusCode": 500,
  "status": "Internal Server Error",
  "simpleMessage": "org.apache.accumulo.core.clientImpl.AccumuloServerException: Error on server accumulo-tserver:9997"
}

Errors in Gaffer REST instance logs:

2024-02-14 09:29:53 core.clientImpl.TabletServerBatchReaderIterator WARN  - Caught exception, but queryThreadPool is not shutdown
2024-02-14 09:29:53 org.apache.accumulo.core.clientImpl.AccumuloServerException: Error on server accumulo-tserver:9997
2024-02-14 09:29:53     at org.apache.accumulo.core.clientImpl.TabletServerBatchReaderIterator.doLookup(TabletServerBatchReaderIterator.java:759)
2024-02-14 09:29:53     at org.apache.accumulo.core.clientImpl.TabletServerBatchReaderIterator$QueryTask.run(TabletServerBatchReaderIterator.java:359)
2024-02-14 09:29:53     at org.apache.htrace.wrappers.TraceRunnable.run(TraceRunnable.java:57)
2024-02-14 09:29:53     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
2024-02-14 09:29:53     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
2024-02-14 09:29:53     at org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
2024-02-14 09:29:53     at java.lang.Thread.run(Thread.java:748)
2024-02-14 09:29:53 Caused by: org.apache.thrift.TApplicationException: Internal error processing startMultiScan
2024-02-14 09:29:53     at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
2024-02-14 09:29:53     at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.recv_startMultiScan(TabletClientService.java:326)
2024-02-14 09:29:53     at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.startMultiScan(TabletClientService.java:302)
2024-02-14 09:29:53     at org.apache.accumulo.core.clientImpl.TabletServerBatchReaderIterator.doLookup(TabletServerBatchReaderIterator.java:677)
2024-02-14 09:29:53     ... 6 more
2024-02-14 09:29:53 core.exception.ErrorFactory ERROR  - Error: org.apache.accumulo.core.clientImpl.AccumuloServerException: Error on server accumulo-tserver:9997
2024-02-14 09:29:53 java.lang.RuntimeException: org.apache.accumulo.core.clientImpl.AccumuloServerException: Error on server accumulo-tserver:9997
2024-02-14 09:29:53     at org.apache.accumulo.core.clientImpl.TabletServerBatchReaderIterator.hasNext(TabletServerBatchReaderIterator.java:177)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.accumulostore.retriever.AccumuloItemRetriever$ElementIterator.hasNext(AccumuloItemRetriever.java:132)
2024-02-14 09:29:53     at uk.gov.gchq.koryphe.iterable.LimitedIterator.hasNext(LimitedIterator.java:79)
2024-02-14 09:29:53     at java.util.Iterator.forEachRemaining(Iterator.java:115)
2024-02-14 09:29:53     at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
2024-02-14 09:29:53     at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
2024-02-14 09:29:53     at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
2024-02-14 09:29:53     at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
2024-02-14 09:29:53     at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
2024-02-14 09:29:53     at java.util.stream.LongPipeline.reduce(LongPipeline.java:439)
2024-02-14 09:29:53     at java.util.stream.LongPipeline.sum(LongPipeline.java:397)
2024-02-14 09:29:53     at java.util.stream.ReferencePipeline.count(ReferencePipeline.java:526)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.store.operation.handler.CountHandler.doOperation(CountHandler.java:32)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.store.operation.handler.CountHandler.doOperation(CountHandler.java:24)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.store.Store.handleOperation(Store.java:961)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.store.operation.handler.OperationChainHandler.doOperation(OperationChainHandler.java:49)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.store.operation.handler.OperationChainHandler.doOperation(OperationChainHandler.java:37)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.store.Store.handleOperation(Store.java:961)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.store.Store.execute(Store.java:373)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.store.Store.execute(Store.java:367)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.graph.Graph._execute(Graph.java:336)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.graph.Graph.execute(Graph.java:187)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.rest.service.v2.AbstractOperationService._execute(AbstractOperationService.java:108)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.rest.controller.OperationController.execute(OperationController.java:137)
2024-02-14 09:29:53     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2024-02-14 09:29:53     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2024-02-14 09:29:53     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2024-02-14 09:29:53     at java.lang.reflect.Method.invoke(Method.java:498)
2024-02-14 09:29:53     at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
2024-02-14 09:29:53     at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
2024-02-14 09:29:53     at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
2024-02-14 09:29:53     at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
2024-02-14 09:29:53     at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
2024-02-14 09:29:53     at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
2024-02-14 09:29:53     at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1072)
2024-02-14 09:29:53     at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:965)
2024-02-14 09:29:53     at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
2024-02-14 09:29:53     at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
2024-02-14 09:29:53     at javax.servlet.http.HttpServlet.service(HttpServlet.java:665)
2024-02-14 09:29:53     at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
2024-02-14 09:29:53     at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
2024-02-14 09:29:53     at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:209)
2024-02-14 09:29:53     at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
2024-02-14 09:29:53     at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
2024-02-14 09:29:53     at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
2024-02-14 09:29:53     at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.rest.filter.GafferHeaderFilter.doFilterInternal(GafferHeaderFilter.java:41)
2024-02-14 09:29:53     at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
2024-02-14 09:29:53     at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
2024-02-14 09:29:53     at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
2024-02-14 09:29:53     at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
2024-02-14 09:29:53     at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
2024-02-14 09:29:53     at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
2024-02-14 09:29:53     at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
2024-02-14 09:29:53     at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
2024-02-14 09:29:53     at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
2024-02-14 09:29:53     at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
2024-02-14 09:29:53     at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
2024-02-14 09:29:53     at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
2024-02-14 09:29:53     at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
2024-02-14 09:29:53     at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
2024-02-14 09:29:53     at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
2024-02-14 09:29:53     at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167)
2024-02-14 09:29:53     at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
2024-02-14 09:29:53     at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481)
2024-02-14 09:29:53     at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130)
2024-02-14 09:29:53     at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
2024-02-14 09:29:53     at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
2024-02-14 09:29:53     at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
2024-02-14 09:29:53     at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:390)
2024-02-14 09:29:53     at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
2024-02-14 09:29:53     at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:926)
2024-02-14 09:29:53     at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1791)
2024-02-14 09:29:53     at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
2024-02-14 09:29:53     at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
2024-02-14 09:29:53     at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
2024-02-14 09:29:53     at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
2024-02-14 09:29:53     at java.lang.Thread.run(Thread.java:748)
2024-02-14 09:29:53 Caused by: org.apache.accumulo.core.clientImpl.AccumuloServerException: Error on server accumulo-tserver:9997
2024-02-14 09:29:53     at org.apache.accumulo.core.clientImpl.TabletServerBatchReaderIterator.doLookup(TabletServerBatchReaderIterator.java:759)
2024-02-14 09:29:53     at org.apache.accumulo.core.clientImpl.TabletServerBatchReaderIterator$QueryTask.run(TabletServerBatchReaderIterator.java:359)
2024-02-14 09:29:53     at org.apache.htrace.wrappers.TraceRunnable.run(TraceRunnable.java:57)
2024-02-14 09:29:53     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
2024-02-14 09:29:53     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
2024-02-14 09:29:53     at org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
2024-02-14 09:29:53     ... 1 more
2024-02-14 09:29:53 Caused by: org.apache.thrift.TApplicationException: Internal error processing startMultiScan
2024-02-14 09:29:53     at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
2024-02-14 09:29:53     at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.recv_startMultiScan(TabletClientService.java:326)
2024-02-14 09:29:53     at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.startMultiScan(TabletClientService.java:302)
2024-02-14 09:29:53     at org.apache.accumulo.core.clientImpl.TabletServerBatchReaderIterator.doLookup(TabletServerBatchReaderIterator.java:677)
2024-02-14 09:29:53     ... 6 more

Errors in Accumulo Monitor logs:

2024-02-14 09:29:53 2024-02-14 09:29:53,881 [scan.LookupTask] WARN : exception while doing multi-scan 
2024-02-14 09:29:53 uk.gov.gchq.gaffer.data.elementdefinition.exception.SchemaException: Failed to load element definitions from bytes
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.data.elementdefinition.ElementDefinitions$BaseBuilder.json(ElementDefinitions.java:323)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.data.elementdefinition.ElementDefinitions$BaseBuilder.json(ElementDefinitions.java:303)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.store.schema.Schema$BaseBuilder.json(Schema.java:472)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.store.schema.Schema.fromJson(Schema.java:116)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.accumulostore.key.impl.AggregatorIterator.init(AggregatorIterator.java:91)
2024-02-14 09:29:53     at org.apache.accumulo.core.conf.IterConfigUtil.loadIterators(IterConfigUtil.java:227)
2024-02-14 09:29:53     at org.apache.accumulo.tserver.tablet.ScanDataSource.createIterator(ScanDataSource.java:248)
2024-02-14 09:29:53     at org.apache.accumulo.tserver.tablet.ScanDataSource.iterator(ScanDataSource.java:140)
2024-02-14 09:29:53     at org.apache.accumulo.core.iterators.system.SourceSwitchingIterator.seek(SourceSwitchingIterator.java:226)
2024-02-14 09:29:53     at org.apache.accumulo.tserver.tablet.Tablet.lookup(Tablet.java:614)
2024-02-14 09:29:53     at org.apache.accumulo.tserver.tablet.Tablet.lookup(Tablet.java:770)
2024-02-14 09:29:53     at org.apache.accumulo.tserver.scan.LookupTask.run(LookupTask.java:116)
2024-02-14 09:29:53     at org.apache.accumulo.tserver.session.ScanSession$ScanMeasurer.run(ScanSession.java:54)
2024-02-14 09:29:53     at org.apache.htrace.wrappers.TraceRunnable.run(TraceRunnable.java:57)
2024-02-14 09:29:53     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
2024-02-14 09:29:53     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
2024-02-14 09:29:53     at org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
2024-02-14 09:29:53     at java.lang.Thread.run(Thread.java:750)
2024-02-14 09:29:53 Caused by: uk.gov.gchq.gaffer.exception.SerialisationException: org.apache.datasketches.memory.Memory
2024-02-14 09:29:53  at [Source: (byte[])"{"edges":{"JunctionLocatedAt":{"source":"junction","destination":"coordinates","directed":"true"},"RoadUse":{"source":"junction","destination":"junction","directed":"true","properties":{"startDate":"date.earliest","endDate":"date.latest","count":"count.long","countByVehicleType":"counts.freqmap"},"groupBy":["startDate","endDate"]},"RoadHasJunction":{"source":"road","destination":"junction","directed":"true"},"RegionContainsLocation":{"source":"region","destination":"location","directed":"true"},"[truncated 2932 bytes]; line: 1, column: 2765] (through reference chain: uk.gov.gchq.gaffer.store.schema.Schema$Builder["types"]->java.util.LinkedHashMap["hllp"]->uk.gov.gchq.gaffer.store.schema.TypeDefinition["class"])
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.jsonserialisation.JSONSerialiser.deserialise(JSONSerialiser.java:334)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.data.elementdefinition.ElementDefinitions$BaseBuilder.json(ElementDefinitions.java:320)
2024-02-14 09:29:53     ... 17 more
2024-02-14 09:29:53 Caused by: hidden.com.fasterxml.jackson.databind.JsonMappingException: org.apache.datasketches.memory.Memory
2024-02-14 09:29:53  at [Source: (byte[])"{"edges":{"JunctionLocatedAt":{"source":"junction","destination":"coordinates","directed":"true"},"RoadUse":{"source":"junction","destination":"junction","directed":"true","properties":{"startDate":"date.earliest","endDate":"date.latest","count":"count.long","countByVehicleType":"counts.freqmap"},"groupBy":["startDate","endDate"]},"RoadHasJunction":{"source":"road","destination":"junction","directed":"true"},"RegionContainsLocation":{"source":"region","destination":"location","directed":"true"},"[truncated 2932 bytes]; line: 1, column: 2765] (through reference chain: uk.gov.gchq.gaffer.store.schema.Schema$Builder["types"]->java.util.LinkedHashMap["hllp"]->uk.gov.gchq.gaffer.store.schema.TypeDefinition["class"])
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:276)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:623)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:611)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:143)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:314)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:609)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:437)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:32)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:158)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.vanillaDeserialize(BuilderBasedDeserializer.java:293)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:217)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4674)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3690)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.jsonserialisation.JSONSerialiser.deserialise(JSONSerialiser.java:332)
2024-02-14 09:29:53     ... 18 more
2024-02-14 09:29:53 Caused by: java.lang.ClassNotFoundException: org.apache.datasketches.memory.Memory
2024-02-14 09:29:53     at org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:148)
2024-02-14 09:29:53     at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
2024-02-14 09:29:53     at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
2024-02-14 09:29:53     at java.lang.Class.forName0(Native Method)
2024-02-14 09:29:53     at java.lang.Class.forName(Class.java:264)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.store.schema.TypeDefinition.setClassString(TypeDefinition.java:91)
2024-02-14 09:29:53     at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
2024-02-14 09:29:53     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2024-02-14 09:29:53     at java.lang.reflect.Method.invoke(Method.java:498)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:141)
2024-02-14 09:29:53     ... 30 more
2024-02-14 09:29:53 2024-02-14 09:29:53,882 [tserver.TabletServer] WARN : Failed to get multiscan result
2024-02-14 09:29:53 java.util.concurrent.ExecutionException: uk.gov.gchq.gaffer.data.elementdefinition.exception.SchemaException: Failed to load element definitions from bytes
2024-02-14 09:29:53     at org.apache.accumulo.tserver.scan.ScanTask.get(ScanTask.java:129)
2024-02-14 09:29:53     at org.apache.accumulo.tserver.TabletServer$ThriftClientHandler.continueMultiScan(TabletServer.java:888)
2024-02-14 09:29:53     at org.apache.accumulo.tserver.TabletServer$ThriftClientHandler.startMultiScan(TabletServer.java:852)
2024-02-14 09:29:53     at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
2024-02-14 09:29:53     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2024-02-14 09:29:53     at java.lang.reflect.Method.invoke(Method.java:498)
2024-02-14 09:29:53     at org.apache.accumulo.core.trace.TraceUtil.lambda$wrapService$8(TraceUtil.java:235)
2024-02-14 09:29:53     at com.sun.proxy.$Proxy21.startMultiScan(Unknown Source)
2024-02-14 09:29:53     at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$startMultiScan.getResult(TabletClientService.java:2756)
2024-02-14 09:29:53     at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$startMultiScan.getResult(TabletClientService.java:2735)
2024-02-14 09:29:53     at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
2024-02-14 09:29:53     at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
2024-02-14 09:29:53     at org.apache.accumulo.server.rpc.TimedProcessor.process(TimedProcessor.java:61)
2024-02-14 09:29:53     at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:518)
2024-02-14 09:29:53     at org.apache.accumulo.server.rpc.CustomNonBlockingServer$CustomFrameBuffer.invoke(CustomNonBlockingServer.java:112)
2024-02-14 09:29:53     at org.apache.thrift.server.Invocation.run(Invocation.java:18)
2024-02-14 09:29:53     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
2024-02-14 09:29:53     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
2024-02-14 09:29:53     at org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
2024-02-14 09:29:53     at java.lang.Thread.run(Thread.java:750)
2024-02-14 09:29:53 Caused by: uk.gov.gchq.gaffer.data.elementdefinition.exception.SchemaException: Failed to load element definitions from bytes
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.data.elementdefinition.ElementDefinitions$BaseBuilder.json(ElementDefinitions.java:323)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.data.elementdefinition.ElementDefinitions$BaseBuilder.json(ElementDefinitions.java:303)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.store.schema.Schema$BaseBuilder.json(Schema.java:472)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.store.schema.Schema.fromJson(Schema.java:116)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.accumulostore.key.impl.AggregatorIterator.init(AggregatorIterator.java:91)
2024-02-14 09:29:53     at org.apache.accumulo.core.conf.IterConfigUtil.loadIterators(IterConfigUtil.java:227)
2024-02-14 09:29:53     at org.apache.accumulo.tserver.tablet.ScanDataSource.createIterator(ScanDataSource.java:248)
2024-02-14 09:29:53     at org.apache.accumulo.tserver.tablet.ScanDataSource.iterator(ScanDataSource.java:140)
2024-02-14 09:29:53     at org.apache.accumulo.core.iterators.system.SourceSwitchingIterator.seek(SourceSwitchingIterator.java:226)
2024-02-14 09:29:53     at org.apache.accumulo.tserver.tablet.Tablet.lookup(Tablet.java:614)
2024-02-14 09:29:53     at org.apache.accumulo.tserver.tablet.Tablet.lookup(Tablet.java:770)
2024-02-14 09:29:53     at org.apache.accumulo.tserver.scan.LookupTask.run(LookupTask.java:116)
2024-02-14 09:29:53     at org.apache.accumulo.tserver.session.ScanSession$ScanMeasurer.run(ScanSession.java:54)
2024-02-14 09:29:53     at org.apache.htrace.wrappers.TraceRunnable.run(TraceRunnable.java:57)
2024-02-14 09:29:53     ... 4 more
2024-02-14 09:29:53 Caused by: uk.gov.gchq.gaffer.exception.SerialisationException: org.apache.datasketches.memory.Memory
2024-02-14 09:29:53  at [Source: (byte[])"{"edges":{"JunctionLocatedAt":{"source":"junction","destination":"coordinates","directed":"true"},"RoadUse":{"source":"junction","destination":"junction","directed":"true","properties":{"startDate":"date.earliest","endDate":"date.latest","count":"count.long","countByVehicleType":"counts.freqmap"},"groupBy":["startDate","endDate"]},"RoadHasJunction":{"source":"road","destination":"junction","directed":"true"},"RegionContainsLocation":{"source":"region","destination":"location","directed":"true"},"[truncated 2932 bytes]; line: 1, column: 2765] (through reference chain: uk.gov.gchq.gaffer.store.schema.Schema$Builder["types"]->java.util.LinkedHashMap["hllp"]->uk.gov.gchq.gaffer.store.schema.TypeDefinition["class"])
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.jsonserialisation.JSONSerialiser.deserialise(JSONSerialiser.java:334)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.data.elementdefinition.ElementDefinitions$BaseBuilder.json(ElementDefinitions.java:320)
2024-02-14 09:29:53     ... 17 more
2024-02-14 09:29:53 Caused by: hidden.com.fasterxml.jackson.databind.JsonMappingException: org.apache.datasketches.memory.Memory
2024-02-14 09:29:53  at [Source: (byte[])"{"edges":{"JunctionLocatedAt":{"source":"junction","destination":"coordinates","directed":"true"},"RoadUse":{"source":"junction","destination":"junction","directed":"true","properties":{"startDate":"date.earliest","endDate":"date.latest","count":"count.long","countByVehicleType":"counts.freqmap"},"groupBy":["startDate","endDate"]},"RoadHasJunction":{"source":"road","destination":"junction","directed":"true"},"RegionContainsLocation":{"source":"region","destination":"location","directed":"true"},"[truncated 2932 bytes]; line: 1, column: 2765] (through reference chain: uk.gov.gchq.gaffer.store.schema.Schema$Builder["types"]->java.util.LinkedHashMap["hllp"]->uk.gov.gchq.gaffer.store.schema.TypeDefinition["class"])
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:276)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:623)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:611)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:143)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:314)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:609)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:437)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:32)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:158)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.vanillaDeserialize(BuilderBasedDeserializer.java:293)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:217)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4674)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3690)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.jsonserialisation.JSONSerialiser.deserialise(JSONSerialiser.java:332)
2024-02-14 09:29:53     ... 18 more
2024-02-14 09:29:53 Caused by: java.lang.ClassNotFoundException: org.apache.datasketches.memory.Memory
2024-02-14 09:29:53     at org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:148)
2024-02-14 09:29:53     at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
2024-02-14 09:29:53     at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
2024-02-14 09:29:53     at java.lang.Class.forName0(Native Method)
2024-02-14 09:29:53     at java.lang.Class.forName(Class.java:264)
2024-02-14 09:29:53     at uk.gov.gchq.gaffer.store.schema.TypeDefinition.setClassString(TypeDefinition.java:91)
2024-02-14 09:29:53     at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
2024-02-14 09:29:53     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2024-02-14 09:29:53     at java.lang.reflect.Method.invoke(Method.java:498)
2024-02-14 09:29:53     at hidden.com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:141)
2024-02-14 09:29:53     ... 30 more

Additional Information
There are similar regular Accumulo errors which occur when it attempts compaction [tablet.MinorCompactor] WARN : MinC failed (Failed to load element definitions from bytes).

Platform

  • OS: Linux
  • Gaffer Version: 2.1.0
@GCHQDeveloper314 GCHQDeveloper314 added the bug Confirmed or suspected bug label Feb 14, 2024
@GCHQDeveloper314 GCHQDeveloper314 added this to the v2.2.0 milestone Feb 14, 2024
@github-actions github-actions bot mentioned this issue Feb 14, 2024
Merged
GCHQDeveloper314 added a commit that referenced this issue Feb 15, 2024
@GCHQDeveloper314
Gh-3159: Datasketches library missing from sketches deploy JAR (#3160)
6d291f6 
* Add missing datasketches library to includes for deploy JAR

* Use wildcard to include datasketches memory dependency
@GCHQDeveloper314 GCHQDeveloper314 self-assigned this Feb 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@GCHQDeveloper314 GCHQDeveloper314

Labels
bug Confirmed or suspected bug
Projects
None yet
Milestone
v2.2.0
Development

Successfully merging a pull request may close this issue.

Gh-3159: Datasketches library missing from sketches deploy JAR gchq/Gaffer
1 participant
@GCHQDeveloper314