Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to check the column lineage and dataset lineage and got below error need some attention #2921

Open
prabakaran-123 opened this issue Oct 11, 2024 · 11 comments
Open

Unable to check the column lineage and dataset lineage and got below error need some attention #2921

prabakaran-123 opened this issue Oct 11, 2024 · 11 comments
Labels
bug Something isn't working
Milestone
0.51.0

Comments

@prabakaran-123
Copy link

prabakaran-123 commented Oct 11, 2024 •
edited
Loading

Read from oracle : oracle://oracledb.cydlut9zgajr.us-east-1.rds.amazonaws.com:1521

image
image
@boring-cyborg boring-cyborg
Copy link

boring-cyborg bot commented Oct 11, 2024

Thanks for opening your first issue in the Marquez project! Please be sure to follow the issue template!

@phixMe
Copy link
Member

phixMe commented Oct 11, 2024

Hello, Do you happen to have any logs on the server that indicate what your issue is?

@prabakaran-123
Copy link
Author

prabakaran-123 commented Oct 14, 2024 •
edited
Loading

@phixMe Here I have attached the logs
lhost:3000/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36" 17
marquez-api-1 | INFO [2024-10-14 17:51:20,631] marquez.logging.LoggingMdcFilter: status: 200
marquez-api-1 | 172.20.0.4 - - [14/Oct/2024:17:51:20 +0000] "GET /api/v1/namespaces/oracle%3A%2F%2Foracledb.cydlut9zgajr.us-east-1.rds.amazonaws.com%3A1521/jobs?limit=20&offset=0 HTTP/1.1" 200 26 "http://localhost:3000/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36" 11
marquez-api-1 | INFO [2024-10-14 17:51:25,712] marquez.logging.LoggingMdcFilter: status: 200
marquez-api-1 | 172.20.0.4 - - [14/Oct/2024:17:51:25 +0000] "GET /api/v1/namespaces/oracle%3A%2F%2Foracledb.cydlut9zgajr.us-east-1.rds.amazonaws.com%3A1521/datasets?limit=20&offset=0 HTTP/1.1" 200 963 "http://localhost:3000/datasets" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36" 27
marquez-api-1 | ERROR [2024-10-14 17:51:27,057] io.dropwizard.jersey.errors.IllegalStateExceptionMapper: Error handling a request: faef7c3122f24251
marquez-api-1 | ! java.lang.IllegalStateException: No match available
marquez-api-1 | ! at java.base/java.util.regex.Matcher.start(Matcher.java:450)
marquez-api-1 | ! at marquez.service.models.NodeId.parts(NodeId.java:233)
marquez-api-1 | ! at marquez.service.models.NodeId.asDatasetId(NodeId.java:257)
marquez-api-1 | ! at marquez.api.BaseResource.throwIfNotExists(BaseResource.java:161)
marquez-api-1 | ! at marquez.api.OpenLineageResource.getLineage(OpenLineageResource.java:118)
marquez-api-1 | ! at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
marquez-api-1 | ! at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
marquez-api-1 | ! at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
marquez-api-1 | ! at java.base/java.lang.reflect.Method.invoke(Method.java:568)
marquez-api-1 | ! at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
marquez-api-1 | ! at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146)
marquez-api-1 | ! at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189)
marquez-api-1 | ! at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176)
marquez-api-1 | ! at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93)
marquez-api-1 | ! at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478)
marquez-api-1 | ! at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400)
marquez-api-1 | ! at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
marquez-api-1 | ! at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256)
marquez-api-1 | ! at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
marquez-api-1 | ! at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
marquez-api-1 | ! at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
marquez-api-1 | ! at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
marquez-api-1 | ! at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
marquez-api-1 | ! at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
marquez-api-1 | ! at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235)
marquez-api-1 | ! at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684)
marquez-api-1 | ! at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
marquez-api-1 | ! at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)
marquez-api-1 | ! at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:358)
marquez-api-1 | ! at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:311)
marquez-api-1 | ! at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
marquez-api-1 | ! at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
marquez-api-1 | ! at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656)
marquez-api-1 | ! at io.dropwizard.servlets.ThreadNameFilter.doFilter(ThreadNameFilter.java:35)
marquez-api-1 | ! at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
marquez-api-1 | ! at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
marquez-api-1 | ! at io.dropwizard.jersey.filter.AllowedMethodsFilter.handle(AllowedMethodsFilter.java:47)
marquez-api-1 | ! at io.dropwizard.jersey.filter.AllowedMethodsFilter.doFilter(AllowedMethodsFilter.java:41)
marquez-api-1 | ! at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
marquez-api-1 | ! at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
marquez-api-1 | ! at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
marquez-api-1 | ! at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
marquez-api-1 | ! at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:322)
marquez-api-1 | ! at io.dropwizard.jetty.RoutingHandler.handle(RoutingHandler.java:52)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772)
marquez-api-1 | ! at io.dropwizard.jetty.ZipExceptionHandlingGzipHandler.handle(ZipExceptionHandlingGzipHandler.java:26)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:54)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:181)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
marquez-api-1 | ! at org.eclipse.jetty.server.Server.handle(Server.java:516)
marquez-api-1 | ! at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
marquez-api-1 | ! at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
marquez-api-1 | ! at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
marquez-api-1 | ! at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
marquez-api-1 | ! at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
marquez-api-1 | ! at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
marquez-api-1 | ! at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
marquez-api-1 | ! at java.base/java.lang.Thread.run(Thread.java:840)
marquez-api-1 | ERROR [2024-10-14 17:51:27,058] marquez.logging.LoggingMdcFilter: status: 500
marquez-api-1 | 172.20.0.4 - - [14/Oct/2024:17:51:27 +0000] "GET /api/v1/lineage?nodeId=dataset:oracle://oracledb.cydlut9zgajr.us-east-1.rds.amazonaws.com:1521:1521/ORCL.CDC_TEST_01&depth=2 HTTP/1.1" 500 110 "http://localhost:3000/lineage/dataset/oracle%3A%2F%2Foracledb.cydlut9zgajr.us-east-1.rds.amazonaws.com%3A1521/1521%2FORCL.CDC_TEST_01" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36" 13

@prabakaran-123
Copy link
Author

prabakaran-123 commented Oct 14, 2024 •
edited
Loading

@phixMe - need some inside on the below logs!

| ERROR [2024-10-14 17:55:05,492] marquez.logging.LoggingMdcFilter: status: 500
marquez-api-1 | 172.20.0.4 - - [14/Oct/2024:17:55:05 +0000] "GET /api/v1/lineage?nodeId=dataset:oracle://oracledb.cydlut9zgajr.us-east-1.rds.amazonaws.com:1521:1521/test_01_newcol&depth=2 HTTP/1.1" 500 110 "http://localhost:3000/lineage/dataset/oracle%3A%2F%2Foracledb.cydlut9zgajr.us-east-1.rds.amazonaws.com%3A1521/1521%2FORCL.cdc_test_01_newcol" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36" 15
postgres-1 | 2024-10-14 17:55:11.090 UTC [27] LOG: checkpoint starting: time
postgres-1 | 2024-10-14 17:55:22.092 UTC [27] LOG: checkpoint complete: wrote 110 buffers (0.7%); 0 WAL file(s) added, 0 removed, 0 recycled; write=10.943 s, sync=0.043 s, total=11.003 s; sync files=96, longest=0.004 s, average=0.001 s; distance=347 kB, estimate=347 kB; lsn=0/20E7A70, redo lsn=0/20E7A38
marquez-api-1 | INFO [2024-10-14 17:56:27,809] marquez.logging.LoggingMdcFilter: status: 200
marquez-api-1 | 172.20.0.4 - - [14/Oct/2024:17:56:27 +0000] "GET /api/v1/namespaces/oracle%3A%2F%2Foracledb.cydlut9zgajr.us-east-1.rds.amazonaws.com%3A1521/datasets?limit=20&offset=0 HTTP/1.1" 200 963 "http://localhost:3000/datasets" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36" 28
marquez-api-1 | ERROR [2024-10-14 17:56:31,426] io.dropwizard.jersey.errors.IllegalStateExceptionMapper: Error handling a request: 8f2e3a6a39c832f3
marquez-api-1 | ! java.lang.IllegalStateException: No match available
marquez-api-1 | ! at java.base/java.util.regex.Matcher.start(Matcher.java:450)
marquez-api-1 | ! at marquez.service.models.NodeId.parts(NodeId.java:233)
marquez-api-1 | ! at marquez.service.models.NodeId.asDatasetId(NodeId.java:257)
marquez-api-1 | ! at marquez.service.ColumnLineageService.getColumnNodes(ColumnLineageService.java:144)
marquez-api-1 | ! at marquez.service.ColumnLineageService.lineage(ColumnLineageService.java:49)
marquez-api-1 | ! at marquez.api.ColumnLineageResource.getLineage(ColumnLineageResource.java:50)
marquez-api-1 | ! at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
marquez-api-1 | ! at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
marquez-api-1 | ! at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
marquez-api-1 | ! at java.base/java.lang.reflect.Method.invoke(Method.java:568)
marquez-api-1 | ! at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
marquez-api-1 | ! at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146)
marquez-api-1 | ! at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189)
marquez-api-1 | ! at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176)
marquez-api-1 | ! at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93)
marquez-api-1 | ! at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478)
marquez-api-1 | ! at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400)
marquez-api-1 | ! at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
marquez-api-1 | ! at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256)
marquez-api-1 | ! at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
marquez-api-1 | ! at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
marquez-api-1 | ! at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
marquez-api-1 | ! at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
marquez-api-1 | ! at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
marquez-api-1 | ! at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
marquez-api-1 | ! at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235)
marquez-api-1 | ! at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684)
marquez-api-1 | ! at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
marquez-api-1 | ! at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)
marquez-api-1 | ! at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:358)
marquez-api-1 | ! at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:311)
marquez-api-1 | ! at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
marquez-api-1 | ! at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
marquez-api-1 | ! at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656)
marquez-api-1 | ! at io.dropwizard.servlets.ThreadNameFilter.doFilter(ThreadNameFilter.java:35)
marquez-api-1 | ! at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
marquez-api-1 | ! at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
marquez-api-1 | ! at io.dropwizard.jersey.filter.AllowedMethodsFilter.handle(AllowedMethodsFilter.java:47)
marquez-api-1 | ! at io.dropwizard.jersey.filter.AllowedMethodsFilter.doFilter(AllowedMethodsFilter.java:41)
marquez-api-1 | ! at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
marquez-api-1 | ! at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
marquez-api-1 | ! at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
marquez-api-1 | ! at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
marquez-api-1 | ! at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:322)
marquez-api-1 | ! at io.dropwizard.jetty.RoutingHandler.handle(RoutingHandler.java:52)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772)
marquez-api-1 | ! at io.dropwizard.jetty.ZipExceptionHandlingGzipHandler.handle(ZipExceptionHandlingGzipHandler.java:26)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:54)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:181)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
marquez-api-1 | ! at org.eclipse.jetty.server.Server.handle(Server.java:516)
marquez-api-1 | ! at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
marquez-api-1 | ! at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
marquez-api-1 | ! at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
marquez-api-1 | ! at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
marquez-api-1 | ! at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
marquez-api-1 | ! at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
marquez-api-1 | ! at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
marquez-api-1 | ! at java.base/java.lang.Thread.run(Thread.java:840)
marquez-api-1 | ERROR [2024-10-14 17:56:31,426] marquez.logging.LoggingMdcFilter: status: 500
marquez-api-1 | 172.20.0.4 - - [14/Oct/2024:17:56:31 +0000] "GET /api/v1/column-lineage?nodeId=dataset:oracle://oracledb.cydlut9zgajr.us-east-1.rds.amazonaws.com:1521:1521/ORCL.cdc_test_01_newcol&depth=2&withDownstream=true HTTP/1.1" 500 110 "http://localhost:3000/datasets/column-level/oracle%253A%252F%252Foracledb.cydlut9zgajr.us-east-1.rds.amazonaws.com%253A1521/1521%2FORCL.cdc_test_01_newcol" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36" 8

@prabakaran-123
Copy link
Author

prabakaran-123 commented Oct 16, 2024 •
edited
Loading

@phixMe
if the dataset name start with a number its creating issue and unable to view the lineage graph

image

@phixMe
Copy link
Member

phixMe commented Oct 16, 2024

Yes, it looks like you have the same issue as:
#2806

@prabakaran-123
Copy link
Author

@phixMe Yes, is there any solution to fix it!

@phixMe
Copy link
Member

phixMe commented Oct 16, 2024

This is logistically a complicated thing for us to fix because it is everywhere including our docs and website and promotional materials...

Essentially, we are in this difficult position where we cannot parse connection strings and use the : colon character as delimiters. There are a bunch of edge cases we are exposed to for example job:trino://trino-integration-test:1337:1234:more:colons it's really unclear what we consider the namespace we what we consider the job... Fixing this for one user might break it for another.

I think the only way to rectify this is to change the delimiter character for our notational, but it's a breaking change for all kinds of users.

@phixMe
Copy link
Member

phixMe commented Oct 16, 2024

I put together a branch of what this would look like on the api server. it would need to be done on the clients as well of course, and maybe others would prefer a different delimiter.
main...modify/node-id-delimiter

@prabakaran-123
Copy link
Author

@phixMe tried with the above branch which you mentioned:

could you please provide me sample for the node id with delimeter, as I tried end up with below error

ERROR [2024-10-17 15:16:52,766] io.dropwizard.jersey.errors.LoggingExceptionMapper: Error handling a request: 11abd68c1ed35ebb
marquez-api-1 | ! java.lang.UnsupportedOperationException: Expected NodeId of type dataset with 3 parts. Got: dataset&oracle://oracledb.cydlut9zgajr.us-east-1.rds.amazonaws.com:1521:1521/orcl.cdc_test_01_newcol
marquez-api-1 | ! at marquez.service.models.NodeId.parts(NodeId.java:218)
marquez-api-1 | ! at marquez.service.models.NodeId.asDatasetId(NodeId.java:257)
marquez-api-1 | ! at marquez.api.BaseResource.throwIfNotExists(BaseResource.java:161)
marquez-api-1 | ! at marquez.api.OpenLineageResource.getLineage(OpenLineageResource.java:121)
marquez-api-1 | ! at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
marquez-api-1 | ! at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
marquez-api-1 | ! at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
marquez-api-1 | ! at java.base/java.lang.reflect.Method.invoke(Method.java:569)
marquez-api-1 | ! at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
marquez-api-1 | ! at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146)
marquez-api-1 | ! at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189)
marquez-api-1 | ! at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176)
marquez-api-1 | ! at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93)
marquez-api-1 | ! at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478)
marquez-api-1 | ! at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400)
marquez-api-1 | ! at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
marquez-api-1 | ! at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256)
marquez-api-1 | ! at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
marquez-api-1 | ! at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
marquez-api-1 | ! at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
marquez-api-1 | ! at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
marquez-api-1 | ! at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
marquez-api-1 | ! at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
marquez-api-1 | ! at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235)
marquez-api-1 | ! at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684)
marquez-api-1 | ! at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
marquez-api-1 | ! at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)
marquez-api-1 | ! at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:358)
marquez-api-1 | ! at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:311)
marquez-api-1 | ! at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
marquez-api-1 | ! at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
marquez-api-1 | ! at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656)
marquez-api-1 | ! at io.dropwizard.servlets.ThreadNameFilter.doFilter(ThreadNameFilter.java:35)
marquez-api-1 | ! at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
marquez-api-1 | ! at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
marquez-api-1 | ! at io.dropwizard.jersey.filter.AllowedMethodsFilter.handle(AllowedMethodsFilter.java:47)
marquez-api-1 | ! at io.dropwizard.jersey.filter.AllowedMethodsFilter.doFilter(AllowedMethodsFilter.java:41)
marquez-api-1 | ! at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
marquez-api-1 | ! at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
marquez-api-1 | ! at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
marquez-api-1 | ! at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
marquez-api-1 | ! at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:322)
marquez-api-1 | ! at io.dropwizard.jetty.RoutingHandler.handle(RoutingHandler.java:52)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772)
marquez-api-1 | ! at io.dropwizard.jetty.ZipExceptionHandlingGzipHandler.handle(ZipExceptionHandlingGzipHandler.java:26)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:54)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:181)
marquez-api-1 | ! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
marquez-api-1 | ! at org.eclipse.jetty.server.Server.handle(Server.java:516)
marquez-api-1 | ! at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
marquez-api-1 | ! at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
marquez-api-1 | ! at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
marquez-api-1 | ! at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
marquez-api-1 | ! at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
marquez-api-1 | ! at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
marquez-api-1 | ! at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
marquez-api-1 | ! at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
marquez-api-1 | ! at java.base/java.lang.Thread.run(Thread.java:840)

Requesting Url format :
(http://localhost:5000/api/v1/lineage?depth=2&nodeId=dataset%26oracle://oracledb.cydlut9zgajr.us-east-1.rds.amazonaws.com:1521:1521/orcl.cdc_test_01_newcol)

@phixMe
Copy link
Member

phixMe commented Oct 17, 2024

Yes, the UI would also need an update... I think the easiest way for you to get around this is to intercept the OpenLineage events and prepend a string to the job names, or rename the jobs that you are testing with to not have this property.

I don't think we can realistically merge anything here without quite a bit of forethought and communication, so that's probably the best I can recommend for now.

@wslulciuc wslulciuc added the bug Something isn't working label Oct 23, 2024
@wslulciuc wslulciuc added this to the 0.51.0 milestone Oct 23, 2024
@wslulciuc wslulciuc mentioned this issue Oct 24, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
Marquez
Status: No status
Milestone
0.51.0
Development

No branches or pull requests
3 participants
@wslulciuc @phixMe @prabakaran-123