Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: PL debugging error: call dbms_debug.attach_session() failed #280

Closed
smallsheeeep opened this issue Sep 8, 2023 · 2 comments · Fixed by #254
Closed

[Bug]: PL debugging error: call dbms_debug.attach_session() failed #280

smallsheeeep opened this issue Sep 8, 2023 · 2 comments · Fixed by #254
Assignees
Milestone

Comments

@smallsheeeep
Copy link
Contributor

smallsheeeep commented Sep 8, 2023

ODC version

4.2.0

OB version

4.2

What happened?

ODC debugging PL, a single node is abnormal. After observer upgraded the repair package (4.2.0.0-100020012023090710), the problem of debugging automatically jumping out without reporting an exception has not recurred. However, the above problem will occur when debugging directly connected to one of the nodes (10.xxx.xxx.33). Directly connected to another one (10.xxx.xxx.30) will not.

What did you expect to happen?

Start PL debug successfully

How can we reproduce it (as minimally and precisely as possible)?

Anything else we need to know?

image image

OBServer 日志:

[2023-09-08 17:43:24.550605] WDIAG [SQL.RESV] get_package_id (ob_schema_checker.cpp:1529) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=13][errcode=-5559] package is not exist(tenant_id=1004, database_id=500004, package_name=DBMS_DEBUG, ret=-5559)
[2023-09-08 17:43:24.550652] WDIAG [SQL.RESV] get_package_id (ob_schema_checker.cpp:1529) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=12][errcode=-5559] package is not exist(tenant_id=1004, database_id=201001, package_name=DBMS_DEBUG, ret=-5559)
[2023-09-08 17:43:24.550675] INFO  [SQL.RESV] get_routine (ob_resolver_utils.cpp:1429) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=4] call ObResolverUtils::get_routine fit routine(db_name=oceanbase, package_name=DBMS_DEBUG, routine_name=ATTACH_SESSION, routine={tenant_id:1, database_id:201001, package_id:310137, owner_id:200001, routine_id:310145, routine_name:"ATTACH_SESSION", overload:0, subprogram_id:8, schema_version:1692003195352384, routine_type:3, flag:9437184, priv_user:"", comp_flag:0, exec_env:"2151677954,45,46,46,", routine_body:"PROCEDURE attach_session(debug_session_id  IN VARCHAR2,
[2023-09-08 17:43:24.550815] WDIAG [PL.DEBUG] get (ob_pl_debugger_manager.cpp:176) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=29][errcode=-4018] failed to fetch debugger from id, debugger not exist(ret=-4018, id=8)
[2023-09-08 17:43:24.550822] WDIAG [PL.DEBUG] get (ob_pl_debugger_manager.cpp:208) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=7][errcode=-4018] fail to exec ObPDBManager::get_instance()->get(id_, pl_debugger_)(ret=-4018)
[2023-09-08 17:43:24.550827] WDIAG [PL] attach_session (ob_dbms_debug.cpp:220) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=3][errcode=-4018] fail to exec guard.get(pl_debugger)(ret=-4018)
[2023-09-08 17:43:24.550839] WDIAG [PL] eh_adjust_call_stack (ob_pl_exception_handling.cpp:176) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=4][errcode=0] got exception! (error_code=-4018, error_code="OB_ENTRY_NOT_EXIST", exception_msg=
[2023-09-08 17:43:24.551831] WDIAG [PL] execute (ob_pl.cpp:2037) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=6][errcode=0] failed to execute it(ret=0, package_id=310164, routine_id=35, subprogram_path=[])
[2023-09-08 17:43:24.552685] WDIAG [PL] execute (ob_pl.cpp:3703) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=7][errcode=-4018] Unhandled exception has occurred in PL(*ctx_.status_=-4018, ret=-4018)
[2023-09-08 17:43:24.552693] WDIAG [PL] execute (ob_pl.cpp:1426) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=6][errcode=-4018] fail to exec pl.execute()(ret=-4018)
[2023-09-08 17:43:24.552697] WDIAG [PL] execute (ob_pl.cpp:2035) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=4][errcode=-4018] fail to exec execute(ctx, allocator, *(ctx.get_package_guard()), *routine, &params, ((0 == nocopy_params.count()) ? __null : &nocopy_params), &result, status, stack_ctx.is_top_stack(), inner_call, routine->is_function() || in_function, false, loc, is_called_from_sql)(ret=-4018)
[2023-09-08 17:43:24.552717] WDIAG [SQL.ENG] execute (ob_routine_executor.cpp:206) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=4][errcode=-4018] failed to execute pl(package_id=310137, routine_id=8, ret=-4018, pkg_id=310137)
[2023-09-08 17:43:24.552723] WDIAG [SQL] open_cmd (ob_result_set.cpp:98) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=6][errcode=-4018] execute cmd failed(ret=-4018)
[2023-09-08 17:43:24.552727] WDIAG [SQL] open (ob_result_set.cpp:157) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=4][errcode=-4018] execute plan failed(ret=-4018)
[2023-09-08 17:43:24.552731] WDIAG [SERVER] response_result (ob_sync_cmd_driver.cpp:126) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=2][errcode=-4018] close result set fail(cret=-4018)
[2023-09-08 17:43:24.552737] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:914) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=4][errcode=-4018] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:0, local_retry_times:0, err_:-4018, err_:"OB_ENTRY_NOT_EXIST", retry_type:0, client_ret:-4018}, need_retry=false)
[2023-09-08 17:43:24.552745] WDIAG [SERVER] response_result (ob_sync_cmd_driver.cpp:132) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=8][errcode=-4018] result set open failed, check if need retry(ret=-4018, cli_ret=-4018, retry_ctrl_.need_retry()=0)
[2023-09-08 17:43:24.552757] INFO  [SERVER] send_error_packet (obmp_packet_sender.cpp:323) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=5] sending error packet(ob_error=-4018, client error=600, extra_err_info=NULL, lbt()="0x11764d1c 0x9a2da6e 0x99ee4bf 0x4f026d5 0x4c296ef 0x4c20066 0x123de4ef 0x4c157b5 0x4c0a581 0x987c8e4 0x11aa75f0 0x7fbe579b7ea5 0x7fbe57476b0d")
[2023-09-08 17:43:24.552775] WDIAG [SERVER] do_process (obmp_query.cpp:827) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=6][errcode=-4018] execute query fail(ret=-4018, timeout_timestamp=4910838204550554)
[2023-09-08 17:43:24.552791] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=4][errcode=0] set logical del time(cache_obj->get_logical_del_time()=11909770241164, cache_obj->added_lc()=false, cache_obj->get_object_id()=36929, cache_obj->get_tenant_id()=1004, lbt()="0x11764d1c 0xa27b158 0xa27ae80 0xa1f2c20 0xd099230 0x4ed3b15 0x4c9b69d 0x4c2b6e4 0x4c28ccd 0x4c20066 0x123de4ef 0x4c157b5 0x4c0a581 0x987c8e4 0x11aa75f0 0x7fbe579b7ea5 0x7fbe57476b0d")
[2023-09-08 17:43:24.552797] INFO  [SQL.PC] before_cache_evicted (ob_i_lib_cache_object.cpp:76) [93133][T1004_L0_G0][T1004][YB420A15FC21-000604C399D2FD67-0-0] [lt=6] before_cache_evicted(this=0x7faf077cc050, this={can_direct_use_param:false, package_id:310137, routine_id:8, out_idx:[], out_name:[], out_type:[], out_type_name:[], out_type_owner:[], is_udt_routine:false})

ODC 日志:

[2023-09-08 17:43:24.493][http-nio-8989-exec-38][cbafca2026004b16,,QTZOFX4O31F0H60W3Z70PLN][WARN][com.oceanbase.odc.service.pldebug.operator.OraclePLOperator][69]: CALL DBMS_DEBUG.PING() occur error StatementCallback; SQL [CALL DBMS_DEBUG.PING()]; (conn=3221864035) ORA-00600: internal error code, arguments: -4018, Entry not exist
[10.21.252.33:2882] [2023-09-08 17:43:24.493725] [YB420A15FC21-000604C39E72B22B-0-0]
at oceanbase.DBMS_DEBUG.PING , line : 292, col : 1; nested exception is java.sql.SQLTransientConnectionException: (conn=3221864035) ORA-00600: internal error code, arguments: -4018, Entry not exist
[10.21.252.33:2882] [2023-09-08 17:43:24.493725] [YB420A15FC21-000604C39E72B22B-0-0]
at oceanbase.DBMS_DEBUG.PING , line : 292, col : 1
[2023-09-08 17:43:24.523][http-nio-8989-exec-38][cbafca2026004b16,,QTZOFX4O31F0H60W3Z70PLN][INFO][com.oceanbase.odc.service.pldebug.PLDebugService][156]: Created OBODC_PL_DEBUG_PACKAGE package successfully
[2023-09-08 17:43:24.546][PLDebug_Thread_1466503260][08b58ae0224c48df,,3BMBQWOXJ8N7BN5WMI9E][INFO][com.oceanbase.odc.service.pldebug.util.PLDebugTask][61]: Start to run debug, debugId=8
[2023-09-08 17:43:24.552][http-nio-8989-exec-38][cbafca2026004b16,,QTZOFX4O31F0H60W3Z70PLN][WARN][com.oceanbase.odc.service.pldebug.session.DebuggerSession][116]: Call dbms_debug.attach_session() failed
java.sql.SQLTransientConnectionException: (conn=3221868582) ORA-00600: internal error code, arguments: -4018, Entry not exist
[10.21.252.33:2882] [2023-09-08 17:43:24.552763] [YB420A15FC21-000604C399D2FD67-0-0]
at package body oceanbase.DBMS_DEBUG.INNER_ATTACH_SESSION , line : 36, col : 1
at oceanbase.DBMS_DEBUG.ATTACH_SESSION , line : 129, col : 0
	at com.oceanbase.jdbc.internal.util.exceptions.ExceptionFactory.createException(ExceptionFactory.java:128) ~[oceanbase-client-2.4.3.jar!/:?]
	at com.oceanbase.jdbc.internal.util.exceptions.ExceptionFactory.create(ExceptionFactory.java:202) ~[oceanbase-client-2.4.3.jar!/:?]
	at com.oceanbase.jdbc.OceanBaseStatement.executeExceptionEpilogue(OceanBaseStatement.java:315) ~[oceanbase-client-2.4.3.jar!/:?]
	at com.oceanbase.jdbc.OceanBaseStatement.executeInternal(OceanBaseStatement.java:488) ~[oceanbase-client-2.4.3.jar!/:?]
	at com.oceanbase.jdbc.OceanBaseStatement.execute(OceanBaseStatement.java:631) ~[oceanbase-client-2.4.3.jar!/:?]
	at com.oceanbase.odc.service.pldebug.session.DebuggerSession.<init>(DebuggerSession.java:114) [odc-service-4.2.0-20230818.jar!/:?]
	at com.oceanbase.odc.service.pldebug.session.PLDebugSession.start(PLDebugSession.java:106) [odc-service-4.2.0-20230818.jar!/:?]
	at com.oceanbase.odc.service.pldebug.PLDebugService.start(PLDebugService.java:165) [odc-service-4.2.0-20230818.jar!/:?]
	at com.oceanbase.odc.server.web.controller.v2.PLDebugController.start(PLDebugController.java:59) [classes!/:?]
	at com.oceanbase.odc.server.web.controller.v2.PLDebugController$$FastClassBySpringCGLIB$$923bdc9a.invoke(<generated>) [classes!/:?]
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) [spring-core-5.3.26.jar!/:5.3.26]
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793) [spring-aop-5.3.27.jar!/:5.3.27]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.3.27.jar!/:5.3.27]
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) [spring-aop-5.3.27.jar!/:5.3.27]
	at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89) [spring-aop-5.3.27.jar!/:5.3.27]
	at com.oceanbase.odc.service.audit.AuditEventAspect.aroundEventAudit(AuditEventAspect.java:146) [odc-service-4.2.0-20230818.jar!/:?]
	at sun.reflect.GeneratedMethodAccessor231.invoke(Unknown Source) ~[?:?]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_312]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_312]
	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634) [spring-aop-5.3.27.jar!/:5.3.27]
	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624) [spring-aop-5.3.27.jar!/:5.3.27]
	at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72) [spring-aop-5.3.27.jar!/:5.3.27]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) [spring-aop-5.3.27.jar!/:5.3.27]
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) [spring-aop-5.3.27.jar!/:5.3.27]
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) [spring-aop-5.3.27.jar!/:5.3.27]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) [spring-aop-5.3.27.jar!/:5.3.27]
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) [spring-aop-5.3.27.jar!/:5.3.27]
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708) [spring-aop-5.3.27.jar!/:5.3.27]
	at com.oceanbase.odc.server.web.controller.v2.PLDebugController$$EnhancerBySpringCGLIB$$f84e8184.start(<generated>) [classes!/:?]
	at sun.reflect.GeneratedMethodAccessor2751.invoke(Unknown Source) ~[?:?]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_312]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_312]
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) [spring-web-5.3.26.jar!/:5.3.26]
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150) [spring-web-5.3.26.jar!/:5.3.26]
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117) [spring-webmvc-5.3.26.jar!/:5.3.26]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895) [spring-webmvc-5.3.26.jar!/:5.3.26]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808) [spring-webmvc-5.3.26.jar!/:5.3.26]
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) [spring-webmvc-5.3.26.jar!/:5.3.26]
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1072) [spring-webmvc-5.3.26.jar!/:5.3.26]
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:965) [spring-webmvc-5.3.26.jar!/:5.3.26]
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) [spring-webmvc-5.3.26.jar!/:5.3.26]
	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909) [spring-webmvc-5.3.26.jar!/:5.3.26]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:681) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) [spring-webmvc-5.3.26.jar!/:5.3.26]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:764) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) [tomcat-embed-websocket-9.0.62.jar!/:?]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) [tomcat-embed-core-9.0.62.jar!/:?]
	at com.oceanbase.odc.server.web.filter.RequestDenyFilter.doFilterInternal(RequestDenyFilter.java:57) [classes!/:?]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) [spring-web-5.3.26.jar!/:5.3.26]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) [tomcat-embed-core-9.0.62.jar!/:?]
	at com.oceanbase.odc.server.web.trace.RequestBodyCopyFilter.doFilterInternal(RequestBodyCopyFilter.java:53) [classes!/:?]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) [spring-web-5.3.26.jar!/:5.3.26]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) [tomcat-embed-core-9.0.62.jar!/:?]
	at com.oceanbase.odc.server.web.trace.RequestIdFilter.doFilterInternal(RequestIdFilter.java:56) [classes!/:?]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) [spring-web-5.3.26.jar!/:5.3.26]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:337) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:122) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:116) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:109) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:223) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:217) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:223) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:217) [spring-security-web-5.7.5.jar!/:5.7.5]
	at com.oceanbase.odc.service.iam.auth.CustomUsernamePasswordAuthenticationFilter.doFilter(CustomUsernamePasswordAuthenticationFilter.java:77) [odc-service-4.2.0-20230818.jar!/:?]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:223) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:217) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) [spring-security-web-5.7.5.jar!/:5.7.5]
	at com.oceanbase.odc.service.iam.auth.local.TestLoginAuthenticationFilter.doFilterInternal(TestLoginAuthenticationFilter.java:46) [obcloud-starter-4.2.0-20230818.jar:?]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) [spring-web-5.3.26.jar!/:5.3.26]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.oauth2.client.web.OAuth2AuthorizationRequestRedirectFilter.doFilterInternal(OAuth2AuthorizationRequestRedirectFilter.java:178) [spring-security-oauth2-client-5.7.5.jar!/:5.7.5]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) [spring-web-5.3.26.jar!/:5.3.26]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.csrf.CsrfFilter.doFilterInternal(CsrfFilter.java:132) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) [spring-web-5.3.26.jar!/:5.3.26]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) [spring-web-5.3.26.jar!/:5.3.26]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:112) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:82) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) [spring-web-5.3.26.jar!/:5.3.26]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.session.DisableEncodeUrlFilter.doFilterInternal(DisableEncodeUrlFilter.java:42) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) [spring-web-5.3.26.jar!/:5.3.26]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:221) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186) [spring-security-web-5.7.5.jar!/:5.7.5]
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) [spring-web-5.3.26.jar!/:5.3.26]
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) [spring-web-5.3.26.jar!/:5.3.26]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) [spring-web-5.3.26.jar!/:5.3.26]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) [spring-web-5.3.26.jar!/:5.3.26]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.springframework.session.web.http.SessionRepositoryFilter.doFilterInternal(SessionRepositoryFilter.java:141) [spring-session-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
	at org.springframework.session.web.http.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:82) [spring-session-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) [spring-web-5.3.26.jar!/:5.3.26]
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) [spring-web-5.3.26.jar!/:5.3.26]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) [spring-web-5.3.26.jar!/:5.3.26]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) [spring-web-5.3.26.jar!/:5.3.26]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) [tomcat-embed-core-9.0.62.jar!/:?]
	at com.oceanbase.odc.server.web.filter.ExceptionHandlerFilter.doFilterInternal(ExceptionHandlerFilter.java:48) [classes!/:?]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) [spring-web-5.3.26.jar!/:5.3.26]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) [tomcat-embed-core-9.0.62.jar!/:?]
	at com.oceanbase.odc.server.web.filter.LocaleFilter.doFilter(LocaleFilter.java:54) [classes!/:?]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:890) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1743) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) [tomcat-embed-core-9.0.62.jar!/:?]
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-9.0.62.jar!/:?]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]
Caused by: com.oceanbase.jdbc.internal.util.exceptions.OceanBaseSqlException: ORA-00600: internal error code, arguments: -4018, Entry not exist
[10.21.252.33:2882] [2023-09-08 17:43:24.552763] [YB420A15FC21-000604C399D2FD67-0-0]
at package body oceanbase.DBMS_DEBUG.INNER_ATTACH_SESSION , line : 36, col : 1
at oceanbase.DBMS_DEBUG.ATTACH_SESSION , line : 129, col : 0
	at com.oceanbase.jdbc.internal.util.exceptions.OceanBaseSqlException.of(OceanBaseSqlException.java:79) ~[oceanbase-client-2.4.3.jar!/:?]
	at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.exceptionWithQuery(AbstractQueryProtocol.java:199) ~[oceanbase-client-2.4.3.jar!/:?]
	at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:312) ~[oceanbase-client-2.4.3.jar!/:?]
	at com.oceanbase.jdbc.OceanBaseStatement.executeInternal(OceanBaseStatement.java:482) ~[oceanbase-client-2.4.3.jar!/:?]
	... 152 more
Caused by: java.sql.SQLException: ORA-00600: internal error code, arguments: -4018, Entry not exist
[10.21.252.33:2882] [2023-09-08 17:43:24.552763] [YB420A15FC21-000604C399D2FD67-0-0]
at package body oceanbase.DBMS_DEBUG.INNER_ATTACH_SESSION , line : 36, col : 1
at oceanbase.DBMS_DEBUG.ATTACH_SESSION , line : 129, col : 0
	at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.readErrorPacket(AbstractQueryProtocol.java:2192) ~[oceanbase-client-2.4.3.jar!/:?]
	at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.readPacket(AbstractQueryProtocol.java:2057) ~[oceanbase-client-2.4.3.jar!/:?]
	at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:1951) ~[oceanbase-client-2.4.3.jar!/:?]
	at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:307) ~[oceanbase-client-2.4.3.jar!/:?]
	at com.oceanbase.jdbc.OceanBaseStatement.executeInternal(OceanBaseStatement.java:482) ~[oceanbase-client-2.4.3.jar!/:?]
	... 152 more
[2023-09-08 17:43:24.566][http-nio-8989-exec-38][cbafca2026004b16,,QTZOFX4O31F0H60W3Z70PLN][INFO][com.oceanbase.odc.common.concurrent.ExecutorUtils][39]: executor debugPingScheduleExecutor terminated success
[2023-09-08 17:43:24.568][http-nio-8989-exec-38][cbafca2026004b16,,QTZOFX4O31F0H60W3Z70PLN][INFO][com.oceanbase.odc.server.web.controller.RestExceptionHandler][375]: ODC_FAILED_REQUEST, httpMethod=POST, fullURL=http://10.16.64.108:8989/api/v2/pldebug/sessions/?currentOrganizationId=1, remoteHost=10.74.33.71, clientAddress=10.74.33.71,  handler=com.oceanbase.odc.server.web.controller.v2.PLDebugController#start[1 args], exceptionType=OBException, message=Call dbms_debug.attach_session() failed, reason=(conn=3221868582) ORA-00600: internal error code, arguments: -4018, Entry not exist
[10.21.252.33:2882] [2023-09-08 17:43:24.552763] [YB420A15FC21-000604C399D2FD67-0-0]
at package body oceanbase.DBMS_DEBUG.INNER_ATTACH_SESSION , line : 36, col : 1
at oceanbase.DBMS_DEBUG.ATTACH_SESSION , line : 129, col : 0, rootReason= 	at com.oceanbase.odc.service.pldebug.session.DebuggerSession.<init>(DebuggerSession.java:118) 	at com.oceanbase.odc.core.shared.exception.OBException.executeFailed(OBException.java:69) at oceanbase.DBMS_DEBUG.ATTACH_SESSION , line : 129, col : 0 at package body oceanbase.DBMS_DEBUG.INNER_ATTACH_SESSION , line : 36, col : 1 com.oceanbase.odc.core.shared.exception.OBException: 执行 SQL 失败,错误详情:Call dbms_debug.attach_session() failed, reason=(conn=3221868582) ORA-00600: internal error code, arguments: -4018, Entry not exist
[10.21.252.33:2882] [2023-09-08 17:43:24.552763] [YB420A15FC21-000604C399D2FD67-0-0]
at package body oceanbase.DBMS_DEBUG.INNER_ATTACH_SESSION , line : 36, col : 1
at oceanbase.DBMS_DEBUG.ATTACH_SESSION , line : 129, col : 0, perfLevel=P2, response=ErrorResponse(error=Error(code=ObExecuteSqlFailed, message=执行 SQL 失败,错误详情:Call dbms_debug.attach_session() failed, reason=(conn=3221868582) ORA-00600: internal error code, arguments: -4018, Entry not exist
[10.21.252.33:2882] [2023-09-08 17:43:24.552763] [YB420A15FC21-000604C399D2FD67-0-0]
at package body oceanbase.DBMS_DEBUG.INNER_ATTACH_SESSION , line : 36, col : 1
at oceanbase.DBMS_DEBUG.ATTACH_SESSION , line : 129, col : 0, details=[Error.Detail(code=ObExecuteSqlFailed, target=null, message=Call dbms_debug.attach_session() failed, reason=(conn=3221868582) ORA-00600: internal error code, arguments: -4018, Entry not exist
[10.21.252.33:2882] [2023-09-08 17:43:24.552763] [YB420A15FC21-000604C399D2FD67-0-0]
at package body oceanbase.DBMS_DEBUG.INNER_ATTACH_SESSION , line : 36, col : 1
at oceanbase.DBMS_DEBUG.ATTACH_SESSION , line : 129, col : 0), Error.Detail(code=OBException, target=null, message=Call dbms_debug.attach_session() failed, reason=(conn=3221868582) ORA-00600: internal error code, arguments: -4018, Entry not exist
[10.21.252.33:2882] [2023-09-08 17:43:24.552763] [YB420A15FC21-000604C399D2FD67-0-0]
at package body oceanbase.DBMS_DEBUG.INNER_ATTACH_SESSION , line : 36, col : 1
at oceanbase.DBMS_DEBUG.ATTACH_SESSION , line : 129, col : 0)]), code=ObExecuteSqlFailed, message=执行 SQL 失败,错误详情:Call dbms_debug.attach_session() failed, reason=(conn=3221868582) ORA-00600: internal error code, arguments: -4018, Entry not exist
[10.21.252.33:2882] [2023-09-08 17:43:24.552763] [YB420A15FC21-000604C399D2FD67-0-0]
at package body oceanbase.DBMS_DEBUG.INNER_ATTACH_SESSION , line : 36, col : 1
at oceanbase.DBMS_DEBUG.ATTACH_SESSION , line : 129, col : 0)

Cloud

No response

@yhilmare
Copy link
Contributor

yhilmare commented Sep 8, 2023

as you can see in ODC 4.2.0 AbstractDebugSession:

// AbstractDebugSession#acquireNewConnection
protected void acquireNewConnection(ConnectionSession connectionSession, boolean useCurrentSession)
            throws Exception {
        ...
        if (!useCurrentSession) {
            this.newDataSource = acquireDirectConnectDataSource();
        }
        this.jdbcOperations = new JdbcTemplate(this.newDataSource);
        this.connection = newDataSource.getConnection();

    }
// AbstractDebugSession#acquireDirectConnectDataSource
private SingleConnectionDataSource acquireDirectConnectDataSource() {
        List<OdcDBSession> sessions = connectionSession.getSyncJdbcExecutor(ConnectionSessionConstants.CONSOLE_DS_KEY)
                .query("show full processlist", new OdcDBSessionRowMapper());

        ConnectionConfig connectionConfig =
                (ConnectionConfig) ConnectionSessionUtil.getConnectionConfig(connectionSession);

        if (sessions.size() == 0) {
            throw new UnexpectedException("Empty db session list");
        }

        String directServerIp = null;
        for (OdcDBSession odcDbSession : sessions) {
            if (StringUtils.isNotBlank(odcDbSession.getSvrIp())) {
                directServerIp = odcDbSession.getSvrIp();
                break;
            }
        }
        if (StringUtils.isEmpty(directServerIp)) {
            throw new UnexpectedException("Empty direct server ip and port from 'show full processlist'");
        }
        String host = directServerIp.split(":")[0];
        int port = Integer.parseInt(directServerIp.split(":")[1]);

        return acquireDataSource(connectionConfig, host, port, connectionSession);
    }

odc will execute show full processlist and select the first ip as the direct host for DebuggeeSession:

// DebuggeeSession
public DebuggeeSession(ConnectionSession connectionSession, ThreadPoolExecutor debugThreadPoolExecutor,
            StartPLDebugReq req) throws Exception {

        acquireNewConnection(connectionSession, false);
        ...
    }

odc will connect to observer without direct link for DebuggerSession:

// DebuggerSession
public DebuggerSession(DebuggeeSession debuggeeSession, StartPLDebugReq req, boolean syncEnabled)
            throws Exception {
        debugId = debuggeeSession.getDebugId();
        debugType = req.getDebugType();
        ddl = req.getAnonymousBlock();
        this.syncEnabled = syncEnabled;

        acquireNewConnection(debuggeeSession.getConnectionSession(), true);
        ...
    }

therefore, DebugeeSession and DebuggerSession may connect to different observer and leads to this issue~

@yhilmare
Copy link
Contributor

yhilmare commented Sep 8, 2023

this pr may fix this issue #254

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

3 participants