Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

使用最新的1.8.2的安装包,k8s部署,默认部署类型看着日志目录的权限不够 #3818

Closed
ghost opened this issue Jul 10, 2021 · 11 comments
Labels

Comments

@ghost
Copy link

ghost commented Jul 10, 2021

使用最新的1.8.2的安装包,k8s部署,默认部署类型看着日志目录的权限不够,请问这如何解决,
kubectl get po,svc -n sre
NAME READY STATUS RESTARTS AGE
pod/apollo-mysql-master-0 1/1 Running 0 37m
pod/statefulset-apollo-config-server-dev-0 1/1 Running 0 20s
pod/statefulset-apollo-config-server-dev-1 1/1 Running 0 18s
pod/statefulset-apollo-config-server-dev-2 1/1 Running 0 12s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/apollo-mysql NodePort 10.24.29.247 3306:30070/TCP 38m
service/apollo-mysql-slave NodePort 10.24.244.122 3306:30071/TCP 38m
service/service-apollo-config-server-dev NodePort 10.24.88.162 8080:30002/TCP 20s
service/service-apollo-meta-server-dev ClusterIP None 8080/TCP 20s
[root@nicetuan-op-cicd-k8s-prod-001 apollo-env-dev]# kubectl logs statefulset-apollo-config-server-dev-0 -n sre
Sat Jul 10 18:34:07 CST 2021 ==== Starting ====
LOG_FOLDER /opt/logs/apollo-config-server does not exist. Falling back to /tmp
Application is running as root (UID 0). This is considered insecure.
Started [51]

@nobodyiam
Copy link
Member

startup-kubernetes.sh 没有自动创建目录,可以尝试按照下面的 startup.sh 脚本改一下试试。
另外,现在 apollo 的镜像已经上传了,所以应该没有必要自己构建镜像了,这块的文档得更新下。

https://github.com/ctripcorp/apollo/blob/72a04989d049e182dd409e2a76e5c0e4448a8a2d/apollo-configservice/src/main/scripts/startup.sh#L23-L25

@ghost
Copy link
Author

ghost commented Jul 12, 2021

目前我增加了上面的配置,没有报错信息了,但是查看log

# kubectl logs statefulset-apollo-config-server-dev-2 -n sre
Mon Jul 12 11:31:40 CST 2021 ==== Starting ====
Application is running as root (UID 0). This is considered insecure.
Started [52]

看着没有报错,我查看是否可以访问,但是运气不好,访问不到

# kubectl get svc -n sre
NAME                               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
apollo-mysql                       NodePort    10.24.29.247    <none>        3306:30070/TCP   45h
apollo-mysql-slave                 NodePort    10.24.244.122   <none>        3306:30071/TCP   45h
service-apollo-config-server-dev   NodePort    10.24.252.201   <none>        8080:30002/TCP   3h33m
service-apollo-meta-server-dev     ClusterIP   None            <none>        8080/TCP         3h33m
[root@nicetuan-op-cicd-k8s-prod-001 apollo-config-server]# curl 10.24.252.201:8080
curl: (7) Failed connect to 10.24.252.201:8080; Connection refused

@ghost
Copy link
Author

ghost commented Jul 12, 2021

我找到对应的维护的地址了,现在可以启动了,我使用的但是,我发现连不上mysql数据库
https://hub.docker.com/r/apolloconfig/apollo-configservice/tags?page=1&ordering=last_updated
这是下面的报错

2021-07-12 15:13:00.297  INFO 1 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'methodSecurityMetadataSource' of type [org.springframework.security.access.method.DelegatingMethodSecurityMetadataSource] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-07-12 15:13:00.603  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
2021-07-12 15:13:00.614  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2021-07-12 15:13:00.615  INFO 1 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.41]
2021-07-12 15:13:00.662  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2021-07-12 15:13:00.662  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 3262 ms
2021-07-12 15:13:00.928  INFO 1 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2021-07-12 15:13:02.083 ERROR 1 --- [           main] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Exception during pool initialization.

com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
	at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174)
	at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64)
	at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:835)
	at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:455)
	at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:240)
	at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:199)
	at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
	at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:358)
	at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
	at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:477)
	at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:560)

我通过svc的地址尝试连接mysql,看起来没有问题

# telnet 10.24.29.247 3306
Trying 10.24.29.247...
Connected to 10.24.29.247.
Escape character is '^]'.
J
8.0.212AX5�!y	lL='&EKmysql_native_password
^]
telnet> quit

最后我的configmap的地址如下,我安装之前的yaml地址更改去连接mysql的svc的地址,但是似乎没有发现任何数据包,连不上

kind: ConfigMap
apiVersion: v1
metadata:
  namespace: sre
  name: configmap-apollo-admin-server-dev
data:
  application-github.properties: |
    spring.datasource.url = jdbc:mysql://apollo-mysql.sre:3306/DevApolloConfigDB?characterEncoding=utf8
    spring.datasource.username = root
    spring.datasource.password = root

@nobodyiam
Copy link
Member

看报错是连不上,试一下到容器里面 telnet 域名 3306 是否正常吧

@ghost
Copy link
Author

ghost commented Jul 16, 2021

我先后尝试了是否可以telnet,但是这个容器没有起来,我使用command - sh - - c - /dev/null,尝试让容器一直运行,但是里面没有telnet命令,后面我怀疑是mysql的授权问题,我准备用ecs的实例单独创建了mysql,分别尝试5.6 、5.7、8.0+的mysql及mariadb,并授权可以访问的库及用户,我尝试使用你们提供的外部的svc的mysql,给出的svc地址,我尝试去telnet svc的mysq地址,默认可以通的,但是在apollo-config里面通过jdbc的方式去连接确还是报错 The driver has not received any packets from the server

@ghost
Copy link
Author

ghost commented Jul 16, 2021

对此非常疑惑,这块你们是否有提供比较稳定的版本及文档的k8s上apollo的方式吗?如果有的话,表示感谢

@nobodyiam
Copy link
Member

建议使用 helm chart 方式一键部署,不过我觉得连不上数据库和 apollo 的部署步骤应该没啥关系

@ghost
Copy link
Author

ghost commented Aug 3, 2021

使用helm部署之后,发现没有其他的问题了,但是有个疑问,默认部署起来只有dev的环境,如果是部署其他的环境,根据原有的配置进行修改就可以了是吗,另外里面没有eureka的连接地址,这个eureka是非必需要的应用必需集成的吗?当然我也尝试进行增加eureka的地址到configserver和adminserver当中,但是似乎eureka没有注册到信息,也没有什么异常的报错信息,这里有点疑惑,eureka在apollo当中扮演了什么角色,重要吗?

@nobodyiam
Copy link
Member

使用 helm chart 部署,会直接使用 k8s 自己的服务发现,不需要 eureka 了,详见 #3054

部署多环境的话,需要为每个环境部署一套 apollo-service,然后在 portal 的 helm chart 中更新环境相关信息即可,详见2.4.1.4.4.4 配置环境信息

@stale
Copy link

stale bot commented Oct 9, 2021

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 7 days unless it is tagged "help wanted" or other activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Oct 9, 2021
@stale
Copy link

stale bot commented Oct 16, 2021

This issue has been automatically closed because it has not had activity in the last 7 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted". Thank you for your contributions.

@stale stale bot closed this as completed Oct 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant