Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] out of memory #1782

Open
8 of 9 tasks
cs50Mu opened this issue Jan 9, 2025 · 7 comments
Open
8 of 9 tasks

[Bug] out of memory #1782

cs50Mu opened this issue Jan 9, 2025 · 7 comments
Labels
bug Something isn't working

Comments

@cs50Mu
Copy link

cs50Mu commented Jan 9, 2025

Verify steps

  • I have read the documentation and understand the meaning of all the configuration items I have written, rather than just piling up seemingly useful options or default values.
  • I have carefully reviewed the documentation and have not resolved the issue.
  • I have searched the Issue Tracker for the issue I want to raise and did not find it.
  • I am a non-Chinese user.
  • I have tested with the latest Alpha branch version, and the issue still persists.
  • I have provided the server and client configuration files and processes that can reproduce the issue locally, rather than a sanitized complex client configuration file.
  • I provided the simplest configuration that can be used to reproduce the errors in my report, rather than relying on remote servers or piling on a lot of unnecessary configurations for reproduction.
  • I have provided complete logs, rather than just the parts I think are useful out of confidence in my own intelligence.
  • I have directly reproduced the error using the Mihomo command-line program, rather than using other tools or scripts.

Operating System

Linux

System Version

Linux RT-AX86U-3E90 4.1.52 #2 SMP PREEMPT Sat Dec 3 14:03:02 EST 2022 aarch64 ASUSWRT-Merlin

Mihomo Version

mihomo-linux-arm64-v1.19.1

Configuration File

# HTTP 代理端口
port: 7890

# SOCKS5 代理端口
socks-port: 7891

# Linux 和 macOS 的 redir 代理端口
redir-port: 7892

# 允许局域网的连接
allow-lan: true

# 规则模式:Rule(规则) / Global(全局代理)/ Direct(全局直连)
mode: Rule

# 设置日志输出级别 (默认级别:silent,即不输出任何内容,以避免因日志内容过大而导致程序内存溢出)。
# 5 个级别:silent / info / warning / error / debug。级别越高日志输出量越大,越倾向于调试,若需要请自行开启。
log-level: error
# Clash 的 RESTful API
external-controller: "0.0.0.0:9090"
experimental:
  ignore-resolve-fail: true
# RESTful API 的口令
secret: ""

Description

it panics due to out of memory after running for almost one day

Reproduction Steps

  1. start the program
  2. wait for one day
  3. it panics due to out of memory

Logs

time="2025-01-07T19:34:19.621770568+08:00" level=info msg="Start initial configuration in progress"
time="2025-01-07T19:34:19.623758325+08:00" level=info msg="Geodata Loader mode: memconservative"
time="2025-01-07T19:34:19.623796306+08:00" level=info msg="Geosite Matcher implementation: succinct"
time="2025-01-07T19:34:19.679491388+08:00" level=info msg="Initial configuration complete, total time: 57ms"
fatal error: runtime: out of memory

runtime stack:
runtime.throw({0xfa4165?, 0x23418?})
	runtime/panic.go:1067 +0x38 fp=0x400199fb30 sp=0x400199fb00 pc=0x82438
runtime.sysMapOS(0x4002000000, 0x400000)
	runtime/mem_linux.go:168 +0x104 fp=0x400199fb70 sp=0x400199fb30 pc=0x239f4
runtime.sysMap(0x4002000000, 0x400000, 0x3a740?)
	runtime/mem.go:155 +0x34 fp=0x400199fb90 sp=0x400199fb70 pc=0x23424
runtime.(*mheap).grow(0x1c13340, 0x4?)
	runtime/mheap.go:1539 +0x278 fp=0x400199fc10 sp=0x400199fb90 pc=0x371d8
runtime.(*mheap).allocSpan(0x1c13340, 0x4, 0x1, 0x0)
	runtime/mheap.go:1244 +0x164 fp=0x400199fcb0 sp=0x400199fc10 pc=0x366d4
runtime.(*mheap).allocManual(0x400199fdb8?, 0x9000000000?, 0xc0?)
	runtime/mheap.go:988 +0x28 fp=0x400199fce0 sp=0x400199fcb0 pc=0x36218
runtime.stackalloc(0x8000)
	runtime/stack.go:408 +0x164 fp=0x400199fd70 sp=0x400199fce0 pc=0x63b04
runtime.copystack(0x4000747c00, 0x8000)
	runtime/stack.go:884 +0x118 fp=0x400199fe70 sp=0x400199fd70 pc=0x649b8
runtime.newstack()
	runtime/stack.go:1126 +0x36c fp=0x400199ffb0 sp=0x400199fe70 pc=0x6500c
runtime.morestack()
	runtime/asm_arm64.s:342 +0x70 fp=0x400199ffb0 sp=0x400199ffb0 pc=0x88860


........
@cs50Mu cs50Mu added the bug Something isn't working label Jan 9, 2025
@cs50Mu
Copy link
Author

cs50Mu commented Jan 9, 2025

clash.log

this is the full log when it crashed, hope it will help to analyze the problem, thanks in advance

@Skyxim
Copy link
Collaborator

Skyxim commented Jan 9, 2025

Check if the memory is exhausted. There is no good solution for memory exhaustion.

@cs50Mu
Copy link
Author

cs50Mu commented Jan 9, 2025

I doubt that it's because there is not enough memory, if so, how to explain that it starts up properly, and running for one day and then panics due to out of memory

@cs50Mu
Copy link
Author

cs50Mu commented Jan 13, 2025

After I reboot the router, the program lives longer(several days till now), whereas before the reboot, the program almost panics as soon as I started it. So it seems like it's because there is not enough memory in deed... I want to verify that further, can you show me the minimum memory requirement of the program? @Skyxim

@Skyxim
Copy link
Collaborator

Skyxim commented Jan 13, 2025

After I reboot the router, the program lives longer(several days till now), whereas before the reboot, the program almost panics as soon as I started it. So it seems like it's because there is not enough memory in deed... I want to verify that further, can you show me the minimum memory requirement of the program? @Skyxim

Memory is related to your configuration. You may need to test how much is needed to start under your own configuration. When there is no connection, the minimum memory. The number of proxy nodes, rules, rule sets, and whether to use GEOIP, etc. These belong to static memory and must be used at startup.

Depending on the number of active connections, TCP/UDP will need a cache. In particular, the QUIC class requires more memory. This part of the memory is the main memory usage.

ps. In my case, I start with about 20MB, and when there are about 30-60 active connections, it takes up about 60-80MB of memory.

@cs50Mu
Copy link
Author

cs50Mu commented Jan 13, 2025

Thanks for the quick response!
I collected the following infos:

this is the memory usage of the system when the program panics:

             total       used       free     shared    buffers     cached
Mem:        933968     708636     225332     109600          0     173148
-/+ buffers/cache:     535488     398480
Swap:            0          0          0

this is the memory usage when the program works fine:

             total       used       free     shared    buffers     cached
Mem:        933968     517864     416104       6956          0      64124
-/+ buffers/cache:     453740     480228
Swap:            0          0          0

this is the memory usage seen from top -m currently with about 300 connections:

Mem total:933968 anon:84308 map:33204 free:416628
 slab:261404 buf:0 cache:64124 dirty:0 write:0
Swap total:0 free:0
  PID^^^VSZ^VSZRW   RSS (SHR) DIRTY (SHR) STACK COMMAND
 2786 1240m 84892 50356     0 33068     0   132 mihomo-linux-arm64-v1.19.1 -d /jffs/clash/

the free memory is indeed smaller when the program panics, but it seems there is still enough memory... can you show me some pointer? Thanks! @Skyxim

@xishang0128
Copy link
Contributor

The system you are using may reserve the minimum available memory to ensure stability; this is not a program issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants