Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tmp #25

Open
wants to merge 502 commits into
base: tmp-main
Choose a base branch
from
Open

Tmp #25

wants to merge 502 commits into from

Conversation

Rosyrain
Copy link
Owner

What type of PR is this?

  • API-change
  • BUG
  • Improvement
  • Documentation
  • Feature
  • Test and CI
  • Code Refactoring

Which issue(s) this PR fixes:

issue #

What this PR does / why we need it:

volgariver6 and others added 30 commits June 18, 2024 13:03
if the txn which access tables under mo_debug database starts
just after tn upgrade, the context will be timeout when sending sub
table request, because the timeout is 30s and tn upgrade lasts
longer than that. so change it to 5m.

Approved by: @zhangxu19830126, @sukki37
1.Refactor the remote execution function and add more logging for it. If the data-receiver prematurely terminates the query, it will always notify the sender to cease operations before closing the connection.

2.Remove numerous fields from the pipeline's Message.

3.Add comments to some obscure code sections.

Approved by: @daviszhen, @reusee, @aunjgr, @badboynt1, @ouyuanning, @sukki37
Fix Rebuild snapshot's table meta

Approved by: @XuPeng-SH, @sukki37
…trixorigin#16985)

retry to get latest ts, but do not fatal if waiter is paused when wait for the ts.

Approved by: @zhangxu19830126, @sukki37
释放掉用于后台任务的batch

Approved by: @qingxinhome, @sukki37
comsume ErrInvalidTruncateLsn

Approved by: @XuPeng-SH, @sukki37
Fix error refers for table meta

Approved by: @XuPeng-SH, @sukki37
…17002)

fix a bug that the TCP context was incorrectly used as a STREAM context during the remote run,
this will cause an inability to properly determine whether the stream had been disconnected.

Approved by: @ouyuanning, @XuPeng-SH, @volgariver6, @zhangxu19830126, @sukki37
…igin#17017)

修改了一个远端执行的context没有指定deadline导致的错误。

Approved by: @badboynt1, @ouyuanning, @sukki37
- Add/Remove some New() functions
- Remove Parts, whose information is included in CStrParts
- Add get db/tbl/col name and nameOrigin functions
- Add lower_case_table_names into BindContext

Approved by: @daviszhen, @aunjgr, @iamlinjunhong, @aressu1985, @reusee, @heni02, @sukki37
1. fix snapshot-read bug.

Approved by: @XuPeng-SH, @sukki37
1. 这行代码的逻辑目前并不重要,但是由于无法正常感知到网络断开,会有问题,因此暂时删除。
在main分支上将恢复这段逻辑。
2. 调整了一个日志的日志等级。
3. 调整了一个map的结构。

Approved by: @reusee, @ouyuanning, @daviszhen, @sukki37
Adjust timeout for orphan transactions.

Approved by: @iamlinjunhong, @sukki37
… to multi function (matrixorigin#16991)

fix ranges bug of more than 2 augments in or expression.

Approved by: @XuPeng-SH, @heni02, @sukki37, @aunjgr, @aressu1985
filter out the CNs with different commit ID, because there is incompatibility
problem if CN servers with different commit ID exuecute the same query.

Approved by: @XuPeng-SH, @reusee, @daviszhen, @w-zr, @zhangxu19830126, @sukki37
为issue3482,增加合适的日志。

Approved by: @qingxinhome, @sukki37
…gin#17054)

修了网络自己发生错误的情况下,不会调用rpc.close()方法的错误,这可能会导致查询结束后资源不被释放。

Approved by: @ouyuanning, @sukki37
add timer to check if the first logtail of table has been received,
to return from the wait function.

Approved by: @XuPeng-SH, @sukki37
返回值里没有将connection正确返回,导致无法被关闭。

Approved by: @ouyuanning, @sukki37
1.  The entries in PartitionState.primaryIndex  are not deleted in some case, leads to memory leak.

Approved by: @reusee, @sukki37
qingxinhome and others added 30 commits September 6, 2024 07:50
Resolve horizontal upgrade exception handling for the same version

Approved by: @daviszhen, @zhangxu19830126, @sukki37
…rixorigin#18588)

changes:
- aggr erorr statement filter with same error msg.

Approved by: @heni02, @qingxinhome, @sukki37
…rigin#18613)

there is no response from server after client send a COM_STMT_CLOSE cmd,
which will cause the session cannot be transferred forever. So ignore the
send time when client send a COM_STMT_CLOSE cmd.

Approved by: @zhangxu19830126, @sukki37
remove unnecessary log print

Approved by: @sukki37, @ouyuanning
1,
leak原因:
profile 和 日志的信息对不齐,现在具体什么原因不确定。
leak现状:
3996 是在v1.2.2-20240715 上出现的。在这个版本之后,已经做了leak方面的下述修改,已经合并在1.2-dev最新的代码中。
后台执行都换成autocommit = true。
后台执行结束清理资源时,会强制结束事务。
修改检查udf function的奇怪逻辑。


leak修改:
footprints直接记录到txnoperator. 能实时记录事务走过那些函数。避免之前要复制的情况。

2,
load data local问题原因:
一种可能原因,在计算引擎还未从pipereader读数据时,
就kill load data local. 然后计算引擎退出,未关闭pipereader. 之后pipewrite.Write阻塞,导致未从load data local退出。

修改:
在load local启动时,启动一个哨兵线程,接受ctx信号。如果ctx在kill时被cancel,立即关闭pipereader. 让pipewriter.write退出。



3,
将incrStatement和rollbackLastStatement的卡住单独列出来,而不含在`found leak txn`。

leak checker新增下述日志。
```
found txn in incr statement

found txn in rollback statement
```

4,
将dropAccount中执行的sql落日志。

Approved by: @XuPeng-SH, @zhangxu19830126, @qingxinhome, @reusee, @sukki37, @triump2020, @aptend
Optimize the io merger to skip if it does not read caches.

Approved by: @sukki37
increase the priority of existing txn

Approved by: @zhangxu19830126, @sukki37
)

dealing with duplicate S3 resource statistics issues

Approved by: @aunjgr, @sukki37
…ct (matrixorigin#18674)

clear the tables in global stats when logtail re-connect.

Approved by: @LeftHandCold, @XuPeng-SH, @sukki37
update CollectDeletesLocked, return tae possible duplicate if delsMap is nil

Approved by: @zhangxu19830126, @XuPeng-SH, @sukki37
fileservice: do not set default memory and disk config in setDefaults

Approved by: @zhangxu19830126, @sukki37
Add summary to speed up scanning txn during pull logtail

Approved by: @XuPeng-SH, @zhangxu19830126, @sukki37
set srv.pu before creating hakeeper client

Approved by: @daviszhen, @sukki37
…rigin#18731)

Method for reading query results during upgrade modification
Change from `UnsafeStringAt()` to call `GetStringAt()`

Approved by: @daviszhen, @zhangxu19830126, @sukki37
fix KillRoutineConnections panic

Approved by: @daviszhen, @sukki37
…gin#18760)

support order by limit optimization for non-sorted columns

Approved by: @ouyuanning, @sukki37
…#18779)

cp to 1.2-dev 'fix a bug that cause hashmap data wrong'

Approved by: @ouyuanning, @heni02, @m-schen, @sukki37
移除了group_concat每行结果不能超过1024的限制

Approved by: @LeftHandCold, @sukki37
…le (matrixorigin#18787)

In the case of "load data local infile" statement, client sends the
first packet, then server sends response, which is "0xFB + filename",
after that, client sends content of filename and an empty packet, at
last, server sends OK packet. The sequence ID of this OK packet is not
1, and will cause the session cannot be transferred after this stmt finished.
So, the solution is: when server sends 0xFB and the sequence ID of
next packet is 3 bigger than last one, the next packet MUST be an
OK packet, and the transfer is allowed.

Approved by: @aylei, @sukki37, @zhangxu19830126
1. fix range checkpoint
2. add log

Approved by: @XuPeng-SH, @sukki37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.