[Bug] (good first issue) The in_memory table attribute for table creation failed when upgrading from version 1.x to version 2.x, causing issues with dynamic partitioning.

[deardeng]

Search before asking

• I had searched in the issues and found no similar issues.

Version

2.1.6

What's Wrong?

Upgrade Process：1.2.4 -> 2.0.15 ->2.1.6

The table was created a long time ago with the in_memory = true parameter. After upgrading to the new version, the creation of new dynamic partitions failed. Upon investigation, it was found that this parameter was causing the issue.
we can force the in_memory parameter to be false. Sometimes, if partitions cannot be created, it leads to data not being able to be ingested, resulting in a series of problems

2024-11-04 12:22:03,459 INFO (DynamicPartitionScheduler|51) [PropertyAnalyzer.analyzeDataProperty():274] Can not assign cool down timestamp to HDD storage medium, ignore user setting.
2024-11-04 12:22:03,460 INFO (DynamicPartitionScheduler|51) [DynamicPartitionScheduler.recordCreatePartitionFailedMsg():627] dynamic add partition failed: errCode = 2, detailMessage = errCode = 2, detailMessage = Not support set 'in_memory'='true' now!, db: dim_blockchain, table: doris_dwm_eth_trans_num_180d
2024-11-04 12:22:03,460 WARN (DynamicPartitionScheduler|51) [DynamicPartitionScheduler.executeDynamicPartition():616] has error
    at org.apache.doris.clone.DynamicPartitionScheduler.executeDynamicPartition(DynamicPartitionScheduler.java:612) ~[doris-fe.jar:1.2-SNAPSHOT]
    at org.apache.doris.clone.DynamicPartitionScheduler.runAfterCatalogReady(DynamicPartitionScheduler.java:678) ~[doris-fe.jar:1.2-SNAPSHOT]

What You Expected?

When upgrading from the old 1.x version where the table was created with in_memory = true, we ignore the in_memory parameter in the 2.x version so that it does not affect the dynamic partition creation

Anything Else?

#18731 This PR deprecates the in_memory parameter, which was removed in version 2.0

https://doris.apache.org/zh-CN/docs/3.0/sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE/ Documentation also reflects this deprecation status

but #18731 compatibility upgrade issues were not considered

Fix Method：

To resolve this, we can modify the gsonPostProcess functions in both TableProperty.java and PartitionInfo.java to force the in_memory parameter to be false after a restart, and add a fe ut to check it

How to Reproduce?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct