Sometimes IDataBase.ExecuteAsync("JSON.ARRAPPEND") Doesn't Append Items to List in AWS ElastiCache Redis

[Daniel-Markov]

Description:

We’re running into an issue with the JSON.ARRAPPEND command using StackExchange.Redis with AWS ElastiCache Redis OSS. Occasionally, when we call this command, items don’t actually get appended to the list, even though the command appears to execute without any errors. This happens randomly, so it’s hard to reproduce consistently.

var result = await IDatabase.ExecuteAsync("JSON.ARRAPPEND", [key, listPath, .. serializedItems]);
_logger.LogDebug("Redis JSON.ARRAPPEND command response: {itemsAppended}, {isNull}", result.Length, result.IsNull);

What we’ve observed:

• When this issue happens, the response shows Length 1 and IsNull False, but the items aren't actually added to the list.
• There are no exceptions or other indicators suggesting that the action has failed.
• This issue doesn't happen every time, but it’s random, making it difficult to reproduce consistently.
• We’re using AWS ElastiCache Redis OSS, and our service is deployed on AWS ECS.

Environment:

• AWS ElastiCache Redis OSS (7.1.0)
• StackExchange.Redis (2.8.16)
• .NET 8
• Deployed on AWS ECS Fargate

Any help or insight into what might be causing this issue would be greatly appreciated. Thanks in advance!

[slorello89]

@Daniel-Markov -

This is probably not a StackExchange.Redis issue, you should probably confirm that the client is sending the commands you think it is (you can do that with MONITOR). Also you can try confirming whether or not this happens using Redis Stack

This is probably an issue with Elasticache. This is not the first time I've heard of issues with Elasticache JSON arrays - particularly with ARRAPPEND, we've heard rumblings about it off and on for a couple years now see this thread in our discord. For some background, Elasticache ported Redis' Source Available RedisJSON module a couple of years ago, but their port is completely closed source so we have no visibility into what it's doing. IDK if they've fixed this in a newer version, probably worth reaching out to their support team.

[Daniel-Markov]

@slorello89
Thanks for the advice, cannot connect to the discord link you've sent.
From what I see NRedisStack is built on top of StackExchange.Redis so I don't think it would help so much.

[slorello89]

@Daniel-Markov - might be an archived channel here was the original message:

Not suggesting to use NRedisStack, you can use that if you want to make it easier to interface with the JSON commands. I'm saying try reproducing against Redis Stack, which uses the actual Redis JSON module rather than the AWS's, port to see if you can reproduce the issue. The user above was able to replicate the issue only in Elasticache, in Redis Stack the command worked fine. This would just be so you can see whether the issue is replicable outside of that environment. Like I said, my guess would be it's AWS's JSON implementation that's the problem - just trying to offer methods for verifying.

[Daniel-Markov]

@slorello89
Was confused with the naming, sorry :) I get what you said now—thanks a lot!