Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementation class of the Memory interface, does not support serialization and deserialization. #1117

Open
lvchzh opened this issue Jul 25, 2024 · 7 comments

Comments

@lvchzh
Copy link

lvchzh commented Jul 25, 2024

Bug description
deserialization. Currently, ChatMemory only implements a memory-based dialogue cache storage. I wanted to implement a Redis-based cache for Memory in my project, but I encountered a problem: the implementation classes of the Memory interface do not implement the Serializable interface, which causes failures in accessing Redis.

Environment
Spring AI 1.0.0-SNAPSHOT
Steps to reproduce
step1 :
`@Slf4j
public class RedisChatMemory implements ChatMemory {

private static final String PREFIX = "chat:";

@Override
public void add(String conversationId, List<Message> messages) {
    messages.forEach(message -> RedisUtils.lSet(PREFIX + conversationId, message, CommonConst.CHAT_EXPIRE_TIME));
}

@Override
public List<Message> get(String conversationId, int lastN) {
    List<Message> messages = RedisUtils.lGet(PREFIX + conversationId, -lastN, -1);
    log.info("history conversation :{}", JSONObject.toJSONString(messages));
    return messages;
}

@Override
public void clear(String conversationId) {
    RedisUtils.del(PREFIX + conversationId);
}

}`
Step2:
Runtime error

Expected behavior
Support for Redis-based session storage

lvchzh added a commit to lvchzh/spring-ai that referenced this issue Jul 25, 2024
@canonxu
Copy link

canonxu commented Aug 14, 2024

i also got this issue
image

@csterwa csterwa added bug Something isn't working Chat Memory redis labels Sep 9, 2024
@csterwa
Copy link

csterwa commented Sep 9, 2024

@lvchzh @canonxu has this issue been resolved in the current milestone release of Spring AI?

@markpollack
Copy link
Member

markpollack commented Nov 12, 2024

Hi. There is an implementation of the ChatMemory interface for Cassandra and there is a PR for pgvector. That said, we would like to provide an impelmentation that delgates to Spring's Cache abstraction along the lines of what was done in llamaindex. I'll create a new issue and post it back here. I don't see an implementation where having ChatMemory implement serializable is needed, entries of the memory may or may not need to be serializable depending upon the storage approach.

@markpollack markpollack removed the bug Something isn't working label Nov 12, 2024
@Joaonic
Copy link

Joaonic commented Nov 25, 2024

I've encountered the same issue with implementing a Redis-based cache for ChatMemory. To address this, I implemented a workaround using Jackson and mixins to handle serialization and deserialization with Redis. This solution is currently functioning correctly in version M4.

Implementation Details

  1. Define Message Mixins

    import com.fasterxml.jackson.annotation.JsonSubTypes;
    import com.fasterxml.jackson.annotation.JsonTypeInfo;
    import org.springframework.ai.chat.messages.AssistantMessage;
    import org.springframework.ai.chat.messages.SystemMessage;
    import org.springframework.ai.chat.messages.ToolResponseMessage;
    import org.springframework.ai.chat.messages.UserMessage;
    
    @JsonTypeInfo(
            use = JsonTypeInfo.Id.NAME,
            include = JsonTypeInfo.As.PROPERTY,
            property = "messageType",
            visible = true
    )
    @JsonSubTypes({
            @JsonSubTypes.Type(value = AssistantMessage.class, name = "ASSISTANT"),
            @JsonSubTypes.Type(value = SystemMessage.class, name = "SYSTEM"),
            @JsonSubTypes.Type(value = UserMessage.class, name = "USER"),
            @JsonSubTypes.Type(value = ToolResponseMessage.class, name = "TOOL")
    })
    public abstract class MessageMixin {
    }
  2. Configure Redis Template with Jackson Serializer

    import com.fasterxml.jackson.databind.ObjectMapper;
    import org.springframework.ai.chat.messages.*;
    import org.springframework.context.annotation.Bean;
    import org.springframework.context.annotation.Configuration;
    import org.springframework.data.redis.connection.RedisConnectionFactory;
    import org.springframework.data.redis.core.RedisTemplate;
    import org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer;
    import org.springframework.data.redis.serializer.StringRedisSerializer;
    
    @Configuration
    public class RedisConfig {
    
        @Bean
        public RedisTemplate<String, Message> redisTemplate(RedisConnectionFactory connectionFactory) {
            RedisTemplate<String, Message> template = new RedisTemplate<>();
            template.setConnectionFactory(connectionFactory);
    
            StringRedisSerializer stringSerializer = new StringRedisSerializer();
            template.setKeySerializer(stringSerializer);
            template.setHashKeySerializer(stringSerializer);
    
            ObjectMapper objectMapper = new ObjectMapper();
    
            objectMapper.addMixIn(Message.class, MessageMixin.class);
            objectMapper.addMixIn(UserMessage.class, UserMessageMixin.class);
            objectMapper.addMixIn(AssistantMessage.class, AssistantMessageMixin.class);
            objectMapper.addMixIn(SystemMessage.class, SystemMessageMixin.class);
            objectMapper.addMixIn(ToolResponseMessage.class, ToolResponseMessageMixin.class);
    
            Jackson2JsonRedisSerializer<Message> jsonSerializer = new Jackson2JsonRedisSerializer<>(Message.class);
            jsonSerializer.setObjectMapper(objectMapper);
            template.setValueSerializer(jsonSerializer);
            template.setHashValueSerializer(jsonSerializer);
    
            template.afterPropertiesSet();
            return template;
        }
    }
  3. Define Specific Mixins

    import com.fasterxml.jackson.annotation.JsonCreator;
    import com.fasterxml.jackson.annotation.JsonProperty;
    import org.springframework.ai.model.Media;
    
    import java.util.Collection;
    import java.util.Map;
    
    public abstract class UserMessageMixin {
    
        @JsonCreator
        public UserMessageMixin(
                @JsonProperty("content") String textContent,
                @JsonProperty("media") Collection<Media> media,
                @JsonProperty("metadata") Map<String, Object> metadata
        ) {
        }
    }

Note: Ensure that all mixin classes (e.g., AssistantMessageMixin, SystemMessageMixin, etc.) are properly defined similar to UserMessageMixin to cover all message types.

@poo0054
Copy link

poo0054 commented Jan 21, 2025

Moreover, Message has not yet implemented Serializable. Now ChatMemory can only be in memory. Can you provide a persistence method so that historical information can be queried normally even if the service is restarted or distributed?

@hardikSinghBehl
Copy link

Used the workaround suggested by @Joaonic in a POC. Was able to persist chat memory across application sessions.

@poo0054
Copy link

poo0054 commented Jan 22, 2025

Thank you very much. In future versions, it will be directly supported without having to rely on Jackson serialization to be compatible with this problem.
If I use other persistence, it won't work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants