Apply TTL via CrudRepository.save(…) [DATACASS-534]

[spring-projects-issues]

SanjayAgnani opened DATACASS-534 and commented

Hello guys ,
I would like to know if we can insert /update row with an feature of TTL in Spring Data Cassandra using repository interfaces.
Able to delete/select based on below method declaration

@AllowFiltering
public void deleteByfirstnameAndLastname(String fn, String ln);

Looking for something similar to insert /save using TTL support

---

0 votes, 5 watchers

[spring-projects-issues]

Mark Paluch commented

Not sure I follow. Spring Data for Apache Cassandra does not support DELETE queries through query derivation. As of now we only support SELECT queries.

To the second part of this ticket: Saving/inserting along WriteOptions is not supported right now. We can't add simply a @TTL(…) to save(…) because these methods are implemented in a static manner by SimpleCassandraRepository and there's no possibility to evaluate the annotation.

What could be possible is adding overloads for save, saveAll, and insert methods accepting WriteOptions

[spring-projects-issues]

Mark Paluch commented

Moved the ticket to https://jira.spring.io/browse/DATACASS as it's not related to JPA

[spring-projects-issues]

SanjayAgnani commented

So it means using CrudRepository/CassandraRepository only select query can be made , for inserting/updating with TTL options on has to use CassandraOperations Interface with Insert/Update options.
Do we have any plan to add this feature , also would like to know if there is any Auditing feature like @CreatedBy @CreatedOn in Spring Data Cassandra along with some pointers in spring-data-cassandra ,to get the executed cql?(like we have option in jpa…showSql?)

[spring-projects-issues]

Mark Paluch commented

There are a couple of issues you're mentioning here. Please check out our issue tracker/reference documentation the next time as all details regarding CQL logging and auditing are mentioned there.

1. Using TTL requires right now direct CassandraTemplate/CqlTemplate use. Applying TTL on writes through repositories was requested a couple times so I think we can add that kind of feature at some point in time.
2. See DATACASS-4 for auditing.
3. See https://docs.spring.io/spring-data/cassandra/docs/current/reference/html/#cassandra.cql-template for CQL logging

[spring-projects-issues]

Sergio Aguayo commented

Mark Paluch I was thinking on implementing this by adding version of some function in CassandraRepository/ReactiveCassandraRepository with additional parameter (int ttl).

Candidates for this would include:

save(S)

saveAll(Iterable<S>)

insert(S)

insert(Iterable<S>)

And their reactive counterparts. I would add a ttl parameter to them. Sounds good?

[spring-projects-issues]

Mark Paluch commented

Can you outline your use-case? Are you using typically a fixed TTL? Or a TTL per entity? We should discuss what is going to be addressed before we start changing code

[spring-projects-issues]

Łukasz Świątek commented

Im also interested is this feature.

 

We are using both fixed TTL for all entities of given type, and in other cases dynamic ttl per entity.

But, fixed ttl can also be set on the cassandra table itself, so if we have to choose, then ttl per entity seems be more important.

 

My company customized some of spring-data-cassandra componesnt to achive the same result, but i would like to see it baseline, so i can drop those customizations.

What we do, is use custom annotation @EntityWriteOptions which allows to specify some of WriteOptions parameters on entity. Additionally if entity implements TtlAware then it provides a method to obtain ttl dynamically. The only constraint  for dynamic ttl is that, entity must be saved with expiration date, to make sure ttl is preserved when entity would is read and written again.

 

Adding WriteOpions/QueryOptions seems to be a better solution as it would allow to specify additional options which canno be defined on annotation (like some complex retry policy, or conditional update)

 

[spring-projects-issues]

Mark Paluch commented

We're now on Cassandra driver 4, so I wonder whether we should introduce a execution profile annotation to specify the execution profile to apply profiles on various repository methods or for the entire repository. That deserves its own ticket. 

From what I hear, having a @TimeToLive annotation on entity/repository level would be appropriate to either control the TTL for the entity on the repository level. Storing the TTL within the entity comes with the mentioned side effect that reading the entity requires reading the TTL back to ensure it can be applied upon the next modification

[yuranich]

Hi team, any update on this?
Having this feature would be awesome!

[nikhilvagyani]

Hi team,
is any latest version supported for dynamic ttl values while inserting data?
or when you are planning to include these feature?

[Aleksei-Ch]

Hi there!

The problem is pretty old, but it seems to be still actual.

So from I know there is like three common workarounds now for dealing with Cassandra WriteOptions:

• Don't use Spring Data repositories at all, write repository classes with plain CassandraOperations calls and customize queries as you wish.
• Extend CassandraRepository<T, ID> with custom methods like save(T entity, Integer ttl), then extend SimpleCassandraRepository<T, ID> and implement there your new Cassandra Repository interface (where modify WriteOptions as you need), and finally define Spring Data repository interface extending your new Cassandra Repository interface and use it.
• Add some custom annotation on repository class/method level with another proxy around calls which will alter the WriteOptions according to some instructions

The first solution makes you write a lot of boilerplate code, and you'll use like only half of the library's toolbox (that's actually what we're doing).

The second one looks better, but the final repository isn't exactly compatible with Spring Data because it's custom (or am I wrong and just don't get its philosophy?), so it doesn't look perfect either.

The third one is a good idea, but honestly I did not get how it could be implemented, seems to me that I'm not that good.

Which all leads me to the point. What do you think, if there was an ability to implement in you project some bean of some predefined interface which would provide the Insert/UpdateOptions based on particular Entity and its CassandraEntityInformation<T, ID> to your repository?

In implementation like this you could've set custom WriteOptions for each table or even entity. Here is an example (looks sloppy but it covers the idea):

package org.springframework.data.cassandra.core;

import org.springframework.data.cassandra.repository.query.CassandraEntityInformation;

import java.util.UUID;

public interface InsertOptionsProvider {
    <T, ID> InsertOptions getInsertOptions(CassandraEntityInformation<T, ID> metadata, T entity);
}


@Component
public class CustomInsertOptionsProvider implements InsertOptionsProvider {

    private final Map<Object, Integer> ttlCache = new WeakHashMap<>();

    @Override
    public <T, ID> InsertOptions getInsertOptions(CassandraEntityInformation<T, ID> metadata, T entity) {
        Integer ttl = ttlCache.get(entity);
        if (ttl != null) {
            return InsertOptions.builder().ttl(ttl).build();
        }
        if (metadata.getJavaType().equals(Model1.class)) {
            return InsertOptions.builder().ttl(100).build();
        }
        return InsertOptionsProvider.builder().build();
    }

    public void setIndividualTtlFor(Object entity, Integer ttl) {
        ttlCache.put(entity, ttl);
    }
}


@Table("model_1")
@Data
@AllArgsConstructor
@NoArgsConstructor
public class Model1 {
    @PrimaryKey
    private String id;
    private String column1;
    private String column2;
}


public interface Model1Repo extends CrudRepository<Model1, String> {
}


@Service
@RequiredArgsConstructo
public class Model1Service {

    private final Model1Repo repo;
    private final CustomInsertOptionsProvider insertOptionsProvider;

    public void save(String column1, String column2, Integer ttl) {
        Model1 a = new Model1();
        a.setId(UUID.randomUUID().toString());
        a.setColumn1(column1);
        a.setColumn2(column2);
        if (ttl != null) {
            insertOptionsProvider.setIndividualTtlFor(a, ttl);
        }
        repo.save(a);
    }
}

So if you defined the InsertOptionsProvider bean, you'll get custom WriteOptions as you want it, otherwise - default. And a CrudRepository is still a CrudRepository.

I'm not sure that it can be the right thing, but I have a working code for it forked from the current main branch (4.3.0). So if the team would like it I would love to contribute.

[Aleksei-Ch]

@mp911de , hi!

What do you think of the suggestion above?

[mp911de]

All examples above introduce quite some complexity without considering the entirety of query options for all repository methods. We haven't found a model yet that would allow providing all sorts of options in a good way.

For the time being, subclassing SimpleCassandraRepository and adding your own methods/overrides seems a better approach to address individual use-cases.

[karuturi]

Having this would be helpful.
@mp911de is this being discussed somewhere? Probably in mailing list? I can help with development effort if needed.

[mp911de]

Discussion for this topic happens here. Currently, we're at the design state, not really into code. There's quite some tension in Repository.save(…) vs. CassandraTemplate.save(…). We generally want to keep our repository method signatures free from store-specific code. Any kind of additional methods/overloads accepting additional arguments violates that principle.