Support storing arrays of primitives (string, int, etc.) as RLMObject properties

[andrewjl]

Reading over issue #1028 it looks like RLMObject doesn't support properties that are arrays of non-RLMObjects. Instead the individual strings each have to be declared as a property of an RLMObject and then stored in an RLMArray.

I'm curious to understand why your support for primitives doesn't extend to collections of objects of those types. As far as Realm is concerned, what's the difference between storing a string or an array of strings? And would you ever add this support to a future release?

---

Subtasks:

• Non-breaking changes for arrays of primitives (Make non-breaking changes to support arrays of primitives #5148)
• Breaking API changes for arrays of primitives (Make breaking changes needed to support arrays of primitives #5149)
• Basic support for arrays of primitives (Add support for arrays of more types #5274)
• Query support for arrays of primitives (Support queries involving arrays of primitives #5334)

[jpsim]

Hi @andrewjl, the reason RLMArray properties can only contain other RLMObject-derived objects is that tables in the underlying db must already exist for those objects. At that point, the RLMArray property just contains pointers to the relevant rows in that object type's table.

There are two fundamental ways we could add support for primitive types in Realm.

1. when a property of type RLMArray<NSString> is added to an RLMObject, we could create a table for NSString with a single column of type tightdb::type_String (Realm's core String type). This would essentially be the same as creating your own RLMObject wrapper around NSString, which is the approach we currently recommend, but with the added advantage of the strings not being "boxed". So this approach only adds syntactic sugar.

2. change Realm's modeling approach altogether by defining models using a protocol rather than subclassing. This way, you could persist any object in Realm, as long as it conforms to the Realm protocol. However, this approach is quite complex to implement and would require a considerable amount of development time, and would probably be more work for users as well.

So although we agree it'd be nice to store RLMArray<NSString> properties or RLMArray<NSDate> properties, it's not currently a high priority. We'll be sure to comment here once we have something to announce.

[andrewjl]

Thanks @jpsim for the explanation. I can see how option 2 would be complex to implement. Option 1 seems more doable, would you be willing to consider a pull request implementing it for arrays of NSString? (As well as the Swift equivalent.)

[jpsim]

I can't speak for the rest of the team, but I'd be open to that.

[michaelforrest]

+1

[andrewjl]

@jpsim based on our chat in person (at the carthage event) it sounds like the best solution would be to do what Realm already requires the user to do automatically, and transparently.

Seems like the best place to do this would be in RLMProperty, inside of setTypeFromRawType, where the RLMPropertyTypeArray case is handled. Instead of throwing an exception when RLMIsObjectSubclass returns false, we instead check the _objectClassName against a white list of class names (Just NSString,and NSNumber would be a good start) and then do what Realm would usually do with an RLMObject the user would provide as the inner class of the array.

Let me know if you think this is in the right direction. Also happy to provide more details if this is too high level.

[andrewjl]

More thoughts... when we detect a white listed inner RLMArray class we can handle it in one of two ways:

• Use the objc runtime to create an RLMObject subclass with a single property of the white listed inner class type. The upside is that we can then pass this dynamically generated class into Realm and it will do the right thing. The downside is that this won't work for Swift (as I don't believe it's possible to create classes on the fly in Swift) and also doing fancy things with the runtime may cause breakages elsewhere.
• We create a new table inside the schema that will store the array's values and write all the querying, updating, and other logic needed to manipulate the table outside of RLMObject. Not sure how much work this entails.

Update: After looking through some more classes, I noticed that RLMAccessor does some meta programming. Perhaps option 1 is more doable than I originally considered.

[thebarndog]

I'm a little late to the party but I'd like to offer some thoughts.

Regarding option 2, that is changing Realm's modeling approach to be protocol based over subclassing, is in my opinion the far superior approach. While it might require more work on the side of the Realm development team, long term it makes more sense because it's far more extensible. In fact one of the few things things I don't like about Realm (it's fantastic overall) is the limited amount of types that can represented, say for example a list of strings. Having to wrap use cases like that in a whole separate class is messy, something I'm sure fans of code style and beauty would shudder at.

A protocol based approach is actually how Mantle does it as of their 2.0 release. It's the only reason that I can use Realm and Mantle together. Mantle doesn't just use a protocol though; they provide a base class of MTLModel that implements the MTLModel protocol (MTLModelProtocol in Swift) so that for users that want to simply subclass their base class, they easily can and for users that need to be free of inheritance like I do can conform their custom model classes to the protocol. One of the concerns brought up originally was would probably be more work for users as well with the combined protocol and subclass approach, that work can easily be mitigated.

Under the hood I'd imagine that the modeling approach would be purely protocol based; the Object subclass would simply exist as a convenience for users.

And finally, So although we agree it'd be nice to store RLMArray<NSString> properties or RLMArray<NSDate> properties, it's not currently a high priority. I have to completely disagree here. In fact one of the reasons I switched from using Core Data was to have arrays of primitive data types persisted in something else than a transformable type. It's an incredibly common use case for a lot of iOS programmers and it shouldn't be treated as "something nice". I believe having that feature is a necessity because not having it, as I said before, really limits the developer on what they can do and leads to the messy pattern of having to make separate classes that encapsulate primitive types.

[peterpaulis]

i do it like this if i want to store simple types

#import "Contributor.h"
@interface Contributor()
@property (nonatomic, strong) NSData * spectrumPeaksData;
@end
@implementation Contributor
+ (nullable NSDictionary *)defaultPropertyValues {
    return @{@"spectrumPeaksData":[NSData data]};
}
////////////////////////////////////////////////////////////////////////
#pragma mark - Peaks
////////////////////////////////////////////////////////////////////////
- (void)setSpectrumPeaks:(NSArray *)spectrumPeaks {
    if ([spectrumPeaks count] == 0) {
        self.spectrumPeaksData = [NSData data];
        return;
    }
    self.spectrumPeaksData = [NSKeyedArchiver archivedDataWithRootObject:spectrumPeaks];
}
- (NSArray *)spectrumPeaks {
    if ([self.spectrumPeaksData length] == 0) {
        return nil;
    }
    return [NSKeyedUnarchiver unarchiveObjectWithData:self.spectrumPeaksData];
}
@end

alternatively you can cache the unarchive object until you call the set method again, where you invalidate the cache

#import "Contributor.h"
@interface Contributor()
@property (nonatomic, strong) NSData * spectrumPeaksData;
@property (nonatomic, strong) NSArray * spectrumPeaksCache;
@end
@implementation Contributor
+ (nullable NSDictionary *)defaultPropertyValues {
    return @{@"spectrumPeaksData":[NSData data]};
}
+ (NSArray *)ignoredProperties {
    return @[@"spectrumPeaksCache"];
}
////////////////////////////////////////////////////////////////////////
#pragma mark - Peaks
////////////////////////////////////////////////////////////////////////
- (void)setSpectrumPeaks:(NSArray *)spectrumPeaks {
    self.spectrumPeaksCache = nil;
    if ([spectrumPeaks count] == 0) {
        self.spectrumPeaksData = [NSData data];
        return;
    }
    self.spectrumPeaksData = [NSKeyedArchiver archivedDataWithRootObject:spectrumPeaks];
}
- (NSArray *)spectrumPeaks {
    if ([self.spectrumPeaksData length] == 0) {
        return nil;
    }
    self.spectrumPeaksCache = [NSKeyedUnarchiver unarchiveObjectWithData:self.spectrumPeaksData];
    return self.spectrumPeaksCache;
}
@end

[thebarndog]

But don't you think that that's an incredible amount of code just to achieve the simple task of storing primary data types? It seems so counter to the point.

[peterpaulis]

yes, you are right... but i list the code as only the current solution

[thebarndog]

What exactly are you try to achieve here, an array of NSData objects?

[peterpaulis]

nope, an array of any object that can be archived (NSString, NSData, NSDate, NSNUmber, NSDictionary etc)

see

• (void)setSpectrumPeaks:(NSArray *)spectrumPeaks;
• (NSArray *)spectrumPeaks;

[jpaas]

I have a need to store arrays of Ints as Object properties. Thanks @peterpaulis for your code sample above. I do agree that is a lot of code for something that seems commonplace and expected.

Although explicit support for collections of primitive types would be great, it seems to me a better fallback position would be to have some sort of a general mechanism for property transformations. Most other ORMs will allow you to define 2 way transforms for "custom" property types.

Another idea would be to create something like a JSON primitive type that could automatically be transformed to/from a Dictionary or Array.

Still, these are both just catchalls, for when the framework can't handle something different. I still would expect collections of primitives to be something that is supported out of the box.

BTW - I'm working on switching my project from couchbase-lite to realm. Array properties were a big source of difficulty for me on couchbase too.

[borut-t]

+1

[narciero]

+1

[nootfly]

+1, It's very bad without this feature. I have two classes which have string arrays. How can I do if I want to delete one class from the database since realm does not support cascading delete? All string are stored in another table.

[BlueMilkApps]

+1000

[andrewjl]

I would love to help get this out sooner. @timanglade @jpsim I have some familiarity with the objective-c part of the codebase. If I can get some guidance as to where to start and what approach to take (at a high level) I'm happy to get the ball rolling on this.

[rex-remind101]

+1000

[cobyanderson]

+1

[AndreiD]

+infinite

[Kalzem]

+over 9000

[NikKovIos]

Anyone knows how it works on Android?

[dmdque]

@jazz-mobility What does the <- operator in your code do? I can't find any mention of it.

[heyzooi]

@dmdque he is probably using this library here: https://github.com/Hearst-DD/ObjectMapper

[dmdque]

I see. Thanks!

[jazz-mobility]

Yes, I was using Realm + ObjectMapper.

[dle-coursera]

This doesn't seem to have been fixed. What's the priority of this?

[jpsim]

The priority is "S:P2 Backlog", which is just behind "S:P1 Backlog". This is publicly reflected in the GitHub issue labels. This is being designed and worked on currently in realm-core, but I can't share any concrete estimates of when we might officially release that functionality.

[ruangchupa]

+1

[aksswami]

@jazz-mobility We can also write transforms for ObjectMapper:-

class RealmString: RLMObject {
    dynamic var stringValue : String = ""
}

// In mapping method
cities                          <- (map["cities"], stringListTransform)

let stringListTransform = TransformOf<RLMArray, [String]>(fromJSON: { (value: [String]?) -> RLMArray? in
        let realmArray = RLMArray(objectClassName: "RealmString")
        guard let items = value else { return realmArray }
        for item in items {
            let realmString = RealmString()
            realmString =  item
            realmArray(realmString)
        }
        return realmArray
        }, toJSON: { (value: RLMArray?) -> [String]? in
            return nil
})

[jazz-mobility]

@aksswami If you check the description, it is clearly mention "Support storing arrays of primitives". I guess this transformation is supported from long time now. But this is not the real solution to the problem.

[isoiphone]

+1
Would love to see support for this-- even if it is just syntactic sugar over the Box solution.

[gerchicov-bp]

The question is:
You can't/don't want to support [String] (array of strings) so why don't you add something like RealmString which includes single field of String type? Why should programmers do this manually always?

[austinzheng]

We are currently working on this feature, as the In Progress tag denotes, and will release it once it's complete and ready. Your patience is greatly appreciated.

[bdash]

Arrays of primitives are supported as of v3.0.0-beta.4.