Codable firestore #838

alickbass · 2018-02-22T15:52:27Z

Hi guys! So this is a WIP PR to solve #627. It is based on #815.

In general, code is almost 1-to-1 port of JSONEncoder from Foundation and PlistEncoder with some minor changes to fit it to Firestore. Most of the code is just boilerplate and the most interesting part is in

func unbox<T : Decodable>(_ value: Any, as type: T.Type) throws -> T?

in FirestoreDecoder and

func box_<T : Encodable>(_ value: T) throws -> NSObject?

in FirestoreEncoder.

I also created internal types Firestore.Encoder and Firestore.Decoder to able to test without needing DocumentReference and DocumentSnapshot. Everything is internal now without any public methods, so that we can already start testing and deciding if it's the right approach without the pain of approving the public API

//сс @wilhuff @paulb777

* Move Example/Firestore_SwiftTests_iOS to Swift/Tests * Delete generated test case * Update Info.plist location to match

Also corresponding xcodeproj changes

googlebot · 2018-02-22T15:52:34Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.
If your company signed a CLA, they designated a Point of Contact who decides which employees are authorized to participate. You may need to contact the Point of Contact for your company and ask to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the project maintainer to go/cla#troubleshoot. The email used to register you as an authorized contributor must be the email used for the Git commit.
In order to pass this check, please resolve this problem and have the pull request author add another comment and the bot will run again. If the bot doesn't comment, it means it doesn't think anything has changed.

alickbass · 2018-02-22T15:53:57Z

I signed it!

googlebot · 2018-02-22T15:54:14Z

CLAs look good, thanks!

wilhuff

I've taken a pass through the decoder and note that there is a ton of effectively duplicate code. It's very hard to tell what's unique about each overload in here.

I've made some suggestions for how to cut down on this.

wilhuff · 2018-02-24T00:56:39Z

Firestore/Swift/Source/Codable/CodableDocumentReference.swift

@@ -0,0 +1,38 @@
+//


All files need a copyright notice to be in this repo. You can copy from the top of any existing file and adjust the date or use this:

/* * Copyright 2018 Google * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */

wilhuff · 2018-02-24T04:28:22Z

Firestore/Swift/Source/Codable/CodableDocumentReference.swift

+import FirebaseFirestore
+
+enum FirestoreDecodingError: Error {
+    case decodingIsNotSupported


Google's coding style requires two space indents. Rather than fix this by hand please wait for PR #847 to land and run that here to fix this.

wilhuff · 2018-02-24T04:29:59Z

Firestore/Swift/Source/Codable/CodableFieldValue.swift

+import FirebaseFirestore
+
+/**
+ * A protocol describing the encodable properties of a FirebaseFirestore.


This is the encodable properties of a FieldValue, no?

wilhuff · 2018-02-24T04:31:58Z

Firestore/Swift/Source/Codable/CodableDocumentReference.swift

@@ -0,0 +1,38 @@
+//


I just re-read our internal style guide and realized that these files should really be Class+Protocol.swift. In this case DocumentReference+Codable.swift. Sorry to lead you astray with the other PR :-(.

wilhuff · 2018-02-24T04:33:37Z

Firestore/Swift/Source/Codable/CodableDocumentReference.swift

+
+import FirebaseFirestore
+
+enum FirestoreDecodingError: Error {


These don't have much to do with DocumentReference. Perhaps put these in an Errors.swift?

wilhuff · 2018-02-24T06:25:58Z

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+            throw DecodingError.valueNotFound(type, DecodingError.Context(codingPath: decoder.codingPath + [_FirestoreKey(index: currentIndex)], debugDescription: "Unkeyed container is at end."))
+        }
+
+        decoder.codingPath.append(_FirestoreKey(index: currentIndex))


If this (and the defer) were at the top of function then we wouldn't have to append to to the codingPath to compose the error message for the !isAtEnd case.

wilhuff · 2018-02-24T06:26:30Z

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+        defer { decoder.codingPath.removeLast() }
+
+        guard let decoded = try decoder.unbox(container[currentIndex], as: Bool.self) else {
+            throw DecodingError.valueNotFound(type, DecodingError.Context(codingPath: decoder.codingPath + [_FirestoreKey(index: currentIndex)], debugDescription: "Expected \(type) but found null instead."))


Also decoder.codingPath already has _FirestoreKey(index: currentIndex) appended to the end of it; this will duplicate the index in the coding path.

wilhuff · 2018-02-24T06:29:05Z

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+        guard !self.isAtEnd else {
+            throw DecodingError.valueNotFound(KeyedDecodingContainer<NestedKey>.self,
+                                              DecodingError.Context(codingPath: self.codingPath,
+                                                                    debugDescription: "Cannot get nested keyed container -- unkeyed container is at end."))


Despite the different error message, this could probably also use expectNotAtEnd. I'm not sure there's any special value in giving a slightly better error message for cases that seem extremely unlikely.

wilhuff · 2018-02-24T06:30:34Z

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+        }
+
+        let value = self.container[self.currentIndex]
+        guard !(value is NSNull) else {


Could use requireValue.

We cannot really use requireValue here as it expects the Optional and we are checking for the NSNull

wilhuff · 2018-02-24T06:37:10Z

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+                                                                    debugDescription: "Cannot get keyed decoding container -- found null value instead."))
+        }
+
+        guard let dictionary = value as? [String : Any] else {


This through the bit that constructs the KeyedDecodingContainer seems as if it can be a method we factor out on the top-level decoder.

public func container<Key>(keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key> { return try container(of: storage.topContainer, keyedBy: type) } public func container<Key>(of value: Any, keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key> { guard !(value is NSNull) else { throw DecodingError.valueNotFound(KeyedDecodingContainer<Key>.self, DecodingError.Context(codingPath: codingPath, debugDescription: "Cannot get keyed decoding container -- found null value instead.")) } guard let dictionary = value as? [String: Any] else { let context = DecodingError.Context(codingPath: codingPath, debugDescription: "Not a dictionary") throw DecodingError.typeMismatch([String: Any].self, context) } let container = _FirestoreKeyedDecodingContainer<Key>(referencing: self, wrapping: dictionary) return KeyedDecodingContainer(container) }

googlebot · 2018-02-27T15:41:07Z

So there's good news and bad news.

👍 The good news is that everyone that needs to sign a CLA (the pull request submitter and all commit authors) have done so. Everything is all good there.

😕 The bad news is that it appears that one or more commits were authored by someone other than the pull request submitter. We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that here in the pull request.

Note to project maintainer: This is a terminal state, meaning the cla/google commit status will not change from this State. It's up to you to confirm consent of the commit author(s) and merge this pull request when appropriate.

alickbass · 2018-02-27T15:44:24Z

@wilhuff I merged master into this branch... And now I am not sure what should I do :) Should I create another PR against master?

wilhuff · 2018-02-28T01:34:54Z

Just keep going. If you're really concerned about the clarity of the PR stack, feel free to rebase your changes and force push to your repository. We're going to squash merge your change into canonical history so it doesn't matter too much if your changes are all that clear at the commit level.

…

On Tue, Feb 27, 2018 at 7:44 AM Oleksii Dykan ***@***.***> wrote: @wilhuff <https://github.com/wilhuff> I merged master into this branch... And now I am not sure what should I do :) Should I create another PR against master? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#838 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJxjkPzXejXuGEt0sJQM_mbBBiYa9jahks5tZCLagaJpZM4SPic9> .

wilhuff · 2018-03-01T05:27:49Z

The base PR from which this started has since been merged to master. I've retargeted this PR against master and now the diff is clean again.

wilhuff

Decoder changes are great. I have minor naming feedback (see specific callouts) and a larger question about how to handle values beyond the range of the intended storage type. JSONDecoder seems to want to fail, but in other context Firestore just does what a native cast would do.

wilhuff · 2018-03-01T05:44:13Z

Firestore/Example/Firestore.xcodeproj/xcshareddata/xcschemes/Firestore-Example.xcscheme

@@ -49,6 +49,16 @@
               ReferencedContainer = "container:Firestore.xcodeproj">
            </BuildableReference>
         </TestableReference>
+         <TestableReference


The base state from which you branched was broken. I've since fixed this, but unfortunately these diffs remain. For all the .xcscheme files, make sure your upstream branch is up-to-date and then git checkout the master version of these files and commit them. After pushing you shouldn't see any additions to project files.

wilhuff · 2018-03-03T04:09:56Z

Firestore/Swift/Source/Codable/Errors.swift

@@ -0,0 +1,16 @@
+//


Copyright notice.

I realize this isn't super important to you, but this is the only way that Google allows me to continue to work in the open with you, so it's very important to me :-).

wilhuff · 2018-03-03T04:18:57Z

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+}
+
+@available(swift 4.0.0)
+fileprivate class _FirestoreDecoder : Decoder {


Is Firestore.DocumentDecoder a possibility? I'm still not clear on exactly what the nested struct within Firestore is for. To me this seems like it should be the public interface on the extension of DocumentSnapshot.

That is to say DocumentDecoder becomes internal, and we create an extension of DocumentSnapshot that uses it (as above) to implement the public API for decoding to an object of a Decodable type.

wilhuff · 2018-03-03T04:21:23Z

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+}
+
+@available(swift 4.0.0)
+fileprivate struct _FirestoreDecodingStorage {


Could this be named DocumentDecodingStorage?

My basic concern is that "Firestore decoding" is too general. For example, we could, in the future decode whole query snapshots, in which case a "Firestore decoder" is insufficient.

wilhuff · 2018-03-03T04:21:57Z

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+}
+
+@available(swift 4.0.0)
+fileprivate struct _FirestoreKeyedDecodingContainer<K : CodingKey> : KeyedDecodingContainerProtocol {


DocumentKeyedDecodingContainer?

wilhuff · 2018-03-03T04:22:32Z

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+    }
+
+    public func decode(_ type: Bool.Type, forKey key: Key) throws -> Bool {
+        let entry = try require(key: key)


This looks substantially better. Thank you for working through it.

wilhuff · 2018-03-03T04:23:17Z

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+}
+
+@available(swift 4.0.0)
+fileprivate struct _FirestoreUnkeyedDecodingContainer : UnkeyedDecodingContainer {


DocumentUnkeyedDecodingContainer?

wilhuff · 2018-03-03T04:25:45Z

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+                return false
+            }
+
+            /* FIXME: If swift-corelibs-foundation doesn't change to use NSNumber, this code path will need to be included and tested:


How are these FIXMEs to be addressed? I assume it comes from the implementation to which you referred.

Do you think there a way to add a check to make sure that we're not messing this up? I'm more concerned with correctness than speed so an extra else case here (after the two cases that are realistically possible) doesn't bother me.

wilhuff · 2018-03-03T04:28:54Z

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+
+        let int8 = number.int8Value
+        guard NSNumber(value: int8) == number else {
+            throw DecodingError.dataCorrupted(DecodingError.Context(codingPath: self.codingPath, debugDescription: "Parsed JSON number <\(number)> does not fit in \(type)."))


"Parsed JSON" no longer applies. Maybe "Firestore Datasnapshot"?

wilhuff · 2018-03-03T04:31:36Z

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+            // * If it was a Float, you will get back the precise value
+            // * If it was a Double or Decimal, you will get back the nearest approximation if it will fit
+            let double = number.doubleValue
+            guard abs(double) <= Double(Float.greatestFiniteMagnitude) else {


@mikelehen What's the right way to respond to this based on our Android implementation?

It seems like CustomClassMapper just relies on "cast" operations (Number#floatValue): non-failing, saturating casts that would have e.g. positive values beyond ~3.4 x 10^38 saturate to Infinity when assigned to a model value containing a Float instead of a double. Isn't a similar response appropriate here?

wilhuff

In general the encoder stuff is far less egregious and doesn't seem like it needs much work (aside from naming).

wilhuff · 2018-03-03T04:58:00Z

Firestore/Swift/Source/Codable/FirestoreEncoder.swift

+}
+
+@available(swift 4.0.0)
+fileprivate class _FirestoreEncoder : Encoder {


The names in here should parallel the resolution on the decoder (i.e. Document*)

phatmann · 2018-04-13T22:33:38Z

Any progress on this? Would be great to have this part of the SDK.

siyao1030 · 2018-05-07T04:09:19Z

Thanks for the hard work @alickbass @wilhuff!
Do you have any updates on the status of this issue/pull request? Would love to use it soon!

jorgej-ramos · 2018-07-03T13:30:31Z

A great work @alickbass @wilhuff. Thank you both.
Any updates about this PR? It's a really needed feature.

swftvsn · 2018-07-25T19:27:54Z

We really need this for our iOS project, so please please keep up the good work @alickbass @wilhuff!
Any updates about this PR? Can I help?

andreasmpet · 2018-11-05T09:36:04Z

Hi guys! What's the progress on this beautiful and highly anticipated feature?

jonathansolorzn · 2018-11-08T03:34:33Z

Any update on this?

paulb777 · 2018-12-13T21:42:38Z

Closing and continuing in #2178

wilhuff and others added 20 commits February 19, 2018 16:27

Add Firestore_SwiftTests_iOS target to Xcode

3598b86

Clean up generated Firestore_SwiftTests_iOS test target

959cc57

* Move Example/Firestore_SwiftTests_iOS to Swift/Tests * Delete generated test case * Update Info.plist location to match

Add FirebaseFirestoreSwift podspec

ad832a1

Add Firestore_SwiftTests_iOS to the Podfile

24daa10

Also corresponding xcodeproj changes

Add CodableGeoPoint

6fc2dd4

Move Firestore_SwiftTests_iOS to the top-level of the Podfile

b5d72b0

Add CodableGeoPointTests to the project

c719b85

Add FirebaseEncoder

448bbbf

Update Encoder

557883e

Fix typo

86bbf4b

Make the extension available only from swift 4

cb7e0a8

Add FirestoreDecoder

6303624

Make all the types in Encoder as available from swift 4

f20e338

Add extension to document snapshot

9f13acf

Refactor to private Decoder and Encoder

a423896

Add internal type Firestore.Decoder

dcc4b4a

Add internal Firestore.Encoder type

493a157

Move to proper sub folder

c9b35c7

Confirm DocumentReference to Codable

05e20fd

Confirm FieldValue to Encodable protocol

735331d

googlebot added the cla: no label Feb 22, 2018

googlebot added cla: yes and removed cla: no labels Feb 22, 2018

paulb777 added the api: firestore label Feb 23, 2018

wilhuff suggested changes Feb 24, 2018

View reviewed changes

Oleksii Dykan added 2 commits February 26, 2018 21:58

Fix copyright

889f29a

Merge master into codable-firestore

4fed07f

Oleksii Dykan added 2 commits February 27, 2018 15:56

Refactor to require value private function

3ea3eba

Refactor to private methods to require values

d23c201

googlebot added cla: no and removed cla: yes labels Feb 27, 2018

wilhuff changed the base branch from wilhuff/codable to master March 1, 2018 04:04

wilhuff added cla: yes and removed cla: no labels Mar 3, 2018

wilhuff reviewed Mar 3, 2018

View reviewed changes

wilhuff mentioned this pull request Jul 30, 2018

Add build infrastructure for Codable support in Firestore #815

Merged

phatmann mentioned this pull request Oct 6, 2018

Support keyDecodingStrategy property for snake case field names alickbass/CodableFirebase#34

Open

paulb777 mentioned this pull request Dec 12, 2018

Continue Firestore Codable #2178

Closed

paulb777 closed this Dec 13, 2018

paulb777 mentioned this pull request Dec 19, 2018

Demonstration of WIP Firestore Codable firebase/quickstart-ios#609

Closed

This was referenced Jan 3, 2019

Squash #2178 to continue Firestore Codable #2228

Merged

Continue Firestore Codable 3 #2229

Closed

This was referenced Mar 26, 2019

Swift Codable and Extension DocumentSnapshot #2635

Closed

Swift Codable and Extension DocumentSnapshot #2636

Closed

firebase locked and limited conversation to collaborators Nov 7, 2019


		import FirebaseFirestore

		enum FirestoreDecodingError: Error {

Codable firestore #838

Codable firestore #838

Conversation

alickbass commented Feb 22, 2018 • edited Loading

googlebot commented Feb 22, 2018

alickbass commented Feb 22, 2018

googlebot commented Feb 22, 2018

wilhuff left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

googlebot commented Feb 27, 2018

alickbass commented Feb 27, 2018

wilhuff commented Feb 28, 2018 via email

wilhuff commented Mar 1, 2018

wilhuff left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wilhuff left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

phatmann commented Apr 13, 2018

siyao1030 commented May 7, 2018 • edited Loading

jorgej-ramos commented Jul 3, 2018

swftvsn commented Jul 25, 2018

andreasmpet commented Nov 5, 2018

jonathansolorzn commented Nov 8, 2018

paulb777 commented Dec 13, 2018

alickbass commented Feb 22, 2018 •

edited

Loading

siyao1030 commented May 7, 2018 •

edited

Loading