Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Implement BundleBuilder #1041

Merged
merged 16 commits into from
May 7, 2020
Merged

feat: Implement BundleBuilder #1041

merged 16 commits into from
May 7, 2020

Conversation

wu-hui
Copy link
Contributor

@wu-hui wu-hui commented Apr 23, 2020

Implements BundleBuilder and added system tests around bundles.

JustinBeckwith and others added 5 commits April 12, 2020 02:16
* Update from googleapis.git

* Add bundle.proto.

* Move bundle.proto to a different folder.

* Move the proto file again

* Update package name.
* Update from googleapis.git

* Add bundle.proto.

* Move bundle.proto to a different folder.

* Add support to convert a Query to a BundledQuery.

* Move the proto file again

* Move the proto file again

* Update package name.

* Update to use new package name.
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Apr 23, 2020
@codecov
Copy link

codecov bot commented Apr 23, 2020

Codecov Report

Merging #1041 into wuandy/Bundles will increase coverage by 1.13%.
The diff coverage is 98.22%.

Impacted file tree graph

@@                Coverage Diff                 @@
##           wuandy/Bundles    #1041      +/-   ##
==================================================
+ Coverage           97.45%   98.59%   +1.13%     
==================================================
  Files                  25       26       +1     
  Lines               16044    16883     +839     
  Branches             1217     1342     +125     
==================================================
+ Hits                15636    16645    +1009     
+ Misses                405      235     -170     
  Partials                3        3              
Impacted Files Coverage Δ
dev/src/field-value.ts 98.13% <85.71%> (+<0.01%) ⬆️
dev/src/bundle.ts 93.75% <93.75%> (ø)
dev/src/types.ts 99.69% <94.11%> (-0.31%) ⬇️
dev/src/index.ts 98.30% <94.44%> (-0.33%) ⬇️
dev/src/document.ts 98.97% <97.67%> (+0.31%) ⬆️
dev/src/v1/firestore_client.ts 97.64% <98.96%> (+2.84%) ⬆️
dev/src/v1beta1/firestore_client.ts 97.38% <99.02%> (+2.89%) ⬆️
dev/src/v1/firestore_admin_client.ts 97.71% <99.07%> (+6.58%) ⬆️
dev/src/reference.ts 99.88% <99.25%> (+0.15%) ⬆️
dev/src/convert.ts 98.27% <100.00%> (-0.03%) ⬇️
... and 19 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c119484...1d378fd. Read the comment docs.

Copy link
Contributor

@schmidt-sebastian schmidt-sebastian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I provided some initial feedback (have yet to look at the tests). Please note that system tests should be added on top of existing unit tests and not act as a replacement. It is much easier to detect failures and prescribe expected behavior with unit tests.

import api = google.firestore.v1;

/**
* Builds a Firestore data bundle with results from given queries and specified
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Builds a Firestore data bundle with results from given queries and specified
* Builds a Firestore data bundle with results from the given queries and specified

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

* Builds a Firestore data bundle with results from given queries and specified
* documents.
*
* For documents in scope for multiple queries, the latest read version will
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* For documents in scope for multiple queries, the latest read version will
* For documents in scope for multiple queries, only the latest version will

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

* be included in the bundle.
*/
export class BundleBuilder {
// DocumentReference and Query added to the bundle.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// DocumentReference and Query added to the bundle.
// DocumentReferences and Queries added to the bundle.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

export class BundleBuilder {
// DocumentReference and Query added to the bundle.
private docRefs: DocumentReference[] = [];
private queries: Map<string, Query> = new Map<string, Query>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you come up with a small inline doc string to describe the key of the Map?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

private docRefs: DocumentReference[] = [];
private queries: Map<string, Query> = new Map<string, Query>();

// DocumentSnapshot and QuerySnapshot added to the bundle.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// DocumentSnapshot and QuerySnapshot added to the bundle.
// DocumentSnapshots and QuerySnapshots added to the bundle.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

}

for (const [name, query] of Array.from(this.queries)) {
if (this.namedQueries.has(name)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be part of the add call.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


const readable = new Readable({
objectMode: false,
read: async size => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read cannot be async as far as I can tell. It needs to return the data that is being read.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note this is the read from ReadableOptions, and its return type is a void.

However I will switch to asyncIterator anyways, so it does not matter any more.

};
this.pushLengthPrefixedString(
readable,
JSON.stringify({metadata} as firestore.IBundleElement)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cast is not valid (since metadata is IBundleMetadata, but not {metadata}). You can just drop the cast.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is valid but unnecessary, because it is casting to IBundleElement.

I slightly prefer to keep these though, because it can serve as a reminder that it is a IBundleElement that gets saved in the bundle.

dev/src/bundle.ts Outdated Show resolved Hide resolved
dev/src/index.ts Show resolved Hide resolved
@wu-hui wu-hui force-pushed the wuandy/Bundles branch 2 times, most recently from 5023791 to c119484 Compare April 24, 2020 21:32
@wu-hui wu-hui removed their assignment Apr 27, 2020

/**
* Builds a Firestore data bundle with results from the given queries and specified
* documents.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should eventually expand on the docs to say how bundles can be used. This can be deferred until we have a client SDK implemented.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree.

dev/src/bundle.ts Outdated Show resolved Hide resolved
constructor(private bundleId: string) {}

add(documentSnapshot: DocumentSnapshot): BundleBuilder;
add(queryName: string, querySnap: QuerySnapshot): BundleBuilder;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/querySnap/querySnapshot/ for consistency.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

* Both the documents data and the query read time will be included in the bundle.
*
* @param {DocumentSnapshot | string=} documentOrName A document snapshot to add or a name of a query.
* @param {Query?=} querySnapshot A query snapshot to add to the bundle, if provided.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/Query?=/Query=

JSDoc for undefined is just =

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

* @example
* let bundle = firestore.bundle('data-bundle');
* const bundleStream =
* await bundle.add(await firestore.doc('abc/123').get()) // Add a document
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you change this to move the calls to bundle.add() to their own lines?
Also, you should not need to use await here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


describe('Bundle building', () => {
// Helper function for testing: converts "13{..}43{..}51{..}" to [[13,43,51],[obj,obj,obj]]
function bundleToLengthsAndObject(bundle: string): [number[], object[]] {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make this easier to digest, consider returning an array with objects: Array<{length: number, content: IBundleElement}>

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

let insertComma = false;

// Turn "13{..}43{..}51{..}" into "{..},{..},{..}"
for (const c of bundle) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a surprising way to read this data. You should take advantage of the data format you build and read this in two phases:

  • Extract the length by reading until you hit "{".
  • Reading the next "length" bytes

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to avoid that actually. What you described is the way client SDKs would be using. I want the tests to be using different (inefficient) ways such that two implementations can cross check (not obvious here, but I imagine this function will be copied over to web SDK).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't yet convinced myself that the logic here adds more test coverage, but I am convinced that it is much harder to parse than the code that we expect users to write would be. In fact, if we don't think that our users will parse bundles like this, then we shouldn't either. We need to validate that the format we are using doesn't just match our specification, but can actually be consumed as intended by our users.

You could use this function in one (or if you want to maybe even most) unit tests, but you should at least have some coverage that uses the "straightforward" way to parse this data. FWIW, there are even packages that you can use: https://www.npmjs.com/package/length-prefixed-json-stream

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks!


afterEach(() => verifyInstance(firestore));

it('succeeds to read length prefixed json with testing function', () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we make this a unit test and move bundleToLengthsAndObject to helpers.ts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason I keep it here is because this is only used in tests, not prod code.

This particular one is definitely a unit test, but because it is only used here in the system tests, we would like to keep them together. It's easier to people to read IMO.

As I suggested below, maybe grouping all bundle system tests (and this one) in a separate file will help?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My apologies..done.

dev/system-test/firestore.ts Show resolved Hide resolved
]);
});

it('succeeds with document snapshots', async () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these tests should be unit tests in their own file (bundle.ts). We can cherry-pick some tests and duplicate them as system tests.

We also need at least one test that consumes stream() directly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually think in this case, integration tests are easier to write than unit tests. All snapshots are constructed nicely, with read time populated correctly. Unit tests are be harder to read IMHO.

Do you keep them as system tests but move them to a separate file will make it better?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

System tests are usually easier to write, but they:

  • are harder to debug
  • are not deterministic
  • make it impossible to test some edge cases
  • cannot be run without Production credentials
  • don't count against code coverage

Please add unit tests :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved some to unit tests, thanks.

@wu-hui wu-hui assigned wu-hui and unassigned schmidt-sebastian Apr 30, 2020
@wu-hui wu-hui assigned schmidt-sebastian and unassigned wu-hui Apr 30, 2020

import api = google.firestore.v1;

/**
* Builds a Firestore data bundle with results from the given queries and specified
* documents.
*
* For documents in scope for multiple queries, only the latest version will
* For documents that are included multiple times, only the latest version will
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent of how we decide to implement this, should we relax this documentation to allow us to send multiple document version if the are processed "oldest to newest" first?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed this section.

* const querySnapshot = await firestore.collection('coll').get();
* const bundleStream = bundle.add(docSnapshot) // Add a document
* .add('coll-query', querySnapshot) // Add a named query.
* .stream();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* .stream();
* bundle.add(docSnapshot) // Add a document
* bundle.add('coll-query', querySnapshot) // Add a named query.
* const bundleStream.stream();

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

if (documentOrName instanceof DocumentSnapshot) {
if (
documentOrName instanceof DocumentSnapshot &&
validateOptional(querySnapshot, {optional: true})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

validateOptional only validates that querySnapshot can be undefined not that it needs to be undefined when documentOrName is a DocumentSnapshot.

This validation should also be moved into the if branch.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rewrote to check arguments.length instead..I found it to be more readable.

dev/src/bundle.ts Show resolved Hide resolved
@@ -113,8 +118,12 @@ export class BundleBuilder {
}
}

private lengthPrefixedBuffer(payload: string): Buffer {
const buffer = Buffer.from(payload, 'utf-8');
// Converts a IBundleElement to a Buffer whose content is the length prefixed JSON representation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For method comments, prefer JSDoc:

/**
 * Converts a IBundleElement...
 * @private
 */

Note that you have to add @private to make sure this doesn't show up in our public docs. JSDoc doesn't know that this method is TypeScript private.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

dev/src/bundle.ts Show resolved Hide resolved

afterEach(() => verifyInstance(firestore));

it('succeeds to read length prefixed json with testing function', () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let insertComma = false;

// Turn "13{..}43{..}51{..}" into "{..},{..},{..}"
for (const c of bundle) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't yet convinced myself that the logic here adds more test coverage, but I am convinced that it is much harder to parse than the code that we expect users to write would be. In fact, if we don't think that our users will parse bundles like this, then we shouldn't either. We need to validate that the format we are using doesn't just match our specification, but can actually be consumed as intended by our users.

You could use this function in one (or if you want to maybe even most) unit tests, but you should at least have some coverage that uses the "straightforward" way to parse this data. FWIW, there are even packages that you can use: https://www.npmjs.com/package/length-prefixed-json-stream

@wu-hui wu-hui removed their assignment May 5, 2020
Copy link
Contributor

@schmidt-sebastian schmidt-sebastian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some belated API suggestions.... but otherwise this looks good.

* @param arg The argument name or argument index (for varargs methods).
* @param value The input to validate.
*/
export function validateDocumentSnapshot(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: No need to export. We generally also have helpers at the bottom of the model.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

* @param arg The argument name or argument index (for varargs methods).
* @param value The input to validate.
*/
export function validateQuerySnapshot(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

* Adds a Firestore document snapshot or query snapshot to the bundle.
* Both the documents data and the query read time will be included in the bundle.
*
* @param {DocumentSnapshot | string=} documentOrName A document snapshot to add or a name of a query.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/string=/string/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

* @returns {BundleBuilder} This instance.
*
* @example
* let bundle = firestore.bundle('data-bundle');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/let/const/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

* const querySnapshot = await firestore.collection('coll').get();
* bundle.add(docSnapshot); // Add a document.
* bundle.add('coll-query', querySnapshot) // Add a named query.
* const bundleBuffer = await bundle.build();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't need to be "awaited" anymore.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

dev/system-test/firestore.ts Show resolved Hide resolved
readTime: snap1.readTime.toProto().timestampValue,
},
snap1.toDocumentProto(),
]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like it should be two different expect calls.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

snap1.toDocumentProto(),
]);
});

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add a test that a second call to build() contains the first round of documents as well as the ones added after the first call?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

bundle: Buffer
): Promise<Array<firestore.IBundleElement>> {
const result: Array<firestore.IBundleElement> = [];
const readable = new PassThrough();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const readable = new PassThrough();
const readable = Readable.from(bundle);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When i try to use from i get "TypeError: stream_1.Readable.from is not a function". Not sure what is wrong.

Also, from the node documentation: "Calling Readable.from(string) or Readable.from(buffer) will not have the strings or buffers be iterated to match the other streams semantics for performance reasons.".

@@ -16,3 +16,4 @@

// TODO(mrschmidt): Come up with actual definitions for these modules.
declare module 'functional-red-black-tree';
declare module 'length-prefixed-json-stream';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see what you did there, adding work to my TODO :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Damn, I should have picked someone else.

@wu-hui wu-hui assigned schmidt-sebastian and unassigned wu-hui May 6, 2020
name: this._ref.formattedName,
fields: this._fieldsProto!,
};
const result: api.IDocument = {name: this._ref.formattedName};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should do this conversion in the caller. The backend has a specified way to return "NoDocuments": https://source.corp.google.com/piper///depot/google3/google/firestore/v1beta1/firestore.proto;l=496

In the calling code, you could do something like

if (doc.exists) ? { document: doc.toProto()} : {name: this._ref.formattedName}

In the Mobile SDKs, you want to use as much of the existing deserialization logic as possible. For that, you want to match your Proto format. From looking at your Proto, it seems that you could also add an exists to IBundledDocumentMetadata and omit the document?

Drive by comment on Proto: We may also want to use "name" for document keys since this is what the backend uses.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added exists and changed to name.

dev/system-test/firestore.ts Show resolved Hide resolved
@@ -24,7 +24,7 @@ import {
} from './util/helpers';
import IBundleElement = firestore.IBundleElement;

describe('Bundle Buidler', () => {
describe.only('Bundle Buidler', () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you merge @thebrianchen's changes form the Node10 branch into your PR you will get a lint check that will catch this :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a lot changes from bulk writer too. I will do the merge later..

arg: string | number,
value: unknown
): void {
function validateQuerySnapshot(arg: string | number, value: unknown): void {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do move to bottom of the module to match the code layout in the rest of the repo.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor Author

@wu-hui wu-hui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also added a version to metadata.

arg: string | number,
value: unknown
): void {
function validateQuerySnapshot(arg: string | number, value: unknown): void {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

name: this._ref.formattedName,
fields: this._fieldsProto!,
};
const result: api.IDocument = {name: this._ref.formattedName};
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added exists and changed to name.

@@ -24,7 +24,7 @@ import {
} from './util/helpers';
import IBundleElement = firestore.IBundleElement;

describe('Bundle Buidler', () => {
describe.only('Bundle Buidler', () => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a lot changes from bulk writer too. I will do the merge later..

@wu-hui wu-hui assigned schmidt-sebastian and unassigned wu-hui May 7, 2020
@wu-hui
Copy link
Contributor Author

wu-hui commented May 7, 2020

Also added a version to metadata.

One more thing: the change to make it non-public will be separate, such that it can be easily reverted.

Copy link
Contributor

@schmidt-sebastian schmidt-sebastian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM minus missing unit test for "addNamedQuery".

@@ -85,6 +88,9 @@ message BundleMetadata {

// Time at which the documents snapshot is taken for this bundle.
google.protobuf.Timestamp create_time = 2;

// The version of bundle.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: "The schema version of the bundle."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -109,10 +85,11 @@ export class BundleBuilder {
private addBundledDocument(snap: DocumentSnapshot) {
const docProto = snap.toDocumentProto();
this.documents.set(snap.id, {
document: docProto,
document: snap.exists ? docProto : undefined,
Copy link
Contributor

@schmidt-sebastian schmidt-sebastian May 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I don't know what snap.toDocumentProto() does if the document doesn't exist. I would not need to check if you called it here inline. :)

Optional.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is another docProto.name below, so I prefer to keeping it this way.

@@ -174,16 +152,19 @@ export class BundleBuilder {
for (const bundledDocument of this.documents.values()) {
const documentMetadata: firestore.IBundledDocumentMetadata =
bundledDocument.metadata;
const document: api.IDocument = bundledDocument.document;
const document = bundledDocument.document;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Move to line 162.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@wu-hui wu-hui assigned schmidt-sebastian and unassigned wu-hui May 7, 2020
Copy link
Contributor

@schmidt-sebastian schmidt-sebastian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the test. Future you (and me) will thank you again.

@wu-hui wu-hui merged commit 98f58eb into wuandy/Bundles May 7, 2020
wu-hui added a commit that referenced this pull request May 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants