Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: improve transaction tests to use async/await #7759

Merged
merged 10 commits into from
Jan 3, 2022
Merged

test: improve transaction tests to use async/await #7759

merged 10 commits into from
Jan 3, 2022

Conversation

cbaker6
Copy link
Contributor

@cbaker6 cbaker6 commented Jan 2, 2022

New Pull Request Checklist

Issue Description

Make the transactions tests (ParseServerRESTController.spec.js and batch.spec.js) use async await.

@davimacedo I cleaned up the ParseServerRESTController.spec.js and ran into a problem with one of the transactions tests for mongo. I noticed in #5849 (comment) you mentioned mongo needed a replicaset when using transactions.

The problem I ran into is the extra reconfiguration of the server:

it('should generate separate session for each call', async () => {
await reconfigureServer();

seems to override the one configuration that uses the replicaSet:

describe('transactions', () => {
beforeEach(async () => {
await TestUtils.destroyAllDataPermanently(true);
if (
semver.satisfies(process.env.MONGODB_VERSION, '>=4.0.4') &&
process.env.MONGODB_TOPOLOGY === 'replicaset' &&
process.env.MONGODB_STORAGE_ENGINE === 'wiredTiger'
) {
await reconfigureServer({
databaseAdapter: undefined,
databaseURI:
'mongodb://localhost:27017/parseServerMongoAdapterTestDatabase?replicaSet=replicaset',
});

Related issue: #7180

Approach

When I remove the extra reconfigure and use the replicaset one, the following failures occur on Mongo (the Mongo CI that doesn't use replicaSet seems to pass in any case):

1) ParseServerRESTController transactions should generate separate session for each call
  - Expected false to equal true.
  - Expected false to equal true.
  - Expected false to equal true.
  - Expected false to equal true.

Note that removing the extra reconfigure on this test seems to work:

it('should handle a batch request with transaction = true', async done => {
await reconfigureServer();

My initial thoughts are the extra reconfigure shouldn't be there and the tests might be catching an actual bug. I don't know enough about Mongo to know if separate sessions are required for transactions.

Also, it looks like the exact set of transaction tests are in ParseServerRESTController.spec.js and batch.spec.js (the same problem I mentioned above exists in both). The difference I see is one is using the ParseServerRESTController in tests. It seems both tests randomly fail with connection issues (I'm not able to replicate this locally) on Postgres (so I've disabled the RESTController transaction tests on Postgres to reduce the amount of random failures in the CI).

if (
(semver.satisfies(process.env.MONGODB_VERSION, '>=4.0.4') &&
process.env.MONGODB_TOPOLOGY === 'replicaset' &&
process.env.MONGODB_STORAGE_ENGINE === 'wiredTiger') ||
process.env.PARSE_SERVER_TEST_DB === 'postgres'
) {
describe('transactions', () => {
beforeEach(async () => {
await TestUtils.destroyAllDataPermanently(true);
if (
semver.satisfies(process.env.MONGODB_VERSION, '>=4.0.4') &&
process.env.MONGODB_TOPOLOGY === 'replicaset' &&
process.env.MONGODB_STORAGE_ENGINE === 'wiredTiger'
) {
await reconfigureServer({
databaseAdapter: undefined,
databaseURI:
'mongodb://localhost:27017/parseServerMongoAdapterTestDatabase?replicaSet=replicaset',
});
} else {
await reconfigureServer();
}
});
it('should handle a batch request with transaction = true', async done => {
await reconfigureServer();
const myObject = new Parse.Object('MyObject'); // This is important because transaction only works on pre-existing collections
myObject
.save()
.then(() => {
return myObject.destroy();
})
.then(() => {
spyOn(databaseAdapter, 'createObject').and.callThrough();
request({
method: 'POST',
headers: headers,
url: 'http://localhost:8378/1/batch',
body: JSON.stringify({
requests: [
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 'value2' },
},
],
transaction: true,
}),
}).then(response => {
expect(response.data.length).toEqual(2);
expect(response.data[0].success.objectId).toBeDefined();
expect(response.data[0].success.createdAt).toBeDefined();
expect(response.data[1].success.objectId).toBeDefined();
expect(response.data[1].success.createdAt).toBeDefined();
const query = new Parse.Query('MyObject');
query.find().then(results => {
expect(databaseAdapter.createObject.calls.count() % 2).toBe(0);
for (let i = 0; i + 1 < databaseAdapter.createObject.calls.length; i = i + 2) {
expect(databaseAdapter.createObject.calls.argsFor(i)[3]).toBe(
databaseAdapter.createObject.calls.argsFor(i + 1)[3]
);
}
expect(results.map(result => result.get('key')).sort()).toEqual([
'value1',
'value2',
]);
done();
});
});
});
});
it('should not save anything when one operation fails in a transaction', async () => {
const myObject = new Parse.Object('MyObject'); // This is important because transaction only works on pre-existing collections
await myObject.save();
await myObject.destroy();
try {
await request({
method: 'POST',
headers: headers,
url: 'http://localhost:8378/1/batch',
body: JSON.stringify({
requests: [
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 10 },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 10 },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 10 },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 10 },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 10 },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 10 },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 10 },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 10 },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 10 },
},
],
transaction: true,
}),
});
} catch (error) {
expect(error).toBeDefined();
const query = new Parse.Query('MyObject');
const results = await query.find();
expect(results.length).toBe(0);
}
});
it('should generate separate session for each call', async () => {
await reconfigureServer();
const myObject = new Parse.Object('MyObject'); // This is important because transaction only works on pre-existing collections
await myObject.save();
await myObject.destroy();
const myObject2 = new Parse.Object('MyObject2'); // This is important because transaction only works on pre-existing collections
await myObject2.save();
await myObject2.destroy();
spyOn(databaseAdapter, 'createObject').and.callThrough();
let myObjectCalls = 0;
Parse.Cloud.beforeSave('MyObject', async () => {
myObjectCalls++;
if (myObjectCalls === 2) {
try {
await request({
method: 'POST',
headers: headers,
url: 'http://localhost:8378/1/batch',
body: JSON.stringify({
requests: [
{
method: 'POST',
path: '/1/classes/MyObject2',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject2',
body: { key: 10 },
},
{
method: 'POST',
path: '/1/classes/MyObject2',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject2',
body: { key: 10 },
},
{
method: 'POST',
path: '/1/classes/MyObject2',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject2',
body: { key: 10 },
},
{
method: 'POST',
path: '/1/classes/MyObject2',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject2',
body: { key: 10 },
},
{
method: 'POST',
path: '/1/classes/MyObject2',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject2',
body: { key: 10 },
},
{
method: 'POST',
path: '/1/classes/MyObject2',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject2',
body: { key: 10 },
},
{
method: 'POST',
path: '/1/classes/MyObject2',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject2',
body: { key: 10 },
},
{
method: 'POST',
path: '/1/classes/MyObject2',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject2',
body: { key: 10 },
},
{
method: 'POST',
path: '/1/classes/MyObject2',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject2',
body: { key: 10 },
},
],
transaction: true,
}),
});
fail('should fail');
} catch (e) {
expect(e).toBeDefined();
}
}
});
const response = await request({
method: 'POST',
headers: headers,
url: 'http://localhost:8378/1/batch',
body: JSON.stringify({
requests: [
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject',
body: { key: 'value2' },
},
],
transaction: true,
}),
});
expect(response.data.length).toEqual(2);
expect(response.data[0].success.objectId).toBeDefined();
expect(response.data[0].success.createdAt).toBeDefined();
expect(response.data[1].success.objectId).toBeDefined();
expect(response.data[1].success.createdAt).toBeDefined();
await request({
method: 'POST',
headers: headers,
url: 'http://localhost:8378/1/batch',
body: JSON.stringify({
requests: [
{
method: 'POST',
path: '/1/classes/MyObject3',
body: { key: 'value1' },
},
{
method: 'POST',
path: '/1/classes/MyObject3',
body: { key: 'value2' },
},
],
}),
});
const query = new Parse.Query('MyObject');
const results = await query.find();
expect(results.map(result => result.get('key')).sort()).toEqual(['value1', 'value2']);
const query2 = new Parse.Query('MyObject2');
const results2 = await query2.find();
expect(results2.length).toEqual(0);
const query3 = new Parse.Query('MyObject3');
const results3 = await query3.find();
expect(results3.map(result => result.get('key')).sort()).toEqual(['value1', 'value2']);
expect(databaseAdapter.createObject.calls.count() >= 13).toEqual(true);
let transactionalSession;
let transactionalSession2;
let myObjectDBCalls = 0;
let myObject2DBCalls = 0;
let myObject3DBCalls = 0;
for (let i = 0; i < databaseAdapter.createObject.calls.count(); i++) {
const args = databaseAdapter.createObject.calls.argsFor(i);
switch (args[0]) {
case 'MyObject':
myObjectDBCalls++;
if (!transactionalSession || (myObjectDBCalls - 1) % 2 === 0) {
transactionalSession = args[3];
} else {
expect(transactionalSession).toBe(args[3]);
}
if (transactionalSession2) {
expect(transactionalSession2).not.toBe(args[3]);
}
break;
case 'MyObject2':
myObject2DBCalls++;
if (!transactionalSession2 || (myObject2DBCalls - 1) % 9 === 0) {
transactionalSession2 = args[3];
} else {
expect(transactionalSession2).toBe(args[3]);
}
if (transactionalSession) {
expect(transactionalSession).not.toBe(args[3]);
}
break;
case 'MyObject3':
myObject3DBCalls++;
expect(args[3]).toEqual(null);
break;
}
}
expect(myObjectDBCalls % 2).toEqual(0);
expect(myObjectDBCalls > 0).toEqual(true);
expect(myObject2DBCalls % 9).toEqual(0);
expect(myObject2DBCalls > 0).toEqual(true);
expect(myObject3DBCalls % 2).toEqual(0);
expect(myObject3DBCalls > 0).toEqual(true);
});
});
}
});

TODOs before merging

  • Add tests
  • A changelog entry is created automatically using the pull request title (do not manually add a changelog entry)

@parse-github-assistant
Copy link

parse-github-assistant bot commented Jan 2, 2022

Thanks for opening this pull request!

  • 🎉 We are excited about your hands-on contribution!

@codecov
Copy link

codecov bot commented Jan 2, 2022

Codecov Report

Merging #7759 (4fb6fbe) into alpha (d05fb9f) will decrease coverage by 0.00%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##            alpha    #7759      +/-   ##
==========================================
- Coverage   93.94%   93.93%   -0.01%     
==========================================
  Files         183      183              
  Lines       13660    13660              
==========================================
- Hits        12833    12832       -1     
- Misses        827      828       +1     
Impacted Files Coverage Δ
src/RestWrite.js 93.88% <0.00%> (-0.16%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d05fb9f...4fb6fbe. Read the comment docs.

@cbaker6 cbaker6 changed the title refactor: improve transaction tests to use async/await test: improve transaction tests to use async/await Jan 2, 2022
@cbaker6
Copy link
Contributor Author

cbaker6 commented Jan 2, 2022

@mtrezza this is ready for review. @davimacedo may want to chime in my comment above.

@cbaker6
Copy link
Contributor Author

cbaker6 commented Jan 2, 2022

@mtrezza looks like we are seeing a higher probability of the test suite passing already. Will increase a little more after this PR.

Note the possibility of the bug doesn't need to be fixed in this PR, but probably should be looked at in the future.

spec/ParseServerRESTController.spec.js Outdated Show resolved Hide resolved
spec/batch.spec.js Show resolved Hide resolved
Copy link
Member

@mtrezza mtrezza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, a great clean-up!

@mtrezza mtrezza merged commit a43638f into parse-community:alpha Jan 3, 2022
@cbaker6 cbaker6 deleted the transactions branch January 3, 2022 23:55
@parseplatformorg
Copy link
Contributor

🎉 This change has been released in version 5.0.0-alpha.17

@parseplatformorg parseplatformorg added the state:released-alpha Released as alpha version label Jan 13, 2022
@parseplatformorg
Copy link
Contributor

🎉 This change has been released in version 5.0.0-beta.10

@parseplatformorg parseplatformorg added the state:released-beta Released as beta version label Mar 15, 2022
@parseplatformorg
Copy link
Contributor

🎉 This change has been released in version 5.1.0

@parseplatformorg parseplatformorg added the state:released Released as stable version label Mar 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
state:released Released as stable version state:released-alpha Released as alpha version state:released-beta Released as beta version
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants