Using Promise.all inside transactions errors with "Cannot pin multiple connections to the same session"

[fiws]

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the bug has not already been reported

Mongoose version

8.4.0

Node.js version

22.0.0

MongoDB server version

7.3.2

Typescript version (if applicable)

No response

Description

Using running mongo queries/commands in Promise.all wrapped in a transaction throws the following error:

TypeError: Cannot pin multiple connections to the same session

I tested this using an atlas serverless database. Is it not allowed to run queries in parallel when using a transaction?

Steps to Reproduce

import mongoose from 'mongoose';
mongoose.connect(process.env.MONGO_URI || 'mongodb://localhost/transaction-test');

const userSchema = new mongoose.Schema({
  name: String,
});

const User = mongoose.model('User', userSchema);

test('user transaction test', async () => {
  await Instance.db.transaction(async session => {
    return Promise.all([
      User.findOne({}, null, { session }),
      User.findOne({}, null, { session }),
    ]);
  });
});

Expected Behavior

No error.

[fiws]

I don't know if this is related but in a more complex application we did also get worrisome side-effects from these kind of transactions, where a findById or findOne looking up a document that has been deleted in the previous transactions suddenly returns something like this:

{
  ok: 1,
  '$clusterTime': {
    clusterTime: new Timestamp({ t: 1715964070, i: 18 }),
    signature: {
      hash: Binary.createFromBase64('QvEPV74zpqEMIRD5lLDYPkfxsV8=', 0),
      keyId: new Long('7330005257113567234')
    }
  },
  operationTime: new Timestamp({ t: 1715964070, i: 18 })
}

This looks like some mongo internals that should never be returned by findById.

Might be another bug tho, and I'm still trying to get this into a small reproducible snippet.

[IslandRhythms]

I noticed in your repo script you're using a local connection. Transactions can only be used on replica sets. Just want to confirm that you tested on a replica set before creating the issue.

[fiws]

Yep. I used a serverless MongoDB atlas for testing.

…

On Tue 21. May 2024 at 21:10, Daniel Diaz ***@***.***> wrote: I noticed in your repo script you're using a local connection. Transactions can only be used on replica sets. Just want to confirm that you tested on a replica set before creating the issue. — Reply to this email directly, view it on GitHub <#14603 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2AQJRSIAEVPZHIE6JQ6OTZDOL4DAVCNFSM6AAAAABH4OYB4KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRTGI3TKNJZGU> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[vkarpov15]

It seems like this issue only occurs with Atlas serverless instances, the following script executes fine against a local mongodb replica set:

const mongoose = require('mongoose');

mongoose.connect('mongodb://127.0.0.1:27017,127.0.0.1:27018/mongoose_test');

const userSchema = new mongoose.Schema({
  name: String,
});


void async function main() {

  const User = mongoose.model('User', userSchema);
  await User.create({ name: 'test' });

  await mongoose.connection.transaction(async session => {
    const docs = await Promise.all([
      User.findOne({}, null, { session }),
      User.findOne({}, null, { session }),
    ]);
    console.log(docs);
  });

  console.log('Done');
}();

Output:

$ node gh-14603.js 
[
  {
    _id: new ObjectId('6650a6d44061785454c11001'),
    name: 'test',
    __v: 0
  },
  {
    _id: new ObjectId('6650a6d44061785454c11001'),
    name: 'test',
    __v: 0
  }
]
Done

[vkarpov15]

I opened issue in MongoDB Jira here: https://jira.mongodb.org/browse/NODE-6190

[vkarpov15]

As a temporary workaround, you can use the following:

  await mongoose.connection.transaction(async session => {
    const docs = await Promise.all([
      User.findOne({}, null, { session }),
      sleep(0).then(() => User.findOne({}, null, { session })),
    ]);
    console.log(docs);
  });

async function sleep(ms) {
  await new Promise(resolve => setTimeout(resolve, ms));
}

[vkarpov15]

It looks like this is expected behavior. From the JIRA ticket:

Here's the docs link. I'm unfortunately going to have to close this issue: Mongoose can't support parallel requests with the same transaction if the MongoDB server doesn't support it.

[fiws]

Thanks for researching this and creating the upstream issue.

Maybe some documentation (similar to the updated mongodb doc) on mongoose for this would be nice. The big problem is that this is undefined behaviour. It does not throw the error every time. We have Promise.all calls that seemingly work (in production). We noticed super weird behaviour recently though, which is the whole reason why i created this issue.

[maxeth]

With .withTransaction() this issue only appears on Serverless instances when the first operation of the transaction is part of the Promise.all().

So this works

      const a = await ModelA.find({}, {}, { session });
      const [b,c] = await Promise.all([...])

while it doesn't if a is included in the Promise.all() array when using a Serverless instance specifically.

Perhaps related to the "Undocumented Issue" mentioned here https://medium.com/@abdelrhman.mbasyoni/mongodb-session-withtransaction-1569293442d4 (Given transaction number X does not match any in-progress transactions)

---

I also noticed the client becoming fully unresponsive at random times after executing a few transactions with an Atlas Serverless instance. Calling a db operation like MyModel.find() just resulted in indefinitely pending promises until I restarted the server.

This only happens with transactions/sessions and with an Atlas Serverless instance so I think it may somehow be related to this thread.

A bizarre solution to this is ending the session manually, like

await session.withTransaction(async () => {}).then(() => session.endSession())

even though the session with .transaction() and .withTransaction() is already being ended automatically / #13719 by mongoose!? 🤔

I ran the same transaction thousands of times with this and it works, while it starts hanging after the ~10th attempt when not ending the session manually.

Seems to me that Atlas Serverless instances aren't reliably usable with mongoose/transactions. Not sure if all of this is even related to mongoose at all (tried up to "mongoose": "^7.4.0", the Atlas Serverless instance is running on MongoDB 8.0.0).

[vkarpov15]

@maxeth can you come up with a repro script that we can run to demonstrate this issue?