Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve ubuntu 22.04 support #1

Closed
rbev opened this issue Nov 12, 2024 · 31 comments
Closed

Improve ubuntu 22.04 support #1

rbev opened this issue Nov 12, 2024 · 31 comments

Comments

@rbev
Copy link

rbev commented Nov 12, 2024

there is a long standing issue with newer linux distributions in EphemeralMongo that I can only assume still exists in this fork

  • When running using the linux-x64 package on newer ubuntu images you get errors because it links to no longer available libraries.
  • if you include the ubuntu-22.04 packages then they are not automatically detected and you have to conditionally change the binpath on startup

asimmon/ephemeral-mongo#51
asimmon/ephemeral-mongo#48 (comment)

I believe that this issue really just stuck around becuase the older ubuntu was needed to run MondoDB 4 - which is now EoL, so the linux-x64 pacakge can probably be updated to work on newer Ubuntu.

@wassim-k
Copy link
Owner

I'm familiar with the issue, which is why I have defaulted both Mongo7 and Mongo8 linux binaries to ubuntu-22.04
Please give it a test and let let me know if the issue still exists.

@rbev
Copy link
Author

rbev commented Nov 12, 2024

Oh cool, I'll try it out tomorrow in our tests and let you know

@rbev
Copy link
Author

rbev commented Nov 13, 2024

Swapping out EphemeralMongo in our tests seems to not work

I'm just getting timeout errors waiting for the db to start - both linux (in container apps build pipeline) and osx (rider)

  ActivatorChain: VetDB.Tests.TestFramework.Specflow.DemoApp.Demo.DemoEventHandler -> λ:VetDB.Tests.TestFramework.Specflow.DemoApp.Demo.IDemoDb
  ----> System.TimeoutException : MongoDB connection availability took longer than the specified timeout of 30 seconds. Consider increasing the value of 'ConnectionTimeout'.
  Stack Trace:
     at Autofac.Core.Resolving.Middleware.ActivatorErrorHandlingMiddleware.Execute(ResolveRequestContext context, Action`1 next)
   <etc>

I also get nothing logged to the test if I set up the stdout logger

@rbev
Copy link
Author

rbev commented Nov 13, 2024

From what i can see all the packages put the right things in the right places, and our config is fairly simple

        _temporaryMongoDb = new Lazy<IMongoRunner>(() =>
            MongoRunner.Run(new MongoRunnerOptions()
            {
                UseSingleNodeReplicaSet = true,
                StandardOuputLogger = _ => { },
                AdditionalArguments = "--nojournal"
            }));

@wassim-k
Copy link
Owner

Which MongoDB.Driver version are you using?

@rbev
Copy link
Author

rbev commented Nov 13, 2024

I tried both 7 and 8 with the same result

@wassim-k
Copy link
Owner

No, I meant the MongoDB.Driver nuget package, which version is referenced in your project?

@rbev
Copy link
Author

rbev commented Nov 13, 2024

oh sorry - 3.0.0

@rbev
Copy link
Author

rbev commented Nov 13, 2024

Just to eliminate any counfounding issues I created a new test project with just a single test:


namespace TestProject1;

public class Tests
{
    [Test]
    public void Test1()
    {
        MongoRunner.Run(new MongoRunnerOptions()
        {
            UseSingleNodeReplicaSet = true, 
            StandardOuputLogger = TestContext.WriteLine, 
            AdditionalArguments = "--nojournal"
        });
    }
}

image

and i get the same behaviour

@rbev
Copy link
Author

rbev commented Nov 13, 2024

and the file listing looks all good for finding the runtimes

❯ find . | grep runtimes
./Debug/net8.0/runtimes
./Debug/net8.0/runtimes/osx-x64
./Debug/net8.0/runtimes/osx-x64/native
./Debug/net8.0/runtimes/osx-x64/native/mongodb
./Debug/net8.0/runtimes/osx-x64/native/mongodb/bin
./Debug/net8.0/runtimes/osx-x64/native/mongodb/bin/mongod
./Debug/net8.0/runtimes/osx-x64/native/mongodb/bin/mongoexport
./Debug/net8.0/runtimes/osx-x64/native/mongodb/bin/mongoimport
./Debug/net8.0/runtimes/osx-x64/native/mongodb/community-server
./Debug/net8.0/runtimes/osx-x64/native/mongodb/community-server/._macos_mongodb.plist
./Debug/net8.0/runtimes/osx-x64/native/mongodb/community-server/LICENSE-Community.txt
./Debug/net8.0/runtimes/osx-x64/native/mongodb/community-server/macos_mongodb.plist
./Debug/net8.0/runtimes/osx-x64/native/mongodb/community-server/MPL-2
./Debug/net8.0/runtimes/osx-x64/native/mongodb/community-server/._THIRD-PARTY-NOTICES
./Debug/net8.0/runtimes/osx-x64/native/mongodb/community-server/._MPL-2
./Debug/net8.0/runtimes/osx-x64/native/mongodb/community-server/THIRD-PARTY-NOTICES
./Debug/net8.0/runtimes/osx-x64/native/mongodb/community-server/README
./Debug/net8.0/runtimes/osx-x64/native/mongodb/community-server/._README
./Debug/net8.0/runtimes/osx-x64/native/mongodb/community-server/._LICENSE-Community.txt
./Debug/net8.0/runtimes/osx-x64/native/mongodb/community-server/version.txt
./Debug/net8.0/runtimes/osx-x64/native/mongodb/community-server/._sbom_from_silk.json
./Debug/net8.0/runtimes/osx-x64/native/mongodb/community-server/sbom_from_silk.json
./Debug/net8.0/runtimes/osx-x64/native/mongodb/database-tools
./Debug/net8.0/runtimes/osx-x64/native/mongodb/database-tools/LICENSE.md
./Debug/net8.0/runtimes/osx-x64/native/mongodb/database-tools/mongodb-database-tools-macos-x86_64-100.10.0.tgz.cdx.json
./Debug/net8.0/runtimes/osx-x64/native/mongodb/database-tools/._LICENSE.md
./Debug/net8.0/runtimes/osx-x64/native/mongodb/database-tools/._THIRD-PARTY-NOTICES
./Debug/net8.0/runtimes/osx-x64/native/mongodb/database-tools/THIRD-PARTY-NOTICES
./Debug/net8.0/runtimes/osx-x64/native/mongodb/database-tools/README.md
./Debug/net8.0/runtimes/osx-x64/native/mongodb/database-tools/._README.md
./Debug/net8.0/runtimes/osx-x64/native/mongodb/database-tools/._mongodb-database-tools-macos-x86_64-100.10.0.tgz.cdx.json
./Debug/net8.0/runtimes/osx-x64/native/mongodb/database-tools/version.txt
./Debug/net8.0/runtimes/win-x64
./Debug/net8.0/runtimes/win-x64/native
./Debug/net8.0/runtimes/win-x64/native/mongodb
./Debug/net8.0/runtimes/win-x64/native/mongodb/bin
./Debug/net8.0/runtimes/win-x64/native/mongodb/bin/mongod.exe
./Debug/net8.0/runtimes/win-x64/native/mongodb/bin/mongoimport.exe
./Debug/net8.0/runtimes/win-x64/native/mongodb/bin/mongoexport.exe
./Debug/net8.0/runtimes/win-x64/native/mongodb/community-server
./Debug/net8.0/runtimes/win-x64/native/mongodb/community-server/LICENSE-Community.txt
./Debug/net8.0/runtimes/win-x64/native/mongodb/community-server/MPL-2
./Debug/net8.0/runtimes/win-x64/native/mongodb/community-server/THIRD-PARTY-NOTICES
./Debug/net8.0/runtimes/win-x64/native/mongodb/community-server/README
./Debug/net8.0/runtimes/win-x64/native/mongodb/community-server/version.txt
./Debug/net8.0/runtimes/win-x64/native/mongodb/community-server/sbom_from_silk.json
./Debug/net8.0/runtimes/win-x64/native/mongodb/database-tools
./Debug/net8.0/runtimes/win-x64/native/mongodb/database-tools/LICENSE.md
./Debug/net8.0/runtimes/win-x64/native/mongodb/database-tools/mongodb-database-tools-windows-x86_64-100.10.0.zip.cdx.json
./Debug/net8.0/runtimes/win-x64/native/mongodb/database-tools/THIRD-PARTY-NOTICES
./Debug/net8.0/runtimes/win-x64/native/mongodb/database-tools/README.md
./Debug/net8.0/runtimes/win-x64/native/mongodb/database-tools/version.txt
./Debug/net8.0/runtimes/linux-x64
./Debug/net8.0/runtimes/linux-x64/native
./Debug/net8.0/runtimes/linux-x64/native/mongodb
./Debug/net8.0/runtimes/linux-x64/native/mongodb/bin
./Debug/net8.0/runtimes/linux-x64/native/mongodb/bin/mongod
./Debug/net8.0/runtimes/linux-x64/native/mongodb/bin/mongoexport
./Debug/net8.0/runtimes/linux-x64/native/mongodb/bin/mongoimport
./Debug/net8.0/runtimes/linux-x64/native/mongodb/community-server
./Debug/net8.0/runtimes/linux-x64/native/mongodb/community-server/LICENSE-Community.txt
./Debug/net8.0/runtimes/linux-x64/native/mongodb/community-server/MPL-2
./Debug/net8.0/runtimes/linux-x64/native/mongodb/community-server/THIRD-PARTY-NOTICES
./Debug/net8.0/runtimes/linux-x64/native/mongodb/community-server/README
./Debug/net8.0/runtimes/linux-x64/native/mongodb/community-server/version.txt
./Debug/net8.0/runtimes/linux-x64/native/mongodb/community-server/sbom_from_silk.json
./Debug/net8.0/runtimes/linux-x64/native/mongodb/database-tools
./Debug/net8.0/runtimes/linux-x64/native/mongodb/database-tools/LICENSE.md
./Debug/net8.0/runtimes/linux-x64/native/mongodb/database-tools/THIRD-PARTY-NOTICES
./Debug/net8.0/runtimes/linux-x64/native/mongodb/database-tools/README.md
./Debug/net8.0/runtimes/linux-x64/native/mongodb/database-tools/mongodb-database-tools-ubuntu2204-x86_64-100.10.0.tgz.cdx.json
./Debug/net8.0/runtimes/linux-x64/native/mongodb/database-tools/version.txt

@wassim-k
Copy link
Owner

Can you please try removing --nojournal option, just to see if that's what's causing it?

@rbev
Copy link
Author

rbev commented Nov 13, 2024

still errors with no options at all

    [Test]
    public void Test1()
    {
        MongoRunner.Run(new MongoRunnerOptions()
        {
            //UseSingleNodeReplicaSet = true, 
            StandardOuputLogger = TestContext.WriteLine, 
            //AdditionalArguments = "--nojournal"
        });
    }

@wassim-k
Copy link
Owner

That's odd, I was sure that the "--nojournal" option is what caused it, as I tested it and it gave me the same error.
And I found this in mongod documentation:

Starting in MongoDB 6.1, journaling is always enabled. As a result, MongoDB removes the storage.journal.enabled option and the corresponding --journal and --nojournal command-line options.

@wassim-k
Copy link
Owner

@rbev Any luck? If not, can you please create a minimal reproduction.

@rbev
Copy link
Author

rbev commented Nov 19, 2024

The minimal replication is literally just consume the nuget package

But here you go:
https://github.com/rbev/MongoSandboxIssueReplication

@wassim-k
Copy link
Owner

wassim-k commented Nov 19, 2024

You're right, that's very minimal. Can you please try running mongod manually and check the output:

.\mongod --dbpath ./db  --tlsMode disabled --port 51401 --bind_ip 127.0.0.1

You'll need to create an empty db directory.

@rbev
Copy link
Author

rbev commented Nov 19, 2024

❯ ./runtimes/osx-x64/native/mongodb/bin/mongod --dbpath ./db --port 51401 --bind_ip 127.0.0.1
zsh: illegal hardware instruction  ./runtimes/osx-x64/native/mongodb/bin/mongod --dbpath ./db --port 51401

I'm on an M2 mackbook pro

Do you need to include osx-arm64 as it's own package?

@rbev
Copy link
Author

rbev commented Nov 19, 2024

Also forcing it to run under x64 does not work

❯ arch -x86_64 ./runtimes/osx-x64/native/mongodb/bin/mongod --dbpath ./db --port 51401 --bind_ip 127.0.0.1
zsh: illegal hardware instruction  arch -x86_64 ./runtimes/osx-x64/native/mongodb/bin/mongod --dbpath ./db --por

@wassim-k
Copy link
Owner

Yes, that's the issue. ARM 64 is not supported as explained here.
Feel free to raise a PR to include it as an optional runtime package.

@rbev
Copy link
Author

rbev commented Nov 19, 2024

i'm not sure that makes sense - we've been running this on apple silicon for literally years with no issues

something has changed

@wassim-k
Copy link
Owner

Which version of MongoDB were you running?

@wassim-k
Copy link
Owner

Can you verify your setup against this compatibility table?

@rbev
Copy link
Author

rbev commented Nov 19, 2024

ok - so if i downgrade MongoSandbox to use mongod v6.x it works fine on my laptop

my OSX version (14.6.1 (23G93)) shows as supported for mongo 7/8 for both x64 and arm builds

@rbev
Copy link
Author

rbev commented Nov 19, 2024

interestingly changing it to mongo v8 works for me

and according to my colleague my replication demo passes on their M1 macbook pro, so it's something with mongod7 + M2

@wassim-k
Copy link
Owner

That's good, we're making progress.
Mongo7 + M2 seems like an oddly specific combo, I wonder why!

@rbev
Copy link
Author

rbev commented Nov 19, 2024

maybe a bad binary from the mongoDB release?

edit: actually that makes no sense

@rbev
Copy link
Author

rbev commented Nov 19, 2024

Ok, so the OSX issue aside - the current 1.0.0 build does NOT work on a 22.04 ubuntu image.

I've just pushed a MongoSandbox8 build of my tests up and it doesn't work, in fact it seems to just lock up.

@wassim-k
Copy link
Owner

That is likely an issue with the code, the github workflow pipeline tests all nuget packages against windows and linux environments, and I'm using MongoSandbox8 in multiple microservices that are developed on windows and deployed and tested in linux pipelines, all without issues.

Try pushing the minimal reproduction up instead and see if it passes then work your way from there.

@rbev
Copy link
Author

rbev commented Nov 19, 2024

In case anyone comes across this issue wondering what the resolution was

  • OSX issue was solved by updating to OSX 15 (some sort of issue with mongod x86 8.0.3 on 14.x's Rosetta?)
  • linux issue is definitely something in our test code, it works if i do some very basic mongo stuff..

@flcdrg
Copy link

flcdrg commented Jan 11, 2025

FYI having issues with MongoSandbox6 on GitHub Actions ubuntu-latest (which are now running Ubuntu 24.04.1) similar errors to those listed above), but MongoSandbox8 seems ok.

@rbev
Copy link
Author

rbev commented Jan 11, 2025

MongoDB 6 goes EoL in July - and I don't think it's supported on that version of linux

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants