Badger failure on Windows Subsystem for Linux (fixed on Badger 2.0) #156

CMCDragonkai · 2020-01-13T13:19:05Z

Subject of the issue

I got this bug while trying out step-ca on WSL: dgraph-io/badger#722

Error opening database of Type badger with source /tmp/step/db: error opening Badger database: Unable to mmap RDWR log file: exec format error

According to the comment at the end, it was fixed when upgrading to badger 2. step-ca still currently uses 1.5.3.

Your environment

OS - Windows 10 with Ubuntu WSL

Steps to reproduce

Use Windows 10 WSL version 1909 OS build 18363.535, get ubuntu wsl. Get step-ca. And try running it after step ca init --ssh.

The text was updated successfully, but these errors were encountered:

dopey · 2020-01-29T00:53:44Z

Looks like the changes to use badger 2 are not very large. However, the data stored by badger v1 is incompatible with the data stored by v2. The change will require users to migrate their databases, or start from scratch. We'll have to discuss with the team to figure out a timeline.

I think most users are probably using the database in a nearly ephemeral way anyway. Unless you're using revocation, in which case you would lose the history of all revoked certs unless you migrated the DB.

strarsis · 2020-01-29T12:58:14Z

@dopey: Although I don't really need revocations because this should be used for localhost development, it would be still nice to have this fully working on WSL. Couldn't badger be used in a way that it autodetects the database file format and offer an optional migration to the user?

Edit: The other supported database, bbolt also got issues with latest WSL (go panic).
How has the ca.json to be configured to not use database at all?
I can't just leave the db option empty: Error opening database of Type with source : database not supported.

dopey · 2020-01-29T21:27:08Z

@dopey: Although I don't really need revocations because this should be used for localhost development, it would be still nice to have this fully working on WSL. Couldn't badger be used in a way that it autodetects the database file format and offer an optional migration to the user?

We discussed this in the morning, and I'll do my best to take a look this week. Current thinking is try to open the db with v2, if that fails then fallback to v1 and log a message that directs the user to the badger migration docs.

Edit: The other supported database, bbolt also got issues with latest WSL (go panic).
How has the ca.json to be configured to not use database at all?
I can't just leave the db option empty: Error opening database of Type with source : database not supported.

Try omitting the db attribute entirely. I've just tried that and it works locally.

strarsis · 2020-01-29T21:30:18Z

*And/or additionally offer a DB "type": "badger-v2".

dopey · 2020-01-29T21:34:48Z

Yeah, that was my initial idea too. If the method I described above isn't working out then I'll fallback to this.

dopey · 2020-01-30T18:21:16Z

Update on this: I tried to load badger and badger/v2 from the same project and the build fails with the following error:

panic: proto: duplicate enum registered: pb.ManifestChange_Operation

goroutine 1 [running]:
github.com/golang/protobuf/proto.RegisterEnum(...)
	/Users/max/pkg/mod/github.com/golang/protobuf@v1.3.1/proto/properties.go:459
github.com/dgraph-io/badger/v2/pb.init.0()
	/Users/max/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1/pb/pb.pb.go:638 +0x459

I believe this is because there is common code being run from the init.go for both packages. So, it doesn't look to me like there's an easy way to make both badger and badger/v2 available from the same package. I think we need to choose one or the other.

Short term I can build a branch of step-ca that only uses badger/v2. We'll need to come up with a longer term plan for migrating users to v2. Maybe just doing the migration for people ...

Would a branch be useful to people or do you prefer to have specific builds in the github releases?

strarsis · 2020-01-30T19:14:26Z

Could the v1 badger package be dynamically loaded using a go plugin?

dopey · 2020-01-30T22:53:39Z

Maybe. We don't use plugins anywhere else in our codebase and I've never used them before. I’m weary of introducing something I don't yet understand.

However, if you'd like to submit a PR that does this, I'd be happy work with you to get it merged.

mmalone · 2020-01-31T22:17:40Z

Could we open an issue with badger to describe the situation and see if they have any guidance?

dopey · 2020-02-02T02:15:43Z

dgraph-io/badger#1208

dopey · 2020-02-06T22:05:25Z

Update on the issue we opened on Badger is that they are looking into this to see if there's anything they can do to ease the transition :/

nicewook · 2020-02-12T22:55:15Z

Let me check "only badger 2.0 working" interim release is useful

dopey · 2020-02-12T22:58:07Z

I'll build an interim release that supports only badger2 and link it tomorrow.

dopey · 2020-02-13T18:09:21Z

Here's a link to a release built using badger2 -> https://github.com/smallstep/certificates/releases/tag/v0.14.0-rc.4.badger2

strarsis · 2020-02-14T23:34:46Z

@dopey: Just pulled the latest image from DockerHub (auto-build, nice!).
I get an error message from traefik, apparently after a restart of traefik and/or the step-ca server: urn:ietf:params:acme:error:serverInternal :: error creating order: error storing order IDs for account mv4GuMH6U2shGwN7r26AM97qc2UaxcQf; order IDs changed since last read, url: " providerName=localca.acme routerName=traefik-sec@docker
Edit: This seems to be some config issue.

nicewook · 2020-02-19T21:10:18Z

Here's a link to a release built using badger2 -> https://github.com/smallstep/certificates/releases/tag/v0.14.0-rc.4.badger2

Thank you @dopey I just tested my Ubuntu in Windows 10.
and I go through the link https://smallstep.com/docs/getting-started/

gclawes · 2020-04-12T22:36:36Z

I'm seeing this on the latest release Smallstep CA/0.14.2 (linux/arm):

root@ca:~# step-ca --version
Smallstep CA/0.14.2 (linux/arm)
Release Date: 2020-04-10 03:50 UTC
root@ca:~# step-ca $(step path)/config/ca.json
Error opening database of Type badger with source /root/.step/db: error opening Badger database: Unable to mmap RDWR log file: invalid argument

dopey · 2020-04-12T23:42:15Z

@gclawes was a previous version of step working for you on linux/arm? Want to make sure this isn't a regression. We just introduced those builds to the releases with this version and I haven't actually verified that they work on those systems.

I just tested on linux/ubuntu and darwin and don't have any trouble opening the DB.

If this is, indeed, an issue with linux/arm not supporting badger v1, then we should be able to fix this in the 0.15 release, which should be a quick follow. Badger recently pushed a fix allowing the use of both badger v1 and v2 simultaneously, which unblocks us being able to support badger v2.

I'm happy to create a release that uses badgerV2 if you'd be willing to try that out to see if it fixes your issue.

gclawes · 2020-04-12T23:51:42Z

This is the first version I am testing on ARM.

I see the same with 0.14.0-dev and 0.14.0-rc.16 (the first ARM release):

root@ca:~# step-ca --version
Smallstep CA/0.14.0-dev (linux/arm)
Release Date: 2020-04-09 16:45 UTC
root@ca:~# step-ca $(step path)/config/ca.json
Error opening database of Type badger with source /root/.step/db: error opening Badger database: Unable to mmap RDWR log file: invalid argument

root@ca:~# step-ca --version
Smallstep CA/0.14.0-rc.16 (linux/arm)
Release Date: 2020-04-08 22:29 UTC
root@ca:~# step-ca $(step path)/config/ca.json
Error opening database of Type badger with source /root/.step/db: error opening Badger database: Unable to mmap RDWR log file: invalid argument

gclawes · 2020-04-12T23:53:07Z

I'd be happy to test anything you want.

My testing platform is this:

root@ca:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="19.10 (Eoan Ermine)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 19.10"
VERSION_ID="19.10"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=eoan
UBUNTU_CODENAME=eoan
root@ca:~# uname -a
Linux ca 4.4.199-a890a5a94ebb621f8f1720c24d12fef1-0 #1 SMP Wed Jan 15 01:42:52 CET 2020 armv7l armv7l armv7l GNU/Linux
root@ca:~# cat /proc/cpuinfo
processor       : 0
model name      : ARMv7 Processor rev 1 (v7l)
BogoMIPS        : 50.00
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpd32
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x4
CPU part        : 0xc09
CPU revision    : 1

processor       : 1
model name      : ARMv7 Processor rev 1 (v7l)
BogoMIPS        : 50.00
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpd32
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x4
CPU part        : 0xc09
CPU revision    : 1

Hardware        : Marvell Armada 380/385 (Device Tree)
Revision        : 0000
Serial          : 0000000000000000

I also have an RPi 4 w/ ubuntu 64-bit.

dopey · 2020-04-13T05:52:36Z

Made a badger2 release - https://github.com/smallstep/certificates/releases v014.3-rc.1.badger2. Let me know if that works (or doesn't).

gclawes · 2020-04-13T23:08:45Z

I'm getting this on my armv7 box:

root@ca:~# step version
Smallstep CLI/0.14.2 (linux/arm)
Release Date: 2020-04-10 03:39 UTC
root@ca:~# step-ca version
Smallstep CA/0.14.3-rc.1.badger2 (linux/arm)
Release Date: 2020-04-13 05:45 UTC
root@ca:~# step-ca $(step path)/config/ca.json
Error opening database of Type badger with source /root/.step/db: error opening database: error opening Badger database: manifest has unsupported version: 4 (we support 7).
Please see https://github.com/dgraph-io/badger/blob/master/README.md#i-see-manifest-has-unsupported-version-x-we-support-y-error on how to fix this.

gclawes · 2020-04-13T23:31:15Z

Removing /root/.step and re-running step ca init fixed this:

root@ca:~# step ca init
✔ What would you like to name your new PKI? (e.g. Smallstep): ca.lan
✔ What DNS names or IP addresses would you like to add to your new CA? (e.g. ca.smallstep.com[,1.1.1.1,etc.]): ca.lan,10.10.0.65
✔ What address will your new CA listen at? (e.g. :443): :443
✔ What would you like to name the first provisioner for your new CA? (e.g. you@smallstep.com): root@ca.lan
✔ What do you want your password to be? [leave empty and we'll generate one]:
✔ Password: w1L>jIlkdui12{Mg6g)6pK5NQE(bcD<&

Generating root certificate...
all done!

Generating intermediate certificate...
all done!

✔ Root certificate: /root/.step/certs/root_ca.crt
✔ Root private key: /root/.step/secrets/root_ca_key
✔ Root fingerprint: ebc95ea862c46bf6dd37d932827c8b71cb5aadc812374cc3e9af7aa7d35388f2
✔ Intermediate certificate: /root/.step/certs/intermediate_ca.crt
✔ Intermediate private key: /root/.step/secrets/intermediate_ca_key
✔ Database folder: /root/.step/db
✔ Default configuration: /root/.step/config/defaults.json
✔ Certificate Authority configuration: /root/.step/config/ca.json

Your PKI is ready to go. To generate certificates for individual services see 'step help ca'.

FEEDBACK 😍 🍻
      The step utility is not instrumented for usage statistics. It does not
      phone home. But your feedback is extremely valuable. Any information you
      can provide regarding how you’re using `step` helps. Please send us a
      sentence or two, good or bad: feedback@smallstep.com or join
      https://gitter.im/smallstep/community.
root@ca:~# step-ca $(step path)/config/ca.json
badger 2020/04/13 23:29:44 INFO: All 0 tables opened in 0s
Please enter the password to decrypt /root/.step/secrets/intermediate_ca_key:
2020/04/13 23:30:13 Serving HTTPS on :443 ...

I originally had over-written an existing /root/.step directory, which resulted in the error above:

root@ca:~# step ca init
✔ What would you like to name your new PKI? (e.g. Smallstep): ca.lan
✔ What DNS names or IP addresses would you like to add to your new CA? (e.g. ca.smallstep.com[,1.1.1.1,etc.]): ca.lan,10.10.0.65
✔ What address will your new CA listen at? (e.g. :443): :443
✔ What would you like to name the first provisioner for your new CA? (e.g. you@smallstep.com): root@ca.lan
✔ What do you want your password to be? [leave empty and we'll generate one]:
✔ Password: $\n9s!>6/:Zw8HF]N6EY>>'mjf4]LpdA

Generating root certificate...
✔ Would you like to overwrite /root/.step/certs/root_ca.crt [y/n]: y
✔ Would you like to overwrite /root/.step/secrets/root_ca_key [y/n]: y
all done!

Generating intermediate certificate...
✔ Would you like to overwrite /root/.step/certs/intermediate_ca.crt [y/n]: y
✔ Would you like to overwrite /root/.step/secrets/intermediate_ca_key [y/n]: y
all done!

✔ Root certificate: /root/.step/certs/root_ca.crt
✔ Root private key: /root/.step/secrets/root_ca_key
✔ Root fingerprint: 8883cf83d996b6fd264c054fb04c3c3ec7b0d9263d9c71de324d24c1810c92fd
✔ Intermediate certificate: /root/.step/certs/intermediate_ca.crt
✔ Intermediate private key: /root/.step/secrets/intermediate_ca_key
✔ Would you like to overwrite /root/.step/config/ca.json [y/n]: y
✔ Would you like to overwrite /root/.step/config/defaults.json [y/n]: y
✔ Database folder: /root/.step/db
✔ Default configuration: /root/.step/config/defaults.json
✔ Certificate Authority configuration: /root/.step/config/ca.json

Your PKI is ready to go. To generate certificates for individual services see 'step help ca'.

FEEDBACK 😍 🍻
      The step utility is not instrumented for usage statistics. It does not
      phone home. But your feedback is extremely valuable. Any information you
      can provide regarding how you’re using `step` helps. Please send us a
      sentence or two, good or bad: feedback@smallstep.com or join
      https://gitter.im/smallstep/community.
root@ca:~# step-ca $(step path)/config/ca.json
Error opening database of Type badger with source /root/.step/db: error opening database: error opening Badger database: manifest has unsupported version: 4 (we support 7).
Please see https://github.com/dgraph-io/badger/blob/master/README.md#i-see-manifest-has-unsupported-version-x-we-support-y-error on how to fix this.

dopey added the bug label Jan 13, 2020

strarsis mentioned this issue Feb 7, 2020

error storing nonce: not implemented #179

Closed

dopey mentioned this issue Apr 14, 2020

Unable to start step-ca on RPi 3 (ARM7) smallstep/cli#256

Closed

dopey added this to the v0.15.0 milestone Apr 15, 2020

dopey closed this as completed Apr 21, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Badger failure on Windows Subsystem for Linux (fixed on Badger 2.0) #156

Badger failure on Windows Subsystem for Linux (fixed on Badger 2.0) #156

CMCDragonkai commented Jan 13, 2020 •

edited

Loading

dopey commented Jan 29, 2020 •

edited

Loading

strarsis commented Jan 29, 2020 •

edited

Loading

dopey commented Jan 29, 2020

strarsis commented Jan 29, 2020 •

edited

Loading

dopey commented Jan 29, 2020

dopey commented Jan 30, 2020

strarsis commented Jan 30, 2020

dopey commented Jan 30, 2020

mmalone commented Jan 31, 2020

dopey commented Feb 2, 2020

dopey commented Feb 6, 2020

nicewook commented Feb 12, 2020

dopey commented Feb 12, 2020

dopey commented Feb 13, 2020

strarsis commented Feb 14, 2020 •

edited

Loading

nicewook commented Feb 19, 2020

gclawes commented Apr 12, 2020

dopey commented Apr 12, 2020

gclawes commented Apr 12, 2020

gclawes commented Apr 12, 2020 •

edited

Loading

dopey commented Apr 13, 2020

gclawes commented Apr 13, 2020 •

edited

Loading

gclawes commented Apr 13, 2020

Badger failure on Windows Subsystem for Linux (fixed on Badger 2.0) #156

Badger failure on Windows Subsystem for Linux (fixed on Badger 2.0) #156

Comments

CMCDragonkai commented Jan 13, 2020 • edited Loading

Subject of the issue

Your environment

Steps to reproduce

dopey commented Jan 29, 2020 • edited Loading

strarsis commented Jan 29, 2020 • edited Loading

dopey commented Jan 29, 2020

strarsis commented Jan 29, 2020 • edited Loading

dopey commented Jan 29, 2020

dopey commented Jan 30, 2020

strarsis commented Jan 30, 2020

dopey commented Jan 30, 2020

mmalone commented Jan 31, 2020

dopey commented Feb 2, 2020

dopey commented Feb 6, 2020

nicewook commented Feb 12, 2020

dopey commented Feb 12, 2020

dopey commented Feb 13, 2020

strarsis commented Feb 14, 2020 • edited Loading

nicewook commented Feb 19, 2020

gclawes commented Apr 12, 2020

dopey commented Apr 12, 2020

gclawes commented Apr 12, 2020

gclawes commented Apr 12, 2020 • edited Loading

dopey commented Apr 13, 2020

gclawes commented Apr 13, 2020 • edited Loading

gclawes commented Apr 13, 2020

CMCDragonkai commented Jan 13, 2020 •

edited

Loading

dopey commented Jan 29, 2020 •

edited

Loading

strarsis commented Jan 29, 2020 •

edited

Loading

strarsis commented Jan 29, 2020 •

edited

Loading

strarsis commented Feb 14, 2020 •

edited

Loading

gclawes commented Apr 12, 2020 •

edited

Loading

gclawes commented Apr 13, 2020 •

edited

Loading