Support for importing 1Password 1pux file format

[changemenemo]

Hi,

no progress on importing 1puf extension database from 1password 8 or were you not aware of the changes? It's basically the same thing than before with small modifications

[droidmonkey]

Link to the spec?

[semicoleon]

I assume they meant "1pux" which appears to be the only non-csv way to export from 1password in version 8. It appears to be documented here. It's a zip archive with a JSON file and attachment files

[droidmonkey]

OK that is great, but that is entirely different from the OPVault concept. In fact they share absolutely no relation to each other. At least this one is unencrypted and not spread amongst hundreds (slight exaggeration) of json files.

[possiblemeatball]

Is there any current progress towards supporting this format? For curiosity's sake, willing to contribute code, just seeking info

[droidmonkey]

No progress and no intent to implement anytime soon. Well written and tested code welcome.

[notDavid]

Hi, so I'm guessing with the new .1PUX format, the section Importing 1Password OPVault of the docs does not work anymore since 1Password 8 (or even 1Password 7) ?

Perhaps it would be a good idea to change these docs, to avoid confusion / people trying to figure out how to do it? :-)

I'm guessing the currently recommended method is to export to .csv ?

[droidmonkey]

They work fine, you need to export your 1Password database as an opvault though.

[notDavid]

@droidmonkey How? I can't pick this option (in 1Password 7 on macOS):

1Password7 > File > Export > All items > :

Also, i don't see it in the 1Password export docs

[droidmonkey]

Looks like they completely removed all references to opvault in their documentation and in their app. You'll have to search reddit or somewhere to see if you can find the original location they are stored. An opvault is automatically created when you set 1Password to be locally unlockable.

[notDavid]

@droidmonkey

1. I'm pretty sure the opvault / database is no longer stored locally at all. And by this i mean when using a 1Password account.

2. Perhaps this is still possible for users who did not opt-in to the 1Password cloud, and are still using local/iCloud accounts while still using 1Password 7.

Also i think in the latest 1Password 8, it is no longer possible to "not use a 1Password account". See for example here :

"1Password 8 only supports 1Password accounts, and doesn't support standalone vaults synced through iCloud, Dropbox, or other third party syncing services. Given this, you'd only be able to access your Personal vault in 1Password 8, not your Primary vault contained in iCloud. Generally speaking, we'd recommend migrating your data to your 1Password account"

[droidmonkey]

Yes looks like the 1PUX format had taken on new urgency. OPVault was a freaking mess anyway, 1PUX will be far easier to implement.

[DaveLeonard]

I agree, there is no other way to export data from 1Password 8 than the .1pux.

I found this gist maybe it could help you: convert json from .1pux in csv

[Remus9oh7]

With the recent LastPass breach, I know several people wanting to move to KeePassXC from 1Password to remove the centralized attack surface. If there were a good time to have this available, now is it. Any updates / progress on this?

[droidmonkey]

The importer code is nearly done but not pretty or with test cases.

[Remus9oh7]

Sounds good, thanks for the update. I would offer to help, but I'm not sure I can - at least not with some hand holding. If there is a simple way to help, let me know. github.ujehm@simplelogin.com

[Skillkiller]

Any notable updates on the topic? Is there a public branch on this?

[droidmonkey]

You can start playing around with this branch but it does not implement everything for 1pux (icons and attachments) and introduces the new Import Wizard which is still WIP: https://github.com/keepassxreboot/keepassxc/tree/feature/1pux-import

[tgwaste]

Can we get a test build for this please?

[Frodox]

@droidmonkey 🙏🏻

[droidmonkey]

Ok, here you go!! I signed this build so you know it came from me, you can check the keepassxc.exe properties to verify.

https://store.dmapps.us/KeePassXC-2.8.0-snapshot-Win64.zip

Instructions:

1. Open the new Import Wizard using Database -> Import ...
   

2. Select the appropriate settings and your 1PUX file, press continue
   

3. Verify the import took and press done. Go through the new database creation wizard as usual.
   

4. Verify your import, NOTE... attachments, icons, and some esoteric data elements are not implemented yet.
   

[droidmonkey]

I'll get this PR buttoned up and posted for merging this weekend. Excited to bring this capability to the table.

[jby]

@tgwaste Sorry, yep. I used "Import..." again and it works. Few minor bugs like default path being system root (read only), but the most important part worked. Hopefully by the end of the weekend I am KeePass converted.

I just downloaded the 2.8 for macOS and I'm getting this same error:
Writing the database failed: Could not save. Database has not been initialised!
No matter how many times I try to re-do the import from my 1pux-file, I'm unable to save it as a kbdx-file... :(

[mobiuscog]

@droidmonkey I just wanted to add a huge Thank You for this PR. It allowed me to migrate easily on Linux, after building, although I didn't have any special requirements.

The only problem I noticed was that TOTP 'keys' in 1Password, often are stored with additional metadata before the secret, so they need to be truncated to only the secret in KeepassXC (in 1Password, they often have some oauth/domain meta as a prefix, that ends with 'secret'). If everything at the start including 'secret' is removed, the remaining key works just fine - at least in my simple testing.

Not all TOTP entries had this (I'm unsure why) so I had to check each manually.

[droidmonkey]

We read and store the otpauth url schema. Can you paste an example with a redacted secret?

I've also made several improvements to this code and haven't pushed them yet. While I was at it here I implemented Bitwarden unencrypted and encrypted json import as well.

[mobiuscog]

We read and store the otpauth url schema. Can you paste an example with a redacted secret?

I don't have them any more, as I converted them all manually as I was switching over, and clearing down my 1password.

There were a number that had otpauth in the name, along with domain information and then the actual key, but the codes generated were wrong until I changed the key to just be the secret. I was checking those with the 2.7.5 release (I just used the PR build to do the import) so perhaps that makes a difference.

Maybe I did something else incorrect.

[mobiuscog]

@droidmonkey I found some backups. An example in the CSV export would be:
otpauth://totp/Dropbox:redactedemail%40redacteddomain.redactedtld?secret=XXXXXXXXXXXXXXXXXXXX&period=30&algorithm=SHA1&digits=6&issuer=Dropbox

If I click on 'Setup TOTP' and look at the secret key, it shows:
otpauthtotpDropboxredactedemailredacteddomainredactedtldsecretXXXXXXXXXXXXXXXXXXXX
which fails as the secret is just XXXXXXXXXXXXXXXXXXXX

If I edit the entry, under advanced, it shows:
otpauth://totp/Dropbox:redactedemail%40redacteddomain.redactedtld?secret=otpauthtotpDropboxredactedemailredacteddomainredactedtldsecretXXXXXXXXXXXXXXXXXXXX%3D&period=30&digits=6&issuer=Dropbox

The %3D at the end of the 'advanced' secret is also curious. Hopefully that helps.

[droidmonkey]

Looks like we need to sometimes do a url decode on load. The reason it is not working is because the %40 should be an @

This is what it should look like: otpauth://totp/Example:alice@google.com?secret=JBSWY3DPEHPK3PXP&issuer=Example

[Baki35]

Can we get a new test build for this please?
Already using the last test build (thanks for that) for my 1Password exports but was wondering if there is a newer version?