Tracking issue for kage MVP

[Skrilltrax]

This is the tracking issue for kage MVP. Below are the requirements to achieve our first goal of making kage compatible with age.

API requirements

The requirements are divided into 3 sections - AgeFile, AgeKey, and Age Client. Once all the requirements are complete we will be able to generate age keys and use those keys to encrypt and decrypt data.

AgeFile

• Implement methods to parse AgeHeader
• Implement methods to write AgeHeader
• Implement methods to parse AgeBody
• Implement methods to write AgeBody
• Implement methods to parse AgeFile
• Implement methods to write AgeFile

AgeKey

• Implement methods to parse AgePublicKey
• Implement methods to write AgePublicKey
• Implement methods to parse AgePrivateKey
• Implement methods to write AgePrivateKey
• Implement methods to parse AgeKey
• Implement methods to write AgeKey

Age Client

• Implement methods to parse and generate X25519 recipients
• Implement crypto methods for X25519 recipients
• Implement methods to parse and generate X25519 identity
• Implement crypto methods for X25519 identity
• Implement a method that takes plaintext and a list of recipients and returns an AgeFile object
• Implement a method that takes an AgeFile and an identity and returns the decrypted data
• Implement convenience methods for library users
• Document public API

Final validation

• Pass all tests from the C2SP/CCTV age test suite

[simao]

Could you clarify a bit the following points:

• Implement a method that takes plaintext and a list of recipients and returns an AgeFile object
• Implement a method that takes an AgeFile and an identity and returns the decrypted data

AgeFile includes a body: ByteArray, should contain plaintext or encrypted data?

Also related, IMO AgeFile#body should be an InputStream, otherwise we'll have to copy the entire blob to memory? But this would also mean that equals and hashCode could not be implemented for AgeFile.

• Implement methods to parse AgeKey
• Implement methods to write AgeKey

These seem to already be implemented. Is there something missing? Maybe a test for AgeKey write?

[Skrilltrax]

Hey @simao

For AgeKey, you're correct. It is already implemented, I'll write the missing test and mark that as complete.

For AgeFile, I believe your point around copying the whole blob in memory is valid. My initial idea was that AgeFile should contain the encrypted data at first. We can then have a decrypt extension method on it. That would still be possible with an InputStream, but we won't be able to use AgeFile inside collections like a HashMap or a Set. Since we can get by without an AgeFile for now, I think we should give it a bit more thought on this before moving forward. What do you think?

[msfjarvis]

I can't think of any cases where putting an AgeFile in a collection that disallows duplicate items is a necessity, so I'm fine with changing the API to use an InputStream.

[simao]

I can see the convenience of putting AgeFiles in a collection, however, I think as a library, kage should signal to the API user that this can cause problems and not make this easy. It's might be OK with a file of say, 10Mb in memory, but if the user is reading hundreds of those files, they should not put them into a collection and pass it around. So I think there also a case to be made that a library should not make this easier.

In any case, I think it's fine to leave it for now, just wanted to point it out.

[msfjarvis]

All the public API we've checked in so far seems to be undocumented so I think we should be including full documentation as an explicit goal as well.

[simao]

Yeah agreed.

What do you think would be the API methods required for kage? What would make sense for android-password-store?

We currently have:

public fun encrypt(recipients: List<Recipient>, plainText: InputStream): AgeFile
public fun decryptStream(identities: List<Identity>, srcStream: InputStream, dstStream: OutputStream)
public fun decrypt(identities: List<Identity>, ageFile: AgeFile): InputStream
public fun decrypt(identity: Identity, ageFile: AgeFile): InputStream

I guess we'd need to expose the methods to write/parse AgeKeyFile, generate Identities/Recipients? What else?

[msfjarvis]

Yeah agreed.

What do you think would be the API methods required for kage? What would make sense for android-password-store?

We currently have:

public fun encrypt(recipients: List<Recipient>, plainText: InputStream): AgeFile
public fun decryptStream(identities: List<Identity>, srcStream: InputStream, dstStream: OutputStream)
public fun decrypt(identities: List<Identity>, ageFile: AgeFile): InputStream
public fun decrypt(identity: Identity, ageFile: AgeFile): InputStream

I guess we'd need to expose the methods to write/parse AgeKeyFile, generate Identities/Recipients? What else?

You can review the usages of PGPainless here to get a feel for what we'll need. Broadly speaking we'd want the following abilities

• Parse a ByteArray to a strongly typed Key
• Get a unique identifier from the Key type, akin to a user ID
• Encrypt a plaintext InputStream and write it into a destination OutputStream
• Decrypt a ciphertext InputStream and write it into a destination OutputStream
• Convert a stringly-typed user ID into a strongly typed Identifier

[msfjarvis]

The age test suite is available in an easily parseable format here, we should run kage against it. The reference Go implementation for the same is here.

[msfjarvis]

#104 lays out the foundation for integrating with the upstream test suite, I'll work on making it actually run the tests soon.

[simao]

I think some of these things are going to be a bit tricky to implement.

Get a unique identifier from the Key type, akin to a user ID

Age doesn't have a KeyID, maybe we could use the public key as KeyId? Or a checksum of the public key?

Parse a ByteArray to a strongly typed Key

We have fun parse(reader: BufferedReader): AgeKeyFile, you mean having fun parse(reader: ByteArray): AgeKeyFile ?

Encrypt a plaintext InputStream and write it into a destination OutputStream

Decrypt a ciphertext InputStream and write it into a destination OutputStream

These we already have, I think, Age.encryptStream and Age.decryptStream ?

Convert a stringly-typed user ID into a strongly typed Identifier

We don't have a GpgIdentifier in Age, how is this used, would this be the equivalent of a public key?

[msfjarvis]

I think some of these things are going to be a bit tricky to implement.

Get a unique identifier from the Key type, akin to a user ID

Age doesn't have a KeyID, maybe we could use the public key as KeyId? Or a checksum of the public key?

Public key should be fine.

Parse a ByteArray to a strongly typed Key

We have fun parse(reader: BufferedReader): AgeKeyFile, you mean having fun parse(reader: ByteArray): AgeKeyFile ?

Correct.

Encrypt a plaintext InputStream and write it into a destination OutputStream
Decrypt a ciphertext InputStream and write it into a destination OutputStream

These we already have, I think, Age.encryptStream and Age.decryptStream ?

Yep, that's covered.

Convert a stringly-typed user ID into a strongly typed Identifier

We don't have a GpgIdentifier in Age, how is this used, would this be the equivalent of a public key?

It's used to parse the .gpg-id file into a list of key IDs so the crypto layer knows what keys it should use. I'll probably just implement it app-side to begin with and upstream it into kage when it's sufficiently robust.

[msfjarvis]

I've raised #110 which takes the parsing code added earlier and dynamically generates tests from it. It has uncovered a bunch of missing and incorrect things that we should be looking into.

[msfjarvis]

Armor might be more work than I anticipated, the Go implementation uses separate readers and writers for armored streams which probably means a fair bit of duplicated code if we follow the same approach. @simao any ideas on this? Armor is the last major hurdle for the interoperability tests.

[FiloSottile]

Hi folks! Thank you for building kage and for running the test suite. If you don’t implement armoring, you’re welcome to just skip those tests. The suite is designed such that each vector tests exactly one thing, and all armored vectors test the armoring implementation, so you’re not missing out on coverage.

[simao]

@FiloSottile thanks that is good to know!

@msfjarvis I will try to take some time to see what we can do to implement armoring, I'd like to have it in kage

[Patagonicus]

Hey, I'd really like to see this project make progress so that android-password-store/Android-Password-Store#2061 gets unblocked. Unfortunately I don't have any Kotlin experience (and don't really trust myself to implement cryptography code).

However, I'd be willing to sponsor this. I won't be able to pay a full Software Engineer salary, but I'd be happy to throw in some 3-digit USD amount for the main contributor(s) if this gets done¹. I'm sure that's not going to be a huge influence and money probably isn't the thing that's missing here, but I still hope that it might help with motivation to work on this. I do hope I'm not offending anyone by this offer.

If there are any tasks that can be done with programming experience but no Kotlin specific knowledge I'd also be happy to look into those myself.

¹ GitHub Sponsorship, Paypal, KoFi, SEPA bank transfer or bug bounty type platforms would all work for me

[Skrilltrax]

Hey @Patagonicus, thanks a lot for offering the money, but as you mentioned, money isn't what we're missing here. I'll try to spend some time with Kage and see if we can reach a 0.x beta version or something along those lines.

[simao]

Yeah not a money problem, it's a time problem. I'll also try to carve some time for this but the next few weeks are busy

[nain-F49FF806]

@simao @Skrilltrax Hi, I am willing to contribute some time. That said, I am new to Kotlin, and kage, so if there's any low-hanging fruit I can work on, please let me know.
Cheers.

[msfjarvis]

With #124 being merged we now pass the age specification's test suite, barring two tests. We'll try to resolve them over the coming weeks and tag an initial 0.1.0 release.

[msfjarvis]

I believe I made a mistake during the rebase of the PR, and we actually do fully pass the test suite. I'll fix it on GitHub in a bit.

Edit: merged as #231

[msfjarvis]

v0.1.0 of kage is now available on Maven Central: https://central.sonatype.com/artifact/com.github.android-password-store/kage

I'll update the README in a few hours.