Abuse of the OWM API key

[robinpaulson]

Somebody is using our api key to make many "one call" requests, taking it over the limit. If this continues, the key will be blocked and forecastie will require a custom key.

[woheller69]

That is why I asked F-Droid to create a way to inject API keys during build. In this case they would not have to be visible in the code.But they do not like it...

https://gitlab.com/fdroid/fdroiddata/-/issues/2654

[robinpaulson]

Cripes, they were shitty answers they gave. Patronising too. Unfortunately, my agreement with you means almost nothing.

They may well be right in what they said, but it's as if they're trying as hard as possible to miss the important point.

I've had multiple messages from OWM about going over our limits, so unless we come up with something soon, Forecastie will be de facto dead within maybe a week, only those users with their own keys will be able to use it.

I'll message Tomas and see if he has any solutions. I think he's looking for a maintainer, perhaps you are interested?

[robinpaulson]

Perhaps we do something in-between what we do now and what we would like F-Droid to do? I wonder if it's possible to put a line in the build script which downloads the key at build time. So long as the key is at a publically-available address, it will fulfil their requirements for "no secrecy", but will make finding the key harder for nefarious actors. Understanding a build script is a little higher bar than reading source code. We could even obfuscate the key, then de-obfuscate it in the code, e.g. by base64 encoding it, then decoding it after it is downloaded. Yes, I understand the security/obscurity line, I've heard the rhyme.

I have no idea if this is technically possible, I do not understand Gradle build files.

Edit: if we do this, it will also make the process of updating they key very simple, I guarantee this abuse will happen again.

[robinpaulson]

https://stackoverflow.com/questions/46959783/running-shell-command-from-gradle-script#46959846

[woheller69]

If you just want to obfuscate you can put the key base64 encoded in the source and then decode at first start and store as shared preference.
I wanted a good future proof solution with F-Droid. There are more and more providers requiring API key and some cost money according to usage or do not allow the key to be published. For my gas prices app they do not even allow to publish the demo key... But they would allow to use one key for all users. But without F-Droid support this is not possible. So everyone needs a key on his own.

In future a user might need to enter several API keys, one for OWM, one for OpenStreetMap (they also somegimes think of API keys), one for a geolocation API, one for...
In the end open source apps will be useless.

[robinpaulson]

I want to do two things: provide an easy way to update keys and mildly obfuscate the keys.

[woheller69]

you can encode your keys here https://www.base64encoder.io/ and then store them in your source code.
In onCreate you check if the version of the app has changed and then decode and save the key.

Maybe you can add some comments in the F-Droid issue. If there are several supporters - maybe in a distant future they will change their mind...

[robinpaulson]

I hear you on the point about access to services via API keys being a problem.

[robinpaulson]

check if the version of the app has changed

given there is no maintainer and may not be for the foreseeable future, I want to avoid anything which relies on app updates

[robinpaulson]

I don't want to get into an argument with the F-Droid maintainers, I've seen what happens when people disagree with them before (the "updates" to the UI a few years ago). They tend to dig their heels in and do anything possible to avoid the topic, as they already did in the issue you opened. To be fair, this is the case for a lot of libre software/services.

[robinpaulson]

Let's see what Tomas says before progressing further.

[woheller69]

you could ask OWM to block OneCallAPI for your key.
For new free keys they want to do this anyway. OneCallAPI will no longer be free for new users.
Future users of my app will need a OneCallAPI 3.0 by Call subscription with 1000 free calls per day to use my app.
https://openweathermap.org/api

[robinpaulson]

you could ask OWM to block OneCallAPI for your key.

yeah, i tried that. they're not interested

[robinpaulson]

For new free keys they want to do this anyway. OneCallAPI will no longer be free for new users.

this is interesting though, it may inadvertently fix our problem. any idea of when they will implement it?

[woheller69]

maybe use a prebuild command on F-Droid like here here https://gitlab.com/fdroid/fdroiddata/-/blob/a2fe970cf5919b83efec4984c2005267058af593/metadata/net.osmand.plus.yml

prebuild:
      - sed -i -e '/qt.*Compile/d' build.gradle
      - sed -i -e "s/System.getenv(\"APK_VERSION\")/\"2.0.4\"/g" build.gradle

something like

echo -n 'bXktc3RyaW5n' | base64 --decode and then send the result to sed to replace a place holder for your API key.

But then the key will be visible in the Tarball published by F-Droid

[woheller69]

For new free keys they want to do this anyway. OneCallAPI will no longer be free for new users.

this is interesting though, it may inadvertently fix our problem. any idea of when they will implement it?

for existing keys this will not be implemented

[robinpaulson]

But then the key will be visible in the Tarball published by F-Droid

that's not a problem per se. as we discussed, this is only about making things a little more difficult. this will help with that.

what interests me more is being able to update the key without making a new release of the whole app.

perhaps every so often Forecastie checks for a new API key and downloads it to the local device? then as soon as a key gets compromised, we update what is on the server and disable the old key. yes, any nefarious actor can look through the code and find the address where it is stored, but it is an extra layer of obfuscation, particularly if we encode it.

[woheller69]

that should be possible, but you need a webserver to store the key

[robinpaulson]

that should be possible, but you need a webserver to store the key

that's the easy bit

[robinpaulson]

as soon as a key gets compromised, we update what is on the server and disable the old key

going one step further, we could automatically generate a new key say every week and make it available. that would make it more difficult to keep up with what we do

[robinpaulson]

For new free keys they want to do this anyway. OneCallAPI will no longer be free for new users.

could you link to some info on this?

[woheller69]

unfortunately not. Few days ago OneCallAPI 1.0 was still mentioned on their web site. And it said it is not available with free keys.
It seems now they completely removed the API documentation.

But I have an email from them:

Yes, all old users can continue to use without changes. But for all new free and startup users, One Call 1.0 will not be available. But for Developer clients and above, it will be available.
Yes, new clients must use syntax 3.0 and subscribe to it. Otherwise, One Call will not be available to them.

[robinpaulson]

I am the person who controls the API access to OWM, so I spoke with their technical contact yesterday, who confirmed there is no access to OneCall for new users. Given this, I'm going to delete and then recreate our API account and get new API keys (we have an "open source" account, so it's a little more convoluted than merely making a new account and generating some API keys). Then I'll have to wait for access to the Github account so I can do a one-time update to the code, with the new keys. This all feels more complicated than necessary, but I've zero interest in fighting with OWM to get them to do it the simple way.

I recall you already maintain an app which uses OWM, so you're likely not interested in asking Tomas that you take over the maintenance of Forecastie?

[woheller69]

I already have 7 apps on F-Droid. At the moment I cannot maintain another one...

You should first try to create an additional account and test the new key. I recently did and OneCallAPI still worked.
I will need to change my app once it does not work anymore for new users.
But I will wait until it happens - maybe it never does and they are just removing documentation

[robinpaulson]

first try to create an additional account and test the new key

yes, good plan.

OneCallAPI still worked

huh, that sounds irritating

[robinpaulson]

@martykan Tomas, any suggestions here? I'm surprised OWM haven't shut down our key already, I think they will at some point though.

[woheller69]

OneCallAPI is blocked now for new subscriptions...

[woheller69]

I built a new app using open-meteo, which does not require an API key.
Maybe someone wants to try...

https://github.com/woheller69/omweather