Major version update planning

[chrisdunelm]

In early 2017 we plan to update to major version 2 of the client libraries. This issue is to track changes we may make that require a major version change.

• Remove support for Silverlight platform.
• Remove support for <.NET4.6
• Remove support for older Windows Phone apps - we will only support Windows 10+ using UWP apps (including native).
• Update all dependencies to the most recent major version - e.g. Newtonsoft.Json, BouncyCastle, ...
• Remove RSACryptoProvider from the public API.
• Change the package and project structure of some targets to contain a single dll, rather than two dlls? (or make the entire support library a single dll/package?)
• Add get-range to IMediaDownloader
• Change the return type from void to IProgressDownload of generated code that uses MediaDownloader.Download(...). This is to expose download errors when using sync calls, which at the moment are completely hidden. This will address Generated download code hides errors #803
• Add request/response logging (Tracing request/response is very fiddly #746)
• Sort out some timezone issues (Recurring yearly appointments created using Google API #853) (or possibly only accept UTC, and throw otherwise?)
• Revisit all auth mechanisms and make sure the as many as are possible are available on as many platforms as possible.

These decisions are not yet final.

[chrisdunelm]

RSACryptoProvider is currently part of the public API (see for example ServiceAccountCredential.cs) which is not available on netstandard and other platforms.
Change the API to use System.Security.Cryptography.RSA (or something similar) instead, so other RSA implementations can be used; whilst providing a consistent API across all platforms.

[LindaLawton]

Windows phone is deprecated? Did I miss Microsoft giving up on it? When did that happen?

[chrisdunelm]

@LindaLawton My mistake, I've updated the issue to make it clearer. We plan to drop support for Windows 8/8.1; and support Windows Phones with Windows 10+ by supporting UWP apps.

[chrisdunelm]

Change the return type from void to IProgressDownload of generated code that uses MediaDownloader.Download(...).
This is to expose download errors when using sync calls, which at the moment are completely hidden. This will address #803

[ghost]

Could you add #853 ? There is reply there: "We won't be able to change the existing behaviour without breaking backward compatibility... so we'll consider it for v2.0, but won't do anything in the v1.X series."

[chrisdunelm]

@obelix30: done.

[Sergio0694]

@chrisdunelm Hello, how's it going with the v2 development, are there any news or is there an ETA on the release? Thanks for your hard work, looking forward to being able to use the library again with UWP apps!

[ghost]

@chrisdunelm See the news on .NET Standard 2.0, some code won't need drastic change.

[mateusz-piasecki]

@chrisdunelm @jskeet hi hi hello - welcome in 'early 2017' - any news about v2?

[LindaLawton]

I don't have any information on this upgrade. However i will see if i cant get in touch with the developers working on it.

Note: Its vacation this week in the UK cant promise we will hear from them before next week.

[chrisdunelm]

We'll have a fairly detailed update here by the end of this week (2016-02-24). Sorry for the silence and delay.

Edit 2016-02-23: Apologies, this update is going to be delayed by a week (hopefully no more). So we now expect to post details here by 2016-03-03

[dpeter99]

@chrisdunelm I think you meant 2017-03-03. So how long should we wait for the update?
Maybe a smaller patch for .net core (#897) while you work on the big update?

[chrisdunelm]

This has become somewhat more complex than we anticipated, mainly due to dependencies from the new gRPC (www.grpc.io) based Google.Cloud.* libraries to Google.Apis.* support libraries within this repo. For example: Grpc.Auth depends on Google.Apis.Auth (and most Google.Cloud.* packages depend - indirectly - on Grpc.Auth); and Google.Cloud.Storage.V1 depends on Google.Apis.Storage.v1 (and then on Google.Apis and Google.Apis.Auth).

gRPC is unlikely to have a major release (from v1.x to v2.x) in the near future. Due to semantic versioning, we cannot upgrade gRPCs dependencies on Google.Apis.* packages form v1.x to v2.x within gRPC v1.x releases.

Although we could upgrade this repo's Google.Apis.* packages to v2 we are very reluctant to do so if this means gRPC cannot depend on them.
This is because we'd then have the situation where a project that has upgraded to Google.Apis.* v2.x would then have to downgrade to v1.x in order to start taking advantage of the new Google.Cloud.* gRPC-based packages. This is absolutely not a situation we want to get in to, as we expect a shift towards the new gRPC-based packages which is where most of our effort is going.

Ideally we'd like to find a way to detach the Google.Cloud.* gRPC-based packages from the Google.Apis.* packages in this repo, removing all cross-dependencies, so we can version each independently.
However, we have as yet been unable to find a way to do this, and to do this in a way that ensures all existing user code will remain unbroken during the transition.

So it looks likely that we'll need to stay with v1.x in this repo; and only move to v2 in step with gRPC. Unfortunately the gRPC release schedule is not determined by our versioning needs for packages within this repo ;)
If we do need to remain in v1.x in the foreseeable future, then we'll implement as many improvements as we can in minor updates within v1.x without breaking backward compatibility. The one breaking change we may consider is upgrading the Newtonsoft.Json dependency to the latest version, as it appears major version updates of Newtonsoft.Json are backwards compatible.

We don't have a firm date for when final decisions will be made, but we are now working through the issues involved.

[Sergio0694]

Hello @chrisdunelm and thanks for the update and for your work.
I'm sorry to hear about these issues you're having, it's a shame the gRPC lib doesn't have an upcoming update planned.

As for the issue I originally opened (#838), any idea on a fix for that? I've been stuck with that problem for months and I have yet to understand why did that happen, as it used to work fine - maybe an issue with a .NET Native update as you suggested?
Anyways, do you think it'd be possible to add a fix for that in an upcoming v1 update, can I hope to see it fixed in the short term? Thanks in advance, have a good day!

[chrisdunelm]

@Sergio0694 Yes, native/UWP support is something that we'll look at fixing with a v1 update; to fix #838 and #752.

[Sergio0694]

@chrisdunelm Awesome, thank you so much, looking forward to it then!

[Sergio0694]

@chrisdunelm Sorry for the double post, just so I can have a general idea of the timeframe, is there a general ETA for the #838 fix? I mean, I'm not even sure if that's a trivial fix or if it'd require lots of work on your end, do you know whether a fix will likely be available in a few days now that you're working on it, or if it's a matter of weeks/months?
Thanks again! 😄

[chrisdunelm]

We have no specific timescale for working on native/UWP support (including #838). It'll be a matter of weeks/months, not a few days.

[mhgamework]

Do I understand correctly from this discussion that it is currently not possible to use the google apis with dotnetcore? I am also having the "GoogleWebAuthorizationBroker is missing" problem.

[chrisdunelm]

In general these libraries do support .NET core, however GoogleWebAuthorizationBroker is not implemented in .NET core, so an alternative authentication mechanism will need to be used.

[mhgamework]

Thanks, do you have any pointers as to how i could do that? My knowledge about this is fairly limited.

[mhgamework]

I managed to implement my own GoogleWebAuthorizationBroker and FileDataStore for dotnetcore using the source for the windows version.
Basically there is a fallthrough in FileDataStore using Environment.getspecialfolder which is more of an utility so that can just be removed. Next i implemented ICodeReceiver because the windows implementation uses the httplistener.

If the small fix to FileDataStore was made, the GoogleWebAuthorizationBroker can be added as is to the dotnetcore libraries and you can use it as is!

[dpeter99]

@mhgamework Can you maybe add it to my repo. So I have it and other people can find an implementation of it easily. As it is already linked in #897

[LindaLawton]

@mhgamework I think it would be a good idea for you to put up your own repo showing how you have done this. If you want to explain how you are well to write a post about it. I am happy to host it.

[Sergio0694]

Hello @chrisdunelm , I've just updated the package to 1.23.0.809 and I noticed a ton of dependencies have been updated as well. Tried running my UWP app again (hope's always a good thing), it still works fine in debug but has the same error when compiling with .NET Native (exactly as explained in #838 ).
Just wanted to leave this feedback, keep up the good work! 👍

[sguergachi]

Updating to 1.24 seems to have completely broken compliation for my UWP app, so I'm dead in the water until it's supported. Is UWP something we have to wait for in V2? Or can that still be worked on for the V1 branch?

[jskeet]

@sguergachi: Could you clarify what you mean by "completely broken compilation"? What exact errors are you getting? Note that there was a problem with 1.24.0, so we had to roll out 1.24.1 quickly. If you saw the problem with 1.24.0, please try with 1.24.1.

[sguergachi]

@jskeet I have 1.24.1, these are the errors that popped up right after updating the Google API NuGet packages, not sure how many are caused by the update but it had no errors before I updated:

Severity	Code	Description	Project	File	Line	Suppression State
Error		Internal compiler error: Unable to cast object of type 'Microsoft.Cci.MutableCodeModel.NamespaceTypeReference' to type 'Microsoft.Cci.INestedTypeReference'.	NudgeToaster			
Error		Type 'Microsoft.Runtime.CompilerServices.ConfiguredTaskAwaitable`1' was not included in compilation, but was referenced in type 'Google.Apis.Auth.OAuth2.ComputeCredential+<RequestAccessTokenAsync>d__9'. There may have been a missing assembly.	NudgeToaster			
Error		Type 'Microsoft.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter' was not included in compilation, but was referenced in type 'Google.Apis.Auth.OAuth2.ComputeCredential+<RequestAccessTokenAsync>d__9'. There may have been a missing assembly.	NudgeToaster			
Error		System.Diagnostics.Process 4.3.0 provides a compile-time reference assembly for System.Diagnostics.Process on UAP,Version=v10.0, but there is no run-time assembly compatible with win10-x64.			0	
Error		System.Diagnostics.Process 4.3.0 provides a compile-time reference assembly for System.Diagnostics.Process on UAP,Version=v10.0, but there is no run-time assembly compatible with win10-x64-aot.			0	
Error		One or more packages are incompatible with UAP,Version=v10.0 (win10-x64).			0	
Error		System.Diagnostics.Process 4.3.0 provides a compile-time reference assembly for System.Diagnostics.Process on UAP,Version=v10.0, but there is no run-time assembly compatible with win10-x86.			0	
Error		One or more packages are incompatible with UAP,Version=v10.0 (win10-arm-aot).			0	
Error		System.Diagnostics.Process 4.3.0 provides a compile-time reference assembly for System.Diagnostics.Process on UAP,Version=v10.0, but there is no run-time assembly compatible with win10-arm-aot.			0	
Error		One or more packages are incompatible with UAP,Version=v10.0 (win10-arm).			0	
Error		One or more packages are incompatible with UAP,Version=v10.0 (win10-x86-aot).			0	
Error		One or more packages are incompatible with UAP,Version=v10.0 (win10-x86).			0	
Error		System.Diagnostics.Process 4.3.0 provides a compile-time reference assembly for System.Diagnostics.Process on UAP,Version=v10.0, but there is no run-time assembly compatible with win10-x86-aot.			0	
Error		System.Diagnostics.Process 4.3.0 provides a compile-time reference assembly for System.Diagnostics.Process on UAP,Version=v10.0, but there is no run-time assembly compatible with win10-arm.			0	
Error		One or more packages are incompatible with UAP,Version=v10.0 (win10-x64-aot).			0	

[sguergachi]

@jskeet On a different system running on VS2017 I'm no longer able to upgrade Google.API.OAuth2.v2 from 1.23.x > 1.24.1 for UWP apps:

Restoring packages for C:\Users\magis\Source\Repos\Nudge\NudgeFrontEnd\NudgeToaster\project.json...
System.Diagnostics.Process 4.3.0 provides a compile-time reference assembly for System.Diagnostics.Process on UAP,Version=v10.0, but there is no run-time assembly compatible with win10-arm.
One or more packages are incompatible with UAP,Version=v10.0 (win10-arm).
System.Diagnostics.Process 4.3.0 provides a compile-time reference assembly for System.Diagnostics.Process on UAP,Version=v10.0, but there is no run-time assembly compatible with win10-arm-aot.
One or more packages are incompatible with UAP,Version=v10.0 (win10-arm-aot).
System.Diagnostics.Process 4.3.0 provides a compile-time reference assembly for System.Diagnostics.Process on UAP,Version=v10.0, but there is no run-time assembly compatible with win10-x64.
One or more packages are incompatible with UAP,Version=v10.0 (win10-x64).
System.Diagnostics.Process 4.3.0 provides a compile-time reference assembly for System.Diagnostics.Process on UAP,Version=v10.0, but there is no run-time assembly compatible with win10-x64-aot.
One or more packages are incompatible with UAP,Version=v10.0 (win10-x64-aot).
System.Diagnostics.Process 4.3.0 provides a compile-time reference assembly for System.Diagnostics.Process on UAP,Version=v10.0, but there is no run-time assembly compatible with win10-x86.
One or more packages are incompatible with UAP,Version=v10.0 (win10-x86).
System.Diagnostics.Process 4.3.0 provides a compile-time reference assembly for System.Diagnostics.Process on UAP,Version=v10.0, but there is no run-time assembly compatible with win10-x86-aot.
One or more packages are incompatible with UAP,Version=v10.0 (win10-x86-aot).
Package restore failed. Rolling back package changes for 'NudgeToaster'.
Time Elapsed: 00:00:01.7039597
========== Finished ==========

Don't know what's changed from the last version to cause this...

[jskeet]

@sguergachi: Hmm. It's possible that this is related to the auth stuff Chris was doing. Any chance you could raise a separate issue for this particular part, with any interesting details about your app? (If you're able to easily reproduce with a new app, that would be great.)

Ultimately though, we really don't support UWP at the moment. We're hoping that can change over time...

[chrisdunelm]

Yes, this will have been an inadvertent effect of #962. The netstandard target now includes a dep on System.Diagnostics.Process; and it's the netstandard target that will currently be used in a UWP app.

As Jon said, we don't support UWP; and this means we don't currently run any tests on the UWP platform before a release.

[chrisdunelm]

We will be remaining in v1.x for the packages in this repo; we will not be producing a v2 release any time soon (if at all).
However, we will continue to address as many issues as we can within v1.x releases.

Going through the items listed above:

Additional platform support

We will be starting to investigate UWP (native) and possibly Xamarin support shortly. We don't have a timeline for when this investigation will be complete, and we don't guarantee that we'll end up supporting these platforms, but will add support if feasible.

Remove support for various older platforms

Following on from the pattern we used in the v1.12 release we will remove support for Silverlight, Windows App 8 (UWP predecessor), Windows Phone 8.1, and Windows Phone Silverlight in the support packages (Google.Apis.Core, Google.Apis, Google.Apis.Auth, and Google.Apis.Auth.Mvc) in a future version. Generated API packages will still support all platforms, and will redirect these deprecated platforms to a suitable version of support libraries. In the same way that targeting .NET4.0 today will bring in v1.10 of the support packages.

Update all dependencies to the most recent major version

Removed dependencies: BouncyCastle (#880), Log4net (#890).
Remaining dependencies:

• ZLib.Portable.Signed: This dependency will be removed once the platform deprecation (above) has been done.
• Newtonsoft.Json: We will upgrade our dependency from v7.0.1 to vLatest in a future release.

Remove RSACryptoProvider from the public API

We cannot do this (would be a breaking change), so will continue with the current solution.
RSACryptoProvider is in the public API in the .NET4.5 target.
RSA is in the public API for netstandard, and will be used for all future targeted platforms.

Change the package and project structure

We cannot change the package structure - i.e. which public entities are in which DLL - (would be a breaking change), so will continue with the current package/dll structure. This is more of an internal project/build issue than a user issue, so only affects people building and working on the project source.

We will tidy up the project structure in a future PR, and probably move to VS2017/csproj for a targets. The produced packages will still be consumable in earlier/different versions of VS.

Add get-range to IMediaDownloader

We cannot do this (would be a breaking change), but may add a DownloadRangeAsync (or something similar) to the underlying public MediaDownloader class in a future release.

Change the return type from void to IProgressDownload of generated code that uses MediaDownloader.Download(...)

We cannot do this directly, as it would be a breaking change.
However we may add an extra method to do this in a future release.

Add request/response logging

This is done #935

Sort out some timezone issues (#853)

We cannot do this, because it would be a breaking change. We expect to leave this as it is.

Revisit all auth mechanisms to ensure platform support

This is done; GoogleWebAuthorizationBroker is now available in netstandard #962, which I think means there is auth mechanism parity between .NET desktop and netstandard.

[chrisdunelm]

All items split out into separate issues. Closing this issue.