Pin dependencies to major version ranges

[felixwang9817]

We have recently had a bunch of issues due to dependencies not being limited appropriately. For example, having google-cloud-bigquery>=2.28.1 led to issues when google-cloud-bigquery released breaking changes in v3.0.0: see #2537 for the issue and #2554 which included the fix. Similarly, #2484 occurred since our protobuf dependency was not limited.

I think we should limit dependencies to the next major version. For example, if we currently use version N of a package, we should also limit it to v<(N+1). This way we are not exposed to breaking changes in all our upstream dependencies, while also maintaining a reasonable amount of flexibility for users. If a version N+1 is released and users want us to support it, they can let us know and we can add support; limiting to v<(N+1) just ensures that we aren't being broken all the time.

[achals]

I like this way of declaring dependencies as well: https://stackoverflow.com/a/50080281/1735989

[felixwang9817]

I like this way of declaring dependencies as well: https://stackoverflow.com/a/50080281/1735989

Ooh this looks super neat, but it seems to only work for minor versions - e.g. if I want to specify >=8.0.0,<9.0.0, I can't do ~=8. But this will definitely work for most of our dependencies.

Edit: oops I'm wrong, the below comment is right.

[judahrand]

if I want to specify >=8.0.0,<9.0.0

That's the same as ~=8.0 isn't it? Which is exactly identical to >=8.0,==8.*. But I suppose that still doesn't allow something even more specific like >=8.0.1,==8.*.