1.6 post-rebase tasks

[sttts]

Blockers:

• @soltysh ensure there are no fields with semantic differences between nil/[] or nil/{}, and if there are, they use alias nullable fields for protobuf and explicitly test round tripping (fuzzer tests won't catch the difference now)
• switch storage version of HPA to autoscaling/v1 for 1.6
• @sttts resurrect extensions/HPA API Resurrect extensions/v1beta1.HPA #14021
• potentially enable GarbageCollection if not having them leads to regressions
• @liggitt The scopes on some types were being serialized incorrectly and this resulted in failing open. I've switched to manual conversions to move the rebase forward TO REVERT: fix SAR serialization sttts/origin#100 (comment) (Fix empty slice conversion for aliased slice types #14132)
• @enj Fix flake in TestEtcd2StoragePath TestEtcd2StoragePath flake #14020 -> Wait for SA creation in etcd test #14051, sync origin authz resources to rbac #14064 (review)

 --- FAIL: TestEtcd2StoragePath (34.55s)
        etcd_storage_path_test.go:960: failed to create stub for Pod from github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/api/v1:
n:""}, Status:"Failure", Message:"pods \"pod1\" is forbidden: service account etcdstoragepathtestnamespace/default was not found, retry af
        etcd_storage_path_test.go:960: failed to create stub for PolicyBinding from github.com/openshift/origin/pkg/authorization/api/v1:
:""}, Status:"Failure", Message:"policybindings.authorization.openshift.io \":default\" already exists", Reason:"AlreadyExists", Details:(
        etcd_storage_path_test.go:960: failed to create stub for Policy from github.com/openshift/origin/pkg/authorization/api/v1: &errors
tatus:"Failure", Message:"policies.authorization.openshift.io \"default\" already exists", Reason:"AlreadyExists", Details:(*v1.StatusDeta
        etcd.go:126: dumping etcd to "/tmp/openshift/test-integration/etcd-dump-TestEtcd2StoragePath.json"

• Re-enable broken e2e tests (compare 8f4cf9b):
  • Basic StatefulSet functionality \[StatefulSetBasic\] should provide basic identity Fix default watch cache size to be 1000 instead of upstream's 100 #14052
  • SchedulerPredicates [Serial] validates resource limits of pods that are allowed to run [Conformance]
• oadm migrate image-references fails in CI oadm migrate image-references fails in CI #13924 – oadm migrate image-references fails in CI #13924 is closed
• @sttts disable rbac storage hook Disable bootstrap-roles post-start hook #14002
• verify preferred versions in API discovery (both order and preferredVersion) (autoscaling/v2alpha1 is marked as preferred, storage.k8s.io v1beta1 vs v1, etc) https://github.com/openshift/origin/pull/15160/files
• @mfojtik disable alpha APIs, @DirectXMan12 decide about hpa v2alpha1 Rebase 1.6.1 #13653 (comment)
• @ncdc error cases for conversion in describe: Rebase 1.6.1 #13653 (comment), Rebase 1.6.1 #13653 (comment). PR Switch back to kapi instead of v1 for RCs in deployment describer #13992
• @ncdc nil-pointer check in describe Rebase 1.6.1 #13653 (comment). PR Don't show policy rules with attribute restrictions #14034
• Fix install + upgrade test 3.6 Upgrade Failure: 50 k8s e2e tests fail #13997
• Fix all-in-one networking GSSAPI extended test failure on deployment config change after 1.6.1 rebase #13927 (comment) Hairpin does not work with the all-in-one #14031

Further work:

• @ncdc get lister-gen and informer-gen working with the new api groups. PR Tweak clientset, lister, informer generated code locations #13982
• Dump etcd v3 contents in integration tests as we do for v2 e2e database pod never comes up #14090 (comment) Issue already open: Dump etcdv3 contents in tests #11837
• switch to generated clients / delete hand-written clients
• switch to external versions in controllers
• use secret shared informer in build/deploy/etc. controllers
• match apigroup package structure with upstream pkg/apis/foo
• get rid of sc.InternalDetermineEffectiveSecurityContext, as this goes away upstream (compare Rebase 1.6.1 #13653 (review))
• convert the describer to use a versioned clientset where possible (eg. deployments, see pkg/cmd/cli/describe/deployments.go) https://github.com/openshift/origin/pull/13653/files/20348bb298cb570f49a78ff37c3ffec00255b2fb..dfd58523e1de61d9debdbd5af7aa9ac3e69111c9#diff-10042ecb2622b0122245db3a12c79f1a
• @ncdc: decide if we want to remove /status access for namespaced roles (https://github.com/openshift/origin/pull/13653/files/326fc9b4a9752f3d0d60fecf9e68017d5aa543f4..55dc4afdb6f775cfadd95c14161c1aaf153c7968#r111140469)
• @derekwaynecarr decide whether to enable taints
• remove setting NetworkPluginDir from pkg/cmd/server/kubernetes/node/node_config.go (cc @dcbw)
• re-enable etcd3, @liggitt to determine what's current re protobuf
• @liggitt replace originLongRunningRequestRE? with our own resource attributes check that tests for a buildconfigs resource and a instantiatebinary or imagestreamimports subresource? Rebase 1.6.1 #13653 (comment)
• @liggitt test for preferred versions in master_config_test.go Rebase 1.6.1 #13653 (comment)
• @liggitt refactor admission temp config files Rebase 1.6.1 #13653 (comment)
• upstream fix for double-counting quota for objects in more than one API group (plus tests) (and ensure admission quotas from both API groups)
• adopt kube rbac roles @deads2k @enj -> start bootstrapping cluster roles from kube #14026 prove the clientbuilder works with RC controller #14033

[mfojtik]

@sttts @enj @deads2k @smarterclayton for the kube alpha API's i found that we already have a way to disabled them: kubernetesMasterConfig.disabledAPIGroupVersions. I don't think we want to disable them by default, but if users want they can selectively disable. Is that enough?

[enj]

Why shouldn't they be disabled by default? I would want all half done features to be off, and then I can turn them on if I understand the risks.

[mfojtik]

@enj if we start setting this now, it will probably break existing clusters? ;-)

[liggitt]

if we start setting this now, it will probably break existing clusters? ;-)

we don't intentionally enable alpha types... what existing 1.6 clusters are you referring to?

[mfojtik]

@liggitt where we disabling them? the kubernetesMasterConfig.disabledAPIGroupVersions seems to be empty for me.

[deads2k]

@liggitt where we disabling them? the kubernetesMasterConfig.disabledAPIGroupVersions seems to be empty for me.

As I recall, there is a default whitelist too. Hardcoded.

[mfojtik]

@deads2k @liggitt @sttts anything critical missing from this list for 3.6?

[sttts]

@mfojtik the item about alpha versions is solved via #14941 I guess. In there, settings.k8s.io is disabled.

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[soltysh]

I think at this point all of the points are either handled or already outdated.
/close