Using for ThirdPartyResource's

[tiaanl]

I've started using this library and it works very well.

When extending the API with ThirdPartyResource resources, things become a major issue. Using the dynamic part doesn't work well because I don't want to convert all my objects into map[string]interface{} objects. The rest part alone also doesn't work well, because there are no Encoders/Decoders.

I'm sure all the tools required to start working with my own resources is available, but get them all up and running is just a major pain point currently.

Right now I don't know enough of the internals to be productive towards a solution, but any solution would be MUCH appreciated.

[lavalamp]

@smarterclayton @krousey Do we have a convenient go struct <-> TPR codec laying around somewhere?

[jimmidyson]

/cc @fabxc

[tiaanl]

Okay, so I've made progress on my side. Using the rest part of the library is quite useful, but the biggest problem is still plenty of little details that is not really reported when not done correctly.

Doing a GET for a list of objects you need the following into object:

type ProjectList struct {
    unversioned.TypeMeta `json:",inline"`
    unversioned.ListMeta `json:"metadata,omitempty"`

    Items []Project `json:"items"`
}

Notice the unversioned.ListMeta here. Without it, you will just not get the right result or something like not a registered type. Very confusing.

The final issue I have here still is that when I try to pull that same list with this call:

type ProjectSpec struct {
    Description string `json:"description"`
}

type Project struct {
    unversioned.TypeMeta `json:",inline"`
    api.ObjectMeta        `json:"metadata,omitempty"`

    Spec ProjectSpec `json:"spec"`
}


    projectList := ProjectList{}
    err := ah.Client.Rest.Get().
        Namespace("default").
        Resource("projects").
        Do().
        Into(&projectList)

The list object gets the right details, but the actual items do not. They are just blank with no data. I've figured out that it is the ObjectMeta type that is causing this error. If I recreate my own ObjectMeta struct, then it works perfectly. So I think there is another interface-detecting mechanism somewhere in the decoding that doesn't like the ObjectMeta type.

If I get all of this to work, I would definitely like to write a doc for how to use all this.

[deads2k]

The list object gets the right details, but the actual items do not. They are just blank with no data. I've figured out that it is the ObjectMeta type that is causing this error. If I recreate my own ObjectMeta struct, then it works perfectly. So I think there is another interface-detecting mechanism somewhere in the decoding that doesn't like the ObjectMeta type.

You might be getting bit by ugorji codec generation. We had a problem with it downstream that we resolved by carrying a patch that deleted the generated codecs until we generated them for our types.

Also, if you're using third party resources, make sure you're aware of the known issues/missing features: kubernetes/enhancements#95

[justinsb]

I hit the same thing, and copying ObjectMeta into my package fixes the problem (thanks for the tip @tiaanl)

Do we know why this happens? Is there an issue tracking this @deads2k?

[deads2k]

Do we know why this happens? Is there an issue tracking this @deads2k?

I think it's because there's an interface check that passes because the object is anonymously included, but I haven't dug into this case to make sure. I don't know of an issue. We discovered it in OpenShift and didn't think it was reasonable to try to force reversion or object shape change to accommodate it. I think we fixed the correctness problems in the codec generator, so it ought to be safe to use for generating a codec now.

[lavalamp]

It's ugorji--either run ugorji to generate en/decoders for your types, or
add a top level Decode() (double check the name) method that calls the
default json machinery.

On Mon, Oct 10, 2016 at 4:37 AM, David Eads notifications@github.com
wrote:

Do we know why this happens? Is there an issue tracking this @deads2k
https://github.com/deads2k?

I think it's because there's an interface check that passes because the
object is anonymously included, but I haven't dug into this case to make
sure. I don't know of an issue. We discovered it in OpenShift and didn't
think it was reasonable to try to force reversion or object shape change to
accommodate it. I think we fixed the correctness problems in the codec
generator, so it ought to be safe to use for generating a codec now.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#8 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAngliBN6xCuAk7JiJnMqysUShjYDzvhks5qyiNfgaJpZM4KP30A
.

[wfarr]

add a top level Decode() (double check the name) method that calls the default json machinery

I'm encountering similar issues to others in this issue.

Please excuse my ignorance here. I'm playing catch-up on the internals of this repository still. Would you be able to point me towards what the Decode() method should be added to? Is it my custom types being passed to Into()?

I can also publish some of the code that I've been working on for our use case publicly to demonstrate what I'm doing (or more likely doing wrong 😉 ) if context is lacking (that said, my work is modeled pretty closely after what I've read from others here in this issue).

[caesarxuchao]

Would you be able to point me towards what the Decode() method should be added to? Is it my custom types being passed to Into()?

Yes.

Or you can take a look at this file, which is the generated ugorji code in the k8s repository, you can generate similar code for your types. The code is generated using this script. (I think write your own Decode() is easier, k8s uses ugorji because it's more efficient than the default json package).

[caesarxuchao]

Paste the slack discussion here since it might be useful for other people.

As ugorji doc says:
"if value implements encoding.TextUnmarshaler and format is a text format, call its UnmarshalText method"

So you need to implement the UnmarshalText() method for your type, you can use json.Unmarshal to implement it. For example:

type YourType struct {
}

func (yourType *YourType) UnmarshalText(data []byte) error {
  return  json.Unmarshal(data, yourType)
}

[tiaanl]

I created a repo to test just accessing the third party resources.

https://github.com/tiaanl/kube-tpr-demo

It has a few simple comands to interact with k8s.

The first issue with using the dynamic package is solved by registering my types with api.Schema.

The only last issue is still the decoding of the json into the correct types. Ive tried all the recommendations in this thread, except for generating that huge file for my type, which tbh is entirely overkill for trying to create something this simple.

Being able to store your meta data inside k8s, using the api versioning and all the other benefits you get for free would be fantastic. No more separate etcd with boilerplate code you have to write yourself.

Any help on this issue would be greatly appreciated.

[caesarxuchao]

@tiaanl I heard the UnmarshalText method I pasted above didn't work. I'll debug it these days and report back here, since it's blocking so many people.

[wfarr]

Thanks so much! I know the indirection and back-and-forth can be really
frustrating and time consuming. I appreciate your attention and effort
towards helping suss these issues out. ❤️
On Sun, Oct 30, 2016 at 6:08 PM Chao Xu notifications@github.com wrote:

@tiaanl https://github.com/tiaanl I heard the UnmarshalText method I
pasted above didn't work. I'll debug it these days and report back here,
since it's blocking so many people.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#8 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAAyxHQQugCTzfwTc_HnmKGL6MV-Rd1ks5q5RVMgaJpZM4KP30A
.

[caesarxuchao]

Finally got it work. See "Problem 1" and "Problem 2" in the code: See wfarr/k8s-tpr-playground#1.

[caesarxuchao]

Reported issue ugorji/go#178