Skip to content

Events can not reference objects that use a different name validation #127594

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
aojea opened this issue Sep 24, 2024 · 12 comments · Fixed by #129790
Closed

Events can not reference objects that use a different name validation #127594

aojea opened this issue Sep 24, 2024 · 12 comments · Fixed by #129790
Assignees
Labels
sig/instrumentation Categorizes an issue or PR as relevant to SIG Instrumentation. triage/accepted Indicates an issue or PR is ready to be actively worked on.

Comments

@aojea
Copy link
Member

aojea commented Sep 24, 2024

Seen in #127588

E0924 12:16:50.003934 1 event_broadcaster.go:270] "Server rejected event (will not retry!)" err="Event "fd00:10:233::103.17f82d400041d81f" is invalid: metadata.name: Invalid value: "fd00:10:233::103.17f82d400041d81f":
a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is 'a-z0-9
?(\.a-z0-9?)*')" event="&Event{ObjectMeta:{fd00:10:233::103.17f82d400041d81f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},EventTime:2024-09-24 12:16:50.002035963 +0000 UTC
m=+179.166665408,Series:nil,ReportingController:ipallocator-repair-controller,ReportingInstance:ipallocator-repair-controller-svc-control-plane,Action:IPAddressAllocation,Reason:IPAddressNotAllocated,Regarding:{IPAddress fd00:1
0:233::103 86ced933-1da5-4fac-b2e3-ab849185e2d5 networking.k8s.io/v1beta1 7070 },Related:nil,Note:IPAddress: fd00:10:233::103 for Service default/v6-service appears to have leaked: cleaning up,Type:Warning,DeprecatedSource:{ },D
eprecatedFirstTimestamp:0001-01-01 00:00:00 +0000 UTC,DeprecatedLastTimestamp:0001-01-01 00:00:00 +0000 UTC,DeprecatedCount:0,}"

An IPAddress object can contain an IPv6 address fd00:10:233::103
The event recorder creates a new Event object based on the referenced object

  r.recorder.Eventf(ipAddress, nil, v1.EventTypeWarning, "IPAddressNotAllocated", "IPAddressAllocation", "IPAddress %s appears to have been modified, not referencing a Service %v: cleaning up", ipAddress.Name, ipAddress.Spec.ParentRef)

func (recorder *recorderImpl) makeEvent(refRegarding *v1.ObjectReference, refRelated *v1.ObjectReference, timestamp metav1.MicroTime, eventtype, reason, message string, reportingController string, reportingInstance string, action string) *v1beta1.Event {
t := metav1.Time{Time: recorder.clock.Now()}
namespace := refRegarding.Namespace
if namespace == "" {
namespace = metav1.NamespaceSystem
}
return &v1beta1.Event{
ObjectMeta: metav1.ObjectMeta{
Name: fmt.Sprintf("%v.%x", refRegarding.Name, t.UnixNano()),

I can see two options:

  1. adapt the controller code to normalize the IPv6 addresses to meet the Event name convention
  2. modify r.recorder.Eventf to make valid names on the Events objects

The result will be the same with 1 or 2, the difference is that option 2. will be valid for all the objects that use names not valid for the existing Events validation

/sig instrumentation

@k8s-ci-robot k8s-ci-robot added sig/instrumentation Categorizes an issue or PR as relevant to SIG Instrumentation. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Sep 24, 2024
@aojea
Copy link
Member Author

aojea commented Sep 24, 2024

/cc @thockin @logicalhan @dgrisonnet

@dgrisonnet
Copy link
Member

The first solution you propose will likely make the Regarding field of the Event invalid since both the name and that field are set based on the given object.

As for changing the name in the client, we will need to completely get rid of the regarded object name, otherwise the validation will never pass for IPAddresses because of the ':' character. But that would be quite a big change since AFAIK Events have always had that form and I am also not sure if any code was built with that naming structure in mind.

Another option could be to soften the event validation from:

allErrs = append(allErrs, ValidateObjectMeta(&event.ObjectMeta, true, apimachineryvalidation.NameIsDNSSubdomain, field.NewPath("metadata"))...)

This check was added for the events/v1 API only, the core/v1 Events' name was never validated before and that's still the case today. I found some context for that change in #91645 (comment), and it sounds fairly safe to soften the validation to support IPv6 values.

/cc @wojtek-t

@thockin
Copy link
Member

thockin commented Sep 24, 2024

There are a few types which use name formats which are not DNS-subdomain compatible:

$ k get roles -A
NAMESPACE     NAME                                              CREATED AT
kube-public   system:controller:bootstrap-signer                2024-03-28T22:56:25Z
kube-system   system::leader-locking-cloud-controller-manager   2024-03-28T22:56:44Z
kube-system   system::leader-locking-kube-controller-manager    2024-03-28T22:56:25Z
kube-system   system::leader-locking-kube-scheduler             2024-03-28T22:56:25Z

$ k get rolebindings -A
NAMESPACE     NAME                                                ROLE                                                   AGE
kube-public   system:controller:bootstrap-signer                  Role/system:controller:bootstrap-signer                179d
kube-system   system::extension-apiserver-authentication-reader   Role/extension-apiserver-authentication-reader         179d
kube-system   system::leader-locking-cloud-controller-manager     Role/system::leader-locking-cloud-controller-manager   179d
kube-system   system::leader-locking-kube-controller-manager      Role/system::leader-locking-kube-controller-manager    179d
kube-system   system::leader-locking-kube-scheduler               Role/system::leader-locking-kube-scheduler             179d

As as aside: I wish we had not done this and I REALLY wish we had not used : in some places and :: in others (@liggitt may know the history).

I don't think it's a valid assumption that the "regarding" object's name can be the basis for the event's name. It's not at all a safe guarantee, and "name" is mostly irrelevant to events. It's common to have deployments and services named the same thing, so I could have events about both and they would just be mixed together.

staging/src/k8s.io/client-go/tools/events/event_recorder.go

func (recorder *recorderImpl) makeEvent(refRegarding *v1.ObjectReference, refRelated *v1.ObjectReference, timestamp metav1.MicroTime, eventtype, reason, message string, reportingController string, reportingInstance string, action string) *eventsv1.Event {                                   
    t := metav1.Time{Time: recorder.clock.Now()}    
    namespace := refRegarding.Namespace     
    if namespace == "" {    
        namespace = metav1.NamespaceDefault     
    }    
    return &eventsv1.Event{    
        ObjectMeta: metav1.ObjectMeta{    
            Name:      fmt.Sprintf("%v.%x", refRegarding.Name, t.UnixNano()),    
            Namespace: namespace,    
        },    

It might be better to use the refRegarding.UID if the goal is to collate events, but the UID may not be present. So maybe md5sum(refRegarding.Namespace + refRegarding.Name) or simply base64 encoded + truncate for length. Or just use generateName and give up on collation?

@liggitt
Copy link
Member

liggitt commented Sep 24, 2024


As as aside: I wish we had not done this and I REALLY wish we had not used : in some places and :: in others (@liggitt may know the history).

#45966 looks like the first and primary addition of :: ... I don't know the rationale there

@dgrisonnet
Copy link
Member

/assign
/triage accepted

@k8s-ci-robot k8s-ci-robot added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Oct 3, 2024
@dgrisonnet
Copy link
Member

dgrisonnet commented Oct 3, 2024

I don't think it's a valid assumption that the "regarding" object's name can be the basis for the event's name. It's not at all a safe guarantee, and "name" is mostly irrelevant to events.

Yes that's true, even kubectl doesn't show the event name by default, you'd have to specify -owide to see it, so I doubt anyone really use the names and it is very unlikely that users would delete or get specific events by name.

It might be better to use the refRegarding.UID if the goal is to collate events, but the UID may not be present. So maybe md5sum(refRegarding.Namespace + refRegarding.Name) or simply base64 encoded + truncate for length. Or just use generateName and give up on collation?

We are already combining events client-side based on the following fields:

type eventKey struct {
eventType string
action string
reason string
reportingController string
reportingInstance string
regarding corev1.ObjectReference
related corev1.ObjectReference
}

I don't think there are use-cases where anyone would need the name to have any particular meaning as all the relevant information to visualize and manage events are already present in the other fields.

We have a notion of event key in both implementations to merge similar events together, maybe we could reuse that and encode it?

core/v1 client code ref:

// getEventKey builds unique event key based on source, involvedObject, reason, message
func getEventKey(event *v1.Event) string {
return strings.Join([]string{
event.Source.Component,
event.Source.Host,
event.InvolvedObject.Kind,
event.InvolvedObject.Namespace,
event.InvolvedObject.Name,
event.InvolvedObject.FieldPath,
string(event.InvolvedObject.UID),
event.InvolvedObject.APIVersion,
event.Type,
event.Reason,
event.Message,
},
"")
}
// getSpamKey builds unique event key based on source, involvedObject
func getSpamKey(event *v1.Event) string {
return strings.Join([]string{
event.Source.Component,
event.Source.Host,
event.InvolvedObject.Kind,
event.InvolvedObject.Namespace,
event.InvolvedObject.Name,
string(event.InvolvedObject.UID),
event.InvolvedObject.APIVersion,
event.Type,
},
"")
}

@wojtek-t
Copy link
Member

wojtek-t commented Oct 4, 2024

I don't think it's a valid assumption that the "regarding" object's name can be the basis for the event's name. It's not at all a safe guarantee, and "name" is mostly irrelevant to events.

+1

We have a notion of event key in both implementations to merge similar events together, maybe we could reuse that and encode it?

+1 to that - this logic was created to distinguish whether two events are about the same or about different things. Making a name to be function of it makes perfect sense to me.

@aojea
Copy link
Member Author

aojea commented Feb 11, 2025

@dgrisonnet @wojtek-t I'd like to get this merged #129790 (comment) and also explain why I think the proposal I'm sending of normalizing the name instead of using the event key is better.

To generate the eventKey you need to first create the event objects, so the code will look like

func (recorder *recorderImpl) makeEvent(refRegarding *v1.ObjectReference, refRelated *v1.ObjectReference, timestamp metav1.MicroTime, eventtype, reason, message string, reportingController string, reportingInstance string, action string) *eventsv1.Event {
	t := metav1.Time{Time: recorder.clock.Now()}
	namespace := refRegarding.Namespace
	if namespace == "" {
		namespace = metav1.NamespaceDefault
	}

   event := &eventsv1.Event{
}}}
event.Name = getEventKey(event)

I don't feel that adding this kind of pattern of generating a name of an object based on the same object without a field is ideal and just normalizing is more simpler

@aojea
Copy link
Member Author

aojea commented Feb 11, 2025

After this comment from @liggitt #129790 (comment) I may step back on my proposal, scratch my previous comment

@dgrisonnet
Copy link
Member

I don't feel that adding this kind of pattern of generating a name of an object based on the same object without a field is ideal and just normalizing is more simpler

I think it is fine for events since we don't use their name to generate the keys or anything else. This field is just here because events are Kubernetes objects and they must have a name.

@aojea
Copy link
Member Author

aojea commented Feb 17, 2025

I think it is fine for events since we don't use their name to generate the keys or anything else. This field is just here because events are Kubernetes objects and they must have a name.

hmmm , we still have the same problem, the event key is not guaranteed to pass the event object name validation

@aojea
Copy link
Member Author

aojea commented Feb 17, 2025

I changed the approach #129790

  1. generated name is valid -> keep it as it is today
  2. is not valid -> change ref name by a random generated string of 63 alphanumeric characters

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/instrumentation Categorizes an issue or PR as relevant to SIG Instrumentation. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants