Can this framework be used to monitor core resources like namespaces?

[kopf-archiver]

An issue by etopeter at 2019-07-08 22:56:26+00:00
Original URL: zalando-incubator/kopf#137
 

Expected Behavior

I want to handle namespaces events. Can this be done with kopf?

Actual Behavior

[2019-07-08 15:42:29,693] kopf.clients.auth    [DEBUG   ] configured via kubeconfig file
[2019-07-08 15:42:30,561] kopf.engines.peering [WARNING ] Default peering object not found, falling back to the standalone mode.
Traceback (most recent call last):
  File "/usr/local/bin/kopf", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/kopf/cli.py", line 28, in wrapper
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/kopf/cli.py", line 59, in run
    peering_name=peering_name,
  File "/usr/local/lib/python3.7/site-packages/kopf/reactor/queueing.py", line 271, in run
    task.result()  # can raise the regular (non-cancellation) exceptions.
  File "/usr/local/lib/python3.7/site-packages/kopf/reactor/queueing.py", line 78, in watcher
    async for event in watching.infinite_watch(resource=resource, namespace=namespace):
  File "/usr/local/lib/python3.7/site-packages/kopf/clients/watching.py", line 132, in infinite_watch
    async for event in streaming_watch(resource=resource, namespace=namespace):
  File "/usr/local/lib/python3.7/site-packages/kopf/clients/watching.py", line 81, in streaming_watch
    rsp = fetching.list_objs(resource=resource, namespace=namespace)
  File "/usr/local/lib/python3.7/site-packages/kopf/clients/fetching.py", line 64, in list_objs
    plural=resource.plural,
  File "/usr/local/lib/python3.7/site-packages/kubernetes/client/apis/custom_objects_api.py", line 1307, in list_cluster_custom_object
    (data) = self.list_cluster_custom_object_with_http_info(group, version, plural, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/kubernetes/client/apis/custom_objects_api.py", line 1413, in list_cluster_custom_object_with_http_info
    collection_formats=collection_formats)
  File "/usr/local/lib/python3.7/site-packages/kubernetes/client/api_client.py", line 334, in call_api
    _return_http_data_only, collection_formats, _preload_content, _request_timeout)
  File "/usr/local/lib/python3.7/site-packages/kubernetes/client/api_client.py", line 168, in __call_api
    _request_timeout=_request_timeout)
  File "/usr/local/lib/python3.7/site-packages/kubernetes/client/api_client.py", line 355, in request
    headers=headers)
  File "/usr/local/lib/python3.7/site-packages/kubernetes/client/rest.py", line 231, in GET
    query_params=query_params)
  File "/usr/local/lib/python3.7/site-packages/kubernetes/client/rest.py", line 222, in request
    raise ApiException(http_resp=r)
kubernetes.client.rest.ApiException: (404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({'Audit-Id': '818d48ae-127f-4dc7-b767-337cc49be806', 'Content-Type': 'text/plain; charset=utf-8', 'X-Content-Type-Options': 'nosniff', 'Date': 'Mon, 08 Jul 2019 22:42:30 GMT', 'Content-Length': '19'})
HTTP response body: 404 page not found

Steps to Reproduce the Problem

1. Create sample code sample.py:

import kopf
@kopf.on.create('core', 'v1', 'namespaces')
def module_handler(meta, spec, namespace, logger, **kwargs):
    logger.info(f"Namespace was created")

2. Run code kopf run sample.py --verbose

Specifications

• Platform:
• Kubernetes version: v1.12.6
• Python version: 3.7.1
• Python packages installed:
  aiohttp==3.5.4
  aiojobs==0.2.2
  async-timeout==3.0.1
  attrs==19.1.0
  awscli==1.16.169
  boto3==1.9.134
  botocore==1.12.159
  cachetools==3.1.1
  certifi==2019.6.16
  chardet==3.0.4
  Click==7.0
  colorama==0.3.9
  docutils==0.14
  google-auth==1.6.3
  idna==2.8
  iso8601==0.1.12
  jmespath==0.9.4
  kopf==0.16
  kubernetes==9.0.0
  multidict==4.5.2
  oauthlib==3.0.1
  packaging==19.0
  pip==19.1.1
  pip-review==1.0
  pyasn1==0.4.5
  pyasn1-modules==0.2.5
  pyparsing==2.4.0
  python-dateutil==2.8.0
  PyYAML==3.13
  requests==2.22.0
  requests-oauthlib==1.2.0
  rsa==3.4.2
  s3transfer==0.2.0
  setuptools==41.0.0
  six==1.12.0
  urllib3==1.24.3
  websocket-client==0.56.0
  wpm==1.51.4
  yarl==1.3.0

---

> <a href="https://github.com/nolar"><img align="left" height="30" src="https://avatars0.githubusercontent.com/u/544296?v=4"></a>            Commented by [nolar](https://github.com/nolar) at _2019-07-09 08:06:30+00:00_
> &nbsp;

[etopeter](https://github.com/etopeter) Yes and no.

**No:** in the current version `0.18`, only the custom resources are supported. Built-in resources are not supported (mostly those of `apiVersion: v1`; those of newer api-versions with `group/version` syntax should work in theory, never were tested in practice).

**Yes:** With #110 merged (switching to pykube-ng internally), all resources will be supported, including pods and namespaces. This is the last step of approximately 2-month journey, and I expect it to be released in few days.

Meanwhile, for experiments, you can try installing [`kopf==0.17.dev1`](https://pypi.org/project/kopf/0.17.dev1/) by explicitly specifying the version number (it is an unofficial temporary pre-release). It contains the code to handle all the builtin resources as an experiment, but lacks some of the newest Kopf's features and bugfixes of versions 0.17 & 0.18 (see [changelogs](https://github.com/nolar/kopf/releases)). Be prepared to switch to 0.19 as soon as it is released — it will contain all the necessary features officially.

**PS:** I tried this code with namespaces, and it worked (with some unwanted effects like a finalizer on the namespaces — at that time; solved now): https://twitter.com/nolar/status/1139630966408994817