Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions providers/apache/kyuubi/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
.. Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

.. http://www.apache.org/licenses/LICENSE-2.0

.. Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.

Package ``apache-airflow-providers-apache-kyuubi``
==================================================

Release: ``1.0.0``


`Apache Kyuubi <https://kyuubi.apache.org/>`__


Provider package
----------------

This is a provider package for ``apache.kyuubi`` provider. All classes for this provider package
are in ``airflow.providers.apache.kyuubi`` python package.

Installation
------------

You can install this package on top of an existing Airflow installation via
``pip install apache-airflow-providers-apache-kyuubi``

Requirements
------------

======================================= ==================
PIP package Version required
======================================= ==================
``apache-airflow`` ``>=2.8.0``
``apache-airflow-providers-apache-hive`` ``>=6.0.0``
======================================= ==================
34 changes: 34 additions & 0 deletions providers/apache/kyuubi/docs/commits.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
.. Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

.. http://www.apache.org/licenses/LICENSE-2.0

.. Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.

.. NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE OVERWRITTEN!

.. IF YOU WANT TO MODIFY THIS FILE, YOU SHOULD MODIFY THE TEMPLATE
`PROVIDER_COMMITS_TEMPLATE.rst.jinja2` IN the `dev/breeze/src/airflow_breeze/templates` DIRECTORY

.. THE REMAINDER OF THE FILE IS AUTOMATICALLY GENERATED. IT WILL BE OVERWRITTEN!

Package apache-airflow-providers-apache-kyuubi
----------------------------------------------

`Apache Kyuubi <https://kyuubi.apache.org/>`__


This is detailed commit list of changes for versions provider package: ``apache.kyuubi``.
For high-level changelog, see :doc:`package information including changelog <index>`.

.. airflow-providers-commits::
49 changes: 49 additions & 0 deletions providers/apache/kyuubi/docs/connections.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
.. Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

.. http://www.apache.org/licenses/LICENSE-2.0

.. Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.

.. _howto/connection:kyuubi:

Apache Kyuubi Connection
========================

The Apache Kyuubi connection type enables integrations with Apache Kyuubi.

Default Connection IDs
----------------------

Kyuubi Hook uses ``kyuubi_default`` by default.

Configuring the Connection
--------------------------

Host (required)
The host to connect to.

Port (required)
The port to connect to. Default is 10009.

Login (optional)
Specify the user name to connect.

Password (optional)
Specify the password to connect.

Extra (optional)
Specify any extra parameters (as json dictionary) that can be used in the connection.

* ``beeline_path``: Path to the Kyuubi Beeline executable.
* ``use_beeline``: Whether to use Beeline or Hive CLI. Default is True.
57 changes: 57 additions & 0 deletions providers/apache/kyuubi/docs/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
.. Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

.. http://www.apache.org/licenses/LICENSE-2.0

.. Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.

``apache-airflow-providers-apache-kyuubi``
============================================

Content
-------

.. toctree::
:maxdepth: 1
:caption: Guides

connections
operators

.. toctree::
:maxdepth: 1
:caption: References

Python API <_api/airflow/providers/apache/kyuubi/index>

.. toctree::
:maxdepth: 1
:caption: Resources

Apache Kyuubi Website <https://kyuubi.apache.org/>

.. toctree::
:maxdepth: 1
:caption: Commits

commits

Package apache-airflow-providers-apache-kyuubi
------------------------------------------------

`Apache Kyuubi <https://kyuubi.apache.org/>`__ is a distributed and multi-tenant gateway to provide serverless SQL on Lakehouses.

Release: 1.0.0

:doc:`Detailed list of commits <commits>`

40 changes: 40 additions & 0 deletions providers/apache/kyuubi/docs/operators.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
.. Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

.. http://www.apache.org/licenses/LICENSE-2.0

.. Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.

.. _howto/operator:KyuubiOperator:

Apache Kyuubi Operator
======================

Use the :class:`~airflow.providers.apache.kyuubi.operators.kyuubi.KyuubiOperator` to execute
Hive Query Language (HQL) statements against Kyuubi.

Using the Operator
^^^^^^^^^^^^^^^^^^

.. code-block:: python

from airflow.providers.apache.kyuubi.operators.kyuubi import KyuubiOperator

t1 = KyuubiOperator(
task_id="kyuubi_task",
hql="SELECT * FROM table",
kyuubi_conn_id="kyuubi_default",
spark_queue="root.default",
spark_app_name="my_kyuubi_app",
spark_conf={"spark.executor.memory": "4g"},
)
50 changes: 50 additions & 0 deletions providers/apache/kyuubi/provider.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

---
package-name: apache-airflow-providers-apache-kyuubi
name: Apache Kyuubi
description: |
`Apache Kyuubi <https://kyuubi.apache.org/>`__

state: ready
source-date-epoch: 1700000000
versions:
- 1.0.0

dependencies:
- apache-airflow>=2.8.0
- apache-airflow-providers-apache-hive>=6.0.0

integrations:
- integration-name: Apache Kyuubi
external-doc-url: https://kyuubi.apache.org/
tags: [apache]

operators:
- integration-name: Apache Kyuubi
python-modules:
- airflow.providers.apache.kyuubi.operators.kyuubi

hooks:
- integration-name: Apache Kyuubi
python-modules:
- airflow.providers.apache.kyuubi.hooks.kyuubi

connection-types:
- hook-class-name: airflow.providers.apache.kyuubi.hooks.kyuubi.KyuubiHook
connection-type: kyuubi
69 changes: 69 additions & 0 deletions providers/apache/kyuubi/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

[build-system]
requires = ["flit_core==3.12.0"]
build-backend = "flit_core.buildapi"

[project]
name = "apache-airflow-providers-apache-kyuubi"
version = "1.0.0"
description = "Provider package apache-airflow-providers-apache-kyuubi for Apache Airflow"
readme = "README.rst"
license = "Apache-2.0"
license-files = ['LICENSE', 'NOTICE']
authors = [
{name="Apache Software Foundation", email="dev@airflow.apache.org"},
]
maintainers = [
{name="Apache Software Foundation", email="dev@airflow.apache.org"},
]
keywords = [ "airflow-provider", "apache.kyuubi", "airflow", "integration" ]
classifiers = [
"Development Status :: 5 - Production/Stable",
"Environment :: Console",
"Environment :: Web Environment",
"Intended Audience :: Developers",
"Intended Audience :: System Administrators",
"Framework :: Apache Airflow",
"Framework :: Apache Airflow :: Provider",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Topic :: System :: Monitoring",
]
requires-python = ">=3.10"

dependencies = [
"apache-airflow>=2.8.0",
"apache-airflow-providers-apache-hive>=6.0.0",
]

[project.urls]
"Documentation" = "https://airflow.apache.org/docs/apache-airflow-providers-apache-kyuubi/1.0.0"
"Bug Tracker" = "https://github.com/apache/airflow/issues"
"Source Code" = "https://github.com/apache/airflow"
"Slack Chat" = "https://s.apache.org/airflow-slack"
"Mastodon" = "https://fosstodon.org/@airflow"
"YouTube" = "https://www.youtube.com/channel/UCSXwxpWZQ7XZ1WL3wqevChA/"

[project.entry-points."apache_airflow_provider"]
provider_info = "airflow.providers.apache.kyuubi.get_provider_info:get_provider_info"

[tool.flit.module]
name = "airflow.providers.apache.kyuubi"
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
Loading
Loading