Skip to content

Commit

Permalink
chore: replace references to Lyft -> Amundsen (amundsen-io#174)
Browse files Browse the repository at this point in the history
Signed-off-by: Tao Feng <fengtao04@gmail.com>
  • Loading branch information
Dorian Johnson authored Aug 19, 2020
1 parent 48b4c71 commit 369685c
Show file tree
Hide file tree
Showing 8 changed files with 32 additions and 32 deletions.
2 changes: 1 addition & 1 deletion CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
This project is governed by [Lyft's code of conduct](https://github.com/lyft/code-of-conduct).
This project is governed by [Amundsen's code of conduct](https://github.com/amundsen-io/amundsen/blob/master/CODE_OF_CONDUCT.md).
All contributors and participants agree to abide by its terms.
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,13 @@

Amundsen Metadata service serves Restful API and is responsible for providing and also updating metadata, such as table & column description, and tags. Metadata service can use Neo4j or Apache Atlas as a persistent layer.

For information about Amundsen and our other services, visit the [main repository](https://github.com/lyft/amundsen#amundsen) `README.md`. Please also see our instructions for a [quick start](https://github.com/lyft/amundsen/blob/master/docs/installation.md#bootstrap-a-default-version-of-amundsen-using-docker) setup of Amundsen with dummy data, and an [overview of the architecture](https://github.com/lyft/amundsen/blob/master/docs/architecture.md#architecture).
For information about Amundsen and our other services, visit the [main repository](https://github.com/amundsen-io/amundsen#amundsen) `README.md`. Please also see our instructions for a [quick start](https://github.com/amundsen-io/amundsen/blob/master/docs/installation.md#bootstrap-a-default-version-of-amundsen-using-docker) setup of Amundsen with dummy data, and an [overview of the architecture](https://github.com/amundsen-io/amundsen/blob/master/docs/architecture.md#architecture).

## Requirements
- Python >= 3.7

## Doc
- https://lyft.github.io/amundsen/
- https://www.amundsen.io/amundsen/

## Instructions to start the Metadata service from distribution
```bash
Expand All @@ -30,7 +30,7 @@ $ curl -v http://localhost:5002/healthcheck

## Instructions to start the Metadata service from the source
```bash
$ git clone https://github.com/lyft/amundsenmetadatalibrary.git
$ git clone https://github.com/amundsen-io/amundsenmetadatalibrary.git
$ cd amundsenmetadatalibrary
$ python3 -m venv venv
$ source venv/bin/activate
Expand Down Expand Up @@ -65,8 +65,8 @@ $ gunicorn metadata_service.metadata_wsgi
Here is [documentation](https://docs.gunicorn.org/en/latest/run.html "documentation") of gunicorn configuration.

### Configuration outside local environment
By default, Metadata service uses [LocalConfig](https://github.com/lyft/amundsenmetadatalibrary/blob/master/metadata_service/config.py "LocalConfig") that looks for Neo4j running in localhost.
In order to use different end point, you need to create [Config](https://github.com/lyft/amundsenmetadatalibrary/blob/master/metadata_service/config.py "Config") suitable for your use case. Once config class has been created, it can be referenced by [environment variable](https://github.com/lyft/amundsenmetadatalibrary/blob/master/metadata_service/metadata_wsgi.py "environment variable"): `METADATA_SVC_CONFIG_MODULE_CLASS`
By default, Metadata service uses [LocalConfig](https://github.com/amundsen-io/amundsenmetadatalibrary/blob/master/metadata_service/config.py "LocalConfig") that looks for Neo4j running in localhost.
In order to use different end point, you need to create [Config](https://github.com/amundsen-io/amundsenmetadatalibrary/blob/master/metadata_service/config.py "Config") suitable for your use case. Once config class has been created, it can be referenced by [environment variable](https://github.com/amundsen-io/amundsenmetadatalibrary/blob/master/metadata_service/metadata_wsgi.py "environment variable"): `METADATA_SVC_CONFIG_MODULE_CLASS`

For example, in order to have different config for production, you can inherit Config class, create Production config and passing production config class into environment variable. Let's say class name is ProdConfig and it's in metadata_service.config module. then you can set as below:

Expand All @@ -77,7 +77,7 @@ This way Metadata service will use production config in production environment.
# Apache Atlas
Amundsen Metadata service can use Apache Atlas as a backend. Some of the benefits of using Apache Atlas instead of Neo4j is that Apache Atlas offers plugins to several services (e.g. Apache Hive, Apache Spark) that allow for push based updates. It also allows to set policies on what metadata is accesible and editable by means of Apache Ranger.

If you would like to use Apache Atlas as a backend for Metadata service you will need to create a [Config](https://github.com/lyft/amundsenmetadatalibrary/blob/master/metadata_service/config.py "Config") as mentioned above. Make sure to include the following:
If you would like to use Apache Atlas as a backend for Metadata service you will need to create a [Config](https://github.com/amundsen-io/amundsenmetadatalibrary/blob/master/metadata_service/config.py "Config") as mentioned above. Make sure to include the following:

```python
PROXY_CLIENT = PROXY_CLIENTS['ATLAS'] # or env PROXY_CLIENT='ATLAS'
Expand Down
4 changes: 2 additions & 2 deletions docs/configurations.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Most of the configurations are set through Flask [Config Class](https://github.com/lyft/amundsenmetadatalibrary/blob/master/metadata_service/config.py).
Most of the configurations are set through Flask [Config Class](https://github.com/amundsen-io/amundsenmetadatalibrary/blob/master/metadata_service/config.py).

#### USER_DETAIL_METHOD `OPTIONAL`
This is a method that can be used to get the user details from any third-party or custom system.
This custom function takes user_id as a parameter, and returns a dictionary consisting user details' fields defined in [UserSchema](https://github.com/lyft/amundsencommon/blob/master/amundsen_common/models/user.py).
This custom function takes user_id as a parameter, and returns a dictionary consisting user details' fields defined in [UserSchema](https://github.com/amundsen-io/amundsencommon/blob/master/amundsen_common/models/user.py).

Example:
```python
Expand Down
26 changes: 13 additions & 13 deletions docs/structure.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,33 @@
Amundsen metadata service consists of three packages, API, Entity, and Proxy.

### [API package](https://github.com/lyft/amundsenmetadatalibrary/tree/master/metadata_service/api "API package")
### [API package](https://github.com/amundsen-io/amundsenmetadatalibrary/tree/master/metadata_service/api "API package")
A package that contains [Flask Restful resources](https://flask-restful.readthedocs.io/en/latest/api.html#flask_restful.Resource "Flask Restful resources") that serves Restful API request.
The [routing of API](https://flask-restful.readthedocs.io/en/latest/quickstart.html#resourceful-routing "routing of API") is being registered [here](https://github.com/lyft/amundsenmetadatalibrary/blob/master/metadata_service/__init__.py#L67 "here").
The [routing of API](https://flask-restful.readthedocs.io/en/latest/quickstart.html#resourceful-routing "routing of API") is being registered [here](https://github.com/amundsen-io/amundsenmetadatalibrary/blob/master/metadata_service/__init__.py#L67 "here").

### [Proxy package](https://github.com/lyft/amundsenmetadatalibrary/tree/master/metadata_service/proxy "Proxy package")
### [Proxy package](https://github.com/amundsen-io/amundsenmetadatalibrary/tree/master/metadata_service/proxy "Proxy package")
Proxy package contains proxy modules that talks dependencies of Metadata service. There are currently three modules in Proxy package,
[Neo4j](https://github.com/lyft/amundsenmetadatalibrary/blob/master/metadata_service/proxy/neo4j_proxy.py "Neo4j"),
[Statsd](https://github.com/lyft/amundsenmetadatalibrary/blob/master/metadata_service/proxy/statsd_utilities.py "Statsd")
and [Atlas](https://github.com/lyft/amundsenmetadatalibrary/blob/master/metadata_service/proxy/atlas_proxy.py "Atlas")
[Neo4j](https://github.com/amundsen-io/amundsenmetadatalibrary/blob/master/metadata_service/proxy/neo4j_proxy.py "Neo4j"),
[Statsd](https://github.com/amundsen-io/amundsenmetadatalibrary/blob/master/metadata_service/proxy/statsd_utilities.py "Statsd")
and [Atlas](https://github.com/amundsen-io/amundsenmetadatalibrary/blob/master/metadata_service/proxy/atlas_proxy.py "Atlas")

Selecting the appropriate proxy (Neo4j or Atlas) is configurable using a config variable `PROXY_CLIENT`,
which takes the path to class name of proxy module available [here](https://github.com/lyft/amundsenmetadatalibrary/blob/master/metadata_service/config.py#L11).
which takes the path to class name of proxy module available [here](https://github.com/amundsen-io/amundsenmetadatalibrary/blob/master/metadata_service/config.py#L11).

_Note: Proxy's host and port are configured using config variables `PROXY_HOST` and `PROXY_PORT` respectively.
Both of these variables can be set using environment variables._

##### [Neo4j proxy module](https://github.com/lyft/amundsenmetadatalibrary/blob/master/metadata_service/proxy/neo4j_proxy.py "Neo4j proxy module")
[Neo4j](https://neo4j.com/docs/ "Neo4j") proxy module serves various use case of getting metadata or updating metadata from or into Neo4j. Most of the methods have [Cypher query](https://neo4j.com/developer/cypher/ "Cypher query") for the use case, execute the query and transform into [entity](https://github.com/lyft/amundsenmetadatalibrary/tree/master/metadata_service/entity "entity").
##### [Neo4j proxy module](https://github.com/amundsen-io/amundsenmetadatalibrary/blob/master/metadata_service/proxy/neo4j_proxy.py "Neo4j proxy module")
[Neo4j](https://neo4j.com/docs/ "Neo4j") proxy module serves various use case of getting metadata or updating metadata from or into Neo4j. Most of the methods have [Cypher query](https://neo4j.com/developer/cypher/ "Cypher query") for the use case, execute the query and transform into [entity](https://github.com/amundsen-io/amundsenmetadatalibrary/tree/master/metadata_service/entity "entity").

##### [Apache Atlas proxy module](https://github.com/lyft/amundsenmetadatalibrary/blob/master/metadata_service/proxy/atlas_proxy.py "Apache Atlas proxy module")
##### [Apache Atlas proxy module](https://github.com/amundsen-io/amundsenmetadatalibrary/blob/master/metadata_service/proxy/atlas_proxy.py "Apache Atlas proxy module")
[Apache Atlas](https://atlas.apache.org/ "Apache Atlas") proxy module serves all of the metadata from Apache Atlas, using [pyatlasclient](https://pyatlasclient.readthedocs.io/en/latest/index.html).
More information on how to setup Apache Atlas to make it compatible with Amundsen can be found [here](proxy/atlas_proxy.md)

##### [Statsd utilities module](https://github.com/lyft/amundsenmetadatalibrary/blob/master/metadata_service/proxy/statsd_utilities.py "Statsd utilities module")
[Statsd](https://github.com/etsy/statsd/wiki "Statsd") utilities module has methods / functions to support statsd to publish metrics. By default, statsd integration is disabled and you can turn in on from [Metadata service configuration](https://github.com/lyft/amundsenmetadatalibrary/blob/master/metadata_service/config.py "Metadata service configuration").
##### [Statsd utilities module](https://github.com/amundsen-io/amundsenmetadatalibrary/blob/master/metadata_service/proxy/statsd_utilities.py "Statsd utilities module")
[Statsd](https://github.com/etsy/statsd/wiki "Statsd") utilities module has methods / functions to support statsd to publish metrics. By default, statsd integration is disabled and you can turn in on from [Metadata service configuration](https://github.com/amundsen-io/amundsenmetadatalibrary/blob/master/metadata_service/config.py "Metadata service configuration").
For specific configuration related to statsd, you can configure it through [environment variable.](https://statsd.readthedocs.io/en/latest/configure.html#from-the-environment "environment variable.")

### [Entity package](https://github.com/lyft/amundsenmetadatalibrary/tree/master/metadata_service/entity "Entity package")
### [Entity package](https://github.com/amundsen-io/amundsenmetadatalibrary/tree/master/metadata_service/entity "Entity package")
Entity package contains many modules where each module has many Python classes in it. These Python classes are being used as a schema and a data holder. All data exchange within Amundsen Metadata service use classes in Entity to ensure validity of itself and improve readability and mainatability.


Expand Down
2 changes: 1 addition & 1 deletion metadata_service/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ class Config:
IS_STATSD_ON = False

# Used to differentiate tables with other entities in Atlas. For more details:
# https://github.com/lyft/amundsenmetadatalibrary/blob/master/docs/proxy/atlas_proxy.md
# https://github.com/amundsen-io/amundsenmetadatalibrary/blob/master/docs/proxy/atlas_proxy.md
ATLAS_TABLE_ENTITY = 'Table'

# The relationalAttribute name of Atlas Entity that identifies the database entity.
Expand Down
4 changes: 2 additions & 2 deletions metadata_service/proxy/neo4j_proxy.py
Original file line number Diff line number Diff line change
Expand Up @@ -887,8 +887,8 @@ def get_dashboard_by_user_relation(self, *, user_email: str, relation_type: User

# FYI, to extract last_successful_execution, it searches for its execution ID which is always
# _last_successful_execution
# https://github.com/lyft/amundsendatabuilder/blob/master/databuilder/models/dashboard/dashboard_execution.py#L18
# https://github.com/lyft/amundsendatabuilder/blob/master/databuilder/models/dashboard/dashboard_execution.py#L24
# https://github.com/amundsen-io/amundsendatabuilder/blob/master/databuilder/models/dashboard/dashboard_execution.py#L18
# https://github.com/amundsen-io/amundsendatabuilder/blob/master/databuilder/models/dashboard/dashboard_execution.py#L24

query = textwrap.dedent(f"""
MATCH {rel_clause}<-[:DASHBOARD]-(dg:Dashboardgroup)<-[:DASHBOARD_GROUP]-(clstr:Cluster)
Expand Down
6 changes: 3 additions & 3 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@
name='amundsen-metadata',
version=__version__,
description='Metadata service for Amundsen',
url='https://www.github.com/lyft/amundsenmetadatalibrary',
maintainer='Lyft',
maintainer_email='amundsen-dev@lyft.com',
url='https://www.github.com/amundsen-io/amundsenmetadatalibrary',
maintainer='Amundsen TSC',
maintainer_email='amundsen-tsc@lists.lfai.foundation',
packages=find_packages(exclude=['tests*']),
include_package_data=True,
zip_safe=False,
Expand Down
8 changes: 4 additions & 4 deletions tests/unit/proxy/test_neo4j_proxy.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,8 +82,8 @@ def setUp(self) -> None:
'last_updated_timestamp': 1,
'owner_records': [
{
'key': 'tester@lyft.com',
'email': 'tester@lyft.com'
'key': 'tester@example.com',
'email': 'tester@example.com'
}
],
'tag_records': [
Expand Down Expand Up @@ -166,7 +166,7 @@ def test_get_table(self) -> None:
end_epoch=2,
stat_type='avg',
stat_val='2')])],
owners=[User(email='tester@lyft.com')],
owners=[User(email='tester@example.com')],
table_writer=Application(application_url=self.table_writer['application_url'],
description=self.table_writer['description'],
name=self.table_writer['name'],
Expand Down Expand Up @@ -217,7 +217,7 @@ def test_get_table_view_only(self) -> None:
end_epoch=2,
stat_type='avg',
stat_val='2')])],
owners=[User(email='tester@lyft.com')],
owners=[User(email='tester@example.com')],
table_writer=Application(application_url=self.table_writer['application_url'],
description=self.table_writer['description'],
name=self.table_writer['name'],
Expand Down

0 comments on commit 369685c

Please sign in to comment.