Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Code format with the right subfolders #198

Merged
merged 62 commits into from
Jun 7, 2022
Merged

Conversation

xiaoyongzhu
Copy link
Member

@xiaoyongzhu xiaoyongzhu commented Apr 30, 2022

Goal of this PR is to add the right folder hierarchy for Feathr code (so that it can be better organized).

Other small improvements include:

  • Updated the pydoc structure to make sure all the pydocs are in the right module name.
  • Renaming frameconfig to feathrconfig and put it into the right folder structure
  • Docs on how to expose the right module in pydocs

pydocs in this PR: https://feathr-xiaoyzhu.readthedocs.io/en/latest/index.html
pydocs in main branch: https://feathr.readthedocs.io/

We should also move the api folder out from the python client. Waiting for @blrchen and @YihuiGuo to fix those as there seems to be some dependencies. It's described at #287.

This should have no end user impact as end users are using this:
from feathr import X

The new file structure is like this:

image

(base) xiazhu@xiazhu-mn1 feathr % tree feathr_project/feathr -I __pycache__
feathr_project/feathr
├── __init__.py
├── api
│   ├── app
│   │   ├── __init__.py
│   │   ├── backends
│   │   │   ├── backend_base.py
│   │   │   └── purview_backend
│   │   │       ├── __init__.py
│   │   │       ├── purview_backend.py
│   │   │       └── purview_backend_config.py
│   │   ├── core
│   │   │   ├── __init__.py
│   │   │   ├── configs.py
│   │   │   ├── error_code.py
│   │   │   ├── error_handling.py
│   │   │   ├── feathr_api_exception.py
│   │   │   └── registry.py
│   │   ├── main.py
│   │   ├── models
│   │   │   └── features.py
│   │   └── routes
│   │       ├── feature.py
│   │       ├── job.py
│   │       ├── offline_feature.py
│   │       ├── online_feature.py
│   │       ├── project.py
│   │       └── transformation.py
│   └── requirements.txt
├── client.py
├── constants.py
├── definition
│   ├── _materialization_utils.py
│   ├── aggregation.py
│   ├── anchor.py
│   ├── dtype.py
│   ├── feathrconfig.py
│   ├── feature.py
│   ├── feature_derivations.py
│   ├── lookup_feature.py
│   ├── materialization_settings.py
│   ├── query_feature_list.py
│   ├── repo_definitions.py
│   ├── settings.py
│   ├── sink.py
│   ├── source.py
│   ├── transformation.py
│   └── typed_key.py
├── exception
├── protobuf
│   ├── __init__.py
│   └── featureValue_pb2.py
├── registry
│   ├── _feature_registry_purview.py
│   └── feature_registry.py
├── secrets
│   └── akv_client.py
├── spark_provider
│   ├── _abc.py
│   ├── _databricks_submission.py
│   ├── _synapse_submission.py
│   └── feathr_configurations.py
├── udf
│   ├── _preprocessing_pyudf_manager.py
│   └── feathr_pyspark_driver_template.py
└── utils
    ├── _envvariableutil.py
    ├── _file_utils.py
    ├── feature_printer.py
    └── job_utils.py

@xiaoyongzhu xiaoyongzhu added the safe to test Tag to execute build pipeline for a PR from forked repo label Apr 30, 2022
@hangfei
Copy link
Collaborator

hangfei commented Apr 30, 2022

Add a screenshot for the new structure?

@hangfei
Copy link
Collaborator

hangfei commented May 3, 2022

Could you provide a screenshot of the new py docs?

@hangfei
Copy link
Collaborator

hangfei commented Jun 7, 2022

Please test notebook as well since this changes a lot of structural stuff.

@xiaoyongzhu
Copy link
Member Author

Please test notebook as well since this changes a lot of structural stuff.

Good catch. I've verified this branch with the latest notebook and all those worked.

@xiaoyongzhu xiaoyongzhu merged commit 6aa3459 into main Jun 7, 2022
@xiaoyongzhu xiaoyongzhu deleted the xiaoyzhu/local_registry branch June 7, 2022 05:50
bozhonghu pushed a commit that referenced this pull request Jun 8, 2022
* main:
  Code format with the right subfolders (#198)
  Added scala dev guide (#306)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants