Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat (format): Introduce buf #519

Merged
merged 6 commits into from
Jun 13, 2024
Merged

feat (format): Introduce buf #519

merged 6 commits into from
Jun 13, 2024

Conversation

SemyonSinchenko
Copy link
Member

Reason for this PR

Introduce using of buf-tool

What changes are included in this PR?

Actual changes are related to the following files:

  • all the files in format
  • buf.yaml
  • buf.gen.yaml

Are these changes tested?

.

Are there any user-facing changes?

.

SemyonSinchenko and others added 6 commits June 7, 2024 09:29
@SemyonSinchenko
feat(spark): Refactoring datasources (apache#514)
d288b7e 
### Reason for this PR
By moving datasources under `org.apache.spark.sql` we are able to access private Spark API. Last time when I was trying to fully migrate datasources to V2 it was a blocker. Detailed motivation is in apache#493 

### What changes are included in this PR?
Mostly refactoring.

### Are these changes tested?
Unit tests are passed

I manually checked the generated JARs:
![image](https://github.com/apache/incubator-graphar/assets/29755009/1b094516-88b1-490a-a2ea-8dcd092a3b1d)

### Are there any user-facing changes?
Mostly not because `GarDataSource` was left under the same package.


Close apache#493
@acezen
feat(dev): Add release and verify scripts (apache#507)
f4da959 
Reason for this PR
Add scripts for developer or release manager to easily release version or verify a version.

What changes are included in this PR?
Add release and verify scripts
related document is updated to website, see Update the release and verify document, and add development document incubator-graphar-website#18
Are these changes tested?
yes

Are there any user-facing changes?
no
---------

Signed-off-by: acezen <qiaozi.zwb@alibaba-inc.com>
@acezen
chore: Bump to version v0.12.0 (Round 1) (apache#517)
8b38f27 
Signed-off-by: acezen <qiaozi.zwb@alibaba-inc.com>
@acezen
chore: Add CHANGELOG.md (apache#513)
ff057cf 
Signed-off-by: acezen <qiaozi.zwb@alibaba-inc.com>
@SemyonSinchenko
Merge branch 'main' into format-definition-dev
cd8896e
@SemyonSinchenko
Introduce buf
2aa13c6 
- v2
- buf.gen
- buf

 On branch format-definition-dev
 Your branch is up to date with 'origin/format-definition-dev'.

 Changes to be committed:
	new file:   buf.gen.yaml
	new file:   buf.yaml
	modified:   format/adjacent_list.proto
	modified:   format/edge_info.proto
	modified:   format/graph_info.proto
	modified:   format/property_group.proto
	modified:   format/types.proto
	modified:   format/vertex_info.proto
@SemyonSinchenko
Copy link
Member Author

@acezen May you make a look please?

To generate files do the following:

  1. Install protoc if you don't have it: https://grpc.io/docs/protoc-installation/
  2. Install buf (>=1.32) if you don't have it: https://buf.build/docs/installation
  3. From the root of the project run buf generate

It creates:

  1. Java classes in maven-projects/info/src/main/java/org/apache/grarphar/info/proto
  2. Cpp classes in cpp/src/proto (I have no understanding where to place them, I'm not a cpp person and I need ur help here)
  3. Python files in pyspark/graphar_pyspark/proto

@SemyonSinchenko
Copy link
Member Author

Using of buf v2 is questionable because it was introduced only in the latest version. Anyway, I see no benefits of v1 because we will need to migrate sooner or later

acezen
acezen approved these changes Jun 13, 2024
View reviewed changes
Copy link
Contributor

@acezen acezen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@acezen
Copy link
Contributor

acezen commented Jun 13, 2024 •
edited
Loading

@acezen May you make a look please?

To generate files do the following:

  1. Install protoc if you don't have it: https://grpc.io/docs/protoc-installation/
  2. Install buf (>=1.32) if you don't have it: https://buf.build/docs/installation
  3. From the root of the project run buf generate

It creates:

  1. Java classes in maven-projects/info/src/main/java/org/apache/grarphar/info/proto
  2. Cpp classes in cpp/src/proto (I have no understanding where to place them, I'm not a cpp person and I need ur help here)
  3. Python files in pyspark/graphar_pyspark/proto

I think the cpp classes may puts to cpp/proto.

@acezen
Copy link
Contributor

acezen commented Jun 13, 2024

@acezen May you make a look please?
To generate files do the following:

  1. Install protoc if you don't have it: https://grpc.io/docs/protoc-installation/
  2. Install buf (>=1.32) if you don't have it: https://buf.build/docs/installation
  3. From the root of the project run buf generate

It creates:

  1. Java classes in maven-projects/info/src/main/java/org/apache/grarphar/info/proto
  2. Cpp classes in cpp/src/proto (I have no understanding where to place them, I'm not a cpp person and I need ur help here)
  3. Python files in pyspark/graphar_pyspark/proto

I think the cpp classes may puts to cpp/proto.

Hi, Sem, I think we can merge this change and I would update the cpp related configuratio.

@SemyonSinchenko SemyonSinchenko marked this pull request as ready for review June 13, 2024 10:02
@SemyonSinchenko SemyonSinchenko merged commit 22aa41f into apache:format-definition-dev Jun 13, 2024
acezen added a commit that referenced this pull request Jun 13, 2024
@SemyonSinchenko @acezen
feat (format): Introduce buf (#519)
2509d40 
* feat(spark): Refactoring datasources (#514)

### Reason for this PR
By moving datasources under `org.apache.spark.sql` we are able to access private Spark API. Last time when I was trying to fully migrate datasources to V2 it was a blocker. Detailed motivation is in #493 

### What changes are included in this PR?
Mostly refactoring.

### Are these changes tested?
Unit tests are passed

I manually checked the generated JARs:
![image](https://github.com/apache/incubator-graphar/assets/29755009/1b094516-88b1-490a-a2ea-8dcd092a3b1d)

### Are there any user-facing changes?
Mostly not because `GarDataSource` was left under the same package.


Close #493

* feat(dev): Add release and verify scripts (#507)

Reason for this PR
Add scripts for developer or release manager to easily release version or verify a version.

What changes are included in this PR?
Add release and verify scripts
related document is updated to website, see Update the release and verify document, and add development document incubator-graphar-website#18
Are these changes tested?
yes

Are there any user-facing changes?
no
---------

Signed-off-by: acezen <qiaozi.zwb@alibaba-inc.com>

* chore: Bump to version v0.12.0 (Round 1) (#517)


Signed-off-by: acezen <qiaozi.zwb@alibaba-inc.com>

* chore: Add CHANGELOG.md (#513)


Signed-off-by: acezen <qiaozi.zwb@alibaba-inc.com>

* Introduce buf

- v2
- buf.gen
- buf

 On branch format-definition-dev
 Your branch is up to date with 'origin/format-definition-dev'.

 Changes to be committed:
	new file:   buf.gen.yaml
	new file:   buf.yaml
	modified:   format/adjacent_list.proto
	modified:   format/edge_info.proto
	modified:   format/graph_info.proto
	modified:   format/property_group.proto
	modified:   format/types.proto
	modified:   format/vertex_info.proto

---------

Signed-off-by: acezen <qiaozi.zwb@alibaba-inc.com>
Co-authored-by: Weibin Zeng <qiaozi.zwb@alibaba-inc.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@acezen acezen acezen approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@SemyonSinchenko @acezen