Bring apis-client-generator in as subtree rather than submodule #713
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- new call_template syntax - noeol/noblank tags - more tests - some type system cleanups - added a sample language generator - several fixes w.r.t. class paths in java - reorganize code tree extensively - better handling of modules w.r.t LanguageModel - snapshot of newest Dart & PHP support
R=smulloni DELTA=15 (10 added, 0 deleted, 5 changed) Revision created by MOE tool push_codebase. MOE_MIGRATION=6529
Extend language model to have a parameterized name transform method.
Use that transform for constant names.
Show it in sample and csharp as a proof of concept.
I think this provides the right help for people doing language models
becuase it gives us a chance to speak about all the transforms in a
uniform way. We can build in the common useful transforms and let
the language author select. Ideally, ToClassName and ToName will
not ever be implemented by a language author.
With the C# change, we get this output.
public class BloggerService : Google.Apis.Services.BaseClientService
{
public const string Version = "v3";
// OAuth2 scopes which may be required for some methods in this API
/** OAuth2 scope: Manage your Blogger account */
public const string BloggerScope = "https://www.googleapis.com/auth/blogger";
/** OAuth2 scope: View your Blogger account */
public const string BloggerReadonlyScope =
"https://www.googleapis.com/auth/blogger.readonly";
R=smulloni
DELTA=143 (134 added, 0 deleted, 9 changed)
Change on 2013-06-18 07:59:25-07:00 by aiuto
Make api_test a little easier to read by cleaning up the use of
Api.MethodByName. It was bothering me while adding tests in
another CL
R=smulloni
DELTA=18 (0 added, 6 deleted, 12 changed)
Change on 2013-06-18 10:33:10-07:00 by aiuto
Fix construction of enums so they have the right data types when we have
a parameter which is a repeated enum. The old construction made parameters
with the structure
p.data_type = ArrayOfDataType(string)
p.enumType = Enum(...)
This requires enumType to be specially handled in several generators, because
the data_type of the element is not really the enum. The actual structure
should be
p.data_type = ArrayOfDataType(Enum(...))
This allows us to write p.data_date.code_type in most generators and have
the right behavior if the pamamter is repeated or not.
Fix gwt templates to use the param.data_type.code_type rather than
parameter.codeType so the enum fix can work.
Remove unneeded codeType override on Enum becuase no code path actually
uses it.
Removed some no-op AnnotateXXX methods in GmsGenerator because they are
unneeded and clutter our attempt to find and fix the hacks in AnnotateXXX
This change has been byte-for-byte tested against a language API matrix
and requires have a global presubmit run before submit.
R=smulloni
DELTA=123 (37 added, 61 deleted, 25 changed)
Change on 2013-06-19 09:06:01-07:00 by aiuto
Next round of LanguageModel features.
- pass the underlying template variable to the ToXxxName methods. This
allows us to extract the api and module from it.
- Add format and show use in memberName on sample class
- add ToClassName, but do not use it yet. That will happn in a revamp
of http://cr/47382629
- Add array_of options, with a fallback to the older CodeTypeForArrayOr
class. sample generator changed to use it, along with module. E.g.
array_of_format='%m::Array<%s>', gets us
< Array<Comment> get_items() {
> com::google::blogger::Array<Comment> get_items() {
- do the same for MapOf (TBD after first round review)
R=smulloni
DELTA=271 (191 added, 39 deleted, 41 changed)
Change on 2013-06-20 10:38:10-07:00 by aiuto
Allow Unicode in Discovery objects.
1. Fix the name_validator to not care about non-ASCII. This is a harmless
change at this time because the non-ASCII characters are still stripped
in template_objects when we initially process descriptions.
1a. Move name_validator to utilities as part of a general cleanup.
2. Add a test case for unicode in descriptions
3. Turn off the unicode stipping on template_objects.
R=smulloni
DELTA=947 (672 added, 272 deleted, 3 changed)
Change on 2013-07-02 08:10:27-07:00 by aiuto
Add the concept of a Constant to the code generator and have the possible
values of an Enum be Constants.
Show the new use in the sample generator, and add a golden test for the sample.
R=peleyal
DELTA=6491 (6432 added, 30 deleted, 29 changed)
Revision created by MOE tool push_codebase.
MOE_MIGRATION=6572
Revision created by MOE tool push_codebase. MOE_MIGRATION=6573
Create 1.16 branch for Java codegen
R=yanivi,aiuto
Change on 2013-07-10 18:37:51-07:00 by aiuto
Add getter/setter policies to language model.
Use them in the sample generator (and add a setter). This results in
changes like:
- object get_address() {
+ object getAddress() {
return _address;
}
+ void setAddress(object v) {
+ _address = v;
+ }
- object get_annotation() {
+ object getAnnotation() {
return _annotation;
}
+ void setAnnotation(object v) {
+ _annotation = v;
+ }
- com::google::myservice::Array<ActivityCategories> get_categories() {
+ com::google::myservice::Array<ActivityCategories> getCategories() {
return _categories;
}
+ void setCategories(com::google::myservice::Array<ActivityCategories> v) {
+ _categories = v;
+ }
R=ewiseblatt
DELTA=2250 (1701 added, 7 deleted, 542 changed)
Change on 2013-07-18 03:01:43-07:00 by ianbarber
Adding an extra language script variable to use the appropriate class naming format for the older templates. Update templates to use these.
R=silvano
DELTA=8 (4 added, 0 deleted, 4 changed)
Change on 2013-07-22 13:45:53-07:00 by smulloni
Fix for csharp nested class names, for the case of classes declared implicitly inside properties. R=peleyal,jcgregorio
DELTA=181 (173 added, 3 deleted, 5 changed)
Change on 2013-07-29 10:51:41-07:00 by smulloni
Added a test verifying that call-template can escape the template directory.
R=aiuto
DELTA=9 (9 added, 0 deleted, 0 changed)
Change on 2013-07-30 05:27:26-07:00 by peleyal
C# templates for code + project
R=smulloni
DELTA=484 (272 added, 165 deleted, 47 changed)
Change on 2013-07-30 13:49:04-07:00 by smulloni
Fix for C# property names that conflict with the name of the enclosing class.
R=aiuto,peleyal
DELTA=32 (32 added, 0 deleted, 0 changed)
Change on 2013-07-31 09:00:25-07:00 by smulloni
Add template variables needed to output pom.xml.
R=aiuto
DELTA=197 (178 added, 4 deleted, 15 changed)
Change on 2013-07-31 15:18:53-07:00 by ngmiceli
Update Java codegen 1.16 "branch" for next development version
R=smulloni
Change on 2013-08-01 12:52:28-07:00 by smulloni
Added a thread-local template context, and adapted one filter (block-comment) to use it to become aware of the current language. This should make it possible to eliminate many uses of the {% language %} tag, among other things.
R=aiuto
DELTA=141 (123 added, 1 deleted, 17 changed)
Change on 2013-08-02 07:19:08-07:00 by smulloni
Upgrade django to 1.4, which will give us {% elif %} and a few other template enhancements. We have to do it someday.
I've gone over the 1.4 release notes (https://docs.djangoproject.com/en/1.4/releases/1.4/) and don't see anything terrifying. But we should still test this rather carefully.
R=aiuto
DELTA=20 (0 added, 0 deleted, 20 changed)
Change on 2013-08-02 07:41:22-07:00 by smulloni
Adds a {% bool x %} tag that coerces the value of the variable x to a Python bool, and then outputs the right boolean literal for the current language.
R=aiuto
DELTA=38 (38 added, 0 deleted, 0 changed)
Change on 2013-08-05 12:14:17-07:00 by smulloni
Fix nested class name in c# that conflicts with property by adding a Data suffix.
For drive/v2 api, the difference this CL makes can be seen here:
/home/smulloni/csharp-nested-class-name-conflict.diff
R=peleyal
DELTA=11 (8 added, 0 deleted, 3 changed)
Change on 2013-08-06 06:56:38-07:00 by peleyal
1. Fix errors in the C# templates
R=smulloni
DELTA=562 (394 added, 113 deleted, 55 changed)
Change on 2013-08-08 06:54:41-07:00 by ianbarber
Support base level methods, missing code names, and odd properties with slashes in
R=silvano,smulloni
DELTA=71 (48 added, 6 deleted, 17 changed)
Change on 2013-08-12 10:28:58-07:00 by smulloni
Avoid name conflicts for deeply nested schemas.
R=peleyal
DELTA=81 (79 added, 1 deleted, 1 changed)
Change on 2013-08-12 12:56:45-07:00 by smulloni
Fix for safeClassName of map types for csharp.
R=peleyal
DELTA=52 (52 added, 0 deleted, 0 changed)
Change on 2013-08-12 13:48:59-07:00 by yanivi
Fix minor javadoc repeated "the the".
R=smulloni,ngmiceli
DELTA=1 (0 added, 0 deleted, 1 changed)
Change on 2013-08-13 09:22:06-07:00 by peleyal
Improve the .NET templates and create a csharp generator
To test - you can run blaze
google3$ build apiserving/clients/csharp:generate_csharp_library && blaze-bin/apiserving/clients/csharp/generate_csharp_library --all_apis --output_dir ~/tmp/apis/default
R=smulloni
DELTA=340 (288 added, 24 deleted, 28 changed)
Change on 2013-08-14 07:16:32-07:00 by smulloni
Make django version consistently 1.4
R=aiuto
DELTA=2 (0 added, 0 deleted, 2 changed)
Change on 2013-08-14 08:27:24-07:00 by smulloni
Customize package path for c#.
Package path is used to setup modules before the framework gives us a good customization hook -- except that we do have an ostensibly deprecated package path override which happens to work. Probably we should generalize this rather than remove it.
R=aiuto,peleyal
DELTA=35 (26 added, 9 deleted, 0 changed)
Change on 2013-08-14 14:08:44-07:00 by yanivi
Use externalTypeName for method request & response schemas, and add proto_lib parameter to JavaApiaryLibrary rule to specify proto_library of model package.
Matches the template logic from JSON java generator except uses "method.requestType.externalTypeName" instead of "method.requestType.fullClassName" and similarly for method.responseType.
I'm not confident about the logic for computing protoFullClassName. The only guarantee I can provide is that it works in one case I tried.
Also convert call-template tags to new syntax as was done in CL 47719178.
Reviewers:
ngmiceli: functionality of proto generator
smulloni: codegen framework
R=smulloni,ngmiceli
DELTA=100 (34 added, 18 deleted, 48 changed)
Change on 2013-08-15 06:55:04-07:00 by peleyal
Small fixes in the C# temapltes (getters to resource inside a resource)
R=smulloni
DELTA=64 (16 added, 16 deleted, 32 changed)
Change on 2013-08-15 11:20:36-07:00 by peleyal
Create 1.5.0-beta and delete 1.1.0-beta and 1.2.0-beta
R=smulloni
Change on 2013-08-16 01:15:07-07:00 by ulfjack
Avoid writing a timestamp to the output zip file to avoid making it non-deterministic.
R=smulloni
DELTA=7 (4 added, 1 deleted, 2 changed)
Change on 2013-08-16 07:42:11-07:00 by aiuto
Add a {% write variable_holding_file_path %} template tag to write a file. The content through {% endwrite %} is rendered and then written to the path contained in the variable. The path is relative to the template source file.
R=lalex,smulloni
DELTA=122 (119 added, 0 deleted, 3 changed)
Change on 2013-08-16 08:18:07-07:00 by smulloni
Change on 2013-08-16 13:08:20-07:00 by smulloni
Move refactored computation of owner to generator itself, and add workaround for discovery bug to generate handler.
R=aiuto
DELTA=195 (78 added, 106 deleted, 11 changed)
Change on 2013-08-19 07:08:43-07:00 by aiuto
Make C# part of open source release.
R=peleyal
DELTA=7 (6 added, 1 deleted, 0 changed)
Change on 2013-08-19 07:25:21-07:00 by peleyal
change .NET 1.5.0-beta version to external
R=smulloni
DELTA=2 (0 added, 0 deleted, 2 changed)
Change on 2013-08-19 08:15:09-07:00 by smulloni
Version bump for codegen open source release.
R=aiuto
DELTA=17 (15 added, 0 deleted, 2 changed)
Change on 2013-08-19 08:28:17-07:00 by smulloni
Un-moe-strip csharp generator in lookup
R=aiuto
DELTA=4 (2 added, 2 deleted, 0 changed)
Revision created by MOE tool push_codebase.
MOE_MIGRATION=6637
Fix maven group id for google apis.
R=yanivi
DELTA=2 (0 added, 0 deleted, 2 changed)
Change on 2013-08-19 10:38:10-07:00 by smulloni
Remove excessively spammy tpvz.
Our tpvz is so slow that we often 500 trying to produce it, and even when we serve it, it is often too slow for our monitoring to fetch it. This removes some cruft that is slowing us down: especially, records for 404 requests for mostly nonexistent apis.
R=aiuto
DELTA=13 (5 added, 8 deleted, 0 changed)
Change on 2013-08-19 11:50:40-07:00 by lalex
Add a property methods_dict to Resource objects that returns a {wireName: method} dict.
R=smulloni,aiuto
DELTA=4 (4 added, 0 deleted, 0 changed)
Change on 2013-08-20 06:31:35-07:00 by jcgregorio
Add METADATA files for Quickstart application and googledata directories
that adds apiary-quickstart to all CLs.
Also focus the list of tests that need to be run for Quickstart.
R=smulloni
DELTA=38 (38 added, 0 deleted, 0 changed)
Change on 2013-08-20 10:28:49-07:00 by lalex
Work on codegen HTML templates so they output documentation in a resource hierarchy.
R=aiuto,smulloni
DELTA=275 (165 added, 107 deleted, 3 changed)
Change on 2013-08-20 12:05:26-07:00 by smulloni
Make our opensource release nicer by adding standard Python packaging.
The old installation/usage experience:
1. Download or check out the generator source.
2. Install dependencies with a series of at least 3 commands.
3. Run commands like:
export PYTHONPATH=$(/bin/pwd)/src:$PYTHONPATH
python $(/bin/pwd)/src/googleapis/codegen/generate_library.py \
--input=src/googleapis/codegen/testdata/moderator.v1.json \
--language=java \
--output_dir=/tmp/generated
The new experience:
1. pip install google-apis-client-generator (from pypi once pushed, or
"pip install google-apis-client-generator-1.4.2.tar.gz" from the
downloaded tarball). Dependencies are installed automatically.
2. Directly run: generate_library --input=... [etc]
Of course, in either case one would be wise to first "mkvirtualenv myenv".
I'm not pinning the dependencies to specific versions; I'd rather find out that
there was a problem with, say, Django 1.5, and fix it. (It seems to work.)
R=aiuto
DELTA=115 (86 added, 16 deleted, 13 changed)
Revision created by MOE tool push_codebase.
MOE_MIGRATION=6639
Fix maven group id for google apis.
R=yanivi
DELTA=2 (0 added, 0 deleted, 2 changed)
Change on 2013-08-19 11:50:40-07:00 by lalex
Add a property methods_dict to Resource objects that returns a {wireName: method} dict.
R=smulloni,aiuto
DELTA=4 (4 added, 0 deleted, 0 changed)
Change on 2013-08-20 10:28:49-07:00 by lalex
Work on codegen HTML templates so they output documentation in a resource hierarchy.
R=aiuto,smulloni
DELTA=275 (165 added, 107 deleted, 3 changed)
Change on 2013-08-20 12:05:26-07:00 by smulloni
Make our opensource release nicer by adding standard Python packaging.
The old installation/usage experience:
1. Download or check out the generator source.
2. Install dependencies with a series of at least 3 commands.
3. Run commands like:
export PYTHONPATH=$(/bin/pwd)/src:$PYTHONPATH
python $(/bin/pwd)/src/googleapis/codegen/generate_library.py \
--input=src/googleapis/codegen/testdata/moderator.v1.json \
--language=java \
--output_dir=/tmp/generated
The new experience:
1. pip install google-apis-client-generator (from pypi once pushed, or
"pip install google-apis-client-generator-1.4.2.tar.gz" from the
downloaded tarball). Dependencies are installed automatically.
2. Directly run: generate_library --input=... [etc]
Of course, in either case one would be wise to first "mkvirtualenv myenv".
I'm not pinning the dependencies to specific versions; I'd rather find out that
there was a problem with, say, Django 1.5, and fix it. (It seems to work.)
R=aiuto
DELTA=115 (86 added, 16 deleted, 13 changed)
Revision created by MOE tool push_codebase.
MOE_MIGRATION=6639
Ensure that apis have a title.
Change on 2013-08-21 07:23:55-07:00 by yanivi
Remove xpp3 dependency and add google-http-client-jdo dependency for generated Apiary Java client library.
Change on 2013-08-21 08:53:29-07:00 by aiuto
Release C++ templates as open source.
Change on 2013-08-21 15:22:14-07:00 by aiuto
Use isVoid check in C# templates to make them easier to read.
From:
{% if method.responseType.fullClassName == 'void' %}...
To:
{% if method.responseType.isVoid %}
Checked with a boid for byte comparison test on several APIs.
Change on 2013-08-22 11:36:18-07:00 by aiuto
Use getter/setter policies in C++ to make the templates a little easier
to read. The reader now sees the concept (getterName) in one token, rather
than having to parse get_{{ var }}. E.g.
From:
{{ property.codeType }} get_{{ property.wireName }}() const {
To:
{{ property.codeType }} {{ property.getterName }}() const {
Tested with before/after byte-by-byte comparision of generated source.
Change on 2013-08-26 09:31:37-07:00 by smulloni
Set owner to the lowercase owner name by default.
Change on 2013-08-26 14:49:05-07:00 by cbao
cl/51181470 is made to add default title value for apis.
Currently, {{api.title}} would return empty string for Endpoints API, because it is not added into directory. This small CL is to make sure that {{api.title}} is returning the right value during bundle generation.
Change on 2013-08-27 13:19:28-07:00 by ngmiceli
Create Java 1.17 codegen branch
Set Java 1.13 generator to "obsolete"
Change on 2013-08-28 15:07:01-07:00 by aiuto
Refactor the naming triplets of transform, format and separator into a
NamingPolicy object.
Change on 2013-08-30 12:12:12-07:00 by lalex
Add the proper Method Response Type section to the HTML templates.
R=aiuto
DELTA=76 (69 added, 0 deleted, 7 changed)
Change on 2013-09-05 15:58:08-07:00 by ngmiceli
Delete Java 1.17 SNAPSHOT codegen directory.
Will be recreated with updated release scripts
Generate Java 1.17 codegen branch (take 2)
Change on 2013-09-06 07:44:43-07:00 by aiuto
Release the experimental HTML generator.
Fix url to python-gflags dependency
…gle-api-cpp-client. The two repos may be out of sync for a few hours. They may be out of sync with the libraries published on https://developers.google.com/ for a few days. Move the building of the full URL for a method from the caller site of creation of the method instance to the callee site. This effectively removes a call of the form StrCat(service->url, constant) from each method in the API and replaces it with a single StrCat() for the entire library. For x86_64, this is approx 100bytes per method. For a large API, this can be significant. E.g. in Google Drive, the .o file reduces by 22K.
API change and upcoming release.
Assumes python is on %PATH%.
generate into new directory remove version from all file names keep existing 'revision' value and set new 'revisionInt16', to make our intent clearer
Add summary to fix warning. Update project URL. Update owner for Google NuGet packages.
Change the generated comment from \mainpage to \brief so that multiple APIs can be generated together and produce reasonable results. Move the Data namespace below the main API namespace so the comment at the top of the file now applies to the latter.
Fix path to google.apis.snk. Remove vestiges of MSBuild NuGet package restore.
The Generate function in generate_library.py sets the package_path option to None by default. When the C# code generator looks for package_path in order to capitalize its components, it finds the None value and stops, instead of reading the packagePath value from the discovery document. Now, if package_path is None, we read the packagePath from the discovery document. When this code is used by Google's code generator server, the real value of package_path is used by default, instead of None, and the C# code actually capitalizes the library components. For example, Google.Apis.Admin.DataTransfer.datatransfer.v1. https://www.nuget.org/packages/Google.Apis.Admin.DataTransfer.datatransfer_v1 (The last component of *that* name, in turn, is lowercase in both cases. This is likely a bug, but one that will remain for backward compatibility.)
Hardcode "Google" as the owner. Update the path for customers to get the package source code.
We've changed how the generator chooses the revision (the fourth part of the version). The new revision numbers may overlap with the old ones, so we bump the version from 1.10.0.X to 1.10.1.X to avoid conflicts.
Pull in version 1.11.0 of support libraries. Since the support libraries no longer support .NET 4.0, we likewise remove support from the generated libraries. We also need to fix up references to the support libraries in the template .csproj.
|
Thanks for your pull request. It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). 📝 Please visit https://cla.developers.google.com/ to sign. Once you've signed, please reply here (e.g.
|
googlebot
added
the
cla: no
|
Hi Matt, I won't have time to review it, but a main concern:
Thanks |
|
This PR brings in all of apis-client-generator. Deleting some would just lead to more difficult merges later if we try to move changes back and forth. The extra languages aren't hurting anything. :) I expect it's "cla:no" because some of the commits are from |
|
/cc @jskeet |
|
Does this change include all languages (I saw only cpp and dart, maybe because this change is huge, those are the only languages github was able to display...)? |
|
Sorry, I could have been more explicit in the description. This PR comprises two changes:
This does include all languages: https://github.com/mmdriley/google-api-dotnet-client/tree/5f108b23dffb492c87d8c414e85938fb3426c752/ClientGenerator/src/googleapis/codegen/languages |
|
@chrsmith Take a look when you can? |
|
This looks good, I even spoke to Matt in person. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reasonable intro to subtrees. Of particular note: this should allow moving changes between github.com/google/apis-client-generator and our fork in
ClientGenerator.After removing the submodule in 21318da, I ran:
git subtree add --prefix=ClientGenerator https://github.com/mmdriley/apis-client-generator masterI chose not to use
--squashbecause I like having history in our repo. A bit of experimentation suggested a more important reason: it avoided some potential conflicts when merging from the original repo.Fixes #688.