Missing SimpleType with xsdata 21.12 compared to 21.11

[pcasenove]

Hi,
Compared to xsdata 21.11, when generating bindings from XSD using 21.12, some SimpleTypes are missing in the generated class.
The XSD set of files are not public, so I can't easily share those; but did something change in the generation code between 21.11 and 21.12?

I've noticed that the 2 simple types that were not generated anymore where not included in ComplexType in the same XSD (but they are included in ComplexTYpes in others XSD that include this one)

Thanks,

[pcasenove]

Some more tests, as I think the issue comes from #629
The simplified setup:
We have xsd_1 which defines SimpleType A and B
xsd_2 defines its own types and one is extending SimpleType A
xsd_3 defines its own types and one is extending SimpleType B

xsd_1, 2 and 3 and in 3 different folders.

When I call xsdata generate on xsd_2 folder, the python class for xsd_1 contains SimpleType A and not B
When I call xsdata generate on xsd_3 folder, the python class for xsd_1 contains SimpleType B and not A.

If the change is really related to the issue 629, we will have to stick to xsdata 21.11, as I didn't see any option to maintain the old behavior

[tefra]

Can you please create a sample xsd or post an excerpt?

Send me an email @pcasenove

[pcasenove]

I came down to a pretty simple example attached, showing the difference between 21.11 and 21.12.

test_case.tar.gz

In "dict" folder, a simple ins.xsd file, defining 3 simple types (type1, type2 and type 3). Only type 1 is used in a ComplexType ctype1.
I then generate the bindings using : xsdata generate -p XXXX /home/user/tmp/dict/ins/
"toto11" is the folder using 21.11: you see that in ins.py, all sympleTypes are there
"toto12" is the folder using 21.12: you see that in ins.py, only type1 is there

And if you create a foo.xsd, with a complexType ctype2, using type 2 from ins.xsd (not in the archive), you get:
21.11: same result
21.12: you see that in ins.py,type1 and type 2 are defined

[tefra]

I see, if those enumerations are not used in the models why are they necessary?

As part of #629 all unused types were blocked from generation.

[pcasenove]

In fact the enumerations are defined in the top ins.xsd file, but not used in it. The file is then imported in other xsd, using one of those enumerations when they need one of those.
But we don't have a xsd where all types are instantiated...
Maybe an option to enable the full generation?

[tefra]

The cli doesn't check for usages only in the current xsd, it should have created those types if there was at least once occurence.

I know it's a private project @pcasenove but I need the original schemas or an excerpt that I can use to reproduce the issue.

email: chris@komposta.net

[pcasenove]

And I forgot another detail: the xsd are in different folders, so as xsdata is not going through folders, it also explain the issue we have

[tefra]

And I forgot another details: the xsd are in different folders, so as xsdata is not going through folders, it also explain the issue we have

Oh I see so you are running the cli command multiple times?

Have you tried using the command with input source the whole directory?

xsdata generate file/path/with/schemas/

[pcasenove]

Yes we run it multiple times, on each folder. We can't put all files in a single directory because the import are done using relative paths. It would require massive changes or inline sed.
I think we tried giving the top dir only, but as we don't have any xsd file at the root, it doesn't generate anything. But I'll have to check this

[pcasenove]

If I give the top folder, it just does nothing:

$ xsdata generate -c xsdata.xml -p test file /path/with/schemas/
Traceback (most recent call last):
File "/cvmfs/euclid-dev.in2p3.fr/CentOS7/EDEN-3.0/bin/xsdata", line 10, in
sys.exit(cli())
File "/cvmfs/euclid-dev.in2p3.fr/CentOS7/EDEN-3.0/lib/python3.9/site-packages/click/core.py", line 1128, in call
return self.main(*args, **kwargs)
File "/cvmfs/euclid-dev.in2p3.fr/CentOS7/EDEN-3.0/lib/python3.9/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/cvmfs/euclid-dev.in2p3.fr/CentOS7/EDEN-3.0/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/cvmfs/euclid-dev.in2p3.fr/CentOS7/EDEN-3.0/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/cvmfs/euclid-dev.in2p3.fr/CentOS7/EDEN-3.0/lib/python3.9/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/cvmfs/euclid-dev.in2p3.fr/CentOS7/EDEN-3.0/lib/python3.9/site-packages/xsdata/cli.py", line 124, in generate
transformer.process(list(resolve_source(source)))
File "/cvmfs/euclid-dev.in2p3.fr/CentOS7/EDEN-3.0/lib/python3.9/site-packages/xsdata/codegen/transformer.py", line 103, in process
self.process_classes()
File "/cvmfs/euclid-dev.in2p3.fr/CentOS7/EDEN-3.0/lib/python3.9/site-packages/xsdata/codegen/transformer.py", line 186, in process_classes
raise CodeGenerationError("Nothing to generate.")
xsdata.exceptions.CodeGenerationError: Nothing to generate.

And as I said, we can't move all xsd in a single folder. I'll send you the whole schemas by email

[tefra]

xsdata architecture is to parse the complete schema definitions and then start analyzing, it's the only way to produce trully valid results.

You are right the cli looks for any xsd files in the root of the input dir. Can you please give it another try with this update #644?

I enabled the recursive globs for the files finder.

$ xsdata generate issue/schemas/ST_DM_Schema/ -ss namespaces

...
Builder: 4 main and 0 inner classes
Analyzer input: 2283 main and 0 inner classes
Analyzer output: 2183 main and 0 inner classes
Generating package: init
Generating package: init
Generating package: init
....

[pcasenove]

Thanks a lot!!! That's great and much easier to generate now :)
You can close the issue or keep it open until the next version, as you want

[tefra]

Awesome! I still have to add some unit tests, but that's it I will try to release a new version this weekend

[tefra]

Thanks for reporting @pcasenove the new recursive mode will be available with the cli flag --recursive