docs: new core data model

README.md

@@ -77,6 +77,10 @@ To see the complete feature list for each language, please click the individual
     <td><a href="./docs/usage.md">OpenAPI</a></td>
     <td>We support the following OpenAPI versions: <em>Swagger 2.0 and OpenAPI 3.0</em>, which generates models for all the defined path request and responses.</td>
   </tr>
+  <tr>
+    <td><a href="./docs/usage.md#generate-models-from-datamodel">DataModel</a></td>
+    <td>This is the internal data model that all inputs gets converted to, and what generators are provided to generate any of the supported outputs. Read more about the </td>
+  </tr>
 </table>
 
 <a id="outputs"></a>

docs/Internal language.md

@@ -0,0 +1,79 @@
+# The model language
+
+It is important to understand the domain-specific language that describes the data models. It is what the input processors converts their inputs to, and the generators use to generate the output.
+
+It is also what developers can use to create their own models, since they can provide a **raw data model** as input. This way you can create your own custom data models that can be interpreted to any output language, with the full sweep of features the generators and presets support. See [Create your own models from the ground up, instead of a supported input](./advanced#create-your-own-models-from-the-ground-up-instead-of-a-supported-input)
+
+## The basics and process
+
+There are two parts to the model, one for the input processors (the **raw data model**), which they convert to, and then the one that the generators receive (the **constrained data model**).
+
+For example (and this accounts for almost all languages) you cannot render a property with the name `my property`, generally, they follow some kind of common naming format such as using camel case `myProperty` or pascal case `MyProperty`.
+
+This is the reason for having two data models because each output (Java, TS, Go, etc) have very specific constraints.
+
+Therefore the **raw data model** does not have any constraints, and it is perfectly normal and expected to name your properties `my property`. Before the model reaches the generator, it gets transformed to a **constrained data model**.
+
+<p align="center">
+  <img src="./img/RenderingProcess.png" />
+</p>
+
+The transformation happens in three stages. 
+
+1. Process the input and transform it into the raw data model. See [The raw data model](#the-raw-data-model) for more information.
+2. Split the raw data model into separate models that are rendered separately. See [The splitting of data models](#The-splitting-of-data-models) for more information. 
+3. Constrain the data models to the output language. See [The constrained data model](#the-constrained-data-model) for more information.
+
+## The raw data model
+The raw data model is what (now and in the future) Protobuf, JSON Schema, JSON Type Definition, Graphql types, are gonna be converted into.
+
+These are the following raw data models that we are gonna support:
+- **ArrayModel** is an unordered collection of a specific **DataModel**.
+- **TupleModel** is an ordered collection of **DataModel**s.
+- **EnumModel** is group of constants.
+- **UnionModel** represent that the model can be either or other **DataModel**s.
+- **ObjectModel** is a structure, that can be generated to class/interface/struct, etc, depending on the generator.
+- **DictionaryModel** is a map/dictionary of key/value **DataModel**s.
+- **ReferencedModel** is used for when a generator splits up models. See [The splitting of data models](#the-splitting-of-data-models).
+- **BooleanModel** represent boolean values.
+- **IntegerModel** represent natural numbers.
+- **FloatModel** represent floating-point numbers. 
+- **StringModel** represent string values.
+- **AnyModel** represent generic values that cannot otherwise be represented by one of the other models.
+
+<p align="center">
+  <img src="./img/RawDataModel.png" />
+</p>
+
+## The splitting of data models
+Each generator requires a different splitting of the models since it varies which models should be rendered as is, and which need to be rendered separately.
+
+For with the current TS generator, we should split:
+- ObjectModel
+- EnumModel (conditional, depends on the provided options, as inline string enums are possible) 
+
+For the Java generator, we should split:
+- ObjectModel
+- EnumModel
+- TupleModel (TS have these models natively supported, Java don't)
+- UnionModel (TS have these models natively supported, Java don't)
+
+## The constrained data model
+
+Extending the raw data models, we introduce the constrained data models. This is the model that generators and presets have access to. 
+
+What exactly get's constrained?
+
+- Enum values and keys
+- Property names
+- Data model names
+- Data model type
+
+How are they constrained?
+
+The answer to this question is not straightforward, cause each language has unique constraints that the data models much adhere to. This is TBD.
+
+<p align="center">
+  <img src="./img/ConstrainedDataModel.png" />
+</p>
+

docs/README.md

@@ -4,6 +4,7 @@
 
 <!-- toc -->
 
+- [Internal language](#internal-language)
 - [Contributing](#contributing)
 - [Usage](#usage)
 - [Advanced](#advanced)
@@ -18,6 +19,9 @@
 
 This document gives the overview of all the available documentation for Modelina.
 
+### [Internal language](./Internal%20language.md)
+Contains all the information you need to understand the internal data models and processes.
+
 ### [Contributing](./contributing.md)
 Contains all the information you need to contribute to this project.
 

docs/advanced.md

@@ -11,7 +11,6 @@ This document contains many of the advanced use-cases that you may stumble upon
 - [Use the models for data transfer](#use-the-models-for-data-transfer)
 - [Extend the logic of an existing renderer](#extend-the-logic-of-an-existing-renderer)
 - [Build your own model renderer](#build-your-own-model-renderer)
-- [Create your own models from the ground up, instead of a supported input](#create-your-own-models-from-the-ground-up-instead-of-a-supported-input)
 - [Add logging to library](#add-logging-to-library)
 - [Change the generated indentation type and size](#change-the-generated-indentation-type-and-size)
 - [Change the naming format for properties](#change-the-naming-format-for-properties)
@@ -49,10 +48,6 @@ TODO
 ## Build your own model renderer
 TODO 
 
-## Create your own models from the ground up, instead of a supported input
-TODO 
-
-
 ## Add logging to library
 When you generate models, by default, nothing is logged to the console or elsewhere.
 

docs/usage.md

@@ -13,6 +13,7 @@ For more specific integration options, please check out the [integration documen
 - [Generate models from AsyncAPI documents](#generate-models-from-asyncapi-documents)
 - [Generate models from JSON Schema documents](#generate-models-from-json-schema-documents)
 - [Generate models from Swagger 2.0 documents](#generate-models-from-swagger-20-documents)
+- [Generate models from DataModel](#generate-models-from-datamodel)
 - [Generate Go models](#generate-go-models)
 - [Generate C# models](#generate-c%23-models)
 - [Generate Java models](#generate-java-models)
@@ -58,6 +59,10 @@ The Swagger input processor expects that the property `swagger` is defined in or
 
 The response payload and `body` parameters, since it is a JSON Schema variant, is [interpreted as a such](./interpretation_of_JSON_Schema.md).
 
+## Generate models from DataModel
+Sometimes, the supported inputs such as AsyncAPI and JSON Schema wont be enough for your use-case. sometimes you want to create your own raw data models while still utilizing the full sweep of features from the generators.
+
+Check out this [example out for a live demonstration](../examples/custom-models).
 
 ## Generate Go models
 TODO 

examples/custom-models/README.md

@@ -0,0 +1,17 @@
+# Generating with custom models
+
+Using the internal data models, you can create your own data models from scratch, and still utilize the generators full sweep of features.
+
+## How to run this example
+
+Run this example using:
+
+```sh
+npm i && npm run start
+```
+
+If you are on Windows, use the `start:windows` script instead:
+
+```sh
+npm i && npm run start:windows
+```

examples/custom-models/index.spec.ts

@@ -0,0 +1,14 @@
+const spy = jest.spyOn(global.console, 'log').mockImplementation(() => { return; });
+import {generate} from './index';
+
+describe('Should be able to render custom data model', () => {
+  afterAll(() => {
+    jest.restoreAllMocks();
+  });
+  test('and should log expected output to console', async () => {
+    await generate();
+    //Generate is called 2x, so even though we expect 1 model, we double it
+    expect(spy.mock.calls.length).toEqual(2);
+    expect(spy.mock.calls[1]).toMatchSnapshot();
+  });
+});

examples/custom-models/index.ts

@@ -0,0 +1,14 @@
+import { TypeScriptGenerator, ObjectModel, StringModel } from '../../src';
+
+const generator = new TypeScriptGenerator();
+const customModel = new ObjectModel();
+const propertyModel = new StringModel();
+customModel.addProperty('test property name', propertyModel);
+
+export async function generate() : Promise<void> {
+  const models = await generator.generate(rootModel);
+  for (const model of models) {
+    console.log(model.result);
+  }
+}
+generate();

examples/custom-models/package.json

@@ -0,0 +1,10 @@
+{
+  "config" : { "example_name" : "custom-models" },
+  "scripts": {
+    "install": "cd ../.. && npm i",
+    "start": "../../node_modules/.bin/ts-node --cwd ../../ ./examples/$npm_package_config_example_name/index.ts",
+    "start:windows": "..\\..\\node_modules\\.bin\\ts-node --cwd ..\\..\\ .\\examples\\%npm_package_config_example_name%\\index.ts",
+    "test": "../../node_modules/.bin/jest --config=../../jest.config.js ./examples/$npm_package_config_example_name/index.spec.ts",
+    "test:windows": "..\\..\\node_modules\\.bin\\jest --config=..\\..\\jest.config.js examples/%npm_package_config_example_name%/index.spec.ts"
+  }
+}

package.json

@@ -71,8 +71,8 @@
     "docker:build": "docker build -t asyncapi/modelina .",
     "docker:test": "npm run docker:build && docker run asyncapi/modelina npm run test",
     "docker:test:blackbox": "npm run docker:build && docker run asyncapi/modelina npm run test:blackbox",
-    "test": "cross-env CI=true jest --coverage --testPathIgnorePatterns ./test/blackbox --testPathIgnorePatterns ./examples/TEMPLATE",
-    "test:examples": "cross-env CI=true jest ./examples --testPathIgnorePatterns ./examples/TEMPLATE",
+    "test": "cross-env CI=true jest --coverage --testPathIgnorePatterns ./test/blackbox --testPathIgnorePatterns ./examples/TEMPLATE  --testPathIgnorePatterns ./examples/custom-models",
+    "test:examples": "cross-env CI=true jest ./examples --testPathIgnorePatterns ./examples/TEMPLATE ./examples/custom-models",
     "test:blackbox": "cross-env CI=true jest ./test/blackbox",
     "test:watch": "jest --watch",
     "docs": "npm run docs:markdown",