docs: new core data model

README.md

@@ -77,6 +77,10 @@ To see the complete feature list for each language, please click the individual
     <td>OpenAPI</td>
     <td>We support the following OpenAPI versions: <em><a href="./docs/usage.md#generate-models-from-swagger-20-documents">Swagger 2.0</a> and <a href="./docs/usage.md#generate-models-from-openapi-documents">OpenAPI 3.0</a></em>, which generates models for all the defined request and response payloads.</td>
   </tr>
+  <tr>
+    <td><a href="./docs/usage.md#generate-models-from-meta-models">Meta model</a></td>
+    <td>This is the internal representation of a model for Modelina, it is what inputs gets converted to, and what generators are provided to generate code. Instead of relying on an input processor, you can create your own models from scratch, and still take advantage on the generators and the features.</td>
+  </tr>
 </table>
 
 <a id="outputs"></a>

docs/README.md

@@ -4,6 +4,7 @@
 
 <!-- toc -->
 
+- [The Processing](#the-processing)
 - [Contributing](#contributing)
 - [Usage](#usage)
 - [Advanced](#advanced)
@@ -18,6 +19,11 @@
 
 This document gives the overview of all the available documentation for Modelina.
 
+### [The Processing](./processing.md)
+Contains information how the internals of the processing works.
+
+This document contains all the information you need to understand the internal meta model and processes.
+
 ### [Contributing](./contributing.md)
 Contains all the information you need to contribute to this project.
 

docs/advanced.md

@@ -11,7 +11,6 @@ This document contains many of the advanced use-cases that you may stumble upon
 - [Use the models for data transfer](#use-the-models-for-data-transfer)
 - [Extend the logic of an existing renderer](#extend-the-logic-of-an-existing-renderer)
 - [Build your own model renderer](#build-your-own-model-renderer)
-- [Create your own models from the ground up, instead of a supported input](#create-your-own-models-from-the-ground-up-instead-of-a-supported-input)
 - [Add logging to library](#add-logging-to-library)
 - [Change the generated indentation type and size](#change-the-generated-indentation-type-and-size)
 - [Change the naming format for properties](#change-the-naming-format-for-properties)
@@ -55,10 +54,6 @@ TODO
 ## Build your own model renderer
 TODO 
 
-## Create your own models from the ground up, instead of a supported input
-TODO 
-
-
 ## Add logging to library
 When you generate models, by default, nothing is logged to the console or elsewhere.
 

docs/processing.md

@@ -0,0 +1,71 @@
+# The Process
+
+In order to generate data models from all kinds of inputs, we need a common structure for how we interact with one. That structure is called `MetaModel` often referred to as `Modelina Meta Model`, `Raw Meta Model`, or `MMM`. And there are two parts to it, there is the **meta model** and then the **constrained meta model**.
+
+# The Meta Model
+The **meta model** is what inputs (now and in the future) such as Protobuf, JSON Schema, JSON Type Definition, GraphQL types, are gonna be converted into. 
+
+These are the meta models and their meaning:
+- **ArrayModel** is an unordered collection of a specific **MetaModel**.
+- **TupleModel** is an ordered collection of **MetaModel**s.
+- **EnumModel** is group of constants.
+- **UnionModel** represent that the model can be either/or other **MetaModel**s.
+- **ObjectModel** is a structure, that can be generated to class/interface/struct, etc, depending on the output language
+- **DictionaryModel** is a map/dictionary of key/value **MetaModel**s.
+- **ReferencedModel** is primarily used for when models should be split up ([see the splitting of meta models](#the-splitting-of-data-models)) and referenced, or it could be an external reference to an external entity.
+- **BooleanModel** represent boolean values.
+- **IntegerModel** represent natural numbers.
+- **FloatModel** represent floating-point numbers. 
+- **StringModel** represent string values.
+- **AnyModel** represent generic values that cannot otherwise be represented by one of the other models.
+
+<p align="center">
+  <img src="./img/MetaModel.png" />
+</p>
+
+
+## The Constrained Meta Model
+
+Before the **meta models**s reaches the generator, it needs to be `constrained` to the output. 
+
+For example, constraining the **EnumModel** in Java means taking the raw enum key (for the **meta model** there are no constrains to what values may be used) such as `something% something` and convert it to a compliant (to the output) enum key that can be accessed directly, without having to call external libraries to find out of the result.
+
+This means that if you accessed `EnumValueModel.key` you would get `something% something`, and with the Java constrained variant `ConstrainedEnumValueModel.key` you get (example) `SOMETHING_PERCENT_SOMETHING`.
+
+How and what are constrained?
+
+The answer to this question is not straightforward, cause each language has unique constraints that the meta models much adhere to. This is TBD.
+
+<p align="center">
+  <img src="./img/ConstrainedMetaModel.png" />
+</p>
+
+
+## The Basics
+
+Inputs generally don't have the faintest idea about the constraints of an output and it is therefore the **meta model** does not have any constraints, and it is perfectly normal and expected to name your properties `my property`. 
+
+Before the model reaches the generator, it gets transformed to a **constrained meta model**. Here it converts the raw **meta model** into only having valid values for the specific output. For example (and this accounts for almost all languages) you cannot render a property with the name `my property`, as they generally follow some kind of common naming format such as using camel case `myProperty` or pascal case `MyProperty`. 
+
+This transformation happen in three stages. 
+
+<p align="center">
+  <img src="./img/RenderingProcess.png" />
+</p>
+
+1. Process the input and transform it into the meta model. See [The meta model](./the_meta_model.md) for more information.
+2. Split the meta model into separate models that are rendered separately. See [The splitting of meta models](#The-splitting-of-data-models) for more information. 
+3. Constrain the meta models to the output language. See [The constrained meta model](#the-constrained-data-model) for more information.
+
+## The splitting of Meta Models
+Each generator requires a different splitting of the **meta model**s because it varies which should be rendered as is, and which need to be rendered separately.
+
+For example with the current TS generator, we split the following models:
+- **ObjectModel**, because we want to generate it into interfaces, or classes
+- **EnumModel**, because we want to generate a representation for enums
+
+For the Java generator, we split the following models:
+- **ObjectModel**, because we want to generate it into a Java Class
+- **EnumModel**, because we want to generate it into a Java Enum.
+- **TupleModel** (TS have these models natively supported, Java don't, so we need to generate alternatives)
+- **UnionModel** (TS have these models natively supported, Java don't, so we need to generate alternatives)

docs/usage.md

@@ -15,6 +15,7 @@ For more specific integration options, please check out the [integration documen
 - [Generate models from JSON Schema documents](#generate-models-from-json-schema-documents)
 - [Generate models from Swagger 2.0 documents](#generate-models-from-swagger-20-documents)
 - [Generate models from OpenAPI documents](#generate-models-from-openapi-documents)
+- [Generate models from Meta models](#generate-models-from-meta-models)
 - [Generate Go models](#generate-go-models)
 - [Generate C# models](#generate-c%23-models)
 - [Generate Java models](#generate-java-models)
@@ -60,6 +61,11 @@ The Swagger input processor expects that the property `swagger` is defined in or
 
 The response payload and `body` parameters, since it is a JSON Schema variant, is [interpreted as a such](./interpretation_of_JSON_Schema.md).
 
+## Generate models from Meta models
+Sometimes, the supported inputs such as AsyncAPI and JSON Schema wont be enough for your use-case and you want to create your own data models while still utilizing the full sweep of features from the generators.
+
+Check out this [example out for a live demonstration](../examples/meta-model).
+
 ## Generate models from OpenAPI documents
 
 There are one way to generate models from an OpenAPI document

examples/README.md

@@ -23,6 +23,7 @@ This directory contains a series of self-contained examples that you can use as
 - [csharp-generate-equals-and-hashcode](./csharp-generate-equals-and-hashcode) - A basic example on how to generate models that overwrite the `Equal` and `GetHashCode` methods
 - [csharp-generate-serializer](./csharp-generate-serializer) - A basic example on how to generate models that include function to serialize the data models to JSON
 - [generate-javascript-models](./generate-javascript-models) - A basic example to generate JavaScript data models
+- [meta-model](./meta-model) - A basic example how to provide a meta model for the generator
 - [javascript-use-esm](./javascript-use-esm) - A basic example that generate the models to use ESM module system.
 - [javascript-use-cjs](./javascript-use-cjs) - A basic example that generate the models to use CJS module system.
 - [javascript-generate-marshalling](./javascript-generate-marshalling) - A basic example of how to use the un/marshalling functionality of the javascript class.

examples/meta-model/README.md

@@ -0,0 +1,17 @@
+# Meta model
+
+Using the internal meta model representation, you can create your own data models from scratch, and still utilize the generators full sweep of features.
+
+## How to run this example
+
+Run this example using:
+
+```sh
+npm i && npm run start
+```
+
+If you are on Windows, use the `start:windows` script instead:
+
+```sh
+npm i && npm run start:windows
+```

examples/meta-model/index.spec.ts

@@ -0,0 +1,14 @@
+const spy = jest.spyOn(global.console, 'log').mockImplementation(() => { return; });
+import {generate} from './index';
+
+describe('Should be able to use generator with meta model', () => {
+  afterAll(() => {
+    jest.restoreAllMocks();
+  });
+  test('and should log expected output to console', async () => {
+    await generate();
+    //Generate is called 2x, so even though we expect 1 model, we double it
+    expect(spy.mock.calls.length).toEqual(2);
+    expect(spy.mock.calls[1]).toMatchSnapshot();
+  });
+});

examples/meta-model/index.ts

@@ -0,0 +1,14 @@
+import { TypeScriptGenerator, ObjectModel, StringModel } from '../../src';
+
+const generator = new TypeScriptGenerator();
+const customModel = new ObjectModel();
+const propertyModel = new StringModel();
+customModel.addProperty('test property name', propertyModel);
+
+export async function generate() : Promise<void> {
+  const models = await generator.generate(rootModel);
+  for (const model of models) {
+    console.log(model.result);
+  }
+}
+generate();

examples/meta-model/package.json

@@ -0,0 +1,10 @@
+{
+  "config" : { "example_name" : "meta-model" },
+  "scripts": {
+    "install": "cd ../.. && npm i",
+    "start": "../../node_modules/.bin/ts-node --cwd ../../ ./examples/$npm_package_config_example_name/index.ts",
+    "start:windows": "..\\..\\node_modules\\.bin\\ts-node --cwd ..\\..\\ .\\examples\\%npm_package_config_example_name%\\index.ts",
+    "test": "../../node_modules/.bin/jest --config=../../jest.config.js ./examples/$npm_package_config_example_name/index.spec.ts",
+    "test:windows": "..\\..\\node_modules\\.bin\\jest --config=..\\..\\jest.config.js examples/%npm_package_config_example_name%/index.spec.ts"
+  }
+}

package.json

@@ -71,8 +71,8 @@
     "docker:build": "docker build -t asyncapi/modelina .",
     "docker:test": "npm run docker:build && docker run asyncapi/modelina npm run test",
     "docker:test:blackbox": "npm run docker:build && docker run asyncapi/modelina npm run test:blackbox",
-    "test": "cross-env CI=true jest --coverage --testPathIgnorePatterns ./test/blackbox --testPathIgnorePatterns ./examples/TEMPLATE",
-    "test:examples": "cross-env CI=true jest ./examples --testPathIgnorePatterns ./examples/TEMPLATE",
+    "test": "cross-env CI=true jest --coverage --testPathIgnorePatterns ./test/blackbox --testPathIgnorePatterns ./examples/TEMPLATE  --testPathIgnorePatterns ./examples/meta-model",
+    "test:examples": "cross-env CI=true jest ./examples --testPathIgnorePatterns ./examples/TEMPLATE ./examples/meta-model",
     "test:blackbox": "cross-env CI=true jest ./test/blackbox",
     "test:watch": "jest --watch",
     "docs": "npm run docs:markdown",
@@ -90,7 +90,11 @@
   },
   "release": {
     "branches": [
-      "master"
+      "master",
+      {
+        "name": "next",
+        "prerelease": true
+      }
     ],
     "plugins": [
       [