more doc clean up, fomatting, modularization- no technical changes (#…

…1016)

docs/modules/ROOT/pages/getting-started/assembly-installing-registry-docker.adoc

@@ -1,5 +1,4 @@
 // Metadata created by nebel
-
 include::{mod-loc}shared/all-attributes.adoc[]
 
 [id="installing-the-registry-docker"]

docs/modules/ROOT/pages/getting-started/assembly-installing-registry-openshift.adoc

@@ -1,5 +1,4 @@
 // Metadata created by nebel
-
 include::{mod-loc}shared/all-attributes.adoc[]
 
 [id="installing-registry-ocp"]

docs/modules/ROOT/pages/getting-started/assembly-intro-to-registry-rules.adoc

@@ -10,7 +10,7 @@ This chapter introduces the optional rules used to govern registry content and p
 * xref:registry-rules[]
 * xref:registry-rules-apply[]
 * xref:registry-rules-work[]
-* xref:registry-rules-config[]  
+* xref:registry-rules-config[]   
 
 //INCLUDES
 include::{mod-loc}getting-started/con-registry-rules.adoc[leveloffset=+1]

docs/modules/ROOT/pages/getting-started/assembly-intro-to-the-registry.adoc

@@ -1,5 +1,4 @@
 // Metadata created by nebel
-
 include::{mod-loc}shared/all-attributes.adoc[]
 
 [id="intro-to-the-registry"]

docs/modules/ROOT/pages/getting-started/assembly-managing-registry-artifacts-api.adoc

@@ -1,5 +1,4 @@
 // Metadata created by nebel
-
 include::{mod-loc}shared/all-attributes.adoc[]
 
 [id="managing-registry-artifacts-api"]

docs/modules/ROOT/pages/getting-started/assembly-managing-registry-artifacts-maven.adoc

@@ -1,5 +1,4 @@
 // Metadata created by nebel
-
 include::{mod-loc}shared/all-attributes.adoc[]
 
 [id="managing-registry-artifacts-maven"]

docs/modules/ROOT/pages/getting-started/assembly-managing-registry-artifacts-ui.adoc

@@ -1,5 +1,4 @@
 // Metadata created by nebel
-
 include::{mod-loc}shared/all-attributes.adoc[]
 
 [id="managing-registry-artifacts-ui"]

docs/modules/ROOT/pages/getting-started/assembly-registry-reference.adoc

@@ -1,5 +1,4 @@
 // Metadata created by nebel
-
 include::{mod-loc}shared/all-attributes.adoc[]
 
 [id="artifact-and-rule-types"]

docs/modules/ROOT/pages/getting-started/assembly-using-kafka-client-serdes.adoc

@@ -1,5 +1,4 @@
 // Metadata created by nebel
-//
 include::{mod-loc}shared/all-attributes.adoc[]
 
 [id="using-kafka-client-serdes"]
@@ -14,6 +13,9 @@ This chapter provides instructions on how to use the Kafka client serializers an
 * xref:registry-serdes-concepts-strategy-{context}[]
 * xref:registry-serdes-concepts-constants-{context}[]
 * xref:registry-serdes-types-serde-{context}[]
+* xref:registry-serdes-types-avro-{context}[]
+* xref:registry-serdes-types-json-{context}[]
+* xref:registry-serdes-types-protobuf-{context}[]
 * xref:registry-serdes-register-{context}[]
 * xref:registry-serdes-config-consumer-{context}[]
 * xref:registry-serdes-config-producer-{context}[]
@@ -34,12 +36,14 @@ include::{mod-loc}getting-started/con-registry-serdes-concepts.adoc[leveloffset=
 include::{mod-loc}getting-started/con-registry-serdes-strategy.adoc[leveloffset=+1]
 include::{mod-loc}getting-started/con-registry-serdes-constants.adoc[leveloffset=+1]
 include::{mod-loc}getting-started/con-registry-serdes-types.adoc[leveloffset=+1]
+include::{mod-loc}getting-started/con-registry-serdes-avro.adoc[leveloffset=+2]
+include::{mod-loc}getting-started/con-registry-serdes-json.adoc[leveloffset=+2]
+include::{mod-loc}getting-started/con-registry-serdes-protobuf.adoc[leveloffset=+2]
 include::{mod-loc}getting-started/proc-registry-serdes-register.adoc[leveloffset=+1]
 include::{mod-loc}getting-started/proc-registry-serdes-config-consumer.adoc[leveloffset=+1]
 include::{mod-loc}getting-started/proc-registry-serdes-config-producer.adoc[leveloffset=+1]
 include::{mod-loc}getting-started/proc-registry-serdes-config-stream.adoc[leveloffset=+1]
 
-
 //.Additional resources (or Next steps)
 
 //* ...

docs/modules/ROOT/partials/getting-started/con-registry-artifacts.adoc

@@ -1,7 +1,7 @@
 // Metadata created by nebel
 
 [id="registry-artifacts"]
-= Schema and API artifacts in {registry}
+= Schema and API artifacts in {registry} 
 
 The items stored in {registry}, such as event schemas and API specifications, are known as registry _artifacts_. The following shows an example of an Apache Avro schema artifact in JSON format for a simple share price application:
 

docs/modules/ROOT/partials/getting-started/con-registry-serdes-avro.adoc

@@ -0,0 +1,90 @@
+// Module included in the following assemblies:
+//  assembly-using-kafka-client-serdes
+
+[id='registry-serdes-types-avro-{context}']
+= Configure Avro SerDe with {registry}
+
+{registry} provides Kafka client serializer and deserializer classes for Apache Avro to make using Avro as
+easy as possible:
+
+* `io.apicurio.registry.utils.serde.AvroKafkaSerializer`
+* `io.apicurio.registry.utils.serde.AvroKafkaDeserializer`
+
+
+.Configure the Avro serializer
+
+You can configure the Avro serializer class in the following ways:
+
+* {registry} location as a URL
+* Artifact ID strategy 
+* Global ID strategy 
+* Global ID location
+* Global ID handler
+* Avro datum provider
+* Avro encoding
+
+.Global ID location
+The serializer passes the unique global ID of the schema as part of the Kafka message so that consumers can use the right schema for deserialization.  The location of that global ID can be in the payload of the message or in the message headers. The default approach is to pass the global ID in the message payload. If you want the ID sent in the message headers instead, you can set the following configuration property:
+----
+props.putIfAbsent(AbstractKafkaSerDe.USE_HEADERS, "true")
+----
+The property name is `apicurio.registry.use.headers`.
+
+
+.Global ID handler
+You can customize precisely how the global ID is encoded when passing it in the Kafka message body. Set
+the configuration property `apicurio.registry.id-handler` to be a class that implements the
+`io.apicurio.registry.utils.serde.strategy.IdHandler` interface. {registry} provides two implementations of
+that interface:
+
+* `io.apicurio.registry.utils.serde.strategy.DefaultIdHandler` - stores the ID as an 8 byte long
+* `io.apicurio.registry.utils.serde.strategy.Legacy4ByteIdHandler` - stores the ID as an 4 byte int
+
+{registry} represents the global ID of an artifact as a long, but for legacy reasons (or for compatibility with other registries or serde classes) you may want to use 4 bytes when sending the ID.
+
+.Avro datum provider
+Avro provides different datum writers and readers to write and read data. {registry} supports three different types:
+
+* Generic
+* Specific
+* Reflect
+
+The {registry} `AvroDatumProvider` is the abstraction on which type is then actually used, where `DefaultAvroDatumProvider` is used by default.
+
+There are two configuration options you can set:
+
+* `apicurio.registry.avro-datum-provider` - provide a fully qualified Java class name of the `AvroDatumProvider` implementation, for example `io.apicurio.registry.utils.serde.avro.ReflectAvroDatumProvider`
+* `apicurio.registry.use-specific-avro-reader` - true or false, to use specific type when using `DefaultAvroDatumProvider`
+
+.Avro encoding
+
+When using Apache Avro to serializer data, it is common to use the Avro binary encoding format. This is so that the data is encoded in as efficient a format as possible.  However, Avro also supports encoding the data as JSON. Encoding as JSON is useful because it is much easier to inspect the payload of each message, often for logging, debugging, or other similar use cases.  The {registry} Avro serializer can be configured to change the encoding to JSON from the default (binary).
+
+Set the Avro encoding to use by configuring the `apicurio.avro.encoding` property. The value must be either
+`JSON` or `BINARY`.
+
+.Configure the Avro deserializer
+
+You must configure the Avro deserializer class to match the configuration settings of the serializer.  As a
+result, you can configure the Avro deserializer class in the following ways:
+
+* {registry} location as a URL
+* Global ID handler
+* Avro datum provider
+* Avro encoding
+
+See the serializer section for these configuration options - the property names and values are the same.
+
+[NOTE] 
+====
+The following options are not needed when configuring the deserializer:
+
+* Artifact ID strategy
+* Global ID strategy
+* Global ID location
+====
+
+The reason these options are not necessary is that the deserializer class can figure this information out from
+the message itself. In the case of the two strategies, they are not needed because the serializer is responsible for sending the global ID of the schema as part of the message. 
+
+The location of that global ID is determined by the deserializer by simply checking for the magic byte at the start of the message payload. If that byte is found, the global ID is read from the message payload using the configured handler.  If the magic byte is not found, the global ID is read from the message headers.

docs/modules/ROOT/partials/getting-started/con-registry-serdes-concepts.adoc

@@ -17,7 +17,8 @@ Schemas can evolve, so you can define rules in {registry}, for example, to ensur
 
 These schema technologies can be used by client applications through Kafka client serializer/deserializer (SerDe) services provided by {registry}.  The maturity and usage of the SerDe classes provided by {registry} may vary. See the type-specific sections below for more details about each.
 
-= Producer schema configuration
+[discrete]
+== Producer schema configuration
 
 A producer client application uses a serializer to put the messages that it sends to a specific broker topic into the correct data format.
 
@@ -35,7 +36,8 @@ After registering your schema, when you start Kafka and {registry}, you can acce
 
 If a schema already exists, you can create a new version using the REST API based on compatibility rules defined in {registry}. Versions are used for compatibility checking as a schema evolves. An artifact ID and schema version represents a unique tuple that identifies a schema.
 
-= Consumer schema configuration
+[discrete]
+== Consumer schema configuration
 A consumer client application uses a deserializer to get the messages that it consumes from a specific broker topic into the correct data format.
 
 To enable a consumer to use {registry} for deserialization:

docs/modules/ROOT/partials/getting-started/con-registry-serdes-json.adoc

@@ -0,0 +1,42 @@
+// Module included in the following assemblies:
+//  assembly-using-kafka-client-serdes
+
+[id='registry-serdes-types-json-{context}']
+= Configure JSON Schema SerDe with {registry}
+
+{registry} provides Kafka client serializer and deserializer classes for JSON Schema to make using JSON Schema as easy as possible:
+
+* `io.apicurio.registry.utils.serde.JsonSchemaKafkaSerializer`
+* `io.apicurio.registry.utils.serde.JsonSchemaKafkaDeserializer`
+
+Unlike Apache Avro, JSON Schema is not actually a serialization technology - it is instead a validation
+technology. As a result, configuration options for JSON Schema are quite different. For example, there is no
+encoding option, because data is always encoded as JSON.
+
+.Configure the JSON Schema serializer
+
+You can configure the JSON Schema serializer class in the following ways:
+
+* {registry} location as a URL
+* Artifact ID strategy 
+* Global ID strategy
+* Validation enabled/disabled
+
+The only non-standard configuration property is whether JSON Schema validation is enabled or
+disabled.  The validation feature is disabled by default but can be enabled by setting
+`apicurio.registry.serdes.json-schema.validation-enabled` to `"true"`. For example:
+----
+props.putIfAbsent(JsonSchemaSerDeConstants.REGISTRY_JSON_SCHEMA_VALIDATION_ENABLED, "true")`
+----
+
+.Configure the JSON Schema deserializer
+
+You can configure the JSON Schema deserializer class in the following ways:
+
+* {registry} location as a URL
+* Validation enabled/disabled
+
+The deserializer is simple to configure. You must provide the location of {registry} so that the schema can be loaded. The only other configuration is whether or not to perform validation.  These
+configuration properties are the same as for the serializer.
+
+NOTE: Deserializer validation only works if the serializer passes the global ID in the Kafka message, which will only happen when validation is enabled in the serializer.

docs/modules/ROOT/partials/getting-started/con-registry-serdes-protobuf.adoc

@@ -0,0 +1,47 @@
+// Module included in the following assemblies:
+//  assembly-using-kafka-client-serdes
+
+[id='registry-serdes-types-protobuf-{context}']
+
+= Configure Protobuf SerDe with {registry}
+
+{registry} provides Kafka client serializer and deserializer classes for Google Protobuf to make using Protobuf as easy as possible:
+
+* `io.apicurio.registry.utils.serde.ProtobufKafkaSerializer`
+* `io.apicurio.registry.utils.serde.ProtobufKafkaDeserializer`
+
+.Configure the Protobuf serializer
+
+You can configure the Protobuf serializer class in the following ways:
+
+* {registry} location as a URL
+* Artifact ID strategy 
+* Global ID strategy 
+* Global ID location
+* Global ID handler
+
+.Configure the Protobuf deserializer
+
+You must configure the Protobuf deserializer class to match the configuration settings of the serializer.  As a result, you can configure the Protobuf deserializer class in the following ways:
+
+* {registry} location as a URL
+* Global ID handler
+
+See the serializer section for these configuration options - the property names and values are the same.
+
+[NOTE]
+====
+The following options are not needed when configuring the deserializer:
+
+* Artifact ID strategy
+* Global ID strategy
+* Global ID location
+====
+
+The reason these options are not necessary is that the deserializer class can figure this information out from
+the message itself. In the case of the two strategies, they are not needed because the serializer is responsible for sending the global ID of the schema as part of the message.  
+
+The location of that global ID is determined (by the deserializer) by simply checking for the magic byte at the start of the message payload. If that byte is found, the global ID is read from the message payload (using the configured handler). If the magic byte is not found, the global ID is read from the message headers.
+
+NOTE: The Protobuf deserializer does not deserialize to your exact Protobuf Message implementation,
+but rather to a `DynamicMessage` instance (because there is no appropriate API to do otherwise).

docs/modules/ROOT/partials/getting-started/con-registry-serdes-strategy.adoc

@@ -29,6 +29,7 @@ Strategies to return an artifact ID based on an implementation of `ArtifactIdStr
 `TopicIdStrategy`:: (Default) strategy that uses the topic name and `key` or `value` suffix.
 `SimpleTopicIdStrategy`:: Simple strategy that only uses the topic name.
 
+
 [discrete]
 == Global ID strategy