Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fragment validator interface & implementation #735

Merged
merged 11 commits into from
Jan 23, 2024

Conversation

ankitk-me
Copy link
Contributor

No description provided.

@@ -170,7 +173,7 @@ public final class Engine implements Collector, AutoCloseable
DispatchAgent agent =
new DispatchAgent(config, tasks, labels, errorHandler, tuning::affinity,
bindings, exporters, guards, vaults, catalogs, metricGroups, converterFactory,
this, coreIndex, readonly);
validatorFactory, this, coreIndex, readonly);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If possible, converters and validators should follow the same pattern as the others, passing a collection instead of a factory.

@@ -139,6 +141,9 @@ Converter createReader(
Converter createWriter(
ConverterConfig converter);

Validator supplyValidator(
ValidatorConfig converter);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move this up next to other supplyXXX methods for guard, vault and catalog.
Also, shouldn't this be returning a ValidatorHandler (per core) instead of a Validator (global) given that it is on EngineContext which is per core?


Validator create(
ValidatorConfig config,
LongFunction<CatalogHandler> supplyCatalog);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems inconsistent, as Validator is global, whereas CatalogHandler is per core.

public String type()
{
return "string";
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move this to first method position for clarity.

String encoding = object.containsKey(ENCODING_NAME)
? object.getString(ENCODING_NAME)
: null;
result = new StringValidatorConfig(encoding);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use builder for both cases.
Also use enum returned from value.getValueType() in a switch instead of if-else with instanceof.

else
{
assert false;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for this assert, covered by schema.

public ValidatorContext supply(
EngineContext context)
{
return new StringValidatorContext();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Usually we pass the engine context in the constructor.


public StringValidator(
StringValidatorConfig config,
LongFunction<CatalogHandler> supplyCatalog)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to pass supplyCatalog here if we can call it from EngineContext passed to supply method instead?

private int incompleteCharCount;
private int incompleteChar;

public StringValidatorHandler(StringValidatorConfig config)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Formatting and naturally need to pull out the encoding from the config.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public StringValidatorHandler(StringValidatorConfig config)
public StringValidatorHandler(
StringValidatorConfig config)

private int incompleteCharCount;
private int incompleteChar;

public StringValidatorHandler(StringValidatorConfig config)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public StringValidatorHandler(StringValidatorConfig config)
public StringValidatorHandler(
StringValidatorConfig config)

Comment on lines 52 to 54
final int charByteCount = (charByte0 & 0b1000_0000) != 0
? Integer.numberOfLeadingZeros((~charByte0 & 0xff) << 24)
: 1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

charByteCount should move to the else clause as it is not needed for the if clause.

@@ -77,31 +73,19 @@ public boolean validate(
}
}
index += charByteCount;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feels like we are double counting the char bytes when fragmented, since index++ will happen for each pending byte and index += charByteCount will happen for complete chars.

Perhaps we can simplify this algorithm to work from the perspective of pendingBytes always, and handle complete chars as a special case, where we need to reinitialize the number of pendingBytes from the first character byte?

Comment on lines 81 to 88
if ((flags & FLAGS_FIN) != 0x00)
{
valid = initialize && incompleteCharCount == 0 && index == limit;
initialize = false;
valid = pendingCharBytes == 0 && index == limit;
}
else
{
valid = initialize && index == limit;
valid = index + pendingCharBytes == limit;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest simplifying to a ternary operator.

value = (flags & FLAGS_FIN) == 0x00
    ? index + pendingCharBytes == limit
    : pendingCharBytes == 0 && index == limit;

Btw, why is index + pendingCharBytes == limit the correct expression for fragmented messages? The index and length represent a single fragment, so why does that relate to pendingBytes?

StringValidatorConfig result = null;
if (value instanceof JsonString)
switch (type)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
switch (type)
switch (value.valueType())

@@ -53,29 +58,23 @@ public JsonValue adaptToJson(
public StringValidatorConfig adaptFromJson(
JsonValue value)
{
JsonValue.ValueType type = value.getValueType();
StringValidatorConfig result = null;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use config instead of result?

@@ -19,14 +19,9 @@

public interface ValidatorContext
{
default ValidatorHandler attach(
default ValidatorHandler supplyHandler(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to remove the default and { return null; } as this must always be implemented.

@jfallows jfallows marked this pull request as ready for review January 23, 2024 03:43
}
},

INVALID
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If schema requires utf_8 as only possibility, this can be removed, agree?

* WARRANTIES OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package io.aklivity.zilla.runtime.types.core;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All non-public code should be in internal subpackage like other modules.

@jfallows jfallows merged commit 2a2aaca into aklivity:feature/schema-registry Jan 23, 2024
3 checks passed
@ankitk-me ankitk-me deleted the fragmentValidator branch January 25, 2024 04:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants