Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added openApi spec formats #22

Merged
merged 6 commits into from
Apr 2, 2021
Merged

Added openApi spec formats #22

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
aa03409
Added openApi spec formats
seriousme Mar 27, 2021
ddbfb10
Merge branch 'master' into master
epoberezkin Mar 28, 2021
c509499
Merge branch 'master' into master
seriousme Mar 31, 2021
7f89297
Reworked float, double, int32 and int64
seriousme Mar 31, 2021
fc389cd
updated tests to force "number" type
seriousme Apr 1, 2021
c2399d0
Merge branch 'master' into master
epoberezkin Apr 2, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,13 @@ The package defines these formats:
- _uuid_: Universally Unique IDentifier according to [RFC4122](http://tools.ietf.org/html/rfc4122).
- _json-pointer_: JSON-pointer according to [RFC6901](https://tools.ietf.org/html/rfc6901).
- _relative-json-pointer_: relative JSON-pointer according to [this draft](http://tools.ietf.org/html/draft-luff-relative-json-pointer-00).
- _byte_: base64 encoded data according to the [openApi 3.0.0 specification](https://spec.openapis.org/oas/v3.0.0#data-types)
- _int32_: signed 32 bits integer according to the [openApi 3.0.0 specification](https://spec.openapis.org/oas/v3.0.0#data-types)
- _int64_: signed 64 bits according to the [openApi 3.0.0 specification](https://spec.openapis.org/oas/v3.0.0#data-types)
- _float_: float according to the [openApi 3.0.0 specification](https://spec.openapis.org/oas/v3.0.0#data-types)
- _double_: double according to the [openApi 3.0.0 specification](https://spec.openapis.org/oas/v3.0.0#data-types)
- _password_: password string according to the [openApi 3.0.0 specification](https://spec.openapis.org/oas/v3.0.0#data-types)
- _binary_: binary string according to the [openApi 3.0.0 specification](https://spec.openapis.org/oas/v3.0.0#data-types)

See regular expressions used for format validation and the sources that were used in [formats.ts](https://github.com/ajv-validator/ajv-formats/blob/master/src/formats.ts).

Expand Down
35 changes: 35 additions & 0 deletions src/formats.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,13 @@ export type FormatName =
| "json-pointer"
| "json-pointer-uri-fragment"
| "relative-json-pointer"
| "byte"
| "int32"
| "int64"
| "float"
| "double"
| "password"
| "binary"

export type DefinedFormats = {
[key in FormatName]: Format
Expand Down Expand Up @@ -62,6 +69,21 @@ export const fullFormats: DefinedFormats = {
"json-pointer-uri-fragment": /^#(?:\/(?:[a-z0-9_\-.!$&'()*+,;:=@]|%[0-9a-f]{2}|~0|~1)*)*$/i,
// relative JSON-pointer: http://tools.ietf.org/html/draft-luff-relative-json-pointer-00
"relative-json-pointer": /^(?:0|[1-9][0-9]*)(?:#|(?:\/(?:[^~/]|~0|~1)*)*)$/,
// the following formats are used by the openapi specification: https://spec.openapis.org/oas/v3.0.0#data-types
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

v3.1.0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually 3.0.0 lists more datatypes than 3.1.0

The OpenApi crew thinks semantic versioning is not relevant since each spec explictly names its version, however for tool builders its a bit annoying that a minor version increase is not backwards compatible :-(

// byte: https://github.com/miguelmota/is-base64
byte: /^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/gm,
// signed 32 bit integer
int32: validateInteger(32),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these should be defined as:

{
  type: "number",
  validateInteger(32),
}

otherwise that would be applied to strings (they should not be) and would not be applied to numbers (they should be)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got it to work using:

{
  type: "number",
  validate: validateInteger(32)
}

The problem that I have now is that the test on a non-numerical value ("x") now returns true instead of false ?!?
Can I just safely remove that test ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to fail on non-numeric value the schema should have type: "number", format on its own should not fail non-numbers.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the general approach of JSON Schema that type should be explicit in the schema, it is never implied by anything else. Even your old definition would pass {} for example on that test, unless there is type in the schema.

// signed 64 bit integer
int64: validateInteger(64),
// C-type float
float: validateNumber(128),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly, float can be constrained by hardcoded IEEE 754 limits (as 2**128 is not precise anyway and would cause run-time conversions)...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will need to read up a bit on this ;-)

// C-type double
double: validateNumber(1024),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this could be defined as {type: "number", validate: isFinite}, although type: "number" would do it already, so maybe just true

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From a javascript perspective yes, but if you validate them before export then a receiver using C will see a difference between floats and doubles. This was the only test I found to check the difference between floats and doubles.
We can set it to true, but it might come back to haunt us ;-)

// hint to the UI to hide input strings
password: true,
// unchecked string payload
binary: true,
}

export const fastFormats: DefinedFormats = {
Expand Down Expand Up @@ -169,6 +191,19 @@ function uri(str: string): boolean {
return NOT_URI_FRAGMENT.test(str) && URI.test(str)
}

function validateInteger(bits: number): (value: number | string) => boolean {
seriousme marked this conversation as resolved.
Show resolved Hide resolved
const max = BigInt(2) ** BigInt(bits - 1)
const min = max * BigInt(-1)
return (value) => Number.isInteger(+value) && BigInt(value) <= max && BigInt(value) >= min
seriousme marked this conversation as resolved.
Show resolved Hide resolved
}

function validateNumber(bits: number): (value: number | string) => boolean {
seriousme marked this conversation as resolved.
Show resolved Hide resolved
const max = Number(BigInt(2) ** BigInt(bits - 1))
const min = max * -1

return (value) => max >= value && min <= value
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

strictly ">" I think (although the limit is slightly lower I think)...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is where it becomes a bit of a challenge
Javascript integers go up to 2^53 hence the BigInt type
Ideally AJV would use BigInt for all its integer handling, however BigInt only plays nice with other BigInts :-(
(e.g. 2*BigInt(2) fails BigInt(2)*BigInt(2) succeeds.
So either we stick to basic javascript integers and skip the whole BigInt stuff of we need to come up with something clever ..

}

const Z_ANCHOR = /[^\\]\\Z/
function regex(str: string): boolean {
if (Z_ANCHOR.test(str)) return false
Expand Down
152 changes: 152 additions & 0 deletions tests/extras/format.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -728,5 +728,157 @@
"valid": true
}
]
},
{
"description": "validation of Byte",
"schema": {"format": "byte"},
"tests": [
{
"description": "'hello world' encoded in base64",
"data": "aGVsbG8gd29ybGQ=",
"valid": true
},
{
"description": "multiline base64",
"data": "VGhpcyBpcyBhIGJhc2U2NCBtdWx0aWxpbmUgc3RyaW5nIHRoYXQgc3BhbnMgbW9yZSB0aGFuIDc2\nIGNoYXJhY3RlcnMgdG8gdGVzdCBtdWx0aWxpbmUgY2FwYWJpbGl0aWVzCg==",
"valid": true
},
{
"description": "Invalid base64",
"data": "aGVsbG8gd29ybG=",
"valid": false
}
]
},
{
"description": "validation of int32",
"schema": {"format": "int32"},
"tests": [
{
"description": "256 is ok",
"data": "256",
"valid": true
},
{
"description": "256.1 fails",
"data": "256.1",
"valid": false
},
{
"description": "2**32 fails",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the test should be that 231 also fails, 232 - 1 succeeds

"data": "4294967296",
"valid": false
},
{
"description": "non-numeric fails",
"data": "x",
"valid": false
}
]
},
{
"description": "validation of int64",
"schema": {"format": "int64"},
"tests": [
{
"description": "256 is ok",
"data": "256",
"valid": true
},
{
"description": "256.1 fails",
"data": "256.1",
"valid": false
},
{
"description": "2**64 fails",
"data": "18446744073709551616",
"valid": false
},
{
"description": "non-numeric fails",
"data": "x",
"valid": false
}
]
},
{
"description": "validation of float",
"schema": {"format": "float"},
"tests": [
{
"description": "256 is ok",
"data": "256",
"valid": true
},
{
"description": "256.1 is ok",
"data": "256.1",
"valid": true
},
{
"description": "2**128 fails",
"data": "3.402823669209385e+38",
"valid": false
},
{
"description": "non-numeric fails",
"data": "x",
"valid": false
}
]
},
{
"description": "validation of double",
"schema": {"format": "double"},
"tests": [
{
"description": "256 is ok",
"data": "256",
"valid": true
},
{
"description": "256.1 is ok",
"data": "256.1",
"valid": true
},
{
"description": "2**1023 is ok",
"data": "8.98846567431158e+307",
"valid": true
},
{
"description": "2**1024 fails",
"data": "179769313486231590772930519078902473361797697894230657273430081157732675805500963132708477322407536021120113879871393357658789768814416622492847430639474124377767893424865485276302219601246094119453082952085005768838150682342462881473913110540827237163350510684586298239947245938479716304835356329624224137216n",
"valid": false
},
{
"description": "non-numeric fails",
"data": "x",
"valid": false
}
]
},
{
"description": "validation of password",
"schema": {"format": "password"},
"tests": [
{
"description": "'password string' is ok",
"data": "password string",
"valid": true
}
]
},
{
"description": "validation of binary",
"schema": {"format": "binary"},
"tests": [
{
"description": "'binary string' is ok",
"data": "binary string",
"valid": true
}
]
}
]