Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exclusive minimum and maximum values are not supported as numeric constraints #856

Closed
rjgladish opened this issue Dec 11, 2023 · 5 comments · Fixed by frictionlessdata/datapackage-v2-draft#11
Assignees
Milestone

Comments

@rjgladish
Copy link

In version 1 of Frictionless Tabular Schema , numeric constraints are limited to closed intervals, i.e. [,], [minimum,], [,maximum], or [minimum, maximum].

However, some observational measurement limits are best expressed as minimum and maximum values that approach a limit.
E.g. minimum temperature measurements should be greater than absolute zero
(0,] Kelvin,
(−273.15,] °C,
(−459.67,] °F

Absent closed interval support, limits must be approximated using arbitrary finite precision, such as 0.000000000001 K, -273.149999999999 °C, and so on. Also, Frictionless min/max constraints that corresponding to SQL "greater than" or "less than" checks get messy. Although this workaround is mostly adequate for integer min/mix, it's messy.

The addition of two optional constraint properties to express open minimum and maximum intervals with exclusive values with permit schema to describe closed interval constraint conditions [minimumEx,], [,maximumEx], and [minimumEx,maximumEx], and half-open intervals, e.g. (minimum, maximumEx], [minimumEx, maximum)

Since it would be optional, it would minimally impact older software, but it would require a new schema.
On the negative side, software that is unaware of exclusive constraints only look for inclusive constraints.

@peterdesmet
Copy link
Member

I had a similar use case, where I want to indicate that the width of a bounding box can be close 0, but not 0. As you describe, I solved this by providing a very small number with an arbitrary fine precision:

"minimum": 1e-15 # same as 0.000000000000001

While two new properties minimumEx and maximumEx allow to express that better, I fear it might lead to even less validation (due to lack of software support). So I don't prefer it over the above "hack".

@rjgladish
Copy link
Author

Thanks for the feedback.
The approach you suggest will work, and realistically, it is the only reasonable interoperable approach using V1 schema, but it may introduce implementation-specific rounding behaviors. IFF the precision of min/max values are greater than the precision supported in the underlying validation implementation, rounding minimum and maximum, down to 0 and up, respectively, may introduce unexpected behaviors that can be difficult to debug

Perhaps this can be revisited in V2, which was recently opened for discussion?

Because the addition of exclusiveMinimum and exclusiveMaximum as constraints would create better alignment with
OAS (OpenAPI) V2.0, V3.0 schema e.g. https://github.com/OAI/OpenAPI-Specification/blob/main/schemas/v3.0/schema.json, and could leverage and json-schema.org Draft 4 http://json-schema.org/draft-04/schema#/properties/exclusiveMinimum and .../exclusiveMaximum, and thr issues raised above, I think it may be worth another look in the next MAJOR schema revision.

@roll roll added this to the v2 milestone Jan 3, 2024
@roll roll changed the title Exclusive minimum and maximum values are not supported as numeric constraints in Frictionless Tabular Schema Exclusive minimum and maximum values are not supported as numeric constraints Jan 3, 2024
@roll roll added the proposal label Jan 6, 2024
@roll roll self-assigned this Jan 6, 2024
@rjgladish
Copy link
Author

To clarify my earlier comments Dec 11 and Dec 23.

JSON Draft 4 and OAS Schema properties exclusiveMinimum and exclusiveMaximum are Boolean types. A default value of FALSE indicates inclusive minimum and inclusive maximum, respectively, preserves the V1 interpretation. In contrast, A value of TRUE specifies that minimum and maximum are interpreted as exclusive minimum and inclusive maximum, respectively.

@peterdesmet
Copy link
Member

@roll the accepted #11 implementation adds exclusiveMinimum and exclusiveMaximum as constraints that behave exactly like minimum and maximum (but excluding the actual value).

I personally think this is a good implementation, but it differs from what @rjgladish proposes in #856 (comment), where those fields are boolean types that affect the behaviour of minimum and maximum. It would be good to express why the former implementation was chosen.

@roll
Copy link
Member

roll commented Jan 25, 2024

@peterdesmet
JSON Schema has fixed draft-4 behavior - https://json-schema.org/understanding-json-schema/reference/numeric#range

In JSON Schema Draft 4, exclusiveMinimum and exclusiveMaximum work differently. There they are boolean values, that indicate whether minimum and maximum are exclusive of the value.

So basically we just stick to the current JSON Schema version (as Table Schema inherits a lot from there)

@roll roll modified the milestones: v2.0-draft, v2.0 Sep 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants