Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(specs): make languages an enum #2865

Merged
merged 5 commits into from
Mar 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 86 additions & 14 deletions specs/common/schemas/IndexSettings.yml
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ baseIndexSettings:
type: array
items:
type: string
example: ['description']
example: [description]
description: Attributes for which to split [camel case](https://wikipedia.org/wiki/Camel_case) words.
default: []
x-categories:
Expand All @@ -158,11 +158,11 @@ baseIndexSettings:
indexLanguages:
type: array
items:
type: string
$ref: '#/supportedLanguage'
example:
- ja
description: |
[ISO code](https://en.wikipedia.org/wiki/List_of_ISO_639_language_codes) for a language for language-specific processing steps, such as word detection and dictionary settings.
Languages for language-specific processing steps, such as word detection and dictionary settings.

**You should always specify an indexing language.**
If you don't specify an indexing language, the search engine uses all [supported languages](https://www.algolia.com/doc/guides/managing-results/optimize-search-results/handling-natural-languages-nlp/in-depth/supported-languages/),
Expand Down Expand Up @@ -519,17 +519,16 @@ indexSettingsAsSearchParams:
queryLanguages:
type: array
items:
type: string
$ref: '#/supportedLanguage'
example:
- es
description: |
[ISO code](https://en.wikipedia.org/wiki/List_of_ISO_639_language_codes) for language-specific settings such as plurals, stop words, and word-detection dictionaries.
Languages for language-specific query processing steps such as plurals, stop-word removal, and word-detection dictionaries.

This setting sets a default list of languages used by the `removeStopWords` and `ignorePlurals` settings.
This setting also sets a dictionary for word detection in the logogram-based [CJK](https://www.algolia.com/doc/guides/managing-results/optimize-search-results/handling-natural-languages-nlp/in-depth/normalization/#normalization-for-logogram-based-languages-cjk) languages.
To support this, you must place the CJK language **first**.


**You should always specify a query language.**
If you don't specify an indexing language, the search engine uses all [supported languages](https://www.algolia.com/doc/guides/managing-results/optimize-search-results/handling-natural-languages-nlp/in-depth/supported-languages/),
or the languages you specified with the `ignorePlurals` or `removeStopWords` parameters.
Expand Down Expand Up @@ -979,14 +978,14 @@ ignorePlurals:
description: |
Treat singular, plurals, and other forms of declensions as equivalent.
You should only use this feature for the languages used in your index.
example: ['ca', 'es']
example: [ca, es]
oneOf:
- type: array
description: |
ISO code for languages for which this feature should be active.
This overrides languages you set with `queryLanguages`.
items:
type: string
description: |
ISO code for languages for which this feature should be active.
This overrides languages you set with `queryLanguages`.
$ref: '#/supportedLanguage'
- type: boolean
description: |
If true, `ignorePlurals` is active for all languages included in `queryLanguages`, or for all supported languages, if `queryLanguges` is empty.
Expand All @@ -1003,12 +1002,12 @@ removeStopWords:
In English, "the", "a", or "and" are stop words.

You should only use this feature for the languages used in your index.
example: ['ca', 'es']
example: [ca, es]
oneOf:
- type: array
description: ISO code for languages for which stop words should be removed. This overrides languages you set in `queryLanguges`.
items:
type: string
description: ISO code for languages for which stop words should be removed. This overrides languages you set in `queryLanguges`.
$ref: '#/supportedLanguage'
- type: boolean
default: false
description: |
Expand Down Expand Up @@ -1107,3 +1106,76 @@ semanticSearch:
type: array
items:
type: string

supportedLanguage:
type: string
description: ISO code for a supported language.
enum:
- af
- ar
- az
- bg
- bn
- ca
- cs
- cy
- da
- de
- el
- en
- eo
- es
- et
- eu
- fa
- fi
- fo
- fr
- ga
- gl
- he
- hi
- hu
- hy
- id
- is
- it
- ja
- ka
- kk
- ko
- ku
- ky
- lt
- lv
- mi
- mn
- mr
- ms
- mt
- nb
- nl
- no
- ns
- pl
- ps
- pt
- 'pt-br'
- qu
- ro
- ru
- sk
- sq
- sv
- sw
- ta
- te
- th
- tl
- tn
- tr
- tt
- uk
- ur
- uz
- zh
6 changes: 0 additions & 6 deletions specs/search/paths/dictionaries/common/parameters.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,6 @@ dictionaryType:
type: string
enum: [plurals, stopwords, compounds]

# misc
language:
description: ISO code of a [supported language](https://www.algolia.com/doc/guides/managing-results/optimize-search-results/handling-natural-languages-nlp/in-depth/supported-languages/).
example: en
type: string

standardEntries:
description: |
Key-value pairs of [supported language ISO codes](https://www.algolia.com/doc/guides/managing-results/optimize-search-results/handling-natural-languages-nlp/in-depth/supported-languages/) and boolean values.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,7 @@ dictionaryEntry:
description: Unique identifier for the dictionary entry.
example: 828afd405e1f4fe950b6b98c2c43c032
language:
type: string
description: ISO code of a [supported language](https://www.algolia.com/doc/guides/managing-results/optimize-search-results/handling-natural-languages-nlp/in-depth/supported-languages/).
example: de
$ref: '../../../../../common/schemas/IndexSettings.yml#/supportedLanguage'
word:
type: string
description: Matching dictionary word for `stopwords` and `compounds` dictionaries.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ post:
hitsPerPage:
$ref: '../../../common/schemas/IndexSettings.yml#/hitsPerPage'
language:
$ref: 'common/parameters.yml#/language'
$ref: '../../../common/schemas/IndexSettings.yml#/supportedLanguage'
responses:
'200':
description: OK
Expand Down
2 changes: 1 addition & 1 deletion templates/kotlin/oneof_interface.mustache
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ internal class {{classname}}Serializer : JsonContentPolymorphicSerializer<{{clas
element is JsonObject -> {{classname}}.{{#lambda.type-to-name}}{{{datatypeWithEnum}}}{{/lambda.type-to-name}}Value.serializer()
{{/isMap}}
{{#isArray}}
element{{#items}}{{#isModel}}.isJsonArrayOfObjects{{/isModel}}{{#isPrimitiveType}}.isJsonArrayOfPrimitives{{/isPrimitiveType}}{{#isModel}}{{#isPrimitiveType}} is JsonArray{{/isPrimitiveType}}{{/isModel}}{{/items}} -> {{{classname}}}.{{#lambda.type-to-name}}{{{datatypeWithEnum}}}{{/lambda.type-to-name}}Value.serializer()
element{{#items}}{{#isModel}}.isJsonArrayOfObjects{{/isModel}}{{#isPrimitiveType}}.isJsonArrayOfPrimitives{{/isPrimitiveType}}{{^isModel}}{{^isPrimitiveType}} is JsonArray{{/isPrimitiveType}}{{/isModel}}{{/items}} -> {{{classname}}}.{{#lambda.type-to-name}}{{{datatypeWithEnum}}}{{/lambda.type-to-name}}Value.serializer()
{{/isArray}}
{{#isEnumRef}}
element.isString -> {{{datatypeWithEnum}}}.serializer()
Expand Down
2 changes: 1 addition & 1 deletion templates/swift/tests/paramValue.mustache
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{{#isObject}}{{objectName}}({{^hasAdditionalProperties}}{{#value}}{{> tests/generateParams}}{{^-last}}, {{/-last}}{{/value}}{{/hasAdditionalProperties}}{{#hasAdditionalProperties}}from: [{{#value}}"{{key}}": {{> tests/paramValue }}{{^-last}}, {{/-last}}{{/value}}]{{/hasAdditionalProperties}}){{/isObject}}{{#isString}}{{#isAnyType}}AnyCodable({{/isAnyType}}"{{{value}}}"{{#isAnyType}}){{/isAnyType}}{{/isString}}{{#isNumber}}{{#isLong}}Int64({{/isLong}}{{{value}}}{{#isLong}}){{/isLong}}{{/isNumber}}{{#isBoolean}}{{{value}}}{{/isBoolean}}{{#isEnum}}{{objectName}}.{{#lambda.swiftIdentifier}}{{#lambda.camelcase}}{{valueEscaped}}{{/lambda.camelcase}}{{/lambda.swiftIdentifier}}{{/isEnum}}{{#isArray}}[{{#value}}{{> tests/generateParams}}{{^-last}}, {{/-last}}{{/value}}]{{/isArray}}{{#isMap}}Map[{{{.}}}]{{/isMap}}{{#isFreeFormObject}}{{#isAnyType}}[{{#value}}{{#entrySet}}"{{key}}": "{{{value}}}"{{/entrySet}}{{/value}}]{{/isAnyType}}{{^isAnyType}}[{{^value}}:{{/value}}{{#value}}{{> tests/generateParams}}{{^-last}}, {{/-last}}{{/value}}]{{/isAnyType}}{{/isFreeFormObject}}{{#isNull}}{{#inClientTest}}TestNull{{{objectName}}}(){{/inClientTest}}{{/isNull}}
{{#isObject}}{{objectName}}({{^hasAdditionalProperties}}{{#value}}{{> tests/generateParams}}{{^-last}}, {{/-last}}{{/value}}{{/hasAdditionalProperties}}{{#hasAdditionalProperties}}from: [{{#value}}"{{key}}": AnyCodable({{> tests/paramValue }}){{^-last}}, {{/-last}}{{/value}}]{{/hasAdditionalProperties}}){{/isObject}}{{#isString}}{{#isAnyType}}AnyCodable({{/isAnyType}}"{{{value}}}"{{#isAnyType}}){{/isAnyType}}{{/isString}}{{#isNumber}}{{#isLong}}Int64({{/isLong}}{{{value}}}{{#isLong}}){{/isLong}}{{/isNumber}}{{#isBoolean}}{{{value}}}{{/isBoolean}}{{#isEnum}}{{objectName}}.{{#lambda.swiftIdentifier}}{{#lambda.camelcase}}{{valueEscaped}}{{/lambda.camelcase}}{{/lambda.swiftIdentifier}}{{/isEnum}}{{#isArray}}[{{#value}}{{> tests/generateParams}}{{^-last}}, {{/-last}}{{/value}}]{{/isArray}}{{#isMap}}Map[{{{.}}}]{{/isMap}}{{#isFreeFormObject}}{{#isAnyType}}[{{#value}}{{#entrySet}}"{{key}}": "{{{value}}}"{{/entrySet}}{{/value}}]{{/isAnyType}}{{^isAnyType}}[{{^value}}:{{/value}}{{#value}}{{> tests/generateParams}}{{^-last}}, {{/-last}}{{/value}}]{{/isAnyType}}{{/isFreeFormObject}}{{#isNull}}{{#inClientTest}}TestNull{{{objectName}}}(){{/inClientTest}}{{/isNull}}
4 changes: 2 additions & 2 deletions tests/CTS/requests/search/search.json
Original file line number Diff line number Diff line change
Expand Up @@ -460,7 +460,7 @@
"personalizationImpact": 0,
"query": "",
"queryLanguages": [
""
"fr"
],
"queryType": "prefixAll",
"ranking": [
Expand Down Expand Up @@ -635,7 +635,7 @@
"personalizationImpact": 0,
"query": "",
"queryLanguages": [
""
"fr"
],
"queryType": "prefixAll",
"ranking": [
Expand Down
16 changes: 8 additions & 8 deletions tests/CTS/requests/search/setSettings.json
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@
"indexName": "theIndexName",
"indexSettings": {
"ignorePlurals": [
"algolia"
"fr"
]
},
"forwardToReplicas": true
Expand All @@ -98,7 +98,7 @@
"method": "PUT",
"body": {
"ignorePlurals": [
"algolia"
"fr"
]
},
"queryParameters": {
Expand Down Expand Up @@ -132,7 +132,7 @@
"indexName": "theIndexName",
"indexSettings": {
"removeStopWords": [
"algolia"
"fr"
]
},
"forwardToReplicas": true
Expand All @@ -142,7 +142,7 @@
"method": "PUT",
"body": {
"removeStopWords": [
"algolia"
"fr"
]
},
"queryParameters": {
Expand Down Expand Up @@ -258,7 +258,7 @@
"hitsPerPage": 10,
"ignorePlurals": false,
"indexLanguages": [
"algolia"
"fr"
],
"keepDiacriticsOnCharacters": "abc",
"maxFacetHits": 20,
Expand All @@ -275,7 +275,7 @@
],
"paginationLimitedTo": 0,
"queryLanguages": [
"algolia"
"fr"
],
"queryType": "prefixLast",
"ranking": [
Expand Down Expand Up @@ -398,7 +398,7 @@
"hitsPerPage": 10,
"ignorePlurals": false,
"indexLanguages": [
"algolia"
"fr"
],
"keepDiacriticsOnCharacters": "abc",
"maxFacetHits": 20,
Expand All @@ -415,7 +415,7 @@
],
"paginationLimitedTo": 0,
"queryLanguages": [
"algolia"
"fr"
],
"queryType": "prefixLast",
"ranking": [
Expand Down
Loading