[Bug]: Query failed with double quotes

[baiyuelanshan]

Is there an existing issue for this?

• I have searched the existing issues

Environment

- Milvus version:2.2.9
- Deployment mode(standalone or cluster):standalone
- MQ type(rocksmq, pulsar or kafka): rocksmq
- SDK version(e.g. pymilvus v2.0.0rc2): milvus-sdk-java 2.2.9
- OS(Ubuntu or CentOS): CentOS
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

when I used scalar filtering on a varchar fieldd with double quotes, I met an expression error:

Error: failed to create query plan: cannot parse expression: tag == "A "test"", error: invalid expression: tag == "A "test"" 

code for setting the expression:
searchParam = builder.withExpr("tag == "A "test""").build();

Expected Behavior

No response

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response

[xiaofan-luan]

you have to do it like 'A "test"' or you have to use Escape character

[yanliang567]

/assign @baiyuelanshan
please retry as comments above.

/unassign

[baiyuelanshan]

acctually, I have use Escape charater, but the text editor didn't display it .

[xiaofan-luan]

@yanliang567
could you try on 2.2.9 and latest 2.3.3?

[yanliang567]

/assign @NicoYuan1986

[NicoYuan1986]

Reproduced on v2.2.9 and master-latest. Needs repair. @yanliang567

[xiaofan-luan]

/assign @xiaocai2333
could you help on it

[xiaofan-luan]

@NicoYuan1986
is this reproducible on python sdk also?

[NicoYuan1986]

@NicoYuan1986 is this reproducible on python sdk also?

Yes.
milvus v2.2.9, I use pymilvus==2.2.9
milvus master-latest, I use pymilvus==2.3.3
I think it's related

[xiaocai2333]

@baiyuelanshan If your expression contains a string, the string must be enclosed in double quotes or single quotes. You can try this:

.withExpr("question == \"A \\\"test\\\"\"")

[NicoYuan1986]

if insert "A "test"" or 'A "test"', can search with expr="tag == "A \"test\"""
if insert "A 'test'", can search with expr="tag == "A 'test'""

[chenxin6]

if insert "A "test"" or 'A "test"', can search with expr="tag == "A "test""" if insert "A 'test'", can search with expr="tag == "A 'test'""

I have inserted a string like A "test" B. But I can't search this data by using the expr tag == "A \"test\" B".

[xiaocai2333]

if insert "A "test"" or 'A "test"', can search with expr="tag == "A "test""" if insert "A 'test'", can search with expr="tag == "A 'test'""

I have inserted a string like A "test" B. But I can't search this data by using the expr tag == "A \"test\" B".

Thank you for bringing up this issue. Our handling of escaped characters in strings wasn't very robust previously. We plan to address this in a future release.
And recommend avoiding escaped characters in the meantime when using the current version.

[yanliang567]

@xiaocai2333 @NicoYuan1986 can we close this issue now?

[xiaocai2333]

@xiaocai2333 @NicoYuan1986 can we close this issue now?

I think it has been fixed.
@NicoYuan1986 please verify it.

[NicoYuan1986]

@xiaocai2333 @NicoYuan1986 can we close this issue now?

I think it has been fixed. @NicoYuan1986 please verify it.

Test case test_search_expression_with_double_quotes has been added. And it performs well in nightly tests.
So close the issue now. version: master-20240111-930735c4