feat: add pg vector index

[huangzhuo1949]

Summary

Close #12341

Screenshots

Before	After
...	...

Checklist

Important

Please review the checklist below before submitting your pull request.

• This change requires a documentation update, included: Dify Document
• I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
• I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
• I've updated the documentation accordingly.
• I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

[crazywoola]

Please link an existing issue or create one in the description. :)
This helps others to understand why you need this.

[huangzhuo1949]

Please link an existing issue or create one in the description. :) This helps others to understand why you need this.

done~

[bowenliang123]

Suggested change

	# DONE: create index https://github.com/pgvector/pgvector?tab=readme-ov-file#indexing
	# PG hnsw index only support 2000 dimension or less
	# pgvector's hnsw index support 2000 or less dimensions
	# ref: https://github.com/pgvector/pgvector?tab=readme-ov-file#indexing

[bowenliang123]

Prevent to use global variables for less memory consumpsion. Consider to use in-line variable inside the function instead.

[bowenliang123]

Consider use sqlalchemy's text aka sql_text method to wrap and templating instead. Or simply format strings.

[bowenliang123]

And how about explictly set the options (m and ef_construction) as well, making the index DDL more readable and helpful?
https://github.com/pgvector/pgvector?tab=readme-ov-file#index-options

CREATE INDEX ON table USING hnsw (embedding vector_l2_ops) WITH (m = 16, ef_construction = 64);

[huangzhuo1949]

And how about explictly set the options (m and ef_construction) as well, making the index DDL more readable and helpful? https://github.com/pgvector/pgvector?tab=readme-ov-file#index-options

CREATE INDEX ON table USING hnsw (embedding vector_l2_ops) WITH (m = 16, ef_construction = 64);

thanks for this suggestion，I have added it~