Updating the swarm sample, to include actual product data. #7
Conversation
| with open(filename, "r") as f: | ||
| data = json.load(f) | ||
|
|
||
| if len(data) > 300: |
There was a problem hiding this comment.
I think this logic means that some records are skipped if number of json docs in the file is greater than 300? What is this for?
| purchase_history_container = database.get_container_client(PURCHASE_HISTORY_CONTAINER) | ||
|
|
||
| # Insert data into CosmosDB with embedding and indexing | ||
| file_prefix = "/Users/aayushkataria/git/multi-agent-swarm/src/data/" |
| query="SELECT * FROM c", | ||
| enable_cross_partition_query=True | ||
|
|
||
| def process_and_insert_data( |
There was a problem hiding this comment.
this could do with being parallelized and increasing default RU to 10000 for the container, or it takes quite a long time.
| ], | ||
| "vectorIndexes": [ | ||
|
|
||
| full_text_policy = { |
There was a problem hiding this comment.
You have added full_text_policy but I can't see full text search being used anywhere? If not being used, I would say better to remove it.
There was a problem hiding this comment.
Thanks @aayush3011 this is great! I added some comments. At minimum, the hardcoded data file location needs to get the path properly. The others are questions/suggestions. Great work, thanks for doing this!
No description provided.