Azure AI Search Python Playground 🌐

Welcome to the Azure AI Search Python Playground! This repository contains a collection of Jupyter notebooks designed to explore the various capabilities of Azure AI Search. You'll find experiments focused on vector search, retrieval-augmented generation (RAG), and other retrieval system tasks using OpenAI's embedding models. Each notebook is a standalone experiment with detailed explanations to help you understand the concepts and apply them in your own projects.

📓 Notebooks

This repository includes a diverse range of Jupyter notebooks demonstrating integrations and functionalities with Azure AI Search:

Foundational Techniques and Embeddings

1. OpenAI Text-Embedding-3-Large Model Exploration
   Enhance search relevance using the OpenAI text-embedding-3-large model.

2. Cohere Embed-v3 Exploration
   Improve embeddings and retrieval quality with the Cohere Embed-v3 model.

3. Indexing and Querying CSV Data
   Learn how to index and query CSV datasets in Azure AI Search.

4. Document Boosting Techniques
   Influence relevance scoring with document boosting strategies.

5. Quantization Techniques
   Optimize efficiency and performance through advanced quantization.

6. Maximum Marginal Relevance
   Post-Process Vector Search Results using MMR (Maximum Marginal Relevance)

RAG Evaluations and Quality Measurement

6. RAG Evaluation with Tonic AI
   Assess Retrieval Augmented Generation (RAG) performance with Tonic AI Validate.

7. RAG Evaluation with TruLens
   Gain insights into RAG solutions using TruLens.

8. RAG Evaluation with Arize Phoenix
   Monitor and improve RAG quality with Arize Phoenix analytics.

9. Measuring Search Relevance with Ranx
   Benchmark and quantify search relevance using Ranx.

10. RAG Evaluation with LlamaIndex, Literal AI, and RAGAS
    Refine and validate RAG pipelines using LlamaIndex, Literal AI logs, and RAGAS metrics.

Advanced Workflows, Agents, and Multimodal Scenarios

11. LlamaIndex Workflows
    Orchestrate asynchronous search workflows with LlamaIndex and Azure AI Search.

12. NVIDIA RAG with LlamaIndex
    Build advanced RAG pipelines leveraging NVIDIA NIM embeddings and LlamaIndex.

13. Multimodal RAG using Contextual Retrieval
    Integrate LlamaParse Premium, Azure OpenAI, and Arize Phoenix for multimodal retrieval.

14. OpenAI Swarm Multi-Agent
    Experiment with multi-agent capabilities using the OpenAI Swarm framework.

15. Getting Started with Voyage Multimodal Embeddings
    Explore voyage-multimodal-3 embeddings for combined text-image retrieval.

Domain-Specific Agents and Applications

17. Legal AI Agent with CrewAI
    Build a CrewAI Agent for complex legal queries, integrating Azure AI Search and Azure OpenAI.

18. Medical AI Agent with CrewAI
    Develop a multi-agent medical assistant for cardiology scenarios.

19. Insurance Claims Agent with Semantic Kernel
    Develop a multi-agent insurance assistant for Hurricane damage to your property.

20. HR Recruitment Agent with Semantic Kernel
    Automate HR candidate screening with a Semantic Kernel-driven agent.

🚀 Getting Started

To begin experimenting with these notebooks, set up your Azure AI environment with the following steps:

1. Set Up an Azure Account: Ensure you have an Azure account. If not, you can sign up for free.

2. Create Azure AI Resources: Follow the Azure AI documentation to create the necessary resources, including Azure OpenAI service instances.

3. Clone This Repository: Clone the repository to your local machine or Azure Notebooks environment:

   git clone https://github.com/farzad528/azure-ai-search-python-playground.git

🛠️ Install Dependencies

Each notebook specifies its dependencies at the beginning. To ensure a smooth experience:

1. Open the Notebook: Navigate to the notebook of interest.

2. Install Dependencies: Follow the markdown instructions at the top of each notebook to install the necessary Python packages.

3. Use a Virtual Environment (Recommended): It's highly recommended to use a virtual environment (via conda, virtualenv, or Python's built-in venv module) to avoid conflicts between project dependencies.

By following the in-notebook instructions, you can ensure that all necessary dependencies for a given experiment are correctly installed.

📚 How to Use

Each notebook is self-contained and includes step-by-step instructions to guide you through the experiments. To get started, simply open a notebook in your Jupyter environment and follow the provided instructions.

🤝 Contributing

We welcome contributions from the community! If you have suggestions for additional experiments or improvements to existing notebooks, please feel free to open an issue or submit a pull request.

🔔 Stay Updated

For more information and updates on using Azure AI for search and retrieval tasks, follow my blog on Hashnode.

Thank you for exploring the Azure AI Search Python Playground. Happy experimenting! 🎉