human-ai-co-op-toward-a-useful-vaccines-summary.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "**Submitted by Merck & Co., Inc.** \n",
    "\n",
    "The CORD-19 Research Database is a growing collection of 50k+ scientific papers relating to information on variants of coronavirus. This enormous dataset poses challenges in terms of finding the right kind of information quickly for the purpose of finding useful and actionable information on today's COVID-19 pandemic. \n",
    "\n",
    "Here we focus efforts on the \"Vaccines\" task due to our specific interests in helping to find the best treatment possible and due to this task being particularly difficult compared to most others because of the need for domain expertise and knowledge. However, we note that our approach is applicable to all tasks.\n",
    "\n",
    "The data science community has gathered together in order to build tools and technologies for the sole purpose of searching, ranking, extracting and aggregating results from this database. However, we found that the extraction + aggregation step of this challenge poses the greatest difficulty. \n",
    "\n",
    "We noticed that even using state of the art NLP technology can only bring us partially to the solution. Therefore, our approach focuses on leveraging existing knowledge + NLP to get us 80% of the way there, and then utilizing human review to help with the final step of aggregating and summarizing the most useful information. Our philosophy is that using human review can get us to the last 20% of the process.\n",
    "\n",
    "**NOTE: It is highly recommended to use a GPU for this notebook to speed up some of the BERT related processes**\n",
    "\n",
    "## Specific Technical Challenges\n",
    "We summarize below the specific challenges we personally encountered and noticed among other submissions.\n",
    "1. Search and rank of information is often limited to titles and information extraction is limited to abstract due to scalability issues\n",
    "2. The BM25 ranking algorithm is a popular way to quickly search full text, but is heavily dependent on using the correct keywords, terms and the full body text being of good quality (when often it is not)\n",
    "3. Deeper context from the full text is usually not available, often lacking the necessary supplementary information to understand the extracted information fully\n",
    "4. Significant human review often not leveraged, resulting in systematic biases that may not be corrected or noticed. Additionally, question Answering or BERT based summarization is not feasible due to lack of corpus specific training and/or very slow inference times. Older non-deep learning methods do not output great results either.\n",
    "5. Relationships between articles are often not considered, which could be an indicator of the quality and strength of the article results (especially if the article was heavily cited).\n",
    "\n",
    "## Our Contribution Toward Resolving the Technical Challenges\n",
    "1. Since titles can be misleading, we focus our search and rank on exclusively the abstract. We accomplish this by embedding each sentence of the abstract and doing a semantic similarity search via a pretrained BioBERT deep learning model on these sentences.\n",
    "2. Like many others, we also use the BM25 algorithm to search the full text. However, we use the results of the previous step (similarity search on abstract sentences) to guide our BM25 query. We take the non-stopword tokens from the similarity results and feed them into the BM25 algorithm to search for the relevant pieces of full body text in order to provide deeper context. We also remove sentences from the full body that are less than 6 words, as they are likely to contain labelings that were accidentally extracted from the PDFs. We believe that using the same language from the abstract leads to more relevant BM25 results. \n",
    "3. We provide deeper context to our results by returning multiple results from the full body instead of just outputting the top result. This is achieved by taking the top 1.5 standard deviations of BM25 scores relative to the average score. Additionally, we identify mentioned antiviral agents or related terms and associate them with each literature to provide context on which antiviral agents are being studied. Finally, we create a **knowledge graph**, the first version of which identifies quickly papers that make references to others. In this way, we can see how many times a specific paper was cited in others.\n",
    "4. We then do a human manual review of our query results to carefully aggregate and summarize the most useful information pertaining to the question. We believe this is the most necessary step that current approaches cannot do automatically. \n",
    "5. We create a knowledge graph that calculates the relationships between articles. Here we contribute to a first implementation on looking at which articles cite each other.\n",
    "\n",
    "## Approach\n",
    "Here, we propose a searching and ranking framework using a combination of BioBERT sentence embeddings and BM25 ranking algorithm to guide reviewers quickly to the relevant pieces of information relative to a particular query. We specifically use a **two-stage process** in order to do 1) an initial search on sentence embeddings on abstract content, and 2) subsequently perform BM25 search of the full text based on the copied language from the first stage results.\n",
    "\n",
    "Specifically:\n",
    "0. An initial exhaustive keyword search is done to filter on the 50k+ articles pertaining to vaccines and vaccine development based on prior knowledge.\n",
    "1. User provides a query related to the question of interest (i.e. \"Antiviral effects of Covid\").\n",
    "2. Sentence embeddings are generated for the user query via pre-trained BioBERT for sentence similarity.\n",
    "3. All abstracts are then split into individual sentences and embedded via the same pre-trained BioBERT for sentence similarity.\n",
    "4. Using cosine distances, the embedded query is then compared against the corpus of abstract sentences to find the most similar sentence.\n",
    "5. All results above a pre-specified similarity threshold (0.65 is the default) are returned (**These are the stage-1 results**). If a known keyword is included in the query, an additional filtering step is done to gaurantee that these results contain that keyword.\n",
    "6. The **stage 1** results are then word tokenized, processed, and then fed into the fast Okapi BM25 ranking algorithm to search for the relevant sections of the full text that may give deeper context to the initial results (**These are the stage-2 results**). We also include metadata on the number of times the found article was cited to give context on the importance of the article\n",
    "7. The output is either printed or saved, and human review is performed on the results to pick and extract out the most relevant pieces of text for final aggregation and summarization.\n",
    "\n",
    "\n",
    "## Future Direction\n",
    "### Knowledge Graphs\n",
    "*Why a knowledge graph* - One of the initial tools in our kit we wanted to explore was instantiating a knowledge graph to represent the myriad information encoded in the corpus of text. In our early review of previous submissions we noted that many approaches were treating articles as independent objects of information. Due to the fact that science is an iterative and progressive process that builds upon historical and recent works - we wanted to capture that context and apply it in our analysis of the corpus. An initial knowledge graph was explored and built representing a variety of different entities in the corpus (authors, articles, citations, text). However, due to technical and time constraints we opted for a 100% in-memory option using networkx on a subset on entities and articles to accomplish some representation of citations.\n",
    "\n",
    "*Technical challenges* - Due to a lack of memory efficient graph storage methods in kaggle, a lot of time was spent exploring options that would work around this constraint. Almost a full library of code was written to encode and store a knowledge graph in sqlite. While progress was made and certain entities robustly represented and encoded, it was evident that any robust graph algorithm would not run acceptably on such a data structure so the approach was shelved. Any team considering such an option must think hard about recursive querying and efficiency in such a structure to achieve something workable.\n",
    "\n",
    "*Future work* - Here are some of the things we're hoping a knowledge graph can help us explore in the future alongside other experiments to better answer the task at hand:\n",
    "\n",
    "- Does this work belong to a series of works?\n",
    "- Can we find the articles that represent the \"supporting knowledge\" for a given article?\n",
    "- Can we determine unique work given the context of all other works?\n",
    "- Is there a network of authors contributing to the same domain?\n",
    "- Can we attribute scientific rigor to certain articles, authors and apply that \"trust\" in the final aggregation step of the solution?\n",
    "- Can we represent articles with their body text and identify similar content in other articles to help find parallel / adjacent / orthogonal work?\n",
    "- Can we represent articles by the language used to cite them?\n",
    "\n",
    "### Question Answering on Full Body Text\n",
    "Question Answering is often not scalable due to the slow inference terms and the sheer number of text available in the full body. However, given if the BM25 searching algorithm can roughly find related content relative to the query, we can in the future go a step further and generate questions based on the query to extract the exact piece of information.\n",
    "\n",
    "### Summarization for Auto Report Generation\n",
    "Often times multiple pieces of text in the results are outputted based on competing similarity scores. The manual review will need to compare the multiple outputs and choose which ones that actually pertain to answering the question well. In the future, we could use a series of deep learning summarization or sentence embedding models to summarize all the results and cluster them based on their similarity. In this way, we can lesson the reading time for reviewers, and even work toward auto-report generation capabilities if paired with question answering capabilities."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Highlighted Results\n",
    "Here, we present our highlighted findings. These highlights were found by using our approach to query on multiple questions pertaining to many aspects of the task. This gets us 80% to the answer. Then, we finish the last 20% by doing a careful human review of the results and compiling them in a summary.\n",
    "\n",
    "## TASK: Effectiveness of drugs being developed and tried to treat COVID-19 patients\n",
    "**Clinical/Observational Studies**\n",
    "* Lopinavir and Ritonavir seem to be common antiviral agents against COVID-19. Studies often compare new drug effectiveness against Lopinavir and Ritonavir\n",
    "* Favipriavir was shown to be more effective than Lopinavir/Ritonavir control arms. In a separate study, it was also shown to be more effective than arbidol control arm\n",
    "* Hydroxychloroquine was shown to be effective for recoverying of pneumonia effects of COVID-19\n",
    "* Azithromycin added to Hydroxychloroquine was shown to be significantly more efficient for virus elimination, possibly because azithromycin was shown to have similar effects as hydroxychloroquine \n",
    "* Danoprevir boosted by ritonavir was shown to be safe and well tolerated in all patients\n",
    "* Early and short doses of a corticosteroid called methylprednisolone was shown to be effective in treatment COVID-19\n",
    "\n",
    "**Notes based on Review Articles**\n",
    "* There have been a number of reports stating that non-steroidal anti-inflammatory drugs (NSAIDs) and corticosteroids may exacerbate symptoms in COVID-19 patients. Proper use of low-dose corticosteroids may bring survival advantages for critically ill patients, but this treatment should be strictly performed.\n",
    "* Although SARS-CoV-2 replication is not entirely suppressed by interferons, viral titers are decreased by several orders of magnitude. It may be useful in the early stages of infection\n",
    "\n",
    "## TASK: Capabilities to discover a therapeutic (not vaccine) for the disease, and clinical effectiveness studies to discover therapeutics, to include antiviral agents.\n",
    "**In Vitro Studies**\n",
    "* Nelfinavir acts as an HIV Protease Inibitor\n",
    "* Azithromycin and Ciprofloxacin have chloroquine effects and may act as alternatives to hydroxychloroquine/chloroquine\n",
    "* Sofosbuvir, Tenofovir, and Alovudine are polymerases that block Sars-Cov-2 incorporation via RdRp\n",
    "* Tenofovir and Emtricitabine terminates SARS-CoV-2 RdRp catalyzed reaction and can act as preventative treatments (PreP)\n",
    "* Terfiflunomide and Leflunomide were shown to have solid antiviral reduction compared to favipiravir, a drug that is already undegoing clinical trials\n",
    "* Darunavir was shown to have no activity against SARS-COV-2 during In Vitro studies\n",
    "\n",
    "**Simulations and Modeling**\n",
    "* Atazanavir, Efavirenz, Dolutegravir, and Saquinavir were shown to be potential candidates of treating COVID-19 based on simulations and modeling\n",
    "\n",
    "**Notes based on Review Articles**\n",
    "* Niclosamide was able to inhibit SARS-CoV replication and totally abolished viral antigen synthesis at a concentration of 1.56 μM\n",
    "* Tocilizumab is a blocker of IL-6R, which can effectively block IL-6 signal transduction pathway\n",
    "\n",
    "## TASK: Assays to evaluate vaccine immune response and process development for vaccines, alongside suitable animal models\n",
    "In querying the CORD-19 corpus for diagnostic assays to evaluate immune responses to COVID-19, it is apparent that there are several assays that are currently being developed or ameliorated. Among these are the gold standards such as ELISA and PCR which have been leveraged both alone, and as per the results below, in combination with other antibody testing methodologies such as IgG/IgM to gain more confidence in CoV-2 infection diagnosis. Among the results that were returned based on the query, there were few assays that seem to be novel in their nature, such as sereological colirometric assays and fluorescence immunochromatographic tests. \n",
    "\n",
    "The results summarized below were manually curated from over 100 hits for the search query used. The curation mainly took into consideration what virus the assay was addressing, as several hits were referencing viruses not in the CoV family. Additional refinement of these results will include queries to better assess the suitable animal models as well as expand to gain insight into assays from a process development standpoint as opposed to simply a diagnostic one. \n",
    "### ELISA\n",
    "* A newly-developed ELISA assay for IgM and IgG antibodies against N protein of SARS-CoV-2 were used to screen the serums of admitted hospital patients with confirmed or suspected SARS-CoV-2 infection. Of the 238 patients, 194 (81.5%) were detected to be antibody (IgM and/or IgG) positive, which was significantly higher than the positive rate of viral RNA (64.3%). There was no difference in the positive rate of antibody between the confirmed patients (83.0%, 127/153) and the suspected patients (78.8%, 67/85) whose nucleic acid tests were negative.\n",
    "\n",
    "### IgG/IgM combined test\n",
    "* The sensitivity and specificity of this ease-of-use IgG/IgM combined test kit were adequate, plus short turnaround time, no specific requirements for additional equipment or skilled technicians, all of these collectively contributed to its competence for mass testing. At the current stage, it cannot take the place of SARA-CoV-2 nucleic acid RT-PCR, but can be served as a complementary option for RT-PCR. The combination of RT-PCR and IgG-IgM combined test kit could provide further insight into SARS-CoV-2 infection diagnosis.\n",
    "\n",
    "### Serological assay\n",
    "* Because most patients have rising antibody titres 10 days after symptom onset, collection of serial serum samples in the convalescent phase would be more useful. Serum IgG amounts can rise at the same time or earlier than those of IgM against SARS-CoV-2. Posterior oropharyngeal saliva samples are a non-invasive specimen more acceptable to patients and health-care workers. Unlike severe acute respiratory syndrome, patients with COVID-19 had the highest viral load near presentation, which could account for the fast-spreading nature of this epidemic. This finding emphasises the importance of stringent infection control and early use of potent antiviral agents, alone or in combination, for high-risk individuals. Serological assay can complement RT-qPCR for diagnosis.\n",
    "\n",
    "### Antibodies assays\n",
    "* Combined use of antibodies assay and qRT-PCR at the same time was able to improve the sensitivities of pathogenic-diagnosis, especially for the throat swabs group at the later stage of illness. Moreover, most of these cases with undetectable viral RNA in throat swabs specimens at the early stage of illness were able to be IgM/IgG seropositive after 7 days.\n",
    "\n",
    "### Gold immunochromatography assay\n",
    "* The colloidal gold immunochromatography assay (GICA) is a rapid diagnostic tool for novel coronavirus disease 2019 (COVID-19) infections. However, with significant numbers of false negatives, improvements to GICA are needed.\n",
    "\n",
    "### Reverse transcription loop-mediated isothermal amplification (RT-LAMP) assay\n",
    "* This assay detected SARS-CoV-2 in the mean (±SD) time of 26.28 ± 4.48 min and the results can be identified with visual observation. \n",
    "\n",
    "### dPCR assays\n",
    "* dPCR could be a confirmatory method for suspected patients diagnosed by RT-qPCR. Furthermore, dPCR is more sensitive and suitable for low virus load specimens from the both patients under isolation and those under observation who may not be exhibiting clinical symptoms. \n",
    "* Another study showed the overall accuracy of dPCR for clinical detection was 96.3%. dPCR was shown to be powerful in detecting asymptomatic patients and suspected patients. Digital PCR is capable of checking the negative results caused by insufficient sample loading by quantifying internal reference gene from human RNA in the PCR reactions. Multi-channel fluorescence dPCR system (FAM/HEX/CY5/ROX) is able to detect more target genes in a single multiplex assay, providing quantitative count of viral load in specimens, which is a powerful tool for monitoring COVID-19 treatment.\n",
    "\n",
    "### Novel luciferase immunosorbent assays (LISA)\n",
    "* The S1-, RBD-, and NP-LISAs were more sensitive than the NTD- and S2-LISAs for the detection of anti-MERS-CoV IgG. These LISAs proved their applicability and reliability for detecting anti-MERS-CoV IgG in samples from camels, monkeys, and mice, among which the RBD-LISA exhibited excellent performance.\"\n",
    "\n",
    "\n",
    "### Rapid serological colorimetric test\n",
    "* Rapid serological test showed a sensitivity of 30% and a specificity of 89% with respect to the standard assay but, interestingly, these performances improve after 8 days of symptoms appearance. After 10 days of symptoms the predictive value of rapid serological test is higher than that of standard assay. It may detect previous exposure to the virus in currently healthy persons.\n",
    "\n",
    "### Fluorescence immunochromatographic assay\n",
    "* Fluorescence immunochromatographic assay experiments were done for detecting nucleocapsid protein of SARS-CoV-2 in nasopharyngeal swab samples and urine within 10 minutes, and evaluated its significance in diagnosis of COVID-19. We measured nucleocapsid protein in nasopharyngeal swab samples in parallel with the nucleic acid test. 100% of nucleocapsid protein positive and negative participants accord with nucleic acid test for same samples.\n",
    "\n",
    "### RT-qPCR\n",
    "* Using flu and RSV clinical specimens,researchers have collected evidence that the RT-qPCR assay can be performed directly on patient sample material from a nasal swab immersed in virus transport medium (VTM) without an RNA extraction step. This approach was used to test for the direct detection of SARS-CoV-2 reference materials spiked in VTM. The data, while preliminary, suggest that using a few microliters of these untreated samples still can lead to sensitive test results. If RNA extraction steps can be omitted without significantly affecting clinical sensitivity, the turn-around time of COVID-19 tests and the backlog we currently experience can be reduced drastically.\n",
    "\n",
    "### Novel in vivo cell-based assay\n",
    "* Reseachers developed a novel in vivo cell-based assay for examining this interaction between the N-protein and packaging signal RNA for SARS-CoV, as well as other viruses within the coronaviridae family. The N-protein specifically recognizes the SARS-CoV packaging signal with greater affinity compared to signals from other coronaviruses or non-coronavirus species. These results describe, for the first time, in vivo evidence for an interaction between the SARS-CoV N-protein and its packaging signal RNA, and demonstrate the feasibility of using this cell-based assay to further probe viral RNA-protein interactions in future studies.\n",
    "\n",
    "## TASK: Efforts to develop prophylaxis clinical studies and prioritize in healthcare workers\n",
    "\n",
    "Prophylaxis describes the efforts and measures to prevent infections and diseases. We summarize some of our findings on these efforts to develop prophylaxis clinical studies by categorizing them into general findings, findings related to α1-AR antagonists, and findings related to risk varied by prognostic factors.\n",
    "\n",
    "### General findings\n",
    "\n",
    "* Risk-adapted treatment strategy may be a useful tool for the treatment of COVID-19 patients. This strategy is associated with significant clinical manifestations alleviation and clinical imaging recovery.\n",
    "* Harmonization of clinically heterogeneous endpoints within and between trials can lead to faster decision making and better management of COVID-19. \n",
    "* Early detection of elevations in serum CRP, combined with a clinical COVID-19 symptom presentation may be used as a surrogate marker for presence and severity of disease.\n",
    "* There are multiple parameters of the clinical course and management of the COVID-19 that need optimization. A hindrance to this development is the vast amount of misinformation present due to scarcely sourced manuscript preprints and social media.\n",
    "* Emphasize evidence-based medicine to evaluate the frequency of presentation of various symptoms to create a stratification system of the most important epidemiological risk factors for COVID-19.\n",
    "* Vitamin C (L-ascorbic acid) has a pleiotropic physiological role, but there is evidence supporting the protective effect of high dose intravenous vitamin C (HDIVC) during sepsis induced ARDS.\n",
    "* Epigenetic control of the ACE2 gene might be a target for prevention and therapy in COVID-19. \n",
    "\n",
    "### α1-AR antagonists\n",
    "Preliminary findings offer a rationale for studying α1-AR antagonists in the prophylaxis of patients with COVID-19 cytokine storm syndrome (CSS) and acute respiratory distress syndrome (ARDS).\n",
    "* Mortality of COVID-19 seems driven by acute respiratory distress syndrome (ARDS)\n",
    "* Emerging evidence suggests that a subset of COVID-19 is characterized by the development of a CSS. \n",
    "* Pre-clinical mouse data suggests that α1-AR antagonists may be a candidate for the treatment of COVID-19.\n",
    "* Using the Truven Health MarketScan Research DataBase, male men who were prescribed α1-AR antagonists in the previous year had lower odds of the composite of need for invasive mechanism ventilation and mortality compared to non-users (AOR 0.80, 95% CI 0.69-0.94, p=0.008) \n",
    "\n",
    "### Relative Risk of COVID-19 for Patients Varies by Prognostic Factors\n",
    "COVID-19 patient outcomes vary by patient characteristics and are important considerations for COVID-19 prophylaxis. Potential important factors include interleukin-6, B lymphocyte proportion, lactate, and CD8+ T cells. \n",
    "* Compared with patients without pneumonia, those with pneumonia were 15 years older and had a higher rate of hypertension, higher frequencies of having a fever and cough, and higher levels of interleukin-6, B lymphocyte proportion, and low account of CD8+ T cells. \n",
    "* Multivariate Cox regression analysis indicated that circulating interleukin-6 and lactate independently predicted COVID-19 progression, with a hazard ratio (95%CI) of 1.052 (1.000-1.107) and 1.082 (1.013-1.155), respectively. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "# Prerequisites\n",
    "\n",
    "We use the following main libraries:\n",
    "* **Transformers** - The wildly popular Transformers library from HuggingFace provides use with the ability to download pretrained model and use them out the box for NLP tasks such as sentence similarity\n",
    "* **Sentence Transformers** - An extension of the Transformer library with the specific goal of training and providing models for sentence similarity tasks\n",
    "* **NTLK** - A popular NLP and text processing library, which we use to process, clean, and tokenize the text data (For Stage 2 results)\n",
    "* **rank_bm25** - A library to perform the BM25 ranking algorithm\n",
    "* **langdetect** - A library to help filter out non-english articles\n",
    "* **networkx** - NetworkX is a Python package for the creation, manipulation, and study of the structure, dynamics, and functions of complex networks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5"
   },
   "outputs": [],
   "source": [
    "%%capture\n",
    "!pip install sentence-transformers\n",
    "!pip install transformers --upgrade\n",
    "!pip install langdetect\n",
    "!pip install rank_bm25\n",
    "!pip install networkx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to /usr/share/nltk_data...\n",
      "[nltk_data]   Package punkt is already up-to-date!\n",
      "[nltk_data] Downloading package stopwords to /usr/share/nltk_data...\n",
      "[nltk_data]   Package stopwords is already up-to-date!\n"
     ]
    }
   ],
   "source": [
    "# This Python 3 environment comes with many helpful analytics libraries installed\n",
    "# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n",
    "# For example, here's several helpful packages to load in \n",
    "\n",
    "import numpy as np # linear algebra\n",
    "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
    "from pathlib import Path, PurePath #Easy path directory\n",
    "import json # Reads json\n",
    "import os\n",
    "import multiprocessing as mp\n",
    "\n",
    "import transformers # NLP task pipeline\n",
    "from transformers import pipeline, AutoModelWithLMHead, AutoModelForQuestionAnswering, AutoTokenizer, AutoModel, AutoTokenizer, AlbertConfig, AlbertForQuestionAnswering, AlbertTokenizer # For downloading pretrained models\n",
    "from sentence_transformers import SentenceTransformer, models # For sentence embeddings trained on semantic similarity\n",
    "\n",
    "from langdetect import detect\n",
    "import re\n",
    "\n",
    "import scipy\n",
    "import statistics \n",
    "from rank_bm25 import BM25Okapi\n",
    "import nltk\n",
    "from nltk.corpus import stopwords\n",
    "\n",
    "# For network analysis on citations\n",
    "import networkx as nx\n",
    "from itertools import chain\n",
    "from tqdm.auto import tqdm\n",
    "tqdm.pandas()\n",
    "\n",
    "nltk.download(\"punkt\")\n",
    "nltk.download('stopwords')\n",
    "pd.set_option('display.max_columns', None)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating a List of Keywords\n",
    "\n",
    "We leverage prior and continuous knowledge about vaccines to do an initial keyword filter of the documents. These keywords containg words pertaining to treatments and antiviral agents that may be vaccine candidates. Additionally, other vaccine related keywords such as \"assay\", \"phrophylaxis\", and \"antibody dependent enhancement\" were included. During our review, if we noticed new keywords that may be helpful in doing an initial filter, then we update the keyword list.\n",
    "\n",
    "Initially, most of these keywords were obtained via a continuously updated report from the National University of Singapore (https://sph.nus.edu.sg/covid-19/research/).\n",
    "\n",
    "**NOTE: We may update this step to instead leverage recent release of specter embeddings in the official CORD-19 database to do our initial filtering**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "keywords = [\n",
    "    'remdesivir',\n",
    "    'azithromycin',\n",
    "    'ciprofloxacin',\n",
    "    'lopinavir',\n",
    "    'ritonavir',\n",
    "    'interferon',\n",
    "    'chloroquine',\n",
    "    'hydroxychloroquine',\n",
    "    'darunavir',\n",
    "    'cobicistat',\n",
    "    'emtricitabine',\n",
    "    'nelfinavir',\n",
    "    'tenofovir',\n",
    "    'saquinavir',\n",
    "    'azuvudine',\n",
    "    'favipiravir',\n",
    "    'umifenovir',\n",
    "    'oseltamivir',\n",
    "    'baloxavir',\n",
    "    'methylprednisolone',\n",
    "    'ribvarin',\n",
    "    'sofosbuvir',\n",
    "    'beclabuvir',\n",
    "    'galidesivir',\n",
    "    'simeprevir',\n",
    "    'nitazoxanide',\n",
    "    'niclosamide',\n",
    "    'naproxen',\n",
    "    'clarithromycin',\n",
    "    'minocyclinethat',\n",
    "    'human monoclonal antibody',\n",
    "    'tocilizumab',\n",
    "    'sarilumab',\n",
    "    'leronlimab',\n",
    "    'foralumab',\n",
    "    'camrelizumab',\n",
    "    'ifx-1',\n",
    "    'ifx',\n",
    "    'arbidol',\n",
    "    'fingolimod',\n",
    "    'brilacidin',\n",
    "    'sirolimus',\n",
    "    'danoprevir',\n",
    "    'rintatolimod',\n",
    "    'cynk-001',\n",
    "    'cynk',\n",
    "    'tmprss2',\n",
    "    'jak',\n",
    "    'zinc',\n",
    "    'quercetin',\n",
    "    'convalescent plasma',\n",
    "    'nanoviricide',\n",
    "    'corticosteroids',\n",
    "    'bevacizumab',\n",
    "    'bxt-25',\n",
    "    'bxt',\n",
    "    'angiotension',\n",
    "    'rhace2',\n",
    "    'pirfenidone',\n",
    "    'thalidomide',\n",
    "    'brohexine hydrochloride',\n",
    "    'dehydroandrographolide succinate',\n",
    "    'antibody dependent enhancement',\n",
    "    'antibody-dependent enhancement',\n",
    "    ' ade ',\n",
    "    'prophylaxis',\n",
    "    'prophylactic',\n",
    "    'vaccine',\n",
    "    'assay',\n",
    "    'elisa',\n",
    "    'th1',\n",
    "    'th2',\n",
    "    'elispot',\n",
    "    'cytometry',\n",
    "    'ctc'\n",
    "]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create Helper Functions\n",
    "\n",
    "We define our functions that aid in data processing, text cleaning, and BM25 search \n",
    "\n",
    "* **get_deeper_context()** - This function performs the BM25 algorithm on the full text against a query of tokenized words and returns the relevant sections of text that meets a threshold criteria\n",
    "* **get_sorted_indices()** - This function takes the BM25 scores, finds all the BM25 scores that meet a threshold crtieria, and returns the indices that map to the section of the full text\n",
    "* **strip_characters(), clean(), tokenize(), preprocess()** - These functions all roll up to preprocess() in order to remove unnecessary punctuation, lower case the text, tokenize the text, and remove stopwords.\n",
    "* **get_keywords()** - This function finds all the keywords in the title and the full text and returns them.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Functions\n",
    "\n",
    "# This is a list of English stopwords that do not contribute to useful information and helps to generate useful tokens for searching\n",
    "english_stopwords = list(set(stopwords.words('english')))\n",
    "\n",
    "# This function performs the BM25 algorithm on the full text against a query of tokenized words and returns the relevant sections of text that meets a threshold criteria\n",
    "def get_deeper_context(row):\n",
    "    # Grab variables from the input data row\n",
    "    sha = row['sha']\n",
    "    results = row['result']\n",
    "    \n",
    "    # Sometimes outputs will contain multiple results, that is separated by the \"\\n\\n\" delimiter. So we make sure to split the results and process them separately.\n",
    "    results = results.split(\"\\n\\n\")\n",
    "    \n",
    "    # We take each results and preprocess them so that useful words are tokenized. These will help perform our BM25 search\n",
    "    tokenized_results = [preprocess(result) for result in results]\n",
    "    \n",
    "    # Create variables to be stored later\n",
    "    paragraphs = []\n",
    "    tokenized_paragraphs = []\n",
    "    candidate_paragraphs = []\n",
    "    candidates = ''\n",
    "    \n",
    "    # Scanning through data to read the relevant file based on the Sha and acquire the full text\n",
    "    for path in Path(directory).rglob('*.json'):\n",
    "        if sha in path.name:\n",
    "            data = json.loads(open(path.absolute()).read())\n",
    "            # Grabs the full body text (which is a list of paragraphs)\n",
    "            body_text = data['body_text']\n",
    "\n",
    "            # Loops through each paragraph and appends them into a list\n",
    "            for i, paragraph in enumerate(body_text):\n",
    "                text = \" \".join(paragraph['text'].split()).replace(\" ,\",\",\")\n",
    "                paragraphs.append(text)\n",
    "                tokenized_paragraphs.append(preprocess(text))\n",
    "            break\n",
    "\n",
    "    try: \n",
    "        # Feed the paragraphs into the BM25 API\n",
    "        bm25 = BM25Okapi(tokenized_paragraphs)\n",
    "        # Loop through the tokenized results and get BM25 scores to see which paragraphs were most relevant to the query.\n",
    "        for tokenized_result in tokenized_results:\n",
    "            doc_scores = bm25.get_scores(tokenized_result) # BM25 scores\n",
    "            candidate_paragraphs.append(get_sorted_indices(doc_scores)) # Saving the indices of relevant paragraphs into a list\n",
    "\n",
    "        # Deduping and sorting the list by index number\n",
    "        candidate_paragraphs = [item for sublist in candidate_paragraphs for item in sublist]\n",
    "        candidate_paragraphs = list(set(candidate_paragraphs))\n",
    "        candidate_paragraphs.sort()\n",
    "\n",
    "        # Combine the relevant paragraphs into a single string.\n",
    "        for index in candidate_paragraphs:\n",
    "            candidates = candidates+\"Paragraph: \"+str(index)+\"\\n\"\n",
    "            candidates = candidates+paragraphs[index]+\"\\n \\n\"\n",
    "    except:\n",
    "        candidates = \"NA\"\n",
    "        \n",
    "    # Saving the results into their own column\n",
    "    row['context'] = candidates\n",
    "\n",
    "    return row\n",
    "\n",
    "# This function takes the BM25 scores, produces the mean and standard deviation of the score, and outputs the relevant scores only if it is 1.5 standard deviations way from the mean\n",
    "def get_sorted_indices(l):\n",
    "    std = statistics.stdev(l) # Standard deviation\n",
    "    mean = statistics.mean(l) # Mean\n",
    "    threshold = mean+(std*1.5) # 1.5 standard deviation treshold\n",
    "    max_score = max(l) # Max scire\n",
    "    \n",
    "    indices = []\n",
    "\n",
    "    # Looping through the scores and applying the threshold\n",
    "    for index, score in enumerate(l):\n",
    "        if score >= threshold:\n",
    "            indices.append(index)\n",
    "    \n",
    "    indices.sort()\n",
    "    \n",
    "    return indices\n",
    "\n",
    "# This function strips characters such as apostrophes, etc.\n",
    "def strip_characters(text):\n",
    "    t = re.sub('\\(|\\)|:|,|;|\\.|’|”|“|\\?|%|>|<', '', text)\n",
    "    t = re.sub('/', ' ', t)\n",
    "    t = t.replace(\"'\",'')\n",
    "    return t\n",
    "\n",
    "# This function calls the strip_characters() function and also lowercases all text\n",
    "def clean(text):\n",
    "    t = text.lower()\n",
    "    t = strip_characters(t)\n",
    "    return t\n",
    "\n",
    "# This function takes a text and cleans and tokenizes it, while removing the stopwords.\n",
    "def tokenize(text):\n",
    "    words = nltk.word_tokenize(text)\n",
    "    return list(set([word for word in words \n",
    "                     if len(word) > 1\n",
    "                     and not word in english_stopwords\n",
    "                     and not (word.isnumeric() and len(word) is not 4)\n",
    "                     and (not word.isnumeric() or word.isalpha())] ))\n",
    "\n",
    "# This is the wrapper function that incorporates the previous functions to clean and tokenize text\n",
    "def preprocess(text):\n",
    "    t = clean(text)\n",
    "    tokens = tokenize(t)\n",
    "    return tokens\n",
    "\n",
    "\n",
    "# This function takes in a piece of text and extracts out the pre-specified keywords found in the text.\n",
    "def get_keywords(row):\n",
    "    found_keywords = []\n",
    "        \n",
    "    # Looping through each row and column of the data\n",
    "    for col in row.iteritems():\n",
    "        # Checking if the column is the title or abstract\n",
    "        if (\"title\" in col[0]) | (\"abstract\" in col[0]):\n",
    "            text = col[1].lower() # lowercass\n",
    "            text = \" \".join(text.split()) # removes useless whitespace\n",
    "            \n",
    "            # Loops through the known keywords and detects if it is found in the text\n",
    "            for keyword in keywords:\n",
    "                if keyword in text:\n",
    "                    found_keywords.append(keyword)\n",
    "            \n",
    "    # De-duplicates the keywords found\n",
    "    found_keywords = set(found_keywords)\n",
    "    \n",
    "    # If no keywords found, return NA\n",
    "    if len(found_keywords) == 0:\n",
    "        row['keywords'] = 'NA'\n",
    "    else:\n",
    "        row['keywords'] = \"; \".join(found_keywords)\n",
    "        \n",
    "    return row\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Initial Filtering of Articles via Keyword Search\n",
    "\n",
    "We perform the following filtering, cleaning and processing steps:\n",
    "1. Read the metadata CSV\n",
    "2. Filter for articles newer than 2019-11-01 (COVID-19 date)\n",
    "3. Filter articles that don't contain full text\n",
    "4. Filter articles that don't contain titles or abstracts\n",
    "5. We found two articles that broke many formattings and encodings of our report, and filtered them out\n",
    "6. Clean the text data by removing unnecessary white space\n",
    "7. Filter for titles or abstracts that contain the pre-specified keyword terms and for each article, save the relevant keywords found\n",
    "8. Drop non-english articles"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of articles\n",
      "(51078, 18)\n",
      "Filter for articles after date: 2019-11-01\n",
      "(6029, 18)\n",
      "Filter for full text\n",
      "(4312, 18)\n",
      "Filter for non-empty title and abstract\n",
      "(3457, 18)\n",
      "Filter for terms relating to treatments\n",
      "(820, 19)\n",
      "Filter for English articles\n",
      "(796, 19)\n"
     ]
    }
   ],
   "source": [
    "# Read the metadata\n",
    "all_data = []\n",
    "directory = '/kaggle/input/CORD-19-research-challenge/'\n",
    "metadata = pd.read_csv(directory+\"metadata.csv\")\n",
    "print(\"Total number of articles\")\n",
    "print(metadata.shape)\n",
    "\n",
    "# Filter for articles newer than 2019-11-01\n",
    "date = '2019-11-01'\n",
    "metadata = metadata[metadata['publish_time'] >= date]\n",
    "print(\"Filter for articles after date: \"+date)\n",
    "print(metadata.shape)\n",
    "\n",
    "# Filter articles that don't have full text\n",
    "metadata = metadata[metadata['has_pdf_parse']]\n",
    "print(\"Filter for full text\")\n",
    "print(metadata.shape)\n",
    "\n",
    "# Filter articles that don't have title or abstract\n",
    "metadata = metadata[metadata['title'].str.len() > 0]\n",
    "metadata = metadata[metadata['abstract'].str.len() > 0]\n",
    "print(\"Filter for non-empty title and abstract\")\n",
    "print(metadata.shape)\n",
    "\n",
    "# Filter certain articles that seem to be of bad quality and messes up export formating\n",
    "metadata = metadata[metadata['sha'] != 'a5293bb4f17ad25a72133cdd9eee8748dd6a4b8d']\n",
    "metadata = metadata[metadata['sha'] != 'b30770ae30b35cdfaf0a173863e74e93edbb0329']\n",
    "\n",
    "# Clean text data\n",
    "metadata['title'] = metadata['title'].apply(lambda x: \" \".join(x.split()))\n",
    "metadata['abstract'] = metadata['abstract'].apply(lambda x: \" \".join(x.split()))\n",
    "\n",
    "# Filter for titles and abstracts that have mention of one of the keywords\n",
    "keyword_query = \"|\".join(keywords)\n",
    "metadata = metadata[metadata['title'].str.contains(keyword_query, flags=re.IGNORECASE, regex=True) | \n",
    "                    metadata['abstract'].str.contains(keyword_query, flags=re.IGNORECASE, regex=True)]\n",
    "\n",
    "# Finds the keywords found in each article and makes a column out of it\n",
    "metadata = metadata.apply(get_keywords, axis=1)\n",
    "print(\"Filter for terms relating to treatments\")\n",
    "print(metadata.shape)\n",
    "\n",
    "# Drop non-english articles\n",
    "for index, row in metadata.iterrows():\n",
    "    title = row['title']\n",
    "    lang = detect(title)\n",
    "    if lang != 'en':\n",
    "        metadata.drop(index, inplace=True)\n",
    "\n",
    "print(\"Filter for English articles\")\n",
    "print(metadata.shape)\n",
    "\n",
    "# Resets the index\n",
    "metadata = metadata.reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting popularity of article via Citations\n",
    "Leveraging the amount of times an article was cited is a useful way of getting a rough ranking on the articles for manual review.\n",
    "\n",
    "## Approach\n",
    "To get these numbers and maintain a result that doesn't blow up our RAM allocation we built a simple strategy to get citations for our relevant articles.\n",
    "\n",
    "1. To maintain memory efficiency, process the json articles in chunks. At any given time we won't have more than :chunk_size: articles read into memory as dictionaries.\n",
    "2. Using as many processes as we can, search each articles json dict for citations.\n",
    "3. With the relevant articles in hand from the above process, extract only those citations pointing to an article in our list. We could speed up this check by including the dates that our relevant articles were published and eliminating those from the checking set that were published after the article we're extracting citations from. If publishing dates are distributed normally this could speed up the process significantly.\n",
    "4. Finally, combine the sharded results and build a directed graph using networkx.\n",
    "5. With the relevant citation graph in hand, produce a function to lookup the number of times an article was cited using its title.\n",
    "\n",
    "You'll notice that even with this strategy, the graph takes up a significant amount of memory. If we want to build a more robust knowledge graph in the future we need a way around this. Our team explored using sqlite but abandoned the approach due to time. It may be something we explore more for submission 2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2cf136ea03bb4cc4b70c8c2f4d5eda14",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Function progress on chunk', max=1.0, style=ProgressStyle…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8d2b229e937940b3b6120b84142efa69",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Task progress bar', max=5000.0, style=ProgressStyle(descr…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ed1dbc3c552a44a4a3cd743c8cdd8dda",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Chunk progress', max=12.0, style=ProgressStyle(descriptio…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done\n"
     ]
    }
   ],
   "source": [
    "def open_json(file_path):\n",
    "    \"\"\"\n",
    "    Helper function to open json file\n",
    "    \"\"\"\n",
    "    with open(file_path, 'r') as f:\n",
    "        json_dict = json.load(f)\n",
    "    return json_dict\n",
    "\n",
    "\n",
    "def json_path_generator(data_path=os.path.abspath(\"/kaggle/input\"), limit=None):\n",
    "    \"\"\"\n",
    "    Helper function to get all the paths of json files in the input directory.\n",
    "    \"\"\"\n",
    "    return_files = []\n",
    "    for dirname, _, filenames in os.walk(data_path):\n",
    "        for filename in filenames:\n",
    "            if filename[-5:] == \".json\":\n",
    "                return_files.append(os.path.join(dirname, filename))\n",
    "                if limit is not None and type(limit) == int:\n",
    "                    if len(return_files) >= limit:\n",
    "                        return return_files\n",
    "    return return_files\n",
    "\n",
    "\n",
    "def get_json_dicts(paths, progress_bar=None):\n",
    "    \"\"\"\n",
    "    Helper function to open a list of paths as json dicts. Careful about memory usage here.\n",
    "    Optionally takes a tqdm bar as input to show progress of loading.\n",
    "    \"\"\"\n",
    "    json_dicts = []\n",
    "    # (I) Max of 2 or number of cpus minus 1, then min of (I) or the number of paths. If limit is used and its small,\n",
    "    # avoid excessive pool sizes.\n",
    "    process_num = min(max(2, os.cpu_count()), len(paths))\n",
    "    with mp.Pool(process_num) as pool:\n",
    "        for result in pool.imap_unordered(open_json, paths):\n",
    "            json_dicts.append(result)\n",
    "            if progress_bar is not None:\n",
    "                progress_bar.update(1)\n",
    "\n",
    "    return json_dicts\n",
    "\n",
    "\n",
    "# Get articles cited by our articles\n",
    "def get_article_citations(article_dict, select_articles=None):\n",
    "    \"\"\"\n",
    "    Function to extract set of (from, to) citation edges from articles.\n",
    "    :select_articles: A set of articles to check so that only citations recorded are those\n",
    "                      where the 'to' node is in the set. Used in this block to limit our citation graph\n",
    "                      to only those citations of the articles we've deemed relevant.\n",
    "    \"\"\"\n",
    "    article_title = article_dict.get(\"metadata\", {}).get(\"title\", \"\")\n",
    "    article_citations = set()\n",
    "    # Get citations and their ids\n",
    "    bib_entries = article_dict.get(\"bib_entries\", {})\n",
    "    for entry in bib_entries:\n",
    "        entry_dict = bib_entries.get(entry, {})\n",
    "        entry_title = entry_dict.get(\"title\", \"\")\n",
    "        if select_articles is not None and type(select_articles) == set:\n",
    "            # Check that article being cited is in our list of articles to look for\n",
    "            if entry_title in select_articles:\n",
    "                article_citations.add((article_title, entry_title))\n",
    "        else:\n",
    "            article_citations.add((article_title, entry_title))\n",
    "    return list(article_citations)\n",
    "\n",
    "\n",
    "def get_article_citations_meta(arg_list):\n",
    "    \"\"\"\n",
    "    Meta function of get_article_citations so that we can parallelize with multiple arguments.\n",
    "    First argument is a single arguments dictionary.\n",
    "    Second argument is the optional set of article titles to limit citations.\n",
    "    \"\"\"\n",
    "    return get_article_citations(arg_list[0], arg_list[1])\n",
    "\n",
    "\n",
    "def divide_chunks(l, n): \n",
    "    \"\"\"\n",
    "    Function shameless taken from stackoverflow to split a list (l) into sublists of size n.\n",
    "    \"\"\"\n",
    "    # looping till length l \n",
    "    for i in range(0, len(l), n):  \n",
    "        yield l[i:i + n]\n",
    "\n",
    "\n",
    "def process_all_articles(tasks: list, function, progress_bar=None):\n",
    "    \"\"\"\n",
    "    A wrapper function to call a single function over a set of tasks with a multiprocessing pool.\n",
    "    \"\"\"\n",
    "    results = []\n",
    "    process_num = min(max(2, os.cpu_count()), len(tasks))\n",
    "    with mp.Pool(process_num) as pool:\n",
    "        for result in pool.imap_unordered(function, tasks):\n",
    "            results.append(result)\n",
    "            if progress_bar is not None:\n",
    "                progress_bar.update(1)\n",
    "    return results\n",
    "\n",
    "\n",
    "def add_citation_edges(graph: nx.DiGraph, edges: list):\n",
    "    \"\"\"\n",
    "    Function to record edges in the networkx DiGraph object.\n",
    "    \"\"\"\n",
    "    all_citation_edges = list(set(chain.from_iterable(edges)))\n",
    "    graph.add_edges_from(all_citation_edges)\n",
    "\n",
    "\n",
    "def build_citation_graph(filter_articles=None, paths=None, limit=None, chunk_size=5000):\n",
    "    \"\"\"\n",
    "    Function to build citation graph from beginning to end.\n",
    "    :filter_articles: Set of article titles to limit citations to. \n",
    "                      Citations will only be recorded if the 'to' article's title is in the set.\n",
    "    :paths: Subset of paths to operate over. If None operate over all.\n",
    "    :limit: Can limit number of paths to this given number.\n",
    "    :chunk_size: Number of article dicts to hold in memory at once to process. 5000 seems like a decent choice.\n",
    "    \"\"\"\n",
    "    # Get paths of articles\n",
    "    if paths is None:\n",
    "        paths = json_path_generator(limit=limit)\n",
    "    # Split the paths into chunks for memory efficiency\n",
    "    chunked_paths = list(divide_chunks(paths, chunk_size))\n",
    "    # Build the citation graph\n",
    "    graph = nx.DiGraph()\n",
    "    functions = [\n",
    "        get_article_citations_meta\n",
    "    ]\n",
    "    function_progress_bar = tqdm(total=len(functions), leave=False, position=1, desc=\"Function progress on chunk\")\n",
    "    if limit is None:\n",
    "        task_num = chunk_size\n",
    "    else:\n",
    "        task_num = min(chunk_size, limit)\n",
    "    task_progress_bar = tqdm(total=task_num, leave=False, position=2, desc=\"Task progress bar\")\n",
    "    all_results = list()\n",
    "    for paths in tqdm(chunked_paths, leave=False, position=0, desc=\"Chunk progress\"):\n",
    "        task_progress_bar.reset(total=len(paths))\n",
    "        path_dicts = get_json_dicts(paths)\n",
    "        tasks = [[x, filter_articles] for x in path_dicts]\n",
    "        function_progress_bar.reset()\n",
    "        for func in functions:\n",
    "            func_name = func.__name__\n",
    "            function_progress_bar.set_description(\"Calling \" + func_name)\n",
    "            results = process_all_articles(tasks, func, task_progress_bar)\n",
    "            # Combine list of sets\n",
    "            result_edges = list(set(chain.from_iterable(results)))\n",
    "            all_results.append(result_edges)\n",
    "            function_progress_bar.update(1)\n",
    "    # Update the graph object\n",
    "    add_citation_edges(graph, all_results)\n",
    "    function_progress_bar.close()\n",
    "    task_progress_bar.close()\n",
    "    print(\"Done\")\n",
    "    return graph\n",
    "\n",
    "\n",
    "relevant_article_citation_graph = build_citation_graph(set(metadata.title.tolist()), chunk_size=5000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>number_citations</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>634</th>\n",
       "      <td>Middle East respiratory syndrome</td>\n",
       "      <td>273</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>660</th>\n",
       "      <td>Detection of 2019 novel coronavirus (2019-nCoV...</td>\n",
       "      <td>64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>66</th>\n",
       "      <td>Comparative therapeutic efficacy of remdesivir...</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>697</th>\n",
       "      <td>SARS-CoV-2 Cell Entry Depends on ACE2 and TMPR...</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>376</th>\n",
       "      <td>The novel coronavirus 2019 (2019-nCoV) uses th...</td>\n",
       "      <td>27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>311</th>\n",
       "      <td>Hydroxychloroquine and azithromycin as a treat...</td>\n",
       "      <td>23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>728</th>\n",
       "      <td>Potent binding of 2019 novel coronavirus spike...</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>713</th>\n",
       "      <td>Prophylactic and therapeutic remdesivir (GS-57...</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>641</th>\n",
       "      <td>Severe acute respiratory syndrome coronavirus ...</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>784</th>\n",
       "      <td>Structure, Function, and Antigenicity of the S...</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                 title  number_citations\n",
       "634                   Middle East respiratory syndrome               273\n",
       "660  Detection of 2019 novel coronavirus (2019-nCoV...                64\n",
       "66   Comparative therapeutic efficacy of remdesivir...                34\n",
       "697  SARS-CoV-2 Cell Entry Depends on ACE2 and TMPR...                31\n",
       "376  The novel coronavirus 2019 (2019-nCoV) uses th...                27\n",
       "311  Hydroxychloroquine and azithromycin as a treat...                23\n",
       "728  Potent binding of 2019 novel coronavirus spike...                20\n",
       "713  Prophylactic and therapeutic remdesivir (GS-57...                19\n",
       "641  Severe acute respiratory syndrome coronavirus ...                15\n",
       "784  Structure, Function, and Antigenicity of the S...                13"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def get_number_citations(article_title: str, citation_graph: nx.DiGraph) -> int:\n",
    "    \"\"\"\n",
    "    Function to get the number of citations received by an article.\n",
    "    :article_title: Title of article to check for numbre of citations received.\n",
    "    :citation_graph: nx.DiGraph instance with edges denoting the number of citations\n",
    "                     built with the orientation (citing article, cited article).\n",
    "    \"\"\"\n",
    "    num_citations = citation_graph.in_degree(article_title)\n",
    "    if type(num_citations) is not int:\n",
    "        return 0\n",
    "    return num_citations\n",
    "\n",
    "# Calculating the number of times an article was cited in each of the articles we filtered for\n",
    "metadata['number_citations'] = metadata.title.apply(lambda title: get_number_citations(title, relevant_article_citation_graph))\n",
    "\n",
    "# Saving the initial filtering in a file\n",
    "metadata.to_pickle('/kaggle/working/metadata.pkl')\n",
    "metadata.to_csv('/kaggle/working/metadata.csv', index=False)\n",
    "\n",
    "# Printing out most popular citations in our filter search\n",
    "metadata.sort_values('number_citations', ascending=False).head(10)[['title', 'number_citations']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Embed Sentences within Abstracts\n",
    "\n",
    "Here we take all the abstracts we found and tokenize them into individual sentences. We also filter out short sentences (sentences less than 6 words) as a cleaning step to filter out and labels.\n",
    "\n",
    "We download a pre-trained BioBERT model from the Transformers library and load the model into the SentenceTransformer class in order to be able to output sentence embeddings of text optimized for semantic similarity comparison.\n",
    "\n",
    "Finally, we take all our sentences and produce sentence embeddings using the pretrained BioBERT model optimized for semantic similarity.\n",
    "\n",
    "**NOTE: Recommend to use GPU to significantly speed up this process.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4d9cb493d84c48f385bfa9c9d3c1c7a8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=1017.0, style=ProgressStyle(description…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1cc062d0ec2d4091b14c1dadb276c06a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=213450.0, style=ProgressStyle(descripti…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4ace4f2901764e2f96307dfc011e9442",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=112.0, style=ProgressStyle(description_…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ea705d2eea004946940324b845b5bef0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=136.0, style=ProgressStyle(description_…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d3020dede0004cceb97602f1c6cd0bd6",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=433288887.0, style=ProgressStyle(descri…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Batches: 100%|██████████| 912/912 [00:26<00:00, 34.64it/s]\n"
     ]
    }
   ],
   "source": [
    "# Create folder to store out BioBERT model\n",
    "if not os.path.exists('/kaggle/working/model'):\n",
    "    os.makedirs('/kaggle/working/model')\n",
    "    \n",
    "# Make dataframe of sentences from abstract\n",
    "sent_dict = {'sha':[],'sentence':[]}\n",
    "\n",
    "# Loop through our filtered list from the metadata\n",
    "for index, row in metadata.iterrows():\n",
    "    sha = row['sha']\n",
    "    abstract = row['abstract']\n",
    "    \n",
    "    # Take the abstract and tokenize them on the sentence level\n",
    "    sentences = nltk.tokenize.sent_tokenize(abstract)\n",
    "    \n",
    "    # Loop through the abstract sentences\n",
    "    for sentence in sentences:\n",
    "        # Make sure sentence is at least 6 words (to filter out useless labelings or headings)\n",
    "        sentence_split = sentence.split()\n",
    "        if len(sentence_split) > 5:\n",
    "            sent_dict['sha'].append(sha)\n",
    "            sent_dict['sentence'].append(sentence)\n",
    "\n",
    "# Convert our list of abstract sentences to a dataframe\n",
    "df_sentences = pd.DataFrame(sent_dict)\n",
    "df_sentences.head()\n",
    "\n",
    "# Download and setup the model\n",
    "tokenizer = AutoTokenizer.from_pretrained(\"gsarti/biobert-nli\")\n",
    "model = AutoModelWithLMHead.from_pretrained(\"gsarti/biobert-nli\")\n",
    "\n",
    "# Initialize and save the model\n",
    "model.save_pretrained(\"/kaggle/working/model\")\n",
    "tokenizer.save_pretrained(\"/kaggle/working/model\")\n",
    "embedding = models.BERT(\"/kaggle/working/model\",max_seq_length=128,do_lower_case=True)\n",
    "pooling_model = models.Pooling(embedding.get_word_embedding_dimension(),\n",
    "                               pooling_mode_mean_tokens=True,\n",
    "                               pooling_mode_cls_token=False,\n",
    "                               pooling_mode_max_tokens=False)\n",
    "\n",
    "model = SentenceTransformer(modules=[embedding, pooling_model])\n",
    "model.save(\"/kaggle/working/model\")\n",
    "encoder = SentenceTransformer(\"/kaggle/working/model\")\n",
    "\n",
    "# Perform the sentence embedding conversion\n",
    "sentences = df_sentences['sentence'].tolist()\n",
    "sentence_embeddings = encoder.encode(sentences)\n",
    "df_sentences['embeddings'] = sentence_embeddings\n",
    "\n",
    "# Save the sentence embeddings dataframe.\n",
    "df_sentences.to_pickle('/kaggle/working/sentence_embeddings.pkl')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Quick Check on Keywords Found\n",
    "\n",
    "We perform a quick check to see which keywords have been successfully identified and extracted. We found that among our list, 50+ of the keywords were found."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'convalescent plasma', 'saquinavir', 'arbidol', 'elispot', 'antibody dependent enhancement', 'ciprofloxacin', 'cobicistat', 'tocilizumab', 'vaccine', 'sofosbuvir', 'remdesivir', 'assay', 'nitazoxanide', 'elisa', 'corticosteroids', 'methylprednisolone', 'quercetin', 'oseltamivir', 'ctc', 'prophylaxis', 'bevacizumab', 'azithromycin', 'tenofovir', 'umifenovir', 'niclosamide', 'emtricitabine', 'prophylactic', 'baloxavir', 'nelfinavir', ' ade ', 'fingolimod', 'sirolimus', 'hydroxychloroquine', 'cytometry', 'ritonavir', 'lopinavir', 'favipiravir', 'th2', 'darunavir', 'chloroquine', 'zinc', 'antibody-dependent enhancement', 'human monoclonal antibody', 'jak', 'th1', 'danoprevir', 'galidesivir', 'interferon', 'tmprss2'}\n",
      "Number of keywords found: 49\n"
     ]
    }
   ],
   "source": [
    "# Some analysis\n",
    "found_keywords = []\n",
    "for keyword in metadata['keywords']:\n",
    "    found_keywords.append(keyword.split(\"; \"))\n",
    "\n",
    "found_keywords = set([item for sublist in found_keywords for item in sublist])\n",
    "\n",
    "print(found_keywords)\n",
    "print(\"Number of keywords found: \"+str(len(found_keywords)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Query and Results\n",
    "\n",
    "Finally, we are ready to query and find information. We achieve this by wrapping a our approach in a function called **execute_query()**.\n",
    "\n",
    "This function takes in several parameters: \n",
    "1. The query\n",
    "2. The metadata from our intial filtering\n",
    "3. The sha number of a particular article for the option of filtering on a single literature\n",
    "4. Similarity score threshold, which is defaulted to 0.65.\n",
    "\n",
    "Within the **execute_query()** function, the steps are:\n",
    "1. Take the query and produce a sentence embeddings from it\n",
    "2. If there is no query and a sha is provided, then set the similarity threshold to 0, as it means we want to output the full text of a single article\n",
    "3. Calculate the cosine distance between the query and the abstract sentences to produce simlarity scores\n",
    "4. Grab all the similarity results above the specified similarity threshold (**Stage 1**)\n",
    "5. Use the keywords from the results, process them, and feed them to the BM25 algorithm in order to extract the relevant pieces of text from the full body to provide a richer and deeper context of the results (**Stage 2**)\n",
    "6. Print out or save the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Make querys against the abstract sentence embeddings to identify candidates\n",
    "\n",
    "def execute_query(query, metadata, sha = '', similarity_threshold = 0.65, print_output=True):\n",
    "    \n",
    "    # 1. Take the query and produce a sentence embeddings from it\n",
    "    query = [query]\n",
    "    query_embedding = encoder.encode(query)\n",
    "    \n",
    "    # See if the query contains any of our keywords\n",
    "    query_keywords = list(set(query[0].split()) & set(keywords))\n",
    "    \n",
    "    # 2. If there is no query and a sha is provided, then set the similarity threshold to 0, as it means we want to output the full text of a single article\n",
    "    similarity_threshold = 0 if ((query[0] == \"\") and len(sha)>0) else similarity_threshold\n",
    "\n",
    "    # 3. Calculate the cosine distance between the query and the abstract sentences to produce simlarity scores\n",
    "    distances = scipy.spatial.distance.cdist(query_embedding, sentence_embeddings, \"cosine\")[0]\n",
    "    results = zip(range(len(distances)), distances) # Pair the indices with the cosine distance\n",
    "    results = sorted(results, key=lambda x: x[0]) # Sort them by index (this is needed to match the cosine scores with the results)\n",
    "\n",
    "    # 4. Grab all the similarity results above the specified similarity threshold  \n",
    "    result_dict = {'sha':[],'result':[]}\n",
    "    \n",
    "    # Loop through the results of the cosine distance calculations\n",
    "    for idx, distance in results:\n",
    "        # The similarity score is 1-distance (so that higher score = better)\n",
    "        similarity_score = 1-distance\n",
    "        \n",
    "        # Get the Sha and the sentence from our sentence dataframe\n",
    "        sentence = df_sentences['sentence'].iloc[idx].strip()\n",
    "        sha_id = df_sentences['sha'].iloc[idx].strip()\n",
    "\n",
    "        # If the similarity score of the sentence is below the threshold, then ignore it\n",
    "        if similarity_score < similarity_threshold:\n",
    "            continue\n",
    "            \n",
    "        # If a single sha id was provided, make sure to skip all the other articles that don't match that id\n",
    "        if len(sha) > 0 and sha_id !=sha:\n",
    "            continue\n",
    "\n",
    "        # If known keywords were found in the query, then make sure that the abstract contains that keyword\n",
    "        if len(query_keywords) > 0:\n",
    "                # Get the abstract from the sha id\n",
    "                abstract = metadata[metadata['sha'] == sha_id]['abstract'].item().lower()\n",
    "                # Determine if the keyword is in the abstract, if so then add that result\n",
    "                if any(keyword in abstract for keyword in query_keywords):\n",
    "                    result_dict['sha'].append(sha_id)\n",
    "                    result_dict['result'].append(sentence)\n",
    "        # If instead a single Sha id was provided, then then if a query was provivded or not and return the results\n",
    "        elif len(sha) > 0 and sha_id == sha:\n",
    "            result_dict['sha'].append(sha_id)\n",
    "            # If a query was not provided, then the result is blank (and later on the full body text will be returned)\n",
    "            if query[0] == \"\":\n",
    "                result_dict['result'].append(\"\")\n",
    "            # Otherwise, return the relevant result\n",
    "            else:\n",
    "                result_dict['result'].append(sentence)\n",
    "        # If no known keywords or single sha was identified, just return all available matches\n",
    "        else:\n",
    "            result_dict['sha'].append(sha_id)\n",
    "            result_dict['result'].append(sentence)\n",
    "    \n",
    "    # Convert the stage 1 results to a dataframe\n",
    "    temp_result = pd.DataFrame(result_dict)\n",
    "\n",
    "    # 5. Use the keywords from the results, process them, and feed them to the BM25 algorithm in order to extract the relevant pieces of text from the full body to provide a richer and deeper context of the results (**Stage 2**)\n",
    "    \n",
    "    # If there are multiple results found, then merge them together in the same string using the \"\\n\\n\" delimeter (So that context from all results are obtained)\n",
    "    temp_result = temp_result.groupby('sha')['result'].apply(\"\\n\\n\".join).reset_index()\n",
    "    # Apply the get_deeper_context() method to all the results in order to grab the deeper context\n",
    "    temp_result = temp_result.apply(get_deeper_context, axis=1)\n",
    "    # Save all the rsults in a dataframe\n",
    "    report = pd.merge(temp_result, metadata[['sha','title','publish_time','abstract','keywords','journal','number_citations']], how='inner')\n",
    "    report['query'] = query[0]\n",
    "\n",
    "    # If the print_output paramter is specified (default = True), then print the results in the console\n",
    "    if print_output:\n",
    "        # Print results\n",
    "        print(\"======= REPORT =========\")\n",
    "\n",
    "        for index, row in report.iterrows():\n",
    "            title = row['title']\n",
    "            sha = row['sha']\n",
    "            result = row['result']\n",
    "            num_citations = row['number_citations']\n",
    "            abstract = row['abstract']\n",
    "            context = row['context']\n",
    "            keywords_found = row['keywords']\n",
    "            publish_date = row['publish_time']\n",
    "\n",
    "            print(\"Query: \"+query[0])\n",
    "            print(\"Sha: \"+sha+\"\\n\")\n",
    "            print(\"Title: \"+title+\"\\n\")\n",
    "            print(\"Published Date: \"+publish_date+\"\\n\")\n",
    "            print(\"Number of times cited: \"+str(num_citations)+\"\\n\")\n",
    "            print(\"Antiviral Related Terms: \"+keywords_found+\"\\n\")\n",
    "            if (len(result.replace(\"\\n\",\"\")) > 0):\n",
    "                print(\"Results: \\n\"+result+\"\\n\")\n",
    "            print(\"Abstract: \")\n",
    "            print(abstract+\"\\n\")\n",
    "            print(\"Context: \")\n",
    "            print(context+\"\\n\")\n",
    "            print(\"----------------\")\n",
    "    \n",
    "    return report\n",
    "                "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Executing on a broad query\n",
    "\n",
    "Here we demonstrate an example of a query. \n",
    "\n",
    "We execute on the query \"antiviral treatments for COVID-19\" to answer one of the task. The query will default to printing the output so that the user can scroll through the results and do a quick evaluate. The results can also be saved into a dataframe and outputted via csv.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Batches: 100%|██████████| 1/1 [00:00<00:00, 58.67it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "======= REPORT =========\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: 0370abacf3b212bd6bded8f23c8c904000f6c2e6\n",
      "\n",
      "Title: Risk-adapted Treatment Strategy For COVID-19 Patients\n",
      "\n",
      "Published Date: 2020-03-27\n",
      "\n",
      "Number of times cited: 0\n",
      "\n",
      "Antiviral Related Terms: methylprednisolone\n",
      "\n",
      "Results: \n",
      "The objective of this study is to investigate the short-term effect of risk-adapted treatment strategy on patients with COVID-19.\n",
      "\n",
      "Abstract: \n",
      "Abstract Background There are no clear expert consensus or guidelines on how to treat 2019 coronavirus disease (COVID-19). The objective of this study is to investigate the short-term effect of risk-adapted treatment strategy on patients with COVID-19. Methods We collected the medical records of 55 COVID-19 patients for analysis. We divided these patients into mild, moderate and severe groups, and risk-adapted treatment approaches were given according to the illness severity. Results Twelve patients were in mild group and 22 were in moderate group (non-severe group, n=34), and 21 patients were in severe group. At the end of the first two weeks after admission, clinical manifestations had completely despeared in 31(91.2%)patients in non-severe group, and 18(85.7%) patients in severe group (p=0.85). Both groups had a satisfied chest CT imaging recovery, which includes 22(64.7%) patients in non-severe group and 12(57.1%) patients in severe group recovered at least 50% of the whole leisions in the first week, and 28(82.4%) and 16(76.2%) recovered at least 75% in the second week, respectively. There were no significant differences in SARS-CoV-2 RNA clearance between two group (p=0.92). There were also no significant differences in the levels of SARS-CoV-2-IgM and IgG antibody production between the two groups (p=0.13, 0.62). There were 45 cases were discharged from the hospital, and no patients died at the time of this clinical analysis. Conclusions Risk-adapted treatment strategy was associated with significant clinical manifestations alleviation and clinical imaging recovery. In severe COVID-19 patients, early and short-term use of low-dose methylprednisolone was beneficial and did not delay SARS-CoV-2 RNA clearance and influence IgG antibody production.\n",
      "\n",
      "Context: \n",
      "Paragraph: 1\n",
      "However, until now, there are no clear expert consensus or guidelines on how to treat COVID-19. We developed a risk-adapted treatment approach according to the illness severity; and the objective of this study is to investigate the short-term effect of this riskadapted treatment strategy on clinical manifestations alleviation, clinical imaging recovery and SARS-CoV-2 RNA clearance.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: 06a1002f9fbea7179ac3572843f66b14568af6e4\n",
      "\n",
      "Title: Nelfinavir was predicted to be a potential inhibitor of 2019-nCov main protease by an integrative approach combining homology modelling, molecular docking and binding free energy calculation\n",
      "\n",
      "Published Date: 2020-01-28\n",
      "\n",
      "Number of times cited: 4\n",
      "\n",
      "Antiviral Related Terms: nelfinavir\n",
      "\n",
      "Results: \n",
      "2019-nCov Mpro is a potential drug target to combat the virus.\n",
      "\n",
      "Therefore, we suggested that nelfinavir might be a potential inhibitor against 2019-nCov Mpro.\n",
      "\n",
      "Abstract: \n",
      "Abstract2019-nCov has caused more than 80 deaths as of 27 January 2020 in China, and infection cases have been reported in more than 10 countries. However, there is no approved drug to treat the disease. 2019-nCov Mpro is a potential drug target to combat the virus. We built homology models based on SARS Mpro structures, and docked 1903 small molecule drugs to the models. Based on the docking score and the 3D similarity of the binding mode to the known Mpro ligands, 4 drugs were selected for binding free energy calculations. Both MM/GBSA and SIE methods voted for nelfinavir, with the binding free energy of −24.69±0.52 kcal/mol and −9.42±0.04 kcal/mol, respectively. Therefore, we suggested that nelfinavir might be a potential inhibitor against 2019-nCov Mpro.\n",
      "\n",
      "Context: \n",
      "Paragraph: 16\n",
      "2019-nCov caused more than 80 deaths in China as of 27 January 2020 and is a potential threat to the global health. However, there is no approved drug to treat the disease.\n",
      " \n",
      "Paragraph: 17\n",
      "2019-nCov M pro is a potential drug target to combat the virus, which shares 96% sequence similarity with the corresponding one in SARS. We built 11 homology models of 2019-nCov M pro and docked 1903 approved small molecule drugs to the 2GTB model. Based on the docking score and the 3D similarity of binding mode to 39 known M pro binders, 15 drugs were selected for further evaluation. The 15 drugs were then docked to all the 11 homology models, leading to 4 drugs for binding energy calculations. Both MM/GBSA and SIE calculations voted for nelfinavir, a HIV-1 protease inhibitor to treat HIV. Therefore, we suggested that nelfinavir might be active against 2019-nCov M pro . In addition, pitavastatin, perampanel, and praziquantel might author/funder. All rights reserved. No reuse allowed without permission.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: 0c9d951acb01cb541671b3065b882bbcb61f9523\n",
      "\n",
      "Title: The epidemiology, diagnosis and treatment of COVID-19\n",
      "\n",
      "Published Date: 2020-03-28\n",
      "\n",
      "Number of times cited: 0\n",
      "\n",
      "Antiviral Related Terms: convalescent plasma; hydroxychloroquine; vaccine; corticosteroids; chloroquine\n",
      "\n",
      "Results: \n",
      "In addition, registered trials investigating treatment options for COVID-19 infection are listed.\n",
      "\n",
      "Abstract: \n",
      "Abstract In December 2019, the outbreak of the novel coronavirus disease (COVID-19) in China spread worldwide, becoming an emergency of major international concern. SARS-CoV-2 infection causes clusters of severe respiratory illness similar to severe acute respiratory syndrome coronavirus. Human-to-human transmission via droplets, contaminated hands or surfaces has been described, with incubation times of 2-14 days. Early diagnosis, quarantine, and supportive treatments are essential to cure patients. This paper reviews the literature on all available information about the epidemiology, diagnosis, isolation and treatments of COVID-19. Treatments, including antiviral agents, chloroquine and hydroxychloroquine, corticosteroids, antibodies, convalescent plasma transfusion and vaccines, are discussed in this article. In addition, registered trials investigating treatment options for COVID-19 infection are listed.\n",
      "\n",
      "Context: \n",
      "Paragraph: 6\n",
      "There is no current evidence from randomized controlled trials (RCTs) to recommend any specific anti-SARS-CoV-2 treatment for patients with a suspected or confirmed COVID-19 infection. Lopinavir (LPV) inhibits the protease activity of coronavirus in vitro and in animal studies. A retrospective, matched-cohort study including 1052 SARS patients showed that LPV/ritonavir as initial treatment was associated with a reduced death rate (2.3% vs. 11.0%) [43] . The protease inhibitor LPV is an effective treatment based on the experience accumulated from the SARS and MERS outbreaks, indicating it is a potential treatment option for COVID-19 [44] . Ribavirin, a guanosine analogue, is an antiviral compound used to treat several virus infections, including respiratory syncytial virus, hepatitis C virus, and some viral hemorrhagic fevers. Promising results were obtained with ribavirin in a MERS-CoV rhesus macaque model [45] . In addition, SARS-CoV-2 RNA-dependent RNA polymerase (RdRp) model is targeted by ribavirin after sequence analysis, modeling, and docking to build the model. This feature increases its potential as an antiviral against SARS-CoV-2 [46] .\n",
      " \n",
      "Paragraph: 7\n",
      "The antiviral agent, remdesivir was designed for the Ebola virus disease [47] . Remdesivir shows broad-spectrum antiviral activity against several RNA viruses, and it may compete for RdRp [48] . Remdesivir and IFNb have superior antiviral activity to LPV and ritonavir in vitro [49] . In a mouse model of SARS-CoV pathogenesis, both prophylactic and therapeutic remdesivir improved pulmonary function and reduced lung viral loads and severe lung pathology [50] . In a rhesus macaque model of MERS-CoV infection, prophylactic remdesivir treatment was initiated 24 h prior to inoculation, and MERS-CoV did not induce clinical disease and did not replicate in respiratory tissues, thus preventing the formation of lung lesions [51] . In cell-based assays, the triphosphate form of remdesivir incorporated at position i, and RNA chain termination was delayed, which explained the high potency of remdesivir against RNA [52] . Remdesivir was used to treat the first case of COVID-19 infection in the United States: the patient's clinical condition improved after only one day of remdesivir treatment [53] . A phase II clinical trial of remdesivir was performed by the University of Nebraska Medical Center, and a phase III clinical trial was performed by the China-Japan Friendship Hospital. The results of these clinical trials will be revealed in April 2020. Remdesivir improved pulmonary function, reduced lung viral loads, and ameliorated severe lung pathology. In contrast, prophylactic LPV/RTV-IFNb only slightly reduced viral loads and did not impact other disease parameters, and therapeutic LPV/RTV-IFNb improved pulmonary function, but did not reduce virus replication or severe lung pathology [49] . Overall, these results indicated that remdesivir showed more potential than LPV/RTV-IFNb [54] . In a case report, lopinavir/ritonavir (Kale-tra®) and arbidol were associated with significant improvements in COVID-19 patients [55] . The efficacy and safety of these antiviral agents for COVID-19 will be assessed in further clinical trials. Thirty-four trials of antiviral agents in patients with COVID-19 have been registered up to March 15, 2020 ( Table 1 ) .\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: 188e7ff1e260864c89f266b5597de26d69a84660\n",
      "\n",
      "Title: Clinical trials on drug repositioning for COVID-19 treatment\n",
      "\n",
      "Published Date: 2020-03-20\n",
      "\n",
      "Number of times cited: 0\n",
      "\n",
      "Antiviral Related Terms: oseltamivir; hydroxychloroquine; arbidol; ritonavir; bevacizumab; lopinavir; favipiravir; remdesivir; methylprednisolone; chloroquine; interferon\n",
      "\n",
      "Results: \n",
      "WHO has named the novel coronavirus disease as COVID-19.\n",
      "\n",
      "Abstract: \n",
      "The World Health Organization (WHO) was informed on December 2019 about a coronavirus pneumonia outbreak in Wuhan, Hubei province (China). Subsequently, on March 12, 2020, 125,048 cases and 4,614 deaths were reported. Coronavirus is an enveloped RNA virus, from the genus Betacoronavirus, that is distributed in birds, humans, and other mammals. WHO has named the novel coronavirus disease as COVID-19. More than 80 clinical trials have been launched to test coronavirus treatment, including some drug repurposing or repositioning for COVID-19. Hence, we performed a search in March 2020 of the clinicaltrials.gov database. The eligibility criteria for the retrieved studies were: contain a clinicaltrials.gov base identifier number; describe the number of participants and the period for the study; describe the participants’ clinical conditions; and utilize interventions with medicines already studied or approved for any other disease in patients infected with the novel coronavirus SARS-CoV-2 (2019-nCoV). It is essential to emphasize that this article only captured trials listed in the clinicaltrials.gov database. We identified 24 clinical trials, involving more than 20 medicines, such as human immunoglobulin, interferons, chloroquine, hydroxychloroquine, arbidol, remdesivir, favipiravir, lopinavir, ritonavir, oseltamivir, methylprednisolone, bevacizumab, and traditional Chinese medicines (TCM). Although drug repurposing has some limitations, repositioning clinical trials may represent an attractive strategy because they facilitate the discovery of new classes of medicines; they have lower costs and take less time to reach the market; and there are existing pharmaceutical supply chains for formulation and distribution.\n",
      "\n",
      "Context: \n",
      "Paragraph: 0\n",
      "The World Health Organization (WHO) was informed on December 31, 2019, about a pneumonia outbreak in Wuhan, Hubei province (China), a city with 11 million inhabitants. By March 12, 2020, there were 125 048 cases and 4 614 deaths (nearly 3.7% of cases) reported for the novel coronavirus (1), named 2019-novel coronavirus (2019-nCoV), and later renamed as severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) (2). The WHO named this novel coronavirus disease as COVID-19 (1, 2), and there have been confirmed cases in 117 countries or territories outside China, including Japan, the United States of America, Italy, Iran, and Brazil (1) . Coronavirus is an enveloped RNA virus, from the genus Betacoronavirus, that is distributed in birds, humans, and other mammals (3, 4) . New evidence indicates a link between SARS-CoV-2 and bat coronavirus (3) . Six species of coronavirus are known as infectious in humans, four of which (229E, OC43, NL63, and HKU1) cause common cold symptoms (4) . However, some authors have claimed that SARS-CoV-2 is even related to the coronavirus species responsible for the severe acute respiratory syndrome (SARS-CoV) as well as Middle East Respiratory Syndrome (MERS-CoV), which have zoonotic origins linked to severe significant illness with higher mortality (3, 4) . For example, in July 2003, the WHO reported 8 437 SARS-CoV cases, especially in China and Hong Kong, with 813 related deaths (5) . Concerning MERS-CoV, from June 2012 to April 2018, 2 206 people were infected in 27 countries, 1 831 cases in Saudi Arabia, with 787 deaths (6) . Unfortunately, there are no vaccines or medicines approved for the novel coronavirus infection (7), but more than 80 clinical trials have been launched to test coronavirus treatments, including some drug repurposing or repositioning for COVID-19 (8) . Drug repositioning for other neglected diseases is an essential and universal strategy in the development of new drugs due to: a) lower costs and reduced time to reach the market because some clinical trial steps might not be required, especially concerning phases I and II; b) existing pharmaceutical supply chains are available for formulation and distribution; c) the possibility of combinations with other drugs in treatments that are more effective than monotherapy; and d) may facilitate the discovery of new mechanisms of action for old drugs and new classes of medicines (9, 10) .\n",
      " \n",
      "Paragraph: 2\n",
      "We performed a search on March 12, 2020, at the clinicaltrials. gov database, with the descriptor [coronavirus] in the simple search field \"conditions or disease\" search, without restrictions on languages, disease conditions, results, or locations. The eligibility criteria for the retrieved studies were: contain a clinicaltrials.gov base identifier number; describe the number of participants and the study period; describe the patient's clinical conditions; and interventions utilize medicines already studied or approved for any other disease in patients with COVID-19. ClinicalTrials.gov is a resource from the US National Library of Medicine, and it contains clinical studies conducted by 209 countries.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: 188f3e97042155ac1709aa8b74c0755760c3b50d\n",
      "\n",
      "Title: Associations between immune-suppressive and stimulating drugs and novel COVID-19—a systematic review of current evidence\n",
      "\n",
      "Published Date: 2020-03-27\n",
      "\n",
      "Number of times cited: 0\n",
      "\n",
      "Antiviral Related Terms: jak\n",
      "\n",
      "Results: \n",
      "This review focuses on the effects of these drugs on host immunity against COVID-19.\n",
      "\n",
      "Abstract: \n",
      "BACKGROUND: Cancer and transplant patients with COVID-19 have a higher risk of developing severe and even fatal respiratory diseases, especially as they may be treated with immune-suppressive or immune-stimulating drugs. This review focuses on the effects of these drugs on host immunity against COVID-19. METHODS: Using Ovid MEDLINE, we reviewed current evidence for immune-suppressing or -stimulating drugs: cytotoxic chemotherapy, low-dose steroids, tumour necrosis factorα (TNFα) blockers, interlukin-6 (IL-6) blockade, Janus kinase (JAK) inhibitors, IL-1 blockade, mycophenolate, tacrolimus, anti-CD20 and CTLA4-Ig. RESULTS: 89 studies were included. Cytotoxic chemotherapy has been shown to be a specific inhibitor for severe acute respiratory syndrome coronavirus in in vitro studies, but no specific studies exist as of yet for COVID-19. No conclusive evidence for or against the use of non-steroidal anti-inflammatory drugs (NSAIDs) in the treatment of COVID-19 patients is available, nor is there evidence indicating that TNFα blockade is harmful to patients in the context of COVID-19. COVID-19 has been observed to induce a pro-inflammatory cytokine generation and secretion of cytokines, such as IL-6, but there is no evidence of the beneficial impact of IL-6 inhibitors on the modulation of COVID-19. Although there are potential targets in the JAK-STAT pathway that can be manipulated in treatment for coronaviruses and it is evident that IL-1 is elevated in patients with a coronavirus, there is currently no evidence for a role of these drugs in treatment of COVID-19. CONCLUSION: The COVID-19 pandemic has led to challenging decision-making about treatment of critically unwell patients. Low-dose prednisolone and tacrolimus may have beneficial impacts on COVID-19. The mycophenolate mofetil picture is less clear, with conflicting data from pre-clinical studies. There is no definitive evidence that specific cytotoxic drugs, low-dose methotrexate for auto-immune disease, NSAIDs, JAK kinase inhibitors or anti-TNFα agents are contraindicated. There is clear evidence that IL-6 peak levels are associated with severity of pulmonary complications.\n",
      "\n",
      "Context: \n",
      "Paragraph: 1\n",
      "The current review focuses on the effects of immune-suppressive or immune-stimulating drugs on the host immunity against COVID-19. Here, we report a short introduction to each drug, followed by a summary of the results from the identified studies for each relative drug.\n",
      " \n",
      "Paragraph: 28\n",
      "This review focuses on how the JAK-STAT pathway can be manipulated to reduce viral entry and inflammation in patients with coronavirus. The main targets that the review highlighted were baricitinib (JAK inhibitor), IRE1α (an endoplasmic reticulum stress sensor, leading to an increased expression of negative regulators of JAK-STAT SOCS1 and SOCS3) and combination therapies using tylophorine-based compounds with JAK2 inhibitors.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: 19a08cc1423d6006b2251bdcfd142f88db7002e7\n",
      "\n",
      "Title: Genetic Profiles in Pharmacogenes Indicate Personalized Drug Therapy for COVID-19\n",
      "\n",
      "Published Date: 2020-03-30\n",
      "\n",
      "Number of times cited: 0\n",
      "\n",
      "Antiviral Related Terms: lopinavir; chloroquine; interferon\n",
      "\n",
      "Results: \n",
      "Many drugs showed potential for COVID-19 therapy.\n",
      "\n",
      "Abstract: \n",
      "Background: The coronavirus disease 2019 (COVID-19) has become a global pandemic currently. Many drugs showed potential for COVID-19 therapy. However, genetic factors which can lead to different drug efficiency and toxicity among populations are still undisclosed in COVID-19 therapy. Methods: We selected 67 potential drugs for COVID-19 therapy (DCTs) from clinical guideline and clinical trials databases. 313 pharmaco-genes related to these therapeutic drugs were included. Variation information in 125,748 exomes were collected for racial differences analyses. The expression level of pharmaco-genes in single cell resolution was evaluated from single-cell RNA sequencing (scRNA-seq) data of 17 healthy adults. Results: Pharmacogenes, including CYP3A4, ABCB1, SLCO1B1, ALB, CYP3A5, were involved in the process of more than multi DCTs. 224 potential drug-drug interactions (DDI) of DCTs were predicted, while 112 of them have been reported. Racial discrepancy of common nonsynonymous mutations was found in pharmacogenes including: VDR, ITPA, G6PD, CYP3A4 and ABCB1 which related to DCTs including ribavirin, α-interferon, chloroquine and lopinavir. Moreover, ACE2, the target of 2019-nCoV, was only found in parts of lung cells, which makes drugs like chloroquine that prevent virus binding to ACE2 more specific than other targeted drugs such as camostat mesylate. Conclusions: At least 17 drugs for COVID-19 therapy with predictable pharmacogenes should be carefully utilized in risk races which are consisted of more risk allele carriers. At least 29 drugs with potential of DDIs are reported to be affected by other DDIs, they should be replaced by similar drugs without interaction if it is possible. Drugs which specifically targeted to infected cells with ACE2 such as chloroquine are preferred in COVID-19 therapy.\n",
      "\n",
      "Context: \n",
      "Paragraph: 1\n",
      "This promotes the importance of personalized therapy for COVID-19 patients.\n",
      " \n",
      "Paragraph: 4\n",
      "Pharmacogenetics (PGx) investigates the affection of genetic variants on drug effects and safety. In the current study, we provide a PGx landscape of drugs with potential to be used in the COVID-19 treatment. Candidate choices from different drugs based on pharmacogenetic analyses were provided.\n",
      " \n",
      "Paragraph: 21\n",
      "Ribavirin is a classic antivirus drug which has been involved in the guideline for 2019-nCoV therapy as well. VDR polymorphism is associated with its efficacy ( Figure 4A ). There is only a common non-synonymous polymorphism M1T (rs2228570) that could hamper the VDR activity and the M1T carriers showed lower efficacy. The frequency of this mutation is over 0.5 in all populations ( Figure 4A ), which means more than half patients will be resistant to this drug.\n",
      " \n",
      "Paragraph: 28\n",
      "As we mentioned above, ABCB1 can pumping out many drugs across the All rights reserved. No reuse allowed without permission.\n",
      " \n",
      "Paragraph: 44\n",
      "Since there are many candidate drugs for COVID-19 therapy, it is important to make a better choice for each patient. can be found in Table 2 .\n",
      " \n",
      "Paragraph: 45\n",
      "As a large part of drugs are metabolized by CYP450 in liver, the activity of CYP450 family may impact many drugs metabolization rate. Due to the activity difference exists among populations, the genotypes of important alleles in CYP450 family should be detected after diagnosis, then treatment strategies for these drugs can be suggested in these patients. Meanwhile, DDI should also be considered as many drugs may be utilized together during COVID-19 treatment as mentioned above. For instance, drugs like lopinavir and ritonavir can inhibit the activity of CYP3A4, thus the drugs dosage should be adjusted when combined with drugs metabolized by CYP3A4. Drugs that were metabolized by CYP450 family in COVID-19 therapy were listed in Table 3 . In addition, when chloroquine is utilized, drugs like losartan which can increase the concentration of chloroquine may be replaced by valsartan.\n",
      " \n",
      "Paragraph: 55\n",
      "Although precision medicine should be considered in COVID-19 treatment, there are still many challenges to face. The first challenge is that pharmacogenes of many drugs for COVID-19 are still not clear, due to that many newly developed drugs are still under clinical trials, their pharmacogenes can hardly be fully identified currently.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: 1f5c1597a84ed1d4f84c488cd19098a091a3d513; 29d15895b6f3c6472f3bab820d6a99feefae60df\n",
      "\n",
      "Title: Asymptomatic carrier state, acute respiratory disease, and pneumonia due to severe acute respiratory syndrome coronavirus 2 (SARSCoV-2): Facts and myths\n",
      "\n",
      "Published Date: 2020\n",
      "\n",
      "Number of times cited: 0\n",
      "\n",
      "Antiviral Related Terms: chloroquine; remdesivir\n",
      "\n",
      "Results: \n",
      "Indeed, age and disease severity may be correlated with the outcomes of COVID-19.\n",
      "\n",
      "Abstract: \n",
      "Since the emergence of coronavirus disease 2019 (COVID-19) (formerly known as the 2019 novel coronavirus [2019-nCoV]) in Wuhan, China in December 2019, which is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), more than 75,000 cases have been reported in 32 countries/regions, resulting in more than 2,000 deaths worldwide. Despite the fact that most COVID-19 cases and mortalities were reported in China, the WHO has declared this outbreak as the sixth public health emergency of international concern. The COVID-19 can present as an asymptomatic carrier state, acute respiratory disease, and pneumonia. Adults represent the population with the highest infection rate; however, neonates, children, and elderly patients can also be infected by SARS-CoV-2. In addition, nosocomial infection of hospitalized patients and healthcare workers, and viral transmission from asymptomatic carriers are possible. The most common finding on chest imaging among patients with pneumonia was ground-glass opacity with bilateral involvement. Severe cases are more likely to be older patients with underlying comorbidities compared to mild cases. Indeed, age and disease severity may be correlated with the outcomes of COVID-19. To date, effective treatment is lacking; however, clinical trials investigating the efficacy of several agents, including remdesivir and chloroquine, are underway in China. Currently, effective infection control intervention is the only way to prevent the spread of SARS-CoV-2.\n",
      "\n",
      "Context: \n",
      "NA\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: 23e7355b5e4e0209f64c9d8d5772092a53b72686\n",
      "\n",
      "Title: Efficacy of hydroxychloroquine in patients with COVID-19: results of a randomized clinical trial\n",
      "\n",
      "Published Date: 2020-03-30\n",
      "\n",
      "Number of times cited: 1\n",
      "\n",
      "Antiviral Related Terms: hydroxychloroquine; chloroquine\n",
      "\n",
      "Results: \n",
      "Aims: Studies have indicated that chloroquine (CQ) shows antagonism against COVID-19 in vitro.\n",
      "\n",
      "This study aims to evaluate the efficacy of hydroxychloroquine (HCQ) in the treatment of patients with COVID-19.\n",
      "\n",
      "Abstract: \n",
      "Aims: Studies have indicated that chloroquine (CQ) shows antagonism against COVID-19 in vitro. However, evidence regarding its effects in patients is limited. This study aims to evaluate the efficacy of hydroxychloroquine (HCQ) in the treatment of patients with COVID-19. Main methods: From February 4 to February 28, 2020, 62 patients suffering from COVID-19 were diagnosed and admitted to Renmin Hospital of Wuhan University. All participants were randomized in a parallel-group trial, 31 patients were assigned to receive an additional 5-day HCQ (400 mg/d) treatment, Time to clinical recovery (TTCR), clinical characteristics, and radiological results were assessed at baseline and 5 days after treatment to evaluate the effect of HCQ. Key findings: For the 62 COVID-19 patients, 46.8% (29 of 62) were male and 53.2% (33 of 62) were female, the mean age was 44.7 (15.3) years. No difference in the age and sex distribution between the control group and the HCQ group. But for TTCR, the body temperature recovery time and the cough remission time were significantly shortened in the HCQ treatment group. Besides, a larger proportion of patients with improved pneumonia in the HCQ treatment group (80.6%, 25 of 32) compared with the control group (54.8%, 17 of 32). Notably, all 4 patients progressed to severe illness that occurred in the control group. However, there were 2 patients with mild adverse reactions in the HCQ treatment group. Significance: Among patients with COVID-19, the use of HCQ could significantly shorten TTCR and promote the absorption of pneumonia.\n",
      "\n",
      "Context: \n",
      "Paragraph: 2\n",
      "The recent publication of results showing the activity of chloroquine (CQ) against SARS-CoV-2 in vitro [7], some experts and researchers also have been recommended the efficacy of this antimalarial drug in patients with 9] . For this, the U.S. Food and Drug Administration (FDA) has been working to investigate the use of CQ in COVID-19 [10] . As a derivative of CQ, hydroxychloroquine (HCQ) has similar therapeutic effects and fewer adverse effects. Based on its characteristics of immunity regulation, antithrombotic activity, and inflammation improvement, HCQ has been routinely used in the clinical treatment of systemic lupus erythematosus (SLE) [11] . However, . CC-BY-NC-ND 4.0 International license It is made available under a author/funder, who has granted medRxiv a license to display the preprint in perpetuity.\n",
      " \n",
      "Paragraph: 3\n",
      "is the (which was not peer-reviewed) The copyright holder for this preprint . https://doi.org/10.1101/2020.03. 22.20040758 doi: medRxiv preprint the efficacy of HCQ in COVID-19 remains unknown.\n",
      " \n",
      "Paragraph: 14\n",
      "is the (which was not peer-reviewed) The copyright holder for this preprint Abbreviations: HCQ, hydroxychloroquine. Figure 1 : Study flow diagram.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: 2f547947bf87380c7fab13ba2c663bbbe9e643ec\n",
      "\n",
      "Title: COVID-19: Epidemiology, Evolution, and Cross-Disciplinary Perspectives\n",
      "\n",
      "Published Date: 2020-03-21\n",
      "\n",
      "Number of times cited: 0\n",
      "\n",
      "Antiviral Related Terms: vaccine\n",
      "\n",
      "Results: \n",
      "Here, we report the epidemiological and virological characteristics of the COVID-19 outbreak.\n",
      "\n",
      "Abstract: \n",
      "The recent outbreak of COVID-19 in Wuhan turned into a public health emergency of international concern. With no antiviral drugs nor vaccines, and the presence of carriers without obvious symptoms, traditional public health intervention measures are significantly less effective. Here, we report the epidemiological and virological characteristics of the COVID-19 outbreak. Originated in bats, 2019-nCoV/ severe acute respiratory syndrome coronavirus (SARS-CoV)-2 likely experienced adaptive evolution in intermediate hosts before transfer to humans at a concentrated source of transmission. Similarities of receptor sequence binding to 2019-nCoV between humans and animals suggest a low species barrier for transmission of the virus to farm animals. We propose, based on the One Health model, that veterinarians and animal specialists should be involved in a cross-disciplinary collaboration in the fight against this epidemic.\n",
      "\n",
      "Context: \n",
      "Paragraph: 0\n",
      "The recent outbreak of COVID-19 in Wuhan turned into a public health emergency of international concern. With no antiviral drugs nor vaccines, and the presence of carriers without obvious symptoms, traditional public health intervention measures are significantly less effective. Here, we report the epidemiological and virological characteristics of the COVID-19 outbreak. Originated in bats, 2019-nCoV/ severe acute respiratory syndrome coronavirus (SARS-CoV)-2 likely experienced adaptive evolution in intermediate hosts before transfer to humans at a concentrated source of transmission. Similarities of receptor sequence binding to 2019-nCoV between humans and animals suggest a low species barrier for transmission of the virus to farm animals. We propose, based on the One Health model, that veterinarians and animal specialists should be involved in a cross-disciplinary collaboration in the fight against this epidemic.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: 5998e7fb99b5c58f7563017ad679fbe2f9974a0c\n",
      "\n",
      "Title: Chloroquine and hydroxychloroquine for the treatment of COVID-19: A living systematic review protocol\n",
      "\n",
      "Published Date: 2020-04-08\n",
      "\n",
      "Number of times cited: 0\n",
      "\n",
      "Antiviral Related Terms: hydroxychloroquine; chloroquine\n",
      "\n",
      "Results: \n",
      "OBJECTIVE: To determine the relative impact of the use of chloroquine and hydroxychloroquine on outcomes important to patients with COVID 19.\n",
      "\n",
      "Abstract: \n",
      "OBJECTIVE: To determine the relative impact of the use of chloroquine and hydroxychloroquine on outcomes important to patients with COVID 19. DESIGN: This is the protocol of a living systematic review. DATA SOURCES: We will conduct searches in PubMed/Medline, Embase, Cochrane Central Register of Controlled Trials (CENTRAL), trial registries, grey literature and in a centralised repository in L-OVE (Living OVerview of Evidence). L-OVE is a platform that maps PICO questions to evidence from Epistemonikos database. In response to the COVID-19 emergency, L-OVE was adapted to expand the range of evidence it covers and customised to group all COVID-19 evidence in one place. The search will cover the period until the day before submission to a journal. ELIGIBILITY CRITERIA FOR SELECTING STUDIES AND METHODS: We will follow a common protocol for multiple parallel systematic reviews, already published and submitted to PROSPERO (awaiting ID allocation). We will include randomised controlled trials evaluating the effect of chloroquine and hydroxychloroquine - as monotherapy or in combination with other drugs - versus placebo or no treatment in patients with COVID-19. Randomised trials evaluating chloroquine and hydroxychloroquine in infections caused by other coronaviruses, such as MERS-CoV and SARS-CoV, and non-randomised studies in COVID-19 will be searched in case no direct evidence from randomised trials is found, or if the direct evidence provides low- or very low-certainty for critical outcomes. Two reviewers will independently screen each study for eligibility, extract data, and assess the risk of bias. We will perform random-effects meta-analyses and use GRADE to assess the certainty of the evidence for each outcome. A living, web-based version of this review will be openly available during the COVID-19 pandemic. We will resubmit it if the conclusions change or there are substantial updates. ETHICS AND DISSEMINATION: No ethics approval is considered necessary. The results of this review will be widely disseminated via peer-reviewed publications, social networks and traditional media.\n",
      "\n",
      "Context: \n",
      "Paragraph: 3\n",
      "Notwithstanding, despite its in vitro activity against several agents, to this date there is no acute viral infection that has been successfully treated by chloroquine or hydroxychloroquine. Moreover, in Chikungunya virus infection, chloroquine showed the paradoxical effect of leading to more chronic complications in the patients who received it, an outcome that would be explained by a delay in immune adaptive response [13] . The pathogenesis of COVID-19 is still unknown, therefore the clinical effects of administration of antimalarials in COVID-19 patients is largely unpredictable [ 14 ] .\n",
      " \n",
      "Paragraph: 6\n",
      "is the (which was not peer-reviewed) The copyright holder for this preprint . https://doi.org/10.1101/2020.04.03.20052530 doi: medRxiv preprint rigorous and continuously updated summary of the evidence available on the relative impact of the use of antimalarials on outcomes important to patients with COVID-19.\n",
      " \n",
      "Paragraph: 25\n",
      "The interventions of interest are the antimalarials chloroquine and hydroxychloroquine. We will not restrict our criteria to any dosage, duration, timing or route of administration.\n",
      " \n",
      "Paragraph: 41\n",
      "is the (which was not peer-reviewed) The copyright holder for this preprint . https://doi.org/10.1101/2020.04.03.20052530 doi: medRxiv preprint outcomes as MID units or inform the results as the difference in the proportion of patients achieving a minimal important effect between intervention and control [30] .\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: 5d13ba76e768fe46ecdf4ad9d2ba10ff7b9bcc48\n",
      "\n",
      "Title: Recommendations for coronavirus infection in rheumatic diseases treated with biologic therapy\n",
      "\n",
      "Published Date: 2020-04-02\n",
      "\n",
      "Number of times cited: 0\n",
      "\n",
      "Antiviral Related Terms: vaccine; tocilizumab\n",
      "\n",
      "Results: \n",
      "Based upon our limited knowledge on the pathogenesis of CoViD-19, a potential role of some anti-rheumatic drugs may be hypothesized, acting as direct antivirals or targeting host immune response.\n",
      "\n",
      "Checkpoints inhibitors such as anti-CD200 and anti-PD1 could have a role in the treatment of CoViD-19.\n",
      "\n",
      "Abstract: \n",
      "Abstract The Coronavirus-associated disease, that was first identified in 2019 in China (CoViD-19), is a pandemic caused by a bat-derived beta-coronavirus, named SARS-CoV2. It shares homology with SARS and MERS-CoV, responsible for past outbreaks in China and in Middle East. SARS-CoV2 spread from China where the first infections were described in December 2019 and is responsible for the respiratory symptoms that can lead to acute respiratory distress syndrome. A cytokine storm has been shown in patients who develop fatal complications, as observed in past coronavirus infections. The management includes ventilatory support and broad-spectrum antiviral drugs, empirically utilized, as a targeted therapy and vaccines have not been developed. Based upon our limited knowledge on the pathogenesis of CoViD-19, a potential role of some anti-rheumatic drugs may be hypothesized, acting as direct antivirals or targeting host immune response. Antimalarial drugs, commonly used in rheumatology, may alter the lysosomal proteases that mediates the viral entry into the cell and have demonstrated efficacy in improving the infection. Anti-IL-1 and anti-IL-6 may interfere with the cytokine storm in severe cases and use of tocilizumab has shown good outcomes in a small cohort. Baricitinib has both antiviral and anti-inflammatory properties. Checkpoints inhibitors such as anti-CD200 and anti-PD1 could have a role in the treatment of CoViD-19. Rheumatic disease patients taking immunosuppressive drugs should be recommended to maintain the chronic therapy, prevent infection by avoiding social contacts and pausing immunosuppressants in case of infection. National and international registries are being created to collect data on rheumatic patients with CoViD-19.\n",
      "\n",
      "Context: \n",
      "Paragraph: 0\n",
      "The Coronavirus-associated disease, that was first identified in 2019 in China, is a pandemic caused by a bat-derived beta-coronavirus, named SARS-CoV2. It shares homology with SARS and MERS-CoV, responsible for past outbreaks in China and in Middle East. SARS-CoV2 spread from China where the first infections were described in December 2019 and is responsible for the respiratory symptoms that can lead to acute respiratory distress syndrome. A cytokine storm has been shown in patients who develop fatal complications, as observed in past coronavirus infections. The management includes ventilatory support and broad-spectrum antiviral drugs, empirically utilized, as a targeted therapy and vaccines have not been developed. Based upon our limited knowledge on the pathogenesis of CoViD-19, a potential role of some anti-rheumatic drugs may be hypothesized, acting as direct antivirals or targeting host immune response. Antimalarial drugs, commonly used in rheumatology, may alter the lysosomal proteases that mediates the viral entry into the cell and have demonstrated efficacy in improving the infection. Anti-IL-1 and anti-IL-6 may interfere with the cytokine storm in severe cases and use of tocilizumab has shown good outcomes in a small cohort. Baricitinib has both antiviral and anti-inflammatory properties. Checkpoints inhibitors such as anti-CD200 and anti-PD1 could have a role in the treatment of CoViD-19. Rheumatic disease patients taking immunosuppressive drugs should be recommended to maintain the chronic therapy, prevent infection by avoiding social contacts and pausing immunosuppressants in case of infection. National and international registries are being created to collect data on rheumatic patients with CoViD-19.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: 5dc4268a42adf3d5c55c87b7f6518de600b057c5\n",
      "\n",
      "Title: Coronavirus Disease 2019 (COVID-19) Pandemic and Pregnancy\n",
      "\n",
      "Published Date: 2020-03-23\n",
      "\n",
      "Number of times cited: 1\n",
      "\n",
      "Antiviral Related Terms: th1; vaccine; th2; assay; chloroquine\n",
      "\n",
      "Results: \n",
      "We discuss the latest options in antiviral therapy and vaccine development, including the novel use of chloroquine in the management of COVID-19.\n",
      "\n",
      "Abstract: \n",
      "Abstract The current coronavirus disease 2019 (COVID-19) pneumonia pandemic, caused by the severe acute respiratory syndrome 2 (SARS-CoV-2) virus, is spreading globally at an accelerated rate, with a basic reproduction number (R0) of 2 – 2.5, indicating that 2 – 3 persons will be infected from an index patient. A serious public health emergency, it is particularly deadly in vulnerable populations and communities in which healthcare providers are insufficiently prepared to manage the infection. As of March 16, 2020, there are more than 180,000 confirmed cases of COVID-19 worldwide, with over 7,000 related deaths. The SARS-CoV-2 virus has been isolated from asymptomatic individuals, and affected patients continue to be infectious two weeks after cessation of symptoms. The substantial morbidity and socioeconomic impact have necessitated drastic measures across all continents, including nationwide lockdowns and border closures. Pregnant women and their fetuses represent a high-risk population during infectious disease outbreaks. To date, the outcomes of 55 pregnant women infected with COVID-19 and 46 neonates have been reported in the literature, with no definite evidence of vertical transmission. Physiological and mechanical changes in pregnancy increase susceptibility to infections in general, particularly when the cardiorespiratory system is affected, and encourage rapid progression to respiratory failure in the gravida. Furthermore, the pregnancy bias towards T-helper 2 (Th2) system dominance which protects the fetus, leaves the mother vulnerable to viral infections, which are more effectively contained by the Th1 system. These unique challenges mandate an integrated approach to pregnancies affected by SARS-CoV-2. Here we present a review of COVID-19 in pregnancy, bringing together the various factors integral to the understanding of pathophysiology and susceptibility, diagnostic challenges with real-time reverse transcriptase polymerase chain reaction (RT-PCR) assays, therapeutic controversies, intrauterine transmission and maternal-fetal complications. We discuss the latest options in antiviral therapy and vaccine development, including the novel use of chloroquine in the management of COVID-19. Fetal surveillance, in view of the predisposition to growth restriction and special considerations during labor and delivery are addressed. Additionally, we focus on keeping frontline obstetric care providers safe while continuing to provide essential services. Our clinical service model is built around the principles of workplace segregation, responsible social distancing, containment of cross-infection to healthcare providers, judicious use of personal protective equipment and telemedicine. Our aim is to share a framework which can be adopted by tertiary maternity units managing pregnant women in the flux of a pandemic while maintaining the safety of the patient and healthcare provider at its core.\n",
      "\n",
      "Context: \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: 672865cb53271ddd7dd4a32a9c35801825b73fcb\n",
      "\n",
      "Title: Understand Research Hotspots Surrounding COVID-19 and Other Coronavirus Infections Using Topic Modeling\n",
      "\n",
      "Published Date: 2020-03-30\n",
      "\n",
      "Number of times cited: 0\n",
      "\n",
      "Antiviral Related Terms: vaccine\n",
      "\n",
      "Results: \n",
      "This study aimed to evaluate the characteristics of publications involving coronaviruses as well as COVID-19 by using a topic modeling analysis.\n",
      "\n",
      "We then analyzed the semantic relationships between topics and compared the topic distribution between COVID-19 and other CoV infections.\n",
      "\n",
      "Abstract: \n",
      "Background: Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is a virus that causes severe respiratory illness in humans, which eventually results in the current outbreak of novel coronavirus disease (COVID-19) around the world. The research community is interested to know what are the hotspots in coronavirus (CoV) research and how much is known about COVID-19. This study aimed to evaluate the characteristics of publications involving coronaviruses as well as COVID-19 by using a topic modeling analysis. Methods: We extracted all abstracts and retained the most informative words from the COVID-19 Open Research Dataset, which contains all the 35,092 pieces of coronavirus related literature published up to March 20, 2020. Using Latent Dirichlet Allocation modeling, we trained an eight-topic model from the corpus. We then analyzed the semantic relationships between topics and compared the topic distribution between COVID-19 and other CoV infections. Results: Eight topics emerged overall: clinical characterization, pathogenesis research, therapeutics research, epidemiological study, virus transmission, vaccines research, virus diagnostics, and viral genomics. It was observed that COVID-19 research puts more emphasis on clinical characterization, epidemiological study, and virus transmission at present. In contrast, topics about diagnostics, therapeutics, vaccines, genomics and pathogenesis only accounted for less than 10% or even 4% of all the COVID-19 publications, much lower than those of other CoV infections. Conclusions: These results identified knowledge gaps in the area of COVID-19 and offered directions for future research. Keywords: COVID-19, coronavirus, topic modeling, hotspots, text mining\n",
      "\n",
      "Context: \n",
      "Paragraph: 5\n",
      "The purpose of this work was to conduct LDA modeling for semantic and quantitative evaluations of the current status of literature on CoV infections as well as COVID-19, identify broad research topics and how these topics interact with one another. More importantly, this work can benefit COVID-19 research coordination by recognizing high priority scientific topics . We found that topics of clinical characterization, epidemiology, and virus transmission are hotspots for COVID-19 at present, while research on pathogenesis, therapeutics, virus diagnostics, vaccines and viral genomics are urgently needed.\n",
      " \n",
      "Paragraph: 18\n",
      "It's common that one article discusses more than one topic, i.e., the topics are not isolated but semantic-related. To identify the semantic relationships between topics, we assigned each article two topics with the highest probabilistic proportion, measured the relationships between two topics by using their co-occurrence statistics, and showed them vividly in a direct chord diagram. 8 interacting less with other topics.\n",
      " \n",
      "Paragraph: 26\n",
      "This present work has several limitations that must be acknowledged to apprehend the findings. First, as a computer model, topic modeling has difficulties with understanding nuances and subtext. We identified eight major topics with frequent appearances, whereas some specific topics such as virus natural history and human-animal interface were buried due to low proportions. Future work will attempt to develop an approach to detect and visualize the conceptual sub-domains. Second, it is also necessary to be aware that the granularity of terms used to label topics may vary a little for different topics. We have tried our best to keep them at the same level through carefully curating the top words in each topic, as well as reviewing text intention of the corresponding publications. Third, the LDA model only provides a cross-sectional profile of the coronavirus research, which is methodologically different from academic disciplines such as evidence-based medicine and clinical assessment that synthesizing evidence centered on a specific question. But this study may facilitate these disciplines by informing what is already available and what is urgently required for the COVID-19 research. Future research could also study the occurrence of topics over time and analyze links to historic events and virus characteristics, in order to better understand possible temporal patterns.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: 7817b03a433df4b3f93127feac4b0d8090c5b56c\n",
      "\n",
      "Title: A systematic review on the efficacy and safety of chloroquine for the treatment of COVID-19\n",
      "\n",
      "Published Date: 2020-03-10\n",
      "\n",
      "Number of times cited: 6\n",
      "\n",
      "Antiviral Related Terms: chloroquine\n",
      "\n",
      "Results: \n",
      "The aim of this systematic review was to summarize the evidence regarding chloroquine for the treatment of COVID-19.\n",
      "\n",
      "Abstract: \n",
      "Abstract Purpose COVID-19 (coronavirus disease 2019) is a public health emergency of international concern. As of this time, there is no known effective pharmaceutical treatment, although it is much needed for patient contracting the severe form of the disease. The aim of this systematic review was to summarize the evidence regarding chloroquine for the treatment of COVID-19. Methods PubMed, EMBASE, and three trial Registries were searched for studies on the use of chloroquine in patients with COVID-19. Results We included six articles (one narrative letter, one in-vitro study, one editorial, expert consensus paper, two national guideline documents) and 23 ongoing clinical trials in China. Chloroquine seems to be effective in limiting the replication of SARS-CoV-2 (virus causing COVID-19) in vitro. Conclusions There is rationale, pre-clinical evidence of effectiveness and evidence of safety from long-time clinical use for other indications to justify clinical research on chloroquine in patients with COVID-19. However, clinical use should either adhere to the Monitored Emergency Use of Unregistered Interventions (MEURI) framework or be ethically approved as a trial as stated by the World Health Organization. Safety data and data from high-quality clinical trials are urgently needed.\n",
      "\n",
      "Context: \n",
      "Paragraph: 2\n",
      "We performed a systematic review of the PubMed and EMBASE databases from inception to 1-March-2020 to find articles providing information on the efficacy and safety of chloroquine and chloroquinerelated formulations in patients with SARS-CoV-2 pneumonia and articles describing related in-vitro studies. As much of the data on COVID-19 are coming from Asia, no language restrictions were imposed (see detailed search strategy in Supplement 1). The search was expanded using a snowballing method applied to the references of retrieved papers. We also searched the Chinese Clinical Trial Registry, Clinicatrial. gov and the International Clinical Trials Registry Platform (WHO ICTRP) to identify ongoing trials. Two authors (AC, MI) independently screened the databases and the trial registries and extracted relevant information (MI, GI). Discrepancies and doubts about relevance of the sources were solved by consensus with two more authors (AG, SE). We did not register the systematic review protocol because we anticipated the very limited available evidence on the topic and due to the urgency of the matter. Journal of Critical Care xxx (2020) xxx\n",
      " \n",
      "Paragraph: 13\n",
      "There is sufficient pre-clinical rationale and evidence regarding the effectiveness of chloroquine for treatment of COVID-19 as well as evidence of safety from long-time use in clinical practice for other indications [3] to justify clinical research on the topic. The current circumstances justify prioritization of ethical review of study proposals above other, less pressing, research topics (i.e. fast track institutional ethical review). Although the use of chloroquine may be supported by expert opinion, clinical use of this drug in patients with COVID-19 should adhere to the MEURI framework or after ethical approval as a trial as stated by the WHO. Data from high-quality, coordinated, clinical trials coming from different locations worldwide are urgently needed.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: 7a08d433124b3fe418029dc663e7b0c268ab29fc\n",
      "\n",
      "Title: Contentious issues and evolving concepts in the clinical presentation and management of patients with COVID-19 infectionwith reference to use of therapeutic and other drugs used in Co-morbid diseases (Hypertension, diabetes etc)\n",
      "\n",
      "Published Date: 2020-06-30\n",
      "\n",
      "Number of times cited: 0\n",
      "\n",
      "Antiviral Related Terms: hydroxychloroquine; chloroquine; prophylaxis\n",
      "\n",
      "Results: \n",
      "Search terms were, COVID-19, treatment of coronavirus, COVID 19 and following terms; chloroquine, hydroxychloroquine, ibuprofen, ACE-inhibitors or angiotensin receptor blockers, cardiovascular disease, diarrhoea, liver, testis and gastrointestinal disease.\n",
      "\n",
      "Abstract: \n",
      "Abstract Background and aims Multiple issues in management of COVID have emerged, but confusion persists regarding rational interpretation. Aim of this brief review is to review these issues based on current literature. Methods This is a narrative review with Pubmed and Google Scholar search till 23 March 2020. Search terms were, COVID-19, treatment of coronavirus, COVID 19 and following terms; chloroquine, hydroxychloroquine, ibuprofen, ACE-inhibitors or angiotensin receptor blockers, cardiovascular disease, diarrhoea, liver, testis and gastrointestinal disease. Results We discuss evidence regarding role of chloroquine and hydroxychloroquine in treatment and prophylaxis, use of inhibitors of the renin angiotensin system, safety of ibuprofen, unusual clinical features like gastrointestinal symptoms and interpretation of tests for cardiac enzymes and biomarkers. Conclusions While our conclusions on management of COVID-19 patients with co-morbidities are based on current evidence, however, data is limited and there is immediate need for fast track research.\n",
      "\n",
      "Context: \n",
      "Paragraph: 1\n",
      "We systematically searched the PubMed database and Google Scholar till March 23, 2020 using the keywordsCOVID-19, treatment of coronavirus, COVID 19 and following terms; chloroquine, hydroxychloroquine, ibuprofen, ACE-inhibitors or angiotensin receptor blockers, cardiovascular disease, diarrhoea, liver, testis and gastrointestinal disease.We also accessed the and retrieved the full text of the relevant cross references from the search results.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: 995847926bde7c0022c3c2b8ac9f3f84756c221d\n",
      "\n",
      "Title: Experimental Treatment with Favipiravir for COVID-19: An Open-Label Control Study\n",
      "\n",
      "Published Date: 2020-03-18\n",
      "\n",
      "Number of times cited: 0\n",
      "\n",
      "Antiviral Related Terms: favipiravir; interferon; lopinavir; ritonavir\n",
      "\n",
      "Results: \n",
      "Herein, we examine the effects of Favipiravir (FPV) versus Lopinavir (LPV)/ritonavir (RTV) for the treatment of COVID-19.\n",
      "\n",
      "Abstract: \n",
      "Abstract An outbreak of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and its caused coronavirus disease 2019 (COVID-19) has been reported in China since December 2019. More than 16% of patients developed acute respiratory distress syndrome, and the fatality ratio was about 1%–2%. No specific treatment has been reported. Herein, we examine the effects of Favipiravir (FPV) versus Lopinavir (LPV)/ritonavir (RTV) for the treatment of COVID-19. Patients with laboratory-confirmed COVID-19 who received oral FPV (Day 1: 1600 mg twice daily; Days 2–14: 600 mg twice daily) plus interferon (IFN)-α by aerosol inhalation (5 million U twice daily) were included in the FPV arm of this study, whereas patients who were treated with LPV/RTV (Days 1–14: 400 mg/100 mg twice daily) plus IFN-α by aerosol inhalation (5 million U twice daily) were included in the control arm. Changes in chest computed tomography (CT), viral clearance, and drug safety were compared between the two groups. For the 35 patients enrolled in the FPV arm and the 45 patients in the control arm, all baseline characteristics were comparable between the two arms. A shorter viral clearance time was found for the FPV arm versus the control arm (median (interquartile range, IQR), 4 (2.5–9) d versus 11 (8–13) d, P < 0.001). The FPV arm also showed significant improvement in chest imaging compared with the control arm, with an improvement rate of 91.43% versus 62.22% (P = 0.004). After adjustment for potential confounders, the FPV arm also showed a significantly higher improvement rate in chest imaging. Multivariable Cox regression showed that FPV was independently associated with faster viral clearance. In addition, fewer adverse reactions were found in the FPV arm than in the control arm. In this open-label nonrandomized control study, FPV showed significantly better treatment effects on COVID-19 in terms of disease progression and viral clearance; if causal, these results should be important information for establishing standard treatment guidelines to combat the SARS-CoV-2 infection.\n",
      "\n",
      "Context: \n",
      "Paragraph: 1\n",
      "One efficient approach to discover effective drugs is to test whether the existing antiviral drugs are effective in treating other related viral infections. Several drugs, such as ribavirin, interferon (IFN), Favipiravir (FPV), and Lopinavir (LPV)/ritonavir (RTV), have been used in patients with SARS or MERS, although the efficacy of some drugs remains controversial. It has recently been demonstrated that, as a prodrug, FPV (half maximal effective concentration (EC 50 ) = 61.88 lmolÁL À1, half-maximal cytotoxic concentration (CC 50 ) > 400-lmolÁL À1, selectivity index (SI) > 6.46) effectively inhibits the SARS-CoV-2 infection in Vero E6 cells (ATCC-1586) [7] . Furthermore, other reports show that FPV is effective in protecting mice against Ebola virus challenge, although its EC 50 value in Vero E6 cells was as high as 67 lmolÁL À1 [8] . Therefore, clinical studies are urgently needed to evaluate the efficacy and safety of this antiviral nucleoside for COVID-19 treatment.\n",
      " \n",
      "Paragraph: 17\n",
      "This study investigated the effect of FPV versus LPV/RTV on the treatment of COVID-19. It was found that FPV was independently associated with faster viral clearance and a higher improvement rate in chest imaging. These findings suggest that FPV has significantly better treatment effects on COVID-19 in terms of disease progression and viral clearance, as compared with LPV/RTV. FPV, which is known as a prodrug, is a novel RNA-dependent RNA polymerase (RdRp) inhibitor, which has been shown to be effective in the treatment of influenza and Ebola virus [8, [11] [12] [13] [14] [15] . Recently, a report from Wang et al. [7] showed that both FPV and remdesivir are effective in reducing the SARS-CoV-2 infection in vitro (EC 50 = 61.88 lmolÁL À1, CC 50 > 400 lmolÁL À1, SI > 6.46). The finding of the preset study confirms the hypotheses conceived from the laboratory finding: that FPV is effective treatment for COVID-19.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: 9e94f9379fd74fcacc4f3a57e03cbe9035efee8e\n",
      "\n",
      "Title: Molecular Modeling Evaluation of the Binding Effect of Ritonavir, Lopinavir and Darunavir to Severe Acute Respiratory Syndrome Coronavirus 2 Proteases\n",
      "\n",
      "Published Date: 2020-02-03\n",
      "\n",
      "Number of times cited: 1\n",
      "\n",
      "Antiviral Related Terms: lopinavir; ritonavir; darunavir\n",
      "\n",
      "Results: \n",
      "AbstractThree anti-HIV drugs, ritonavir, lopinavir and darunavir, might have therapeutic effect on coronavirus disease 2019 (COVID-19).\n",
      "\n",
      "Abstract: \n",
      "AbstractThree anti-HIV drugs, ritonavir, lopinavir and darunavir, might have therapeutic effect on coronavirus disease 2019 (COVID-19). In this study, the structure models of two severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) proteases, coronavirus endopeptidase C30 (CEP_C30) and papain like viral protease (PLVP), were built by homology modeling. Ritonavir, lopinavir and darunavir were then docked to the models, respectively, followed by energy minimization of the protease-drug complexes. In the simulations, ritonavir can bind to CEP_C30 most suitably, and induce significant conformation changes of CEP_C30; lopinavir can also bind to CEP_C30 suitably, and induce significant conformation changes of CEP_C30; darunavir can bind to PLVP suitably with slight conformation changes of PLVP. It is suggested that the therapeutic effect of ritonavir and lopinavir on COVID-19 may be mainly due to their inhibitory effect on CEP_C30, while ritonavir may have stronger efficacy; the inhibitory effect of darunavir on SARS-CoV-2 and its potential therapeutic effect may be mainly due to its inhibitory effect on PLVP.\n",
      "\n",
      "Context: \n",
      "Paragraph: 1\n",
      "The trial fifth edition of diagnosis and treatment guideline of COVID-19 issued by National Health Commission of the People's Republic of China (http://www.nhc.gov.cn/) recommends to use Kaletra for treatment. Kaletra, an anti-HIV drug which is composed of two protease inhibitors, ritonavir (CAS#: 155213-67-5) and lopinavir (CAS#: 192725-17-0), might have therapeutic effect on coronavirus diseases like SARS and MERS [7] [8] [9] [10] . However, whether it can inhibit SARS-CoV-2 or treat COVID-19 lacks clinical evidences and randomized clinical trials, and the safety of its use in COVID-19 patients is unclear. Otherwise, Lanjuan Li, infectious disease scientist, academician of Chinese Academy of Engineering, recommended darunavir (CAS#: 206361-99-1), also an HIV protease inhibitor, as a treatment for COVID-19. Although the inhibitory effect of darunavir on SARS-CoV-2 has been verified in vitro, its therapeutic effect on COVID-19 is still unknown. At the same time, the mechanism of how these drugs inhibit SARS-CoV-2 is also unknown.\n",
      " \n",
      "Paragraph: 21\n",
      "In conclusion, the therapeutic effect of ritonavir and lopinavir on COVID-19 and other coronavirus disease may be mainly due to their inhibitory effect on CEP_C30, while ritonavir may have stronger efficacy; the inhibitory effect of darunavir on SARS-CoV-2 and its potential therapeutic effect may be mainly due to its inhibitory effect on PLVP. However, there are still limitations in this study. As the catalytic mechanisms of CEP_C30 and PLVP are still unknown, in further studies, it should still be focused on to figure out these mechanisms and how ritonavir, lopinavir and darunavir block these procedures. The safety and actual effects of these drugs on human body should also be carried out and verified by randomized clinical control studies further. author/funder. All rights reserved. No reuse allowed without permission.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: a0327c89dd46ce06ec8e979ddbad8c7b9c32660b\n",
      "\n",
      "Title: Temporal profiles of viral load in posterior oropharyngeal saliva samples and serum antibody responses during infection by SARS-CoV-2: an observational cohort study\n",
      "\n",
      "Published Date: 2020-03-23\n",
      "\n",
      "Number of times cited: 3\n",
      "\n",
      "Antiviral Related Terms: assay\n",
      "\n",
      "Results: \n",
      "We included patients with laboratory-confirmed COVID-19.\n",
      "\n",
      "Abstract: \n",
      "Summary Background Coronavirus disease 2019 (COVID-19) causes severe community and nosocomial outbreaks. Comprehensive data for serial respiratory viral load and serum antibody responses from patients infected with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) are not yet available. Nasopharyngeal and throat swabs are usually obtained for serial viral load monitoring of respiratory infections but gathering these specimens can cause discomfort for patients and put health-care workers at risk. We aimed to ascertain the serial respiratory viral load of SARS-CoV-2 in posterior oropharyngeal (deep throat) saliva samples from patients with COVID-19, and serum antibody responses. Methods We did a cohort study at two hospitals in Hong Kong. We included patients with laboratory-confirmed COVID-19. We obtained samples of blood, urine, posterior oropharyngeal saliva, and rectal swabs. Serial viral load was ascertained by reverse transcriptase quantitative PCR (RT-qPCR). Antibody levels against the SARS-CoV-2 internal nucleoprotein (NP) and surface spike protein receptor binding domain (RBD) were measured using EIA. Whole-genome sequencing was done to identify possible mutations arising during infection. Findings Between Jan 22, 2020, and Feb 12, 2020, 30 patients were screened for inclusion, of whom 23 were included (median age 62 years [range 37–75]). The median viral load in posterior oropharyngeal saliva or other respiratory specimens at presentation was 5·2 log10 copies per mL (IQR 4·1–7·0). Salivary viral load was highest during the first week after symptom onset and subsequently declined with time (slope −0·15, 95% CI −0·19 to −0·11; R 2=0·71). In one patient, viral RNA was detected 25 days after symptom onset. Older age was correlated with higher viral load (Spearman's ρ=0·48, 95% CI 0·074–0·75; p=0·020). For 16 patients with serum samples available 14 days or longer after symptom onset, rates of seropositivity were 94% for anti-NP IgG (n=15), 88% for anti-NP IgM (n=14), 100% for anti-RBD IgG (n=16), and 94% for anti-RBD IgM (n=15). Anti-SARS-CoV-2-NP or anti-SARS-CoV-2-RBD IgG levels correlated with virus neutralisation titre (R 2>0·9). No genome mutations were detected on serial samples. Interpretation Posterior oropharyngeal saliva samples are a non-invasive specimen more acceptable to patients and health-care workers. Unlike severe acute respiratory syndrome, patients with COVID-19 had the highest viral load near presentation, which could account for the fast-spreading nature of this epidemic. This finding emphasises the importance of stringent infection control and early use of potent antiviral agents, alone or in combination, for high-risk individuals. Serological assay can complement RT-qPCR for diagnosis. Funding Richard and Carol Yu, May Tam Mak Mei Yin, The Shaw Foundation Hong Kong, Michael Tong, Marina Lee, Government Consultancy Service, and Sanming Project of Medicine.\n",
      "\n",
      "Context: \n",
      "Paragraph: 1\n",
      "We included consecutive patients with laboratory confirmed COVID-19 who were admitted to Princess Margaret Hospital and Queen Mary Hospital in Hong Kong. In Hong Kong, patients were tested for SARS-CoV-2 based on clinical and epidemiological criteria as outlined and updated by the Hospital Authority. Initial laboratory confirmation was done using nasopharyngeal or sputum specimens at the Public Health Laboratory Centre of Hong Kong. We excluded patients if archived saliva or serum samples were insufficient for testing.\n",
      " \n",
      "Paragraph: 2\n",
      "This study was approved by the Institutional Review Board of the University of Hong Kong/Hospital Authority Hong Kong West Cluster (UW 13-372). Since archived specimens were used, written informed consent was waived. 12 of 23 patients included in this study have been reported previously, 6 but their clinical information, viral load by single copy RNA-dependent RNA polymerasehelicase gene, antibody response, or viral genome analysis has not been reported before.\n",
      " \n",
      "Paragraph: 11\n",
      "Between Jan 22, 2020, and Feb 12, 2020, 30 patients were screened for inclusion, of whom 23 were included (13 male and ten female). Ten patients had severe COVID-19, of whom all required oxygen supplementation, and 13 patients had mild disease. The median age of patients was 62 years (range 37-75). 11 (48%) of 23 patients had chronic medical illnesses, and the most common under lying diseases were hypertension in six (26%) patients and diabetes in four (17%). Chronic comorbidities were more common among patients with severe COVID-19 (seven [70%] patients with severe disease had chronic comorbidities vs four [31%] with mild disease), although this difference was not significant (table). Five patients were admitted to the ICU, including three who required intubation. Two patients died.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: a1ddaf941a4c1a0145ce1af098cf1d9da207ed35\n",
      "\n",
      "Title: Broad Spectrum Antiviral Agent Niclosamide and Its Therapeutic Potential\n",
      "\n",
      "Published Date: 2020-03-03\n",
      "\n",
      "Number of times cited: 2\n",
      "\n",
      "Antiviral Related Terms: niclosamide\n",
      "\n",
      "Results: \n",
      "In this brief review, we summarize the broad antiviral activity of niclosamide and highlight its potential clinical use in the treatment of COVID-19.\n",
      "\n",
      "Abstract: \n",
      "[Image: see text] The recent outbreak of coronavirus disease 2019 (COVID-19) highlights an urgent need for therapeutics. Through a series of drug repurposing screening campaigns, niclosamide, an FDA-approved anthelminthic drug, was found to be effective against various viral infections with nanomolar to micromolar potency such as SARS-CoV, MERS-CoV, ZIKV, HCV, and human adenovirus, indicating its potential as an antiviral agent. In this brief review, we summarize the broad antiviral activity of niclosamide and highlight its potential clinical use in the treatment of COVID-19.\n",
      "\n",
      "Context: \n",
      "Paragraph: 1\n",
      "Drug repurposing screens have emerged as an attractive strategy to accelerate new drug discovery and development. This strategy offers various advantages over de novo drug discovery featured with key benefits including reduced time, cost, and risk as well as the unique means for safer and more effective drugs to be accessed by patients. 8, 9 Niclosamide is an FDA-approved anthelminthic drug that has been widely used in humans to treat tapeworm infections for several decades and is currently listed on the World Health Organization's list of essential medicines. 10,11 Niclosamide exerts its anticestodal effect by inhibiting oxidative phosphorylation and stimulating adenosine triphosphatase activity in the mitochondria. 12 Over the past several years, niclosamide has been identified as a multifunctional drug via drug repurposing screens. It can regulate multiple signaling pathways and biological processes including Wnt/β-catenin, mTORC1, STAT3, NF-κB, Notch, NS2B-NS3 interaction, and pH, 13,14 indicating its potential to treat other human conditions 15 such as cancer, 16−18 bacterial and viral infections, 19−22 and metabolic diseases. 23 These broad biological activities of niclosamide including relevant cell signaling pathways were briefly reviewed by Chen et al. 15 In this short review, we focus on summarizing the broad antiviral activities of niclosamide ( Figure 1 ) and highlighting its therapeutic potential in combating COVID-19.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: db9b9f34fb37dfbeef7e0e900e834e982536de56\n",
      "\n",
      "Title: Reply to Gautret et al. 2020: A Bayesian reanalysis of the effects of hydroxychloroquine and azithromycin on viral carriage in patients with COVID-19\n",
      "\n",
      "Published Date: 2020-04-03\n",
      "\n",
      "Number of times cited: 0\n",
      "\n",
      "Antiviral Related Terms: azithromycin; hydroxychloroquine; chloroquine\n",
      "\n",
      "Results: \n",
      "This data has triggered speculation whether these drugs should be considered as candidates for the treatment of severe COVID-19.\n",
      "\n",
      "Abstract: \n",
      "Gautret and colleagues reported results of a non-randomised open-label case series which examined the effects of hydroxychloroquine and azithromycin on viral load in the upper respiratory tract of Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) patients. The authors report that hydroxychloroquine (HCQ) had significant virus reducing effects, and that dual treatment of both HCQ and azithromycin further enhanced virus reduction. This data has triggered speculation whether these drugs should be considered as candidates for the treatment of severe COVID-19. However, questions have been raised regarding the study's data integrity, statistical analyses, and experimental design. We therefore reanalysed the original data to interrogate the main claims of the paper. Here we apply Bayesian statistics to assess the robustness of the original papers claims by testing four variants of the data: 1) The original data; 2) Data including patients who deteriorated; 3) Data including patients who deteriorated with exclusion of untested patients in the comparison group; 4) Data that includes patients who deteriorated with the assumption that untested patients were negative. To ask if HCQ monotherapy is effective, we performed an A/B test for a model which assumes a positive effect, compared to a model of no effect. We find that the statistical evidence is highly sensitive to these data variants. Statistical evidence for the positive effect model ranged from strong for the original data (BF ~11), to moderate when including patients who deteriorated (BF ~4.35), to anecdotal when excluding untested patients (BF ~2), and to anecdotal negative evidence if untested patients were assumed positive (BF ~0.6). To assess whether HCQ is more effective when combined with AZ, we performed the same tests, and found only anecdotal evidence for the positive effect model for the original data (BF ~2.8), and moderate evidence for all other variants of the data (BF ~5.6). Our analyses only explore the effects of different assumptions about excluded and untested patients. These assumptions are not adequately reported, nor are they justified in the original paper, and we find that varying them causes substantive changes to the evidential support for the main claims of the original paper. This statistical uncertainty is exacerbated by the fact that the treatments were not randomised, and subject to several confounding variables including the patients' consent to treatment, different care centres, and clinical decision-making. Furthermore, while the viral load measurements were noisy, showing multiple reversals between test outcomes, there is greater certainty around other clinical outcomes such as the 4 patients who seriously deteriorated. The fact that all of these belonged to the HCQ monotherapy group should be assigned greater weight when evaluating the potential clinical efficacy of HCQ. Randomised controlled trials are currently underway, and will be critical in resolving this uncertainty as to whether HCQ and AZ are effective as a treatment for COVID-19.\n",
      "\n",
      "Context: \n",
      "Paragraph: 6\n",
      "(2) Data det includes deteriorated patients . It is questionable to exclude several patients who could not complete the treatment because their condition deteriorated. This could introduce a selection bias that inflates the effect of the treatment. We therefore modified the original data as follows: of the HCQ mono group, six were originally described as being excluded. One patient died, three deteriorated into intensive care, one patient stopped because of nausea and one left the hospital. These can be considered counterfactual cases that are necessary to entertain for a conservative estimate of the effects of HCQ mono . In the following, we add the patients who died or entered intensive care to the positive test cases for day 6. This is tabulated in Fig. 1a . We exclude both the patient that stopped treatment due to nausea, and the one patient who left the hospital, due to the ambiguity of their cases. This means that four cases are added to the HCQ mono that tested positive for SARS-CoV-2.\n",
      " \n",
      "Paragraph: 32\n",
      "We were able to qualitatively reproduce the finding of an improvement of HCQ +AZ over HCQ momo . However, although this finding was statistically significant in the original finding, our reanalysis revealed only anecdotal evidence for the positive effect of HCQ +AZ over HCQ momo . However, when we included the deteriorated patients into the analysis, this evidence increased to moderate. We also performed another test, which is to compare HCQ +AZ against the comparison group. It should be noted that this test is not the most relevant test because it varies two drugs at the same time. Nevertheless, the statistical evidence for the positive effect of the combined treatment over no treatment is very strong for the original data, and drops down to moderate when excluding the untested patients (Fig. 3) .\n",
      " \n",
      "Paragraph: 40\n",
      "Clinical safety. While the viral load measurements were noisy, showing multiple reversals between test outcomes, there is greater certainty around other clinical outcomes such as the 4 patients whose condition seriously deteriorated. It is important to stress that all of these belonged to the HCQ mono group, a fact that did not adequately temper the central claims of the original paper regarding the clinical potential of HCQ. Another way to state this would be that, though there is varying degrees of evidence for an effect of HCQ on viral load, it is known with greater certainty that all of the deteriorations occurred in the HCQ treatment group. Greater weight should be placed on this fact, when stating the possible clinical benefits of HCQ in the treatment of Covid-19.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: eabdc5478663e01b52476fa2988145532b0a5968\n",
      "\n",
      "Title: Nelfinavir inhibits replication of severe acute respiratory syndrome coronavirus 2 in vitro.\n",
      "\n",
      "Published Date: 2020-04-08\n",
      "\n",
      "Number of times cited: 0\n",
      "\n",
      "Antiviral Related Terms: nelfinavir; lopinavir\n",
      "\n",
      "Results: \n",
      "These results suggest that nelfinavir is a potential candidate drug for the treatment of COVID-19 and should be assessed in patients with COVID-19.\n",
      "\n",
      "Abstract: \n",
      "In December 2019, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) emerged in Wuhan, Hubei Province, China. No specific treatment has been established against coronavirus disease-2019 (COVID-19) so far. Therefore, it is urgently needed to identify effective antiviral agents for the treatment of this disease, and several approved drugs such as lopinavir have been evaluated. Here, we report that nelfinavir, an HIV-1 protease inhibitor, potently inhibited replication of SARS-CoV-2. The effective concentrations for 50% and 90% inhibition (EC50 and EC90) of nelfinavir were 1.13 micro M and 1.76 micro M respectively, the lowest of the nine HIV-1 protease inhibitors including lopinavir. The trough and peak serum concentrations of nelfinavir were three to six times higher than EC50 of this drug. These results suggest that nelfinavir is a potential candidate drug for the treatment of COVID-19 and should be assessed in patients with COVID-19.\n",
      "\n",
      "Context: \n",
      "Paragraph: 9\n",
      "These results suggest that nelfinavir, lopinavir, and tipranavir can achieve EC 50 of each drug in human and are effective in the treatment of COVID-19 patients.\n",
      " \n",
      "Paragraph: 12\n",
      "Our findings reveal that nelfinavir is highly effective in inhibiting SARS-CoV-2 replication in vitro and has the high C max -EC 50 and C tough -EC 50 ratio. We suggest that nelfinavir is a potential candidate drug for the treatment of COVID-19 and should be assessed in the patients with COVID-19. author/funder. All rights reserved. No reuse allowed without permission.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: eba1da4b90ee7cff2811391e12e773baa5cd6c2e\n",
      "\n",
      "Title: A brief review of antiviral drugs evaluated in registered clinical trials for COVID-19\n",
      "\n",
      "Published Date: 2020-03-20\n",
      "\n",
      "Number of times cited: 0\n",
      "\n",
      "Antiviral Related Terms: hydroxychloroquine; ritonavir; lopinavir; favipiravir; umifenovir; remdesivir; methylprednisolone; chloroquine\n",
      "\n",
      "Results: \n",
      "We included interventional clinical trials focusing on patients with COVID-19 and assessing antiviral drugs or agents.\n",
      "\n",
      "Abstract: \n",
      "Background: Although a number of antiviral agents have been evaluated for coronaviruses there are no approved drugs available. To provide an overview of the landscape of therapeutic research for COVID-19, we conducted a review of registered clinical trials. Methods: A review of currently registered clinical trials was performed on registries, including the Chinese (chictr.org.cn) and US (clinicaltrials.gov) databases to identify relevant studies up to March, 7th 2020. The search was conducted using the search terms “2019-nCoV”, “COVID-19”, “SARS-CoV-2”, “Hcov-19”, “new coronavirus”, “novel coronavirus”. We included interventional clinical trials focusing on patients with COVID-19 and assessing antiviral drugs or agents. Findings: Out of the 353 studies identified, 115 clinical trials were selected for data extraction. Phase IV trials were the most commonly reported study type (n=27, 23%). However, 62 trials (54%) did not describe the phase of the study. Eighty percent (n=92) of the trials were randomized with parallel assignment and the median number of planned inclusions was 63 (IQR, 36-120). Open-label studies were the most frequent (46%) followed by double-blind (13%) and single blind studies (10%). The most frequently assessed therapies were: stem cells therapy (n=23 trials), lopinavir/ritonavir (n=15), chloroquine (n=11), umifenovir (n=9), hydroxychloroquine (n=7), plasma treatment (n=7), favipiravir (n=7), methylprednisolone (n=5), and remdesivir (n=5). Remdesivir was tested in 5 trials with a median of 400 (IQR, 394-453) planned inclusions per trial, while stem cells therapy was tested in 23 trials, but had a median of 40 (IQR, 23-60) planned inclusions per trial. Lopinavir/ritonavir was associated with the highest total number of planned inclusions (2606) followed by remdesivir (2155). Only 52% of the clinical trials reported the treatment dose (n=60) and only 34% (n=39) the duration. The primary outcome was clinical in 76 studies (66%), virological in 27 (23%); radiological in 9 (8%) or immunological in three studies (3%). Interpretation: Numerous clinical trials have been registered since the beginning of the COVID-19 outbreak, however, a number of information regarding drugs or trial design were lacking. Funding: None\n",
      "\n",
      "Context: \n",
      "Paragraph: 5\n",
      " Population: patients with COVID-19,  Intervention/Comparator: any antiviral agent or drug. We excluded trials evaluating traditional Chinese medicine, homeopathy, dietary supplements, and therapeutic strategies whose description was not sufficient to identify a specific drug.  Outcomes: any outcomes,  Study type: interventional clinical trial. We excluded traditional Chinese medicine and homeopathy as we have no expertise to analyse clinical trials testing these agents that rely on controversial concepts. 10, 11 Dietary supplements were also excluded as their potential in treating COVID-19 seems limited.\n",
      " \n",
      "Paragraph: 13\n",
      "This review of ongoing clinical trials assessing COVID-19 treatments shows the important amount of research that is currently being conducted on this topic. However, although the number of trials identified is high, there are a number of caveats.\n",
      " \n",
      "Paragraph: 16\n",
      "Due to the pandemic context associated with COVID-19, the number of clinical trials registered is increasing day after day. A previous review conducted up to February 18 th, 2020 found 74 clinical trials evaluating antiviral agents or drugs. 36 Our review adds to this evidence by screening a larger number of clinical trial registries and reporting the studies design, randomization, allocation, and number of planned inclusions as well as treatment dose, duration, disease severity, and primary outcomes used.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: ef6f1dea64a581ce9f21fc860d2eea0042e1d921\n",
      "\n",
      "Title: Acetazolamide, Nifedipine and Phosphodiesterase Inhibitors: Rationale for Their Utilization as Adjunctive Countermeasures in the Treatment of Coronavirus Disease 2019 (COVID-19)\n",
      "\n",
      "Published Date: 2020-03-20\n",
      "\n",
      "Number of times cited: 0\n",
      "\n",
      "Antiviral Related Terms: vaccine\n",
      "\n",
      "Results: \n",
      "The utilization of medications found to be effective in HAPE, for the treatment of COVID-19, is proposed.\n",
      "\n",
      "Abstract: \n",
      "Effective treatments for Coronavirus Disease 2019 (COVID-19) outbreak are urgently needed. While anti-viral approaches and vaccines are being considered immediate countermeasures are unavailable. The aim of this article is to outline a perspective on the pathophysiology of COVID-19 in the context of the currently available clinical data published in the literature. This article appreciates clinical data published on COVID-19 in the context of another respiratory illness - high altitude pulmonary edema (HAPE). Both conditions have significant similarities that portend pathophysiologic trajectories. Following this potential treatment options emerge. Both COVID-19 and HAPE exhibit a decreased ratio of arterial oxygen partial pressure to fractional inspired oxygen with concomitant hypoxia and tachypnea. There also appears to be a tendency for low carbon dioxide levels in both as well. Radiologic findings of ground glass opacities are present in up to 86% of patients with COVID-19 in addition to patchy infiltrates. Patients with HAPE also exhibit patchy infiltrates throughout the pulmonary fields, often in an asymmetric pattern and CT findings reveal increased lung markings and ground glass-like changes as well. Widespread ground-glass opacities are most commonly a manifestation of hydrostatic pulmonary edema. Similarly, elevated fibrinogen levels in both conditions are likely an epiphenomenon of edema formation rather than coagulation activation. Autopsy results of a COVID-19 fatality revealed bilateral diffuse alveolar damage associated with pulmonary edema, pro-inflammatory concentrates, and indications of early-phase acute respiratory distress syndrome (ARDS). HAPE itself is initially caused by an increase in pulmonary capillary pressure and induces altered alveolar-capillary permeability via high pulmonary artery hydrostatic pressures that lead to a protein-rich and mildly hemorrhagic edema. It appears that COVID-19 and HAPE both discretely converge on ARDS. In light of this, a countermeasure that has been shown to be effective in the analogous condition of HAPE is Acetazolamide. Acetazolamide has a myriad of effects on different organ systems, potently reduces hypoxic pulmonary vasoconstriction, improves minute ventilation and expired vital capacity. Other therapeutics to consider that are also directed towards decreased pulmonary pressure include Nifedipine and Phosphodiesterase inhibitors. This review describes COVID-19 in parallel to HAPE. Deranged respiratory parameters that are present in both conditions are highlighted. The utilization of medications found to be effective in HAPE, for the treatment of COVID-19, is proposed. Given the medical emergency of a growing contagion and the thousands of lives at stake, expedient attempts to improve survival are needed. Acetazolamide, Nifedipine and Phosphodiesterase inhibitors may be potential countermeasures.\n",
      "\n",
      "Context: \n",
      "Paragraph: 13\n",
      "This review describes COVID-19 in parallel to HAPE. Deranged respiratory parameters that are present in both conditions are highlighted. The utilization of medications found to be effective in HAPE for the treatment of COVID-19 is proposed. Given the medical emergency of a growing contagion and the thousands of lives at stake, expedient attempts to improve survival are needed. Acetazolamide, Nifedipine and Phosphodiesterase inhibitors may present an opportunity for countermeasure development.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: f0b09e91c6b3d3a40d43110d256198b5c85a0d86\n",
      "\n",
      "Title: Potential Treatments for COVID-19; a Narrative Literature Review\n",
      "\n",
      "Published Date: 2020-03-21\n",
      "\n",
      "Number of times cited: 0\n",
      "\n",
      "Antiviral Related Terms: vaccine\n",
      "\n",
      "Results: \n",
      "Therefore, in this article, new potential COVID-19 therapies are briefly reviewed.\n",
      "\n",
      "Abstract: \n",
      "SARS-CoV-2 is a newly emerging human infectious coronavirus that causes COVID-19, which has been recognized as a pandemic by the World Health Organization (WHO) on March 11(th). There is still no vaccine or definitive treatment for this virus because its pathogenesis and proliferation pathways are still unknown. Therefore, in this article, new potential COVID-19 therapies are briefly reviewed.\n",
      "\n",
      "Context: \n",
      "Paragraph: 0\n",
      "SARS-CoV-2 (Severe Acute Respiratory Syndrome Coronavirus 2) is a newly emerging human infectious coronavirus, originated in Wuhan, China, and has been spreading rapidly in China and other countries since December 2019 (1) . The World Health Organization (WHO) also declared a global emergency on January 31 s t due to increasing concerns over its fast spread, and on March 11 t h the disease was recognized as a pandemic. Since the bases for pathogenesis of this virus and its proliferation is unclear, there is still no vaccine or definitive treatment against it. Thus, medications used against SARS-CoV-2 are mainly based on their effectiveness on earlier strains of coronavirus, SARS-CoV and MERS-CoV. Therefore, the immediate introduction of potential COVID-19 treatments can be essential and salvaging. In this article, new potential COVID-19 therapies are briefly reviewed.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n",
      "Query: antiviral treatments for COVID-19\n",
      "Sha: ff54e3e961a72eb1d2500166809b3651b2f98cf6\n",
      "\n",
      "Title: Predicting commercially available antiviral drugs that may act on the novel coronavirus (2019-nCoV), Wuhan, China through a drug-target interaction deep learning model\n",
      "\n",
      "Published Date: 2020-02-02\n",
      "\n",
      "Number of times cited: 1\n",
      "\n",
      "Antiviral Related Terms: lopinavir; ritonavir; darunavir\n",
      "\n",
      "Results: \n",
      "Overall, we suggest that the list of antiviral drugs identified by the MT-DTI model should be considered, when establishing effective treatment strategies for 2019-nCoV.\n",
      "\n",
      "Abstract: \n",
      "AbstractThe infection of a novel coronavirus found in Wuhan of China (2019-nCoV) is rapidly spreading, and the incidence rate is increasing worldwide. Due to the lack of effective treatment options for 2019-nCoV, various strategies are being tested in China, including drug repurposing. In this study, we used our pretrained deep learning-based drug-target interaction model called Molecule Transformer-Drug Target Interaction (MT-DTI) to identify commercially available drugs that could act on viral proteins of 2019-nCoV. The result showed that atazanavir, an antiretroviral medication used to treat and prevent the human immunodeficiency virus (HIV), is the best chemical compound, showing a inhibitory potency with Kd of 94.94 nM against the 2019-nCoV 3C-like proteinase, followed by efavirenz (199.17 nM), ritonavir (204.05 nM), and dolutegravir (336.91 nM). Interestingly, lopinavir, ritonavir, and darunavir are all designed to target viral proteinases. However, in our prediction, they may also bind to the replication complex components of 2019-nCoV with an inhibitory potency with Kd < 1000 nM. In addition, we also found that several antiviral agents, such as Kaletra, could be used for the treatment of 2019-nCoV, although there is no real-world evidence supporting the prediction. Overall, we suggest that the list of antiviral drugs identified by the MT-DTI model should be considered, when establishing effective treatment strategies for 2019-nCoV.\n",
      "\n",
      "Context: \n",
      "Paragraph: 2\n",
      "In this study, we applied our pre-trained MT-DTI model to identify commercially available antiviral drugs that could potentially disrupt 2019-nCoV's viral components, such as proteinase, RNA-dependent RNA polymerase, and/or helicase. Since the model utilizes simplified molecular-input line-entry system (SMILES) strings and amino acid (AA) sequences, which are 1D string inputs, it is possible to quickly apply target proteins that do not have experimentally confirmed 3D crystal structures, such as viral proteins of 2019-nCoV. We share a list of top commercially available antiviral drugs that could potentially hinder the multiplication cycle of 2019-nCoV with the hope that effective drugs can be developed based on these AI-proposed drug candidates and act against 2019-nCoV.\n",
      " \n",
      "\n",
      "\n",
      "----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sha</th>\n",
       "      <th>result</th>\n",
       "      <th>context</th>\n",
       "      <th>title</th>\n",
       "      <th>publish_time</th>\n",
       "      <th>abstract</th>\n",
       "      <th>keywords</th>\n",
       "      <th>journal</th>\n",
       "      <th>number_citations</th>\n",
       "      <th>query</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0370abacf3b212bd6bded8f23c8c904000f6c2e6</td>\n",
       "      <td>The objective of this study is to investigate ...</td>\n",
       "      <td>Paragraph: 1\\nHowever, until now, there are no...</td>\n",
       "      <td>Risk-adapted Treatment Strategy For COVID-19 P...</td>\n",
       "      <td>2020-03-27</td>\n",
       "      <td>Abstract Background There are no clear expert ...</td>\n",
       "      <td>methylprednisolone</td>\n",
       "      <td>International Journal of Infectious Diseases</td>\n",
       "      <td>0</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>06a1002f9fbea7179ac3572843f66b14568af6e4</td>\n",
       "      <td>2019-nCov Mpro is a potential drug target to c...</td>\n",
       "      <td>Paragraph: 16\\n2019-nCov caused more than 80 d...</td>\n",
       "      <td>Nelfinavir was predicted to be a potential inh...</td>\n",
       "      <td>2020-01-28</td>\n",
       "      <td>Abstract2019-nCov has caused more than 80 deat...</td>\n",
       "      <td>nelfinavir</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0c9d951acb01cb541671b3065b882bbcb61f9523</td>\n",
       "      <td>In addition, registered trials investigating t...</td>\n",
       "      <td>Paragraph: 6\\nThere is no current evidence fro...</td>\n",
       "      <td>The epidemiology, diagnosis and treatment of C...</td>\n",
       "      <td>2020-03-28</td>\n",
       "      <td>Abstract In December 2019, the outbreak of the...</td>\n",
       "      <td>convalescent plasma; hydroxychloroquine; vacci...</td>\n",
       "      <td>International Journal of Antimicrobial Agents</td>\n",
       "      <td>0</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>188e7ff1e260864c89f266b5597de26d69a84660</td>\n",
       "      <td>WHO has named the novel coronavirus disease as...</td>\n",
       "      <td>Paragraph: 0\\nThe World Health Organization (W...</td>\n",
       "      <td>Clinical trials on drug repositioning for COVI...</td>\n",
       "      <td>2020-03-20</td>\n",
       "      <td>The World Health Organization (WHO) was inform...</td>\n",
       "      <td>oseltamivir; hydroxychloroquine; arbidol; rito...</td>\n",
       "      <td>Rev Panam Salud Publica</td>\n",
       "      <td>0</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>188f3e97042155ac1709aa8b74c0755760c3b50d</td>\n",
       "      <td>This review focuses on the effects of these dr...</td>\n",
       "      <td>Paragraph: 1\\nThe current review focuses on th...</td>\n",
       "      <td>Associations between immune-suppressive and st...</td>\n",
       "      <td>2020-03-27</td>\n",
       "      <td>BACKGROUND: Cancer and transplant patients wit...</td>\n",
       "      <td>jak</td>\n",
       "      <td>Ecancermedicalscience</td>\n",
       "      <td>0</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>19a08cc1423d6006b2251bdcfd142f88db7002e7</td>\n",
       "      <td>Many drugs showed potential for COVID-19 therapy.</td>\n",
       "      <td>Paragraph: 1\\nThis promotes the importance of ...</td>\n",
       "      <td>Genetic Profiles in Pharmacogenes Indicate Per...</td>\n",
       "      <td>2020-03-30</td>\n",
       "      <td>Background: The coronavirus disease 2019 (COVI...</td>\n",
       "      <td>lopinavir; chloroquine; interferon</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1f5c1597a84ed1d4f84c488cd19098a091a3d513; 29d1...</td>\n",
       "      <td>Indeed, age and disease severity may be correl...</td>\n",
       "      <td>NA</td>\n",
       "      <td>Asymptomatic carrier state, acute respiratory ...</td>\n",
       "      <td>2020</td>\n",
       "      <td>Since the emergence of coronavirus disease 201...</td>\n",
       "      <td>chloroquine; remdesivir</td>\n",
       "      <td>Journal of Microbiology, Immunology and Infection</td>\n",
       "      <td>0</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>23e7355b5e4e0209f64c9d8d5772092a53b72686</td>\n",
       "      <td>Aims: Studies have indicated that chloroquine ...</td>\n",
       "      <td>Paragraph: 2\\nThe recent publication of result...</td>\n",
       "      <td>Efficacy of hydroxychloroquine in patients wit...</td>\n",
       "      <td>2020-03-30</td>\n",
       "      <td>Aims: Studies have indicated that chloroquine ...</td>\n",
       "      <td>hydroxychloroquine; chloroquine</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2f547947bf87380c7fab13ba2c663bbbe9e643ec</td>\n",
       "      <td>Here, we report the epidemiological and virolo...</td>\n",
       "      <td>Paragraph: 0\\nThe recent outbreak of COVID-19 ...</td>\n",
       "      <td>COVID-19: Epidemiology, Evolution, and Cross-D...</td>\n",
       "      <td>2020-03-21</td>\n",
       "      <td>The recent outbreak of COVID-19 in Wuhan turne...</td>\n",
       "      <td>vaccine</td>\n",
       "      <td>Trends in Molecular Medicine</td>\n",
       "      <td>0</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>5998e7fb99b5c58f7563017ad679fbe2f9974a0c</td>\n",
       "      <td>OBJECTIVE: To determine the relative impact of...</td>\n",
       "      <td>Paragraph: 3\\nNotwithstanding, despite its in ...</td>\n",
       "      <td>Chloroquine and hydroxychloroquine for the tre...</td>\n",
       "      <td>2020-04-08</td>\n",
       "      <td>OBJECTIVE: To determine the relative impact of...</td>\n",
       "      <td>hydroxychloroquine; chloroquine</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>5d13ba76e768fe46ecdf4ad9d2ba10ff7b9bcc48</td>\n",
       "      <td>Based upon our limited knowledge on the pathog...</td>\n",
       "      <td>Paragraph: 0\\nThe Coronavirus-associated disea...</td>\n",
       "      <td>Recommendations for coronavirus infection in r...</td>\n",
       "      <td>2020-04-02</td>\n",
       "      <td>Abstract The Coronavirus-associated disease, t...</td>\n",
       "      <td>vaccine; tocilizumab</td>\n",
       "      <td>Journal of Autoimmunity</td>\n",
       "      <td>0</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>5dc4268a42adf3d5c55c87b7f6518de600b057c5</td>\n",
       "      <td>We discuss the latest options in antiviral the...</td>\n",
       "      <td></td>\n",
       "      <td>Coronavirus Disease 2019 (COVID-19) Pandemic a...</td>\n",
       "      <td>2020-03-23</td>\n",
       "      <td>Abstract The current coronavirus disease 2019 ...</td>\n",
       "      <td>th1; vaccine; th2; assay; chloroquine</td>\n",
       "      <td>American Journal of Obstetrics and Gynecology</td>\n",
       "      <td>1</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>672865cb53271ddd7dd4a32a9c35801825b73fcb</td>\n",
       "      <td>This study aimed to evaluate the characteristi...</td>\n",
       "      <td>Paragraph: 5\\nThe purpose of this work was to ...</td>\n",
       "      <td>Understand Research Hotspots Surrounding COVID...</td>\n",
       "      <td>2020-03-30</td>\n",
       "      <td>Background: Severe acute respiratory syndrome ...</td>\n",
       "      <td>vaccine</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>7817b03a433df4b3f93127feac4b0d8090c5b56c</td>\n",
       "      <td>The aim of this systematic review was to summa...</td>\n",
       "      <td>Paragraph: 2\\nWe performed a systematic review...</td>\n",
       "      <td>A systematic review on the efficacy and safety...</td>\n",
       "      <td>2020-03-10</td>\n",
       "      <td>Abstract Purpose COVID-19 (coronavirus disease...</td>\n",
       "      <td>chloroquine</td>\n",
       "      <td>Journal of Critical Care</td>\n",
       "      <td>6</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>7a08d433124b3fe418029dc663e7b0c268ab29fc</td>\n",
       "      <td>Search terms were, COVID-19, treatment of coro...</td>\n",
       "      <td>Paragraph: 1\\nWe systematically searched the P...</td>\n",
       "      <td>Contentious issues and evolving concepts in th...</td>\n",
       "      <td>2020-06-30</td>\n",
       "      <td>Abstract Background and aims Multiple issues i...</td>\n",
       "      <td>hydroxychloroquine; chloroquine; prophylaxis</td>\n",
       "      <td>Diabetes &amp; Metabolic Syndrome: Clinical Resear...</td>\n",
       "      <td>0</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>995847926bde7c0022c3c2b8ac9f3f84756c221d</td>\n",
       "      <td>Herein, we examine the effects of Favipiravir ...</td>\n",
       "      <td>Paragraph: 1\\nOne efficient approach to discov...</td>\n",
       "      <td>Experimental Treatment with Favipiravir for CO...</td>\n",
       "      <td>2020-03-18</td>\n",
       "      <td>Abstract An outbreak of severe acute respirato...</td>\n",
       "      <td>favipiravir; interferon; lopinavir; ritonavir</td>\n",
       "      <td>Engineering</td>\n",
       "      <td>0</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>9e94f9379fd74fcacc4f3a57e03cbe9035efee8e</td>\n",
       "      <td>AbstractThree anti-HIV drugs, ritonavir, lopin...</td>\n",
       "      <td>Paragraph: 1\\nThe trial fifth edition of diagn...</td>\n",
       "      <td>Molecular Modeling Evaluation of the Binding E...</td>\n",
       "      <td>2020-02-03</td>\n",
       "      <td>AbstractThree anti-HIV drugs, ritonavir, lopin...</td>\n",
       "      <td>lopinavir; ritonavir; darunavir</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>a0327c89dd46ce06ec8e979ddbad8c7b9c32660b</td>\n",
       "      <td>We included patients with laboratory-confirmed...</td>\n",
       "      <td>Paragraph: 1\\nWe included consecutive patients...</td>\n",
       "      <td>Temporal profiles of viral load in posterior o...</td>\n",
       "      <td>2020-03-23</td>\n",
       "      <td>Summary Background Coronavirus disease 2019 (C...</td>\n",
       "      <td>assay</td>\n",
       "      <td>The Lancet Infectious Diseases</td>\n",
       "      <td>3</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>a1ddaf941a4c1a0145ce1af098cf1d9da207ed35</td>\n",
       "      <td>In this brief review, we summarize the broad a...</td>\n",
       "      <td>Paragraph: 1\\nDrug repurposing screens have em...</td>\n",
       "      <td>Broad Spectrum Antiviral Agent Niclosamide and...</td>\n",
       "      <td>2020-03-03</td>\n",
       "      <td>[Image: see text] The recent outbreak of coron...</td>\n",
       "      <td>niclosamide</td>\n",
       "      <td>ACS Infect Dis</td>\n",
       "      <td>2</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>db9b9f34fb37dfbeef7e0e900e834e982536de56</td>\n",
       "      <td>This data has triggered speculation whether th...</td>\n",
       "      <td>Paragraph: 6\\n(2) Data det includes deteriorat...</td>\n",
       "      <td>Reply to Gautret et al. 2020: A Bayesian reana...</td>\n",
       "      <td>2020-04-03</td>\n",
       "      <td>Gautret and colleagues reported results of a n...</td>\n",
       "      <td>azithromycin; hydroxychloroquine; chloroquine</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>eabdc5478663e01b52476fa2988145532b0a5968</td>\n",
       "      <td>These results suggest that nelfinavir is a pot...</td>\n",
       "      <td>Paragraph: 9\\nThese results suggest that nelfi...</td>\n",
       "      <td>Nelfinavir inhibits replication of severe acut...</td>\n",
       "      <td>2020-04-08</td>\n",
       "      <td>In December 2019, severe acute respiratory syn...</td>\n",
       "      <td>nelfinavir; lopinavir</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>eba1da4b90ee7cff2811391e12e773baa5cd6c2e</td>\n",
       "      <td>We included interventional clinical trials foc...</td>\n",
       "      <td>Paragraph: 5\\n Population: patients with COVI...</td>\n",
       "      <td>A brief review of antiviral drugs evaluated in...</td>\n",
       "      <td>2020-03-20</td>\n",
       "      <td>Background: Although a number of antiviral age...</td>\n",
       "      <td>hydroxychloroquine; ritonavir; lopinavir; favi...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>ef6f1dea64a581ce9f21fc860d2eea0042e1d921</td>\n",
       "      <td>The utilization of medications found to be eff...</td>\n",
       "      <td>Paragraph: 13\\nThis review describes COVID-19 ...</td>\n",
       "      <td>Acetazolamide, Nifedipine and Phosphodiesteras...</td>\n",
       "      <td>2020-03-20</td>\n",
       "      <td>Effective treatments for Coronavirus Disease 2...</td>\n",
       "      <td>vaccine</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>f0b09e91c6b3d3a40d43110d256198b5c85a0d86</td>\n",
       "      <td>Therefore, in this article, new potential COVI...</td>\n",
       "      <td>Paragraph: 0\\nSARS-CoV-2 (Severe Acute Respira...</td>\n",
       "      <td>Potential Treatments for COVID-19; a Narrative...</td>\n",
       "      <td>2020-03-21</td>\n",
       "      <td>SARS-CoV-2 is a newly emerging human infectiou...</td>\n",
       "      <td>vaccine</td>\n",
       "      <td>Arch Acad Emerg Med</td>\n",
       "      <td>0</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>ff54e3e961a72eb1d2500166809b3651b2f98cf6</td>\n",
       "      <td>Overall, we suggest that the list of antiviral...</td>\n",
       "      <td>Paragraph: 2\\nIn this study, we applied our pr...</td>\n",
       "      <td>Predicting commercially available antiviral dr...</td>\n",
       "      <td>2020-02-02</td>\n",
       "      <td>AbstractThe infection of a novel coronavirus f...</td>\n",
       "      <td>lopinavir; ritonavir; darunavir</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>antiviral treatments for COVID-19</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                  sha  \\\n",
       "0            0370abacf3b212bd6bded8f23c8c904000f6c2e6   \n",
       "1            06a1002f9fbea7179ac3572843f66b14568af6e4   \n",
       "2            0c9d951acb01cb541671b3065b882bbcb61f9523   \n",
       "3            188e7ff1e260864c89f266b5597de26d69a84660   \n",
       "4            188f3e97042155ac1709aa8b74c0755760c3b50d   \n",
       "5            19a08cc1423d6006b2251bdcfd142f88db7002e7   \n",
       "6   1f5c1597a84ed1d4f84c488cd19098a091a3d513; 29d1...   \n",
       "7            23e7355b5e4e0209f64c9d8d5772092a53b72686   \n",
       "8            2f547947bf87380c7fab13ba2c663bbbe9e643ec   \n",
       "9            5998e7fb99b5c58f7563017ad679fbe2f9974a0c   \n",
       "10           5d13ba76e768fe46ecdf4ad9d2ba10ff7b9bcc48   \n",
       "11           5dc4268a42adf3d5c55c87b7f6518de600b057c5   \n",
       "12           672865cb53271ddd7dd4a32a9c35801825b73fcb   \n",
       "13           7817b03a433df4b3f93127feac4b0d8090c5b56c   \n",
       "14           7a08d433124b3fe418029dc663e7b0c268ab29fc   \n",
       "15           995847926bde7c0022c3c2b8ac9f3f84756c221d   \n",
       "16           9e94f9379fd74fcacc4f3a57e03cbe9035efee8e   \n",
       "17           a0327c89dd46ce06ec8e979ddbad8c7b9c32660b   \n",
       "18           a1ddaf941a4c1a0145ce1af098cf1d9da207ed35   \n",
       "19           db9b9f34fb37dfbeef7e0e900e834e982536de56   \n",
       "20           eabdc5478663e01b52476fa2988145532b0a5968   \n",
       "21           eba1da4b90ee7cff2811391e12e773baa5cd6c2e   \n",
       "22           ef6f1dea64a581ce9f21fc860d2eea0042e1d921   \n",
       "23           f0b09e91c6b3d3a40d43110d256198b5c85a0d86   \n",
       "24           ff54e3e961a72eb1d2500166809b3651b2f98cf6   \n",
       "\n",
       "                                               result  \\\n",
       "0   The objective of this study is to investigate ...   \n",
       "1   2019-nCov Mpro is a potential drug target to c...   \n",
       "2   In addition, registered trials investigating t...   \n",
       "3   WHO has named the novel coronavirus disease as...   \n",
       "4   This review focuses on the effects of these dr...   \n",
       "5   Many drugs showed potential for COVID-19 therapy.   \n",
       "6   Indeed, age and disease severity may be correl...   \n",
       "7   Aims: Studies have indicated that chloroquine ...   \n",
       "8   Here, we report the epidemiological and virolo...   \n",
       "9   OBJECTIVE: To determine the relative impact of...   \n",
       "10  Based upon our limited knowledge on the pathog...   \n",
       "11  We discuss the latest options in antiviral the...   \n",
       "12  This study aimed to evaluate the characteristi...   \n",
       "13  The aim of this systematic review was to summa...   \n",
       "14  Search terms were, COVID-19, treatment of coro...   \n",
       "15  Herein, we examine the effects of Favipiravir ...   \n",
       "16  AbstractThree anti-HIV drugs, ritonavir, lopin...   \n",
       "17  We included patients with laboratory-confirmed...   \n",
       "18  In this brief review, we summarize the broad a...   \n",
       "19  This data has triggered speculation whether th...   \n",
       "20  These results suggest that nelfinavir is a pot...   \n",
       "21  We included interventional clinical trials foc...   \n",
       "22  The utilization of medications found to be eff...   \n",
       "23  Therefore, in this article, new potential COVI...   \n",
       "24  Overall, we suggest that the list of antiviral...   \n",
       "\n",
       "                                              context  \\\n",
       "0   Paragraph: 1\\nHowever, until now, there are no...   \n",
       "1   Paragraph: 16\\n2019-nCov caused more than 80 d...   \n",
       "2   Paragraph: 6\\nThere is no current evidence fro...   \n",
       "3   Paragraph: 0\\nThe World Health Organization (W...   \n",
       "4   Paragraph: 1\\nThe current review focuses on th...   \n",
       "5   Paragraph: 1\\nThis promotes the importance of ...   \n",
       "6                                                  NA   \n",
       "7   Paragraph: 2\\nThe recent publication of result...   \n",
       "8   Paragraph: 0\\nThe recent outbreak of COVID-19 ...   \n",
       "9   Paragraph: 3\\nNotwithstanding, despite its in ...   \n",
       "10  Paragraph: 0\\nThe Coronavirus-associated disea...   \n",
       "11                                                      \n",
       "12  Paragraph: 5\\nThe purpose of this work was to ...   \n",
       "13  Paragraph: 2\\nWe performed a systematic review...   \n",
       "14  Paragraph: 1\\nWe systematically searched the P...   \n",
       "15  Paragraph: 1\\nOne efficient approach to discov...   \n",
       "16  Paragraph: 1\\nThe trial fifth edition of diagn...   \n",
       "17  Paragraph: 1\\nWe included consecutive patients...   \n",
       "18  Paragraph: 1\\nDrug repurposing screens have em...   \n",
       "19  Paragraph: 6\\n(2) Data det includes deteriorat...   \n",
       "20  Paragraph: 9\\nThese results suggest that nelfi...   \n",
       "21  Paragraph: 5\\n Population: patients with COVI...   \n",
       "22  Paragraph: 13\\nThis review describes COVID-19 ...   \n",
       "23  Paragraph: 0\\nSARS-CoV-2 (Severe Acute Respira...   \n",
       "24  Paragraph: 2\\nIn this study, we applied our pr...   \n",
       "\n",
       "                                                title publish_time  \\\n",
       "0   Risk-adapted Treatment Strategy For COVID-19 P...   2020-03-27   \n",
       "1   Nelfinavir was predicted to be a potential inh...   2020-01-28   \n",
       "2   The epidemiology, diagnosis and treatment of C...   2020-03-28   \n",
       "3   Clinical trials on drug repositioning for COVI...   2020-03-20   \n",
       "4   Associations between immune-suppressive and st...   2020-03-27   \n",
       "5   Genetic Profiles in Pharmacogenes Indicate Per...   2020-03-30   \n",
       "6   Asymptomatic carrier state, acute respiratory ...         2020   \n",
       "7   Efficacy of hydroxychloroquine in patients wit...   2020-03-30   \n",
       "8   COVID-19: Epidemiology, Evolution, and Cross-D...   2020-03-21   \n",
       "9   Chloroquine and hydroxychloroquine for the tre...   2020-04-08   \n",
       "10  Recommendations for coronavirus infection in r...   2020-04-02   \n",
       "11  Coronavirus Disease 2019 (COVID-19) Pandemic a...   2020-03-23   \n",
       "12  Understand Research Hotspots Surrounding COVID...   2020-03-30   \n",
       "13  A systematic review on the efficacy and safety...   2020-03-10   \n",
       "14  Contentious issues and evolving concepts in th...   2020-06-30   \n",
       "15  Experimental Treatment with Favipiravir for CO...   2020-03-18   \n",
       "16  Molecular Modeling Evaluation of the Binding E...   2020-02-03   \n",
       "17  Temporal profiles of viral load in posterior o...   2020-03-23   \n",
       "18  Broad Spectrum Antiviral Agent Niclosamide and...   2020-03-03   \n",
       "19  Reply to Gautret et al. 2020: A Bayesian reana...   2020-04-03   \n",
       "20  Nelfinavir inhibits replication of severe acut...   2020-04-08   \n",
       "21  A brief review of antiviral drugs evaluated in...   2020-03-20   \n",
       "22  Acetazolamide, Nifedipine and Phosphodiesteras...   2020-03-20   \n",
       "23  Potential Treatments for COVID-19; a Narrative...   2020-03-21   \n",
       "24  Predicting commercially available antiviral dr...   2020-02-02   \n",
       "\n",
       "                                             abstract  \\\n",
       "0   Abstract Background There are no clear expert ...   \n",
       "1   Abstract2019-nCov has caused more than 80 deat...   \n",
       "2   Abstract In December 2019, the outbreak of the...   \n",
       "3   The World Health Organization (WHO) was inform...   \n",
       "4   BACKGROUND: Cancer and transplant patients wit...   \n",
       "5   Background: The coronavirus disease 2019 (COVI...   \n",
       "6   Since the emergence of coronavirus disease 201...   \n",
       "7   Aims: Studies have indicated that chloroquine ...   \n",
       "8   The recent outbreak of COVID-19 in Wuhan turne...   \n",
       "9   OBJECTIVE: To determine the relative impact of...   \n",
       "10  Abstract The Coronavirus-associated disease, t...   \n",
       "11  Abstract The current coronavirus disease 2019 ...   \n",
       "12  Background: Severe acute respiratory syndrome ...   \n",
       "13  Abstract Purpose COVID-19 (coronavirus disease...   \n",
       "14  Abstract Background and aims Multiple issues i...   \n",
       "15  Abstract An outbreak of severe acute respirato...   \n",
       "16  AbstractThree anti-HIV drugs, ritonavir, lopin...   \n",
       "17  Summary Background Coronavirus disease 2019 (C...   \n",
       "18  [Image: see text] The recent outbreak of coron...   \n",
       "19  Gautret and colleagues reported results of a n...   \n",
       "20  In December 2019, severe acute respiratory syn...   \n",
       "21  Background: Although a number of antiviral age...   \n",
       "22  Effective treatments for Coronavirus Disease 2...   \n",
       "23  SARS-CoV-2 is a newly emerging human infectiou...   \n",
       "24  AbstractThe infection of a novel coronavirus f...   \n",
       "\n",
       "                                             keywords  \\\n",
       "0                                  methylprednisolone   \n",
       "1                                          nelfinavir   \n",
       "2   convalescent plasma; hydroxychloroquine; vacci...   \n",
       "3   oseltamivir; hydroxychloroquine; arbidol; rito...   \n",
       "4                                                 jak   \n",
       "5                  lopinavir; chloroquine; interferon   \n",
       "6                             chloroquine; remdesivir   \n",
       "7                     hydroxychloroquine; chloroquine   \n",
       "8                                             vaccine   \n",
       "9                     hydroxychloroquine; chloroquine   \n",
       "10                               vaccine; tocilizumab   \n",
       "11              th1; vaccine; th2; assay; chloroquine   \n",
       "12                                            vaccine   \n",
       "13                                        chloroquine   \n",
       "14       hydroxychloroquine; chloroquine; prophylaxis   \n",
       "15      favipiravir; interferon; lopinavir; ritonavir   \n",
       "16                    lopinavir; ritonavir; darunavir   \n",
       "17                                              assay   \n",
       "18                                        niclosamide   \n",
       "19      azithromycin; hydroxychloroquine; chloroquine   \n",
       "20                              nelfinavir; lopinavir   \n",
       "21  hydroxychloroquine; ritonavir; lopinavir; favi...   \n",
       "22                                            vaccine   \n",
       "23                                            vaccine   \n",
       "24                    lopinavir; ritonavir; darunavir   \n",
       "\n",
       "                                              journal  number_citations  \\\n",
       "0        International Journal of Infectious Diseases                 0   \n",
       "1                                                 NaN                 4   \n",
       "2       International Journal of Antimicrobial Agents                 0   \n",
       "3                             Rev Panam Salud Publica                 0   \n",
       "4                               Ecancermedicalscience                 0   \n",
       "5                                                 NaN                 0   \n",
       "6   Journal of Microbiology, Immunology and Infection                 0   \n",
       "7                                                 NaN                 1   \n",
       "8                        Trends in Molecular Medicine                 0   \n",
       "9                                                 NaN                 0   \n",
       "10                            Journal of Autoimmunity                 0   \n",
       "11      American Journal of Obstetrics and Gynecology                 1   \n",
       "12                                                NaN                 0   \n",
       "13                           Journal of Critical Care                 6   \n",
       "14  Diabetes & Metabolic Syndrome: Clinical Resear...                 0   \n",
       "15                                        Engineering                 0   \n",
       "16                                                NaN                 1   \n",
       "17                     The Lancet Infectious Diseases                 3   \n",
       "18                                     ACS Infect Dis                 2   \n",
       "19                                                NaN                 0   \n",
       "20                                                NaN                 0   \n",
       "21                                                NaN                 0   \n",
       "22                                                NaN                 0   \n",
       "23                                Arch Acad Emerg Med                 0   \n",
       "24                                                NaN                 1   \n",
       "\n",
       "                                query  \n",
       "0   antiviral treatments for COVID-19  \n",
       "1   antiviral treatments for COVID-19  \n",
       "2   antiviral treatments for COVID-19  \n",
       "3   antiviral treatments for COVID-19  \n",
       "4   antiviral treatments for COVID-19  \n",
       "5   antiviral treatments for COVID-19  \n",
       "6   antiviral treatments for COVID-19  \n",
       "7   antiviral treatments for COVID-19  \n",
       "8   antiviral treatments for COVID-19  \n",
       "9   antiviral treatments for COVID-19  \n",
       "10  antiviral treatments for COVID-19  \n",
       "11  antiviral treatments for COVID-19  \n",
       "12  antiviral treatments for COVID-19  \n",
       "13  antiviral treatments for COVID-19  \n",
       "14  antiviral treatments for COVID-19  \n",
       "15  antiviral treatments for COVID-19  \n",
       "16  antiviral treatments for COVID-19  \n",
       "17  antiviral treatments for COVID-19  \n",
       "18  antiviral treatments for COVID-19  \n",
       "19  antiviral treatments for COVID-19  \n",
       "20  antiviral treatments for COVID-19  \n",
       "21  antiviral treatments for COVID-19  \n",
       "22  antiviral treatments for COVID-19  \n",
       "23  antiviral treatments for COVID-19  \n",
       "24  antiviral treatments for COVID-19  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "execute_query(\"antiviral treatments for COVID-19\", metadata, similarity_threshold=0.70)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Executing on a paper specific query\n",
    "\n",
    "Sometimes, we want to be able to zoom in on a specific detail and a more detailed exploration. \n",
    "\n",
    "In this sample, we find a paper about a hydroxychloroqine study and want to view the full paper details. To accomplish this, we enter the sha number into the parameters and leave the query blank. Note, however, that you can still enter keywords in the query section to get a targeted search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Batches: 100%|██████████| 1/1 [00:00<00:00, 19.09it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "======= REPORT =========\n",
      "Query: \n",
      "Sha: 23e7355b5e4e0209f64c9d8d5772092a53b72686\n",
      "\n",
      "Title: Efficacy of hydroxychloroquine in patients with COVID-19: results of a randomized clinical trial\n",
      "\n",
      "Published Date: 2020-03-30\n",
      "\n",
      "Number of times cited: 1\n",
      "\n",
      "Antiviral Related Terms: hydroxychloroquine; chloroquine\n",
      "\n",
      "Abstract: \n",
      "Aims: Studies have indicated that chloroquine (CQ) shows antagonism against COVID-19 in vitro. However, evidence regarding its effects in patients is limited. This study aims to evaluate the efficacy of hydroxychloroquine (HCQ) in the treatment of patients with COVID-19. Main methods: From February 4 to February 28, 2020, 62 patients suffering from COVID-19 were diagnosed and admitted to Renmin Hospital of Wuhan University. All participants were randomized in a parallel-group trial, 31 patients were assigned to receive an additional 5-day HCQ (400 mg/d) treatment, Time to clinical recovery (TTCR), clinical characteristics, and radiological results were assessed at baseline and 5 days after treatment to evaluate the effect of HCQ. Key findings: For the 62 COVID-19 patients, 46.8% (29 of 62) were male and 53.2% (33 of 62) were female, the mean age was 44.7 (15.3) years. No difference in the age and sex distribution between the control group and the HCQ group. But for TTCR, the body temperature recovery time and the cough remission time were significantly shortened in the HCQ treatment group. Besides, a larger proportion of patients with improved pneumonia in the HCQ treatment group (80.6%, 25 of 32) compared with the control group (54.8%, 17 of 32). Notably, all 4 patients progressed to severe illness that occurred in the control group. However, there were 2 patients with mild adverse reactions in the HCQ treatment group. Significance: Among patients with COVID-19, the use of HCQ could significantly shorten TTCR and promote the absorption of pneumonia.\n",
      "\n",
      "Context: \n",
      "Paragraph: 0\n",
      "Coronaviruses are enveloped positive-sense single-stranded RNA viruses belonging to the family Coronaviridae and are broadly distributed in humans and other vertebrates, eventually causing damage in digestive, respiratory and even multiple systems. In December 2019, a series of pneumonia cases of unknown etiology appeared in Wuhan, Hubei, China [1] . Sequencing analysis of throat swabs samples and electron microscope observations indicated a novel coronavirus, which was named SARS-CoV-2 (formerly known as 2019-nCoV) [2] . Coronavirus disease 2019 (COVID-19) caused by SARS-CoV-2 has been confirmed to have obvious human-to-human characteristics [3, 4] . As of March 20, 2020, more than two hundred thousand confirmed cases have been identified globally, for a total of 8778 deaths [5] .\n",
      " \n",
      "Paragraph: 1\n",
      "As the epidemic is spreading to many countries, COVID-19 poses a severe threat to global health [6] . Therefore, it is urgent to develop effective drugs against COVID-19.\n",
      " \n",
      "Paragraph: 2\n",
      "The recent publication of results showing the activity of chloroquine (CQ) against SARS-CoV-2 in vitro [7], some experts and researchers also have been recommended the efficacy of this antimalarial drug in patients with 9] . For this, the U.S. Food and Drug Administration (FDA) has been working to investigate the use of CQ in COVID-19 [10] . As a derivative of CQ, hydroxychloroquine (HCQ) has similar therapeutic effects and fewer adverse effects. Based on its characteristics of immunity regulation, antithrombotic activity, and inflammation improvement, HCQ has been routinely used in the clinical treatment of systemic lupus erythematosus (SLE) [11] . However, . CC-BY-NC-ND 4.0 International license It is made available under a author/funder, who has granted medRxiv a license to display the preprint in perpetuity.\n",
      " \n",
      "Paragraph: 3\n",
      "is the (which was not peer-reviewed) The copyright holder for this preprint . https://doi.org/10.1101/2020.03. 22.20040758 doi: medRxiv preprint the efficacy of HCQ in COVID-19 remains unknown.\n",
      " \n",
      "Paragraph: 4\n",
      "Interestingly, through a follow-up survey, we found that none of our 80 SLE patients who took long-term oral HCQ had been confirmed to have SARS-CoV-2 infection or appeared to have related symptoms. In addition, among the 178 patients diagnosed with COVID-19 pneumonia in our hospital, none were receiving HCQ treatment before admission. All predicting the use of HCQ in SARS-CoV-2 infections. As one of the clinical research registration units in China, we aimed to investigate the efficiency of HCQ in patients with COVID-19 in this study.\n",
      " \n",
      "Paragraph: 5\n",
      "The clinical research protocol was reviewed and approved by the Ethics Committee in Renmin 62 patients who met the trial criteria were randomly assigned in a to two groups, all received the . CC-BY-NC-ND 4.0 International license It is made available under a author/funder, who has granted medRxiv a license to display the preprint in perpetuity.\n",
      " \n",
      "Paragraph: 6\n",
      "is the (which was not peer-reviewed) The copyright holder for this preprint . https://doi.org/10.1101/2020.03. 22.20040758 doi: medRxiv preprint standard treatment (oxygen therapy, antiviral agents, antibacterial agents, and immunoglobulin, with or without corticosteroids), patients in the HCQ treatment group received additional oral HCQ (hydroxychloroquine sulfate tablets, Shanghai Pharma) 400 mg/d (200 mg/bid) between days 1 and 5 ( Figure 1), patients in the control group with the standard treatment only. Randomization was performed through a computer-generated list stratified by site. Treatments were assigned after confirming the correctness of the admission criteria. Neither the research performers nor the patients were aware of the treatment assignments. Pulmonary recovery is defined as three levels: exacerbated, unchanged, and improved, moderately improved when less than 50 % of pneumonia were absorbed, and more than 50 % means significantly improved.\n",
      " \n",
      "Paragraph: 7\n",
      "Data were described as the mean (standard deviation, SD), n (%), the t-test or χ ² test was used to compare the differences between the two groups. A two-sided p-value of less than 0.05 was considered statistically significant. Statistical analyses were performed using Graphpad Prism, version 6.0.\n",
      " \n",
      "Paragraph: 8\n",
      "62 patients were identified as having COVID-19 and enrolled in this study, none quit ( Figure 1 ). As shown in Table 1, For all patients, the age was 44.7 (15.3) years old, 46.8% (29 of 62) were male and 53.2% (33 of 62) were female. Patients were randomly assigned into two groups. There was no significant difference in the age and sex distribution between the two groups of patients, but there are . CC-BY-NC-ND 4.0 International license It is made available under a author/funder, who has granted medRxiv a license to display the preprint in perpetuity. CC-BY-NC-ND 4.0 International license It is made available under a author/funder, who has granted medRxiv a license to display the preprint in perpetuity.\n",
      " \n",
      "Paragraph: 9\n",
      "is the (which was not peer-reviewed) The copyright holder for this preprint . https://doi.org/10.1101/2020.03. 22.20040758 doi: medRxiv preprint and effects of CQ in patients with COVID-19, we registered this trial in ChiCTR and chose HCQ (the sulfate and phosphate salts of CQ) as the intervention agent. The data in this study revealed that after 5 days of HCQ treatment, the symptoms of patients with COVID-19 were significantly relieved, manifesting as shorten in the recovery time for cough and fever. At the same time, a larger proportion of patients with pulmonary inflammatory has been partially absorbed in the HCQ treatment group, indicating the immune modulation and anti-inflammatory properties of HCQ in non-malarial diseases [17] . At present, the multiple actions of HCQ such as regulation in pro-inflammatory cytokines [e.g.\n",
      " \n",
      "Paragraph: 10\n",
      "Tumor necrosis factor-α (TNF-α), interleukin-1 (IL-1), interleukin-1 (IL-6)], antioxidant activities, also promote it widely performed in rheumatic diseases such as SLE [11, 17] . According to current research, a higher pro-inflammatory cytokine storm existed in COVID-19 patients with a severe or critical illness, eventually affected the prognosis [18] . For this, IL-6 antibody blocker, transfusion of convalescent plasma, and other therapies have been applied to counteract the cytokine storm [19, 20] . Therefore, with antiviral and autoimmune regulation effects, HCQ should be a protector in SARS-CoV-2 infection. In the present study, the reduced risk of progression to severe illness in patients with HCQ treatment also explained the intervention effect of HCQ on the pathological process of the COVID-19.\n",
      " \n",
      "Paragraph: 11\n",
      "Although HCQ has proven to be effective, with advantages of inexpensive and easily accessible, its potential detrimental effects in viral diseases must also be taken seriously. Retinopathy is one of the major adverse reactions of long-term therapy with HCQ [21] . Besides, patients with rheumatoid diseases treated with HCQ occasionally experience arrhythmias [22] . Other rare adverse reactions caused by HCQ include gastrointestinal reactions, cramps, liver dysfunction, itching, headache, dizziness, insomnia, peripheral neuropathy [13] . Fortunately, deciding on individual treatment plans scientifically, monitoring adverse reactions timely, to avoid overdose, short-term application of HCQ is relatively safe.\n",
      " \n",
      "Paragraph: 12\n",
      "Despite our small number of cases, the potential of HCQ in the treatment of COVID-19 has been partially confirmed. Considering that there is no better option at present, it is a promising practice to apply HCQ to COVID-19 under reasonable management. However, Large-scale clinical and basic research is still needed to clarify its specific mechanism and to continuously optimize the treatment plan.\n",
      " \n",
      "Paragraph: 13\n",
      ". CC-BY-NC-ND 4.0 International license It is made available under a author/funder, who has granted medRxiv a license to display the preprint in perpetuity.\n",
      " \n",
      "Paragraph: 14\n",
      "is the (which was not peer-reviewed) The copyright holder for this preprint Abbreviations: HCQ, hydroxychloroquine. Figure 1 : Study flow diagram.\n",
      " \n",
      "Paragraph: 15\n",
      "Abbreviations: COVID-19, severe acute respiratory syndrome coronavirus; HCQ, hydroxychloroquine;\n",
      " \n",
      "Paragraph: 16\n",
      "CT, computed tomography.\n",
      " \n",
      "Paragraph: 17\n",
      ". CC-BY-NC-ND 4.0 International license It is made available under a author/funder, who has granted medRxiv a license to display the preprint in perpetuity.\n",
      " \n",
      "Paragraph: 18\n",
      "is the (which was not peer-reviewed) The copyright holder for this preprint . https://doi.org/10.1101/2020.03. 22.20040758 doi: medRxiv preprint\n",
      " \n",
      "\n",
      "\n",
      "----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sha</th>\n",
       "      <th>result</th>\n",
       "      <th>context</th>\n",
       "      <th>title</th>\n",
       "      <th>publish_time</th>\n",
       "      <th>abstract</th>\n",
       "      <th>keywords</th>\n",
       "      <th>journal</th>\n",
       "      <th>number_citations</th>\n",
       "      <th>query</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>23e7355b5e4e0209f64c9d8d5772092a53b72686</td>\n",
       "      <td>\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n</td>\n",
       "      <td>Paragraph: 0\\nCoronaviruses are enveloped posi...</td>\n",
       "      <td>Efficacy of hydroxychloroquine in patients wit...</td>\n",
       "      <td>2020-03-30</td>\n",
       "      <td>Aims: Studies have indicated that chloroquine ...</td>\n",
       "      <td>hydroxychloroquine; chloroquine</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                        sha  \\\n",
       "0  23e7355b5e4e0209f64c9d8d5772092a53b72686   \n",
       "\n",
       "                                 result  \\\n",
       "0  \\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n   \n",
       "\n",
       "                                             context  \\\n",
       "0  Paragraph: 0\\nCoronaviruses are enveloped posi...   \n",
       "\n",
       "                                               title publish_time  \\\n",
       "0  Efficacy of hydroxychloroquine in patients wit...   2020-03-30   \n",
       "\n",
       "                                            abstract  \\\n",
       "0  Aims: Studies have indicated that chloroquine ...   \n",
       "\n",
       "                          keywords journal  number_citations query  \n",
       "0  hydroxychloroquine; chloroquine     NaN                 1        "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "execute_query(\"\", metadata, sha='23e7355b5e4e0209f64c9d8d5772092a53b72686')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Executing on many queries\n",
    "\n",
    "We execute on several queries and save the output. We leave it up to the user to explore which queries are most appropriate.\n",
    "\n",
    "Some of our queries include:\n",
    "* antiviral treatments for COVID-19\n",
    "* {insert_drug_name} for treatment of COVID-19 (i.e. hydroxychloroquine for treatment of COVID-19)\n",
    "* preventative clinical studies for COVID-19\n",
    "* anti-viral prophylaxis studies for COVID-19\n",
    "* prophylaxis studies for COVID-19\n",
    "* diagnostic assay for COV response\n",
    "* immunoassay for antibody or cell response\n",
    "* ELISA or flow cytometry assay for cov\n",
    "* mouse or ferret model for assay evaluation\n",
    "\n",
    "**Note: the code below only executes subset of queries that we evaluated.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Batches: 100%|██████████| 1/1 [00:00<00:00, 67.50it/s]\n",
      "Batches: 100%|██████████| 1/1 [00:00<00:00, 74.33it/s]\n",
      "/opt/conda/lib/python3.6/site-packages/ipykernel_launcher.py:43: FutureWarning: `item` has been deprecated and will be removed in a future version\n",
      "Batches: 100%|██████████| 1/1 [00:00<00:00, 72.68it/s]\n",
      "Batches: 100%|██████████| 1/1 [00:00<00:00, 64.57it/s]\n",
      "Batches: 100%|██████████| 1/1 [00:00<00:00, 70.38it/s]\n",
      "Batches: 100%|██████████| 1/1 [00:00<00:00, 63.31it/s]\n",
      "Batches: 100%|██████████| 1/1 [00:00<00:00, 72.27it/s]\n",
      "Batches: 100%|██████████| 1/1 [00:00<00:00, 75.97it/s]\n",
      "Batches: 100%|██████████| 1/1 [00:00<00:00, 75.80it/s]\n"
     ]
    }
   ],
   "source": [
    "queries = [\n",
    "    'antiviral treatments for COVID-19',\n",
    "    'hydroxychloroquine for treatment of COVID-19',\n",
    "    'preventative clinical studies for COVID-19',\n",
    "    'anti-viral prophylaxis studies for COVID-19',\n",
    "    'prophylaxis studies for COVID-19',\n",
    "    'diagnostic assay for COV response',\n",
    "    'immunoassay for antibody or cell response',\n",
    "    'ELISA or flow cytometry assay for cov',\n",
    "    'mouse or ferret model for assay evaluation'\n",
    "]\n",
    "\n",
    "for query in queries:\n",
    "    result = execute_query(query, metadata, print_output = False)\n",
    "    result.to_csv(\"/kaggle/working/query - \"+query+\".csv\", index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Manual Review and Summary\n",
    "We go through the query results manually to make sure relevant information is found and compiled. Overall, we spent half a day reviewing these queries and generating a compiled output. \n",
    "\n",
    "We save this and it can be found in our Github Repository https://github.com/Weilin37/CORD-19-Kaggle-Challenge/tree/master/Report. Each report is formatted differently due to the work being split up amongst different people. Additionally, not all tasks are suitable for a standardized output.\n",
    "\n",
    "We present our summaries in the beginning of our notebook, sectioned \"Highlighted Results\". Please take a look!\n",
    "\n",
    "Thank you for your time and please let us know your comments and questions!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "0103bd9900b549a68bc2c87471ba0e2e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": "initial"
      }
     },
     "08a8931798b940a4a156403615f0147d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": "initial"
      }
     },
     "0cfb8a43b0a14472a8e2be62434ca211": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": "initial"
      }
     },
     "1380de0b13c44211b0a66f5d27dabaf9": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "1801c68c5fd346f9a1386b55eaf6a2f1": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_b5d35fd921a344c0adaebd17aa0f8106",
       "placeholder": "​",
       "style": "IPY_MODEL_5dca3a853b2d4f07aff4376f0271ae26",
       "value": " 213k/213k [00:00&lt;00:00, 503kB/s]"
      }
     },
     "1960b4db361c47f09e1c1a1d8b7e714b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": "initial"
      }
     },
     "1cc062d0ec2d4091b14c1dadb276c06a": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_8ef06e89066345d992b62c6348101c23",
        "IPY_MODEL_1801c68c5fd346f9a1386b55eaf6a2f1"
       ],
       "layout": "IPY_MODEL_72d20e380e0b431387a43ff361df8169"
      }
     },
     "201840d5972c4af7be5369b5dddb9be2": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "21681e79a86b4da28574e875a9d77c49": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_3ed217b9aebe4ffb8406d8cb7b229e43",
       "placeholder": "​",
       "style": "IPY_MODEL_83e89a5aecfe44e0ad3779208bb5cbae",
       "value": " 112/112 [00:00&lt;00:00, 168B/s]"
      }
     },
     "2bd90aa3c64f469baabefa97c0f2873c": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "2cf136ea03bb4cc4b70c8c2f4d5eda14": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_9b345db4622b4ab39f83237920d0cc6c",
        "IPY_MODEL_8a36029cdc844eecbc207371abf37b65"
       ],
       "layout": "IPY_MODEL_6dafdf77f0544eff99df4076ef931a07"
      }
     },
     "30a7ad5e639b45ac93f45b18daa0a487": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "3395149268ca485a881fe5aef675eac6": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "",
       "description": "Task progress bar: 100%",
       "description_tooltip": null,
       "layout": "IPY_MODEL_e482b0d868df461795be8173b28ef065",
       "max": 4311.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_0cfb8a43b0a14472a8e2be62434ca211",
       "value": 4311.0
      }
     },
     "3c9d779b5106455083d6a74aaef175e9": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "3ed217b9aebe4ffb8406d8cb7b229e43": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "3fa6e14add804fe2b155c4f640fb4610": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "4ace4f2901764e2f96307dfc011e9442": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_9127c5a3f07c443b94447244ab81546f",
        "IPY_MODEL_21681e79a86b4da28574e875a9d77c49"
       ],
       "layout": "IPY_MODEL_1380de0b13c44211b0a66f5d27dabaf9"
      }
     },
     "4d9cb493d84c48f385bfa9c9d3c1c7a8": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_fb0518ff95ca45e3b0de790a1c017214",
        "IPY_MODEL_84f52b0adf2b4a33aeeebd786ca1aefa"
       ],
       "layout": "IPY_MODEL_30a7ad5e639b45ac93f45b18daa0a487"
      }
     },
     "4e35647f722e4b529553b69ccbac35f6": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": "initial"
      }
     },
     "4e84a47661aa4997b5d22889d5ec44f0": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "Downloading: 100%",
       "description_tooltip": null,
       "layout": "IPY_MODEL_e948c5848a104d8fb061f676ac5002fa",
       "max": 433288887.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_4e35647f722e4b529553b69ccbac35f6",
       "value": 433288887.0
      }
     },
     "56736942462748a3b0290fa267ce0b9b": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "5dca3a853b2d4f07aff4376f0271ae26": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "5f12cd5b71604066a9f357c394bbdf3d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "",
       "description": "Chunk progress: 100%",
       "description_tooltip": null,
       "layout": "IPY_MODEL_65a642d6bcd14612a7807c8671a17ddf",
       "max": 12.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_93e7a3c13ca94ca9abd9146dacc2b792",
       "value": 12.0
      }
     },
     "65a642d6bcd14612a7807c8671a17ddf": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "6862abcde090494088b40fe76146edcd": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "6dafdf77f0544eff99df4076ef931a07": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "6e4f9ebb99d649269a7b0984c6f8ba52": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "72d20e380e0b431387a43ff361df8169": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "7baa5da6b58149899e4988ae3b67a46b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "7d58ba339c0a4e4e8f4864aa57b2ed34": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "803162e67eb74168b7f98a4abd40903c": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_bd090e53e75d454f822d49dca6061229",
       "placeholder": "​",
       "style": "IPY_MODEL_201840d5972c4af7be5369b5dddb9be2",
       "value": " 4311/4311 [00:17&lt;00:00, 826.44it/s]"
      }
     },
     "83e89a5aecfe44e0ad3779208bb5cbae": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "84f52b0adf2b4a33aeeebd786ca1aefa": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_3c9d779b5106455083d6a74aaef175e9",
       "placeholder": "​",
       "style": "IPY_MODEL_3fa6e14add804fe2b155c4f640fb4610",
       "value": " 1.02k/1.02k [00:01&lt;00:00, 944B/s]"
      }
     },
     "8a36029cdc844eecbc207371abf37b65": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_2bd90aa3c64f469baabefa97c0f2873c",
       "placeholder": "​",
       "style": "IPY_MODEL_8e557b6d16c340a0b359cbcd6a0073dd",
       "value": " 1/1 [00:06&lt;00:00,  8.07s/it]"
      }
     },
     "8d2b229e937940b3b6120b84142efa69": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_3395149268ca485a881fe5aef675eac6",
        "IPY_MODEL_803162e67eb74168b7f98a4abd40903c"
       ],
       "layout": "IPY_MODEL_e98c999e45d54404884a8213987ec606"
      }
     },
     "8e557b6d16c340a0b359cbcd6a0073dd": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "8ef06e89066345d992b62c6348101c23": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "Downloading: 100%",
       "description_tooltip": null,
       "layout": "IPY_MODEL_6e4f9ebb99d649269a7b0984c6f8ba52",
       "max": 213450.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_0103bd9900b549a68bc2c87471ba0e2e",
       "value": 213450.0
      }
     },
     "9101e11ced0743fea04614cf3875bd06": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "9127c5a3f07c443b94447244ab81546f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "Downloading: 100%",
       "description_tooltip": null,
       "layout": "IPY_MODEL_ca4e3492a4db4d8bac5c50a39d58dd41",
       "max": 112.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_08a8931798b940a4a156403615f0147d",
       "value": 112.0
      }
     },
     "92fa36c3f9d64760b26eb594f86015f2": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "93e7a3c13ca94ca9abd9146dacc2b792": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": "initial"
      }
     },
     "93f8028626844b5dba5934a4479fcb26": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_fefe78cf66ce49a5a0307fd0f7e8745e",
       "placeholder": "​",
       "style": "IPY_MODEL_92fa36c3f9d64760b26eb594f86015f2",
       "value": " 12/12 [05:14&lt;00:00, 23.91s/it]"
      }
     },
     "9b345db4622b4ab39f83237920d0cc6c": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "",
       "description": "Calling get_article_citations_meta: 100%",
       "description_tooltip": null,
       "layout": "IPY_MODEL_7d58ba339c0a4e4e8f4864aa57b2ed34",
       "max": 1.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_a6e959b3b7074890b69941f1b4ac77eb",
       "value": 1.0
      }
     },
     "a6e959b3b7074890b69941f1b4ac77eb": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": "initial"
      }
     },
     "a80ec397584949d58333924787a02c68": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_9101e11ced0743fea04614cf3875bd06",
       "placeholder": "​",
       "style": "IPY_MODEL_6862abcde090494088b40fe76146edcd",
       "value": " 433M/433M [00:13&lt;00:00, 32.1MB/s]"
      }
     },
     "b1bfe675fd6e45c788991f3719ca8a9b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "Downloading: 100%",
       "description_tooltip": null,
       "layout": "IPY_MODEL_bb2110b8a64a4182a4e4415294c2c236",
       "max": 136.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_ef69f4cfb93c4d6085e1e0093fa39950",
       "value": 136.0
      }
     },
     "b5d35fd921a344c0adaebd17aa0f8106": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "b76e5da0f22b4025a32bb628da62a3fc": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "bb2110b8a64a4182a4e4415294c2c236": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "bd090e53e75d454f822d49dca6061229": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "ca4e3492a4db4d8bac5c50a39d58dd41": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "d3020dede0004cceb97602f1c6cd0bd6": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_4e84a47661aa4997b5d22889d5ec44f0",
        "IPY_MODEL_a80ec397584949d58333924787a02c68"
       ],
       "layout": "IPY_MODEL_f18edef6c1ec48a68823b922e02adf37"
      }
     },
     "e482b0d868df461795be8173b28ef065": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "e948c5848a104d8fb061f676ac5002fa": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "e98c999e45d54404884a8213987ec606": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "e99164c2ff874511943e6ba7948e67e8": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "ea705d2eea004946940324b845b5bef0": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_b1bfe675fd6e45c788991f3719ca8a9b",
        "IPY_MODEL_fc6802080ef94ab18037058bd71e1bdb"
       ],
       "layout": "IPY_MODEL_f8e076e4dbb14de3aa1bea1837f6c07f"
      }
     },
     "ed1dbc3c552a44a4a3cd743c8cdd8dda": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_5f12cd5b71604066a9f357c394bbdf3d",
        "IPY_MODEL_93f8028626844b5dba5934a4479fcb26"
       ],
       "layout": "IPY_MODEL_e99164c2ff874511943e6ba7948e67e8"
      }
     },
     "ef69f4cfb93c4d6085e1e0093fa39950": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": "initial"
      }
     },
     "f18edef6c1ec48a68823b922e02adf37": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "f8e076e4dbb14de3aa1bea1837f6c07f": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "fb0518ff95ca45e3b0de790a1c017214": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "Downloading: 100%",
       "description_tooltip": null,
       "layout": "IPY_MODEL_56736942462748a3b0290fa267ce0b9b",
       "max": 1017.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_1960b4db361c47f09e1c1a1d8b7e714b",
       "value": 1017.0
      }
     },
     "fc6802080ef94ab18037058bd71e1bdb": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_b76e5da0f22b4025a32bb628da62a3fc",
       "placeholder": "​",
       "style": "IPY_MODEL_7baa5da6b58149899e4988ae3b67a46b",
       "value": " 136/136 [00:00&lt;00:00, 2.37kB/s]"
      }
     },
     "fefe78cf66ce49a5a0307fd0f7e8745e": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}