텍스트분석7,10.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyPQwvqE7+5r6ETsqyFtTcBa",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/wlsisl/ESAA_/blob/main/%ED%85%8D%EC%8A%A4%ED%8A%B8%EB%B6%84%EC%84%9D7%2C10.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 7. 문서 군집화 소개와 실습 (Opinion Review 데이터 세트)\n",
        "\n",
        "### 문서 군집화 개념 \n",
        "문서 군집화 : 비슷한 텍스트 구성의 문서를 군집화하는 것이다. 동일한 군집에 속하는 문서를 같은 카테고리 소속으로 분류할 수 있으므로 앞에서 소개한 텍스트 분류 기반의문서 분류와 유사하다.  \n",
        "\n",
        "* 문서 군집하는 학습 데이터 세트가 필요없느 ㄴ비지도 학습 기반으로 동작한다.\n",
        "\n",
        "### Opinion Review 데이터 세트를 이용한 문서 군집화화수행\n"
      ],
      "metadata": {
        "id": "aKtQZqJgxVq2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "import glob, os\n",
        "import nltk\n",
        "nltk.download('punkt')\n",
        "nltk.download('wordnet')\n",
        "\n",
        "from google.colab import drive\n",
        "drive.mount(\"/content/drive\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OuAg-2iN0J0O",
        "outputId": "753f66b7-04e8-4eda-c143-42c4736fe454"
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Unzipping tokenizers/punkt.zip.\n",
            "[nltk_data] Downloading package wordnet to /root/nltk_data...\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jpubAD9IxJEk"
      },
      "outputs": [],
      "source": [
        "path=r'/content/drive/MyDrive/topics'\n",
        "all_files=glob.glob(os.path.join(path, '*.data'))\n",
        "filename_list=[]\n",
        "opinion_text=[]"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "#개별 파일의 파일명은 filename_list로 취합,\n",
        "#개별 파일의 파일 내용은 df 로딩 후 다시 string으로 변환해 opinion_text list로 취합합\n",
        "for file_ in all_files:\n",
        "  #개별 파일을 읽어서 df로 생성성\n",
        "  df=pd.read_table(file_, index_col=None, header=0, encoding='latin1')\n",
        "\n",
        "  #절대 경로로 주어진 파일명을 가공.\n",
        "  #맨 마지막 .data 확장자도 제거거\n",
        "  filename_=file_.split('\\\\')[-1]\n",
        "  filename=filename_.split('.')[0]\n",
        "\n",
        "  #파일명 list와 파일 내용 list에 파일명과 파일 내용을 추가가\n",
        "  filename_list.append(filename)\n",
        "  opinion_text.append(df.to_string())\n",
        "\n",
        "#파일명 list와 파일 내용 list 객체를 df로 생성성\n",
        "document_df=pd.DataFrame({'filename':filename_list, 'opinion_text':opinion_text})\n",
        "document_df.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "L-JCXXlaExvv",
        "outputId": "7d142cdd-f7c3-4ca9-84d5-33aeac3c5563"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                            filename  \\\n",
              "0  /content/drive/MyDrive/topics/accuracy_garmin_...   \n",
              "1  /content/drive/MyDrive/topics/bathroom_bestwes...   \n",
              "2  /content/drive/MyDrive/topics/eyesight-issues_...   \n",
              "3  /content/drive/MyDrive/topics/buttons_amazon_k...   \n",
              "4    /content/drive/MyDrive/topics/features_windows7   \n",
              "\n",
              "                                        opinion_text  \n",
              "0                                                ...  \n",
              "1                                                ...  \n",
              "2                                                ...  \n",
              "3                                                ...  \n",
              "4                                                ...  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-032a0635-3650-49e1-958d-e616d3c208cf\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>filename</th>\n",
              "      <th>opinion_text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>/content/drive/MyDrive/topics/accuracy_garmin_...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>/content/drive/MyDrive/topics/bathroom_bestwes...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>/content/drive/MyDrive/topics/eyesight-issues_...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>/content/drive/MyDrive/topics/buttons_amazon_k...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>/content/drive/MyDrive/topics/features_windows7</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-032a0635-3650-49e1-958d-e616d3c208cf')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-032a0635-3650-49e1-958d-e616d3c208cf button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-032a0635-3650-49e1-958d-e616d3c208cf');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "각 파일 이름 자체만으로 의견의 텍스트가 어떠한 제품/ 서비스에 대한 리뷰인지 알 수 있다. \n",
        "\n",
        "문서를 TF-IDF 형태로 피처 벡터화해보자."
      ],
      "metadata": {
        "id": "5L6uesMy0Mz8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from nltk.stem import WordNetLemmatizer\n",
        "import nltk\n",
        "import string\n",
        "\n",
        "# 단어 원형 추출 함수\n",
        "lemmar = WordNetLemmatizer()\n",
        "def LemTokens(tokens):\n",
        "    return [lemmar.lemmatize(token) for token in tokens]\n",
        "\n",
        "# 특수 문자 사전 생성: {33: None ...}\n",
        "# ord(): 아스키 코드 생성\n",
        "remove_punct_dict = dict((ord(punct), None) for punct in string.punctuation)\n",
        "\n",
        "# 특수 문자 제거 및 단어 원형 추출\n",
        "def LemNormalize(text):\n",
        "    # 텍스트 소문자 변경 후 특수 문자 제거\n",
        "    text_new = text.lower().translate(remove_punct_dict)\n",
        "    \n",
        "    # 단어 토큰화\n",
        "    word_tokens = nltk.word_tokenize(text_new)\n",
        "    \n",
        "    # 단어 원형 추출\n",
        "    return LemTokens(word_tokens)"
      ],
      "metadata": {
        "id": "H1RW3y-j0cLH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "nltk.download('punkt')\n",
        "nltk.download('wordnet')\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "\n",
        "tfidf_vect=TfidfVectorizer(tokenizer=LemNormalize, stop_words='english', ngram_range=(1,2), min_df=0.05, max_df=0.85)\n",
        "#opinion_text 칼럼 값으로 피처 벡터화 수행\n",
        "feature_vect=tfidf_vect.fit_transform(document_df['opinion_text'])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7M-vV-fp0gKz",
        "outputId": "e3e94a26-b9c1-48d1-899c-c1e042f63395"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Package punkt is already up-to-date!\n",
            "[nltk_data] Downloading package wordnet to /root/nltk_data...\n",
            "[nltk_data]   Package wordnet is already up-to-date!\n",
            "/usr/local/lib/python3.10/dist-packages/sklearn/feature_extraction/text.py:528: UserWarning: The parameter 'token_pattern' will not be used since 'tokenizer' is not None'\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.10/dist-packages/sklearn/feature_extraction/text.py:409: UserWarning: Your stop_words may be inconsistent with your preprocessing. Tokenizing the stop words generated tokens ['ha', 'le', 'u', 'wa'] not in stop_words.\n",
            "  warnings.warn(\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "문서별 텍스트가 TF-IDF 변환된 피처 벡터화 행렬 데이터에 대해서 군집화를 수행해 어떤 문서끼리 군집되는지 확인. \n",
        "\n",
        "군집화 기법 : K-평균\n",
        "먼저 5개의 중심 기반으로 어떻게 군집화되는지 확인해보자."
      ],
      "metadata": {
        "id": "GVKtvnPJ0M56"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.cluster import KMeans\n",
        "\n",
        "#5개 집합으로 군집화 수행\n",
        "km_cluster=KMeans(n_clusters=5, max_iter=10000, random_state=0)\n",
        "km_cluster.fit(feature_vect)\n",
        "cluster_label=km_cluster.labels_\n",
        "cluster_centers=km_cluster.cluster_centers_"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WXM1q26V2Mrv",
        "outputId": "d27c58db-dedc-4b27-f31a-dcfa226d114f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/cluster/_kmeans.py:870: FutureWarning: The default value of `n_init` will change from 10 to 'auto' in 1.4. Set the value of `n_init` explicitly to suppress the warning\n",
            "  warnings.warn(\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "각 데이터별로 할당된 군집의 레이블을 파일명과 파일 내용을 가지고 있는 데이터프레임에 칼럼 추가해 생성하자."
      ],
      "metadata": {
        "id": "5BnxRJVL2URl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "document_df['cluster_label']=cluster_label\n",
        "document_df.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "XmJwuBcA2UXm",
        "outputId": "9e671500-a8f0-4c03-b96a-0cd27eb3cc5c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                            filename  \\\n",
              "0  /content/drive/MyDrive/topics/accuracy_garmin_...   \n",
              "1  /content/drive/MyDrive/topics/bathroom_bestwes...   \n",
              "2  /content/drive/MyDrive/topics/eyesight-issues_...   \n",
              "3  /content/drive/MyDrive/topics/buttons_amazon_k...   \n",
              "4    /content/drive/MyDrive/topics/features_windows7   \n",
              "\n",
              "                                        opinion_text  cluster_label  \n",
              "0                                                ...              4  \n",
              "1                                                ...              1  \n",
              "2                                                ...              0  \n",
              "3                                                ...              0  \n",
              "4                                                ...              2  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-3b8f524d-e424-4eb1-add8-a6d0b94e8bba\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>filename</th>\n",
              "      <th>opinion_text</th>\n",
              "      <th>cluster_label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>/content/drive/MyDrive/topics/accuracy_garmin_...</td>\n",
              "      <td>...</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>/content/drive/MyDrive/topics/bathroom_bestwes...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>/content/drive/MyDrive/topics/eyesight-issues_...</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>/content/drive/MyDrive/topics/buttons_amazon_k...</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>/content/drive/MyDrive/topics/features_windows7</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-3b8f524d-e424-4eb1-add8-a6d0b94e8bba')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-3b8f524d-e424-4eb1-add8-a6d0b94e8bba button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-3b8f524d-e424-4eb1-add8-a6d0b94e8bba');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#cluster_label=0인 데이터는 호텔에 대한 리뷰로 군집화\n",
        "document_df[document_df['cluster_label']==0].sort_values(by='filename')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "COcCLhrx2dRE",
        "outputId": "36cae052-7ddf-415f-a864-e4bae3a2ca9a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                             filename  \\\n",
              "3   /content/drive/MyDrive/topics/buttons_amazon_k...   \n",
              "2   /content/drive/MyDrive/topics/eyesight-issues_...   \n",
              "7   /content/drive/MyDrive/topics/fonts_amazon_kindle   \n",
              "19  /content/drive/MyDrive/topics/navigation_amazo...   \n",
              "36  /content/drive/MyDrive/topics/price_amazon_kindle   \n",
              "\n",
              "                                         opinion_text  cluster_label  \n",
              "3                                                 ...              0  \n",
              "2                                                 ...              0  \n",
              "7                                                 ...              0  \n",
              "19                                                ...              0  \n",
              "36                                                ...              0  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-4f28373f-46f9-432e-9ff4-893fff2a9d23\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>filename</th>\n",
              "      <th>opinion_text</th>\n",
              "      <th>cluster_label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>/content/drive/MyDrive/topics/buttons_amazon_k...</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>/content/drive/MyDrive/topics/eyesight-issues_...</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>/content/drive/MyDrive/topics/fonts_amazon_kindle</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>/content/drive/MyDrive/topics/navigation_amazo...</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>36</th>\n",
              "      <td>/content/drive/MyDrive/topics/price_amazon_kindle</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4f28373f-46f9-432e-9ff4-893fff2a9d23')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-4f28373f-46f9-432e-9ff4-893fff2a9d23 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-4f28373f-46f9-432e-9ff4-893fff2a9d23');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#cluster_label=1인 데이터는 킨들, 아이팟, 넷북 등 포터블 전자기기에 대한 리뷰\n",
        "document_df[document_df['cluster_label']==1].sort_values(by='filename')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 551
        },
        "id": "LNlL7Zky2odO",
        "outputId": "f95a06ac-a72f-4e2e-d779-a7f1f49ee096"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                             filename  \\\n",
              "1   /content/drive/MyDrive/topics/bathroom_bestwes...   \n",
              "11  /content/drive/MyDrive/topics/food_holiday_inn...   \n",
              "16  /content/drive/MyDrive/topics/food_swissotel_c...   \n",
              "20  /content/drive/MyDrive/topics/free_bestwestern...   \n",
              "24  /content/drive/MyDrive/topics/location_bestwes...   \n",
              "15  /content/drive/MyDrive/topics/location_holiday...   \n",
              "23  /content/drive/MyDrive/topics/parking_bestwest...   \n",
              "28  /content/drive/MyDrive/topics/price_holiday_in...   \n",
              "33  /content/drive/MyDrive/topics/room_holiday_inn...   \n",
              "38  /content/drive/MyDrive/topics/rooms_bestwester...   \n",
              "32  /content/drive/MyDrive/topics/rooms_swissotel_...   \n",
              "39  /content/drive/MyDrive/topics/service_bestwest...   \n",
              "27  /content/drive/MyDrive/topics/service_holiday_...   \n",
              "48  /content/drive/MyDrive/topics/service_swissote...   \n",
              "47  /content/drive/MyDrive/topics/staff_bestwester...   \n",
              "46  /content/drive/MyDrive/topics/staff_swissotel_...   \n",
              "\n",
              "                                         opinion_text  cluster_label  \n",
              "1                                                 ...              1  \n",
              "11                                                ...              1  \n",
              "16                                                ...              1  \n",
              "20                                                ...              1  \n",
              "24                                                ...              1  \n",
              "15                                                ...              1  \n",
              "23                                                ...              1  \n",
              "28                                                ...              1  \n",
              "33                                                ...              1  \n",
              "38                                                ...              1  \n",
              "32                                                ...              1  \n",
              "39                                                ...              1  \n",
              "27                                                ...              1  \n",
              "48                                                ...              1  \n",
              "47                                                ...              1  \n",
              "46                                                ...              1  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-17b951ac-7a08-4d8f-bfe4-0a4d491df8aa\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>filename</th>\n",
              "      <th>opinion_text</th>\n",
              "      <th>cluster_label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>/content/drive/MyDrive/topics/bathroom_bestwes...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>/content/drive/MyDrive/topics/food_holiday_inn...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>/content/drive/MyDrive/topics/food_swissotel_c...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>20</th>\n",
              "      <td>/content/drive/MyDrive/topics/free_bestwestern...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24</th>\n",
              "      <td>/content/drive/MyDrive/topics/location_bestwes...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>/content/drive/MyDrive/topics/location_holiday...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>23</th>\n",
              "      <td>/content/drive/MyDrive/topics/parking_bestwest...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28</th>\n",
              "      <td>/content/drive/MyDrive/topics/price_holiday_in...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>33</th>\n",
              "      <td>/content/drive/MyDrive/topics/room_holiday_inn...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>38</th>\n",
              "      <td>/content/drive/MyDrive/topics/rooms_bestwester...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>32</th>\n",
              "      <td>/content/drive/MyDrive/topics/rooms_swissotel_...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>39</th>\n",
              "      <td>/content/drive/MyDrive/topics/service_bestwest...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27</th>\n",
              "      <td>/content/drive/MyDrive/topics/service_holiday_...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>48</th>\n",
              "      <td>/content/drive/MyDrive/topics/service_swissote...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>47</th>\n",
              "      <td>/content/drive/MyDrive/topics/staff_bestwester...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>46</th>\n",
              "      <td>/content/drive/MyDrive/topics/staff_swissotel_...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-17b951ac-7a08-4d8f-bfe4-0a4d491df8aa')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-17b951ac-7a08-4d8f-bfe4-0a4d491df8aa button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-17b951ac-7a08-4d8f-bfe4-0a4d491df8aa');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#cluster_label=2인 데이터은 1과 비슷하지만 주로 차량용 네이게이션으로 군집\n",
        "document_df[document_df['cluster_label']==2].sort_values(by='filename')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 457
        },
        "id": "7PW2a0jm2tYs",
        "outputId": "1741796d-ed9d-4033-f9f6-b27eea8a736b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                             filename  \\\n",
              "9   /content/drive/MyDrive/topics/battery-life_ama...   \n",
              "5   /content/drive/MyDrive/topics/battery-life_ipo...   \n",
              "6   /content/drive/MyDrive/topics/battery-life_net...   \n",
              "4     /content/drive/MyDrive/topics/features_windows7   \n",
              "17  /content/drive/MyDrive/topics/keyboard_netbook...   \n",
              "25  /content/drive/MyDrive/topics/performance_netb...   \n",
              "35  /content/drive/MyDrive/topics/screen_garmin_nu...   \n",
              "29  /content/drive/MyDrive/topics/screen_ipod_nano...   \n",
              "34  /content/drive/MyDrive/topics/screen_netbook_1...   \n",
              "40  /content/drive/MyDrive/topics/size_asus_netboo...   \n",
              "42  /content/drive/MyDrive/topics/sound_ipod_nano_8gb   \n",
              "49  /content/drive/MyDrive/topics/video_ipod_nano_8gb   \n",
              "50  /content/drive/MyDrive/topics/voice_garmin_nuv...   \n",
              "\n",
              "                                         opinion_text  cluster_label  \n",
              "9                                                 ...              2  \n",
              "5                                                 ...              2  \n",
              "6                                                 ...              2  \n",
              "4                                                 ...              2  \n",
              "17                                                ...              2  \n",
              "25                                                ...              2  \n",
              "35                                                ...              2  \n",
              "29                                                ...              2  \n",
              "34                                                ...              2  \n",
              "40                                                ...              2  \n",
              "42      headphone jack i got a clear case for it a...              2  \n",
              "49                                                ...              2  \n",
              "50                                                ...              2  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-255adf1a-0438-400b-a418-b4e6cf744991\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>filename</th>\n",
              "      <th>opinion_text</th>\n",
              "      <th>cluster_label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>/content/drive/MyDrive/topics/battery-life_ama...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>/content/drive/MyDrive/topics/battery-life_ipo...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>/content/drive/MyDrive/topics/battery-life_net...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>/content/drive/MyDrive/topics/features_windows7</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>/content/drive/MyDrive/topics/keyboard_netbook...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25</th>\n",
              "      <td>/content/drive/MyDrive/topics/performance_netb...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>35</th>\n",
              "      <td>/content/drive/MyDrive/topics/screen_garmin_nu...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>29</th>\n",
              "      <td>/content/drive/MyDrive/topics/screen_ipod_nano...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>34</th>\n",
              "      <td>/content/drive/MyDrive/topics/screen_netbook_1...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>40</th>\n",
              "      <td>/content/drive/MyDrive/topics/size_asus_netboo...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>42</th>\n",
              "      <td>/content/drive/MyDrive/topics/sound_ipod_nano_8gb</td>\n",
              "      <td>headphone jack i got a clear case for it a...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49</th>\n",
              "      <td>/content/drive/MyDrive/topics/video_ipod_nano_8gb</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50</th>\n",
              "      <td>/content/drive/MyDrive/topics/voice_garmin_nuv...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-255adf1a-0438-400b-a418-b4e6cf744991')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-255adf1a-0438-400b-a418-b4e6cf744991 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-255adf1a-0438-400b-a418-b4e6cf744991');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#cluster_label=3인 데이터은 0과 같이 호텔에 대한 리뷰로 군집화화\n",
        "document_df[document_df['cluster_label']==3].sort_values(by='filename')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 332
        },
        "id": "gMjUqU212tau",
        "outputId": "64db6d59-36db-435a-ab33-2dcfeb6083ac"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                             filename  \\\n",
              "8   /content/drive/MyDrive/topics/comfort_honda_ac...   \n",
              "13  /content/drive/MyDrive/topics/comfort_toyota_c...   \n",
              "22  /content/drive/MyDrive/topics/gas_mileage_toyo...   \n",
              "26  /content/drive/MyDrive/topics/interior_honda_a...   \n",
              "21  /content/drive/MyDrive/topics/interior_toyota_...   \n",
              "14  /content/drive/MyDrive/topics/mileage_honda_ac...   \n",
              "18  /content/drive/MyDrive/topics/performance_hond...   \n",
              "37  /content/drive/MyDrive/topics/quality_toyota_c...   \n",
              "31  /content/drive/MyDrive/topics/seats_honda_acco...   \n",
              "\n",
              "                                         opinion_text  cluster_label  \n",
              "8                                                 ...              3  \n",
              "13                                                ...              3  \n",
              "22                                                ...              3  \n",
              "26                                                ...              3  \n",
              "21                                                ...              3  \n",
              "14                                                ...              3  \n",
              "18                                                ...              3  \n",
              "37                                                ...              3  \n",
              "31                                                ...              3  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-af9af945-fdf7-4eb6-86e8-508357b9c69a\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>filename</th>\n",
              "      <th>opinion_text</th>\n",
              "      <th>cluster_label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>/content/drive/MyDrive/topics/comfort_honda_ac...</td>\n",
              "      <td>...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>/content/drive/MyDrive/topics/comfort_toyota_c...</td>\n",
              "      <td>...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>22</th>\n",
              "      <td>/content/drive/MyDrive/topics/gas_mileage_toyo...</td>\n",
              "      <td>...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>26</th>\n",
              "      <td>/content/drive/MyDrive/topics/interior_honda_a...</td>\n",
              "      <td>...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>21</th>\n",
              "      <td>/content/drive/MyDrive/topics/interior_toyota_...</td>\n",
              "      <td>...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>/content/drive/MyDrive/topics/mileage_honda_ac...</td>\n",
              "      <td>...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>/content/drive/MyDrive/topics/performance_hond...</td>\n",
              "      <td>...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>37</th>\n",
              "      <td>/content/drive/MyDrive/topics/quality_toyota_c...</td>\n",
              "      <td>...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>31</th>\n",
              "      <td>/content/drive/MyDrive/topics/seats_honda_acco...</td>\n",
              "      <td>...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-af9af945-fdf7-4eb6-86e8-508357b9c69a')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-af9af945-fdf7-4eb6-86e8-508357b9c69a button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-af9af945-fdf7-4eb6-86e8-508357b9c69a');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#cluster_label=4인 데이터는 토요타와 혼다등 자동차에 대한 리뷰뷰\n",
        "document_df[document_df['cluster_label']==4].sort_values(by='filename')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 300
        },
        "id": "Ug9L6a4d20cX",
        "outputId": "63d9e258-73ed-42b1-f640-53fc0ab1035a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                             filename  \\\n",
              "0   /content/drive/MyDrive/topics/accuracy_garmin_...   \n",
              "12  /content/drive/MyDrive/topics/directions_garmi...   \n",
              "10  /content/drive/MyDrive/topics/display_garmin_n...   \n",
              "30  /content/drive/MyDrive/topics/satellite_garmin...   \n",
              "44  /content/drive/MyDrive/topics/speed_garmin_nuv...   \n",
              "43       /content/drive/MyDrive/topics/speed_windows7   \n",
              "41  /content/drive/MyDrive/topics/transmission_toy...   \n",
              "45  /content/drive/MyDrive/topics/updates_garmin_n...   \n",
              "\n",
              "                                         opinion_text  cluster_label  \n",
              "0                                                 ...              4  \n",
              "12                                                ...              4  \n",
              "10                                                ...              4  \n",
              "30                                                ...              4  \n",
              "44                                                ...              4  \n",
              "43                                                ...              4  \n",
              "41                                                ...              4  \n",
              "45                                                ...              4  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-79dc6ba1-41b0-4b67-b6a6-c2e39f425aeb\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>filename</th>\n",
              "      <th>opinion_text</th>\n",
              "      <th>cluster_label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>/content/drive/MyDrive/topics/accuracy_garmin_...</td>\n",
              "      <td>...</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>/content/drive/MyDrive/topics/directions_garmi...</td>\n",
              "      <td>...</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>/content/drive/MyDrive/topics/display_garmin_n...</td>\n",
              "      <td>...</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>30</th>\n",
              "      <td>/content/drive/MyDrive/topics/satellite_garmin...</td>\n",
              "      <td>...</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>44</th>\n",
              "      <td>/content/drive/MyDrive/topics/speed_garmin_nuv...</td>\n",
              "      <td>...</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>43</th>\n",
              "      <td>/content/drive/MyDrive/topics/speed_windows7</td>\n",
              "      <td>...</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41</th>\n",
              "      <td>/content/drive/MyDrive/topics/transmission_toy...</td>\n",
              "      <td>...</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45</th>\n",
              "      <td>/content/drive/MyDrive/topics/updates_garmin_n...</td>\n",
              "      <td>...</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-79dc6ba1-41b0-4b67-b6a6-c2e39f425aeb')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-79dc6ba1-41b0-4b67-b6a6-c2e39f425aeb button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-79dc6ba1-41b0-4b67-b6a6-c2e39f425aeb');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "5개로 설정하니 되게 세분화 되어버림. 중심 개수를 3개로 낮춰서 군집화를 해보자."
      ],
      "metadata": {
        "id": "7iAyh6Xu21-g"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.cluster import KMeans\n",
        "\n",
        "#3개의 집합으로 군집화\n",
        "km_cluster=KMeans(n_clusters=3, max_iter=10000, random_state=0)\n",
        "km_cluster.fit(feature_vect)\n",
        "cluster_label=km_cluster.labels_\n",
        "cluster_centers=km_cluster.cluster_centers_\n",
        "\n",
        "#소속 군집을 cluster_label 칼럼으로 할당하고 cluster_label 값으로 정렬\n",
        "document_df['cluster_label']=cluster_label\n",
        "document_df.sort_values(by='cluster_label')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "_MB4TmKX22Cn",
        "outputId": "b1dc5145-e661-45ba-9dad-5f2a2455da97"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/cluster/_kmeans.py:870: FutureWarning: The default value of `n_init` will change from 10 to 'auto' in 1.4. Set the value of `n_init` explicitly to suppress the warning\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                             filename  \\\n",
              "50  /content/drive/MyDrive/topics/voice_garmin_nuv...   \n",
              "21  /content/drive/MyDrive/topics/interior_toyota_...   \n",
              "22  /content/drive/MyDrive/topics/gas_mileage_toyo...   \n",
              "14  /content/drive/MyDrive/topics/mileage_honda_ac...   \n",
              "13  /content/drive/MyDrive/topics/comfort_toyota_c...   \n",
              "41  /content/drive/MyDrive/topics/transmission_toy...   \n",
              "8   /content/drive/MyDrive/topics/comfort_honda_ac...   \n",
              "42  /content/drive/MyDrive/topics/sound_ipod_nano_8gb   \n",
              "26  /content/drive/MyDrive/topics/interior_honda_a...   \n",
              "37  /content/drive/MyDrive/topics/quality_toyota_c...   \n",
              "31  /content/drive/MyDrive/topics/seats_honda_acco...   \n",
              "49  /content/drive/MyDrive/topics/video_ipod_nano_8gb   \n",
              "18  /content/drive/MyDrive/topics/performance_hond...   \n",
              "20  /content/drive/MyDrive/topics/free_bestwestern...   \n",
              "27  /content/drive/MyDrive/topics/service_holiday_...   \n",
              "24  /content/drive/MyDrive/topics/location_bestwes...   \n",
              "23  /content/drive/MyDrive/topics/parking_bestwest...   \n",
              "28  /content/drive/MyDrive/topics/price_holiday_in...   \n",
              "32  /content/drive/MyDrive/topics/rooms_swissotel_...   \n",
              "39  /content/drive/MyDrive/topics/service_bestwest...   \n",
              "33  /content/drive/MyDrive/topics/room_holiday_inn...   \n",
              "16  /content/drive/MyDrive/topics/food_swissotel_c...   \n",
              "15  /content/drive/MyDrive/topics/location_holiday...   \n",
              "11  /content/drive/MyDrive/topics/food_holiday_inn...   \n",
              "46  /content/drive/MyDrive/topics/staff_swissotel_...   \n",
              "47  /content/drive/MyDrive/topics/staff_bestwester...   \n",
              "48  /content/drive/MyDrive/topics/service_swissote...   \n",
              "1   /content/drive/MyDrive/topics/bathroom_bestwes...   \n",
              "38  /content/drive/MyDrive/topics/rooms_bestwester...   \n",
              "40  /content/drive/MyDrive/topics/size_asus_netboo...   \n",
              "43       /content/drive/MyDrive/topics/speed_windows7   \n",
              "44  /content/drive/MyDrive/topics/speed_garmin_nuv...   \n",
              "45  /content/drive/MyDrive/topics/updates_garmin_n...   \n",
              "36  /content/drive/MyDrive/topics/price_amazon_kindle   \n",
              "35  /content/drive/MyDrive/topics/screen_garmin_nu...   \n",
              "0   /content/drive/MyDrive/topics/accuracy_garmin_...   \n",
              "30  /content/drive/MyDrive/topics/satellite_garmin...   \n",
              "29  /content/drive/MyDrive/topics/screen_ipod_nano...   \n",
              "19  /content/drive/MyDrive/topics/navigation_amazo...   \n",
              "17  /content/drive/MyDrive/topics/keyboard_netbook...   \n",
              "12  /content/drive/MyDrive/topics/directions_garmi...   \n",
              "10  /content/drive/MyDrive/topics/display_garmin_n...   \n",
              "9   /content/drive/MyDrive/topics/battery-life_ama...   \n",
              "7   /content/drive/MyDrive/topics/fonts_amazon_kindle   \n",
              "6   /content/drive/MyDrive/topics/battery-life_net...   \n",
              "5   /content/drive/MyDrive/topics/battery-life_ipo...   \n",
              "4     /content/drive/MyDrive/topics/features_windows7   \n",
              "3   /content/drive/MyDrive/topics/buttons_amazon_k...   \n",
              "2   /content/drive/MyDrive/topics/eyesight-issues_...   \n",
              "34  /content/drive/MyDrive/topics/screen_netbook_1...   \n",
              "25  /content/drive/MyDrive/topics/performance_netb...   \n",
              "\n",
              "                                         opinion_text  cluster_label  \n",
              "50                                                ...              0  \n",
              "21                                                ...              0  \n",
              "22                                                ...              0  \n",
              "14                                                ...              0  \n",
              "13                                                ...              0  \n",
              "41                                                ...              0  \n",
              "8                                                 ...              0  \n",
              "42      headphone jack i got a clear case for it a...              0  \n",
              "26                                                ...              0  \n",
              "37                                                ...              0  \n",
              "31                                                ...              0  \n",
              "49                                                ...              0  \n",
              "18                                                ...              0  \n",
              "20                                                ...              1  \n",
              "27                                                ...              1  \n",
              "24                                                ...              1  \n",
              "23                                                ...              1  \n",
              "28                                                ...              1  \n",
              "32                                                ...              1  \n",
              "39                                                ...              1  \n",
              "33                                                ...              1  \n",
              "16                                                ...              1  \n",
              "15                                                ...              1  \n",
              "11                                                ...              1  \n",
              "46                                                ...              1  \n",
              "47                                                ...              1  \n",
              "48                                                ...              1  \n",
              "1                                                 ...              1  \n",
              "38                                                ...              1  \n",
              "40                                                ...              2  \n",
              "43                                                ...              2  \n",
              "44                                                ...              2  \n",
              "45                                                ...              2  \n",
              "36                                                ...              2  \n",
              "35                                                ...              2  \n",
              "0                                                 ...              2  \n",
              "30                                                ...              2  \n",
              "29                                                ...              2  \n",
              "19                                                ...              2  \n",
              "17                                                ...              2  \n",
              "12                                                ...              2  \n",
              "10                                                ...              2  \n",
              "9                                                 ...              2  \n",
              "7                                                 ...              2  \n",
              "6                                                 ...              2  \n",
              "5                                                 ...              2  \n",
              "4                                                 ...              2  \n",
              "3                                                 ...              2  \n",
              "2                                                 ...              2  \n",
              "34                                                ...              2  \n",
              "25                                                ...              2  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-5915baf7-9f4a-4099-b674-6b89f8283ecd\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>filename</th>\n",
              "      <th>opinion_text</th>\n",
              "      <th>cluster_label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>50</th>\n",
              "      <td>/content/drive/MyDrive/topics/voice_garmin_nuv...</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>21</th>\n",
              "      <td>/content/drive/MyDrive/topics/interior_toyota_...</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>22</th>\n",
              "      <td>/content/drive/MyDrive/topics/gas_mileage_toyo...</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>/content/drive/MyDrive/topics/mileage_honda_ac...</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>/content/drive/MyDrive/topics/comfort_toyota_c...</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41</th>\n",
              "      <td>/content/drive/MyDrive/topics/transmission_toy...</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>/content/drive/MyDrive/topics/comfort_honda_ac...</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>42</th>\n",
              "      <td>/content/drive/MyDrive/topics/sound_ipod_nano_8gb</td>\n",
              "      <td>headphone jack i got a clear case for it a...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>26</th>\n",
              "      <td>/content/drive/MyDrive/topics/interior_honda_a...</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>37</th>\n",
              "      <td>/content/drive/MyDrive/topics/quality_toyota_c...</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>31</th>\n",
              "      <td>/content/drive/MyDrive/topics/seats_honda_acco...</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49</th>\n",
              "      <td>/content/drive/MyDrive/topics/video_ipod_nano_8gb</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>/content/drive/MyDrive/topics/performance_hond...</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>20</th>\n",
              "      <td>/content/drive/MyDrive/topics/free_bestwestern...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27</th>\n",
              "      <td>/content/drive/MyDrive/topics/service_holiday_...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24</th>\n",
              "      <td>/content/drive/MyDrive/topics/location_bestwes...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>23</th>\n",
              "      <td>/content/drive/MyDrive/topics/parking_bestwest...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28</th>\n",
              "      <td>/content/drive/MyDrive/topics/price_holiday_in...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>32</th>\n",
              "      <td>/content/drive/MyDrive/topics/rooms_swissotel_...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>39</th>\n",
              "      <td>/content/drive/MyDrive/topics/service_bestwest...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>33</th>\n",
              "      <td>/content/drive/MyDrive/topics/room_holiday_inn...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>/content/drive/MyDrive/topics/food_swissotel_c...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>/content/drive/MyDrive/topics/location_holiday...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>/content/drive/MyDrive/topics/food_holiday_inn...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>46</th>\n",
              "      <td>/content/drive/MyDrive/topics/staff_swissotel_...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>47</th>\n",
              "      <td>/content/drive/MyDrive/topics/staff_bestwester...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>48</th>\n",
              "      <td>/content/drive/MyDrive/topics/service_swissote...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>/content/drive/MyDrive/topics/bathroom_bestwes...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>38</th>\n",
              "      <td>/content/drive/MyDrive/topics/rooms_bestwester...</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>40</th>\n",
              "      <td>/content/drive/MyDrive/topics/size_asus_netboo...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>43</th>\n",
              "      <td>/content/drive/MyDrive/topics/speed_windows7</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>44</th>\n",
              "      <td>/content/drive/MyDrive/topics/speed_garmin_nuv...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45</th>\n",
              "      <td>/content/drive/MyDrive/topics/updates_garmin_n...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>36</th>\n",
              "      <td>/content/drive/MyDrive/topics/price_amazon_kindle</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>35</th>\n",
              "      <td>/content/drive/MyDrive/topics/screen_garmin_nu...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>/content/drive/MyDrive/topics/accuracy_garmin_...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>30</th>\n",
              "      <td>/content/drive/MyDrive/topics/satellite_garmin...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>29</th>\n",
              "      <td>/content/drive/MyDrive/topics/screen_ipod_nano...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>/content/drive/MyDrive/topics/navigation_amazo...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>/content/drive/MyDrive/topics/keyboard_netbook...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>/content/drive/MyDrive/topics/directions_garmi...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>/content/drive/MyDrive/topics/display_garmin_n...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>/content/drive/MyDrive/topics/battery-life_ama...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>/content/drive/MyDrive/topics/fonts_amazon_kindle</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>/content/drive/MyDrive/topics/battery-life_net...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>/content/drive/MyDrive/topics/battery-life_ipo...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>/content/drive/MyDrive/topics/features_windows7</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>/content/drive/MyDrive/topics/buttons_amazon_k...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>/content/drive/MyDrive/topics/eyesight-issues_...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>34</th>\n",
              "      <td>/content/drive/MyDrive/topics/screen_netbook_1...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25</th>\n",
              "      <td>/content/drive/MyDrive/topics/performance_netb...</td>\n",
              "      <td>...</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-5915baf7-9f4a-4099-b674-6b89f8283ecd')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-5915baf7-9f4a-4099-b674-6b89f8283ecd button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-5915baf7-9f4a-4099-b674-6b89f8283ecd');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "0은 포터블 전자기기 리뷰, 1은 호텔 리뷰, 2는 자동차 리뷰로 군집 구성됨.\n",
        "\n",
        "### 군집별 핵심 단어 추출하기 \n",
        "\n",
        "각 군집에 속한 문서는 핵심 단어를 주축으로 군집화돼 있을 것이다. \n",
        "\n",
        "* clusters_enters_ : KMeans 객체는 각 군집을 구성하는 단어 피처가 군집의 중심을 기준으로 얼마나 가깝게 위치해 있는지 나타냄\n",
        "  - 행은 개별 군집, 열은 개별 피처를 의미한다.\n",
        "  - 배열값으로 제공된다.\n",
        "  - 각 배열 내의 값은 개별 군집 내의 상대 위치를 숫자 값으로 표현한 일종의 좌표 값이다. \n",
        "  - ex) cluster_centers[0,1]은 0번 군집에서 두 번째 피처의 위치값이다.\n"
      ],
      "metadata": {
        "id": "1KSjpf0k2dV-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "cluster_centers=km_cluster.cluster_centers_\n",
        "print('cluster_centers shape :', cluster_centers.shape)\n",
        "print(cluster_centers)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XjOmJpg22daF",
        "outputId": "86dd2316-8f0c-4f5c-a944-313d04e1b39a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "cluster_centers shape : (3, 4611)\n",
            "[[0.00067703 0.00071193 0.         ... 0.         0.         0.        ]\n",
            " [0.         0.00099499 0.00174637 ... 0.         0.00183397 0.00144581]\n",
            " [0.01102405 0.         0.         ... 0.00802599 0.         0.        ]]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "(3,2409) 배열인데, 이는 군집이 3개, word피처가 2409개로 구성되었음을 의미한다.\n",
        "\n",
        "2409개의 피처가 개별 중심과 얼마나 가까운가를 상대 값으로 나타낸 것이다. 1에 가까울수록 중심과 가까운 값을 의미한다.\n",
        "\n",
        "이제 해당 속성값을 이용해 각 군집별 핵심 단어를 찾아보자. \n",
        "\n",
        "* argsort()[:,::-1] : 배열 내 값이 큰 순으로 정렬된 위치 인덱스 값을 반환.\n",
        "\n",
        "인덱스 값이 필요한 이유: 핵심 단어 피처의 이름을 출력하기 위해서\n",
        "\n",
        "**<위를 처리할 함수 생성>**"
      ],
      "metadata": {
        "id": "lFz3rLnh2ddD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#군집별 top n 핵심 단어, 그 단어의 중심 위치 상댓값, 대상 파일명을 반환함.\n",
        "def get_cluster_details(cluster_model, cluster_data, feature_names, clusters_num, top_n_features=10):\n",
        "  cluster_details={}\n",
        "\n",
        "  #cluster_centers array 값이 큰 순으로 정렬된 인덱스 값을 반환\n",
        "  #군집 중심점 별 할당된 word 피처들의 거리값이 큰 순으로 값을 구하기 위함\n",
        "  centroid_feature_ordered_ind=cluster_model.cluster_centers_.argsort()[:, ::-1]\n",
        "\n",
        "  #개별 군집별로 반복하며 핵심 단어, 그 단어의 중심 위치 상댓값, 대상 파일명 입력\n",
        "  for cluster_num in range(clusters_num):\n",
        "    #개별 군집별 정보를 담을 데이터 초기화\n",
        "    cluster_details[cluster_num]={}\n",
        "    cluster_details[cluster_num]['cluster']=cluster_num\n",
        "\n",
        "    #cluster_centers_.argsort()로 구한 인덱스를 이용해 top n 피처 단어를 구함\n",
        "    top_feature_indexes=centroid_feature_ordered_ind[cluster_num, :top_n_features]\n",
        "    top_features=[feature_names[ind] for ind in top_feature_indexes]\n",
        "\n",
        "    #top_feature_indexes를 이용해 해당 피처 단어의 중심 위치 상댓값 구함.\n",
        "    top_feature_values=cluster_model.cluster_centers_[cluster_num, top_feature_indexes].tolist()\n",
        "\n",
        "    #cluster_details 딕셔너리 객체에 개별 군집별 핵심단어와 중심위치 상댓값, 해당 파일명 입력\n",
        "    cluster_details[cluster_num]['top_features']=top_features\n",
        "    cluster_details[cluster_num]['top_features_value']=top_feature_values\n",
        "    filenames=cluster_data[cluster_data['cluster_label']==cluster_num]['filename']\n",
        "    filenames=filenames.values.tolist()\n",
        "\n",
        "    cluster_details[cluster_num]['filenames']=filenames\n",
        "\n",
        "  return cluster_details\n"
      ],
      "metadata": {
        "id": "YX8TiHjT2dir"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#보기 좋게 표현하기 위한 함수수\n",
        "\n",
        "def print_cluster_details(cluster_details):\n",
        "  for cluster_num, cluster_detail in cluster_details.items():\n",
        "    print('##### Cluster {0}'.format(cluster_num))\n",
        "    print('Top features:', cluster_detail['top_features'])\n",
        "    print('reviews 파일명:', cluster_detail['filenames'][:7])\n",
        "    print('==============================================')"
      ],
      "metadata": {
        "id": "nPmjW0hiOOq1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#함수 호출\n",
        "feature_names=tfidf_vect.get_feature_names_out() #IF-IDF 변환된 tfidf_vect객체에서 추출.\n",
        "\n",
        "cluster_details=get_cluster_details(cluster_model=km_cluster, cluster_data=document_df, feature_names=feature_names, clusters_num=3, top_n_features=10)\n",
        "print_cluster_details(cluster_details)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BrUET6NaOOue",
        "outputId": "90e2231f-4f70-46a3-dad3-3430b064cd25"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "##### Cluster 0\n",
            "Top features: ['interior', 'seat', 'mileage', 'comfortable', 'quality', 'gas', 'gas mileage', 'transmission', 'car', 'performance']\n",
            "reviews 파일명: ['/content/drive/MyDrive/topics/comfort_honda_accord_2008', '/content/drive/MyDrive/topics/comfort_toyota_camry_2007', '/content/drive/MyDrive/topics/mileage_honda_accord_2008', '/content/drive/MyDrive/topics/performance_honda_accord_2008', '/content/drive/MyDrive/topics/interior_toyota_camry_2007', '/content/drive/MyDrive/topics/gas_mileage_toyota_camry_2007', '/content/drive/MyDrive/topics/interior_honda_accord_2008']\n",
            "==============================================\n",
            "##### Cluster 1\n",
            "Top features: ['room', 'hotel', 'service', 'staff', 'food', 'location', 'bathroom', 'clean', 'price', 'parking']\n",
            "reviews 파일명: ['/content/drive/MyDrive/topics/bathroom_bestwestern_hotel_sfo', '/content/drive/MyDrive/topics/food_holiday_inn_london', '/content/drive/MyDrive/topics/location_holiday_inn_london', '/content/drive/MyDrive/topics/food_swissotel_chicago', '/content/drive/MyDrive/topics/free_bestwestern_hotel_sfo', '/content/drive/MyDrive/topics/parking_bestwestern_hotel_sfo', '/content/drive/MyDrive/topics/location_bestwestern_hotel_sfo']\n",
            "==============================================\n",
            "##### Cluster 2\n",
            "Top features: ['screen', 'battery', 'keyboard', 'battery life', 'life', 'kindle', 'direction', 'size', 'map', 'button']\n",
            "reviews 파일명: ['/content/drive/MyDrive/topics/accuracy_garmin_nuvi_255W_gps', '/content/drive/MyDrive/topics/eyesight-issues_amazon_kindle', '/content/drive/MyDrive/topics/buttons_amazon_kindle', '/content/drive/MyDrive/topics/features_windows7', '/content/drive/MyDrive/topics/battery-life_ipod_nano_8gb', '/content/drive/MyDrive/topics/battery-life_netbook_1005ha', '/content/drive/MyDrive/topics/fonts_amazon_kindle']\n",
            "==============================================\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "군집화 0은 화면과 배터리 수명 등이 핵심 단어\n",
        "\n",
        "군집화 1는 방과 서비스 등이 핵심 단어\n",
        "\n",
        "군집화 2는 실내 인테리어, 좌석, 연료 효율 등이 핵심 단어\n",
        "\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "## 10 텍스트 분석 실습 - 캐글 Mercary Price Suggestion Challenge\n",
        "\n",
        "일본의 대형 온라인 쇼핑몰인 Mercari사의 제품에 대해 가격을 예측하는 과제다. \n",
        "제공되는 데이터 세트는 제품에 대한 여러 속성 및 제품 설명 등의 텍스트 데이터로 구성.\n",
        "\n",
        "\n",
        "\n",
        "---\n",
        "price : 타겟변수. \n",
        "\n",
        "회귀 피처를 학습한 뒤 price를 예측하는 문제이다. \n",
        "\n",
        "기존 회귀와 다른점은 item_Description과 같은 텍스트 형태의 비정형 데이터와 다른 정형 속성을 같이 적용해 회귀를 수행한다는 점이다.\n",
        "\n",
        "### 데이터 전처리"
      ],
      "metadata": {
        "id": "1l0hvlNuOrMz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "os.chdir(\"/content/drive/MyDrive/ESAA_data/\")"
      ],
      "metadata": {
        "id": "7BlRNnZiRCmw"
      },
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install pyunpack"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vkaJfMNxQyIL",
        "outputId": "ad5fcffa-307a-46ad-c414-d5a4dbc33948"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: pyunpack in /usr/local/lib/python3.10/dist-packages (0.3)\n",
            "Requirement already satisfied: easyprocess in /usr/local/lib/python3.10/dist-packages (from pyunpack) (1.1)\n",
            "Requirement already satisfied: entrypoint2 in /usr/local/lib/python3.10/dist-packages (from pyunpack) (1.1)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install patool"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3SYMLA1sQ1Zt",
        "outputId": "69d330d9-bdb0-4519-f63b-dadc4a281c45"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: patool in /usr/local/lib/python3.10/dist-packages (1.12)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from pyunpack import Archive\n",
        "Archive('train.tsv.7z').extractall('/content/drive/MyDrive/ESAA_data')"
      ],
      "metadata": {
        "id": "MHqc-zTmQpMw"
      },
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import Ridge, LogisticRegression\n",
        "from sklearn.model_selection import train_test_split, cross_val_score\n",
        "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\n",
        "import pandas as pd\n",
        "\n",
        "mercari_df=pd.read_csv('/content/drive/MyDrive/ESAA_data/train.tsv', sep='\\t')\n",
        "print(mercari_df.shape)\n",
        "mercari_df.head(3)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 162
        },
        "id": "ffr5NFDYPDH-",
        "outputId": "f4598070-3e7a-4948-d8d6-d462359ed344"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(1482535, 8)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   train_id                                 name  item_condition_id  \\\n",
              "0         0  MLB Cincinnati Reds T Shirt Size XL                  3   \n",
              "1         1     Razer BlackWidow Chroma Keyboard                  3   \n",
              "2         2                       AVA-VIV Blouse                  1   \n",
              "\n",
              "                                       category_name brand_name  price  \\\n",
              "0                                  Men/Tops/T-shirts        NaN   10.0   \n",
              "1  Electronics/Computers & Tablets/Components & P...      Razer   52.0   \n",
              "2                        Women/Tops & Blouses/Blouse     Target   10.0   \n",
              "\n",
              "   shipping                                   item_description  \n",
              "0         1                                 No description yet  \n",
              "1         0  This keyboard is in great condition and works ...  \n",
              "2         1  Adorable top with a hint of lace and a key hol...  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-9c1f494e-d02d-4202-8ef0-43e6d3b40b18\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>train_id</th>\n",
              "      <th>name</th>\n",
              "      <th>item_condition_id</th>\n",
              "      <th>category_name</th>\n",
              "      <th>brand_name</th>\n",
              "      <th>price</th>\n",
              "      <th>shipping</th>\n",
              "      <th>item_description</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>MLB Cincinnati Reds T Shirt Size XL</td>\n",
              "      <td>3</td>\n",
              "      <td>Men/Tops/T-shirts</td>\n",
              "      <td>NaN</td>\n",
              "      <td>10.0</td>\n",
              "      <td>1</td>\n",
              "      <td>No description yet</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>Razer BlackWidow Chroma Keyboard</td>\n",
              "      <td>3</td>\n",
              "      <td>Electronics/Computers &amp; Tablets/Components &amp; P...</td>\n",
              "      <td>Razer</td>\n",
              "      <td>52.0</td>\n",
              "      <td>0</td>\n",
              "      <td>This keyboard is in great condition and works ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>AVA-VIV Blouse</td>\n",
              "      <td>1</td>\n",
              "      <td>Women/Tops &amp; Blouses/Blouse</td>\n",
              "      <td>Target</td>\n",
              "      <td>10.0</td>\n",
              "      <td>1</td>\n",
              "      <td>Adorable top with a hint of lace and a key hol...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-9c1f494e-d02d-4202-8ef0-43e6d3b40b18')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-9c1f494e-d02d-4202-8ef0-43e6d3b40b18 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-9c1f494e-d02d-4202-8ef0-43e6d3b40b18');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#null확인\n",
        "print(mercari_df.info())"
      ],
      "metadata": {
        "id": "PHS1i-ixPDJq",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3e8dc6da-45ec-456b-c8fd-0a6bc29a06df"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 1482535 entries, 0 to 1482534\n",
            "Data columns (total 8 columns):\n",
            " #   Column             Non-Null Count    Dtype  \n",
            "---  ------             --------------    -----  \n",
            " 0   train_id           1482535 non-null  int64  \n",
            " 1   name               1482535 non-null  object \n",
            " 2   item_condition_id  1482535 non-null  int64  \n",
            " 3   category_name      1476208 non-null  object \n",
            " 4   brand_name         849853 non-null   object \n",
            " 5   price              1482535 non-null  float64\n",
            " 6   shipping           1482535 non-null  int64  \n",
            " 7   item_description   1482531 non-null  object \n",
            "dtypes: float64(1), int64(3), object(4)\n",
            "memory usage: 90.5+ MB\n",
            "None\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "null데이터는 이후에 적절한 문자열로 치환해준다.\n",
        "\n",
        "target값인 price 칼럼의 데이터 분포도를 살펴보자.\n",
        " 왜곡돼 있을 경우 로그를를 씌워서 변환하면 정규 분포 형태를 갖게 된다."
      ],
      "metadata": {
        "id": "E7azs8AyPGsK"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#target값 price 칼럼 분포도 \n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "%matplotlib inline\n",
        "\n",
        "y_train_df=mercari_df['price']\n",
        "plt.figure(figsize=(6,4))\n",
        "sns.distplot(y_train_df, kde=False)"
      ],
      "metadata": {
        "id": "-RA-oiJZPGzl",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 623
        },
        "outputId": "e43533e9-1576-472d-889c-7a59e1ce60b1"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-9-d1032245e053>:8: UserWarning: \n",
            "\n",
            "`distplot` is a deprecated function and will be removed in seaborn v0.14.0.\n",
            "\n",
            "Please adapt your code to use either `displot` (a figure-level function with\n",
            "similar flexibility) or `histplot` (an axes-level function for histograms).\n",
            "\n",
            "For a guide to updating your code to use the new functions, please see\n",
            "https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751\n",
            "\n",
            "  sns.distplot(y_train_df, kde=False)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<Axes: xlabel='price'>"
            ]
          },
          "metadata": {},
          "execution_count": 9
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 600x400 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgQAAAGCCAYAAACfA9vOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAnY0lEQVR4nO3de1zVdYL/8fcB5eDt4AW5GYiXvDQqEOaJzNKJQseh2nZ2HHWDYbxsjbYWk6N0gbEacSY1dstyMi+1M4q5j7SZZC1DWackXVGmLCXvuCaouXAUDZDz+f3Rj1NnuMhBrvp6Ph7fx0M+38/n+/18Hh/5njffy/lajDFGAADghubV2h0AAACtj0AAAAAIBAAAgEAAAABEIAAAACIQAAAAEQgAAIAIBAAAQAQCAAAgAgEAAFA7CwQ7duxQfHy8QkJCZLFYtGnTJo+3YYzR4sWLNWjQIFmtVvXp00e//e1vm76zAAC0Ix1auwOeKCsrU0REhH7xi1/ooYceatQ25syZow8++ECLFy/W8OHDdf78eZ0/f76JewoAQPtiaa8vN7JYLNq4caMefPBBV1l5ebmefvpprVu3TiUlJRo2bJh+97vfaezYsZKkAwcOaMSIEdq/f78GDx7cOh0HAKANaleXDK5m9uzZys3NVWZmpj799FP90z/9k8aPH69Dhw5Jkv7yl7+of//+eu+999SvXz+Fh4dr+vTpnCEAANzwrptAUFhYqNWrV2vDhg0aM2aMBgwYoCeffFJ33nmnVq9eLUk6evSoTpw4oQ0bNuitt97SmjVrlJeXp5/85Cet3HsAAFpXu7qHoD6fffaZqqqqNGjQILfy8vJy9erVS5LkdDpVXl6ut956y1Vv5cqVio6OVkFBAZcRAAA3rOsmEFy8eFHe3t7Ky8uTt7e327quXbtKkoKDg9WhQwe30DB06FBJ355hIBAAAG5U100giIqKUlVVlc6cOaMxY8bUWmf06NG6cuWKjhw5ogEDBkiSvvzyS0lS3759W6yvAAC0Ne3qKYOLFy/q8OHDkr4NAEuXLtW4cePUs2dPhYWF6Z//+Z/18ccfa8mSJYqKitLZs2eVnZ2tESNGaOLEiXI6nbrtttvUtWtXZWRkyOl0atasWbLZbPrggw9aeXQAALSedhUIcnJyNG7cuBrliYmJWrNmjSorK/XCCy/orbfe0qlTp+Tv76/bb79dCxYs0PDhwyVJX331lR577DF98MEH6tKliyZMmKAlS5aoZ8+eLT0cAADajHYVCAAAQPO4bh47BAAAjUcgAAAA7eMpA6fTqa+++krdunWTxWJp7e4AANBuGGN04cIFhYSEyMur7vMA7SIQfPXVVwoNDW3tbgAA0G6dPHlSN910U53r20Ug6Natm6RvB2Oz2Vq5NwAAtB8Oh0OhoaGuz9K6tItAUH2ZwGazEQgAAGiEq11y56ZCAABAIAAAAAQCAAAgAgEAABCBAAAAiEAAAABEIAAAACIQAAAAEQgAAIAIBAAAQAQCAACgdvIug+aydldhveun2MNaqCcAALQuzhAAAAACAQAAIBAAAAARCAAAgAgEAABABAIAACACAQAAEIEAAACIQAAAAEQgAAAAakQg2LFjh+Lj4xUSEiKLxaJNmzbVW/+dd97Rvffeq969e8tmsykmJkbvv/9+Y/sLAACagceBoKysTBEREVq2bFmD6u/YsUP33nuvsrKylJeXp3Hjxik+Pl779u3zuLMAAKB5ePxyowkTJmjChAkNrp+RkeH288KFC/Xuu+/qL3/5i6KiojzdPQAAaAYt/rZDp9OpCxcuqGfPnnXWKS8vV3l5uetnh8PREl0DAOCG1eI3FS5evFgXL17UT3/60zrrpKeny8/Pz7WEhoa2YA8BALjxtGggWLt2rRYsWKC3335bAQEBddZLSUlRaWmpazl58mQL9hIAgBtPi10yyMzM1PTp07VhwwbFxsbWW9dqtcpqtbZQzwAAQIucIVi3bp2SkpK0bt06TZw4sSV2CQAAPODxGYKLFy/q8OHDrp+PHTum/Px89ezZU2FhYUpJSdGpU6f01ltvSfr2MkFiYqL+7d/+TXa7XUVFRZKkTp06yc/Pr4mGAQAAroXHZwj27NmjqKgo1yODycnJioqKUmpqqiTp9OnTKiwsdNV//fXXdeXKFc2aNUvBwcGuZc6cOU00BAAAcK08PkMwduxYGWPqXL9mzRq3n3NycjzdBQAAaGG8ywAAABAIAAAAgQAAAIhAAAAARCAAAAAiEAAAABEIAACACAQAAEAEAgAAIAIBAAAQgQAAAIhAAAAARCAAAAAiEAAAABEIAACACAQAAEAEAgAAIAIBAAAQgQAAAIhAAAAARCAAAAAiEAAAABEIAACACAQAAEAEAgAAIAIBAAAQgQAAAIhAAAAARCAAAAAiEAAAABEIAACACAQAAEAEAgAAIAIBAAAQgQAAAIhAAAAA1IhAsGPHDsXHxyskJEQWi0WbNm26apucnBzdeuutslqtGjhwoNasWdOIrgIAgObicSAoKytTRESEli1b1qD6x44d08SJEzVu3Djl5+fr8ccf1/Tp0/X+++973FkAANA8OnjaYMKECZowYUKD6y9fvlz9+vXTkiVLJElDhw7VRx99pJdeeklxcXGe7h4AADSDZr+HIDc3V7GxsW5lcXFxys3NrbNNeXm5HA6H2wIAAJpPsweCoqIiBQYGupUFBgbK4XDo8uXLtbZJT0+Xn5+fawkNDW3ubgIAcENrk08ZpKSkqLS01LWcPHmytbsEAMB1zeN7CDwVFBSk4uJit7Li4mLZbDZ16tSp1jZWq1VWq7W5uwYAAP6/Zj9DEBMTo+zsbLeyrVu3KiYmprl3DQAAGsjjQHDx4kXl5+crPz9f0rePFebn56uwsFDSt6f7ExISXPUfeeQRHT16VL/+9a918OBBvfrqq3r77bf1xBNPNM0IAADANfM4EOzZs0dRUVGKioqSJCUnJysqKkqpqamSpNOnT7vCgST169dPmzdv1tatWxUREaElS5bojTfe4JFDAADaEIsxxrR2J67G4XDIz89PpaWlstlsTbbdtbsK610/xR7WZPsCAKA1NPQztE0+ZQAAAFoWgQAAABAIAAAAgQAAAIhAAAAARCAAAAAiEAAAABEIAACACAQAAEAEAgAAIAIBAAAQgQAAAIhAAAAARCAAAAAiEAAAABEIAACACAQAAEAEAgAAIAIBAAAQgQAAAIhAAAAARCAAAAAiEAAAABEIAACACAQAAEAEAgAAIAIBAAAQgQAAAIhAAAAARCAAAAAiEAAAABEIAACACAQAAEAEAgAAIAIBAABQIwPBsmXLFB4eLl9fX9ntdu3evbve+hkZGRo8eLA6deqk0NBQPfHEE/rmm28a1WEAAND0PA4E69evV3JystLS0rR3715FREQoLi5OZ86cqbX+2rVrNX/+fKWlpenAgQNauXKl1q9fr6eeeuqaOw8AAJqGx4Fg6dKlmjFjhpKSknTLLbdo+fLl6ty5s1atWlVr/Z07d2r06NGaMmWKwsPDdd9992ny5MlXPasAAABajkeBoKKiQnl5eYqNjf1uA15eio2NVW5ubq1t7rjjDuXl5bkCwNGjR5WVlaUf/ehHde6nvLxcDofDbQEAAM2ngyeVz507p6qqKgUGBrqVBwYG6uDBg7W2mTJlis6dO6c777xTxhhduXJFjzzySL2XDNLT07VgwQJPugYAAK5Bsz9lkJOTo4ULF+rVV1/V3r179c4772jz5s16/vnn62yTkpKi0tJS13Ly5Mnm7iYAADc0j84Q+Pv7y9vbW8XFxW7lxcXFCgoKqrXNs88+q4cffljTp0+XJA0fPlxlZWWaOXOmnn76aXl51cwkVqtVVqvVk64BAIBr4NEZAh8fH0VHRys7O9tV5nQ6lZ2drZiYmFrbXLp0qcaHvre3tyTJGONpfwEAQDPw6AyBJCUnJysxMVEjR47UqFGjlJGRobKyMiUlJUmSEhIS1KdPH6Wnp0uS4uPjtXTpUkVFRclut+vw4cN69tlnFR8f7woGAACgdXkcCCZNmqSzZ88qNTVVRUVFioyM1JYtW1w3GhYWFrqdEXjmmWdksVj0zDPP6NSpU+rdu7fi4+P129/+tulGAQAAronFtIPz9g6HQ35+fiotLZXNZmuy7a7dVVjv+in2sCbbFwAAraGhn6G8ywAAABAIAAAAgQAAAIhAAAAARCAAAAAiEAAAABEIAACACAQAAEAEAgAAIAIBAAAQgQAAAIhAAAAARCAAAAAiEAAAABEIAACACAQAAEAEAgAAIAIBAAAQgQAAAIhAAAAARCAAAAAiEAAAABEIAACACAQAAEAEAgAAIAIBAAAQgQAAAIhAAAAARCAAAAAiEAAAABEIAACACAQAAEAEAgAAIAIBAAAQgQAAAKiRgWDZsmUKDw+Xr6+v7Ha7du/eXW/9kpISzZo1S8HBwbJarRo0aJCysrIa1WEAAND0OnjaYP369UpOTtby5ctlt9uVkZGhuLg4FRQUKCAgoEb9iooK3XvvvQoICNB//ud/qk+fPjpx4oS6d+/eFP0HAABNwONAsHTpUs2YMUNJSUmSpOXLl2vz5s1atWqV5s+fX6P+qlWrdP78ee3cuVMdO3aUJIWHh19brwEAQJPy6JJBRUWF8vLyFBsb+90GvLwUGxur3NzcWtv8+c9/VkxMjGbNmqXAwEANGzZMCxcuVFVV1bX1HAAANBmPzhCcO3dOVVVVCgwMdCsPDAzUwYMHa21z9OhRbdu2TVOnTlVWVpYOHz6sX/7yl6qsrFRaWlqtbcrLy1VeXu762eFweNJNAADgoWZ/ysDpdCogIECvv/66oqOjNWnSJD399NNavnx5nW3S09Pl5+fnWkJDQ5u7mwAA3NA8CgT+/v7y9vZWcXGxW3lxcbGCgoJqbRMcHKxBgwbJ29vbVTZ06FAVFRWpoqKi1jYpKSkqLS11LSdPnvSkmwAAwEMeBQIfHx9FR0crOzvbVeZ0OpWdna2YmJha24wePVqHDx+W0+l0lX355ZcKDg6Wj49PrW2sVqtsNpvbAgAAmo/HlwySk5O1YsUKvfnmmzpw4IAeffRRlZWVuZ46SEhIUEpKiqv+o48+qvPnz2vOnDn68ssvtXnzZi1cuFCzZs1qulEAAIBr4vFjh5MmTdLZs2eVmpqqoqIiRUZGasuWLa4bDQsLC+Xl9V3OCA0N1fvvv68nnnhCI0aMUJ8+fTRnzhzNmzev6UYBAACuicUYY1q7E1fjcDjk5+en0tLSJr18sHZXYb3rp9jDmmxfAAC0hoZ+hvIuAwAAQCAAAAAEAgAAIAIBAAAQgQAAAIhAAAAARCAAAAAiEAAAABEIAACACAQAAEAEAgAAIAIBAAAQgQAAAIhAAAAARCAAAAAiEAAAABEIAACACAQAAEAEAgAAIAIBAAAQgQAAAIhAAAAARCAAAAAiEAAAABEIAACACAQAAEAEAgAAIAIBAAAQgQAAAIhAAAAARCAAAAAiEAAAABEIAACACAQAAEAEAgAAIAIBAABQIwPBsmXLFB4eLl9fX9ntdu3evbtB7TIzM2WxWPTggw82ZrcAAKCZeBwI1q9fr+TkZKWlpWnv3r2KiIhQXFyczpw5U2+748eP68knn9SYMWMa3VkAANA8PA4ES5cu1YwZM5SUlKRbbrlFy5cvV+fOnbVq1ao621RVVWnq1KlasGCB+vfvf00dBgAATc+jQFBRUaG8vDzFxsZ+twEvL8XGxio3N7fOds8995wCAgI0bdq0Bu2nvLxcDofDbQEAAM3Ho0Bw7tw5VVVVKTAw0K08MDBQRUVFtbb56KOPtHLlSq1YsaLB+0lPT5efn59rCQ0N9aSbAADAQ836lMGFCxf08MMPa8WKFfL3929wu5SUFJWWlrqWkydPNmMvAQBAB08q+/v7y9vbW8XFxW7lxcXFCgoKqlH/yJEjOn78uOLj411lTqfz2x136KCCggINGDCgRjur1Sqr1epJ1wAAwDXw6AyBj4+PoqOjlZ2d7SpzOp3Kzs5WTExMjfpDhgzRZ599pvz8fNdy//33a9y4ccrPz+dSAAAAbYRHZwgkKTk5WYmJiRo5cqRGjRqljIwMlZWVKSkpSZKUkJCgPn36KD09Xb6+vho2bJhb++7du0tSjXIAANB6PA4EkyZN0tmzZ5WamqqioiJFRkZqy5YtrhsNCwsL5eXFFyACANCeWIwxprU7cTUOh0N+fn4qLS2VzWZrsu2u3VVY7/op9rAm2xcAAK2hoZ+h/CkPAAAIBAAAgEAAAABEIAAAACIQAAAAEQgAAIAIBAAAQAQCAAAgAgEAABCBAAAAiEAAAABEIAAAACIQAAAAEQgAAIAIBAAAQAQCAAAgAgEAABCBAAAAiEAAAABEIAAAACIQAAAAEQgAAIAIBAAAQAQCAAAgAgEAABCBAAAAiEAAAABEIAAAACIQAAAAEQgAAIAIBAAAQAQCAAAgAgEAABCBAAAAiEAAAADUyECwbNkyhYeHy9fXV3a7Xbt3766z7ooVKzRmzBj16NFDPXr0UGxsbL31AQBAy/M4EKxfv17JyclKS0vT3r17FRERobi4OJ05c6bW+jk5OZo8ebK2b9+u3NxchYaG6r777tOpU6euufMAAKBpWIwxxpMGdrtdt912m1555RVJktPpVGhoqB577DHNnz//qu2rqqrUo0cPvfLKK0pISGjQPh0Oh/z8/FRaWiqbzeZJd+u1dldhveun2MOabF8AALSGhn6GenSGoKKiQnl5eYqNjf1uA15eio2NVW5uboO2cenSJVVWVqpnz5511ikvL5fD4XBbAABA8/EoEJw7d05VVVUKDAx0Kw8MDFRRUVGDtjFv3jyFhIS4hYq/l56eLj8/P9cSGhrqSTcBAICHWvQpg0WLFikzM1MbN26Ur69vnfVSUlJUWlrqWk6ePNmCvQQA4MbTwZPK/v7+8vb2VnFxsVt5cXGxgoKC6m27ePFiLVq0SB9++KFGjBhRb12r1Sqr1epJ1wAAwDXw6AyBj4+PoqOjlZ2d7SpzOp3Kzs5WTExMne1+//vf6/nnn9eWLVs0cuTIxvcWAAA0C4/OEEhScnKyEhMTNXLkSI0aNUoZGRkqKytTUlKSJCkhIUF9+vRRenq6JOl3v/udUlNTtXbtWoWHh7vuNejatau6du3ahEMBAACN5XEgmDRpks6ePavU1FQVFRUpMjJSW7Zscd1oWFhYKC+v7048vPbaa6qoqNBPfvITt+2kpaXpN7/5zbX1HgAANAmPv4egNfA9BAAANE6zfA8BAAC4PhEIAAAAgQAAABAIAACAGvGUwY3kajcdStx4CAC4PnCGAAAAEAgAAACBAAAAiEAAAABEIAAAACIQAAAAEQgAAIAIBAAAQAQCAAAgAgEAABCBAAAAiEAAAABEIAAAACIQAAAAEQgAAIAIBAAAQAQCAAAgAgEAABCBAAAAiEAAAABEIAAAACIQAAAAEQgAAIAIBAAAQAQCAAAgAgEAABCBAAAASOrQ2h1o79buKqx3/RR7WAv1BACAxuMMAQAAIBAAAAACAQAAUCMDwbJlyxQeHi5fX1/Z7Xbt3r273vobNmzQkCFD5Ovrq+HDhysrK6tRnQUAAM3D45sK169fr+TkZC1fvlx2u10ZGRmKi4tTQUGBAgICatTfuXOnJk+erPT0dP34xz/W2rVr9eCDD2rv3r0aNmxYkwyiLbvaTYcSNx4CAFqfxRhjPGlgt9t122236ZVXXpEkOZ1OhYaG6rHHHtP8+fNr1J80aZLKysr03nvvucpuv/12RUZGavny5Q3ap8PhkJ+fn0pLS2Wz2Tzpbr0a8mHdFhAYAACN1dDPUI/OEFRUVCgvL08pKSmuMi8vL8XGxio3N7fWNrm5uUpOTnYri4uL06ZNm+rcT3l5ucrLy10/l5aWSvp2UE3pUtmFJt1ec3lj2+fNvo+fjgxt9n0AAFpe9Wfn1f7+9ygQnDt3TlVVVQoMDHQrDwwM1MGDB2ttU1RUVGv9oqKiOveTnp6uBQsW1CgPDeVDq7nMaO0OAACa1YULF+Tn51fn+jb5xUQpKSluZxWcTqfOnz+vXr16yWKxNMk+HA6HQkNDdfLkySa9DNFarrfxSIypPbjexiNdf2O63sYjMSZPGWN04cIFhYSE1FvPo0Dg7+8vb29vFRcXu5UXFxcrKCio1jZBQUEe1Zckq9Uqq9XqVta9e3dPutpgNpvtuvkPJV1/45EYU3twvY1Huv7GdL2NR2JMnqjvzEA1jx479PHxUXR0tLKzs11lTqdT2dnZiomJqbVNTEyMW31J2rp1a531AQBAy/P4kkFycrISExM1cuRIjRo1ShkZGSorK1NSUpIkKSEhQX369FF6erokac6cObr77ru1ZMkSTZw4UZmZmdqzZ49ef/31ph0JAABoNI8DwaRJk3T27FmlpqaqqKhIkZGR2rJli+vGwcLCQnl5fXfi4Y477tDatWv1zDPP6KmnntLNN9+sTZs2tfp3EFitVqWlpdW4NNFeXW/jkRhTe3C9jUe6/sZ0vY1HYkzNxePvIQAAANcf3mUAAAAIBAAAgEAAAABEIAAAALpBA4Gnr29uLenp6brtttvUrVs3BQQE6MEHH1RBQYFbnbFjx8pisbgtjzzyiFudwsJCTZw4UZ07d1ZAQIDmzp2rK1eutORQXH7zm9/U6O+QIUNc67/55hvNmjVLvXr1UteuXfWP//iPNb7Yqi2NR5LCw8NrjMlisWjWrFmS2v4c7dixQ/Hx8QoJCZHFYqnxnhFjjFJTUxUcHKxOnTopNjZWhw4dcqtz/vx5TZ06VTabTd27d9e0adN08eJFtzqffvqpxowZI19fX4WGhur3v/99q4ypsrJS8+bN0/Dhw9WlSxeFhIQoISFBX331lds2apvXRYsWtcqYrjZHP//5z2v0dfz48W512tMcSar1d8pisejFF1901WlLc9SQ43VTHd9ycnJ06623ymq1auDAgVqzZk3TDMLcYDIzM42Pj49ZtWqV+fzzz82MGTNM9+7dTXFxcWt3rYa4uDizevVqs3//fpOfn29+9KMfmbCwMHPx4kVXnbvvvtvMmDHDnD592rWUlpa61l+5csUMGzbMxMbGmn379pmsrCzj7+9vUlJSWmNIJi0tzfzgBz9w6+/Zs2dd6x955BETGhpqsrOzzZ49e8ztt99u7rjjDtf6tjYeY4w5c+aM23i2bt1qJJnt27cbY9r+HGVlZZmnn37avPPOO0aS2bhxo9v6RYsWGT8/P7Np0ybzt7/9zdx///2mX79+5vLly64648ePNxEREeaTTz4xf/3rX83AgQPN5MmTXetLS0tNYGCgmTp1qtm/f79Zt26d6dSpk/nDH/7Q4mMqKSkxsbGxZv369ebgwYMmNzfXjBo1ykRHR7tto2/fvua5555zm7fv/+615JiuNkeJiYlm/Pjxbn09f/68W532NEfGGLexnD592qxatcpYLBZz5MgRV522NEcNOV43xfHt6NGjpnPnziY5Odl88cUX5uWXXzbe3t5my5Yt1zyGGy4QjBo1ysyaNcv1c1VVlQkJCTHp6emt2KuGOXPmjJFk/vu//9tVdvfdd5s5c+bU2SYrK8t4eXmZoqIiV9lrr71mbDabKS8vb87u1iotLc1ERETUuq6kpMR07NjRbNiwwVV24MABI8nk5uYaY9reeGozZ84cM2DAAON0Oo0x7WuO/v7A7HQ6TVBQkHnxxRddZSUlJcZqtZp169YZY4z54osvjCTzP//zP646//Vf/2UsFos5deqUMcaYV1991fTo0cNtPPPmzTODBw9u5hHVHFNtdu/ebSSZEydOuMr69u1rXnrppTrbtNaY6goEDzzwQJ1troc5euCBB8wPf/hDt7K2OkfG1DxeN9Xx7de//rX5wQ9+4LavSZMmmbi4uGvu8w11yaD69c2xsbGusqu9vrktqX4NdM+ePd3K//SnP8nf31/Dhg1TSkqKLl265FqXm5ur4cOHu71xMi4uTg6HQ59/3vyvVa7NoUOHFBISov79+2vq1KkqLCyUJOXl5amystJtfoYMGaKwsDDX/LTF8XxfRUWF/vjHP+oXv/iF24u42tscVTt27JiKiorc5sTPz092u91tTrp3766RI0e66sTGxsrLy0u7du1y1bnrrrvk4+PjqhMXF6eCggL93//9XwuNpm6lpaWyWCw13pmyaNEi9erVS1FRUXrxxRfdTt22tTHl5OQoICBAgwcP1qOPPqqvv/7ara/teY6Ki4u1efNmTZs2rca6tjpHf3+8bqrjW25urts2qus0xWdYm3zbYXNpzOub2wqn06nHH39co0ePdvuWxylTpqhv374KCQnRp59+qnnz5qmgoEDvvPOOpLpfP129rqXZ7XatWbNGgwcP1unTp7VgwQKNGTNG+/fvV1FRkXx8fGoclL//uuy2Np6/t2nTJpWUlOjnP/+5q6y9zdH3Ve+/vleYFxUVKSAgwG19hw4d1LNnT7c6/fr1q7GN6nU9evRolv43xDfffKN58+Zp8uTJbi+V+dd//Vfdeuut6tmzp3bu3KmUlBSdPn1aS5culdS2xjR+/Hg99NBD6tevn44cOaKnnnpKEyZMUG5urry9vdv9HL355pvq1q2bHnroIbfytjpHtR2vm+r4Vlcdh8Ohy5cvq1OnTo3u9w0VCNqzWbNmaf/+/froo4/cymfOnOn69/DhwxUcHKx77rlHR44c0YABA1q6m1c1YcIE179HjBghu92uvn376u23376m/8htxcqVKzVhwgS314y2tzm6kVRWVuqnP/2pjDF67bXX3NZ9/xXsI0aMkI+Pj/7lX/5F6enpbe4rc3/2s5+5/j18+HCNGDFCAwYMUE5Oju65555W7FnTWLVqlaZOnSpfX1+38rY6R3Udr9u6G+qSQWNe39wWzJ49W++99562b9+um266qd66drtdknT48GFJdb9+unpda+vevbsGDRqkw4cPKygoSBUVFSopKXGr8/35acvjOXHihD788ENNnz693nrtaY6q91/f70xQUJDOnDnjtv7KlSs6f/58m5636jBw4sQJbd269aqvnLXb7bpy5YqOHz8uqW2OqVr//v3l7+/v9n+sPc6RJP31r39VQUHBVX+vpLYxR3Udr5vq+FZXHZvNds1/VN1QgaAxr29uTcYYzZ49Wxs3btS2bdtqnPqqTX5+viQpODhY0revn/7ss8/cDgbVB79bbrmlWfrtiYsXL+rIkSMKDg5WdHS0Onbs6DY/BQUFKiwsdM1PWx7P6tWrFRAQoIkTJ9Zbrz3NUb9+/RQUFOQ2Jw6HQ7t27XKbk5KSEuXl5bnqbNu2TU6n0xV+YmJitGPHDlVWVrrqbN26VYMHD26VU9HVYeDQoUP68MMP1atXr6u2yc/Pl5eXl+vUe1sb0/f97//+r77++mu3/2PtbY6qrVy5UtHR0YqIiLhq3daco6sdr5vq+BYTE+O2jeo6TfIZds23JbYzmZmZxmq1mjVr1pgvvvjCzJw503Tv3t3trs624tFHHzV+fn4mJyfH7bGaS5cuGWOMOXz4sHnuuefMnj17zLFjx8y7775r+vfvb+666y7XNqofY7nvvvtMfn6+2bJli+ndu3erPab3q1/9yuTk5Jhjx46Zjz/+2MTGxhp/f39z5swZY8y3j+WEhYWZbdu2mT179piYmBgTExPTZsdTraqqyoSFhZl58+a5lbeHObpw4YLZt2+f2bdvn5Fkli5davbt2+e6437RokWme/fu5t133zWffvqpeeCBB2p97DAqKsrs2rXLfPTRR+bmm292e6StpKTEBAYGmocfftjs37/fZGZmms6dOzfbI231jamiosLcf//95qabbjL5+fluv1vVd3Lv3LnTvPTSSyY/P98cOXLE/PGPfzS9e/c2CQkJrTKm+sZz4cIF8+STT5rc3Fxz7Ngx8+GHH5pbb73V3Hzzzeabb75xbaM9zVG10tJS07lzZ/Paa6/VaN/W5uhqx2tjmub4Vv3Y4dy5c82BAwfMsmXLeOzwWrz88ssmLCzM+Pj4mFGjRplPPvmktbtUK0m1LqtXrzbGGFNYWGjuuusu07NnT2O1Ws3AgQPN3Llz3Z5xN8aY48ePmwkTJphOnToZf39/86tf/cpUVla2woi+fTwmODjY+Pj4mD59+phJkyaZw4cPu9ZfvnzZ/PKXvzQ9evQwnTt3Nv/wD/9gTp8+7baNtjSeau+//76RZAoKCtzK28Mcbd++vdb/Z4mJicaYbx89fPbZZ01gYKCxWq3mnnvuqTHOr7/+2kyePNl07drV2Gw2k5SUZC5cuOBW529/+5u58847jdVqNX369DGLFi1qlTEdO3aszt+t6u+OyMvLM3a73fj5+RlfX18zdOhQs3DhQrcP2JYcU33juXTpkrnvvvtM7969TceOHU3fvn3NjBkzavyR057mqNof/vAH06lTJ1NSUlKjfVubo6sdr41puuPb9u3bTWRkpPHx8TH9+/d328e14PXHAADgxrqHAAAA1I5AAAAACAQAAIBAAAAARCAAAAAiEAAAABEIAACACAQAGun48eOyWCyur2IG0L7xxUQAGqWqqkpnz56Vv7+/OnTgxalAe0cgAOCxiooK+fj4tHY3ADQhLhkA0NixYzV79mzNnj1bfn5+8vf317PPPqvqvxfCw8P1/PPPKyEhQTabTTNnzqz1ksHnn3+uH//4x7LZbOrWrZvGjBmjI0eOuNa/8cYbGjp0qHx9fTVkyBC9+uqrLT1UAHXgPB8ASdKbb76padOmaffu3dqzZ49mzpypsLAwzZgxQ5K0ePFipaamKi0trdb2p06d0l133aWxY8dq27Ztstls+vjjj3XlyhVJ0p/+9CelpqbqlVdeUVRUlPbt26cZM2aoS5cuSkxMbLFxAqgdlwwAaOzYsTpz5ow+//xzWSwWSdL8+fP15z//WV988YXCw8MVFRWljRs3utocP35c/fr10759+xQZGamnnnpKmZmZKigoUMeOHWvsY+DAgXr++ec1efJkV9kLL7ygrKws7dy5s/kHCaBeXDIAIEm6/fbbXWFAkmJiYnTo0CFVVVVJkkaOHFlv+/z8fI0ZM6bWMFBWVqYjR45o2rRp6tq1q2t54YUX3C4pAGg9XDIA0CBdunSpd32nTp3qXHfx4kVJ0ooVK2S3293WeXt7X3vnAFwzAgEASdKuXbvcfv7kk0908803N/gDe8SIEXrzzTdVWVlZ4yxBYGCgQkJCdPToUU2dOrXJ+gyg6XDJAIAkqbCwUMnJySooKNC6dev08ssva86cOQ1uP3v2bDkcDv3sZz/Tnj17dOjQIf3Hf/yHCgoKJEkLFixQenq6/v3f/11ffvmlPvvsM61evVpLly5triEB8ABnCABIkhISEnT58mWNGjVK3t7emjNnjmbOnNng9r169dK2bds0d+5c3X333fL29lZkZKRGjx4tSZo+fbo6d+6sF198UXPnzlWXLl00fPhwPf744800IgCe4CkDABo7dqwiIyOVkZHR2l0B0Eq4ZAAAAAgEAACASwYAAECcIQAAACIQAAAAEQgAAIAIBAAAQAQCAAAgAgEAABCBAAAAiEAAAABEIAAAAJL+H/hhdU6kzWufAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "매우 왜곡 -> 로그 값으로 변환환"
      ],
      "metadata": {
        "id": "8V9y4614PIIE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#변환하고 다시 살펴보기기\n",
        "import numpy as np\n",
        "\n",
        "y_train_df=np.log1p(y_train_df)\n",
        "sns.distplot(y_train_df, kde=False)"
      ],
      "metadata": {
        "id": "FpU6IaFtPINj",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 669
        },
        "outputId": "ad760134-983d-4983-c264-0b0954f0ba2f"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-10-2686a97a1d55>:5: UserWarning: \n",
            "\n",
            "`distplot` is a deprecated function and will be removed in seaborn v0.14.0.\n",
            "\n",
            "Please adapt your code to use either `displot` (a figure-level function with\n",
            "similar flexibility) or `histplot` (an axes-level function for histograms).\n",
            "\n",
            "For a guide to updating your code to use the new functions, please see\n",
            "https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751\n",
            "\n",
            "  sns.distplot(y_train_df, kde=False)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<Axes: xlabel='price'>"
            ]
          },
          "metadata": {},
          "execution_count": 10
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAGwCAYAAABFFQqPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA4vElEQVR4nO3de3RU9b3//1cu5CIyEy4mYb4mkFbKpSBRAiFyUQ5ZRI20UWwJpJJqhKUnoYQgN8WALTUaSgsIklK7DKslR+ScQ4qgwTQoaSUECOQAKUS0KCidBBdkBlJJQjK/P7qyfwxQIDphSPbzsdZeq7M/7733+zPazqv7Fh+Xy+USAACACfl6uwEAAABvIQgBAADTIggBAADTIggBAADTIggBAADTIggBAADTIggBAADT8vd2A7eylpYWnTp1St26dZOPj4+32wEAADfA5XLp3Llzstls8vW99jkfgtA1nDp1ShEREd5uAwAAfAMnT57UnXfeec0agtA1dOvWTdK/vkiLxeLlbgAAwI1wOp2KiIgwfsevhSB0Da2XwywWC0EIAIAO5kZua+FmaQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFr+3m4AwJUKyk9ct2ZqbORN6AQAOjfOCAEAANMiCAEAANNqcxAqLS3VxIkTZbPZ5OPjo8LCwitqjhw5oh/84AeyWq3q2rWrhg8frhMn/v9T/RcuXFB6erp69uyp22+/XZMmTVJNTY3bPk6cOKHExETddtttCg0N1dy5c3Xx4kW3mg8//FD33nuvAgMDdddddyk/P/+KXtasWaO+ffsqKChIsbGx2rNnT1unDAAAOqk2B6H6+noNHTpUa9asuer4p59+qtGjR2vAgAH68MMPdfDgQb344osKCgoyambPnq133nlHmzZt0s6dO3Xq1Ck99thjxnhzc7MSExPV2NioXbt2af369crPz1d2drZRc/z4cSUmJmrcuHGqrKxUZmamnn76aW3fvt2o2bhxo7KysrR48WLt379fQ4cOVUJCgmpra9s6bQAA0An5uFwu1zfe2MdHmzdvVlJSkrEuOTlZXbp00R/+8IerbuNwOHTHHXeooKBAjz/+uCTp6NGjGjhwoMrKyjRy5Ei99957euSRR3Tq1CmFhYVJkvLy8jR//nydPn1aAQEBmj9/vrZt26bDhw+7Hbuurk5FRUWSpNjYWA0fPlyrV6+WJLW0tCgiIkIzZ87UggULruitoaFBDQ0Nxmen06mIiAg5HA5ZLJZv+jUBbcbN0gDwzTmdTlmt1hv6/fboPUItLS3atm2bvve97ykhIUGhoaGKjY11u3xWUVGhpqYmxcfHG+sGDBigyMhIlZWVSZLKyso0ZMgQIwRJUkJCgpxOp6qqqoyaS/fRWtO6j8bGRlVUVLjV+Pr6Kj4+3qi5XE5OjqxWq7FERER8uy8EAADc0jwahGpra3X+/Hm98sorevDBB/X+++/r0Ucf1WOPPaadO3dKkux2uwICAhQSEuK2bVhYmOx2u1FzaQhqHW8du1aN0+nU119/ra+++krNzc1XrWndx+UWLlwoh8NhLCdPnvxmXwQAAOgQPPoeoZaWFknSD3/4Q82ePVuSFB0drV27dikvL0/333+/Jw/ncYGBgQoMDPR2GwAA4Cbx6BmhXr16yd/fX4MGDXJbP3DgQOOpsfDwcDU2Nqqurs6tpqamRuHh4UbN5U+RtX6+Xo3FYlFwcLB69eolPz+/q9a07gMAAJibR4NQQECAhg8frurqarf1H3/8sfr06SNJGjZsmLp06aKSkhJjvLq6WidOnFBcXJwkKS4uTocOHXJ7uqu4uFgWi8UIWXFxcW77aK1p3UdAQICGDRvmVtPS0qKSkhKjBgAAmFubL42dP39en3zyifH5+PHjqqysVI8ePRQZGam5c+dq8uTJGjt2rMaNG6eioiK98847+vDDDyVJVqtVaWlpysrKUo8ePWSxWDRz5kzFxcVp5MiRkqQJEyZo0KBBeuKJJ5Sbmyu73a5FixYpPT3duHT1zDPPaPXq1Zo3b56eeuop7dixQ2+//ba2bdtm9JaVlaXU1FTFxMRoxIgRWrFiherr6/Xkk09+m+8MAAB0Fq42+uCDD1ySrlhSU1ONmt///veuu+66yxUUFOQaOnSoq7Cw0G0fX3/9tes///M/Xd27d3fddtttrkcffdT1j3/8w63ms88+cz300EOu4OBgV69evVxz5sxxNTU1XdFLdHS0KyAgwPWd73zH9eabb17R72uvveaKjIx0BQQEuEaMGOHavXv3Dc/V4XC4JLkcDscNbwN4wobdn193AQBcXVt+v7/Ve4Q6u7a8hwDwJN4jBADfnNfeIwQAANCREIQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBptTkIlZaWauLEibLZbPLx8VFhYeG/rX3mmWfk4+OjFStWuK0/c+aMUlJSZLFYFBISorS0NJ0/f96t5uDBgxozZoyCgoIUERGh3NzcK/a/adMmDRgwQEFBQRoyZIjeffddt3GXy6Xs7Gz17t1bwcHBio+P17Fjx9o6ZQAA0Em1OQjV19dr6NChWrNmzTXrNm/erN27d8tms10xlpKSoqqqKhUXF2vr1q0qLS3VjBkzjHGn06kJEyaoT58+qqio0LJly7RkyRKtW7fOqNm1a5emTJmitLQ0HThwQElJSUpKStLhw4eNmtzcXK1atUp5eXkqLy9X165dlZCQoAsXLrR12gAAoBPycblcrm+8sY+PNm/erKSkJLf1X375pWJjY7V9+3YlJiYqMzNTmZmZkqQjR45o0KBB2rt3r2JiYiRJRUVFevjhh/XFF1/IZrNp7dq1euGFF2S32xUQECBJWrBggQoLC3X06FFJ0uTJk1VfX6+tW7caxx05cqSio6OVl5cnl8slm82mOXPm6LnnnpMkORwOhYWFKT8/X8nJydedn9PplNVqlcPhkMVi+aZfE9BmBeUnrlszNTbyJnQCAB1PW36/PX6PUEtLi5544gnNnTtX3//+968YLysrU0hIiBGCJCk+Pl6+vr4qLy83asaOHWuEIElKSEhQdXW1zp49a9TEx8e77TshIUFlZWWSpOPHj8tut7vVWK1WxcbGGjWXa2hokNPpdFsAAEDn5fEg9Oqrr8rf318/+9nPrjput9sVGhrqts7f3189evSQ3W43asLCwtxqWj9fr+bS8Uu3u1rN5XJycmS1Wo0lIiLiuvMFAAAdl0eDUEVFhVauXKn8/Hz5+Ph4ctc3xcKFC+VwOIzl5MmT3m4JAAC0I48Gob/85S+qra1VZGSk/P395e/vr88//1xz5sxR3759JUnh4eGqra112+7ixYs6c+aMwsPDjZqamhq3mtbP16u5dPzS7a5Wc7nAwEBZLBa3BQAAdF4eDUJPPPGEDh48qMrKSmOx2WyaO3eutm/fLkmKi4tTXV2dKioqjO127NihlpYWxcbGGjWlpaVqamoyaoqLi9W/f391797dqCkpKXE7fnFxseLi4iRJUVFRCg8Pd6txOp0qLy83agAAgLn5t3WD8+fP65NPPjE+Hz9+XJWVlerRo4ciIyPVs2dPt/ouXbooPDxc/fv3lyQNHDhQDz74oKZPn668vDw1NTUpIyNDycnJxqP2U6dO1UsvvaS0tDTNnz9fhw8f1sqVK/Wb3/zG2O+sWbN0//33a/ny5UpMTNRbb72lffv2GY/Y+/j4KDMzU0uXLlW/fv0UFRWlF198UTab7Yqn3AAAgDm1OQjt27dP48aNMz5nZWVJklJTU5Wfn39D+9iwYYMyMjI0fvx4+fr6atKkSVq1apUxbrVa9f777ys9PV3Dhg1Tr169lJ2d7fauofvuu08FBQVatGiRnn/+efXr10+FhYUaPHiwUTNv3jzV19drxowZqqur0+jRo1VUVKSgoKC2ThsAAHRC3+o9Qp0d7xGCt/AeIQD45rz6HiEAAICOgiAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMy9/bDQDoHArKT1y3Zmps5E3oBABuHGeEAACAaRGEAACAaRGEAACAaRGEAACAaRGEAACAafHUGGByPO0FwMw4IwQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyrzUGotLRUEydOlM1mk4+PjwoLC42xpqYmzZ8/X0OGDFHXrl1ls9k0bdo0nTp1ym0fZ86cUUpKiiwWi0JCQpSWlqbz58+71Rw8eFBjxoxRUFCQIiIilJube0UvmzZt0oABAxQUFKQhQ4bo3XffdRt3uVzKzs5W7969FRwcrPj4eB07dqytUwYAAJ1Um4NQfX29hg4dqjVr1lwx9s9//lP79+/Xiy++qP379+t///d/VV1drR/84AdudSkpKaqqqlJxcbG2bt2q0tJSzZgxwxh3Op2aMGGC+vTpo4qKCi1btkxLlizRunXrjJpdu3ZpypQpSktL04EDB5SUlKSkpCQdPnzYqMnNzdWqVauUl5en8vJyde3aVQkJCbpw4UJbpw0AADohH5fL5frGG/v4aPPmzUpKSvq3NXv37tWIESP0+eefKzIyUkeOHNGgQYO0d+9excTESJKKior08MMP64svvpDNZtPatWv1wgsvyG63KyAgQJK0YMECFRYW6ujRo5KkyZMnq76+Xlu3bjWONXLkSEVHRysvL08ul0s2m01z5szRc889J0lyOBwKCwtTfn6+kpOTrzs/p9Mpq9Uqh8Mhi8XyTb8moM1u5rt9PHUs3kcE4FbRlt/vdr9HyOFwyMfHRyEhIZKksrIyhYSEGCFIkuLj4+Xr66vy8nKjZuzYsUYIkqSEhARVV1fr7NmzRk18fLzbsRISElRWViZJOn78uOx2u1uN1WpVbGysUXO5hoYGOZ1OtwUAAHRe7RqELly4oPnz52vKlClGIrPb7QoNDXWr8/f3V48ePWS3242asLAwt5rWz9eruXT80u2uVnO5nJwcWa1WY4mIiGjznAEAQMfRbkGoqalJP/7xj+VyubR27dr2OoxHLVy4UA6Hw1hOnjzp7ZYAAEA7ape/NdYagj7//HPt2LHD7fpceHi4amtr3eovXryoM2fOKDw83Kipqalxq2n9fL2aS8db1/Xu3dutJjo6+qp9BwYGKjAwsK3TBbyCe3IA4Nvz+Bmh1hB07Ngx/fnPf1bPnj3dxuPi4lRXV6eKigpj3Y4dO9TS0qLY2FijprS0VE1NTUZNcXGx+vfvr+7duxs1JSUlbvsuLi5WXFycJCkqKkrh4eFuNU6nU+Xl5UYNAAAwtzYHofPnz6uyslKVlZWS/nVTcmVlpU6cOKGmpiY9/vjj2rdvnzZs2KDm5mbZ7XbZ7XY1NjZKkgYOHKgHH3xQ06dP1549e/TRRx8pIyNDycnJstlskqSpU6cqICBAaWlpqqqq0saNG7Vy5UplZWUZfcyaNUtFRUVavny5jh49qiVLlmjfvn3KyMiQ9K8n2jIzM7V06VJt2bJFhw4d0rRp02Sz2a75lBsAADCPNl8a27dvn8aNG2d8bg0nqampWrJkibZs2SJJV1x++uCDD/TAAw9IkjZs2KCMjAyNHz9evr6+mjRpklatWmXUWq1Wvf/++0pPT9ewYcPUq1cvZWdnu71r6L777lNBQYEWLVqk559/Xv369VNhYaEGDx5s1MybN0/19fWaMWOG6urqNHr0aBUVFSkoKKit0wYAAJ3Qt3qPUGfHe4TgLTdy/8+NuJnv/7mZPQPAtdxS7xECAAC4VRGEAACAaRGEAACAaRGEAACAaRGEAACAaRGEAACAaRGEAACAabXL3xoDcGvw1Lt9AKCz4owQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQAAAwrTYHodLSUk2cOFE2m00+Pj4qLCx0G3e5XMrOzlbv3r0VHBys+Ph4HTt2zK3mzJkzSklJkcViUUhIiNLS0nT+/Hm3moMHD2rMmDEKCgpSRESEcnNzr+hl06ZNGjBggIKCgjRkyBC9++67be4FAACYV5uDUH19vYYOHao1a9ZcdTw3N1erVq1SXl6eysvL1bVrVyUkJOjChQtGTUpKiqqqqlRcXKytW7eqtLRUM2bMMMadTqcmTJigPn36qKKiQsuWLdOSJUu0bt06o2bXrl2aMmWK0tLSdODAASUlJSkpKUmHDx9uUy8AAMC8fFwul+sbb+zjo82bNyspKUnSv87A2Gw2zZkzR88995wkyeFwKCwsTPn5+UpOTtaRI0c0aNAg7d27VzExMZKkoqIiPfzww/riiy9ks9m0du1avfDCC7Lb7QoICJAkLViwQIWFhTp69KgkafLkyaqvr9fWrVuNfkaOHKno6Gjl5eXdUC/X43Q6ZbVa5XA4ZLFYvunXBLRZQfkJb7fgZmps5HVrPNXzjRwLAK6lLb/f/p488PHjx2W32xUfH2+ss1qtio2NVVlZmZKTk1VWVqaQkBAjBElSfHy8fH19VV5erkcffVRlZWUaO3asEYIkKSEhQa+++qrOnj2r7t27q6ysTFlZWW7HT0hIMC7V3Ugvl2toaFBDQ4Px2el0fuvvBOgMbrVgBgCe4tGbpe12uyQpLCzMbX1YWJgxZrfbFRoa6jbu7++vHj16uNVcbR+XHuPf1Vw6fr1eLpeTkyOr1WosERERNzBrAADQUfHU2CUWLlwoh8NhLCdPnvR2SwAAoB15NAiFh4dLkmpqatzW19TUGGPh4eGqra11G7948aLOnDnjVnO1fVx6jH9Xc+n49Xq5XGBgoCwWi9sCAAA6L48GoaioKIWHh6ukpMRY53Q6VV5erri4OElSXFyc6urqVFFRYdTs2LFDLS0tio2NNWpKS0vV1NRk1BQXF6t///7q3r27UXPpcVprWo9zI70AAABza3MQOn/+vCorK1VZWSnpXzclV1ZW6sSJE/Lx8VFmZqaWLl2qLVu26NChQ5o2bZpsNpvxZNnAgQP14IMPavr06dqzZ48++ugjZWRkKDk5WTabTZI0depUBQQEKC0tTVVVVdq4caNWrlzpdnP0rFmzVFRUpOXLl+vo0aNasmSJ9u3bp4yMDEm6oV4AAIC5tfmpsX379mncuHHG59Zwkpqaqvz8fM2bN0/19fWaMWOG6urqNHr0aBUVFSkoKMjYZsOGDcrIyND48ePl6+urSZMmadWqVca41WrV+++/r/T0dA0bNky9evVSdna227uG7rvvPhUUFGjRokV6/vnn1a9fPxUWFmrw4MFGzY30AgAAzOtbvUeos+M9QvAWMz+uznuEAHxbbfn95qkxAABgWgQhAABgWgQhAABgWgQhAABgWgQhAABgWgQhAABgWgQhAABgWgQhAABgWgQhAABgWgQhAABgWgQhAABgWgQhAABgWgQhAABgWgQhAABgWgQhAABgWgQhAABgWv7ebgAALlVQfuK6NVNjI29CJwDMgDNCAADAtAhCAADAtAhCAADAtAhCAADAtAhCAADAtAhCAADAtAhCAADAtAhCAADAtAhCAADAtAhCAADAtAhCAADAtAhCAADAtAhCAADAtAhCAADAtAhCAADAtDwehJqbm/Xiiy8qKipKwcHB+u53v6tf/OIXcrlcRo3L5VJ2drZ69+6t4OBgxcfH69ixY277OXPmjFJSUmSxWBQSEqK0tDSdP3/erebgwYMaM2aMgoKCFBERodzc3Cv62bRpkwYMGKCgoCANGTJE7777rqenDAAAOiiPB6FXX31Va9eu1erVq3XkyBG9+uqrys3N1WuvvWbU5ObmatWqVcrLy1N5ebm6du2qhIQEXbhwwahJSUlRVVWViouLtXXrVpWWlmrGjBnGuNPp1IQJE9SnTx9VVFRo2bJlWrJkidatW2fU7Nq1S1OmTFFaWpoOHDigpKQkJSUl6fDhw56eNgAA6IB8XJeeqvGARx55RGFhYfr9739vrJs0aZKCg4P1xz/+US6XSzabTXPmzNFzzz0nSXI4HAoLC1N+fr6Sk5N15MgRDRo0SHv37lVMTIwkqaioSA8//LC++OIL2Ww2rV27Vi+88ILsdrsCAgIkSQsWLFBhYaGOHj0qSZo8ebLq6+u1detWo5eRI0cqOjpaeXl5152L0+mU1WqVw+GQxWLx2HcEXE9B+Qlvt3BLmxob6e0WANzC2vL77fEzQvfdd59KSkr08ccfS5L+7//+T3/961/10EMPSZKOHz8uu92u+Ph4Yxur1arY2FiVlZVJksrKyhQSEmKEIEmKj4+Xr6+vysvLjZqxY8caIUiSEhISVF1drbNnzxo1lx6ntab1OJdraGiQ0+l0WwAAQOfl7+kdLliwQE6nUwMGDJCfn5+am5v1y1/+UikpKZIku90uSQoLC3PbLiwszBiz2+0KDQ11b9TfXz169HCriYqKumIfrWPdu3eX3W6/5nEul5OTo5deeumbTBsAAHRAHj8j9Pbbb2vDhg0qKCjQ/v37tX79ev3qV7/S+vXrPX0oj1u4cKEcDoexnDx50tstAQCAduTxM0Jz587VggULlJycLEkaMmSIPv/8c+Xk5Cg1NVXh4eGSpJqaGvXu3dvYrqamRtHR0ZKk8PBw1dbWuu334sWLOnPmjLF9eHi4ampq3GpaP1+vpnX8coGBgQoMDPwm0wYAAB2Qx88I/fOf/5Svr/tu/fz81NLSIkmKiopSeHi4SkpKjHGn06ny8nLFxcVJkuLi4lRXV6eKigqjZseOHWppaVFsbKxRU1paqqamJqOmuLhY/fv3V/fu3Y2aS4/TWtN6HAAAYG4eD0ITJ07UL3/5S23btk2fffaZNm/erF//+td69NFHJUk+Pj7KzMzU0qVLtWXLFh06dEjTpk2TzWZTUlKSJGngwIF68MEHNX36dO3Zs0cfffSRMjIylJycLJvNJkmaOnWqAgIClJaWpqqqKm3cuFErV65UVlaW0cusWbNUVFSk5cuX6+jRo1qyZIn27dunjIwMT08bAAB0QB5/fP7cuXN68cUXtXnzZtXW1spms2nKlCnKzs42nvByuVxavHix1q1bp7q6Oo0ePVqvv/66vve97xn7OXPmjDIyMvTOO+/I19dXkyZN0qpVq3T77bcbNQcPHlR6err27t2rXr16aebMmZo/f75bP5s2bdKiRYv02WefqV+/fsrNzdXDDz98Q3Ph8Xl4C4/PXxuPzwO4lrb8fns8CHUmBCF4C0Ho2ghCAK7Fq+8RAgAA6CgIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLT8vd0AALRVQfmJ69ZMjY28CZ0A6Og4IwQAAEyrXYLQl19+qZ/85Cfq2bOngoODNWTIEO3bt88Yd7lcys7OVu/evRUcHKz4+HgdO3bMbR9nzpxRSkqKLBaLQkJClJaWpvPnz7vVHDx4UGPGjFFQUJAiIiKUm5t7RS+bNm3SgAEDFBQUpCFDhujdd99tjykDN6yg/MR1FwDAzeHxIHT27FmNGjVKXbp00Xvvvae//e1vWr58ubp3727U5ObmatWqVcrLy1N5ebm6du2qhIQEXbhwwahJSUlRVVWViouLtXXrVpWWlmrGjBnGuNPp1IQJE9SnTx9VVFRo2bJlWrJkidatW2fU7Nq1S1OmTFFaWpoOHDigpKQkJSUl6fDhw56eNgAA6IB8XC6Xy5M7XLBggT766CP95S9/ueq4y+WSzWbTnDlz9Nxzz0mSHA6HwsLClJ+fr+TkZB05ckSDBg3S3r17FRMTI0kqKirSww8/rC+++EI2m01r167VCy+8ILvdroCAAOPYhYWFOnr0qCRp8uTJqq+v19atW43jjxw5UtHR0crLy7uit4aGBjU0NBifnU6nIiIi5HA4ZLFYPPMFwfQ443NzcI8QYF5Op1NWq/WGfr89fkZoy5YtiomJ0Y9+9COFhobqnnvu0e9+9ztj/Pjx47Lb7YqPjzfWWa1WxcbGqqysTJJUVlamkJAQIwRJUnx8vHx9fVVeXm7UjB071ghBkpSQkKDq6mqdPXvWqLn0OK01rce5XE5OjqxWq7FERER8y28DAADcyjwehP7+979r7dq16tevn7Zv365nn31WP/vZz7R+/XpJkt1ulySFhYW5bRcWFmaM2e12hYaGuo37+/urR48ebjVX28elx/h3Na3jl1u4cKEcDoexnDx5ss3zBwAAHYfHH59vaWlRTEyMXn75ZUnSPffco8OHDysvL0+pqamePpxHBQYGKjAw0NttAACAm8TjQah3794aNGiQ27qBAwfqf/7nfyRJ4eHhkqSamhr17t3bqKmpqVF0dLRRU1tb67aPixcv6syZM8b24eHhqqmpcatp/Xy9mtZxAObG+4gAePzS2KhRo1RdXe227uOPP1afPn0kSVFRUQoPD1dJSYkx7nQ6VV5erri4OElSXFyc6urqVFFRYdTs2LFDLS0tio2NNWpKS0vV1NRk1BQXF6t///7GE2pxcXFux2mtaT0OAAAwN48HodmzZ2v37t16+eWX9cknn6igoEDr1q1Tenq6JMnHx0eZmZlaunSptmzZokOHDmnatGmy2WxKSkqS9K8zSA8++KCmT5+uPXv26KOPPlJGRoaSk5Nls9kkSVOnTlVAQIDS0tJUVVWljRs3auXKlcrKyjJ6mTVrloqKirR8+XIdPXpUS5Ys0b59+5SRkeHpaQMAgA7I45fGhg8frs2bN2vhwoX6+c9/rqioKK1YsUIpKSlGzbx581RfX68ZM2aorq5Oo0ePVlFRkYKCgoyaDRs2KCMjQ+PHj5evr68mTZqkVatWGeNWq1Xvv/++0tPTNWzYMPXq1UvZ2dlu7xq67777VFBQoEWLFun5559Xv379VFhYqMGDB3t62gAAoAPy+HuEOpO2vIcAuFG8R+jmuJF7e7hHCOicvPoeIQAAgI6CIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEzL439iAzAz3hoNAB0LZ4QAAIBpEYQAAIBpcWkMQKfEZUoAN4IzQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLQIQgAAwLTaPQi98sor8vHxUWZmprHuwoULSk9PV8+ePXX77bdr0qRJqqmpcdvuxIkTSkxM1G233abQ0FDNnTtXFy9edKv58MMPde+99yowMFB33XWX8vPzrzj+mjVr1LdvXwUFBSk2NlZ79uxpj2kCAIAOqF2D0N69e/Xb3/5Wd999t9v62bNn65133tGmTZu0c+dOnTp1So899pgx3tzcrMTERDU2NmrXrl1av3698vPzlZ2dbdQcP35ciYmJGjdunCorK5WZmamnn35a27dvN2o2btyorKwsLV68WPv379fQoUOVkJCg2tra9pw2AADoIHxcLperPXZ8/vx53XvvvXr99de1dOlSRUdHa8WKFXI4HLrjjjtUUFCgxx9/XJJ09OhRDRw4UGVlZRo5cqTee+89PfLIIzp16pTCwsIkSXl5eZo/f75Onz6tgIAAzZ8/X9u2bdPhw4eNYyYnJ6uurk5FRUWSpNjYWA0fPlyrV6+WJLW0tCgiIkIzZ87UggULrui5oaFBDQ0Nxmen06mIiAg5HA5ZLJb2+JrQyRSUn/B2C/CwqbGR3m4BQBs5nU5ZrdYb+v1utzNC6enpSkxMVHx8vNv6iooKNTU1ua0fMGCAIiMjVVZWJkkqKyvTkCFDjBAkSQkJCXI6naqqqjJqLt93QkKCsY/GxkZVVFS41fj6+io+Pt6ouVxOTo6sVquxREREfItvAAAA3OraJQi99dZb2r9/v3Jycq4Ys9vtCggIUEhIiNv6sLAw2e12o+bSENQ63jp2rRqn06mvv/5aX331lZqbm69a07qPyy1cuFAOh8NYTp48eeOTBgAAHY6/p3d48uRJzZo1S8XFxQoKCvL07ttVYGCgAgMDvd0GAAC4STx+RqiiokK1tbW699575e/vL39/f+3cuVOrVq2Sv7+/wsLC1NjYqLq6OrftampqFB4eLkkKDw+/4imy1s/Xq7FYLAoODlavXr3k5+d31ZrWfQAAAHPzeBAaP368Dh06pMrKSmOJiYlRSkqK8Z+7dOmikpISY5vq6mqdOHFCcXFxkqS4uDgdOnTI7emu4uJiWSwWDRo0yKi5dB+tNa37CAgI0LBhw9xqWlpaVFJSYtQAAABz8/ilsW7dumnw4MFu67p27aqePXsa69PS0pSVlaUePXrIYrFo5syZiouL08iRIyVJEyZM0KBBg/TEE08oNzdXdrtdixYtUnp6unHp6plnntHq1as1b948PfXUU9qxY4fefvttbdu2zThuVlaWUlNTFRMToxEjRmjFihWqr6/Xk08+6elpAwCADsjjQehG/OY3v5Gvr68mTZqkhoYGJSQk6PXXXzfG/fz8tHXrVj377LOKi4tT165dlZqaqp///OdGTVRUlLZt26bZs2dr5cqVuvPOO/XGG28oISHBqJk8ebJOnz6t7Oxs2e12RUdHq6io6IobqAEAgDm123uEOoO2vIcAkHiPUGfEe4SAjueWeI8QAADArY4gBAAATIsgBAAATIsgBAAATIsgBAAATIsgBAAATIsgBAAATIsgBAAATIsgBAAATMsrf2ID6Ih4azQAdD6cEQIAAKbFGSFAnO0BALPijBAAADAtghAAADAtghAAADAtghAAADAtbpYGgGvw5I30U2MjPbYvAJ7BGSEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBa/NFVALhJbuQPuPKHWYGbizNCAADAtAhCAADAtDwehHJycjR8+HB169ZNoaGhSkpKUnV1tVvNhQsXlJ6erp49e+r222/XpEmTVFNT41Zz4sQJJSYm6rbbblNoaKjmzp2rixcvutV8+OGHuvfeexUYGKi77rpL+fn5V/SzZs0a9e3bV0FBQYqNjdWePXs8PWUAANBBeTwI7dy5U+np6dq9e7eKi4vV1NSkCRMmqL6+3qiZPXu23nnnHW3atEk7d+7UqVOn9Nhjjxnjzc3NSkxMVGNjo3bt2qX169crPz9f2dnZRs3x48eVmJiocePGqbKyUpmZmXr66ae1fft2o2bjxo3KysrS4sWLtX//fg0dOlQJCQmqra319LQBAEAH5ONyuVzteYDTp08rNDRUO3fu1NixY+VwOHTHHXeooKBAjz/+uCTp6NGjGjhwoMrKyjRy5Ei99957euSRR3Tq1CmFhYVJkvLy8jR//nydPn1aAQEBmj9/vrZt26bDhw8bx0pOTlZdXZ2KiookSbGxsRo+fLhWr14tSWppaVFERIRmzpypBQsWXLd3p9Mpq9Uqh8Mhi8Xi6a8Gt5AbuYkVuBm4WRr49try+93u9wg5HA5JUo8ePSRJFRUVampqUnx8vFEzYMAARUZGqqysTJJUVlamIUOGGCFIkhISEuR0OlVVVWXUXLqP1prWfTQ2NqqiosKtxtfXV/Hx8UbN5RoaGuR0Ot0WAADQebVrEGppaVFmZqZGjRqlwYMHS5LsdrsCAgIUEhLiVhsWFia73W7UXBqCWsdbx65V43Q69fXXX+urr75Sc3PzVWta93G5nJwcWa1WY4mIiPhmEwcAAB1Cuwah9PR0HT58WG+99VZ7HsZjFi5cKIfDYSwnT570dksAAKAdtdsLFTMyMrR161aVlpbqzjvvNNaHh4ersbFRdXV1bmeFampqFB4ebtRc/nRX61Nll9Zc/qRZTU2NLBaLgoOD5efnJz8/v6vWtO7jcoGBgQoMDPxmE4ZX8II6AMC34fEzQi6XSxkZGdq8ebN27NihqKgot/Fhw4apS5cuKikpMdZVV1frxIkTiouLkyTFxcXp0KFDbk93FRcXy2KxaNCgQUbNpftorWndR0BAgIYNG+ZW09LSopKSEqMGAACYm8fPCKWnp6ugoEB/+tOf1K1bN+N+HKvVquDgYFmtVqWlpSkrK0s9evSQxWLRzJkzFRcXp5EjR0qSJkyYoEGDBumJJ55Qbm6u7Ha7Fi1apPT0dOOMzTPPPKPVq1dr3rx5euqpp7Rjxw69/fbb2rZtm9FLVlaWUlNTFRMToxEjRmjFihWqr6/Xk08+6elpAwCADsjjQWjt2rWSpAceeMBt/Ztvvqmf/vSnkqTf/OY38vX11aRJk9TQ0KCEhAS9/vrrRq2fn5+2bt2qZ599VnFxceratatSU1P185//3KiJiorStm3bNHv2bK1cuVJ33nmn3njjDSUkJBg1kydP1unTp5WdnS273a7o6GgVFRVdcQM1AAAwp3Z/j1BHxnuEbn2eukeI9wjhVsE9bcC3d0u9RwgAAOBW1W5PjQG3Cs72AAD+Hc4IAQAA0yIIAQAA0yIIAQAA0yIIAQAA0+JmaQC4hfBnY4CbiyAEAB0MYQnwHC6NAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0+JPbABAJ8Sf4QBuDGeEAACAaXFGCABMirNGAGeEAACAiRGEAACAaRGEAACAaXGPEADg3+I+InR2nBECAACmRRACAACmxaUxAMC3wuUzdGScEQIAAKbFGSEAQLvjrBFuVZwRAgAApsUZIQDALYGzRvAGUwShNWvWaNmyZbLb7Ro6dKhee+01jRgxwtttAQDaiLAET+v0QWjjxo3KyspSXl6eYmNjtWLFCiUkJKi6ulqhoaHebg8A4GGEJbSFj8vlcnm7ifYUGxur4cOHa/Xq1ZKklpYWRUREaObMmVqwYME1t3U6nbJarXI4HLJYLB7vjf+yfns38h0CwDfB//52XG35/e7UZ4QaGxtVUVGhhQsXGut8fX0VHx+vsrKyK+obGhrU0NBgfHY4HJL+9YW2h3/Wn7tuTXsdu7O4ke8QAL6JN3ZUebuFNvtxTIS3W7gltP523si5nk4dhL766is1NzcrLCzMbX1YWJiOHj16RX1OTo5eeumlK9ZHRHjvX6zpXjsyAKCj4TfD3blz52S1Wq9Z06mDUFstXLhQWVlZxueWlhadOXNGPXv2lI+Pj0eP5XQ6FRERoZMnT7bLZbdbiZnmKplrvsy1czLTXCVzzdcsc3W5XDp37pxsNtt1azt1EOrVq5f8/PxUU1Pjtr6mpkbh4eFX1AcGBiowMNBtXUhISHu2KIvF0qn/ZbyUmeYqmWu+zLVzMtNcJXPN1wxzvd6ZoFad+oWKAQEBGjZsmEpKSox1LS0tKikpUVxcnBc7AwAAt4JOfUZIkrKyspSamqqYmBiNGDFCK1asUH19vZ588klvtwYAALys0wehyZMn6/Tp08rOzpbdbld0dLSKioquuIH6ZgsMDNTixYuvuBTXGZlprpK55stcOyczzVUy13zNNNcb1enfIwQAAPDvdOp7hAAAAK6FIAQAAEyLIAQAAEyLIAQAAEyLIOQla9asUd++fRUUFKTY2Fjt2bPH2y15XGlpqSZOnCibzSYfHx8VFhZ6u6V2k5OTo+HDh6tbt24KDQ1VUlKSqqurvd1Wu1m7dq3uvvtu46VscXFxeu+997zdVrt75ZVX5OPjo8zMTG+30i6WLFkiHx8ft2XAgAHebqvdfPnll/rJT36inj17Kjg4WEOGDNG+ffu83Va76Nu37xX/bH18fJSenu7t1ryOIOQFGzduVFZWlhYvXqz9+/dr6NChSkhIUG1trbdb86j6+noNHTpUa9as8XYr7W7nzp1KT0/X7t27VVxcrKamJk2YMEH19fXebq1d3HnnnXrllVdUUVGhffv26T/+4z/0wx/+UFVVHe+PVN6ovXv36re//a3uvvtub7fSrr7//e/rH//4h7H89a9/9XZL7eLs2bMaNWqUunTpovfee09/+9vftHz5cnXv3t3brbWLvXv3uv1zLS4uliT96Ec/8nJntwAXbroRI0a40tPTjc/Nzc0um83mysnJ8WJX7UuSa/Pmzd5u46apra11SXLt3LnT263cNN27d3e98cYb3m6jXZw7d87Vr18/V3Fxsev+++93zZo1y9sttYvFixe7hg4d6u02bor58+e7Ro8e7e02vGbWrFmu7373u66WlhZvt+J1nBG6yRobG1VRUaH4+Hhjna+vr+Lj41VWVubFzuBJDodDktSjRw8vd9L+mpub9dZbb6m+vr7T/uma9PR0JSYmuv33trM6duyYbDabvvOd7yglJUUnTpzwdkvtYsuWLYqJidGPfvQjhYaG6p577tHvfvc7b7d1UzQ2NuqPf/yjnnrqKY//QfGOiCB0k3311Vdqbm6+4s3WYWFhstvtXuoKntTS0qLMzEyNGjVKgwcP9nY77ebQoUO6/fbbFRgYqGeeeUabN2/WoEGDvN2Wx7311lvav3+/cnJyvN1Ku4uNjVV+fr6Kioq0du1aHT9+XGPGjNG5c+e83ZrH/f3vf9fatWvVr18/bd++Xc8++6x+9rOfaf369d5urd0VFhaqrq5OP/3pT73dyi2h0/+JDeBmS09P1+HDhzvtvRWt+vfvr8rKSjkcDv33f/+3UlNTtXPnzk4Vhk6ePKlZs2apuLhYQUFB3m6n3T300EPGf7777rsVGxurPn366O2331ZaWpoXO/O8lpYWxcTE6OWXX5Yk3XPPPTp8+LDy8vKUmprq5e7a1+9//3s99NBDstls3m7llsAZoZusV69e8vPzU01Njdv6mpoahYeHe6kreEpGRoa2bt2qDz74QHfeeae322lXAQEBuuuuuzRs2DDl5ORo6NChWrlypbfb8qiKigrV1tbq3nvvlb+/v/z9/bVz506tWrVK/v7+am5u9naL7SokJETf+9739Mknn3i7FY/r3bv3FaF94MCBnfZSYKvPP/9cf/7zn/X00097u5VbBkHoJgsICNCwYcNUUlJirGtpaVFJSUmnvb/CDFwulzIyMrR582bt2LFDUVFR3m7ppmtpaVFDQ4O32/Co8ePH69ChQ6qsrDSWmJgYpaSkqLKyUn5+ft5usV2dP39en376qXr37u3tVjxu1KhRV7zi4uOPP1afPn281NHN8eabbyo0NFSJiYnebuWWwaUxL8jKylJqaqpiYmI0YsQIrVixQvX19XryySe93ZpHnT9/3u3/SR4/flyVlZXq0aOHIiMjvdiZ56Wnp6ugoEB/+tOf1K1bN+N+L6vVquDgYC9353kLFy7UQw89pMjISJ07d04FBQX68MMPtX37dm+35lHdunW74j6vrl27qmfPnp3y/q/nnntOEydOVJ8+fXTq1CktXrxYfn5+mjJlirdb87jZs2frvvvu08svv6wf//jH2rNnj9atW6d169Z5u7V209LSojfffFOpqany9+fn3+Dtx9bM6rXXXnNFRka6AgICXCNGjHDt3r3b2y153AcffOCSdMWSmprq7dY87mrzlOR68803vd1au3jqqadcffr0cQUEBLjuuOMO1/jx413vv/++t9u6KTrz4/OTJ0929e7d2xUQEOD6f//v/7kmT57s+uSTT7zdVrt55513XIMHD3YFBga6BgwY4Fq3bp23W2pX27dvd0lyVVdXe7uVW4qPy+VyeSeCAQAAeBf3CAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEwlc8++0w+Pj6qrKz0disAbgG8WRqAqTQ3N+v06dPq1asXf28JAEEIgHk0NjYqICDA220AuIVwaQxAh/XAAw8oIyNDGRkZslqt6tWrl1588UW1/v+7vn376he/+IWmTZsmi8WiGTNmXPXSWFVVlR555BFZLBZ169ZNY8aM0aeffmqMv/HGGxo4cKCCgoI0YMAAvf766zd7qgDaCeeFAXRo69evV1pamvbs2aN9+/ZpxowZioyM1PTp0yVJv/rVr5Sdna3Fixdfdfsvv/xSY8eO1QMPPKAdO3bIYrHoo48+0sWLFyVJGzZsUHZ2tlavXq177rlHBw4c0PTp09W1a1elpqbetHkCaB9cGgPQYT3wwAOqra1VVVWVfHx8JEkLFizQli1b9Le//U19+/bVPffco82bNxvbfPbZZ4qKitKBAwcUHR2t559/Xm+99Zaqq6vVpUuXK45x11136Re/+IWmTJlirFu6dKneffdd7dq1q/0nCaBdcWkMQIc2cuRIIwRJUlxcnI4dO6bm5mZJUkxMzDW3r6ys1JgxY64agurr6/Xpp58qLS1Nt99+u7EsXbrU7dIZgI6LS2MAOrWuXbteczw4OPjfjp0/f16S9Lvf/U6xsbFuY35+ft++OQBeRxAC0KGVl5e7fd69e7f69et3w0Hl7rvv1vr169XU1HTFWaGwsDDZbDb9/e9/V0pKisd6BnDr4NIYgA7txIkTysrKUnV1tf7rv/5Lr732mmbNmnXD22dkZMjpdCo5OVn79u3TsWPH9Ic//EHV1dWSpJdeekk5OTlatWqVPv74Yx06dEhvvvmmfv3rX7fXlADcRJwRAtChTZs2TV9//bVGjBghPz8/zZo1SzNmzLjh7Xv27KkdO3Zo7ty5uv/+++Xn56fo6GiNGjVKkvT000/rtttu07JlyzR37lx17dpVQ4YMUWZmZjvNCMDNxFNjADqsBx54QNHR0VqxYoW3WwHQQXFpDAAAmBZBCAAAmBaXxgAAgGlxRggAAJgWQQgAAJgWQQgAAJgWQQgAAJgWQQgAAJgWQQgAAJgWQQgAAJgWQQgAAJjW/wdT5t4hoW+aewAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "정규분포를 따른다. price칼럼을 원래 값에서 로그로 변환된 값으로 변경한다."
      ],
      "metadata": {
        "id": "slMp_XZ1Ud-x"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#다른 피처 값도 살펴보기\n",
        "mercari_df['price']=np.log1p(mercari_df['price'])\n",
        "mercari_df['price'].head(3)"
      ],
      "metadata": {
        "id": "23UMrK4RPJUY",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "30820890-5406-4609-da37-16c950ebfa1e"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    2.397895\n",
              "1    3.970292\n",
              "2    2.397895\n",
              "Name: price, dtype: float64"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "* shipping칼럼은 배송비 유무이며, 값이 비교적 균일하다. \n",
        "\n",
        "* Item_condition_id는 판매자가 제공하는 제품 상태로서 각 값이 의미하는 바는 캐글에 기재돼 있지 않아서 알 수 없지만, 1,2,3값이 주를 이루고 있다.\n",
        "* item_description칼럼은 null값은 별로 없지만 설명이 없는 경우 no라고 되어있다."
      ],
      "metadata": {
        "id": "vhwdWLkEUkWR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#no description 갯수 보기\n",
        "boolean_cond=mercari_df['item_description']=='No description yet'\n",
        "mercari_df[boolean_cond]['item_description'].count()\n",
        "     "
      ],
      "metadata": {
        "id": "7CdxY8RaUkdc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "bb82c80d-9aae-4ba3-8d89-33675ef4a9ac"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "82489"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "null과 마찬가지 이므로 속상값으로 사용될 수 없다. 적절한 값으로 변경해야한다.\n",
        "\n",
        "\n",
        "- category_name\n",
        "\n",
        "'/'로 분리된 카테고리를 하나의 문자열로 나타내고 있다. 가령 대분류, 중분류, 소분류로 나눌 수 있다. 이것은 텍스트이므로 피처 추출 시 tokenizer를'/'로 하여 단어를 분리해 벡터화할 수도 있지만 이것을 기준으로 단어를 토큰화해 각각 별도의 피처로 저장하고 이를 이용해 알고리즘을 학습시킨다.\n",
        "\n",
        "split_cat()을 이용하여 대, 중, 소분류를 분리한다."
      ],
      "metadata": {
        "id": "QwA2X_M-U8N5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#apply lambda 에서 호출되는 대, 중, 소 분할 함수 생성, 대, 중, 소 값을 리스트로 반환\n",
        "def split_cat(category_name):\n",
        "  try:\n",
        "    return category_name.split('/')\n",
        "  except:\n",
        "    return ['Other_Null', 'Other_Null', 'Other_Null'] #대, 중, 소에 모두 other_null 넣기기\n",
        "\n",
        "#위의 split_cat()을 apply lambda에서 호출해 대, 중, 소 칼럼을 mercari_df에 생성.\n",
        "mercari_df['cat_dae'], mercari_df['cat_jung'], mercari_df['cat_so']=zip(*mercari_df['category_name'].apply(lambda x:split_cat(x)))"
      ],
      "metadata": {
        "id": "hw9KIe8kVUh2"
      },
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#대분류만 값의 유형과 건수를 살펴보고, 중분류, 소분류는 값의 유형이 많으므로 분류 개수만 추출\n",
        "print('대분류 유형 :\\n', mercari_df['cat_dae'].value_counts())\n",
        "print('중분류 개수 :', mercari_df['cat_jung'].unique())\n",
        "print('소분류 개수 :', mercari_df['cat_so'].unique())"
      ],
      "metadata": {
        "id": "t3jEyg43VjWd",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0783f74b-a238-42a4-c195-d84cd8ce4ca6"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "대분류 유형 :\n",
            " Women                     664385\n",
            "Beauty                    207828\n",
            "Kids                      171689\n",
            "Electronics               122690\n",
            "Men                        93680\n",
            "Home                       67871\n",
            "Vintage & Collectibles     46530\n",
            "Other                      45351\n",
            "Handmade                   30842\n",
            "Sports & Outdoors          25342\n",
            "Other_Null                  6327\n",
            "Name: cat_dae, dtype: int64\n",
            "중분류 개수 : ['Tops' 'Computers & Tablets' 'Tops & Blouses' 'Home Décor' 'Jewelry'\n",
            " 'Other' 'Swimwear' 'Apparel' 'Collectibles' 'Makeup' 'Fragrance'\n",
            " 'Dresses' 'Office supplies' 'Shoes' 'Gear' 'Athletic Apparel'\n",
            " 'Cell Phones & Accessories' 'Jeans' 'Underwear' 'Skin Care' 'Toys'\n",
            " \"Women's Handbags\" 'Video Games & Consoles' 'Coats & Jackets' 'Pants'\n",
            " 'Girls (4+)' 'Antique' 'Kitchen & Dining' 'Sweaters' 'Boys 0-24 Mos'\n",
            " 'Girls 0-24 Mos' 'Maternity' 'Bedding' 'Exercise' 'Trading Cards'\n",
            " 'Boys (4+)' 'Storage & Organization' 'Fan Shop' 'Girls 2T-5T'\n",
            " \"Men's Accessories\" 'Boys 2T-5T' \"Women's Accessories\"\n",
            " 'Daily & Travel items' 'Other_Null' 'Skirts' 'Hair Care' 'Pet Supplies'\n",
            " 'Book' 'Tools & Accessories' 'Team Sports' 'Home Appliances'\n",
            " 'Accessories' 'Bags and Purses' 'Sweats & Hoodies' 'Shorts'\n",
            " 'TV, Audio & Surveillance' 'Outdoors' 'Bath & Body'\n",
            " 'Car Seats & Accessories' 'Toy' 'Art' 'Seasonal Décor' 'Footwear'\n",
            " 'Housewares' 'Media' 'Cameras & Photography' 'Glass' 'Paper Goods'\n",
            " 'Cleaning Supplies' 'Books' \"Kids' Home Store\" 'Electronics' 'Diapering'\n",
            " 'Bath' 'Children' 'Magazines' 'Artwork' 'Automotive' 'Serving' 'Clothing'\n",
            " 'Supplies' 'Feeding' 'Suits & Blazers' 'Golf' 'Music' 'Pets'\n",
            " 'Health & Baby Care' 'Nursery' 'Furniture' 'Geekery'\n",
            " 'Musical instruments' 'Books and Zines' 'Safety' 'Crochet' 'Home Decor'\n",
            " 'Holidays' 'Strollers' 'Pregnancy & Maternity' 'Needlecraft'\n",
            " 'Paper Ephemera' 'Car Audio, Video & GPS' 'Bathing & Skin Care'\n",
            " 'Dolls and Miniatures' 'Suits' 'Weddings' 'Others' 'Knitting' 'Patterns'\n",
            " 'Blazers & Sport Coats' 'Woodworking' 'Potty Training' 'Candles'\n",
            " 'Ceramics and Pottery' 'Quilts']\n",
            "소분류 개수 : ['T-shirts' 'Components & Parts' 'Blouse' 'Home Décor Accents' 'Necklaces'\n",
            " 'Other' 'Two-Piece' 'Girls' 'Doll' 'Face' 'Women' 'Above Knee, Mini'\n",
            " 'School Supplies' 'Boots' 'Makeup Sets' 'Eyes' 'Backpacks & Carriers'\n",
            " 'Makeup Palettes' 'Tank, Cami' 'Sports Bras' 'Cell Phones & Smartphones'\n",
            " 'Chargers & Cradles' 'T-Shirts' 'Athletic' 'Cases, Covers & Skins'\n",
            " 'Pants, Tights, Leggings' 'One-Piece' 'Boot Cut' 'Bras'\n",
            " 'Stuffed Animals & Plush' 'Totes & Shoppers' 'Shirts & Tops' 'Consoles'\n",
            " 'Glass' 'Vest' 'Arts & Crafts' 'Capris, Cropped' 'Messenger & Crossbody'\n",
            " 'Shoes' 'Collectibles' 'Coffee & Tea Accessories' 'Brooch' 'Headsets'\n",
            " 'Rings' 'Shorts' 'Fleece Jacket' 'Dolls & Accessories' 'Crewneck'\n",
            " 'Jackets' 'Home Fragrance' 'Accessories' 'Tops & Blouses'\n",
            " 'Sheets & Pillowcases' 'Fitness technology' 'Dress Up & Pretend Play'\n",
            " 'Animation' 'Jewelry Boxes & Organizers' 'Games' 'NCAA'\n",
            " 'Electronics for Kids' 'Watches' 'Knee-Length' 'Wallets'\n",
            " 'Tracksuits & Sweats' 'Loafers & Slip-Ons' 'Pumps' 'Full Zip'\n",
            " 'Medical Supplies & Equipment' 'Other_Null' 'Slim, Skinny' 'Belts'\n",
            " 'Tapestries' 'Figurine' 'Lips' 'Mini' 'Fashion Sneakers' 'Dresses'\n",
            " 'Shampoo & Conditioner Sets' 'Video Gaming Merchandise' 'Top & T-shirts'\n",
            " 'Dogs' 'Shoulder Bag' 'Hobo' 'Nails' 'Comics' 'Household Supplies'\n",
            " 'Halter' 'Learning & Education' 'Makeup Brushes & Tools' 'Earrings'\n",
            " 'Baby Gyms & Playmats' 'Cardigan' 'Shipping Supplies' 'Collared'\n",
            " 'Panties' 'Jewelry' 'Sandals' 'Baseball & Softball' 'Casual Pants'\n",
            " 'Candles & Home Scents' 'Kitchen Appliances' 'Hooded' 'Cables & Adapters'\n",
            " 'Coats & Jackets' 'Keychain' 'Asymmetrical Hem' 'Wristlet' 'Clutch'\n",
            " 'Full-Length' 'Sweatshirt, Pullover' 'Casual Shorts' 'Gadgets' 'Fishing'\n",
            " 'Leggings' 'Jeans' 'iPad' 'Sports' 'Bath' 'Bracelets' 'Body' 'Socks'\n",
            " 'Sports Nutrition' 'Action Figure' 'Hats' 'Portable Audio & Accessories'\n",
            " 'Photo Albums & Frames' 'Party Supplies' 'Action Figures & Statues'\n",
            " 'Fitness accessories' 'Sunglasses' 'Photography' 'Eyewear' 'Jean Jacket'\n",
            " 'Halloween' 'Dining & Entertaining' 'Kitchen Utensils & Gadgets'\n",
            " 'Button-Front' 'Tops & T-Shirts' 'Backpack Style' 'Cleats' 'Health Care'\n",
            " 'Tunic' 'Ceramic' 'DVD' 'Cell Phone Accessories' 'Ink & Toner' 'Necklace'\n",
            " 'Personal Care' 'Digital Cameras' 'Bottles' 'Sticker' 'Jerseys' 'CD'\n",
            " 'Camera & Photo Accessories' 'Tote' 'Flats' 'Polo, Rugby'\n",
            " 'Styling Products' 'Household Cleaners' 'Blankets & Throws' 'Purse'\n",
            " 'Literature & Fiction' 'Bottoms' 'Peacoat' 'One-Pieces' 'V-Neck'\n",
            " \"Children's Books\" 'Small Appliances' 'Football' 'Hair Accessories'\n",
            " 'Education & Teaching' 'Slippers' 'Swimwear' \"Kids' Room Décor\"\n",
            " 'Duvet Covers & Sets' 'Track & Sweat Pants' 'Cosmetic Bags' 'Hair Color'\n",
            " 'All Other Sports' 'Maxi' 'Lamps& Accessories' 'Radio' 'Conditioners'\n",
            " 'NFL' 'Disposable Diapers' 'Beach Accessories' 'Straight, Pencil'\n",
            " 'Bathroom Accessories' 'Baby & Toddler Toys' 'Asymmetrical' 'Baby'\n",
            " 'Towels' 'Lifestyle & Cultures' 'Military' 'Turtleneck, Mock' 'Paintings'\n",
            " 'Building Toys' 'Button Down Shirt' 'Scarves & Wraps' 'Feet'\n",
            " 'Khakis, Chinos' 'Straight Leg' 'Book' 'MLB' 'Cowl Neck' 'Valentine'\n",
            " 'Hair Styling Tools' 'Poncho' 'Video Game' 'Desktops & All-In-Ones'\n",
            " 'Thanksgiving' 'Mid-Calf' 'Cloth Diapers' 'Mules & Clogs' 'Hobbies'\n",
            " 'Car Electronics & Accessories' \"Kids' Bedding\" 'Tank' 'Electronics'\n",
            " 'Backpacks, Bags & Briefcases' 'Dress Pants' 'Cover-Ups' 'Painting'\n",
            " 'Screen Protectors' 'Boys' 'Cleansers' 'Comforters & Sets' 'Hair'\n",
            " 'Blu-Ray' 'Drives, Storage & Media' 'Kids' 'Bakeware' 'Mug'\n",
            " 'G-Strings & Thongs' 'Cape' 'Sweater' 'Headphones' 'Polo Shirt' 'Charm'\n",
            " 'Clothing & Closet Storage' 'Wall Decor' 'Scrapbooking' 'NBA' 'Handbag'\n",
            " 'Pacifiers & Accessories' 'Capri, Cropped' 'Ties' '100 Years or Older'\n",
            " 'Pouch' 'Playards' 'Costume' 'Vacuums & Floor Care' 'Makeup Remover'\n",
            " 'Blazer' 'Brushes & Applicators' \"Men's Golf Clubs\" 'Film Photography'\n",
            " 'Medical Books' 'Bottle-Feeding' 'Toy' 'Laptops & Netbooks' 'Vinyl'\n",
            " 'Collar' 'Garment Steamers' 'Clocks' 'Tools & Equipment' 'Cookbook'\n",
            " 'Pleated' 'Cards' 'Windbreaker' 'Bedding' 'Pants'\n",
            " 'Diaper Pails & Refills' 'Cup' 'Arts & Photography'\n",
            " 'Storage & Containers' 'VHS' 'Souvenir' 'Sets & Kits'\n",
            " 'Exterior Accessories' 'Home Office Furniture' 'Scrubs & Body Treatments'\n",
            " 'Classic, Straight Leg' 'Wine Accessories' 'Wrap' 'Food' 'Boxing & MMA'\n",
            " 'Luggage' 'Ring' 'Car Seats' 'Diaper Bags' 'Motorcycle' 'Shampoos'\n",
            " 'Hoodie' 'Knit Top' 'Henley' 'Drawings' 'Cookware' 'Electronic' 'Formal'\n",
            " 'Yoga & Pilates' 'Networking & Connectivity' 'Humidifiers' 'Sets'\n",
            " 'Dress Shirts' 'Writing' 'Volleyball' 'Styling Tools'\n",
            " \"Kids' Furniture, Décor & Storage\" 'Puffer' 'Science' 'Bracelet'\n",
            " 'Christmas' '50 To 75 Years' 'Game' 'Calendars' 'Batteries'\n",
            " 'Swings, Jumpers & Bouncers' 'Others' 'DVD & Blu-ray Players' 'Raincoat'\n",
            " 'Athletic Training' 'Hiking & Camping' 'Scoop Neck' 'Cargo'\n",
            " 'Breastfeeding' 'Pant Suit' 'Bowl' 'A-Line' 'Wind & Woodwind Instruments'\n",
            " 'Overalls' 'Dance' 'Bookmark' 'Kitchen & Table Linens' 'Waxing'\n",
            " 'Water Coolers & Filters' 'Sun' 'Work & Safety' 'Monitors'\n",
            " 'Air Fresheners' 'Lingerie' 'Wipes & Holders' 'Men' 'Pinback Button'\n",
            " 'Candles & Holders' 'Leg Warmers' 'Highchairs & Booster Seats'\n",
            " 'Camcorders' 'Puzzles' 'Tumbler' 'Vintage' 'Supplies' 'Business & Money'\n",
            " 'Bike & Skate' 'Turtleneck' 'Candle Holder' 'Satchel' 'Organization'\n",
            " 'Skirt' 'Bathroom Storage & Organization' 'Baskets' 'Stationery'\n",
            " 'Hair & Scalp Treatments' 'Relaxed' 'Lanyard' 'Coaster' 'Books and Zines'\n",
            " 'Indoor' 'NHL' 'Soccer' 'Reference' 'Posters & Prints' 'Nail Tools'\n",
            " 'Leash' 'Boyfriend' 'Baby Seats' 'Car' 'Televisions' 'Vases'\n",
            " 'Area Rugs & Pads' 'Tray' 'Wine, Beer & Beverage Coolers' 'Flare'\n",
            " 'Dress - Flat Front' 'Trench' 'Prenatal Monitoring Devices' 'Easter'\n",
            " 'Performance Parts & Accessories' 'Toy Remote Control & Play Vehicles'\n",
            " 'Varsity' 'Travel Beds' 'Tennis & Racquets' 'Board Shorts' 'Dress'\n",
            " 'Hands & Nails' '75 To 100 Years' 'Action, Adventure' 'Pendant'\n",
            " 'Cabinet Locks & Straps' 'Golf Balls' 'Lights & Lighting Accessories'\n",
            " 'Tweezers' 'Animal' 'Bags & Cases' 'Sweatercoat' 'Needlepoint'\n",
            " 'Skateboard' 'Storage & Organization' 'Teething Relief' 'Stamps'\n",
            " 'Sports & Outdoor Play' 'Gloves' 'Mirrors' 'Basketball' 'Apron' 'Walkers'\n",
            " 'Skirts, Skorts & Dresses' 'Snowboard' 'GPS Units & Equipment' 'Linens'\n",
            " 'Soaps & Cleansers' 'Sculptures' 'Hockey' 'Oxfords'\n",
            " 'Instrument Accessories' 'Fabric' 'Home Decor' 'Parka' 'Bead'\n",
            " 'Vest, Sleeveless' 'Laundry Storage & Organization' 'Guitars' 'Patch'\n",
            " 'Interior Accessories' \"St Patrick's\" 'Window Treatments' 'Corset'\n",
            " 'Golf Apparel' 'Scale Dollhouse Miniature' 'Skin Care' 'Clothing' 'Pin'\n",
            " 'Biographies & Memoirs' 'Outdoor' 'Kitchen' 'Swim Trunks'\n",
            " 'Bathing Accessories' 'Food Service Equipment & Supplies'\n",
            " 'Christian Books & Bibles' 'Decorative Pillows' 'Shampoo' 'Two Button'\n",
            " 'Bed Pillows' 'Home Speakers & Subwoofers' 'Cake Toppers'\n",
            " 'Baskets & Bins' 'Sci-Fi, Fantasy' 'Magic' 'Stationery & Party Supplies'\n",
            " 'Quilts' 'Tricycles, Scooters & Wagons' 'Teacup' 'Clock'\n",
            " 'Changing Pads & Covers' 'Health' 'Microphones & Accessories'\n",
            " 'Basic Supplies' 'Tripods & Supports' 'Wool' 'Embroidery' 'Bath Rugs'\n",
            " 'Home Surveillance' 'Space Heaters' 'Box' 'Glassware' 'Rainwear'\n",
            " 'Sweeping' 'Family Planning Tests' 'Baby & Child Care'\n",
            " 'Other Accessories' 'GPS Accessories & Mounts' 'Leather'\n",
            " 'Knitting Supplies' 'Corduroys' 'Board, Surf' 'History'\n",
            " 'Studio Recording Equipment' 'Athletic Apparel'\n",
            " 'Shampoo Plus Conditioner' 'Jacket' 'Baggy, Loose' 'Fans'\n",
            " 'Washcloths & Towels' 'Art' 'Biography' 'Shopping Cart Covers' 'Standard'\n",
            " 'Scarf' 'Linen' 'Flight' 'Sun Protection' 'Skirt Suit' 'One Button'\n",
            " 'Pet Lover' 'Track & Sweat Suits' 'Keyboards' 'Sewing'\n",
            " 'Electrical Safety' 'Cuff Links' 'Binoculars & Telescopes' 'Belt'\n",
            " 'Papermaking' 'Vests' 'Stringed Instruments' 'Bowls' 'Snowsuits & Bibs'\n",
            " 'Hat' 'Postcard' 'Breastfeeding Pillows & Stools' 'Home Audio'\n",
            " 'Bathing Tubs & Seats' 'Travel Systems' 'Printers, Scanners & Supplies'\n",
            " 'Full Skirt' 'Change Purse' 'Politics & Social Sciences' 'Children'\n",
            " 'Washers & Dryers' 'Air Purifiers' 'Teethers'\n",
            " 'Decorative Pillows, Inserts & Covers' 'Pillows & Stools'\n",
            " 'Kitchen Knives & Cutlery Accessories' 'Thermal Underwear' 'Poster'\n",
            " 'Shrug' 'Dress Suit' 'Religion & Spirituality' 'Nursery Décor'\n",
            " 'Novelty & Gag Toys' 'Racks, Shelves & Drawers' 'Porcelain' 'Cage'\n",
            " 'Fiction' 'Maternity' 'Print' 'Yarn' 'Gift Wrap' 'Bed' 'Wallet'\n",
            " 'Water Sports' 'Lenses & Filters' 'Bedroom Furniture' 'Crafting'\n",
            " 'Nasal Aspirators' 'Car Care' 'Toothbrushes' 'Mattress Pads'\n",
            " 'Decorations' 'Patriotic' 'Humidifiers & Vaporizers' 'Housewares'\n",
            " 'Birthday' 'Track & Field' 'Journal' 'Strength training' 'Puzzle'\n",
            " 'Microwaves' 'Hair Loss Products' 'Dress - Pleat' 'Illustration'\n",
            " 'Strategy Guides' 'Poetry' 'Historical, Military'\n",
            " 'DJ, Electronic Music & Karaoke' 'Backpack' 'Car Speakers & Systems'\n",
            " 'Feeding' 'Salt and Pepper Shakers' 'Paper Towels' 'Air Conditioners'\n",
            " 'Car Stereos & Components' 'Irons & Ironing Boards' 'Science & Math'\n",
            " 'Hair Relaxers' 'Camera' 'Replacement Parts & Tools' 'Mopping' 'Baguette'\n",
            " 'Signs' 'Cabochon' 'Calendar' 'Sweaters' 'Epilators' 'Watch'\n",
            " 'Cross Stitch' 'Swim Briefs' 'Vase' 'Bass Guitars' 'Harnesses & Leashes'\n",
            " 'Wall Hanging' 'Lighting' 'Carving' 'Case' 'Engineering & Transportation'\n",
            " 'How to' 'Peasant' 'Lighting & Studio' 'Shirt' 'Motorcycle & Powersports'\n",
            " 'Bags and Purses' 'Notebook' 'Equipment' 'Dress Shorts'\n",
            " 'Amplifiers & Effects' 'Hip Bag' 'Drums & Percussion'\n",
            " 'Travel Bathing Kits' 'Collages' 'Favors' 'Dehumidifiers' 'Three Button'\n",
            " 'Polo' 'Grooming & Healthcare Kits' 'Training Pants' 'Magnet' 'Telephone'\n",
            " 'Lacrosse' 'Hawaiian' 'Religious' 'Outdoor Safety' 'Wide Leg'\n",
            " 'Personalized' 'Joggers' 'Tshirt' 'Toiletry Kits' 'Ornaments'\n",
            " 'Golf Shoes' \"Kids' Furniture\" 'Trash Bags' 'Nursery Bedding'\n",
            " 'Diaper Bag' 'Videogame' 'Planter' 'Potties & Seats'\n",
            " 'Automotive Enthusiast Merchandise' 'Buckle' 'Printmaking' 'Photographs'\n",
            " 'Casserole' 'Shams, Bed Skirts & Bed Frame Draperies' 'Plate'\n",
            " 'Garage Storage & Organization' 'Tires & Wheels' 'Instrument' 'Paper'\n",
            " 'Bowling' 'Refrigerators' 'Inflatable Beds'\n",
            " 'Kitchen Storage & Organization' 'Tiered' 'Other Furniture' 'Frame'\n",
            " 'Teapot' \"New Year's\" 'Thermometers' 'Bathroom' 'Television' 'Laptop'\n",
            " 'Living Room Furniture' \"Kids' Bath\" 'Sleep Positioners' 'Accessory'\n",
            " 'Magnets' 'Humor' 'Slipcovers' 'Bathroom Safety' 'Small Animal'\n",
            " 'Flatware' 'Diaper Stackers & Caddies' 'Bubble Bath' 'Gift Sets'\n",
            " 'Hair Perms & Texturizers' 'Skirts' 'Track Jacket' 'Paperweights'\n",
            " 'Outerwear' 'Custom' 'Finding' 'Maternity Pillows' 'Invitations'\n",
            " 'Live Sound & Stage' 'Graphic Design' 'Cream and Sugar Set'\n",
            " 'Dinnerware Set' 'Draft Stoppers' 'Comic' 'Nonfiction' 'Tuxedo'\n",
            " 'Lithographs, Etchings & Woodcuts' 'Car Subwoofers'\n",
            " 'Activity Centers & Entertainers' 'Pitcher' 'Button' 'Shawl'\n",
            " 'Bedspreads & Coverlets' 'Gadget' 'Paint, Body & Trim' 'Carpenter'\n",
            " 'Guest Books' 'Toys' 'Stained Glass' 'Block' 'Bouquets'\n",
            " 'Replacement Parts' 'Denim' 'Golf Bags' 'Tape' 'Storage Cabinets'\n",
            " 'Furniture' 'Art Doll' 'Bubble' 'Brushes' 'Child Friendly' 'Car Video'\n",
            " 'Brass Instruments' 'Safety Caps' 'Fireplaces & Accessories'\n",
            " 'Presentation' 'Doll Clothes' 'Human Figure Doll' 'Suits & Blazers'\n",
            " 'Beads' 'Home Entertainment Furniture' 'Fantasy' 'Bath Linen Sets'\n",
            " 'Pillow' 'Day of the Dead' 'Badminton' 'Kitchen Safety' 'Knitting'\n",
            " 'Lightweight' 'Decorating' 'Flashes & Flash Accessories'\n",
            " 'Garbage Disposals' 'Plush' 'Planters' 'Office' 'Gates & Doorways'\n",
            " 'Boxes' 'Carpenter, Utility' 'Afghan' 'Trash & Recycling' 'Pillows'\n",
            " 'Playard Bedding' 'Instructional' 'Changing Kits' 'Crochet'\n",
            " 'RV Parts & Accessories' 'Educational' 'Origami' 'Band & Orchestra'\n",
            " 'Chain' 'Pattern' 'Home Brewing & Wine Making' 'Doll Clothing'\n",
            " 'Figurines' 'Double Breasted' 'Table' 'Hair Coloring Tools' 'Music'\n",
            " 'Illustrated' 'Prams' 'Car Security & Convenience' \"Women's Golf Clubs\"\n",
            " 'Dusting' 'Freezers & Ice Makers' 'Basket' 'Trim' 'Oils & Fluids'\n",
            " 'Religion' 'Bed in a Bag' 'Storage' 'Blanket' 'Powders & Lotions'\n",
            " 'Non-Slip Bath Mats' 'Something Blue' 'House' 'Cleaning' 'Handkerchief'\n",
            " 'Animals' 'Scifi' 'Tiles' 'Mixed Media' 'Sponges' 'Felted' 'Aceo'\n",
            " 'Frames' 'Fixture' 'Serving' 'Beading' 'Grooming' 'Seat Covers'\n",
            " 'Advertisement' 'Pad' 'Nail Care' 'Holiday Décor Storage' 'Dishes'\n",
            " 'Coasters' 'Fabric Postcard' 'Toddler' 'Chair' 'Candles'\n",
            " 'Bathroom Furniture Sets' 'Price Guides & Publications' 'Paper Ephemera'\n",
            " 'Horror' 'Nursery Furniture' 'Light' 'Patchwork' 'Tag' 'Handmade'\n",
            " 'Primitive' 'Necktie' 'Bicycle Child Seats & Trailers' 'Sport'\n",
            " 'Home Bar Furniture' 'Storage Solutions' 'Novelty' 'Pot Holder' 'Boating'\n",
            " 'Map' 'Shelf' 'Entertaining' 'Scale Models' 'Competitive Swimwear'\n",
            " 'International Publications' 'Cozy' 'Miniature' 'St Patricks'\n",
            " 'Fashion Dolls Apparel' 'Butter Dish' 'Mirror' 'Just Married'\n",
            " 'Cotton & Swabs' 'Pet Food' 'Bathroom Furniture' 'Cuff' 'Hanukkah'\n",
            " 'Feather Beds' 'Burning' 'Compact' 'Quilt' 'New Years'\n",
            " 'Edge & Corner Guards' 'Doorstops' 'Seasonal' 'Bathroom Shelves'\n",
            " \"Kids' Flatware\" 'Towel' 'Crib Netting' 'Four Button' 'Entertainment'\n",
            " 'Platter' 'Zipper' 'Step Stools' 'Dishwashers' 'Bear' 'Artist Bears'\n",
            " 'Doormats' 'Ephemera']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "대분류의 경우 women, beauty, kids 등의 분류가 매우 많다.\n",
        "\n",
        "나머지 칼럼의 null값은 일괄적으로 other null로 동일하게 변경하자. \n",
        "\n",
        "brand_name은 price값 결정에 영향을 줄 것으로 판단되지만 null값이 매우많다. 따라서 null값을 다른 값으로 변경하기 보다 other null로 변경해준다. "
      ],
      "metadata": {
        "id": "FZh3DpFvVmqe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "mercari_df['brand_name']=mercari_df['brand_name'].fillna(value='Other_Null')\n",
        "mercari_df['category_name']=mercari_df['category_name'].fillna(value='Other_Null')\n",
        "mercari_df['item_description']=mercari_df['item_description'].fillna(value='Other_Null')"
      ],
      "metadata": {
        "id": "jaQ90wepV3SC"
      },
      "execution_count": 15,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#각 칼럼별로 null값 건수 확인, 모두 0이 나옴\n",
        "mercari_df.isnull().sum()"
      ],
      "metadata": {
        "id": "s0xGDrjsV7_v",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "41040cf1-1070-4ae1-8a0a-a3ee1fa4221f"
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "train_id             0\n",
              "name                 0\n",
              "item_condition_id    0\n",
              "category_name        0\n",
              "brand_name           0\n",
              "price                0\n",
              "shipping             0\n",
              "item_description     0\n",
              "cat_dae              0\n",
              "cat_jung             0\n",
              "cat_so               0\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 피처 인코딩과 피처 벡터화\n",
        "\n",
        "문자열 칼럼 중 레이블 또는 원핫인코딩을 수행하거나 피처 벡터화로 변환할 칼럼을 선별해보자.\n",
        "\n",
        "* 선형회귀모델, 회귀트리모델 모두 이용한다.\n",
        "\n",
        "* 선형회귀모델에서는 원핫인코딩을 적용해준다.\n",
        "\n",
        "* 피처 벡터화의 경우 비교적 짧은 텍스트의 경우는 count기반, 긴 텍스트는 TD-IDF기반 적용\n",
        "\n"
      ],
      "metadata": {
        "id": "Ods_iGIQV9bd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#상품 브랜드명\n",
        "\n",
        "print('brand name의 유형 건수 :', mercari_df['brand_name'].unique())\n",
        "print('brand name sample 5건 : \\n', mercari_df['brand_name'].value_counts()[:5])"
      ],
      "metadata": {
        "id": "08vb1c7BWZVO",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e4398eac-c567-4241-ca2e-fcbbdc6c91cb"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "brand name의 유형 건수 : ['Other_Null' 'Razer' 'Target' ... 'Astroglide' 'Cumberland Bay'\n",
            " 'Kids Only']\n",
            "brand name sample 5건 : \n",
            " Other_Null           632682\n",
            "PINK                  54088\n",
            "Nike                  54043\n",
            "Victoria's Secret     48036\n",
            "LuLaRoe               31024\n",
            "Name: brand_name, dtype: int64\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "대부분 명료한 문자열로 돼 있다. 별도의 피처 벡터화 형태로 만들 필요 없이 인코딩 변환을 적용한다. "
      ],
      "metadata": {
        "id": "jiwUCkZyWdCQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#상품명 유형\n",
        "\n",
        "print('name의 유형 건수 :', mercari_df['name'].unique())\n",
        "print('name sample 10건 : \\n', mercari_df['name'].value_counts()[:10])"
      ],
      "metadata": {
        "id": "Z6A1aJoKWj_w",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "69349bbf-d469-433d-c288-cc358f0ae579"
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "name의 유형 건수 : ['MLB Cincinnati Reds T Shirt Size XL' 'Razer BlackWidow Chroma Keyboard'\n",
            " 'AVA-VIV Blouse' ... '21 day fix containers and eating plan'\n",
            " 'World markets lanterns' 'Brand new lux de ville wallet']\n",
            "name sample 10건 : \n",
            " Bundle                 2232\n",
            "Reserved                453\n",
            "Converse                445\n",
            "BUNDLE                  418\n",
            "Dress                   410\n",
            "Coach purse             404\n",
            "Lularoe TC leggings     396\n",
            "Romper                  353\n",
            "Nike                    340\n",
            "Vans                    334\n",
            "Name: name, dtype: int64\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "상품명은 name속성의 경우 종류가 매우 많다. 유형이 매우 많고 적은 단어 위주의 텍스트 형태로 돼 있으므로 count기반으로 피처 벡터화 변환 적용한다.\n",
        "\n",
        "shipping과 category_name 모두 원핫인코딩을 적용한다.\n",
        "\n",
        "item_description은 가장 긴 텍스트를 가지고 있다. "
      ],
      "metadata": {
        "id": "277lDWoMWndl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "pd.set_option('max_colwidth', 200)\n",
        "\n",
        "#item_description의 평균 문자열 크기\n",
        "print('item_description 평균 문자열 크기:', mercari_df['item_description'].str.len().mean())\n",
        "\n",
        "mercari_df['item_description'][:2]"
      ],
      "metadata": {
        "id": "wifsW6k4W54O",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "473ccd75-9695-49b4-c776-5600ef2faf16"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "item_description 평균 문자열 크기: 145.7113889385411\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0                                                                                                                                                                              No description yet\n",
              "1    This keyboard is in great condition and works like it came out of the box. All of the ports are tested and work perfectly. The lights are customizable via the Razer Synapse app on your PC.\n",
              "Name: item_description, dtype: object"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "비교적 문자열이 크므로 해당 칼럼은 TF-IDF로 변환한다.\n",
        "\n",
        "이제 각각 피처벡터화를 해보자."
      ],
      "metadata": {
        "id": "JACTDenMW7Dr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#name 속성 피처 벡터화 변환\n",
        "cnt_vec=CountVectorizer()\n",
        "X_name=cnt_vec.fit_transform(mercari_df.name)\n",
        "\n",
        "#item_description 피처 벡터화 변환\n",
        "tfidf_descp=TfidfVectorizer(max_features=50000, ngram_range=(1,3), stop_words='english')\n",
        "X_descp=tfidf_descp.fit_transform(mercari_df['item_description'])\n",
        "\n",
        "print('name vectorization shape:', X_name.shape)\n",
        "print('item_description vectorization shape:', X_descp.shape)\n",
        "     "
      ],
      "metadata": {
        "id": "epxSXyPAXCXn",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f7b66e1a-a3fc-4f6e-85d7-239a024ea5e2"
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "name vectorization shape: (1482535, 105757)\n",
            "item_description vectorization shape: (1482535, 50000)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        ".fit_transform로 반환하는 데이터는 희소행렬 형태이다.\n",
        "\n",
        "희소 행렬 객체 변수인 X_name과 X_descp를 새로 결합해 새로운 데이터 세트로 구성해야하고, 앞으로 인코딩 될 모든 변수들도 이 둘과 결합돼 ML모델을 실행하는 기반 데이터 세트로 재구성돼야한다.\n",
        "\n",
        "-> 인코딩 대상 칼ㄹ럼도 밀집 행렬 형태가 아닌 희소 행렬 형태로 인코딩\n",
        "\n",
        "* LabelBinarizer : 희소 행렬 형태의 원핫인코딩 지원\n",
        "* hstack() : 희소행렬 결합할 때 사용"
      ],
      "metadata": {
        "id": "WRlFznf1XFSH"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#인코딩\n",
        "from sklearn.preprocessing import LabelBinarizer\n",
        "\n",
        "#brand_name, item_condition_id, shipping 각 피처들을 희소 행렬 원-핫 인코딩 변환\n",
        "lb_brand_name=LabelBinarizer(sparse_output=True)\n",
        "X_brand=lb_brand_name.fit_transform(mercari_df['brand_name'])\n",
        "lb_item_cond_id=LabelBinarizer(sparse_output=True)\n",
        "X_item_cond_id=lb_item_cond_id.fit_transform(mercari_df['item_condition_id'])\n",
        "lb_shipping=LabelBinarizer(sparse_output=True)\n",
        "X_shipping=lb_shipping.fit_transform(mercari_df['shipping'])\n",
        "\n",
        "#cat_dae 등 각 피처들을 희소 행렬 원-핫 인코딩 변환\n",
        "lb_cat_dae=LabelBinarizer(sparse_output=True)\n",
        "X_cat_dae=lb_cat_dae.fit_transform(mercari_df['cat_dae'])\n",
        "lb_cat_jung=LabelBinarizer(sparse_output=True)\n",
        "X_cat_jung=lb_cat_jung.fit_transform(mercari_df['cat_jung'])\n",
        "lb_cat_so=LabelBinarizer(sparse_output=True)\n",
        "X_cat_so=lb_cat_so.fit_transform(mercari_df['cat_so'])"
      ],
      "metadata": {
        "id": "SmRoN-cTXujP"
      },
      "execution_count": 21,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(type(X_brand), type(X_item_cond_id), type(X_shipping))\n",
        "print('X_brand shape:{0}, X_item_cond_id shape:{1}'.format(X_brand.shape, X_item_cond_id.shape))\n",
        "print('X_shipping shape:{0}, X_cat_dae shape:{1}'.format(X_shipping.shape, X_cat_dae.shape))\n",
        "print('X_cat_jung shape:{0}, X_cat_so shape:{1}'.format(X_cat_jung.shape, X_cat_so.shape))"
      ],
      "metadata": {
        "id": "6QdFsMtqXxG9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7447dd2b-ffaf-4acf-ac1a-e97b09d13cb6"
      },
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'scipy.sparse._csr.csr_matrix'> <class 'scipy.sparse._csr.csr_matrix'> <class 'scipy.sparse._csr.csr_matrix'>\n",
            "X_brand shape:(1482535, 4810), X_item_cond_id shape:(1482535, 5)\n",
            "X_shipping shape:(1482535, 1), X_cat_dae shape:(1482535, 11)\n",
            "X_cat_jung shape:(1482535, 114), X_cat_so shape:(1482535, 871)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "인코딩 컬럼이 매우 만힝 생겼지만 피처 벡터화로 텍스트 형태의 문자열이 가지는 벡터 형태의 매우 많은 칼럼과 함께 결합되므로 크게 문제가 없다. "
      ],
      "metadata": {
        "id": "3WoB0hZmXzp8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#결합\n",
        "\n",
        "from scipy.sparse import hstack\n",
        "import gc\n",
        "\n",
        "sparse_matrix_list=(X_name, X_descp, X_brand, X_item_cond_id,\n",
        "                    X_shipping, X_cat_dae, X_cat_jung, X_cat_so)\n",
        "\n",
        "#hstack 함수를 이용해 인코딩과 벡터화를 수행한 데이터 세트를 모두 결합\n",
        "X_features_sparse=hstack(sparse_matrix_list).tocsr()\n",
        "print(type(X_features_sparse), X_features_sparse.shape)"
      ],
      "metadata": {
        "id": "PbWPdOZBYK7q",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7e2ef3bc-17a3-4205-cebd-dd15115991d8"
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'scipy.sparse._csr.csr_matrix'> (1482535, 161569)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "이제 이렇게 만들어진 데이터 세트에 회귀를 적용해 price값을 예측할 수 있도록 모델을 만들 차례이다.\n",
        "\n",
        "## 릿지 회귀 모델 구축 및 평가\n",
        "\n",
        "여러 알고리즘 모델과 희소 행렬을 변환하고 예측 성능을 비교하면서 테스트를 수행할 것이므로 로직을 함수화 해야한다.\n",
        "\n",
        "* 평가로직 함수화\n",
        "\n",
        "RMSLE를 사용한다. 주의할 점은 로그 변환을 취했기 때문에 예측도 로그 변환한 값 수준이다. 따라서 학습 모델을 이용한 예측된 PRICE값은 다시 로그의 역변환인 지수 변환을 수행해 복원한다."
      ],
      "metadata": {
        "id": "EaGOdyERYPqA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def rmsle(y, y_pred):\n",
        "  #underflow, overflow를 막기 위해 log1p로 rmsle 계산\n",
        "  return np.sqrt(np.mean(np.power(np.log1p(y)-np.log1p(y_pred),2)))\n",
        "\n",
        "def evaluate_org_price(y_test, preds):\n",
        "  #원본 데이터는 log1p 변환되었으므로 expm1으로 원복 필요.\n",
        "  preds_exmpm=np.expm1(preds)\n",
        "  y_test_exmpm=np.expm1(y_test)\n",
        "\n",
        "  #rmsle로 RMSLE 값 추출\n",
        "  rmsle_result=rmsle(y_test_exmpm, preds_exmpm)\n",
        "  return rmsle_result"
      ],
      "metadata": {
        "id": "IhOjKiogYluJ"
      },
      "execution_count": 24,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "학습용 데이터를 생성하고 모델을 학습/예측하는 로직을 별도의 함수로 만들자. "
      ],
      "metadata": {
        "id": "37xo-9djYpmr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#최종 데이터 세트로 결합할 희소 행렬 리스트를 가진다.\n",
        "import gc\n",
        "from scipy.sparse import hstack\n",
        "\n",
        "def model_train_predict(model, matrix_list):\n",
        "  #scipy.sparse 모듈의 hstack을 이용해 희소 행렬 결합\n",
        "  X=hstack(matrix_list).tocsr()\n",
        "\n",
        "  X_train, X_test, y_train, y_test=train_test_split(X, mercari_df['price'], test_size=0.2, random_state=156)\n",
        "\n",
        "  #모델 학습 및 예측\n",
        "  model.fit(X_train, y_train)\n",
        "  preds=model.predict(X_test)\n",
        "\n",
        "  del X, X_train, X_test, y_train\n",
        "  gc.collect()\n",
        "\n",
        "  return preds, y_test"
      ],
      "metadata": {
        "id": "EnbIugTIYudW"
      },
      "execution_count": 25,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 릿지 예측 수행\n",
        "\n",
        "개별 함수를 만들었으면 이제 이를 이용해 먼저 릿지로 회귀 예측을 수행해보자. \n",
        "\n",
        "수행 전에 Merari상품 가격 예측에 item_description과 같은 텍스트 형태의 속성이 얼마나 영향을 미치는지 알아보자."
      ],
      "metadata": {
        "id": "XKfBRsmOY3pN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "linear_model=Ridge(solver='lsqr', fit_intercept=False)\n",
        "\n",
        "sparse_matrix_list=(X_name, X_brand, X_item_cond_id,\n",
        "                    X_shipping, X_cat_dae, X_cat_jung, X_cat_so)\n",
        "linear_preds, y_test=model_train_predict(model=linear_model, matrix_list=sparse_matrix_list)\n",
        "print('item description을 제외했을 때 rmsle 값 ;', evaluate_org_price(y_test, linear_preds))\n",
        "\n",
        "sparse_matrix_list=(X_descp, X_name, X_brand, X_item_cond_id,\n",
        "                    X_shipping, X_cat_dae, X_cat_jung, X_cat_so)\n",
        "linear_preds, y_test=model_train_predict(model=linear_model, matrix_list=sparse_matrix_list)\n",
        "print('item description을 포함함했을 때 rmsle 값 ;', evaluate_org_price(y_test, linear_preds))"
      ],
      "metadata": {
        "id": "4faWU_MDZFgp",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0e274afc-1fb9-447d-8efa-17fee52b0119"
      },
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "item description을 제외했을 때 rmsle 값 ; 0.4983990938999374\n",
            "item description을 포함함했을 때 rmsle 값 ; 0.4680329023962086\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "rmsle값이 많이 감소했다. item_Description영향이 중요하다.\n",
        "\n",
        "## LightGBM 회귀 모델 구축과 앙상블을 이용한 최종 예측 평가\n",
        "\n",
        "다음으로는 LightGBM을 이용해 회귀를 수행하고, 위에서 구한 릿지 모델 예측값과 이번에 구할 예측값을 간단한 앙상블 방식으로 섞어 평가하자.\n",
        "\n",
        "* lightGBM 회귀 수행행"
      ],
      "metadata": {
        "id": "zVDfC6rhZH5S"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from lightgbm import LGBMRegressor\n",
        "\n",
        "sparse_matrix_list=(X_descp, X_name, X_brand, X_item_cond_id,\n",
        "                    X_shipping, X_cat_dae, X_cat_jung, X_cat_so)\n",
        "\n",
        "lgbm_model=LGBMRegressor(n_estimators=100, learning_rate=0.5, num_leaves=125, random_state=156)\n",
        "lgbm_preds, y_test=model_train_predict(model=lgbm_model, matrix_list=sparse_matrix_list)\n",
        "print('lgbm rmsle 값 ;', evaluate_org_price(y_test, lgbm_preds))"
      ],
      "metadata": {
        "id": "-9bEWKwOWJ_T"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "preds=lgbm_preds*0.45+linear_preds*0.55\n",
        "print('lgbm과 ridge를 앙상블 한 최종 rmsle 값 :', evaluate_org_price(y_test, preds))"
      ],
      "metadata": {
        "id": "a_ORtwZdWLhv"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "예측 성능이 나아졌다다. \n",
        "\n",
        "이제 앙상블해서 최종 예측 결과값을을 평가해보자."
      ],
      "metadata": {
        "id": "lQSnLxIvZd6B"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "성능이 개선되었다다.\n",
        "\n",
        "\n",
        "---\n",
        "## 11. 정리\n",
        "\n",
        "텍스트 분석을 위한 기반 프로세스를 알아보았다. \n",
        "텍스트 분류, 감성분석, 토픽 모델링, 텍스트 군집화 및 유사도 측정 등을 직접 파이썬 코드를 이용해 구현했다.\n",
        "\n",
        "**<머신러닝 기반 텍스트 분석 프로세스>**\n",
        "\n",
        "1. 텍스트 사전 정제 작업 등 텍스트 정규화 작업 수행\n",
        "2. 이들 단어들을 피처 벡터화로 변환\n",
        "3. 생성된 피처 벡터 데이터 세트에 머신러닝 모델 학습, 예측, 평가\n",
        "\n",
        "* 정규화 작업 \n",
        "  - 텍스트 클렌징, 대소문자 변경, 단어 토큰화 , 의미 없느 ㄴ단어 필터링, 어근 추출\n",
        "\n",
        "* 피처 벡터화\n",
        "  - BOW의 대표 방식인 count기반과 TF-IDF 기반 피처 벡터화 \n",
        "\n",
        "-> 희소행렬 데이터세트이다.\n",
        "\n",
        "머신러닝은 이러한 희소 행렬 기반에서 최적화되어야 한다.\n",
        "\n",
        "**<텍스트 분류절>**\n",
        "\n",
        "문서들을 피처 벡터화한 후 로지스틱 회귀를 적용해 문서를 지도학습 방식으로 예측 분류해 보았다. \n",
        "\n",
        "<감성 분석절>\n",
        "**굵은 텍스트**\n",
        "지도학습 기반으로 긍정/부정 이진 분류를 적용한 방식과 감성사전을 이용한 방식 두가지를 살펴봄\n",
        "\n",
        "**<토픽 모델링>**\n",
        "\n",
        "lda를 이용해 뉴스그룹 내의 많은 문서들이 공통적으로 가지는 토픽들을 추출\n",
        "\n",
        "**<텍스트 군집화>**\n",
        "\n",
        "K-평균군집화 기법을 이용해 비슷한 문서들끼리 군집화했음.\n",
        "\n",
        "또한 한글 형태소 분석기인 KoNLPy패키지를 이용해 네이버에서 제공하는 한글로 된 영화리뷰에 이진진 분류를 적용해보았다.\n"
      ],
      "metadata": {
        "id": "KdVNauDAZH_O"
      }
    }
  ]
}