la_05_interpretation_matrix_multiplication.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[],"authorship_tag":"ABX9TyPhqdPk15wYvHCpiFFapPTK"},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"}},"cells":[{"cell_type":"markdown","source":["#### Interpretation of matrix-vector multiplication"],"metadata":{"id":"cAvNgUEGZHF1"}},{"cell_type":"markdown","source":["$y\\in \\mathbf{R}^m,\\,A\\in \\mathbf{R}^{m\\times n},\\, x\\in \\mathbf{R}^n$"],"metadata":{"id":"FFUmogHNXZlo"}},{"cell_type":"markdown","source":["1st interpretation, as `mixture of columns` of $A$\n","\n","$$A=\\begin{bmatrix}a_1& a_2 & \\cdots & a_n\\end{bmatrix}$$\n","\n","where each column $a_i\\in\\mathbf{R}^m$"],"metadata":{"id":"h6eZHgB-h3xr"}},{"cell_type":"markdown","source":["For $x$\n","\n","$$x=\\begin{bmatrix}x_1\\\\x_2\\\\\\vdots\\\\x_n\\end{bmatrix}$$\n","\n","we have\n","\n","$$\\boxed{y=\\sum_{i=1}^n x_ia_i}$$"],"metadata":{"id":"JuipZqvjh7w4"}},{"cell_type":"markdown","source":["2nd interpretation, as `inner product with rows` of $A$\n","\n","$$A=\\begin{bmatrix}\\tilde{a_1}^T\\\\\\tilde{a_2}^T\\\\\\vdots\\\\\\tilde{a_m}^T\\end{bmatrix}$$\n","\n","where each row $\\tilde{a_i}^T\\in\\mathbf{R}^n$"],"metadata":{"id":"BnGwxHzHiWd6"}},{"cell_type":"markdown","source":["Then, we have\n","\n","$$\\boxed{y=\\begin{bmatrix}\\tilde{a_1}^T x\\\\\\tilde{a_2}^T x\\\\\\vdots\\\\\\tilde{a_m}^T x\\end{bmatrix}}$$"],"metadata":{"id":"e0FvC9Cli4E5"}},{"cell_type":"markdown","source":["#### Interpretation of matrix-matrix multiplication"],"metadata":{"id":"9f8dOTuHjQQQ"}},{"cell_type":"markdown","source":["$C\\in \\mathbf{R}^{m\\times p},\\,A\\in \\mathbf{R}^{m\\times n},\\, B\\in \\mathbf{R}^{n\\times p}$"],"metadata":{"id":"W9DQ6y0JjX0J"}},{"cell_type":"markdown","source":["1st interpretation, $A$ acting on `columns` of $B$\n","\n","$$\\boxed{C=A\\begin{bmatrix}b_1& b_2 & \\cdots & b_p\\end{bmatrix}}$$\n","\n","where each column $b_i\\in\\mathbf{R}^n$\n","\n","That is, each `column` in $C$ is some `linear combination of columns` in $A$"],"metadata":{"id":"Mm3OFMTujgMs"}},{"cell_type":"markdown","source":["2nd interpretation, $B$ acting on `rows` of $A$\n","\n","$$\\boxed{C=\\begin{bmatrix}\\tilde{a_1}^TB\\\\\\tilde{a_2}^TB\\\\\\vdots\\\\\\tilde{a_m}^TB\\end{bmatrix}}$$\n","\n","where each row $\\tilde{a_i}^T\\in\\mathbf{R}^n$\n","\n","That is, each `row` in $C$ is some `linear combination of rows` in $B$"],"metadata":{"id":"n0UBnHocj1bI"}},{"cell_type":"markdown","source":["3rd interpretation, `sum of outer products`\n","\n","Write\n","\n","$$A=\\begin{bmatrix}a_1& a_2 & \\cdots & a_n\\end{bmatrix}$$\n","\n","and\n","\n","$$B=\\begin{bmatrix}\\tilde{b_1}^T\\\\\\tilde{b_2}^T\\\\\\vdots\\\\\\tilde{b_n}^T\\end{bmatrix}$$\n","\n","where each column $a_i\\in \\mathbf{R}^m$ and each row $\\tilde{b_i}^T \\in \\mathbf{R}^p$\n","\n","Then\n","\n","$$\\boxed{C=\\sum_{i=i}^na_i\\tilde{b_i}^T}$$\n","\n","Each $a_i\\tilde{b_i}^T\\in \\mathbf{R}^{m\\times p}$ is a `low-rank approximation` of $C$\n"],"metadata":{"id":"oCKx-ioHkni9"}},{"cell_type":"code","source":[],"metadata":{"id":"25gUmmn9jXeS"},"execution_count":null,"outputs":[]}]}