Merge branch 'pluralitybook:main' into main

audreyt · Mar 23, 2024 · 27b7444 · 27b7444
2 parents 833bd99 + cdb9a44
commit 27b7444
Show file tree

Hide file tree

Showing 15 changed files with 2,203 additions and 1,414 deletions.
diff --git a/contents/english/6-0-from-⿻-to-reality.md b/contents/english/6-0-from-⿻-to-reality.md
diff --git a/contents/english/6-1-workplace.md b/contents/english/6-1-workplace.md
diff --git a/contents/english/6-2-health.md b/contents/english/6-2-health.md
diff --git a/contents/english/6-3-media.md b/contents/english/6-3-media.md
diff --git a/contents/english/6-4-environment.md b/contents/english/6-4-environment.md
diff --git a/scripts/index/Plurality Book Indexing Exercise - Main.csv b/scripts/index/Plurality Book Indexing Exercise - Main.csv
diff --git a/scripts/index/README.md b/scripts/index/README.md
@@ -2,12 +2,15 @@
 
 - `Plurality Book Indexing Exercise - Main.csv`: raw file exported from [Spreadsheet](https://docs.google.com/spreadsheets/d/1gmyjFbErt_CW8-qLKChSpciLlCDGUhLriYFov0HO3qA/edit#gid=0)
 - `main.py`: output POC count, occurence of each keywords in each sections, and the count of occurences 
+- `ignore.txt`: keywords which should avoid mechine search
 
 ## output
 - `contributors.tsv`: number of contribution on the spreadsheet
 - `keyword_occurrence.tsv`: occurrence of each keywords in each sections
 - `section_occurrence.tsv`: number of occurrences in each sections of any keywords. It is to find less-covered sections.
 - `no_occurence.txt`: Keywords which does not occur in the contents.
+- `too_many_occurrence.tsv`: Keywords which occur in more than 5 sections.
+- `similar_keywords.txt`: Output if there are keywords like `Neural network` and `Neural Network`.
 
 
 ## memo
@@ -17,4 +20,7 @@
 - no_occurence: Some looks mistake (e.g. `W. Mitchell Waldrop` does not occur but `M. Mitchell Waldrop` occurs), some may because of acronym in palens (e.g. `Distributed Ledger Technology (DLT)`)
 - Changed `Universal Record Locator` to `Uniform Resource Locator`, and fixed `W. Mitchell Waldrop`.
 - Fix some upper/lower diversity (e.g. `Virtual Reality` and `Virtual reality`)
-- Keywords with acronym such as `Artificial Intelligence (AI)`: If it does not occerred, remove after palens and search again.
+- Keywords with acronym such as `Artificial Intelligence (AI)`: If it does not occerred, remove after palens and search again.
+- Keywords with quotes such as `Diversity of "groups"`: remove quotes
+- `keyword_occurrence.tsv`: Output "by human" keywords and "by script" keywords on the different columns
+- Fix bug: I ignored `X` derived from `X (formerly Twitter)` but the comparison was done after lower().
diff --git a/scripts/index/contributors.tsv b/scripts/index/contributors.tsv
@@ -1,5 +1,5 @@
-cFQ	1123
+cFQ	1378
 glenweyl	1
-gnomevan	27
-nishio	156
+gnomevan	56
+nishio	155
 tsuzumik	44
diff --git a/scripts/index/ignore.txt b/scripts/index/ignore.txt
@@ -0,0 +1,11 @@
+X
+Her
+UN
+Author
+Writing
+Translation
+Signal
+Participation
+Outcome
+Output
+Administration