updating docs for vector migration by genetics changes. (#9)

Updated docs to add VECTOR_MIGRATION_BY_GENETICS information, added migration file creating scripts.

docs/csv/vector-migration-by-genetics-input.csv

@@ -0,0 +1,18 @@
+FromNodeID,ToNodeID,[],"[[""X1"",""Y2""]]","[[""a1"", ""a1""],[""b1"",""b0""], [""X1"", ""X1""]]","[[""*"", ""a0""], [""X1"", ""Y1""]]","[[""a1"", ""a1""], [""b1"", ""b1""]]"
+5,1,0.1,0,0,0,0
+5,2,0,0.1,0,0,0
+5,3,0,0,0.1,0,0
+5,4,0,0,0,0.1,0
+5,6,0,0,0,0,0.1
+5,7,0.1,0.1,0,0,0
+5,8,0.1,0,0.1,0.05,0.01
+5,9,0,0.1,0,0,0.1
+1,2,1,0,0,0,0
+1,3,0,1,0,0,0
+1,4,0,0,1,0,0
+1,6,0,0,0,1,0
+3,6,0,0,0,0,0.5
+3,7,0,0.5,0,0,0
+3,8,0.5,0,0,0.0,0.5
+3,9,0,0.5,0,0,0.0
+

docs/csv/vector-migration-by-sex-input.csv

@@ -0,0 +1,9 @@
+FromNodeID,ToNodeID,RateMales,RateFemales
+5,1,0.1,0.02
+5,2,0.1,0.02
+5,3,0.1,0.02
+5,4,0.1,0.02
+5,6,0,0.02
+5,7,0,0.02
+5,8,0.1,0
+5,9,0.1,0

docs/emod/model-migration.rst

@@ -9,7 +9,7 @@ nodes without disease and introduce disease transmission into that node. Nodes a
 can represent everything from individual households to entire countries or anything in between.
 Therefore, to include migration in a simulation, you must define multiple nodes.
 
-At each time step, individuals in each node have a defined probability of migrating out of their
+At each time step, individuals in each node have a defined rate of migration out of their
 current node to another. You can also define the average length of time individuals will stay in
 their destination node before migrating again. If you are using timesteps longer than one day and
 the time to next migration falls between timesteps, individuals will migrate at the following

docs/emod/software-migration-creation-vector.rst

@@ -0,0 +1,185 @@
+====================================
+How to create vector migration files
+====================================
+
+You can create the JSON metadata and binary migration files needed by |EMOD_s| to run simulations
+from CSV data using Python script below. You can assign the same probability of migration to
+each vector in a node or you can assign different migration rates based on gender or genetics of the vector.
+
+#.  Run the 'convert_csv_to_bin_vector_migration.py' script using the format below::
+
+        python -m emodpy_malaria.migration.convert_csv_to_bin_vector_migration [input-migration-csv] [idreference(optional)]
+
+
+.. note:: 
+
+    The **IdReference** must match the value in the demographics file. The bin.json metadata file will be created without a valid
+    **IdReference** with expectations that the user will set it themselves if that argument is not passed in.
+
+
+CSV Input Configurations
+========================
+Below are different csv file input configurations you can use to create vector migration.
+
+One rate for all vectors
+------------------------
+
+Header (optional):  FromNodeID, ToNodeID, Rate (Average # of Trips Per Day)
+If the csv/text file has three columns with no headers, this is the format we assume.
+
+.. csv-table::
+    :header: Parameter, Data type,  Min, Max, Default, Description
+    :widths: 10,5,5,5,5,20
+
+    FromNodeID, integer, 1, 2147480000, NA,"NodeID, matching NodeIDs in demographics file, from which the vector/human will travel."
+    ToNodeID, integer, 1, 2147480000, NA,"NodeID, matching NodeIDs in demographics file, to which the vector/human will travel."
+    Rate, float, 0, 3.40282e+38, NA, "Rate at which the all the vectors/humans will travel from the FromNodeID to ToNodeID."
+
+Example:
+
+.. csv-table::
+    :header: FromNodeID, ToNodeID, Rate
+    :widths: 5,5,5
+
+    5,1,0.1
+    5,2,0.1
+    5,3,0.1
+    5,4,0.1
+    5,6,0
+    5,7,0
+    5,8,0.1
+    5,9,0.1
+
+Actual csv:
+
+.. literalinclude:: ../csv/migration-input-file-simple.csv
+
+
+Different rates for male and female vectors
+-------------------------------------------
+
+Header (optional):  FromNodeID, ToNodeID, RateMales, RateFemales
+If the csv/text file has four columns with no headers, this is the format we assume.
+
+.. csv-table::
+    :header: Parameter, Data type,  Min, Max, Default, Description
+    :widths: 10,5,5,5,5,20
+
+    FromNodeID, integer, 1, 2147480000, NA, "NodeID, matching NodeIDs in demographics file, from which the vector/human will travel."
+    ToNodeID, integer, 1, 2147480000, NA,"NodeID, matching NodeIDs in demographics file, to which the vector/human will travel."
+    RateMales, float,0, 3.40282e+38, NA,  "Rate at which the vector/human of male sex will travel from the FromNodeID to ToNodeID."
+    RateFemales, float, 0, 3.40282e+38, NA, "Rate at which the vector/human of female sex will travel from the FromNodeID to ToNodeID."
+
+Example:
+
+.. csv-table::
+    :header: FromNodeID, ToNodeID, RateMales, RateFemales
+    :widths: 5,5,5,5
+
+    5,1,0.1,0.02
+    5,2,0.1,0.02
+    5,3,0.1,0.02
+    5,4,0.1,0.02
+    5,6,0,0.02
+    5,7,0,0.02
+    5,8,0.1,0
+    5,9,0.1,0
+
+Actual csv:
+
+.. literalinclude:: ../csv/vector-migration-by-sex-input.csv
+
+
+Different rates depending on genetics of the vector
+---------------------------------------------------
+
+Header (required):  FromNodeID, ToNodeID, [], arrays denoting Allele_Combinations
+Allele_Combinations example: [["a1", "a1"], ["b1", "b1"]] or  [["X1","Y2"]] or [["*", "a0"], ["X1", "Y1"]]
+Due to use of commas in headers, it is best to use Excel to create them (or look at a sample text csv).
+This is to support VECTOR_MIGRATION_BY_GENETICS. Headers are required for this csv file.
+The first (empty, []) array is used as a "default rate" if the vector's genetics doesn't match any of the
+Allele_Combinations. The other column headers denote the rate that the vector will travel at if it matches the
+Allele_Combination listed. Vectors are checked against Allele_Combinations from most-specific, to least-specific,
+regardless of the order in the csv file. Allele_Combinations can, but don't have to, include sex-alleles. Without
+specified sex-alleles, any vector that matches the alleles regardless of sex will travel at that rate. Use '*' as a
+wildcard if the second allele does not matter and can be matched with anything.
+
+.. csv-table::
+    :header: Parameter, Data type, Min, Max, Default, Description
+    :widths: 10,5,5,5,5,20
+
+    FromNodeID, integer, 1, 2147480000, NA, "NodeID, matching NodeIDs in demographics file, from which the vector/human will travel."
+    ToNodeID, integer, 1, 2147480000, NA, "NodeID, matching NodeIDs in demographics file, to which the vector/human will travel."
+    [], float, 0, 3.40282e+38, NA, "Default rate at which the vector that doesn't match any other allele combinations will travel from the FromNodeID to ToNodeID."
+    User-defined Allele Combination, float, 0, 3.40282e+38, NA, "Rate at which the vector that matches this and not a more-specific allele combination will travel from the FromNodeID to ToNodeID."
+
+Example:
+
+.. csv-table::
+    :header: FromNodeID, ToNodeID, [], "[['a1', 'a1'], ['b1', 'b1']]", "[['*', 'a0'], ['X1', 'Y1']]", "[['X1','Y2']]"
+    :widths: 5,5,5,5,5,5
+
+    5,1,0.1,0,0,0
+    5,2,0,0.1,0,0
+    5,3,0,0,0.1,0
+    5,4,0,0,0,0.1
+    5,6,0,0,0,0
+    5,7,0.1,0.1,0,0
+    5,8,0.1,0,0.1,0.05
+    5,9,0,0.1,0,0
+    1,2,1,0,0,0
+    1,3,0,1,0,0
+    1,4,0,0,1,0
+    1,6,0,0,0,1
+    3,6,0,0,0,0
+    3,7,0,0.5,0,0
+    3,8,0.5,0,0,0.0
+    3,9,0,0.5,0,0
+
+
+Actual csv:
+
+.. literalinclude:: ../csv/vector-migration-by-genetics-input.csv
+
+
+Migration binary file
+=====================
+
+For information, see :ref:`binary_migration_file`.
+
+
+JSON metadata file
+==================
+
+The metadata file is a JSON-formatted file that includes a metadata section and a node offsets
+section. The **Metadata** section contains a JSON object with parameters that help |EMOD_s| interpret the migration
+binary file. The users are encouraged to add their own parameters to the section to remind themselves about the source,
+reason, purpose of the binary file and the data it contains. Non-required parameters are ignored.
+
+
+Vector Migration Metadata File Parameters
+------------------------------------------
+
+.. csv-table::
+    :header: Parameter, Data type, Description
+    :widths: 10,5,20
+
+    IdReference, string, "Required. A unique id to match demographics, climate, and migration files that work together."
+    DatavalueCount, integer, "Required.The number of outbound data values per node (max 100). The number must be the same across every node in the binary file."
+    GenderDataType, enum, "Required. Denotes whether data is provided for each gender separately, is the same for both, or depends on vector genetics. Accepted values are ONE_FOR_BOTH_GENDERS, ONE_FOR_EACH_GENDER, VECTOR_MIGRATION_BY_GENETICS."
+    AlleleCombinations, array, "Required for GenderDataType: VECTOR_MIGRATION_BY_GENETICS. An array of Allele_Combinations, starting with an emtpy array to mark the default migration rate."
+    NodeCount, integer, "Required. The number of 'from' nodes in the data. Used to verify size NodeOffsets - 16*NodeCount = # chars in NodeOffsets."
+    NodeOffsets, string, "Required. The number of rates/'to' nodes for each 'from' node. Max of 100."
+    DateCreated , string, (Informational for user only) Date and time the file was generated by the script.
+    Tool, string, (Informational for user only) The script used to create the file.
+    User-created parameter, string, (Informational for user only) Example of a user-created parameter
+
+
+Example
+-------
+
+.. literalinclude:: ../json/vector-migration-metadata.json
+   :language: json
+
+
+

docs/emod/software-migration-creation.rst

@@ -41,9 +41,9 @@ To use the same average migration rate for every individual in a node, create th
     Rate
         The average number of trips per day.
 
-#.  Run the `convert_txt_to_bin.py <https://github.com/InstituteforDiseaseModeling/EMOD/blob/master/Scripts/MigrationTools/convert_txt_to_bin.py>`_ script using the command format below::
+#.  Run the 'convert_txt_to_bin.py' script using the command format below::
 
-        python convert_txt_to_bin.py [input-migration-csv] [output-bin] [migration-type] [idreference]
+        python -m emodpy_malaria.migration.convert_txt_to_bin [input-migration-csv] [output-bin] [migration-type] [idreference]
 
 
 This will create both the metadata and binary file needed by |EMOD_s|. 
@@ -61,9 +61,9 @@ To vary the average migration rate based on age and/or gender, create the migrat
 
 #.  Create a JSON file with the structure described in the sections below.
 
-#.  Run the `convert_json_to_bin.py <https://github.com/InstituteforDiseaseModeling/EMOD/blob/master/Scripts/MigrationTools/convert_json_to_bin.py>`_ script using the command format below::
+#.  Run the 'convert_json_to_bin.py' script using the command format below::
 
-        python convert_json_to_bin.py [input-json] [output-bin] [migration-type]
+        python -m emodpy_malaria.migration.convert_json_to_bin [input-json] [output-bin] [migration-type]
 
 This will create both the metadata and binary file needed by |EMOD_s|. 
 

docs/emod/software-migration-vector.rst

@@ -1,39 +1,53 @@
-======================
-Vector migration files
-======================
+=================
+Vector Migration
+=================
 
 Vector migration file describes the rate of migration of vectors *out* of a geographic :term:`node`
-analogously to human migration (see :doc:`software-migration` for more information), although vector
-migration supports gender parameters, it does not support migration by age and age-based migration 
-in the migration file will cause an error. Vector migration is one way, such that each trip made by
-a vector is independent of previous trips made by the vector. For **Vector_Sampling_Type** set to 
-"TRACK_ALL_VECTORS" or "SAMPLE_IND_VECTORS", the rates in the file are used to control whether or not 
-a female vector will migrate: the rate specified is used to get a "time to leave on trip" value
-from an exponential distribution. If the value is less than one day, then the female vector will migrate.
-For male vectors (who are always in cohorts/compartments), and for female vectors when 
-**Vector_Sampling_Type** is set to "VECTOR_COMPARTMENTS_NUMBER" or "VECTOR_COMPARTMENTS_PERCENT", 
-the rates in the file are used to calculate what fraction of the population is traveling out of the node
-on that day based on a total rate of travel out of that node and the traveling vectors are distributed 
-to their destination nodes in proportion of rates to those nodes to the total outbound rate. 
-
-If default geography is used (the configuration parameter **Enable_Demographics_Builtin** is set to 1, 
+analogously to human migration (see :doc:`software-migration` for more information), the vector
+model does not support migration by age and age-based migration in the migration file will cause an
+error. Vector migration does support migration by gender as well as migration based on genetics (see below).
+Vector migration is one way, such that each trip made by a vector is independent of previous trips made
+by the vector.
+
+Note: If default geography is used (the configuration parameter **Enable_Demographics_Builtin** is set to 1,
 and **Default_Geography_Initial_Node_Population** and **Default_Geography_Torus_Size** are configured), 
 vector migration will be built internally and vectors will automatically migrate. It is a known issue,
 please see https://github.com/InstituteforDiseaseModeling/EMOD/issues/43
 
-Vectors "LOCAL_MIGRATION" MigrationType for all their migration needs, but are not limited to the default 
-maximum data values of 8 (destinations).
+Vectors do not have a "MigrationType" as each species use only one file for all their migration needs.
 
 Each vector species has its own **Vector_Migration_Filename**, if it is left as an empty string, no 
 migration will happen for that species. The **Vector_Migration_Modifier_Equation** and its parameters 
 can influence female vector migration to particular nodes over others, while **x_Vector_Migration** is
 a multiplier affects the migration rates for both genders. See :doc:`parameter-configuration` for more
 information on the parameters governing vector migration.
 
-Migration Files
-==================
+Vector Migration Using Genetics
+===============================
+
+Vectors have a type of migration not available to humans set with "GenderDataType": VECTOR_MIGRATION_BY_GENETICS in the
+migration metadata file (usually a .bin.json file).
+
+The way this migration works is that you can define a migration rate for each defined Allele_Combination.
+Allele_Combinations example: [ [], [["a1", "a1"], ["b1", "b1"]],  [["X1","Y2"]], [["*", "a0"], ["X1", "Y1"]]]
+The first (empty, []) array is used as a "default rate" if the vector's genetics doesn't match any of the
+Allele_Combinations. The other column Allele_Combinations will be associated with the rate that the vector will use
+if it matches with that Allele_Combination. Vectors are checked against Allele_Combinations from most-specific,
+to least-specific, regardless of the order in the Allele_Combinations parameter. Allele_Combinations can,
+but don't have to, include sex-alleles. Without specified sex-alleles, any vector that matches the alleles regardless
+of sex will travel at that rate. Please see Vector Migration File creation for more information.
+
+
+Vector Migration Files
+======================
+
+The Binary file structure for the vector migration files is the same as it is for human files.
+The Allele_Combinations array, when present for vector VECTOR_MIGRATION_BY_GENETICS, is used in the same capacity as
+AgesYears array would be to maintain the same structure of the file. Please see migration file creation
+for more details.
 
-Vectors use the same migration files as humans, with two caveats:
+.. toctree::
+   :maxdepth: 3
+   :titlesonly:
 
-1. Vectors do not migrate by-age, so multiple AgeBins in the by-age by-gender migration file will cause an error.
-2. Vector migration only uses MigrationType "LOCAL_MIGRATION" all other migration types will cause an error.
+   software-migration-creation-vector

docs/emod/software-migration.rst

@@ -98,6 +98,9 @@ Example
 .. literalinclude:: ../json/migration-metadata.json
    :language: json
 
+
+.. _binary_migration_file:
+
 Binary file
 ===========
 
@@ -118,5 +121,6 @@ The following image shows how a binary file with a **DatavalueCount** value of 8
    :maxdepth: 3
    :titlesonly:
 
-   software-migration-vector
    software-migration-creation
+   software-migration-vector
+

docs/json/vector-migration-metadata.json

@@ -0,0 +1,13 @@
+{
+    "Metadata": {
+        "IdReference": "9-nodes",
+        "DateCreated": "Thu Nov 21 17:41:47 2024",
+        "Tool": "convert_csv_to_bin_vector_migration.py",
+        "DatavalueCount": 8,
+        "GenderDataType": "VECTOR_MIGRATION_BY_GENETICS",
+        "AlleleCombinations": [[], [["X1","Y2"]], [["a1","a1"], ["b1","b0"], ["X1","X1"]], [["*","a0"], ["X1","Y1"]], [["a1","a1"],["b1","b1"]]],
+        "NodeCount": 3,
+        "Project": "Migration based on Dr. Acula research."
+    },
+    "NodeOffsets": "0000000500000000000000010000006000000003000000C0"
+}

emodpy_malaria/migration/README.md

@@ -0,0 +1,10 @@
+# Migration
+
+This submodule provides scripts for creating and reading vector migration input files that are directly ingested by 
+EMOD for determining how individuals migrate between nodes over time during a simulation. Going forward, all reading 
+and writing of these migration control files should be done via this submodule.
+
+The script files convert from human-readable files (txt, json, csv) to binary migration files. Please see documentation
+for more details. 
+
+