Merge branch 'master' into no-overlap

README.md

@@ -11,7 +11,7 @@ Need help? Got something working? Share it!
 
 [Discord Server](Https://discord.gg/trunk-recorder) - and don't forget the [Wiki](https://github.com/robotastic/trunk-recorder/wiki)
 
-Trunk Recorder is able to record the calls on trunked and conventional radio systems. It uses 1 or more Software Defined Radios (SDRs) to do this. The SDRs capture large swathes of RF and then use software to process what was received. [GNURadio](https://gnuradio.org/) is used to do this processing because it provides lots of convenient RF blocks that can be pieced together to allow for complex RF processing. The libraries from the amazing [OP25](http://op25.osmocom.org/trac/wiki) project are used for a lot of the P25 functionality. Multiple radio systems can be recorded at the same time.
+Trunk Recorder is able to record the calls on trunked and conventional radio systems. It uses 1 or more Software Defined Radios (SDRs) to do this. The SDRs capture large swathes of RF and then use software to process what was received. [GNU Radio](https://gnuradio.org/) is used to do this processing because it provides lots of convenient RF blocks that can be pieced together to allow for complex RF processing. The libraries from the amazing [OP25](http://op25.osmocom.org/trac/wiki) project are used for a lot of the P25 functionality. Multiple radio systems can be recorded at the same time.
 
 
 Trunk Recorder currently supports the following:

docs/CONFIGURE.md

@@ -172,7 +172,7 @@ Here is a map of the different sections of the *config.json* file:
 | broadcastifySystemId   |          |                                | number                                                       | [*if broadcastifyCallsServer is set*] System ID for Broadcastify Calls <br />(this is an integer, and different from the RadioReference system ID) |
 | uploadScript           |          |                                | string                                                       | This is the filename of a script that is called after each recording has finished. Checkout *encode-upload.sh.sample* as an example. The script should be located in the same directory as the trunk-recorder executable. |
 | compressWav            |          | true                           | bool                                                         | Convert the recorded .wav file to an .m4a file. **This is required for both OpenMHz and Broadcastify!** The `sox` and `fdkaac` packages need to be installed for this command to work. |
-| unitScript             |          |                                | string                                                       | This is the filename of a script that runs when a radio (unit) registers (is turned on), affiliates (joins a talk group), deregisters (is turned off), sends an acknowledgment response or transmits. Passed as parameters:  `shortName radioID on\|join\|off\|ackresp\|call`. On joins and transmissions, `talkgroup` is passed as a fourth parameter.  On joins and transmissions, `patchedTalkgroups`  (comma separated list of talkgroup IDs) is passed as a fifth parameter if the talkgroup is part of a patch on the system. See *examples/unit-script.sh* for a logging example. Note that for paths relative to recorder, this should start with `./`( or `../`). |
+| unitScript             |          |                                | string                                                       | This is the filename of a script that runs when a radio (unit) registers (is turned on), affiliates (joins a talk group), deregisters (is turned off), gets an acknowledgment response, transmits, gets a data channel grant, a unit-unit answer request or a Location Registration Response. Passed as parameters:  `shortName radioID on\|join\|off\|ackresp\|call\|data\|ans_req\|location`. On joins and transmissions, `talkgroup` is passed as a fourth parameter; on answer requests, the `source` is.  On joins and transmissions, `patchedTalkgroups`  (comma separated list of talkgroup IDs) is passed as a fifth parameter if the talkgroup is part of a patch on the system. See *examples/unit-script.sh* for a logging example. Note that for paths relative to recorder, this should start with `./`( or `../`). |
 | audioArchive           |          | true                           | **true** / **false**                                         | Should the recorded audio files be kept after successfully uploading them? |
 | transmissionArchive    |          | false                          | **true** / **false**                                         | Should each of the individual transmission be kept? These transmission are combined together with other recent ones to form a single call. |
 | callLog                |          | false                          | **true** / **false**                                         | Should a json file with the call details be kept after successful uploads? |
@@ -183,7 +183,8 @@ Here is a map of the different sections of the *config.json* file:
 | recordUnknown          |          | true                           | **true** / **false**                                         | Record talkgroups if they are not listed in the Talkgroups File. |
 | hideEncrypted          |          | false                          | **true** / **false**                                         | Hide encrypted talkgroups log entries                        |
 | hideUnknownTalkgroups  |          | false                          | **true** / **false**                                         | Hide unknown talkgroups log entries                          |
-| minDuration            |          | 0<br />(which is disabled)     | number                                                       | The minimum call (transmission) duration in seconds (decimals allowed), calls below this number will have recordings deleted and will not be uploaded. |
+| minDuration            |          | 0<br />(which is disabled)     | number                                                       | The minimum call duration in seconds (decimals allowed), calls below this number will have recordings deleted and will not be uploaded. |
+| minTransmissionDuration|          | 0<br />(which is disabled)     | number                                                       | The minimum transmission duration in seconds (decimals allowed), transmissions below this number will not be added to their corresponding call. |
 | talkgroupDisplayFormat |          | "id"                           | **"id" "id_tag"** or **"tag_id"**                            | The display format for talkgroups in the console and log file. (*id_tag* and *tag_id* is only valid if **talkgroupsFile** is specified) |
 | bandplan               |          | "800_standard"                 | **"800_standard" "800_reband" "800_splinter"** or **"400_custom"** | [SmartNet only] this is the SmartNet bandplan that will be used. |
 | bandplanBase           |          |                                | number                                                       | [SmartNet, 400_custom only] this is for the *400_custom* bandplan only. This is the base frequency, specified in Hz. |
@@ -280,7 +281,7 @@ This example will stream audio from talkgroup 58914 on system "CountyTrunked" to
             "address":"127.0.0.1",
             "port":9123,
             "sendTGID":false,
-	    "shortName":"CountyTrunked"}
+            "shortName":"CountyTrunked"}
         }
 ```
 
@@ -295,12 +296,12 @@ This example will stream audio from talkgroup 58914 from System CountyTrunked to
             "address":"127.0.0.1",
             "port":9123,
             "sendTGID":false,
-	    "shortName":"CountyTrunked"},
+            "shortName":"CountyTrunked"},
            {"TGID":58916,
             "address":"127.0.0.1",
             "port":9124,
             "sendTGID":false,
-	    "shortName":"StateTrunked"}
+            "shortName":"StateTrunked"}
           ]}
         }
 ```
@@ -334,7 +335,7 @@ This example will stream audio from all talkgroups being recorded on System Coun
             "address":"127.0.0.1",
             "port":9123,
             "sendTGID":true,
-	    "shortName":"CountyTrunked"}
+            "shortName":"CountyTrunked"}
         }
 ```
 ##### Example - Sending Audio to pulseaudio
@@ -358,8 +359,8 @@ The matching simplestream config to send audio from talkgroup 58918 to TCP port
             "address":"127.0.0.1",
             "port":9125,
             "sendTGID":true,
-	    "shortName":"CountyTrunked",
-	    "useTCP":true}
+            "shortName":"CountyTrunked",
+            "useTCP":true}
         }
 ```
 

lib/gr_blocks/transmission_sink.cc

@@ -252,6 +252,7 @@ void transmission_sink::end_transmission() {
     strcpy(transmission.filename, current_filename); // Copy the filename
     strcpy(transmission.base_filename, current_base_filename);
     this->add_transmission(transmission);
+
     d_sample_count = 0;
     d_error_count = 0;
     d_spike_count = 0;

plugins/unit_script/unit_script.cc

@@ -27,7 +27,7 @@ int unit_registration(System *sys, long source_id) {
   std::string system_script = get_system_script(sys->get_short_name());
   if ((system_script != "") && (source_id != 0)) {
     char shell_command[200];
-    sprintf(shell_command, "./%s %s %li on &", system_script.c_str(), sys->get_short_name().c_str(), source_id);
+    sprintf(shell_command, "%s %s %li on &", system_script.c_str(), sys->get_short_name().c_str(), source_id);
     int rc =  system(shell_command);
     return 0;
   }
@@ -39,7 +39,7 @@ int unit_deregistration(System *sys, long source_id) {
   std::string system_script = get_system_script(sys->get_short_name());
   if ((system_script != "") && (source_id != 0)) {
     char shell_command[200];
-    sprintf(shell_command, "./%s %s %li off &", system_script.c_str(), sys->get_short_name().c_str(), source_id);
+    sprintf(shell_command, "%s %s %li off &", system_script.c_str(), sys->get_short_name().c_str(), source_id);
     int rc =  system(shell_command);
     return 0;
   }
@@ -49,16 +49,56 @@ int unit_acknowledge_response(System *sys, long source_id) {
   std::string system_script = get_system_script(sys->get_short_name());
   if ((system_script != "") && (source_id != 0)) {
     char shell_command[200];
-    sprintf(shell_command, "./%s %s %li ackresp &", system_script.c_str(), sys->get_short_name().c_str(), source_id);
+    sprintf(shell_command, "%s %s %li ackresp &", system_script.c_str(), sys->get_short_name().c_str(), source_id);
     int rc =  system(shell_command);
     return 0;
   }
     return 1;
 }
 
+int unit_group_affiliation(System *sys, long source_id, long talkgroup_num) {
+    unit_affiliations[source_id] = talkgroup_num;
+    std::string system_script = get_system_script(sys->get_short_name());
+  if ((system_script != "") && (source_id != 0)) {
+    char shell_command[200];
+    std::vector<unsigned long> talkgroup_patches = sys->get_talkgroup_patch(talkgroup_num);
+    std::string patch_string;
+    bool first = true;
+    BOOST_FOREACH (auto& TGID, talkgroup_patches) {
+      if (!first) { patch_string += ","; }
+      first = false;
+      patch_string += std::to_string(TGID);
+    }
+    sprintf(shell_command, "%s %s %li join %li %s &", system_script.c_str(), sys->get_short_name().c_str(), source_id, talkgroup_num, patch_string.c_str());
+    int rc =  system(shell_command);
+    return 0;
+  }
+    return 1;
+}
 
+int unit_data_grant(System *sys, long source_id) {
+    std::string system_script = get_system_script(sys->get_short_name());
+  if ((system_script != "") && (source_id != 0)) {
+    char shell_command[200];
+    sprintf(shell_command, "%s %s %li data &", system_script.c_str(), sys->get_short_name().c_str(), source_id);
+    int rc =  system(shell_command);
+    return 0;
+  }
+    return 1;
+}
 
-int unit_group_affiliation(System *sys, long source_id, long talkgroup_num) {
+int unit_answer_request(System *sys, long source_id, long talkgroup) {
+    std::string system_script = get_system_script(sys->get_short_name());
+  if ((system_script != "") && (source_id != 0)) {
+    char shell_command[200];
+    sprintf(shell_command, "%s %s %li ans_req %li &", system_script.c_str(), sys->get_short_name().c_str(), source_id, talkgroup);
+    int rc =  system(shell_command);
+    return 0;
+  }
+    return 1;
+}
+
+int unit_location(System *sys, long source_id, long talkgroup_num) {
     unit_affiliations[source_id] = talkgroup_num;
     std::string system_script = get_system_script(sys->get_short_name());
   if ((system_script != "") && (source_id != 0)) {
@@ -71,7 +111,7 @@ int unit_group_affiliation(System *sys, long source_id, long talkgroup_num) {
       first = false;
       patch_string += std::to_string(TGID);
     }
-    sprintf(shell_command, "./%s %s %li join %li %s &", system_script.c_str(), sys->get_short_name().c_str(), source_id, talkgroup_num, patch_string.c_str());
+    sprintf(shell_command, "%s %s %li location %li %s &", system_script.c_str(), sys->get_short_name().c_str(), source_id, talkgroup_num, patch_string.c_str());
     int rc =  system(shell_command);
     return 0;
   }
@@ -93,7 +133,7 @@ int call_start(Call *call) {
       first = false;
       patch_string += std::to_string(TGID);
     }
-    sprintf(shell_command, "./%s %s %li call %li %s &", system_script.c_str(), short_name.c_str(), source_id, talkgroup_num, patch_string.c_str());
+    sprintf(shell_command, "%s %s %li call %li %s &", system_script.c_str(), short_name.c_str(), source_id, talkgroup_num, patch_string.c_str());
     int rc =  system(shell_command);
     return 0;
   }

trunk-recorder/call_concluder/call_concluder.cc

@@ -32,7 +32,7 @@ int combine_wav(std::string files, char *target_filename) {
 int convert_media(char *filename, char *converted) {
   char shell_command[400];
 
-  int nchars = snprintf(shell_command, 400, "sox %s --norm -t wav - | fdkaac --silent  -p 2 --ignorelength -b 8000 -o %s -", filename, converted);
+  int nchars = snprintf(shell_command, 400, "sox %s --norm=-.01 -t wav - | fdkaac --silent  -p 2 --ignorelength -b 8000 -o %s -", filename, converted);
 
   if (nchars >= 400) {
     BOOST_LOG_TRIVIAL(error) << "Call uploader: Command longer than 400 characters";
@@ -277,6 +277,21 @@ Call_Data_t Call_Concluder::create_call_data(Call *call, System *sys, Config con
     for (std::vector<Transmission>::iterator it = call_info.transmission_list.begin(); it != call_info.transmission_list.end(); ++it) {
       Transmission t = *it;
       char formattedTalkgroup[62];
+
+      if(t.length < sys->get_min_tx_duration())
+      {
+        if (!call_info.transmission_archive) {
+
+          snprintf(formattedTalkgroup, 61, "%c[%dm%10ld%c[0m", 0x1B, 35, call_info.talkgroup, 0x1B);
+          std::string talkgroup_display = boost::lexical_cast<std::string>(formattedTalkgroup);
+          BOOST_LOG_TRIVIAL(info) << "[" << call_info.short_name << "]\t\033[0;34m" << call_info.call_num<< "C\033[0m\tTG: " << talkgroup_display << "\tFreq: " << format_freq(call_info.freq) << "\tRemoving transmission less than " << sys->get_min_tx_duration() <<" seconds. Actual length: " << t.length << "." << std::endl;
+
+          if (checkIfFile(t.filename)) {
+            remove(t.filename);
+          }
+        }
+        continue;
+      } 
       snprintf(formattedTalkgroup, 61, "%c[%dm%10ld%c[0m", 0x1B, 35, call_info.talkgroup, 0x1B);
       std::string talkgroup_display = boost::lexical_cast<std::string>(formattedTalkgroup);
       BOOST_LOG_TRIVIAL(info) << "[" << call_info.short_name << "]\t\033[0;34m" << call_info.call_num << "C\033[0m\tTG: " << talkgroup_display << "\tFreq: " << format_freq(call_info.freq) << "\t- Transmission src: " << t.source << " pos: " << total_length << " length: " << t.length;

trunk-recorder/main.cc

@@ -261,12 +261,11 @@ bool load_config(string config_file) {
         BOOST_LOG_TRIVIAL(info) << "Control Channels: ";
         BOOST_FOREACH (boost::property_tree::ptree::value_type &sub_node, node.second.get_child("control_channels")) {
           double control_channel = sub_node.second.get<double>("", 0);
-
           BOOST_LOG_TRIVIAL(info) << "  " << format_freq(control_channel);
-          system->add_control_channel(control_channel);
-          system->set_talkgroups_file(node.second.get<std::string>("talkgroupsFile", ""));
-          BOOST_LOG_TRIVIAL(info) << "Talkgroups File: " << system->get_talkgroups_file();
+          system->add_control_channel(control_channel); 
         }
+        system->set_talkgroups_file(node.second.get<std::string>("talkgroupsFile", ""));
+        BOOST_LOG_TRIVIAL(info) << "Talkgroups File: " << system->get_talkgroups_file();
       } else {
         BOOST_LOG_TRIVIAL(error) << "System Type in config.json not recognized";
         return false;
@@ -382,6 +381,8 @@ bool load_config(string config_file) {
       BOOST_LOG_TRIVIAL(info) << "Minimum Call Duration (in seconds): " << system->get_min_duration();
       system->set_max_duration(node.second.get<double>("maxDuration", 0));
       BOOST_LOG_TRIVIAL(info) << "Maximum Call Duration (in seconds): " << system->get_max_duration();
+      system->set_min_tx_duration(node.second.get<double>("minTransmissionDuration", 0));
+      BOOST_LOG_TRIVIAL(info) << "Minimum Transmission Duration (in seconds): " << system->get_min_tx_duration();
 
 
       if (!system->get_compress_wav()) {
@@ -941,8 +942,19 @@ void unit_group_affiliation(System *sys, long source_id, long talkgroup_num) {
   plugman_unit_group_affiliation(sys, source_id, talkgroup_num);
 }
 
+void unit_data_grant(System *sys, long source_id) {
+  plugman_unit_data_grant(sys, source_id);
+}
+
+void unit_answer_request(System *sys, long source_id, long talkgroup) {
+  plugman_unit_answer_request(sys, source_id, talkgroup);
+}
+
+void unit_location(System *sys, long source_id, long talkgroup_num) {
+  plugman_unit_location(sys, source_id, talkgroup_num);
+}
 
-void handle_call_grant(TrunkMessage message, System *sys) {
+void handle_call(TrunkMessage message, System *sys) {
   bool call_found = false;
   bool call_retune = false;
   bool recording_started = false;
@@ -1119,7 +1131,10 @@ void handle_message(std::vector<TrunkMessage> messages, System *sys) {
       current_system_status(message, sys);
       break;
 
-    case LOCATION:        // currently not handling, TODO: expand plugin system to handle this
+    case LOCATION:
+      unit_location( sys, message.source, message.talkgroup);
+      break;
+
     case ACKNOWLEDGE:
       unit_acknowledge_response( sys, message.source);
       break;
@@ -1130,6 +1145,15 @@ void handle_message(std::vector<TrunkMessage> messages, System *sys) {
     case PATCH_DELETE:
       sys->delete_talkgroup_patch(message.patch_data);
       break;
+
+    case DATA_GRANT:
+      unit_data_grant(sys, message.source);
+      break;
+
+    case UU_ANS_REQ:
+      unit_answer_request(sys, message.source, message.talkgroup);
+      break;
+
     case UNKNOWN:
       break;
     }
@@ -1270,7 +1294,23 @@ void monitor_messages() {
   while (1) {
 
     if (exit_flag) { // my action when signal set it 1
-      printf("\n Signal caught!\n");
+        BOOST_LOG_TRIVIAL(info) << "Caught Exit Signal...";
+        for (vector<Call *>::iterator it = calls.begin(); it != calls.end();) {
+         Call *call = *it;
+
+        if (call->get_state() != MONITORING) {
+          call->set_state(COMPLETED);
+          call->conclude_call();
+        }
+
+         it = calls.erase(it);
+         delete call;
+       }
+
+       BOOST_LOG_TRIVIAL(info) << "Cleaning up & Exiting...";
+
+       // Sleep for 5 seconds to allow for all of the Call Concluder threads to finish.
+       boost::this_thread::sleep(boost::posix_time::milliseconds(5000));
       return;
     }
 
@@ -1299,6 +1339,10 @@ void monitor_messages() {
         }
       }
 
+      if (msg->type() == -1) {
+        BOOST_LOG_TRIVIAL(error) << "[" << sys->get_short_name() << "]\t process_data_unit timeout";
+      }
+
       msg.reset();
     } else {
       current_time = time(NULL);
@@ -1563,7 +1607,7 @@ int main(int argc, char **argv) {
     logging::add_file_log(
         keywords::file_name = config.log_dir + "/%m-%d-%Y_%H%M_%2N.log",
         keywords::format = "[%TimeStamp%] (%Severity%)   %Message%",
-        keywords::rotation_size = 10 * 1024 * 1024,
+        keywords::rotation_size = 100 * 1024 * 1024,
         keywords::time_based_rotation = sinks::file::rotation_at_time_point(0, 0, 0),
         keywords::auto_flush = true);
   }