Add flag/config for "lastModifiedWithin"

[harrisonSoS]

It'd be nice to have a config item to only pilfer files e.g last modified in the last 4 years, which will help reduce noise of creds that have since been rolled.

It could very well already exist but don't see such an option in the README/sample config.

[l0ss]

last modified date is already being collected - can you not do your filtering on the log file?

[harrisonSoS]

Are there efficiencies to be made in pulling file metadata, then only proceeding to pull / parse / pilfer if the file was last modified within the last X?

[l0ss]

yeah actually there would be.

If you want to take a swing at implementing this, here's roughly what you'd need to do:

Add a thing to handle a ClassifierRule.MatchLocation using the modified date into:
https://github.com/SnaffCon/Snaffler/blob/master/SnaffCore/Classifiers/FileClassifier.cs

Then (assuming you want to be able to pass the date as an argument) you'd need to add a thing to parse that arg, generate a discard rule using that modified date, and insert it into the ruleset at runtime.

If you don't wanna try it yourself, I'll probably get to it eventually, but probably not soon.

[mepher]

As a horrible hack, add / update / overwrite / merge this horror into SnaffCore TreeWalker.cs around line 38:

                string[] files = Directory.GetFiles(currentDir);
                DateTime oneYearAgo = DateTime.Now.AddYears(-1);
                // check if we actually like the files
                foreach (string file in files)
                {
                    FileInfo fileInfo = new FileInfo(file);
                    DateTime lastModified = fileInfo.LastWriteTime;
                    if (lastModified >= oneYearAgo)
                    {
                        FileTaskScheduler.New(() =>
                        {
                            try
                            {
                                FileScanner.ScanFile(file);
                            }
                            catch (Exception e)
                            {
                                Mq.Error("Exception in FileScanner task for file " + file);
                                Mq.Trace(e.ToString());
                            }
                        });
                    }
                    // the unsaid "else, move along. "
                    // 

[BlueFootedBird]

I created a hack/workaround for implementing a timeframe so that Snaffler only reports files within a specified window of time. I originally was filtering through the data manually in logs, but figured that since so much data was being sent that I didn't want, it would be more OPSEC and efficient by transmitting only the files that we care about. The sample code shown below was a modification to "SnaffleRunner.cs" in the ProcessMessage() function. The rest just requires minor tweaks to Config.cs and Options.cs.

case SnafflerMessageType.FileResult:
...
if (Options.TimeFrame != null)
{
   String[] dateParts = (Options.TimeFrame).split(',');
   DateTime after;
   DateTime before;
   if (DateTime.TryParse(dataParts[0], out after) && DateTime.TryParse(dataParts[1], out before))
   {
      DateTime modifiedStamp = message.FileResult.FileInfo.LastWriteTime.ToUniversalTime();
      if (after > modifiedStamp || modifiedStamp > before)
      {
         break;
      }
   }
   else
   {
      Console.WriteLine("Check your date format. Exiting...");
      Environment.Exit(1);
   }
}
...