Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create scan to sync WAS finding data between environments #798

Merged
merged 24 commits into from
Feb 20, 2025

Conversation

DJensen94
Copy link
Contributor

🗣 Description

This PR creates a scan that pulls WAS findings from the scanning environment and saves it to the LZ mini data lake.

💭 Motivation and context

This allows us to keep the commercial and gov cloud environments WAS finding data in sync and will allow the Cyhy Dashboard to query these findings in a more robust manner, instead of being forced into the current generic crossfeed vulnerability.

🧪 Testing

Tested locally against the live API endpoint.

✅ Pre-approval checklist

  • This PR has an informative and human-readable title.
  • Changes are limited to a single goal - eschew scope creep!
  • All future TODOs are captured in issues, which are referenced
    in code comments.
  • All relevant type-of-change labels have been added.
  • I have read the CONTRIBUTING document.
  • These code changes follow cisagov code standards.

✅ Pre-merge checklist

  • Revert dependencies to default branches.
  • Finalize version.

Add shodan and xpanse mdl syncs to XFD
Create scan that pulls credential breaches from the commercial mdl
pin pytz version
Run linter
remove unnecessary else statements
remove pytz library
fix data_source reference, that was accessing the wrong dictionary
pylint fixes
final lint changes
Clean up comments
update scan list in the frontend
Fix failing timezone call, which was calling from the wrong datetime object
Add logging for failed runs
Remove test break statements and add additional response handling
Add WAS dmz sync
Merge scan sync branch into was Sync branch
run linter on WAS sync
remove unused model imports
lint on unrelated files
update date function to allow None return
update scan to pull more often
@DJensen94 DJensen94 marked this pull request as ready for review February 19, 2025 21:06
Copy link
Collaborator

@cduhn17 cduhn17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comments regarding repeated code

remove duplicate code for headers and date calculations
run linter and update reused variable name
@cduhn17 cduhn17 merged commit 6384ce1 into develop Feb 20, 2025
18 of 19 checks passed
@cduhn17 cduhn17 deleted the DJ_Was_mdl_sync_scan_WIP branch February 20, 2025 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants