Unpredictable results returned

[GreenSaaSDev]

Describe the bug
We set up a Data Lake for our Non-profit client in December. The initial set-up returned all rows for the selected objects we pulled in (Account, Contact, Opportunity, Lead and a couple of custom objects). Subsequent monthly runs of the data lake failed to process and upon inspection this was due to some blocking locks. Now when the process is run manually through the State Machine, # of connections spikes and processing (sf_importprocess) of the data in S3 buckets fails due to postgres

dbinstance-ftienvalzd5x
CPU
13.54%
Status
Available
Class
db.t3.micro
Role
Instance
Current activity
58 Connections
Engine
PostgreSQL
Region & AZ
us-east-1b

To Reproduce
Steps to reproduce the behavior:

1. Go to Step Function state machine
2. Click on Execute
3. Watch as DB connections to Postgres climb to limits
4. Notice that although there are hundreds of thousands of records in Opportunity, Contact, Accounts only a limited subset are actually processed into the import tables
5. 17 "Account" "2023-05-31 22:43:19.710286" 1500 168 1421 1618927413
   18 "Lead" "2023-05-31 22:43:19.751178" 0 78 26 22194445
   19 "Opportunity" "2023-05-31 22:43:21.157252" 3400 250 1123 1777352241

Expected behavior
Expect that all rows are processed into the data lake

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

[simsek97]

I tried the same but I couldn't replicate the issue.