Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: stock balance report #35348

Merged

Conversation

rohitwaghchaure
Copy link
Collaborator

@rohitwaghchaure rohitwaghchaure commented May 17, 2023

Why refactor?

For the large db the stock balance report was not generating even though prepared report and the state was remaining in the Queued state and it was never completing

image

Investigation

During investigation, we found that if the user has not set item or warehouse filters then this issue happens. System fetch all the stock ledger entries to calculate the Opening Quantity and Opening Stock Value. If the number of stock ledger entries are more than 1 million and multiple users try to access the stock balance report without filters then RAM gets consumed fully and get the error "Work-horse was terminated unexpectedly (waitpid returned 15)".

New Changes

  1. "Closing Stock Balance"

Added new doctype "Closing Stock Balance" to avoid calculation of Opening Quantity and Opening Stock Value. If you have closed the period then create the Closing Stock Balance for that period so that while preparing the stock balance report system will use the data of Closing Stock Balance and avoid Opening Quantity and Opening Stock calculations.

Screenshot 2023-05-22 at 4 06 18 PM
  1. Changed IN clause with JOIN

While watching process log we have found that one query is taking time and that was item_code in () (see below image).
The IN operator works slowly if the number of data is huge and it's better to use join query
Screenshot 2023-05-22 at 4 25 51 PM

Profiling Result

Before Change

Screenshot 2023-05-23 at 1 29 52 PM

After Change

Screenshot 2023-05-23 at 1 28 23 PM

@rohitwaghchaure rohitwaghchaure requested a review from s-aga-r as a code owner May 17, 2023 18:22
@rohitwaghchaure rohitwaghchaure marked this pull request as draft May 17, 2023 18:22
@rohitwaghchaure rohitwaghchaure force-pushed the refactor-stock-balance-report branch 5 times, most recently from 0193cf0 to cfb835d Compare May 22, 2023 11:49
@codecov
Copy link

codecov bot commented May 22, 2023

Codecov Report

Merging #35348 (3f548ac) into develop (3940deb) will decrease coverage by 0.05%.
The diff coverage is 65.30%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop   #35348      +/-   ##
===========================================
- Coverage    63.82%   63.77%   -0.05%     
===========================================
  Files          813      814       +1     
  Lines        60260    60456     +196     
===========================================
+ Hits         38460    38556      +96     
- Misses       21800    21900     +100     
Impacted Files Coverage Δ
...ype/closing_stock_balance/closing_stock_balance.py 0.00% <0.00%> (ø)
...rpnext/stock/report/stock_balance/stock_balance.py 79.92% <77.11%> (-4.38%) ⬇️
...xt/stock/report/stock_analytics/stock_analytics.py 91.92% <87.50%> (-1.99%) ⬇️
...value/warehouse_wise_item_balance_age_and_value.py 91.96% <91.17%> (-0.63%) ⬇️
erpnext/stock/report/stock_ageing/stock_ageing.py 90.95% <100.00%> (ø)

... and 3 files with indirect coverage changes

@rohitwaghchaure rohitwaghchaure force-pushed the refactor-stock-balance-report branch 3 times, most recently from 2e43dd2 to 2d6dfea Compare May 23, 2023 11:35
@rohitwaghchaure rohitwaghchaure force-pushed the refactor-stock-balance-report branch from 2d6dfea to 3f548ac Compare May 23, 2023 11:36
@rohitwaghchaure rohitwaghchaure marked this pull request as ready for review May 23, 2023 11:36
@rohitwaghchaure rohitwaghchaure merged commit 0bc9c32 into frappe:develop May 24, 2023
@rohitwaghchaure rohitwaghchaure added the backport version-14-hotfix backport to version 14 label May 24, 2023
rohitwaghchaure added a commit that referenced this pull request May 24, 2023
…-35348

refactor: stock balance report (backport #35348)
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 4, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
backport version-14-hotfix backport to version 14
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant