Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve Runners #4501

Open
astrojuanlu opened this issue Feb 19, 2025 · 0 comments
Open

Improve Runners #4501

astrojuanlu opened this issue Feb 19, 2025 · 0 comments
Labels
Type: Parent Issue
Milestone
Improve Runners (...

Comments

@astrojuanlu
Copy link
Member

Original milestone description:

Kedro has a lot of features to improve the performance and efficiency of Kedro runs.

  • Different Runner types
  • CachedDataSet
  • --async

But perhaps we can make the default way Kedro is run more efficient without the user needing to use any of the above features. A "smart" runner that can automatically use the settings that will make your run the fastest and compute efficient.

Priorities are

  • Solve issues with ParallelRunner
  • Sort out thread safety issues with SequentialRunner
  • Decouple runners and catalog
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Type: Parent Issue
Projects
Kedro 🔶
Status: No status
Milestone
Improve Runners (code & efficiency)
Development

No branches or pull requests
1 participant
@astrojuanlu