Add a property bag to node metadata

[EngHabu]

Why are the changes needed?

This allows plugins to pass execution instructions to the backend plugin without having to introduce new protos..

What changes were proposed in this pull request?

Allow passing cluster_assignment in Workflow NodeMetadata and LaunchPlan.

How was this patch tested?

Deployed to an internal tenant and verified the bits are passed correctly.

Setup process

Screenshots

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

Summary by Bito

This PR introduces a new config property bag in NodeMetadata enabling plugins to pass execution instructions without new proto definitions, and adds support for cluster_assignment in LaunchPlan specifications. The changes span across multiple language implementations including Go, TypeScript, Python, and Rust, enhancing the flexibility of plugin configuration and execution control.

Unit tests added: False

Estimated effort to review (1-5, lower is better): 5

[flyte-bot]

Code Review Agent Run Status

• Limitations and other issues: ❌ Failure - The AI Code Review Agent skipped reviewing this change because it is configured to exclude certain pull requests based on the source/target branch or the pull request status. You can change the settings here, or contact the agent instance creator at eduardo@union.ai.

[codecov]

Codecov Report

Attention: Patch coverage is 16.66667% with 10 lines in your changes missing coverage. Please review.

Project coverage is 37.03%. Comparing base (05c85a8) to head (4d7c359).
Report is 4 commits behind head on master.

Files with missing lines	Patch %	Lines
...lyteidl/gen/pb-go/flyteidl/admin/launch_plan.pb.go	16.66%	5 Missing ⚠️
flyteidl/gen/pb-go/flyteidl/core/workflow.pb.go	16.66%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6169      +/-   ##
==========================================
- Coverage   37.05%   37.03%   -0.02%     
==========================================
  Files        1318     1318              
  Lines      132583   132594      +11     
==========================================
- Hits        49122    49100      -22     
- Misses      79211    79243      +32     
- Partials     4250     4251       +1     

Flag	Coverage Δ
unittests-datacatalog	51.58% <ø> (ø)
unittests-flyteadmin	54.31% <ø> (-0.03%)	⬇️
unittests-flytecopilot	30.99% <ø> (ø)
unittests-flytectl	62.29% <ø> (-0.05%)	⬇️
unittests-flyteidl	7.23% <16.66%> (+<0.01%)	⬆️
unittests-flyteplugins	53.85% <ø> (ø)
unittests-flytepropeller	42.62% <ø> (-0.05%)	⬇️
unittests-flytestdlib	55.13% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[flyte-bot]

Code Review Agent Run #812b90

Actionable Suggestions - 0

Review Details

• Files reviewed - 14 · Commit Range: 4d7c359..4d7c359

  • flyteidl/gen/pb-es/flyteidl/admin/launch_plan_pb.ts
  • flyteidl/gen/pb-es/flyteidl/core/workflow_pb.ts
  • flyteidl/gen/pb-go/flyteidl/admin/launch_plan.pb.go
  • flyteidl/gen/pb-go/flyteidl/core/workflow.pb.go
  • flyteidl/gen/pb-js/flyteidl.d.ts
  • flyteidl/gen/pb-js/flyteidl.js
  • flyteidl/gen/pb_python/flyteidl/admin/launch_plan_pb2.py
  • flyteidl/gen/pb_python/flyteidl/admin/launch_plan_pb2.pyi
  • flyteidl/gen/pb_python/flyteidl/core/workflow_pb2.py
  • flyteidl/gen/pb_python/flyteidl/core/workflow_pb2.pyi
  • flyteidl/gen/pb_rust/flyteidl.admin.rs
  • flyteidl/gen/pb_rust/flyteidl.core.rs
  • flyteidl/protos/flyteidl/admin/launch_plan.proto
  • flyteidl/protos/flyteidl/core/workflow.proto
• Files skipped - 2

  • flyteidl/clients/go/assets/admin.swagger.json - Reason: Filter setting
  • flyteidl/gen/pb-go/gateway/flyteidl/service/admin.swagger.json - Reason: Filter setting
• Tools

  • Golangci-lint (Linter) - ✖︎ Failed
  • Whispers (Secret Scanner) - ✔︎ Successful
  • Detect-secrets (Secret Scanner) - ✔︎ Successful

---

AI Code Review powered by

[flyte-bot]

Changelist by Bito

This pull request implements the following key changes.

Key Change	Files Impacted
Feature Improvement - Enhanced Node Metadata and Launch Plan Configuration	- launch_plan_pb.ts - Added cluster assignment support to LaunchPlan specification - workflow_pb.ts - Added config property bag to NodeMetadata for plugin instructions - launch_plan.pb.go - Implemented cluster assignment functionality in Go protobuf
Feature Improvement - Enhanced Launch Plan and Node Configuration	- launch_plan.pb.go - Updated protobuf definitions to support cluster assignment and property bag configuration - launch_plan_pb.ts - Added cluster assignment support to LaunchPlan specification - workflow_pb.ts - Added config property bag to NodeMetadata for plugin instructions
Feature Improvement - Enhanced Node Configuration System	- launch_plan.pb.go - Added cluster assignment support and updated protobuf dependencies - workflow.pb.go - Added config property bag to NodeMetadata for plugin configuration - launch_plan_pb.ts - Added cluster assignment support to LaunchPlan specification - workflow_pb.ts - Added config property bag to NodeMetadata for plugin instructions
Feature Improvement - Node Metadata Property Bag Enhancement	- workflow.pb.go - Added config property bag to NodeMetadata and updated protobuf message types - flyteidl.d.ts - Added TypeScript definitions for NodeMetadata config and LaunchPlanSpec clusterAssignment - flyteidl.js - Implemented JavaScript support for NodeMetadata config property bag
Feature Improvement - Enhanced Node Configuration System	- flyteidl.js - Added cluster assignment support to LaunchPlanSpec and updated protobuf definitions - launch_plan_pb2.py - Updated Python protobuf definitions to include cluster assignment support - workflow_pb2.py - Added config property bag to NodeMetadata and updated serialization - workflow_pb2.pyi - Added type definitions for NodeMetadata config property bag - flyteidl.admin.rs - Added cluster assignment support to LaunchPlanSpec in Rust - flyteidl.core.rs - Added config property bag to NodeMetadata in Rust implementation - launch_plan.proto - Added cluster assignment field to LaunchPlanSpec proto definition - workflow.proto - Added config property bag field to NodeMetadata proto definition