Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stacked @card v1.2 : Customizing @card with current.card #894

Merged
merged 53 commits into from
Jan 25, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
f61e200
allow passing userland `MetaflowCardComponents`
valayDave Dec 22, 2021
e32d6f0
Merge branch 'multiple-card-decorators' into multiple-card-decorators…
valayDave Dec 25, 2021
072f47a
`current.cards` with `id` support.
valayDave Jan 2, 2022
597b5a8
passing card `id` to subprocess and storing it
valayDave Jan 2, 2022
113f400
Added `customize=True` in `@card`
valayDave Jan 3, 2022
69a3692
`get` + `id` related changes + comments + warnings
valayDave Jan 3, 2022
7516fa9
Fix comment.
valayDave Jan 3, 2022
45da014
added json serialization checks
valayDave Jan 3, 2022
e2a8c5b
Checking if `id` matches regex pattern.
valayDave Jan 4, 2022
d56e51e
Bug fix in the card id + id check in `@card`
valayDave Jan 4, 2022
6872741
fixing @card registers incorrect number of other @card
valayDave Jan 4, 2022
8e237d9
Fixing comment
valayDave Jan 4, 2022
daa9799
Added error details to _serialize_components
valayDave Jan 5, 2022
518aac9
Merge branch 'multiple-card-decorators' into multiple-card-decorators…
valayDave Jan 6, 2022
3d7506b
Merge branch 'multiple-card-decorators' into multiple-card-decorators…
valayDave Jan 6, 2022
e0d66df
converting`bool` to `str` for `self.attribute`
valayDave Jan 7, 2022
d13e570
Merge branch 'multiple-card-decorators' into multiple-card-decorators…
valayDave Jan 7, 2022
17580d2
Merge branch 'multiple-card-decorators' into multiple-card-decorators…
valayDave Jan 8, 2022
3d09d38
Changed `current.cards` to `current.card`
valayDave Jan 10, 2022
177fdb3
Removing warning code when > 1 card has no id
valayDave Jan 10, 2022
f3ee72f
Warning fix.
valayDave Jan 10, 2022
91e429e
Merge branch 'multiple-card-decorators' into multiple-card-decorators…
valayDave Jan 11, 2022
4bbf8ab
stateful warnings in `current.card`
valayDave Jan 12, 2022
1684f2d
Added warning component for `current.card`
valayDave Jan 12, 2022
a2d60b9
Merge branch 'mfcards-s0-multi-deco' into mfcards-s1-current-card
valayDave Jan 14, 2022
b73ae73
Merge branch 'multiple-card-decorators' into multiple-card-decorators…
valayDave Jan 14, 2022
a37f7b6
Merge branch 'master' into multiple-card-decorators
valayDave Jan 14, 2022
d6a470c
Merge branch 'mfcards-s0-multi-deco' into mfcards-s1-current-card
valayDave Jan 17, 2022
a53a555
Merge branch 'multiple-card-decorators' into multiple-card-decorators…
valayDave Jan 17, 2022
7ca6498
Merge branch 'master' into multiple-card-decorators
valayDave Jan 17, 2022
98ae8e4
Merge branch 'multiple-card-decorators' into multiple-card-decorators…
valayDave Jan 18, 2022
0492e20
Merge branch 'master' into multiple-card-decorators
valayDave Jan 18, 2022
d9690f4
Merge branch 'mfcards-s0-multi-deco' into mfcards-s1-current-card
valayDave Jan 18, 2022
e8f2197
Fixing bugs for bad card types.
valayDave Jan 18, 2022
0cd4685
Merge branch 'master' into multiple-card-decorators
valayDave Jan 20, 2022
759bd6b
Merge branch 'multiple-card-decorators' into multiple-card-decorators…
valayDave Jan 20, 2022
58bea9b
Merge branch 'mfcards-s0-multi-deco' into mfcards-s1-current-card
valayDave Jan 20, 2022
4d8aae8
Merge branch 'mfcards-s0-multi-deco' into mfcards-s1-current-card
valayDave Jan 21, 2022
86b54bd
Merge branch 'master' into multiple-card-decorators
valayDave Jan 21, 2022
db96dd1
Merge branch 'multiple-card-decorators' into multiple-card-decorators…
valayDave Jan 21, 2022
e41ca22
Suppressing warnings by default for card components.
valayDave Jan 21, 2022
178af91
Merge branch 'multiple-card-decorators-custom-components-final-api-de…
valayDave Jan 21, 2022
3727949
Nit fixes from romain
valayDave Jan 22, 2022
7619b25
Merge branch 'multiple-card-decorators-custom-components-final-api-de…
valayDave Jan 22, 2022
cab38e7
Fixing bug
valayDave Jan 22, 2022
049bf2e
Merge branch 'multiple-card-decorators-custom-components-final-api-de…
valayDave Jan 22, 2022
4b8e92f
handling component warning suppression via config
valayDave Jan 23, 2022
172a11f
Merge branch 'multiple-card-decorators-custom-components-final-api-de…
valayDave Jan 23, 2022
5122753
Merge branch 'mfcards-s0-multi-deco' into mfcards-s1-current-card
valayDave Jan 24, 2022
e2bdc3b
Merge branch 'mfcards-s0-multi-deco' into mfcards-s1-current-card
valayDave Jan 24, 2022
f8f94cb
Fixing warning component formatting.
valayDave Jan 24, 2022
fe8596a
Merge branch 'mfcards-s0-multi-deco' into mfcards-s1-current-card
valayDave Jan 25, 2022
8694045
Stacked @card v1.2: Read cli changes for Supporting Multiple`@card`s …
valayDave Jan 25, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
306 changes: 306 additions & 0 deletions docs/cards.md

Large diffs are not rendered by default.

18 changes: 8 additions & 10 deletions metaflow/cards.py
Original file line number Diff line number Diff line change
@@ -1,21 +1,19 @@
from metaflow.plugins.cards.card_client import get_cards
from metaflow.plugins.cards.card_modules.card import MetaflowCardComponent, MetaflowCard
from metaflow.plugins.cards.card_modules.components import (
Artifact,
Table,
Image,
Error,
Markdown,
)
from metaflow.plugins.cards.card_modules.basic import (
DefaultCard,
TitleComponent,
SubTitleComponent,
SectionComponent,
ImageComponent,
BarChartComponent,
LineChartComponent,
TableComponent,
DagComponent,
PageComponent,
ArtifactsComponent,
RENDER_TEMPLATE_PATH,
TaskToDict,
DefaultComponent,
ChartComponent,
TaskInfoComponent,
ErrorCard,
BlankCard,
)
2 changes: 2 additions & 0 deletions metaflow/graph.py
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,8 @@ def node_to_dict(name, node):
d["foreach_artifact"] = node.foreach_param
elif d["type"] == "split-parallel":
d["num_parallel"] = node.num_parallel
if node.matching_join:
d["matching_join"] = node.matching_join
return d

def populate_block(start_name, end_name):
Expand Down
9 changes: 6 additions & 3 deletions metaflow/metaflow_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,26 +64,29 @@ def from_conf(name, default=None):
DATATOOLS_SUFFIX = from_conf("METAFLOW_DATATOOLS_SUFFIX", "data")
DATATOOLS_S3ROOT = from_conf(
"METAFLOW_DATATOOLS_S3ROOT",
"%s/%s" % (from_conf("METAFLOW_DATASTORE_SYSROOT_S3"), DATATOOLS_SUFFIX)
os.path.join(from_conf("METAFLOW_DATASTORE_SYSROOT_S3"), DATATOOLS_SUFFIX)
if from_conf("METAFLOW_DATASTORE_SYSROOT_S3")
else None,
)
# Local datatools root location
DATATOOLS_LOCALROOT = from_conf(
"METAFLOW_DATATOOLS_LOCALROOT",
"%s/%s" % (from_conf("METAFLOW_DATASTORE_SYSROOT_LOCAL"), DATATOOLS_SUFFIX)
os.path.join(from_conf("METAFLOW_DATASTORE_SYSROOT_LOCAL"), DATATOOLS_SUFFIX)
if from_conf("METAFLOW_DATASTORE_SYSROOT_LOCAL")
else None,
)

# Cards related config variables
DATASTORE_CARD_SUFFIX = "mf.cards"
DATASTORE_CARD_LOCALROOT = from_conf("METAFLOW_CARD_LOCALROOT")
DATASTORE_CARD_S3ROOT = from_conf(
"METAFLOW_CARD_S3ROOT",
"%s/%s" % (from_conf("METAFLOW_DATASTORE_SYSROOT_S3"), DATASTORE_CARD_SUFFIX)
os.path.join(from_conf("METAFLOW_DATASTORE_SYSROOT_S3"), DATASTORE_CARD_SUFFIX)
if from_conf("METAFLOW_DATASTORE_SYSROOT_S3")
else None,
)
CARD_NO_WARNING = from_conf("METAFLOW_CARD_NO_WARNING", False)

# S3 endpoint url
S3_ENDPOINT_URL = from_conf("METAFLOW_S3_ENDPOINT_URL", None)
S3_VERIFY_CERTIFICATE = from_conf("METAFLOW_S3_VERIFY_CERTIFICATE", None)
Expand Down
21 changes: 18 additions & 3 deletions metaflow/plugins/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -150,17 +150,32 @@ def get_plugin_cli():
_merge_lists(FLOW_DECORATORS, _ext_plugins["FLOW_DECORATORS"], "name")

# Cards
from .cards.card_modules.basic import DefaultCard, TaskSpecCard, ErrorCard
from .cards.card_modules.test_cards import TestErrorCard, TestTimeoutCard, TestMockCard
from .cards.card_modules.basic import DefaultCard, TaskSpecCard, ErrorCard, BlankCard
from .cards.card_modules.test_cards import (
TestErrorCard,
TestTimeoutCard,
TestMockCard,
TestPathSpecCard,
TestEditableCard,
TestEditableCard2,
TestNonEditableCard,
)
from .cards.card_modules import MF_EXTERNAL_CARDS

CARDS = [
DefaultCard,
TaskSpecCard,
ErrorCard,
BlankCard,
TestErrorCard,
TestTimeoutCard,
TestMockCard,
]
TestPathSpecCard,
TestEditableCard,
TestEditableCard2,
TestNonEditableCard,
BlankCard,
] + MF_EXTERNAL_CARDS
# Sidecars
from ..mflog.save_logs_periodically import SaveLogsPeriodicallySidecar
from metaflow.metadata.heartbeat import MetadataHeartBeat
Expand Down
Loading