Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge path_utils into fileio & refactor what was left #168

Merged
Merged
Changes from 3 commits
Commits
Show all changes
104 commits
Select commit Hold shift + click to select a range
2070c7f
Merge remote-tracking branch 'origin/main' into alex/ENG-61-pathutils…
strickvl Nov 3, 2021
6874f54
Add caching option to step decorator
schustmi Nov 4, 2021
e04ffd3
add paperspace env check
strickvl Nov 5, 2021
e65d3b8
add metadata annotation
strickvl Nov 5, 2021
85ac0af
Merge branch 'main' into michael/ENG-24-caching
schustmi Nov 5, 2021
4676b6c
Improve step parameter handling
schustmi Nov 5, 2021
d2cbf79
Fix error type in base step parameter handling
schustmi Nov 5, 2021
eb9a6d0
Implement cache disabling on a per step basis
schustmi Nov 5, 2021
d88737b
Merge pull request #156 from zenml-io/alex/add-paperspace-check
htahir1 Nov 5, 2021
f35b5be
Fix existing tests and add some more for step initialization
schustmi Nov 5, 2021
4b68e54
Merge pull request #157 from zenml-io/michael/ENG-24-caching
htahir1 Nov 5, 2021
008dd3f
Fixed import
htahir1 Nov 5, 2021
45928d4
refactor out the walk function
strickvl Nov 5, 2021
d2cdd44
Merge remote-tracking branch 'origin/main' into alex/ENG-61-pathutils…
strickvl Nov 5, 2021
72ddc7d
refactor out the is_root function
strickvl Nov 5, 2021
f7f36bc
refactor out the isdir function
strickvl Nov 5, 2021
f6c41d1
add todo for future refactor
strickvl Nov 5, 2021
0f03355
refactor out the find_files function
strickvl Nov 5, 2021
4e598ce
refactor out the is_remote function
strickvl Nov 5, 2021
119838c
add comment about naming convention
strickvl Nov 5, 2021
ee8ada6
refactor out the is_gcs_path function
strickvl Nov 5, 2021
e818eaf
refactor out the list_dir function
strickvl Nov 5, 2021
1ceced1
refactor out duplicated listdir function
strickvl Nov 5, 2021
f62479f
refactor out create_file_if_not_exists
strickvl Nov 5, 2021
acaf5bf
refactor out append_file
strickvl Nov 5, 2021
e78d908
refactor out create_dir_if_not_exists
strickvl Nov 5, 2021
f64ab35
refactor out create_dir_recursive_if_not_exists
strickvl Nov 5, 2021
e4d7f06
refactor out resolve_relative_path
strickvl Nov 5, 2021
4a1de9b
refactor out file_exists
strickvl Nov 5, 2021
ee4a5b8
remove exists function from fileio
strickvl Nov 5, 2021
6d157be
merge Hamza's changes
strickvl Nov 5, 2021
abcc883
Merge branch 'baris/ENG-37-standard-import-check' into alex/ENG-61-pa…
htahir1 Nov 5, 2021
2471caf
Add documentation for pipeline step parameter and run name configuration
schustmi Nov 5, 2021
2cf7c38
Merge pull request #158 from zenml-io/michael/ENG-67-pipeline-config-…
htahir1 Nov 5, 2021
1910d04
Merge remote-tracking branch 'origin/main' into alex/ENG-61-pathutils…
strickvl Nov 5, 2021
1b97453
Preparing version: 0.5.2
htahir1 Nov 5, 2021
f241558
refactor out copy function
strickvl Nov 5, 2021
440921b
Automatically disable caching if the step function code has changed
schustmi Nov 5, 2021
86837d9
Merge pull request #159 from zenml-io/michael/automatic-step-cache-in…
htahir1 Nov 5, 2021
e0b1244
Fixed logic of pulling examples from cli in edge cases of unreleased …
htahir1 Nov 5, 2021
eb5b840
Got rid of stray click.echos and replaced with cli.utils
htahir1 Nov 5, 2021
c068662
Merge remote-tracking branch 'origin/main' into alex/ENG-61-pathutils…
strickvl Nov 5, 2021
59320d7
Added release notes for 0.5.2
htahir1 Nov 5, 2021
d71b02b
Added release notes for 0.5.2
htahir1 Nov 5, 2021
8f6d61e
Updated quickstart
htahir1 Nov 5, 2021
defcd1c
fix path issues
strickvl Nov 5, 2021
8ba823f
refactor out copy_dir function
strickvl Nov 5, 2021
ef902da
refactor out move function
strickvl Nov 5, 2021
7f22f91
refactor out rm_dir function and delete rmtree
strickvl Nov 5, 2021
fee833c
fix broken path issue
strickvl Nov 5, 2021
abe15a4
add random testing order
strickvl Nov 8, 2021
9a81f59
Merge remote-tracking branch 'origin/main' into alex/ENG-61-pathutils…
strickvl Nov 8, 2021
cf147e0
Merge pull request #160 from zenml-io/alex/ENG-76-random-pytest
htahir1 Nov 8, 2021
ecbff43
Merge remote-tracking branch 'origin/main' into alex/ENG-61-pathutils…
strickvl Nov 8, 2021
d8ce7f0
GitBook: [#547] Fixed chapter 7 images
htahir1 Nov 8, 2021
8c55de6
Add link to config yaml format, improve type annotations
schustmi Nov 9, 2021
c1c3537
Merge remote-tracking branch 'origin/main' into alex/ENG-61-pathutils…
strickvl Nov 9, 2021
84c5a2c
refactor out rm_file function
strickvl Nov 9, 2021
e90d269
refactor out read_file_contents_as_string function
strickvl Nov 9, 2021
3897710
refactor out write_file_contents_as_string function
strickvl Nov 9, 2021
8f9705e
Allow args when connecting steps, improve error handling
schustmi Nov 9, 2021
0e15bf9
refactor out get_grandparent
strickvl Nov 9, 2021
47dcc30
refactor out get_parent
strickvl Nov 9, 2021
21fc5ad
refactor load_csv_column_names and load_csv_header
strickvl Nov 9, 2021
51bf165
refactor out load_csv_header
strickvl Nov 9, 2021
d57eadd
refactor out create_tarfile
strickvl Nov 9, 2021
8209723
refactor out extract_tarfile
strickvl Nov 9, 2021
4b66150
refactor out is_zenml_dir
strickvl Nov 9, 2021
480ddf7
refactor out get_zenml_dir
strickvl Nov 9, 2021
9eadd55
Added repobeats statistics
htahir1 Nov 9, 2021
9cfe122
Added repobeats statistics
htahir1 Nov 9, 2021
e2027b0
refactor out get_zenml_config_dir
strickvl Nov 9, 2021
94e49c4
Move location of repobeats image
alex-zenml Nov 9, 2021
fbfbd90
Add TODO to replace Channels with ZenML class
schustmi Nov 9, 2021
2b4c1aa
remove path_utils completely
strickvl Nov 9, 2021
ecb0a26
fix typo
strickvl Nov 9, 2021
faee827
Merge pull request #162 from zenml-io/michael/small-fixes
htahir1 Nov 9, 2021
03fecbf
standardise spelling
strickvl Nov 9, 2021
dfe1d35
fix typos
strickvl Nov 9, 2021
172a078
Merge remote-tracking branch 'origin/main' into alex/ENG-61-pathutils…
strickvl Nov 9, 2021
f19ac8e
amend type
strickvl Nov 9, 2021
9be1b5b
amend type
strickvl Nov 9, 2021
5e14f2b
move below contributing
strickvl Nov 9, 2021
c09555e
add extra line break
alex-zenml Nov 9, 2021
c3759f7
Merge pull request #163 from zenml-io/alex/repobeats-location-shift
htahir1 Nov 9, 2021
741cae3
Allow args when creating a pipeline, improve error handling
schustmi Nov 10, 2021
832b88e
Add tests for pipeline initialization
schustmi Nov 10, 2021
913ca87
Add missing type annotation
schustmi Nov 10, 2021
e75b250
Update error message to include BaseStep subclasses as allowed inputs…
schustmi Nov 10, 2021
8ddc30a
GitBook: [#548] Added Sam
htahir1 Nov 10, 2021
3ad9bcb
Add expected steps/artifacts to error message
schustmi Nov 10, 2021
118ad73
Added Sam
htahir1 Nov 10, 2021
9036f46
Added Sam
htahir1 Nov 10, 2021
f36b910
Added sam images
htahir1 Nov 10, 2021
49f30c1
switched the div and style to tables as github sanitizes css
htahir1 Nov 10, 2021
282e118
Extract pipeline and step creation into fixtures, separate pipeline i…
schustmi Nov 10, 2021
e37b461
Merge pull request #165 from zenml-io/hamza/add_sam
htahir1 Nov 10, 2021
d742be6
Merge pull request #164 from zenml-io/michael/pipeline-with-args
schustmi Nov 10, 2021
3fbcf6a
Improve detection of third party modules during class resolving
schustmi Nov 10, 2021
0e040bc
Add test for third party module detection
schustmi Nov 10, 2021
77fc06f
Merge pull request #167 from zenml-io/michael/third-party-source-dete…
schustmi Nov 10, 2021
33569a0
fix class import path
strickvl Nov 11, 2021
b9babf8
Merge remote-tracking branch 'origin/main' into alex/ENG-61-pathutils…
strickvl Nov 11, 2021
e6e81ee
fix import order for linting
strickvl Nov 11, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions src/zenml/utils/analytics_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,13 @@ def in_google_colab() -> bool:
return False


def in_paperspace_gradient() -> bool:
"""Returns: True if running in a Paperspace Gradient env, else False"""
if "PAPERSPACE_NOTEBOOK_REPO_ID" in os.environ:
return True
return False


def get_system_info() -> Dict[str, Any]:
"""Returns system info as a dict.

Expand Down Expand Up @@ -146,6 +153,7 @@ def track_event(event: str, metadata: Optional[Dict[str, Any]] = None) -> None:
{
"in_docker": in_docker(),
"in_google_colab": in_google_colab(),
"in_paperspace_gradient": in_paperspace_gradient(),
"version": __version__,
}
)
Expand Down