Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Running a federation with GaNDLF Documentation (#794)
* Initial commit of Running the federation with GaNDLF Documentation Signed-off-by: Patrick Foley <psfoley@gmail.com> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu> Signed-off-by: Patrick Foley <psfoley@gmail.com> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu> Signed-off-by: Patrick Foley <psfoley@gmail.com> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu> Signed-off-by: Patrick Foley <psfoley@gmail.com> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu> Signed-off-by: Patrick Foley <psfoley@gmail.com> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu> Signed-off-by: Patrick Foley <psfoley@gmail.com> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu> Signed-off-by: Patrick Foley <psfoley@gmail.com> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu> Signed-off-by: Patrick Foley <psfoley@gmail.com> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu> Signed-off-by: Patrick Foley <psfoley@gmail.com> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu> Signed-off-by: Patrick Foley <psfoley@gmail.com> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu> Signed-off-by: Patrick Foley <psfoley@gmail.com> * Update README.md Removed references to Intel's ownship, given it's now owned by the LF AI and Data. Signed-off-by: Patrick Foley <psfoley@gmail.com> * Fix Flake8 C419 for Ubuntu CI (#800) C419 Unnecessary list comprehension passed to any()/all() prevents short-circuiting - rewrite as a generator Signed-off-by: Aleksander Kantak <aleksander.kantak@intel.com> Signed-off-by: Patrick Foley <psfoley@gmail.com> * Update README.md Signed-off-by: Patrick Foley <psfoley@gmail.com> * Fix warnings and issues in docs (#825) * Fix warnings and issues in docs Signed-off-by: Aleksander Kantak <aleksander.kantak@intel.com> * fixup! Fix warnings and issues in docs Signed-off-by: Aleksander Kantak <aleksander.kantak@intel.com> --------- Signed-off-by: Aleksander Kantak <aleksander.kantak@intel.com> Signed-off-by: Patrick Foley <psfoley@gmail.com> * Add Logo (#827) * Add Logo * Update README.md Signed-off-by: Patrick Foley <psfoley@gmail.com> * Change OpenFL documentation font to improve accessibility (#809) This replaces the font of OpenFL documents with Intel One Mono font for low vision developers. Known issues: 1. The text font within the images has not been changed. 2. Some icons that do not exist in the new font cannot be displayed properly. Fixes #799 Co-authored-by: Wang, Le <le3.wang@intel.com> Signed-off-by: He, Dan H <dan.h.he@intel.com> Signed-off-by: Jiang, Jiaqiu <jiaqiu.jiang@intel.com> Signed-off-by: Li, Qingqing <qingqing.li@intel.com> Signed-off-by: Wang, Le <le3.wang@intel.com> Signed-off-by: Wu, Caili <caili.wu@intel.com> Signed-off-by: He, Dan H <dan.h.he@intel.com> Co-authored-by: He, Dan H <dan.h.he@intel.com> Signed-off-by: Patrick Foley <psfoley@gmail.com> * Update unit tests to improve code coverage (#821) * Update ci config Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> * Add unit test for following files - openfl/federated/plan/plan.py - openfl/interface/aggregation_functions/core/adaptive_aggregation.py Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> * Add some test cases for databases module Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> * Fix bugs for databases module unittest Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> * Update unit tests for component module Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> * Restore workflow config and update some comments Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> * Enable save_ test case. Add yaml under test dir for unit test usage. Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> * Remove plan to new dir. Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> * Remove plan to new dir. Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> * Add aggregator start test cases. Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> * Add 2 aggregator test cases. Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> * Add 1 aggregator test case. Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> * Format code. Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> * Refactor code. Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> * Add collaborator start test cases. Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> * Add 1 collaborator test case. Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> * Format with flake8 Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> * Remove TODO comments Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> --------- Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> Co-authored-by: Wang, Wenjie <wenjie2.wang@intel.com> Co-authored-by: Lei5 Chen <lei5.chen@intel.com> Signed-off-by: Patrick Foley <psfoley@gmail.com> * Add PyTorch linear regression example (#808) This adds a new tutorial example on distributing a linear regression task over OpenFL cluster. The model is defined by Pytorch which is able to run over both cpu (by default) and gpu. The dataset is generated by make_regression from sklearn.datasets with pre-defined parameters. Fixes #797 Co-authored-by: Jiang, Jiaqiu <jiaqiu.jiang@intel.com> Signed-off-by: He, Dan H <dan.h.he@intel.com> Signed-off-by: Jiang, Jiaqiu <jiaqiu.jiang@intel.com> Signed-off-by: Li, Qingqing <qingqing.li@intel.com> Signed-off-by: Wang, Le <le3.wang@intel.com> Signed-off-by: Wu, Caili <caili.wu@intel.com> Signed-off-by: He, Dan H <dan.h.he@intel.com> Signed-off-by: Patrick Foley <psfoley@gmail.com> * This prints out the hash of the CSR to disk for both the aggregator and (#813) * This prints out the hash of the CSR to disk for both the aggregator and collaborator. The user then compares and approves this hash with the hash printed out of the file to validate the CSR. In addition, a warning message is pritned if certify is run in silent mode. Fixes #692 Signed-off-by: Grant Baker <grant.baker@intel.com> * Refactor read_csr function to use get_csr_hash Signed-off-by: Grant Baker <grant.baker@intel.com> * Ask to check hashes before prompt --------- Signed-off-by: Grant Baker <grant.baker@intel.com> Co-authored-by: Grant Baker <grant.baker@intel.com> Signed-off-by: Patrick Foley <psfoley@gmail.com> * Improve workspace requirements import (#810) Remove the dump_requirement_file operation in export_ method. Fixes #767 Co-authored-by: Li, Qingqing <qingqing.li@intel.com> Co-authored-by: Wu, Caili <caili.wu@intel.com> Signed-off-by: He, Dan H <dan.h.he@intel.com> Signed-off-by: Jiang, Jiaqiu <jiaqiu.jiang@intel.com> Signed-off-by: Li, Qingqing <qingqing.li@intel.com> Signed-off-by: Wang, Le <le3.wang@intel.com> Signed-off-by: Wu, Caili <caili.wu@intel.com> Signed-off-by: He, Dan H <dan.h.he@intel.com> Signed-off-by: Patrick Foley <psfoley@gmail.com> * Issue 506 Added Example using FedProx (#818) * created new ineractive_api dir to hold pytorch fedprox mnist example corrected files changed to FedProxOptimizer and ran set_old_weights for new FedProx Pytorch example renamed FedProx notebook used mode.parrameters() to get pytorch model weights got weights using state_dict changed old wieghts to list (for serialization) and fixed README input wieghts before zero_grad [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer. Fixes: #506 Signed-off-by: Klemme, Beverly <beverly.klemme@intel.com> Signed-off-by: Baker, Grant <grant.baker@intel.com> Signed-off-by: ELizabeth Simon, Neethu <neethu.elizabeth.simon@intel.com> Signed-off-by: Jillela, Emmanuel <emmanuel.jillela@intel.com> Signed-off-by: Beverly Klemme <beverly.klemme@intel.com> * [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer. Fixes: #506 Signed-off-by: Klemme, Beverly <beverly.klemme@intel.com> Signed-off-by: Baker, Grant <grant.baker@intel.com> Signed-off-by: Elizabeth Simon, Neethu <neethu.elizabeth.simon@intel.com> Signed-off-by: Jillela, Emmanuel <emmanuel.jillela@intel.com> Signed-off-by: Beverly Klemme <beverly.klemme@intel.com> * Update README.md Signed-off-by: Beverly Klemme <beverly.klemme@intel.com> * addressed comments by psfoley: corrected words in the jupyter notebook metadata and added a link to the FedProx paper in the README. Signed-off-by: Beverly Klemme <beverly.klemme@intel.com> --------- Signed-off-by: Klemme, Beverly <beverly.klemme@intel.com> Signed-off-by: Baker, Grant <grant.baker@intel.com> Signed-off-by: ELizabeth Simon, Neethu <neethu.elizabeth.simon@intel.com> Signed-off-by: Jillela, Emmanuel <emmanuel.jillela@intel.com> Signed-off-by: Beverly Klemme <beverly.klemme@intel.com> Signed-off-by: Elizabeth Simon, Neethu <neethu.elizabeth.simon@intel.com> Signed-off-by: Patrick Foley <psfoley@gmail.com> * [Bug: 768] FX CLI: Separate create, cert gen commands (#807) This change separates existing command "fx collaborator.py generate-cert-request" command into two commands. "fx collaborator create -n {NAME} -d {DATA_PATH: optional}". "fx collaborator generate-cert-request -n {NAME}". Fixes #768 Signed-off-by: Emmanuel Jillela <emmanuel.jillela@intel.com> Co-authored-by: Emmanuel Jillela <emmanuel.jillela@intel.com> Signed-off-by: Patrick Foley <psfoley@gmail.com> * Add new tutorial example to OpenFL interactive API (#812) * Add new tutorial example to OpenFL interactive API This adds a new tutorial example on distributing a linear regression task over OpenFL cluster The model is defined by scikit-learn which is able to run over both cpu (by default) and gpu. The dataset is 1-dimensional noisy data of sinusoid with pre-defined parameters. Fixes #798 Co-authored-by: Beverly Klemme <beverly.klemme@intel.com> Co-authored-by: Grant Baker <grant.baker@intel.com> Signed-off-by: Yi CAO <yi2.cao@intel.com> * reduced requirements.txt in workspace Signed-off-by: Beverly Klemme <beverly.klemme@intel.com> --------- Signed-off-by: Yi CAO <yi2.cao@intel.com> Signed-off-by: Beverly Klemme <beverly.klemme@intel.com> Co-authored-by: Yi CAO <yi2.cao@intel.com> Signed-off-by: Patrick Foley <psfoley@gmail.com> * build(deps): bump tensorflow in /openfl-workspace/tf_cnn_histology (#776) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](tensorflow/tensorflow@v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Patrick Foley <psfoley@gmail.com> * build(deps): bump tensorflow (#777) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](tensorflow/tensorflow@v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Patrick Foley <psfoley@gmail.com> * Update GaNDLF repo location and test CI with master branch Signed-off-by: Patrick Foley <psfoley@gmail.com> * Update GaNDLF repo location and test CI with master branch Signed-off-by: Patrick Foley <psfoley@gmail.com> * Update GaNDLF repo location and test CI with master branch Signed-off-by: Patrick Foley <psfoley@gmail.com> * Fix documentation links. Change path names and templates for CI Signed-off-by: Patrick Foley <psfoley@gmail.com> * Fix paths Signed-off-by: Patrick Foley <psfoley@gmail.com> * Fix paths Signed-off-by: Patrick Foley <psfoley@gmail.com> * Fix breaking tests Signed-off-by: Patrick Foley <psfoley@gmail.com> * Add compatible onnx version to requirements.txt file Signed-off-by: Patrick Foley <psfoley@gmail.com> * Fix wrong csv file name Signed-off-by: Patrick Foley <psfoley@gmail.com> * Fix wrong csv file name Signed-off-by: Patrick Foley <psfoley@gmail.com> * Fix wrong names in workflow file Signed-off-by: Patrick Foley <psfoley@gmail.com> * Fix wrong data path Signed-off-by: Patrick Foley <psfoley@gmail.com> * Fix lint in test_gandlf.py Signed-off-by: Patrick Foley <psfoley@gmail.com> * Fix lint errors Signed-off-by: Patrick Foley <psfoley@gmail.com> --------- Signed-off-by: Patrick Foley <psfoley@gmail.com> Signed-off-by: Aleksander Kantak <aleksander.kantak@intel.com> Signed-off-by: He, Dan H <dan.h.he@intel.com> Signed-off-by: Fang, Xiaoran <xiaoran.fang@intel.com> Signed-off-by: Klemme, Beverly <beverly.klemme@intel.com> Signed-off-by: Baker, Grant <grant.baker@intel.com> Signed-off-by: ELizabeth Simon, Neethu <neethu.elizabeth.simon@intel.com> Signed-off-by: Jillela, Emmanuel <emmanuel.jillela@intel.com> Signed-off-by: Beverly Klemme <beverly.klemme@intel.com> Signed-off-by: Elizabeth Simon, Neethu <neethu.elizabeth.simon@intel.com> Signed-off-by: Emmanuel Jillela <emmanuel.jillela@intel.com> Signed-off-by: Yi CAO <yi2.cao@intel.com> Co-authored-by: Sarthak Pati <sarthak.pati@pennmedicine.upenn.edu> Co-authored-by: Prashant Shah <40899779+SprashAI@users.noreply.github.com> Co-authored-by: akantak <aleksander.kantak@intel.com> Co-authored-by: wangleflex <106506636+wangleflex@users.noreply.github.com> Co-authored-by: He, Dan H <dan.h.he@intel.com> Co-authored-by: xiaoranf <desolatefang@gmail.com> Co-authored-by: Wang, Wenjie <wenjie2.wang@intel.com> Co-authored-by: Lei5 Chen <lei5.chen@intel.com> Co-authored-by: Beverly Klemme <35578090+bjklemme-intel@users.noreply.github.com> Co-authored-by: Emmanuel Jillela <emmanuel.jillela@intel.com> Co-authored-by: Yi CAO <yi2.cao@intel.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
- Loading branch information
13 people
authored
Jun 29, 2023
1 parent
29e3978
commit 029a575