Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI][Docker][Cortex-M]Update scripts to update ci_cortexm to Ubuntu 20.04 #13736

Merged
merged 8 commits into from
Jan 11, 2023

Conversation

mehrdadh
Copy link
Member

@mehrdadh mehrdadh commented Jan 9, 2023

This PR makes changed to update ci_cortexm to 20.04. Here is a list of changes:

  • Updates nrfjprog to latest (10.18.1) to fix issues related to running microTVM tests inside docker. The main issue was that with older version commands like nrfjprog --com was not working even with docker --privileged.
  • Update python install script to support python 3.7 in 20.04 -> we keep using python 3.7 in ubuntu 20.04 until its EOL
  • Add script to install LLVM in ubuntu 20.04

cc @driazati @gromero

@tvm-bot
Copy link
Collaborator

tvm-bot commented Jan 9, 2023

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

Generated by tvm-bot

@github-actions github-actions bot requested review from driazati and gromero January 9, 2023 18:59
@mehrdadh
Copy link
Member Author

mehrdadh commented Jan 9, 2023

@tvm-bot rerun

@mehrdadh
Copy link
Member Author

mehrdadh commented Jan 9, 2023

cc @Mousius

@mehrdadh mehrdadh force-pushed the ci_update_cortexm_ubuntu20 branch from 70e98dd to c1456ae Compare January 9, 2023 22:26
@mehrdadh
Copy link
Member Author

@tvm-bot rerun

Copy link
Member

@Mousius Mousius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks a lot more secure @mehrdadh 😸 one last change and then I think it's good to go.

echo deb http://apt.llvm.org/focal/ llvm-toolchain-focal-13 main\
>> /etc/apt/sources.list.d/llvm.list

wget -q -O - http://apt.llvm.org/llvm-snapshot.gpg.key|sudo apt-key add -
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should get a specific key with a specific fingerprint from a reasonably trust worthy key server:

Suggested change
wget -q -O - http://apt.llvm.org/llvm-snapshot.gpg.key|sudo apt-key add -
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 15CF4D18AF4F7421

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

COPY install/ubuntu1804_install_llvm.sh /install/ubuntu1804_install_llvm.sh
RUN bash /install/ubuntu1804_install_llvm.sh
COPY install/ubuntu2004_install_llvm.sh /install/ubuntu2004_install_llvm.sh
RUN bash /install/ubuntu2004_install_llvm.sh
Copy link
Contributor

@ashutosh-arm ashutosh-arm Jan 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mehrdadh, thanks for this upgrade. Since Ubuntu20.04 has default installation of python3.8 and python3.7 EOL is June 2023 as per https://peps.python.org/pep-0537/#lifespan. Since I have not seen any other discussions, my assumption is that we still do wish to support older version until its EOL. We most likely would require python3.7 installation from https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ashutosh-arm do we need python3.7 for cortexm CI image? If so, we should add it in a separate PR.
This PR doesn't update the image yet.

Copy link
Contributor

@ashutosh-arm ashutosh-arm Jan 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we will need to support python3.7 until its EOL. Considering this, the next thing that pops up in my brain is when the ci_cortexm gets updated eventually corresponding to this (current) change, will the default venv still remain at python3.7 or we would move cortex-m docker image to use install/ubuntu2004_install_python.sh?

ENV TVM_VENV /venv/apache-tvm-py3.7

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

install/ubuntu2004_install_python.sh installs python3.8 as well. We have to add another script to install 3.7 in ubuntu20.04 if we need it.

Switching the ci image to 3.8 doesn't mean we're not supporting 3.7 anymore. Could you elaborate if there is a specific reason that this image has to run 3.7? We have switched other images like hexagon to 3.8 already.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see your point, I will change the scripts to install python3.7 in ci_cortexm.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

08ce7f8 addresses your concern.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mehrdadh

@mehrdadh
Copy link
Member Author

@Mousius PTAL and explicitly approve if there's no other suggestions. thanks!

@mehrdadh
Copy link
Member Author

@tvm-bot rerun

@mehrdadh mehrdadh force-pushed the ci_update_cortexm_ubuntu20 branch from 6a8124d to 08ce7f8 Compare January 10, 2023 23:23
Copy link
Contributor

@ashutosh-arm ashutosh-arm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have had a look at the changes done for python3.7 support. LGTM! Thanks @mehrdadh for the update.

Copy link
Member

@Mousius Mousius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update the commit message @mehrdadh? The scope of this PR has expanded massively, I'd actually agree with your initial suggestion of splitting the Python 3.7 stuff into a separate PR as the changes you were trying to make are now dwarfed by unrelated ones.

@mehrdadh mehrdadh changed the title [CI][Docker][Cortex-M]Update ci_cortexm to Ubuntu 20.04 and update nrfjprog version to 10.18.1 [CI][Docker][Cortex-M]Update scripts to update ci_cortexm to Ubuntu 20.04 Jan 11, 2023
@mehrdadh
Copy link
Member Author

@Mousius I updated both PR description which will be used for commit message and PR tilte

Copy link
Member

@Mousius Mousius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks ok to me, thanks for your diligence on the security points here @mehrdadh 😺

@Mousius Mousius merged commit a9c6f13 into apache:main Jan 11, 2023
@mehrdadh mehrdadh deleted the ci_update_cortexm_ubuntu20 branch January 11, 2023 16:56
fzi-peccia pushed a commit to fzi-peccia/tvm that referenced this pull request Mar 27, 2023
…0.04 (apache#13736)

- Updates nrfjprog to latest (10.18.1) to fix issues related to running microTVM tests inside docker. The main issue was that with older version commands like nrfjprog --com was not working even with docker --privileged.
- Update python install script to support python 3.7 in 20.04 -> we keep using python 3.7 in ubuntu 20.04 until its EOL
- Add script to install LLVM in ubuntu 20.04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants