Implement example archive importer

[edan-bainglass]

This PR implements a new notebook (linked from the home page via new button) to import example calculations from aiidalab-qe-examples. The notebook includes:

• Documentation/instructions
• A selector of examples populated from a repo-fetched examples list
• An import button, an dynamic info message reflecting the current operation
• A link to the calculation history
• A collapsible log of verdi archive import

Future improvements

• ArchiveImporter could be made more general, allowing imports from any URL (MC for example)
• Move ArchiveImporter to AWB
• Stream import log

Closes #1117

[codecov]

Codecov Report

Attention: Patch coverage is 18.03279% with 50 lines in your changes missing coverage. Please review.

Project coverage is 71.99%. Comparing base (192050d) to head (60e7ec5).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/aiidalab_qe/common/widgets.py	18.03%	50 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1126      +/-   ##
==========================================
- Coverage   72.49%   71.99%   -0.50%     
==========================================
  Files         105      105              
  Lines        6810     6871      +61     
==========================================
+ Hits         4937     4947      +10     
- Misses       1873     1924      +51     

Flag	Coverage Δ
python-3.11	71.99% <18.03%> (-0.50%)	⬇️
python-3.9	72.00% <18.03%> (-0.50%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[mikibonacci]

I will put, in the readme of the aiidalab-qe-examples package, an instruction on how to add a line in the examples_list.txt file, so that this parsing will always work also in the future.

[edan-bainglass]

Note that I committed some mods to the examples repo to simplify fetching.

[mikibonacci]

This download will happen in the qe-app root folder, but I am not sure it is the recommended way: if the download or import is interrupted (closing the notebook - the case I tried - , or any other reason), the file will stay there because we don't run the delete_archive of the widget. A issue can be that then we have then storage occupied by this file.

@edan-bainglass, do you agree with this? maybe I am missing something in the code.

My suggestion is:
If we do the download and import from a temporary directory, we can automatically skip the need of deleting the file (so the last call, delete_archive, in all cases (successfull or not). Is there any reason which we cannot do the process in a tempdir?

I suggest to add destination and source input arguments to download_archive and import_archive, and run them to use a tempdir:

def import_examples(self, _):
    if self.logger and self.clear_log_on_import:
        self.logger.value = ""
    for archive in self.selector.value:
        with tempfile.TemporaryDirectory() as tempdir:
            if self.download_archive(filename=archive, destination=tempdir):
                self.import_archive(filename=archive, source=tempdir)
                #self.delete_archive(archive)  <--- no need for this

then, the filename variable in the two methods should be probably changed to be new_filename_or_path = pathlib.Path(destination or source) / <filename>

[edan-bainglass]

No objections. This is great. I forget tempdir exists 😅 Will implement 👍

[edan-bainglass]

So I ended up bypassing the download altogether, as verdi archive import can directly import from a URL! Very nice 👌

[mikibonacci]

Great! Even better!

[mikibonacci]

Hi @edan-bainglass, this is great! I just added a comment on the possibility to do the download in a temporary directory, so we avoid unwanted partially downloaded files in the qe-app root folder, in case something goes wrong.

If the change is not needed, we can discuss about it and I will remove the request.

Thanks!

[edan-bainglass]

Thanks @mikibonacci. Yes, let's switch to tempdir. Will update the PR.

update

Skipped the download in favor of importing directly from the URL 👍

[mikibonacci]

Hi @edan-bainglass, LGTM!

[edan-bainglass]

Hi @edan-bainglass, LGTM!

Thanks @mikibonacci. I just pushed an update with a link to the calculation history. Please give it a quick review 🙏 Also, if you can please quickly review #1124 (on which this is based) and its aiidalab-home dependency PR 🙏🙏

[mikibonacci]

Hi @edan-bainglass, LGTM!