Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the jobs status script so it works with the new gbasf2 version #209

Merged
merged 3 commits into from
Nov 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 14 additions & 9 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,19 +8,22 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm

## Unreleased

### Fixed
### Added

[#203](https://github.com/nils-braun/b2luigi/pull/203) @AlexanderHeidelbach
* **gbasf2:** Fix `gbasf2_setup_path` setting not being passed through in some function calls.
* **gbasf2:** Local basf2 log level setting is now passed over to the grid jobs. You can now limit the log size of jobs with many warnings via `basf2.set_log_level(basf2.LogLevel.ERROR)`. This could fix some errors due to too large log sizes. Implemented by pickling local `basf2.logging.log_level`.

### Changed
### Fixed

* **gbasf2**: Make gbasf2 wrapper work with gbasf2 release `v5.8.2`:
- Change the default gbasf2 setup script path to CVMFS location in gbasf2 v5.8.2, i.e.
```
/cvmfs/belle.kek.jp/grid/gbasf2/pro/bashrc
```
Reminder that this can still be customized via the `gbasf2_setup_path` setting.
[#207](https://github.com/nils-braun/b2luigi/pull/207)
- Fix script to get gbasf2 project status as JSON for latest gbasf2 release. [#209](https://github.com/nils-braun/b2luigi/pull/209) @eckerpatrick

* **gbasf2:** Change the default gbasf2 setup script path to CVMFS location in gbasf2 v5.8.2, i.e.
```
/cvmfs/belle.kek.jp/grid/gbasf2/pro/bashrc
```
Reminder that this can still be customized via the `gbasf2_setup_path` setting. Resolves issue ![#206](https://github.com/nils-braun/b2luigi/issues/206).
* **gbasf2:** Fix `gbasf2_setup_path` setting not being passed through in some function calls. [#203](https://github.com/nils-braun/b2luigi/pull/203) @AlexanderHeidelbach

### Removed

Expand All @@ -33,6 +36,8 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
### Added

* **gbasf2:** New setting `gbasf2_setup_path` which can be used to customize the path to the gbasf2 setup file directly (default: `"/cvmfs/belle.kek.jp/grid/gbasf2/pro/tools/setup.sh"`). It is a more flexible replacement for the `gbasf2_install_directory` setting, which will be removed in the future, since we can't predict potential name and path changes of the setup script between gbasf2 releases. @meliache [#162](https://github.com/nils-braun/b2luigi/pull/162)
* **gbasf2:** Raise error for all non-zero return codes of ``gbas2_job_status.py``. This should result in better and earlier error messages if getting the job statuses in a gbasf2 project fails due to unexpected reasons.


### Fixed

Expand Down
4 changes: 4 additions & 0 deletions b2luigi/batch/processes/gbasf2.py
Original file line number Diff line number Diff line change
Expand Up @@ -1091,6 +1091,10 @@ def get_gbasf2_project_job_status_dict(
f"\nCould not find any jobs for project {gbasf2_project_name} on the grid.\n"
+ "Probably there was an error during the project submission when running the gbasf2 command.\n"
)
if proc.returncode > 0:
raise subprocess.CalledProcessError(
proc.returncode, job_status_command, output=proc.stdout, stderr=proc.stderr
)
job_status_json_string = proc.stdout
return json.loads(job_status_json_string)

Expand Down
4 changes: 3 additions & 1 deletion b2luigi/batch/processes/gbasf2_utils/gbasf2_job_status.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@
import datetime

from BelleDIRAC.gbasf2.lib.job.information_collector import InformationCollector
from BelleDIRAC.Client.helpers.auth import userCreds
from BelleDIRAC.gbasf2.lib.auth import userCreds
from BelleDIRAC.Client.helpers.common import initializeCS


class JobStatusEncoder(json.JSONEncoder):
Expand Down Expand Up @@ -74,5 +75,6 @@ def get_job_status_dict(project_name, user_name, group_name):
parser.add_argument('-u', '--user', type=str, default=None, help="grid username")
parser.add_argument('-g', '--group', type=str, default="belle", help="gbasf2 group name")
args = parser.parse_args()
initializeCS()
job_status_dict = get_job_status_dict(args.project, args.user, args.group)
print(json.dumps(job_status_dict, cls=JobStatusEncoder))