Add LongBench validation #1220
Add LongBench validation #1220
Conversation
6c96bd0 to
eefe4f2
Compare
|
is it still valid? |
000d2a4 to
7eb1e54
Compare
6c3e950 to
becdf78
Compare
d71206d to
15d3c77
Compare
| LongBenchTestData("trec", 3.2, 2.0, 3.3), | ||
| LongBenchTestData("qasper", 5.8, 1.7, 3.6), | ||
| ]) | ||
| def test_optimized_generation_longbench(qwen2_converted_model, test_struct): |
There was a problem hiding this comment.
is it explicitly not marked as @pytest.mark.precommit ?
how long does this test take?
There was a problem hiding this comment.
Yes
On my local machine it takes 24 minutes, but on CI it takes about 60 minutes.
There was a problem hiding this comment.
Can we run this test in nightly jobs?
There was a problem hiding this comment.
yes, please add mark for nightly
| @@ -0,0 +1,245 @@ | |||
| import re | |||
There was a problem hiding this comment.
This code is probably taken from 3rd party repo. We need to keep the license in this case.
There was a problem hiding this comment.
I would suggest to check if Protex will actually catch it. In general it's not a problem if the license allows reusing code. In that case you need clearly state that in your file.
15d3c77 to
1e88d10
Compare
| # Copyright (C) 2023-2025 Intel Corporation | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # | ||
| # This file includes utility functions copied from the LongBench repository: | ||
| # https://github.com/THUDM/LongBench | ||
| # | ||
| # Copyright (c) 2023 THU-KEG & Zhipu AI | ||
| # Licensed under the MIT License |
There was a problem hiding this comment.
@moslex please take a look at the license. Could you confirm if it is sufficient for code adapted from a 3rd party repo?
There was a problem hiding this comment.
Yes, you need to keep the original copyrights and should mark the entire file as Apache 2.0 to be compliant with the OpenVINO license. This is in case you copy-paste some code and make your own changes. Everything is good now.
If you copy-paste the full file/component without any modifications, it is better to keep its original license and move this file/component to a separate folder named "thirdparty" and update the thirdparty.txt file by adding the license for this component.
There was a problem hiding this comment.
@l-bat , you can also check how it's done in NNCF. I can guide you if needed.
315576d to
85b5e32
Compare
No description provided.