[OTX][HOT-FIX] Revert otx cli tools with data.yaml check

[harimkang]

Summary

Due to the effect of that #1497, data.yaml of otx-workspace becomes unusable, so I revert this.

[cih9088]

The problem of original implementation was that ./data.yaml used while testing sometimes does not get removed.
If ./data.yaml is not removed, all the following tests would be failed. I didn't look into the root cause though.
Revert it for now and if a similar issue comes up, lets fix them then.

[harimkang]

The problem of original implementation was that ./data.yaml used while testing sometimes does not get removed. If ./data.yaml is not removed, all the following tests would be failed. I didn't look into the root cause though. Revert it for now and if a similar issue comes up, lets fix them then.

Oh, there was such an issue. Let me take a look.

[sungmanc]

@harimkang @cih9088 , Did you execute the parallel tests ? (conduct >= 2 classification/detection/segmentation tests at the same time).

When I executed 2 task tests, I got the same error as @cih9088 's said.

[harimkang]

@harimkang @cih9088 , Did you execute the parallel tests ? (conduct >= 2 classification/detection/segmentation tests at the same time).

When I executed 2 task tests, I got the same error as @cih9088 's said.

Hi @sungchul2, I think the issue is caused by the function used in the test below. Could you please confirm?

training_extensions/tests/integration/cli/classification/test_classification.py

Line 615 in 3e5ff26

def set_dummy_data(func):

[cih9088]

When I executed 2 task tests, I got the same error as @cih9088 's said.

Yes, exactly. As I said, if you have .data.yaml in your CWD, because of whatever reasons, like pytest didn't remove it properly or one executed tests parallel, which means at some point you got data.yaml in your CWD, you would have an error.

[harimkang]

When I executed 2 task tests, I got the same error as @cih9088 's said.

Yes, exactly. As I said, if you have .data.yaml in your CWD, because of whatever reasons, like pytest didn't remove it properly or one executed tests parallel, which means at some point you got data.yaml in your CWD, you would have an error.

As sungman said, if we run multiple tests at the same time, it seems to be reproduced. I think maybe the cause I found is correct. I guess this needs to be addressed as I don't think test should affect root.

[sungmanc]

When I executed 2 task tests, I got the same error as @cih9088 's said.

Yes, exactly. As I said, if you have .data.yaml in your CWD, because of whatever reasons, like pytest didn't remove it properly or one executed tests parallel, which means at some point you got data.yaml in your CWD, you would have an error.

Let's assume this case,

test process 1: classification test
test process 2: detection test

Then, test process 1 made data.yaml during the self-sl test. In the mean time, test process 2 might get error because there was data.yaml file made by test process 1.

So, if we don't use parallel tests, then it will be fine

[sungmanc]

When I executed 2 task tests, I got the same error as @cih9088 's said.

Yes, exactly. As I said, if you have .data.yaml in your CWD, because of whatever reasons, like pytest didn't remove it properly or one executed tests parallel, which means at some point you got data.yaml in your CWD, you would have an error.

As sungman said, if we run multiple tests at the same time, it seems to be reproduced. I think maybe the cause I found is correct. I guess this needs to be addressed as I don't think test should affect root.

Agreed. and this problem will be resolved after the merge of Datumaro PR. @sungchul2 , @cih9088 , @harimkang .
So, how about keep this as is?

Because, when we use the Datumaro, we don't need to directly make data.yaml for test case.

Making .yaml file will be removed

[sungchul2]

I agree. After Datumaro integration, I think we can remove the function.