Support suffix for test classes when using suite.

[xpconanfan]

So the same test class can be run with different configs and reported differently in the same suite.

Aliases are only possible in a suite as they are set when calling add_test_class.

---

This change is 

[winterfroststrom]

Review status: 0 of 2 files reviewed, 1 unresolved discussion (waiting on @xpconanfan, @winterfroststrom, and @k2fong)

---

tests/mobly/test_runner_test.py, line 292 at r1 (raw file):

        record1 = tr.results.executed[0]
        record2 = tr.results.executed[1]
        self.assertEqual(record1.test_class, 'IntegrationTest-FirstConfig')

I don't see a check for the base functionality, so add a new unit test or add asserts to test_run_two_test_classes

---

Comments from Reviewable

[xpconanfan]

Review status: 0 of 2 files reviewed, 1 unresolved discussion (waiting on @xpconanfan, @winterfroststrom, and @k2fong)

---

tests/mobly/test_runner_test.py, line 292 at r1 (raw file):

Previously, winterfroststrom wrote…

I don't see a check for the base functionality, so add a new unit test or add asserts to test_run_two_test_classes

what is the "base functionality"?

---

Comments from Reviewable

[winterfroststrom]

Review status: 0 of 2 files reviewed, 1 unresolved discussion (waiting on @winterfroststrom and @k2fong)

---

tests/mobly/test_runner_test.py, line 292 at r1 (raw file):

Previously, xpconanfan (Ang Li) wrote…

what is the "base functionality"?

that the tag is set to the class name?

---

Comments from Reviewable

[winterfroststrom]

Review status: 0 of 2 files reviewed, 1 unresolved discussion (waiting on @winterfroststrom and @k2fong)

---

tests/mobly/test_runner_test.py, line 292 at r1 (raw file):

Previously, winterfroststrom wrote…

that the tag is set to the class name?

err, I should say that the record.test_class is the class name

---

Comments from Reviewable

[xpconanfan]

Review status: 0 of 2 files reviewed, 1 unresolved discussion (waiting on @xpconanfan, @winterfroststrom, and @k2fong)

---

tests/mobly/test_runner_test.py, line 292 at r1 (raw file):

Previously, winterfroststrom wrote…

err, I should say that the record.test_class is the class name

isn't that what this assert equal checks?

---

Comments from Reviewable

[winterfroststrom]

Review status: 0 of 2 files reviewed, 1 unresolved discussion (waiting on @winterfroststrom and @k2fong)

---

tests/mobly/test_runner_test.py, line 292 at r1 (raw file):

Previously, xpconanfan (Ang Li) wrote…

isn't that what this assert equal checks?

This assert checks that you can change the default functionality to set a custom class alias, it does not check that the original class name is as expected

---

Comments from Reviewable

[k2fong]

Review status: 0 of 2 files reviewed, 2 unresolved discussions (waiting on @xpconanfan, @winterfroststrom, and @k2fong)

---

tests/mobly/test_runner_test.py, line 293 at r1 (raw file):

IntegrationTest-SecondConfig

Will the original test class name be preserve somewhere in the output? or will it just be the alias if specified?

---

Comments from Reviewable

[xpconanfan]

Review status: 0 of 2 files reviewed, 2 unresolved discussions (waiting on @xpconanfan, @winterfroststrom, and @k2fong)

---

tests/mobly/test_runner_test.py, line 293 at r1 (raw file):

Previously, k2fong wrote…

IntegrationTest-SecondConfig

Will the original test class name be preserve somewhere in the output? or will it just be the alias if specified?

Just the alias. It changes how the class is referred to.

---

Comments from Reviewable

[xpconanfan]

Review status: 0 of 2 files reviewed, 2 unresolved discussions (waiting on @winterfroststrom and @k2fong)

---

tests/mobly/test_runner_test.py, line 292 at r1 (raw file):

Previously, winterfroststrom wrote…

This assert checks that you can change the default functionality to set a custom class alias, it does not check that the original class name is as expected

Done.

---

Comments from Reviewable

[winterfroststrom]

---

Review status: 0 of 2 files reviewed, 1 unresolved discussion (waiting on @k2fong)

---

Comments from Reviewable

[winterfroststrom]

Review status: 0 of 2 files reviewed, 2 unresolved discussions (waiting on @k2fong and @xpconanfan)

---

tests/mobly/test_runner_test.py, line 266 at r2 (raw file):

        record2 = tr.results.executed[1]
        self.assertEqual(record1.test_class, 'IntegrationTest')
        self.assertEqual(record1.test_class, 'IntegrationTest')

Wait, actually this check seems wrong

Additionally, Integration2Test is added first

---

Comments from Reviewable

[k2fong]

Review status: 0 of 2 files reviewed, 2 unresolved discussions (waiting on @k2fong and @xpconanfan)

---

tests/mobly/test_runner_test.py, line 293 at r1 (raw file):

Previously, xpconanfan (Ang Li) wrote…

Just the alias. It changes how the class is referred to.

Will it make sense to preserve the original class name and append (or some reader friendly way) the extra string from the user to differentiate the different tests instead of replacing?

---

Comments from Reviewable

[xpconanfan]

Review status: 0 of 2 files reviewed, 2 unresolved discussions (waiting on @xpconanfan and @k2fong)

---

tests/mobly/test_runner_test.py, line 293 at r1 (raw file):

Previously, k2fong wrote…

Will it make sense to preserve the original class name and append (or some reader friendly way) the extra string from the user to differentiate the different tests instead of replacing?

It sounds good, but I can't think of a use case.
Also it requires adding field in the record obj, so I'm not sure.
What do you think?

---

Comments from Reviewable

[k2fong]

Review status: 0 of 2 files reviewed, 2 unresolved discussions (waiting on @k2fong and @xpconanfan)

---

tests/mobly/test_runner_test.py, line 293 at r1 (raw file):

Previously, xpconanfan (Ang Li) wrote…

It sounds good, but I can't think of a use case.
Also it requires adding field in the record obj, so I'm not sure.
What do you think?

The use case will be same as this pull request. On a different system I have seem them do this

IntegrationTest [FirstConfig]
IntegrationTest [SecondConfig]

user can pass in what goes into the bracket.

If there might be worry about adding field, couldn't the new appended name just replace the default just like you have done for alias?

---

Comments from Reviewable

[winterfroststrom]

Review status: 0 of 2 files reviewed, 1 unresolved discussion (waiting on @xpconanfan and @k2fong)

---

tests/mobly/test_runner_test.py, line 293 at r1 (raw file):

Previously, k2fong wrote…

The use case will be same as this pull request. On a different system I have seem them do this

IntegrationTest [FirstConfig]
IntegrationTest [SecondConfig]

user can pass in what goes into the bracket.

If there might be worry about adding field, couldn't the new appended name just replace the default just like you have done for alias?

why is it better for mobly to do this rather than the user?
Since they can get the class name and set the logged name arbitrarily, so they could just implement it like that?

---

Comments from Reviewable

[k2fong]

Review status: 0 of 2 files reviewed, 1 unresolved discussion (waiting on @xpconanfan and @k2fong)

---

tests/mobly/test_runner_test.py, line 293 at r1 (raw file):

Previously, winterfroststrom wrote…

why is it better for mobly to do this rather than the user?
Since they can get the class name and set the logged name arbitrarily, so they could just implement it like that?

agree, user can do it them self, but having mobly do this can enforce some consistency across all users.

---

Comments from Reviewable

[winterfroststrom]

Review status: 0 of 2 files reviewed, 1 unresolved discussion (waiting on @xpconanfan and @k2fong)

---

tests/mobly/test_runner_test.py, line 293 at r1 (raw file):

Previously, k2fong wrote…

agree, user can do it them self, but having mobly do this can enforce some consistency across all users.

Hmm, there's question is what the consistency buys us.

If we ever want to add some analysis/report generation stuff for example,
then consistency could be helpful as it can establiss invariants/assumptions.

I think I could go either way on this.

---

Comments from Reviewable

[xpconanfan]

Reviewable status: 0 of 4 files reviewed, 1 unresolved discussion (waiting on @k2fong)

---

tests/mobly/test_runner_test.py, line 293 at r1 (raw file):

Previously, winterfroststrom wrote…

Hmm, there's question is what the consistency buys us.

If we ever want to add some analysis/report generation stuff for example,
then consistency could be helpful as it can establiss invariants/assumptions.

I think I could go either way on this.

Done.

[k2fong]

Reviewable status: 0 of 4 files reviewed, 1 unresolved discussion (waiting on @k2fong)