The new Authentication TCK old-tck-runner/pom.xml is missing the old Authentication TCK

[scottmarlow]

jakartaee/specifications#460 referenced the new and (staged) old Authentication TCK:

The URL of the staging directory on downloads.eclipse.org for the proposed EFTL TCK binary:
https://download.eclipse.org/ee4j/jakartaee-tck/jakartaee10/staged/eftl/jakarta-authentication-tck-3.0.0-dist.zip
https://download.eclipse.org/ee4j/jakartaee-tck/jakartaee10/staged/eftl/jakarta-authentication-tck-3.0.0.zip [part of dist, loaded as bundle]

The new Authentication TCK got promoted to https://download.eclipse.org/jakartaee/authentication/3.0/jakarta-authentication-tck-3.0.0.zip, which contains the old-tck-runner/pom.xml for running old Authentication TCK.

The old Authentication 3.0 TCK https://download.eclipse.org/ee4j/jakartaee-tck/jakartaee10/staged/eftl/jakarta-authentication-tck-3.0.0.zip should also be published to https://download.eclipse.org/jakartaee/authentication/3.0 but didn't get published yet.

To add to the confusion around this, I downloaded https://download.eclipse.org/jakartaee/authentication/3.0/jakarta-authentication-tck-3.0.0.zip and ran the old-tck-runner/pom.xml which downloaded the new TCK instead of the old Authenticiation TCK.

Should old-tck-runner/pom.xml instead download the old (not yet published) Authentication TCK?

CC @arjantijms @starksm64 @alwin-joseph

[arjantijms]

Btw, the old/new archive thing was done as a time saver as we were so close to the deadline and it (kinda) worked this way. We probably should transfer the old authentication TCK code to the authentication project in the same was as was done for Security and Faces, and build a single bundle for a service release.

[scottmarlow]

We probably should transfer the old authentication TCK code to the authentication project in the same was as was done for Security and Faces, and build a single bundle for a service release.

Would that be using the old authentication TCK (Ant + Java Test Harness based) tests as is?

Just as a point of reference, the old https://download.eclipse.org/ee4j/jakartaee-tck/jakartaee10/staged/eftl/jakarta-authentication-tck-3.0.0.zip contents are shown at https://gist.github.com/scottmarlow/f6a9fb1ea753a6ff973f93395f95148d

[arjantijms]

Would that be using the old authentication TCK (Ant + Java Test Harness based) tests as is?

Yes, essentially having what is now a separate bundle internally in the one and only new bundle.

[alwin-joseph]

Btw, the old/new archive thing was done as a time saver as we were so close to the deadline and it (kinda) worked this way. We probably should transfer the old authentication TCK code to the authentication project in the same was as was done for Security and Faces, and build a single bundle for a service release.

Created #167 for transferring the old tck. Hopefully a new service release can be done after this changes are integrated.

Also the Platform requirement will need to be mentioned to run the old tck tests in platform mode(full and web profile) as mentioned here : #167 (comment) .

Does the new Authentication TCK tests need to be executed for platform compatibility too ?

[arjantijms]

Does the new Authentication TCK tests need to be executed for platform compatibility too ?

Yes, that would indeed be needed.

[scottmarlow]

Also the Platform requirement will need to be mentioned to run the old tck tests in platform mode(full and web profile) as mentioned here : #167 (comment) .

There will be an Authentication TCK service release that brings the old Authentication TCK tests in so that Platform implementations could run the old tests for their own validation but not as a EE 10 Platform requirement since the old Auth TCK tests will not be validated until after the start of the EE 10 Platform ballot.

Does this make sense @arjantijms

[arjantijms]

Makes perfect sense Scott, I'll get to it soon.

[scottmarlow]

@arjantijms during the Platform TCK call, we discussed how the Authentication 3.0 Ballot referenced (old + new standalone) TCK bundles but as we know, the old TCK bundle didn't get released.

I'm going to reach out to the Specification team to get their input on how we proceed with regard to addressing ^.

[scottmarlow]

@arjantijms in https://www.eclipse.org/lists/jakarta.ee-spec/msg02650.html, IVar states that if there is agreement, he can publish the missing https://download.eclipse.org/ee4j/jakartaee-tck/jakartaee10/staged/eftl/jakarta-authentication-tck-3.0.0.zip file that was previously created by the Platform TCK when it still contained the Authentication TCK tests (e.g. [9.1 branch still has them](https://github.com/eclipse-ee4j/jakartaee-tck/tree/9.1.x/src/com/sun/ts/tests/jaspic for reference)). He will also update the Authentication 3.0 spec page to link to both TCK zips.

Your input is of course welcome directly on the spec mailing list conversation.

The previous response from Ed Bratt mentions that we need to resolve where the old TCK source is maintained for the purpose of resolving Authentication 3.0 TCK challenges. Currently that is checked into https://github.com/jakartaee/authentication but has been removed from the github.com/eclipse-ee4j/jakartaee-tck repository. In the interest of deciding which source repo is to be used to handle Authentication 3.0 (old) TCK challenges, do you agree that either repo could be used as the source of truth for handling (old) TCK challenges?

Regardless of which source repo contains the truth for handling (old) TCK challenges, I think we need to validate that building the (old) TCK from that repo can produce an identical TCK archive. Do you agree? If the only difference is EPL license in root versus EFTL, that is fine.

cc @alwin-joseph

[alwin-joseph]

FYI we executed the authentication TCK with latest glassfish nightly bundles for Platform Certification. If the new service release to integrate the old tck is done we can rerun the same by modifying the job config. The old tck is yet to be released as mentioned in previous comment.

Authentication TCK Run against Glassfish (Includes new + old tck tests)
old tck bundle used for below run : https://download.eclipse.org/ee4j/jakartaee-tck/jakartaee10/staged/eftl/jakarta-authentication-tck-3.0.0.zip

JDK17
Web profile : https://ci.eclipse.org/jakartaee-tck/job/10/job/jakarta-authentication-tck-glassfish-run-jdk17/4/
Full Profile : https://ci.eclipse.org/jakartaee-tck/job/10/job/jakarta-authentication-tck-glassfish-run-jdk17/3/

JDK11
Web Profile: https://ci.eclipse.org/jakartaee-tck/job/10/job/jakarta-authentication-tck-glassfish-run/33/
Full Profile : https://ci.eclipse.org/jakartaee-tck/job/10/job/jakarta-authentication-tck-glassfish-run/30/

Full profile : 70 new tests + 68 old tck tests
Web profile : 70 new tests + 61 old tck tests
We used the keywords parameter for running the different set of tests in old tck as expected. The same set of tests were run for platform certification in the previous releases.

From the Jenkins job config :

if [[ ${PROFILE} == "web" ]];then
	sed -i.bak "s#<artifactId>glassfish</artifactId>#<artifactId>web</artifactId>#g" pom.xml
    sed -i.bak "s#<artifactId>glassfish</artifactId>#<artifactId>web</artifactId>#g" old-tck-runner/pom.xml
    sed -i.bak "s#<includeArtifactIds>glassfish</includeArtifactIds>#<includeArtifactIds>web</includeArtifactIds>#g" old-tck-runner/pom.xml
    sed -i.bak "s#<arg value=\"runclient\" if:set=\"run.test\" />#<arg value=\"runclient\" if:set=\"run.test\" />  <arg value=\"-Dkeywords=jaspic_web_profile\" /> #g" old-tck-runner/pom.xml
else
    sed -i.bak "s#<arg value=\"runclient\" if:set=\"run.test\" />#<arg value=\"runclient\" if:set=\"run.test\" />  <arg value=\"-Dkeywords=jaspic_baseline|javaee\" /> #g" old-tck-runner/pom.xml
fi