Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed test step result sync, made it independent from updating test issues #17

Merged
merged 11 commits into from
Nov 2, 2022
77 changes: 49 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,11 @@ A list of other mapping implementations.

#### DefaultSummaryMapper

This maps Java test methods to Jira *Tests* and Java classes to Jira *Test Sets* by their name, when no keys are present in the annotations. Additionally, it creates the issues when they don't exist. You enable that feature by passing that mapper in your `XrayResultsSynchronizer`.
This maps Java test methods to Jira *Tests* and Java classes to Jira *Test Sets* by their name, when no keys are present in the annotations.

Please note, that this mapper creates the issues when they don't exist! See above for more details how it's work.

You enable that feature by passing that mapper in your `XrayResultsSynchronizer`.

```java
public class MyXrayResultsSynchronizer extends AbstractXrayResultsSynchronizer {
Expand Down Expand Up @@ -227,60 +231,64 @@ Please note, that
- `queryTest` is also called if you use `@XrayTest` annotation, but without key attribute
- `queryTestSet` is also called if you `@XrayTestSet` annotation, but without key attribute

#### Update entities
#### Creating new entities

The `XrayMapper` also provides callbacks for updating entities.
By default, the Xray connector doesn't create any issues. You can enable that by passing `true` in the interface.

Please note, that existing issues will be updated automatically. All manual changes like test steps will be overwritten.

```java
public class GenericMapper implements XrayMapper {

@Override
public void updateTestExecution(XrayTestExecutionIssue xrayTestExecutionIssue, ExecutionContext executionContext) {
xrayTestExecutionIssue.getTestEnvironments().add("Test");
xrayTestExecutionIssue.setFixVersions(List.of(new JiraNameReference("1.0")));
public boolean shouldCreateNewTestSet(ClassContext classContext) {
return true;
}

@Override
public void updateTestSet(XrayTestSetIssue xrayTestSetIssue, ClassContext classContext) {
xrayTestSetIssue.getLabels().add("TestAutomation");
public boolean shouldCreateNewTest(MethodContext methodContext) {
return true;
}

@Override
public void updateTest(XrayTestIssue xrayTestIssue, MethodContext methodContext) {
xrayTestIssue.getLabels().add("TestAutomation");
public String getDefaultTestIssueSummery(MethodContext methodContext) {
return String.format("%s_%s", methodContext.getClassContext().getName(), methodContext.getName());
}
}
```

You can use these methods to update the Jira issues right before importing. Please mind, that not all features are supported by the [Xray import API](#references).
If you create new test issues, Xray connector will use the method `getDefaultTestIssueSummery` to generate a new issue summary.

#### Creating new entities
In the example above new created test issues get the summary according to the format `<TestClass_TestMethod>` , e.g. `MyTestClass_testSomething`.

By default, the Xray connector doesn't create any issues. You can enable that by passing `true` in the interface.
#### Updating existing entities

The `XrayMapper` also provides callbacks for updating entities.

To update Xray testsets and test issues you have to allow creating new issues (see [Creating new entities](#creating-new-entities)).

```java
public class GenericMapper implements XrayMapper {

@Override
public boolean shouldCreateNewTestSet(ClassContext classContext) {
return true;
public void updateTestExecution(XrayTestExecutionIssue xrayTestExecutionIssue, ExecutionContext executionContext) {
xrayTestExecutionIssue.getTestEnvironments().add("Test");
xrayTestExecutionIssue.setFixVersions(List.of(new JiraNameReference("1.0")));
}

@Override
public boolean shouldCreateNewTest(MethodContext methodContext) {
return true;
public void updateTestSet(XrayTestSetIssue xrayTestSetIssue, ClassContext classContext) {
xrayTestSetIssue.getLabels().add("TestAutomation");
}

@Override
public String getDefaultTestIssueSummery(MethodContext methodContext) {
return String.format("%s_%s", methodContext.getClassContext().getName(), methodContext.getName());
public void updateTest(XrayTestIssue xrayTestIssue, MethodContext methodContext) {
xrayTestIssue.getLabels().add("TestAutomation");
}
}
```

If you create new test issues, Xray connector will use the method `getDefaultTestIssueSummery` for generate new issue summary.

In the example above new created test issues get the summery according the format `<TestClass_TestMethod>` like `MyTestClass_testSomething`.
You can use these methods to update the Jira issues right before importing. Please mind, that not all features are supported by the [Xray import API](#references).

#### How to use JqlQuery

Expand Down Expand Up @@ -390,9 +398,9 @@ You can retrieve these IDs directly from a Jira by **editing** an Xray **Test Ex
|xray.webresource.filter.getrequestsonly.enabled|false|Enable this for debugging to avoid PUT/POST/DELETE requests sent to Jira|
|xray.webresource.filter.getrequestsonly.fake.response.key|FAKE-666666|This key will returned, when `xray.webresource.filter.getrequestsonly.enabled` set to `true` and PUT/POST/DELETE request was sent.|

---
## Additional information

## Troubleshooting
### Troubleshooting

Hints for the following occuring symptoms:

Expand All @@ -401,6 +409,23 @@ Symptom | Explanation | Solution
`{"error": "...java.sql.SQLIntegrityConstraintViolationException: ORA-00001: unique constraint (JIRA_SCHEMA.SYS_C00134897) violated` | An issue could not be imported because it already exists. | Make sure that the issue key for an existing issue could be found via. the `query()` methods of the `XrayMapper`.
`{"errorMessages":["We can't create this issue for you right now, it could be due to unsupported content you've entered into one or more of the issue fields...` | Missing data on the issue. | Try to create an issue manually, call the REST API for this issue and check which fields are set by default. |


### References

1. Import Xray results: https://docs.getxray.app/display/XRAY/Import+Execution+Results
2. Xray JSON import format: https://docs.getxray.app/display/XRAY/Import+Execution+Results#ImportExecutionResults-XrayJSONformat
3. Jira REST API: https://developer.atlassian.com/server/jira/platform/jira-rest-api-examples/

### Known issues

1. Adding labels for a new Test execution will be ignored because it is not supported by Xray JSON import format.
2. Using ``DefaultSummeryMapper`` and __identical__ test method names:
* The xray connector could create issues with identical summaries and can sync the results.
* On the second run Xray connector find existing issue with the given summary = method name. Xray connector uses the first find, it cannot distinguish the other issues with the same summary to the correct Xray issues.
* Please note, that in that case the sync is not working as expected. Please extend the mapping, e.g. <classname_methodname>

---

## Publication

This module is deployed and published to Maven Central. All JAR files are signed via Gradle signing plugin.
Expand All @@ -422,10 +447,6 @@ If all properties are set, call the following to build, deploy and release this
gradle publish closeAndReleaseRepository
````

## References

1. Import Xray results: https://docs.getxray.app/display/XRAY/Import+Execution+Results

## Code of Conduct

This project has adopted the [Contributor Covenant](https://www.contributor-covenant.org/) in version 2.0 as our code of conduct. Please see the details in our [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md). All contributors must abide by the code of conduct.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
/*
* Testerra Xray-Connector
*
* (C) 2022, Martin Großmann, T-Systems Multimedia Solutions GmbH, Deutsche Telekom AG
*
* Deutsche Telekom AG and all other contributors /
* copyright owners license this file to you under the Apache
* License, Version 2.0 (the "License"); you may not use this
* file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*
*/
package eu.tsystems.mms.tic.testerra.plugins.xray.mapper.jira;

import java.util.Arrays;
import java.util.List;

/**
* Created on 2022-10-11
*
* @author mgn
*/
public class JiraError {

martingrossmann marked this conversation as resolved.
Show resolved Hide resolved
private String[] messages;
private String summary;
private String project;

public List<String> getMessages() {
return Arrays.asList(messages);
}

public String getSummary() {
return summary;
}

public String getProject() {
return project;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@

import com.fasterxml.jackson.annotation.JsonFormat;
import eu.tsystems.mms.tic.testerra.plugins.xray.jql.predefined.TestType;
import eu.tsystems.mms.tic.testerra.plugins.xray.mapper.jira.JiraError;
import eu.tsystems.mms.tic.testerra.plugins.xray.mapper.jira.JiraIssue;
import eu.tsystems.mms.tic.testerra.plugins.xray.mapper.jira.JiraKeyReference;
import eu.tsystems.mms.tic.testerra.plugins.xray.mapper.jira.JiraNameReference;
Expand Down Expand Up @@ -91,14 +92,20 @@ public ResultTestIssueImport getTestIssues() {

public static class ResultTestIssueImport {

private JiraKeyReference[] success;
private JiraKeyReference[] success = {};

private JiraError[] error = {};

public ResultTestIssueImport() {
}

public List<JiraKeyReference> getSuccess() {
return Arrays.asList(success);
}

public List<JiraError> getError() {
return Arrays.asList(this.error);
}
}

public static class Info extends AbstractInfo {
Expand Down Expand Up @@ -327,17 +334,6 @@ public void addEvidence(Evidence evidence) {
public TestRun() {
}

public TestRun(JiraIssue issue) {
this(issue.getKey());
this.testInfo = new Info();
this.testInfo.setDescription(issue.getDescription());
this.testInfo.setSummary(issue.getSummary());
this.testInfo.setLabels(issue.getLabels());
this.testInfo.setDefinition(issue.getSummary());
this.testInfo.setType(TestType.AutomatedGeneric);
this.testInfo.setProjectKey(issue.getProject().getKey());
}

public TestRun(String testKey) {
this.testKey = testKey;
this.setStatus(Status.FAIL);
Expand Down
Loading