Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge SAP_COM_0948 #4735

Closed
wants to merge 72 commits into from
Closed

Conversation

DanielMieg
Copy link
Member

Changes

  • Tests
  • Documentation

daskuznetsova and others added 30 commits September 27, 2023 14:59
…ntials config (#4600)

* added logging if unauthorized for github

* refactored

* fixed log message & added logging github response

* deleted extra log

* refactored log message
… commit to trust enterprise github instances (#4602)

* downloading ca cert bundle when added as config

* adding logging statements

* allowing bats test to handle ca cert

* adding info message

* hard coding file names

* including correct http client util bundle

* removing logging message not needed

* adding cert bundle to commit and push

* improving the condition to add ca cert in commit and push

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test
* add containerDockerfilePath param to multipleImages

* rename ContainerDockerfilePath param to DockerfilePath

* Fix trailing spaces

---------

Co-authored-by: Egor Balakin <egor.balakin@sap.com>
Co-authored-by: Vyacheslav Starostin <32613074+vstarostin@users.noreply.github.com>
…yer (#4604)

forward sourceRepositoryCredentialsId from groovy to go layer in the same way how this is done for the targetRepositoryCredentialsId
…anges (#4605)

* exporting generateConfig function and applying minor changes

* Added setConfigOptions to set configOptions variable.
Added possibility to set format output, json or yaml for now.

* Correcting mistake on cmd/getDefaults.go

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>

---------

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
…4595)

* added logic of fetching golang private packages for whitesource step and detectExecuteScan step

* changed logic of checking by config.PrivateModulesGitToken

* moved func prepareGolangPrivatePackages to golangBuild.go

* fix (gitOpsUpdateDeployment) add CA bundle options to plain clone and commit to trust enterprise github instances (#4602)

* downloading ca cert bundle when added as config

* adding logging statements

* allowing bats test to handle ca cert

* adding info message

* hard coding file names

* including correct http client util bundle

* removing logging message not needed

* adding cert bundle to commit and push

* improving the condition to add ca cert in commit and push

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test

* feat(kanikoExecute): add dockerfilePath param to multipleImages (#4569)

* add containerDockerfilePath param to multipleImages

* rename ContainerDockerfilePath param to DockerfilePath

* Fix trailing spaces

---------

Co-authored-by: Egor Balakin <egor.balakin@sap.com>
Co-authored-by: Vyacheslav Starostin <32613074+vstarostin@users.noreply.github.com>

* fix(helm): forward sourceRepositoryCredentialsId from groovy to go layer (#4604)

forward sourceRepositoryCredentialsId from groovy to go layer in the same way how this is done for the targetRepositoryCredentialsId

* feat(config): exporting generateConfig function and applying minor changes (#4605)

* exporting generateConfig function and applying minor changes

* Added setConfigOptions to set configOptions variable.
Added possibility to set format output, json or yaml for now.

* Correcting mistake on cmd/getDefaults.go

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>

---------

Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>

* moved func prepareGolangPrivatePackages to pkg/golang

---------

Co-authored-by: Akramdzhon Azamov <MY_NAME@example.com>
Co-authored-by: Andrei Kireev <andrei.kireev@sap.com>
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
Co-authored-by: Egor Balakin <14162703+m1ron0xFF@users.noreply.github.com>
Co-authored-by: Egor Balakin <egor.balakin@sap.com>
Co-authored-by: Vyacheslav Starostin <32613074+vstarostin@users.noreply.github.com>
Co-authored-by: Marcus Holl <marcus.holl@sap.com>
Co-authored-by: Jk1484 <35270240+Jk1484@users.noreply.github.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
#4579)

* enabling publish to only publish sub packages

* changing directory and then coming back to original after the publish runs

* searching the glob tar and npmrc in the current directory

* excluding build descriptor check and addtional target tool check

* changing the npm pack before publish to run only in sub packages

* removing commented code clean up

* adding the correct npm pack

* improve logging

* fix error handling and a bit style fix

* fix unit tests

* remove commented lines

* respecting build descriptor list when provided

* improve docu for the step param

* fixing linting issues

* improve docu

---------

Co-authored-by: Gulom Alimov <gulomjon.alimov@sap.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
…be stashed/unstashed (#4573)

* Add stash param to mavenBuild

* go generate

* Delete default stash group

---------

Co-authored-by: Anil Keshav <anil.keshav@sap.com>
* Add CertificateDownload func to certutils package

* Add customTlsCertificateLinks param for artifactPrepareVersion

* Add the possibility to provide custom certs for artifactPrepareVersion

* Update tests

* Return back build flags

* Return back build flags

* Update pkg/certutils/certutils.go

Apply suggestion from code review

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>

---------

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
* structure vault logs

* Update client.go
* Update default_pipeline_environment.yml

Add filterRegex parameter to jmeter configuration. Set it to an empty string as generated from pipelinesyntax reference in jenkins

* Update testsPublishResults.groovy

Add the filterRegex param to the publishJmeterReport plugin in testPublishResults

* Update documentation to include filterRegex

---------

Co-authored-by: Anil Keshav <anil.keshav@sap.com>
* Adapting unit test to run for packages with namespace

* Handling classes with namespace

* support namespace obj for ATC
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.15.0 to 0.17.0.
- [Commits](golang/net@v0.15.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…pload and export to node (#4624)

* Provide additional log message on successful upload and export to node

---------

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
…ity] (#4550)

* fix(deps): update module github.com/hashicorp/vault to v1.14.3 [security]

* update to v1.14.1 instead due to Go version

* github.com/hashicorp/vault@v1.14.1

* go mod tidy

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: jliempt <>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
* fix(cxone): 1st scan failed due to 403 forbiden

* fix(cxone): 1st scan failed due to 403 forbiden

* fix(cxone): 1st scan failed due to 403 forbiden
…nks parameter (#4631)

* Add conditions for customTlsCertificateLinks

* go generate
* Add new unified fields to Mend and Blackduck SARIF

* fmt project

---------

Co-authored-by: Dmitrii Pavlukhin <dmitrii.pavlukhin@sap.com>
* retry Vault request errors and add extensive logging

* fix sigsegv and change logging

* refining logs

* changing debug to info

---------

Co-authored-by: jliempt <>
Co-authored-by: Muhammadali Nazarov <muhammadalinazarov@gmail.com>
* Remove confusing comments in tmsClient.go
* Add warning message when cf native builds uses bg deployment

---------

Co-authored-by: Oliver Feldmann <oliver.feldmann@sap.com>
…hub (#4652)

* fixed unzipping db sources without db dir

* fixed tests
Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>
Co-authored-by: Gulom Alimov <gulomjon.alimov@sap.com>
…#4653)

* Initialize Splunk client if dsn or prodCriblEndpoint provided

* Fix tests

* Apply suggestions from code review

* go generate

---------

Co-authored-by: Christopher Fenner <26137398+CCFenner@users.noreply.github.com>
vstarostin and others added 26 commits November 27, 2023 17:04
…ions (#4685)

* Align build and job urls with what is expected by piper

* Add comments, delete unused func

* Clean up

* Update tests

* Update GetJobURL

* Fix test

* Update

* Clean up
* Allow cALM service keys

* Fix typo

Co-authored-by: Srinikitha Kondreddy <srinikitha.kondreddy@sap.com>

* fix typo

Co-authored-by: Srinikitha Kondreddy <srinikitha.kondreddy@sap.com>

* Hardcode tms endpoint in calm test case

* Add new serviceKey parameter

* Use new serviceKey parameter

With deprecation warning if old tmsServiceKey parameter is used

* Add unit tests and optimise

* Remove tms from service key log message

* Apply suggestions from code review

Co-authored-by: Artem Bannikov <62880541+artembannikov@users.noreply.github.com>

* Remove unused json fields mapping

* Apply review suggestion

* Apply further review suggestions

* Use new parameter name in groovy

* Generate again

* Fix groovy test

---------

Co-authored-by: Srinikitha Kondreddy <srinikitha.kondreddy@sap.com>
Co-authored-by: Artem Bannikov <62880541+artembannikov@users.noreply.github.com>
* Initial API Manager

* Intermediate part

* Intermediate step

* Fix utils tests

* Adapt pull

* Migrate Checkout

* Refactor createTags

* Refactoring

* Setup tests for SAP_COM_0510

* Add tests

* Refactor parsing

* Add retry to clone

* refactor

* Refactor and tests

* Fix function call

* Adapt create tag tests

* Adapt tests

* Add tests

* Fix tests

* Fix test

* Fix client mock

* Add unit test comments

* Add missing parameters

* Branch not mandatory for clone

* Improve switch branch trigger

---------

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
…#4695)

In #4595 a typo was committed that prevents Go private packages from
being correctly set up in the `detectExecute` step

Co-authored-by: Anil Keshav <anil.keshav@sap.com>
* imagePushToRegistry new step

* adding copy and push functionality

* including only copy correctly

* groovy step for imagePushToRegistry

* create .docker folder

* imagePushToRegistry new step

* adding copy and push functionality

* including only copy correctly

* groovy step for imagePushToRegistry

* create .docker folder

* fix CopyImage

* test

* test

* Correct docker config path

* Update

* Update

* Update

* Update

* Update

* Use creds from Vault

* Use creds from Vault

* Use creds from Vault

* Use creds from Vault

* Test

* Comment some logic

* Test: move regexp logic

* Test

* Update

* Update

* Clean up

* Update

* Update

* Update interface

* Rename function

* imagePushToRegistry: small refactoring (#4688)

* imagePushToRegistry new step

* adding copy and push functionality

* including only copy correctly

* groovy step for imagePushToRegistry

* create .docker folder

* Correct docker config path

* Update

* Update

* Update

* Update

* Update

* Use creds from Vault

* Use creds from Vault

* Use creds from Vault

* Use creds from Vault

* Test

* Comment some logic

* Test: move regexp logic

* Test

* Update

* Update

* Clean up

* Update

* Update

---------

Co-authored-by: Keshav <anil.keshav@sap.com>
Co-authored-by: Muhammadali Nazarov <muhammadalinazarov@gmail.com>

* Update step yaml file

* Update interface

* Rename func

* Update tests

* Update interface, create mock methods, update tests

* Update mock

* Add md file

* Fix groovy doc, unit test, go unit test

* Update

* Add unit tests

* Support tagLatest param

* Fetch source creds from Vault

* Update yaml file

* Support multiple images

* Update test

* Support copy images in parallel

* Update yaml

* Clean up

* Return err if no creds provided

* Fix tests

* Add err msg

* Add debug log

* Do not use CPE for targetImages

* Support platform

* Delete Jenkins specific creds

* Update groovy: do not handle Jenkins creds

* Delete unused code

* Fix: Support platform

* Fix: Support platform

* Apply suggestion from code review

Co-authored-by: Egor Balakin <14162703+m1ron0xFF@users.noreply.github.com>

* Apply suggestion from code review

Co-authored-by: Egor Balakin <14162703+m1ron0xFF@users.noreply.github.com>

* Add tests for parseDockerImageName

* Add comment that tagArtifactVersion is not supported yet

* Set limit of running goroutines

* Fix: Set limit of running goroutines

* The tagArtifactVersion is not supported yet

---------

Co-authored-by: Muhammadali Nazarov <muhammadalinazarov@gmail.com>
Co-authored-by: Egor Balakin <egor.balakin@sap.com>
Co-authored-by: Vyacheslav Starostin <vyacheslav.starostin@sap.com>
Co-authored-by: Vyacheslav Starostin <32613074+vstarostin@users.noreply.github.com>
Co-authored-by: Egor Balakin <14162703+m1ron0xFF@users.noreply.github.com>
…ettingsFile (#4702)

* added settings file params

* added checking build tool
* Add support for volume mounts

* Adatpt unit test to include VolumeMounts

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* Only accept volumeMounts with the name volume

---------

Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
* fix(deps): update module golang.org/x/mod to v0.14.0

* go mod tidy

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: jliempt <>
* fix(deps): update module golang.org/x/oauth2 to v0.15.0

* go mod tidy

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: jliempt <>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Jordi van Liempt <35920075+jliempt@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…4689)

Bumps [github.com/go-jose/go-jose/v3](https://github.com/go-jose/go-jose) from 3.0.0 to 3.0.1.
- [Release notes](https://github.com/go-jose/go-jose/releases)
- [Changelog](https://github.com/go-jose/go-jose/blob/v3/CHANGELOG.md)
- [Commits](go-jose/go-jose@v3.0.0...v3.0.1)

---
updated-dependencies:
- dependency-name: github.com/go-jose/go-jose/v3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: jliempt <>
…4707)

* Add imageTag param

* Make imageTag mandatory if tagArtifactVersion is true && update logic

* Make sourceRegistryURL mandatory if localDockerImagePath is not set

* Make some param mandatoryIf

* Change format of sourceImages param

* Add source image tag

* Update sourceImages and targetImages params

* Delete unused function

* Clean up tests

* Update

* Update metadata file

* Update tests

* Fix test

* Fix tests
* Initial in progress

* compiling but not yet functional

* Missed file

* updated checkmarxone step

* Working up to fetching a project then breaks

* Missed file

* Breaks when retrieving projects+proxy set

* Create project & run scan working, now polling

* Fixed polling

* added back the zipfile remove command

* Fixed polling again

* Generates and downloads PDF report

* Updated and working, prep for refactor

* Added compliance steps

* Cleanup, reporting, added groovy connector

* fixed groovy file

* checkmarxone to checkmarxOne

* checkmarxone to checkmarxOne

* split credentials (id+secret, apikey), renamed pullrequestname to branch, groovy fix

* Fixed filenames & yaml

* missed the metadata_generated.go

* added json to sarif conversion

* fix:type in new checkmarxone package

* fix:type in new checkmarxone package

* removed test logs, added temp error log for creds

* extra debugging to fix crash

* improved auth logging, fixed query parse issue

* fixed bug with group fetch when using oauth user

* CWE can be -1 if not defined, can't be uint

* Query also had CweID

* Disabled predicates-fetch in sarif generation

* Removing leftover info log message

* Better error handling

* fixed default preset configuration

* removing .bat files - sorry

* Cleanup per initial review

* refactoring per Gist, fixed project find, add apps

* small fix - sorry for commit noise while testing

* Fixing issues with incremental scans.

* removing maxretries

* Updated per PR feedback, further changes todo toda

* JSON Report changes and reporting cleanup

* removing .bat (again?)

* adding docs, groovy unit test, linter fixes

* Started adding tests maybe 15% covered

* fix(checkmarxOne): test cases for pkg and reporting

* fix(checkmarxOne):fix formatting

* feat(checkmarxone): update interface with missing method

* feat(checkmarxone):change runStep signature to be able to inject dependency

* feat(checkmarxone): add tests for step (wip)

* Adding a bit more coverage

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix integration test PR

* adding scan-summary bug workaround, reportgen fail

* enforceThresholds fix when no results passed in

* fixed gap when preset empty in yaml & project conf

* fixed another gap in preset selection

* fix 0-result panic

* fail when no preset is set anywhere

* removed comment

* initial project-under-app support

* fixing sarif reportgen

* some cleanup of error messages

* post-merge test fixes

* revert previous upstream merge

* adding "incremental" to "full" triggers

* wrong boolean

* project-in-application api change prep

* Fixing SARIF report without preset access

* fix sarif deeplink

* removing comments

* fix(cxone):formatting

* fix(cxone):formatting

---------

Co-authored-by: thtri <trinhthanhhai@gmail.com>
Co-authored-by: Thanh-Hai Trinh <thanh.hai.trinh@sap.com>
…4703)

* added filtering issues by tag

* added optional group of issues

* fixed tests

---------

Co-authored-by: sumeet patil <sumeet.patil@sap.com>
Co-authored-by: Anil Keshav <anil.keshav@sap.com>
* fix sidecar conditionals

Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>

* Fix unit tests

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* Consider parameter used in conditions of sidecars

Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>

---------

Co-authored-by: Johannes Dillmann <j.dillmann@sap.com>
Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>
There is no testModule parameter in gatlingExecuteTests, the correct parameter to be used is pomPath
* fix(checkmarx):disable failOnMissingReports

* fix(checkmarx):disable failOnMissingReports

---------

Co-authored-by: sumeet patil <sumeet.patil@sap.com>
* SAP_COM_0948 clone, pull & checkout

* Fix log output

* Enable compatibility for old tests

* Fix tests

* Add tests for SAP_COM_0948

* Change message

* Add tags for test

* add retry for error code 501

---------

Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com>
…4714)

* added initializing empty repo

* updated go.mod

* updated go.mod

* updated go.sum

* updated go.mod

* updated go.mod

* updated go.mod

---------

Co-authored-by: sumeet patil <sumeet.patil@sap.com>
@DanielMieg DanielMieg closed this Dec 21, 2023
@DanielMieg DanielMieg reopened this Dec 21, 2023
Copy link

Quality Gate Passed Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

@DanielMieg DanielMieg closed this Dec 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.