Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed the validation when use relative links #895

Merged
merged 27 commits into from
Sep 16, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
09d1589
Failed the validation when use relative links when enable the checkLi…
sima-zhu Aug 18, 2020
6d75043
Merge branch 'master' into relative_link
sima-zhu Sep 8, 2020
1b200a7
Addressed feedback
sima-zhu Sep 9, 2020
04887d7
Fixed the bug of pkg name.
sima-zhu Sep 10, 2020
fcc0f0c
Update eng/common/scripts/Verify-Links.ps1
sima-zhu Sep 10, 2020
6acc9af
Merge branch 'relative_link' of https://github.com/sima-zhu/azure-sdk…
sima-zhu Sep 10, 2020
02d51a6
Added link guidance link
sima-zhu Sep 10, 2020
901acdf
Merge branch 'master' into relative_link
sima-zhu Sep 10, 2020
b48ad15
Update eng/common/scripts/Verify-Links.ps1
sima-zhu Sep 10, 2020
e93148b
Update eng/common/scripts/Verify-Links.ps1
sima-zhu Sep 10, 2020
519b473
Move all vars above
sima-zhu Sep 11, 2020
1e5a167
Silent the archor links
sima-zhu Sep 11, 2020
122f192
Silent the archor links
sima-zhu Sep 11, 2020
824a3e1
Romove root url
sima-zhu Sep 11, 2020
3f2d8cd
change the var name
sima-zhu Sep 11, 2020
b3a1f7e
missing parathesis
sima-zhu Sep 11, 2020
0fe62c0
adjust the parameters
sima-zhu Sep 11, 2020
27e9f6f
Pass the right parameters
sima-zhu Sep 11, 2020
ea224d4
default value
sima-zhu Sep 11, 2020
53fd6e3
remove comma
sima-zhu Sep 11, 2020
7f8cf43
fix the wrong check
sima-zhu Sep 11, 2020
d943b89
Skip the empty link
sima-zhu Sep 11, 2020
9ba0e2e
Silent the empty link
sima-zhu Sep 11, 2020
1cba484
small changes
sima-zhu Sep 11, 2020
b96fd2a
Added check for empty string
sima-zhu Sep 11, 2020
f4d1f6b
Added comments and more info in warning message
sima-zhu Sep 15, 2020
21e23aa
Turn off the feature
sima-zhu Sep 16, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions eng/common/TestResources/New-TestResources.ps1.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ It is passed as to the ARM
template as 'testApplicationOid'

For more information on the relationship between AAD Applications and Service
Principals see: https://docs.microsoft.com/en-us/azure/active-directory/develop/app-objects-and-service-principals
Principals see: https://docs.microsoft.com/azure/active-directory/develop/app-objects-and-service-principals

```yaml
Type: String
Expand Down Expand Up @@ -457,6 +457,3 @@ This cmdlet supports the common parameters: -Debug, -ErrorAction, -ErrorVariable
## NOTES

## RELATED LINKS

[Remove-TestResources.ps1]()

4 changes: 2 additions & 2 deletions eng/common/TestResources/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,8 +125,8 @@ New-MarkdownHelp -Command .\New-TestResources.ps1 -OutputFolder . -Force

PowerShell markdown documentation created with [platyPS][].

[New-TestResources.ps1]: ./New-TestResources.ps1.md
[Remove-TestResources.ps1]: ./Remove-TestResources.ps1.md
[New-TestResources.ps1]: https://github.com/Azure/azure-sdk-tools/blob/master/eng/common/TestResources/New-TestResources.ps1.md
[Remove-TestResources.ps1]: https://github.com/Azure/azure-sdk-tools/blob/master/eng/common/TestResources/Remove-TestResources.ps1.md
[PowerShell]: https://github.com/PowerShell/PowerShell
[PowerShellAz]: https://docs.microsoft.com/powershell/azure/install-az-ps
[platyPS]: https://github.com/PowerShell/platyPS
3 changes: 0 additions & 3 deletions eng/common/TestResources/Remove-TestResources.ps1.md
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,3 @@ This cmdlet supports the common parameters: -Debug, -ErrorAction, -ErrorVariable
## NOTES

## RELATED LINKS

[New-TestResources.ps1]()

40 changes: 24 additions & 16 deletions eng/common/pipelines/templates/steps/verify-links.yml
Original file line number Diff line number Diff line change
@@ -1,19 +1,27 @@
parameters:
Directory: 'not-specified'
IgnoreLinksFile: "$(Build.SourcesDirectory)/eng/ignore-links.txt"

IgnoreLinksFile: '$(Build.SourcesDirectory)/eng/ignore-links.txt'
WorkingDirectory: '$(System.DefaultWorkingDirectory)'
ScriptDirectory: 'eng/common/scripts'
Recursive: $false
CheckLinkGuidance: $false
Urls: '(Get-ChildItem -Path ./ -Recurse -Include *.md)'
BranchReplaceRegex: "^(${env:SYSTEM_PULLREQUEST_SOURCEREPOSITORYURI}.*/(?:blob|tree)/)master(/.*)$"
BranchReplacementName: "${env:SYSTEM_PULLREQUEST_SOURCECOMMITID}"

steps:
- task: PowerShell@2
displayName: Link verification check
inputs:
pwsh: true
workingDirectory: $(Build.SourcesDirectory)/${{ parameters.Directory }}
filePath: eng/common/scripts/Verify-Links.ps1
arguments: >
-urls $(dir -r -i *.md)
-rootUrl "file://$(Build.SourcesDirectory)/${{ parameters.Directory }}"
-recursive: $false
-ignoreLinksFile ${{ parameters.IgnoreLinksFile }}
-branchReplaceRegex "^($env:SYSTEM_PULLREQUEST_SOURCEREPOSITORYURI.*/(?:blob|tree)/)master(/.*)$"
-branchReplacementName $env:SYSTEM_PULLREQUEST_SOURCECOMMITID
-devOpsLogging: $true
- task: PowerShell@2
displayName: Link verification check
inputs:
pwsh: true
workingDirectory: '${{ parameters.WorkingDirectory }}/${{ parameters.Directory }}'
filePath: ${{ parameters.ScriptDirectory }}/Verify-Links.ps1
arguments: >
-urls ${{ parameters.Urls }}
-rootUrl "file://${{ parameters.WorkingDirectory }}/${{ parameters.Directory }}"
-recursive: ${{ parameters.Recursive }}
-ignoreLinksFile ${{ parameters.IgnoreLinksFile }}
-branchReplaceRegex "${{ parameters.BranchReplaceRegex }}"
-branchReplacementName ${{ parameters.BranchReplacementName }}
-devOpsLogging: $true
-checkLinkGuidance: ${{ parameters.CheckLinkGuidance }}
54 changes: 35 additions & 19 deletions eng/common/scripts/Verify-Links.ps1
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
param (
# url list to verify links. Can either be a http address or a local file request. Local file paths support md and html files.
# url list to verify links. Can either be a http address or a local file request. Local file paths support md and html files.
[string[]] $urls,
# file that contains a set of links to ignore when verifying
[string] $ignoreLinksFile = "$PSScriptRoot/ignore-links.txt",
Expand All @@ -17,14 +17,14 @@ param (
[string] $branchReplaceRegex = "^(https://github.com/.*/(?:blob|tree)/)master(/.*)$",
# the substitute branch name or SHA commit
[string] $branchReplacementName = "",
# flag to allow checking against azure sdk link guidance.
# flag to allow checking against azure sdk link guidance. Check link guidance here: https://aka.ms/azsdk/guideline/links
[bool] $checkLinkGuidance = $false
)

$ProgressPreference = "SilentlyContinue"; # Disable invoke-webrequest progress dialog
# Regex of the locale keywords.
$locale = "/en-us/"

$emptyLinkMessage = "There is at least one empty link in the page. Please replace with absolute link. Check here for more infomation: https://aka.ms/azsdk/guideline/links"
function NormalizeUrl([string]$url){
if (Test-Path $url) {
$url = "file://" + (Resolve-Path $url).ToString();
Expand Down Expand Up @@ -85,16 +85,18 @@ function ResolveUri ([System.Uri]$referralUri, [string]$link)
$linkUri = [System.Uri]$link;
# Our link guidelines do not allow relative links so only resolve them when we are not
# validating links against our link guidelines (i.e. !$checkLinkGuideance)
if(!$checkLinkGuidance) {
if (!$linkUri.IsAbsoluteUri) {
if ($checkLinkGuidance -and !$linkUri.IsAbsoluteUri) {
return $linkUri
}

if (!$linkUri.IsAbsoluteUri) {
# For rooted paths resolve from the baseUrl
if ($link.StartsWith("/")) {
Write-Verbose "rooturl = $rootUrl"
$linkUri = new-object System.Uri([System.Uri]$rootUrl, ".$link");
}
else {
$linkUri = new-object System.Uri($referralUri, $link);
}
if ($link.StartsWith("/")) {
Write-Verbose "rooturl = $rootUrl"
$linkUri = new-object System.Uri([System.Uri]$rootUrl, ".$link");
}
else {
$linkUri = new-object System.Uri($referralUri, $link);
}
}

Expand Down Expand Up @@ -134,6 +136,10 @@ function ParseLinks([string]$baseUri, [string]$htmlContent)

function CheckLink ([System.Uri]$linkUri)
{
if(!$linkUri.ToString().Trim()) {
LogWarning "Found Empty link. Please use absolute link instead. Check here for more infomation: https://aka.ms/azsdk/guideline/links"
return $false
}
if ($checkedLinks.ContainsKey($linkUri)) {
if (!$checkedLinks[$linkUri]) {
LogWarning "broken link $linkUri"
Expand Down Expand Up @@ -193,11 +199,19 @@ function CheckLink ([System.Uri]$linkUri)
}
}

# Check if link uri includes locale info.
if ($checkLinkGuidance -and ($linkUri -match $locale)) {
LogWarning "DO NOT include locale $locale information in links: $linkUri."
$linkValid = $false
if ($checkLinkGuidance) {
# Check if the url is relative links, suppress the archor link validation.
if (!$linkUri.IsAbsoluteUri -and !$linkUri.ToString().StartsWith("#")) {
LogWarning "DO NOT use relative link $linkUri. Please use absolute link instead. Check here for more infomation: https://aka.ms/azsdk/guideline/links"
$linkValid = $false
}
# Check if link uri includes locale info.
if ($linkUri -match $locale) {
LogWarning "DO NOT include locale $locale information in links: $linkUri. Check here for more information: https://aka.ms/azsdk/guideline/links"
$linkValid = $false
}
}

$checkedLinks[$linkUri] = $linkValid
return $linkValid
}
Expand Down Expand Up @@ -270,7 +284,6 @@ $checkedPages = @{};
$checkedLinks = @{};
$badLinks = @{};
$pageUrisToCheck = new-object System.Collections.Queue

foreach ($url in $urls) {
$uri = NormalizeUrl $url
$pageUrisToCheck.Enqueue($uri);
Expand All @@ -286,9 +299,12 @@ while ($pageUrisToCheck.Count -ne 0)
Write-Host "Found $($linkUris.Count) links on page $pageUri";
$badLinksPerPage = @();
foreach ($linkUri in $linkUris) {
$linkUri = ReplaceGithubLink $linkUri
$isLinkValid = CheckLink $linkUri
$replacedLink = ReplaceGithubLink $linkUri
$isLinkValid = CheckLink $replacedLink
if (!$isLinkValid -and !$badLinksPerPage.Contains($linkUri)) {
if (!$linkUri.ToString().Trim()) {
$linkUri = $emptyLinkMessage
}
$badLinksPerPage += $linkUri
}
if ($recursive -and $isLinkValid) {
Expand Down