Skip to content

Commit

Permalink
Merge branch 'autorestv3' of https://github.com/Azure/autorest.python
Browse files Browse the repository at this point in the history
…into inherit_customization

* 'autorestv3' of https://github.com/Azure/autorest.python:
  Update package.json
  Update ChangeLog.md
  default optional const parameter to none (#1171)
  add storage to ci checks (#1176)
  fail on regeneration diff (#1173)
  Drop Python 2.7 (#1175)
  • Loading branch information
iscai-msft committed Mar 3, 2022
2 parents d8af865 + f737949 commit 880c02f
Show file tree
Hide file tree
Showing 80 changed files with 683 additions and 587 deletions.
2 changes: 2 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,7 @@
# Automation
################

* @iscai-msft @msyyc @changlong-liu

# Git Hub bot rules
/.github/fabricbot.json @AlexGhiondea @jsquire
14 changes: 13 additions & 1 deletion ChangeLog.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Change Log

### 2022-xx-xx - 5.12.7
### 2022-03-03 - 5.13.0

| Library | Min Version
| --------------- | -------
Expand All @@ -10,11 +10,23 @@
|`msrest` dep of generated code | `0.6.21`
|`azure-mgmt-core` dep of generated code (If generating mgmt plane code) | `1.3.0`

**Breaking Changes in Version Tolerant Generation**

- We now generate with optional constant parameters as None by defaulting `--default-optional-constants-to-none` to True #1171

**New Features**

- Add flag `--default-optional-constants-to-none` with which optional constant parameters is default to None #1171

**Bug Fixes**

- Add default value consistently for parameters #1164
- Make `content_type` param keyword-only if there are multiple content types #1167

**Other Changes**

- Drop testing support for 2.7 packages #1175

### 2022-02-09 - 5.12.6

| Library | Min Version
Expand Down
3 changes: 3 additions & 0 deletions autorest/codegen/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -289,6 +289,9 @@ def _build_code_model_options(self) -> Dict[str, Any]:
"low_level_client": low_level_client,
"combine_operation_files": self._autorestapi.get_boolean_value("combine-operation-files", version_tolerant),
"python3_only": python3_only,
"default_optional_constants_to_none": self._autorestapi.get_boolean_value(
"default-optional-constants-to-none", low_level_client or version_tolerant
),
}

if options["builders_visibility"] is None:
Expand Down
15 changes: 14 additions & 1 deletion autorest/codegen/models/parameter.py
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,10 @@ def description(self):
if description:
description += " "
description += f"{self.schema.extra_description_information}"
if isinstance(self.schema, ConstantSchema) and not self.constant:
if description:
description += " "
description += f"Possible values are {self.schema.get_declaration(self.schema.value)} or {None}."
if self.has_default_value and not any(
l for l in ["default value is", "default is"] if l in description.lower()
):
Expand Down Expand Up @@ -237,7 +241,12 @@ def _default_value(self) -> Tuple[Optional[Any], str, str]:
default_value_declaration = "None"
else:
if isinstance(self.schema, ConstantSchema):
default_value = self.schema.get_declaration(self.schema.value)
if (self.required or
self.is_content_type or
not self.code_model.options["default_optional_constants_to_none"]):
default_value = self.schema.get_declaration(self.schema.value)
else:
default_value = None
default_value_declaration = default_value
else:
default_value = self.schema.default_value
Expand Down Expand Up @@ -319,6 +328,10 @@ def is_hidden(self) -> bool:
self.yaml_data["implementation"] == "Client" and self.constant
)

@property
def is_content_type(self) -> bool:
return self.rest_api_name == "Content-Type" and self.location == ParameterLocation.Header

@property
def is_positional(self) -> bool:
return self.in_method_signature and not (self.is_keyword_only or self.is_kwarg)
Expand Down
49 changes: 29 additions & 20 deletions eng/pipelines/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,17 +20,26 @@ parameters:
default:
- name: azure-messaging-webpubsubservice
service: webpubsub
branchName: main
- name: azure-storage-blob
service: storage
branchName: feature/storage-autorest
- name: azure-storage-queue
service: storage
branchName: feature/storage-autorest
- name: azure-storage-file-datalake
service: storage
branchName: feature/storage-autorest
- name: azure-storage-file-share
service: storage
branchName: feature/storage-autorest

jobs:
- job: "AutoRest_Python_CI"
displayName: "Run AutoRest CI Checks"

strategy:
matrix:
Linux_Python27:
OSName: "Linux"
OSVmImage: "MMSUbuntu20.04"
PythonVersion: "2.7"
Linux_Python37:
OSName: "Linux"
OSVmImage: "MMSUbuntu20.04"
Expand Down Expand Up @@ -74,10 +83,8 @@ jobs:
- script: |
inv regenerate
displayName: 'Regenerate Code'
- script: |
git add -A # 'add' first so 'diff' includes untracked files
git diff --staged -w
displayName: 'Diff regeneration'
- script: node ./eng/scripts/check-for-changed-files.js
displayName: Fail on regeneration diff
- task: UsePythonVersion@0
displayName: 'Use Python $(PythonVersion)'
inputs:
Expand All @@ -102,9 +109,9 @@ jobs:
cd $(TestFolder)/azure/low-level
tox -e ci
displayName: 'Execute low-level "azure" Tests - Python $(PythonVersion)'
condition: and(succeededOrFailed(), ne(variables['PythonVersion'], '2.7'))
condition: succeededOrFailed()
- task: PublishTestResults@2
condition: and(succeededOrFailed(), ne(variables['PythonVersion'], '2.7'))
condition: succeededOrFailed()
displayName: 'Publish low-level "azure" Test Results'
inputs:
testRunTitle: "$(OSName) Python $(PythonVersion) - low-level azure"
Expand All @@ -114,9 +121,9 @@ jobs:
cd $(TestFolder)/azure/version-tolerant
tox -e ci
displayName: 'Execute version-tolerant "azure" Tests - Python $(PythonVersion)'
condition: and(succeededOrFailed(), ne(variables['PythonVersion'], '2.7'))
condition: succeededOrFailed()
- task: PublishTestResults@2
condition: and(succeededOrFailed(), ne(variables['PythonVersion'], '2.7'))
condition: succeededOrFailed()
displayName: 'Publish version-tolerant "azure" Test Results'
inputs:
testRunTitle: "$(OSName) Python $(PythonVersion) - version-tolerant azure"
Expand All @@ -143,10 +150,10 @@ jobs:
cd $(TestFolder)/vanilla/low-level
tox -e ci
displayName: 'Execute low level "vanilla" Tests - Python $(PythonVersion)'
condition: and(succeededOrFailed(), ne(variables['PythonVersion'], '2.7'))
condition: succeededOrFailed()
- task: PublishTestResults@2
displayName: 'Publish low level "vanilla" Test Results'
condition: and(succeededOrFailed(), ne(variables['PythonVersion'], '2.7'))
condition: succeededOrFailed()
inputs:
testRunTitle: "$(OSName) Python $(PythonVersion) - low level vanilla"
searchFolder: "$(TestFolder)/vanilla/low-level"
Expand All @@ -155,10 +162,10 @@ jobs:
cd $(TestFolder)/vanilla/version-tolerant
tox -e ci
displayName: 'Execute version-tolerant "vanilla" Tests - Python $(PythonVersion)'
condition: and(succeededOrFailed(), ne(variables['PythonVersion'], '2.7'))
condition: succeededOrFailed()
- task: PublishTestResults@2
displayName: 'Publish version-tolerant "vanilla" Test Results'
condition: and(succeededOrFailed(), ne(variables['PythonVersion'], '2.7'))
condition: succeededOrFailed()
inputs:
testRunTitle: "$(OSName) Python $(PythonVersion) - version-tolerant vanilla"
searchFolder: "$(TestFolder)/vanilla/version-tolerant"
Expand All @@ -173,12 +180,12 @@ jobs:
cd $(TestFolder)/dpg/low-level
tox -e ci
displayName: 'Execute low level update "dpg" Tests - Python $(PythonVersion)'
condition: and(succeededOrFailed(), ne(variables['PythonVersion'], '2.7'))
condition: succeededOrFailed()
- script: |
cd $(TestFolder)/dpg/version-tolerant
tox -e ci
displayName: 'Execute version-tolerant update "dpg" Tests - Python $(PythonVersion)'
condition: and(succeededOrFailed(), ne(variables['PythonVersion'], '2.7'))
condition: succeededOrFailed()
- script: |
cd $(TestFolder)/dpg/version-tolerant
tox -e lint
Expand Down Expand Up @@ -208,9 +215,9 @@ jobs:
displayName: "Publish coverage report to storage account"
condition: and(succeededOrFailed(), eq(variables['PythonVersion'], '3.10'))
- ${{ if eq(and(ne(variables['Build.Reason'], 'PullRequest'), ne(variables['PythonVersion'], '2.7')), 'True') }}:
- ${{ if eq(ne(variables['Build.Reason'], 'PullRequest'), 'True') }}:
- script: |
git clone https://github.com/Azure/azure-sdk-for-python.git --depth=1 --branch main $(Build.SourcesDirectory)/regenerated/
git clone https://github.com/Azure/azure-sdk-for-python.git $(Build.SourcesDirectory)/regenerated/
displayName: 'Clone python repo'
- script: |
Expand All @@ -220,6 +227,8 @@ jobs:
- ${{ each package in parameters.VerificationPackages }}:
- script: |
git fetch origin
git checkout ${{ package.branchName }}
autorest ./swagger/README.md --use=$(Build.SourcesDirectory)
displayName: 'Regenerate target package ${{ package.name }}'
workingDirectory: '$(Build.SourcesDirectory)/regenerated/sdk/${{ package.service }}/${{ package.name }}'
Expand Down
23 changes: 23 additions & 0 deletions eng/scripts/check-for-changed-files.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
// @ts-check

const { run } = require("./helpers.js");

const proc = run("git", ["status", "--porcelain"], {
encoding: "utf-8",
stdio: [null, "pipe", "pipe"],
});

if (proc.stdout) {
console.log(proc.stdout);
}

if (proc.stderr) {
console.error(proc.stderr);
}

if (proc.stdout || proc.stderr) {
console.error(
`ERROR: There are diffs in regeneration. Please run 'inv regenerate' and re-run. You may also have to remove 'node_modules' and re-run 'npm install' to get the latest testserver.`,
);
process.exit(1);
}
83 changes: 83 additions & 0 deletions eng/scripts/helpers.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
// @ts-check
const { spawn, spawnSync } = require("child_process");
const { resolve } = require("path");

const repoRoot = resolve(__dirname, "../..");
const prettier = resolve(
repoRoot,
"packages/extensions/core/node_modules/.bin/prettier"
);
const tsc = resolve(repoRoot, "packages/extensions/core/node_modules/.bin/tsc");

const isCmdOnWindows = ["rush", "npm", "code", "code-insiders", tsc, prettier];

function run(command, args, options) {
console.log();
console.log(`> ${command} ${args.join(" ")}`);

options = {
stdio: "inherit",
sync: true,
throwOnNonZeroExit: true,
...options,
};

if (process.platform === "win32" && isCmdOnWindows.includes(command)) {
command += ".cmd";
}

const proc = (options.sync ? spawnSync : spawn)(command, args, options);
if (proc.error) {
if (options.ignoreCommandNotFound && proc.error.code === "ENOENT") {
console.log(`Skipped: Command \`${command}\` not found.`);
} else {
throw proc.error;
}
} else if (
options.throwOnNonZeroExit &&
proc.status !== undefined &&
proc.status !== 0
) {
throw new CommandFailedError(
`Command \`${command} ${args.join(" ")}\` failed with exit code ${
proc.status
}`,
proc
);
}

return proc;
}

class CommandFailedError extends Error {
constructor(msg, proc) {
super(msg);
this.proc = proc;
}
}

function runPrettier(...args) {
run(
prettier,
[
...args,
"--config",
".prettierrc.yml",
"--ignore-path",
".prettierignore",
"**/*.{ts,js,cjs,mjs,json,yml,yaml,cadl,md}",
],
{
cwd: repoRoot,
}
);
}

module.exports = {
repoRoot,
prettier,
tsc,
run,
runPrettier,
CommandFailedError,
};
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@autorest/python",
"version": "5.12.6",
"version": "5.13.0",
"description": "The Python extension for generators in AutoRest.",
"scripts": {
"prepare": "node run-python3.js prepare.py",
Expand Down
6 changes: 0 additions & 6 deletions test/azure/legacy/AcceptanceTests/asynctests/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,3 @@
# IN THE SOFTWARE.
#
# --------------------------------------------------------------------------

import sys
import pytest

pytestmark = pytest.mark.skipif(sys.version_info < (3,6),
reason="requires python3.6")
5 changes: 0 additions & 5 deletions test/azure/legacy/AcceptanceTests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,6 @@ def testserver():
yield
terminate_server_process(server)

# Ignore collection of async tests for Python 2
collect_ignore = []
if sys.version_info < (3,5):
collect_ignore.append("asynctests")


class CookiePolicy(SansIOHTTPPolicy):
def __init__(self, *args, **kwargs):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -75,12 +75,8 @@ def test_custom_base_uri_bad_host(self, client):
def test_models(self):
from custombaseurl.models import Error

if sys.version_info >= (3,5):
from custombaseurl.models._models_py3 import Error as ErrorPy3
assert Error == ErrorPy3
else:
from custombaseurl.models._models import Error as ErrorPy2
assert Error == ErrorPy2
from custombaseurl.models._models_py3 import Error as ErrorPy3
assert Error == ErrorPy3

def test_operation_groups(self):
from custombaseurl.operations import PathsOperations
Expand Down
8 changes: 2 additions & 6 deletions test/azure/legacy/AcceptanceTests/test_azure_url.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,8 @@ def test_azure_url(self, credential, authentication_policy):
def test_models(self):
from subscriptionidapiversion.models import Error

if sys.version_info >= (3,5):
from subscriptionidapiversion.models._models_py3 import Error as ErrorPy3
assert Error == ErrorPy3
else:
from subscriptionidapiversion.models._models import Error as ErrorPy2
assert Error == ErrorPy2
from subscriptionidapiversion.models._models_py3 import Error as ErrorPy3
assert Error == ErrorPy3

def test_operation_groups(self):
from subscriptionidapiversion.operations import GroupOperations
Expand Down
Loading

0 comments on commit 880c02f

Please sign in to comment.