Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: azure publish plugin #2733

Merged
merged 90 commits into from
May 5, 2020
Merged

feat: azure publish plugin #2733

merged 90 commits into from
May 5, 2020

Conversation

benbrown
Copy link
Contributor

@benbrown benbrown commented Apr 21, 2020

Description

Task Item

close #2461
close #2757

Screenshots

image

@benbrown
Copy link
Contributor Author

benbrown commented Apr 21, 2020

@VanyLaw We need to simplify this in a few ways.

do not provision in the plugin

The provisioning part (botProjectDeploy.create) should not be part of the Composer publish action. This should happen outside of the plugin, via a command line tool. We need to figure out how to make this an easy to use script -- possibly bundle it with the ejected code.

Ideally, we would be able to do something like this:

provisionComposer --name <bot> --location westus --environment dev --appPassword <password> --luisAuthoringKey --<key> --token <token> --subscription <subid>

This would perform the provision, and after completion, output a configuration profile, like:

Your Azure hosting environment has been created! Copy paste the following configuration into a new profile in Composer's Publishing tab.

{
accessToken: <token>,
subscription: <subid>,
environment: <environment>,
luisAuthoringKey: <key>,
luisAuthoringRegion: <region>,
... other settings possibly including all provisioned services...
}

make sure the plugin does not expect the declarative assets to be local

Right now, the plugin is doing a simple recursive copy of the files from the bot project's source folder. However, this will not work for all storage providers.

Instead, you should use the project.files array to access the file content and write them to the appropriate location.

Consider alternative mechanisms for building/deploying

Right now, the plugin copies the declarative assets from the bot project along with the c sharp code project from the template into a single TEMP folder. Inside this temp folder, it updates an appsettings.deployment.settings file after provisioning, and then uses that for the deploy.

There are a number of flaws in this approach, though a big one is that this requires provisioning to occur immediately before deploying and will only work for one project at a time, and only work once since provisioning status is held in the plugin's memory and not serialized anywhere.

  • where should information about the provisioned resources live? For maximum flexibility, this should probably live in the publishing profile - this would allow Composer to write the appropriate appsettings.json file during deploy. (Is this correct?)
  • How will this work with an ejected runtime? Is it as easy as just copying the source code from a different path? (I think so)
  • When using an ejected runtime, should the build occur in a temp folder or should it occur in the normal project folder? (not sure!)

Error handling during provisioning

Right now, a lot of the errors just return and cause no error to be raised. These critical errors should probably throw an error and cause the promise to reject so that they can be handled properly.

@VanyLaw
Copy link
Contributor

VanyLaw commented Apr 22, 2020

@benbrown Can't agree more. I did some update today.

  1. To handle the token expire more user-friendly, I split the login into a js script which return a credential. user can simpliy run this before start composer. And no need to update the profile even the token expired. The credential will be read before composer start and refresh in memory.
  2. Instead of the a single TEMP folder, I used the hask key of profile config(name, location,env,````) as a folder name to save those declarative assets and runtime. and add the field create to defined provision or not. If user use the same config to publish different bots, it will only replace the composerDialogs folder in it and use the same config and appsettings.Deployment.json to publish.

I will keep doing the rest and move the provision out of the plugin next.

@github-actions
Copy link

Coverage Status

Coverage remained the same at 0.0% when pulling a6e0f45 on wenyluo/azure into 3a7dbce on master.

@benbrown benbrown marked this pull request as ready for review May 1, 2020 22:24
@cwhitten cwhitten merged commit 21a0f8f into master May 5, 2020
@cwhitten cwhitten deleted the wenyluo/azure branch May 5, 2020 20:29
alanlong9278 added a commit that referenced this pull request May 7, 2020
* master: (58 commits)
  fix: Copy skill manifests to the correct directory in the localPublish plugin (#2932)
  feat: Goto Begin Dialog after clicking dialog (#2922)
  fix: Improved Electron auto update UX (#2925)
  fix: Action Flow gradual left alignment (#2909)
  fix: word wrap in SendActivity (#2908)
  fix: Fixed various onboarding issues and updated content (#2900)
  chore: Component Governance (#2899)
  perf: improve property editor performance (#2921)
  fix: paste blank node (#2905)
  extract memory variables at lg lsp server (#2902)
  feat: manage samples via plugin (#2805)
  can not use event capture in visual editor (#2913)
  style: make focus styles more consistent (#2898)
  feat: azure publish plugin (#2733)
  fix: unable to clear form title (#2885)
  fix: Populate env variable with AppData folder (#2894)
  a11y: use Key/Value aria labels in object field (#2890)
  Fix border issue in visual editor (#2891)
  fix: changes manifest type from '.manifest' to '.json' (#2888)
  Fixed packaged folder structure. (#2887)
  ...
lei9444 pushed a commit to lei9444/BotFramework-Composer-1 that referenced this pull request Jun 15, 2021
* split the configuration as object

* fix deployment script, change settings path

* fix bugs in deploy script and change credential type:

* finish gethistory, getstatus, update history and status after publish

* add some code comments for clarity
add many TODOs in code

* more code comments

* more code comments for clarity

* add login script and hash the config as folder name

* add bot deploy package build into lib build

* split the provision part out of composer

* fix luis appid unfound error

* fix typo

* polish

* use bot root to config the path of settings

* differ bot to different layer

* change the order of loading settings

* update schema

* change login and provision script, from save file to output to console

* add comment

* fix merge conflict

* format resulting profile

* add comment about security

* use Bearer token auth when doing zipdeploy
remove use of websiteclient

* move provision out of composer, and make sure it not depend on internal package

* add provision subfields schema

* use token replace the credential

* fix output of token

* change the error message when token expire

* add provision error details, add configurable

* update schema require and log message return

* fix parse

* fix template

* update schema and provision script

* fix provision luis configurable

* detail the token's error handle

* Add support for using the ejected runtime code instead of the built in template when publishing to Azure

* Make it optional to persist the history to disk.
Keep it in memory instead.

* move provision script into shared asset folder that is copied into all new projets
change hashing to include bot name to avoid possible overlap when using shared resources

* add to readme

* more readme

* more readme

* clean up tmp folder after publish completes or fails

* remove empty config in provision and appPassword in provision result, Change provision default to true

* add token error handle when zipdeploy fail

* improve output of provision script:
* add usage if missing parameters
* colorize and format output
* add some additional error formatting

* improve output

* remove webManagerClient and use token instead

* clean up the order of fields, remove unused fields from profile
update schema with full form

* use publisher description rather than package name

* feat: Runtime refactor, new directory structure and Azure Function introduction phase 1 (microsoft#2855)

* Runtime: new folder structure, refactor common c# code into core, create function runtime

* Remove deprecated Bot Project

* Runtime: Fix tests

* Update runtime code owners

* Runtime: Part 1 of updating composer server to honor new runtime paths

* Runtime: tweaks for local publish to run prior to merge

* Runtime: fixes post merge

* Update azure publish to use the azure web app template.

* Azure publish: update bot project deploy to new directory structure

* Fix codeowners alias mistake

* Revert "Merge branch 'master' of https://github.com/microsoft/BotFramework-Composer into wenyluo/azure"

This reverts commit af077e248e47e32750629dee7b9b8c8c6e98d84a, reversing
changes made to e24071d0ddbce5554d498c1ba34eb18a93b8948d.

* WebApp + new runtime: deployment and local runtime all working

* Runtime: move nuget.config to dotnet runtime root

* Asset manager: add mock folder to reflect new runtime structure

* Local publish: remove unnecessary space

* Runtime: Add copyright header to all missing files

* Fix bad merge

* Runtime: rename js -> node and update readmes

* add an optional instructions field to the publish plugin so we ca nexplain how to get a config

* merge master

* fix linter warning

* handle errors in kill procss

* remove unnecessary dupe fields from schema

* fix for modified schema path

Co-authored-by: Wenyi Luo <wenyluo@microsoft.com>
Co-authored-by: Qi Kang <qika@microsoft.com>
Co-authored-by: Chris Whitten <christopher.whitten@microsoft.com>
Co-authored-by: Carlos Castro <carlosscastro@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants