Revision of build process part 1 #3137
Conversation
|
@donker I'm not sure what the issue is here, but this showing lots of conflicts that restrict the merge? |
|
\azp run |
|
I've rebased on |
SolutionInfo.cs
Outdated
| [assembly: AssemblyVersion("9.4.2.0")] | ||
| [assembly: AssemblyFileVersion("9.4.2.5782")] | ||
| [assembly: AssemblyInformationalVersion("9.4.2-unstable.5782+Branch.reorg1.Sha.b37c5e45ed0dde233bee7c8572d70940e5bda93e")] |
There was a problem hiding this comment.
Is this supposed to be added in here? Or when the manifests are reverted, should this be reverted as well?
There was a problem hiding this comment.
Good catch. I'll repair after merge in a fresh PR.
There was a problem hiding this comment.
Ok, so from a read of the changes, I have nothing to say, this is a major improvement!
However I tried to install a clean site and the Persona Bar does not load because of this missing file
I would like that we get to a state that works before merging. I need to catch a few zzz, but I will ping you up in the afternoon...
I do not get the same result here. Strange. |
|
I will try to give it a more detailed look in a few hours, sorry. |
|
I have tested this locally and am seeing the same as @valadas |
|
Ok, one difference here needs to be fixed: |
|
Here is one finding, the resources.zip on Dnn.PersonaBar.Extensions is missing the output from the yarn build and has other files unrelated, maybe this is getting zipped from the wrong source path or something? |
I can't replicate that, Daniel. Here it builds this resource zip correctly from Yarn. Can you check your build output? |
I just fixed this. |
|
@donker well, I see the yarn build running but then the packaging does not have the build result, I am thinking maybe the build script has the wrong source folder. David had the same results as me. Can you try re-cloning into a brand new folder and building to see, maybe you have some ignored files from past builds that makes it work for you but not for us... I am still a bit swamped today but if you want to take a look at it together I should have a bit more time starting tomorrow... Just ping me up. |
|
It is probably due to the various paths in the two MS build solutions running in parallel. It's a royal clusterf. We could try to solve this as well by collapsing those two into a single one. But it requires quite a bit of work to figure out all the dependencies. I just gave it a shot but no cigar yet. I propose to move AE's module.build to a file called AEModule.build and similar with Package.targets. These are the two files that are different from the main files and specifically tweaked for the AE. |
|
Cool, well ping me up if you want an extra pair of eyes, I got it working before this PR by referencing the main build script for everything but having 1 override for the AE Extensions project which was a bit different. |
|
I can confirm that a clean install results in It seems like there may be timing issues around when things are built/packaged/cleaned. |
|
Ok, so just updated with the changes that fixed previous build issue and it did not solve this one, it looks like we have missing stuff in PersonaBar.UI and EditBar.UI packages for some reason. |
|
I went ahead and merged the build scripts from DNN and AE into one folder. Hopefully this resolves the quirks you were seeing. |
|
I've rebased this PR on release/9.4.x again. Once it's done building again, we can check the artifacts produced in Azure DevOps and see if they're still missing folders. |
|
The new build's artifacts are still missing |
|
That The Dnn.Platform/Build/BuildScripts/AEPackage.targets Lines 49 to 52 in 087ee36 The Dnn.Platform/Build/BuildScripts/AEPackage.targets Lines 13 to 25 in 087ee36 The According to MSBuild docs:
Therefore, we'd expect Does anyone know of a good way to trace what MSBuild thinks it's doing? |
Oddly enough I can't replicate this. I just recloned and just ran the build script. |
|
Definitely weird. You're just getting a fresh clone and running |
|
It seems like it could be a timing issue, but it's weird that it seems consistent on DevOps and my machine. Are you using VS 2019 or 2017? I wonder if that might be a contributing factor. I'm using 2019 (and I asssume DevOps is also). |
Completely fresh clone. Then: .\Build.ps1 -Target BuildAll So that should be equivalent, yes. I have both 2017 and 2019 installed on my machine. But build is called from the scripts so cannot see how that is a factor. |
|
Yes and then the next builds are fine. It only happens on the first build. Same happens in CI. |
|
So in answer to @bdukes : I compared my successful build log with the one in DevOps and in my case the zip was created on line 8587 and the export-bundle on 8331. So it seems they are indeed reversed. But ... but. On a rebuild it's reversed. Hmm. |
|
Alright, I think I've tracked this down. Not quite sure what the right answer is. While the The easiest solution is probably to move the Yarn commands into Cake, and make sure they're called before |
|
Together with Daniel we worked at this and added the PersonaBar.Extensions project as dependency for the PersonaBar.UI project. |
|
Conflict has been addressed, once it's done building let's go ahead and approve this PR with a squash (I'm heading out of town, so don't wait for me to push the squash button) |
* Do not store these portal settings when setting their property on the portalsettings object * Removed DnnHttpControllerActivator and added DependencyResolver implementation using the IServiceProvider * Added xml comments for the new DnnDependencyResolver * Added DnnDependencyResolver Unit Tests * Replaced IServiceProvider mock with full IServiceProvider implementation to work with Web API tests * Added more unit tests for the DnnDependencyResolver to cover GetServices (multiple) and BeginScope * Do not show "No Search results" message when no search is performed. (#3203) * Revision of build process part 1 (#3137) * Move website folder * repairs * Update Cake * Move external task to its own file * Begin packaging scripts in cake * Further work on packaging * Ensure Telerik project also gets new version nr * Move Telerik PDF to its proper place * New logic for packages that are already in final form in the repo and just need to be zipped. * Bring fips Lucene lib back under git control * Newtonsoft package and repair to telerik package * Splitting off other tasks * Upgrade and deploy packages * Symbols package * Get rid of platform ver in manifest * Final fixes - working packages * Move compilation stuff * House cleaning * peg utils version * Fix to ckep script * suppress warnings * Variable initialization issue * Removing unused scripts * Revert change to Telerik package version * Move "other" to "thirdparty" * Get rid of hard coded Newtonsoft version * Max cpu to 4 for compiling * Repairs to MSBuild scripts to get AE to build properly * Correcting paths * Cleaning up * Trying to fix ignore file * Update Build/Cake/thirdparty.json Co-Authored-By: Brian Dukes <bdukes@engagesoftware.com> * Remove absolute path from .gitignore * Fix extension of DNC library package * Merging AE and DNN build scripts * Fixes to scripts * Clean up symbols project for AE since they are included in DNN project now * Add previously untracked Yahoo compressor dlls * Remove version information from SolutionInfo.cs * Fix errors introduced in rebase * Fix incorrect paths * Change build order of MSBuild to build extensions before personabar ui. * Make white-space visible in log viewer (#3198) * DNN-34236: check the url with case insensitive. * DNN-29417 Username is changed to email address in the user profile if the "Use Email Address as Username" is enabled. * Fix to installer and the dependencies of the HTML and DA modules * DNN-24945 - Always redirect to the source portal page after creating (#3223) group * Fix Typo (#3234) * Improve querystring parse (#3242) * Improve props popup * Improve input checking of sites (#3243) * Fixes an upgrade issue that affects 9.4.1 => 9.4.2 upgrades (#3282) * Fixes an issue with a duplication of assembly binding * Removed hardcode version in manifest * Further reorganization of build process (#3236) * Introduce settings and add custom version nr * Ability to create a local dev site * Move node workspaces to root * Reshuffling of variables in MSBuild * Repairs * Fix * Allow local override of global build variables * Only save manifests if the file is not there. This prevents overwriting in case of a failed build before. * Ensure projects build in debug to the website path specified by the platform build settings * Don't track vscode folder * Fixes #3168 Test Sending Email Issues (#3237) * Use Admin Email for Send if smtpHostMode false * Fix Test Email Settings TO and FROM Addresses If I did this right and it should make sense is I set the test email to be able to go to the user testing. After all they are who are trying to make it work. It can be any email used by anyone and usually to see this feature you are a host or portal administrator so security should not be as much of a concern here. The FROM address will use the HOST email settings if set to HOST mode in DNN for the portal, otherwise it should use the Portal Administrators Email account to send from. This however does not address the issue of using the email address supplied in settings and creating another setting which will be applied next to figure out the correct logic. * Update ServerSettingsSmtpAdminController.cs * Fix build issue to previous changes. PortalSettings.UserInfo.Email is this the current user email? I am still getting my things setup. I ran VS 2019 to build successfully and I am trying to read through the namespaces correctly so I can understand the changes. I noticed a number of files possibly needing updates due to being deprecated in 9.4.2 but after I changed the namespace issue those warnings went away. One last thought is how do you get intellisense working on DNN solution. Is that a dream to have or am I missing something in my setup? * Change to address to the current user's email testing the server. I believe a UI to set the TO address would be nice to help troubleshoot and address issues trying to send to a specific user. * Corrected To address for current user email * TO Current User Email * DNN-29110 Site Assets > Select All is not working (#3251) * DNN-29110 Site Assets > Select All is not working * Code review fix * Moves Remember Login above Login Button: referenced in issue #2471 #3255 (#3256) * Move Remember Login Above Login Button * Update Login.ascx * file copy paste issue... * DNN-34250 Search is not working even after re-Indexing Search for All DNN Portals (#3260) * Fixes duplication in DotNetNuke.Website.csproj * Removed reference to missing LeftMenu files
* Do not store these portal settings when setting their property on the portalsettings object * Removed DnnHttpControllerActivator and added DependencyResolver implementation using the IServiceProvider * Added xml comments for the new DnnDependencyResolver * Added DnnDependencyResolver Unit Tests * Replaced IServiceProvider mock with full IServiceProvider implementation to work with Web API tests * Added more unit tests for the DnnDependencyResolver to cover GetServices (multiple) and BeginScope * Do not show "No Search results" message when no search is performed. (#3203) * Revision of build process part 1 (#3137) * Move website folder * repairs * Update Cake * Move external task to its own file * Begin packaging scripts in cake * Further work on packaging * Ensure Telerik project also gets new version nr * Move Telerik PDF to its proper place * New logic for packages that are already in final form in the repo and just need to be zipped. * Bring fips Lucene lib back under git control * Newtonsoft package and repair to telerik package * Splitting off other tasks * Upgrade and deploy packages * Symbols package * Get rid of platform ver in manifest * Final fixes - working packages * Move compilation stuff * House cleaning * peg utils version * Fix to ckep script * suppress warnings * Variable initialization issue * Removing unused scripts * Revert change to Telerik package version * Move "other" to "thirdparty" * Get rid of hard coded Newtonsoft version * Max cpu to 4 for compiling * Repairs to MSBuild scripts to get AE to build properly * Correcting paths * Cleaning up * Trying to fix ignore file * Update Build/Cake/thirdparty.json Co-Authored-By: Brian Dukes <bdukes@engagesoftware.com> * Remove absolute path from .gitignore * Fix extension of DNC library package * Merging AE and DNN build scripts * Fixes to scripts * Clean up symbols project for AE since they are included in DNN project now * Add previously untracked Yahoo compressor dlls * Remove version information from SolutionInfo.cs * Fix errors introduced in rebase * Fix incorrect paths * Change build order of MSBuild to build extensions before personabar ui. * Make white-space visible in log viewer (#3198) * DNN-34236: check the url with case insensitive. * DNN-29417 Username is changed to email address in the user profile if the "Use Email Address as Username" is enabled. * Fix to installer and the dependencies of the HTML and DA modules * DNN-24945 - Always redirect to the source portal page after creating (#3223) group * Fix Typo (#3234) * Improve querystring parse (#3242) * Improve props popup * Improve input checking of sites (#3243) * Fixes an upgrade issue that affects 9.4.1 => 9.4.2 upgrades (#3282) * Fixes an issue with a duplication of assembly binding * Removed hardcode version in manifest * Further reorganization of build process (#3236) * Introduce settings and add custom version nr * Ability to create a local dev site * Move node workspaces to root * Reshuffling of variables in MSBuild * Repairs * Fix * Allow local override of global build variables * Only save manifests if the file is not there. This prevents overwriting in case of a failed build before. * Ensure projects build in debug to the website path specified by the platform build settings * Don't track vscode folder * Fixes #3168 Test Sending Email Issues (#3237) * Use Admin Email for Send if smtpHostMode false * Fix Test Email Settings TO and FROM Addresses If I did this right and it should make sense is I set the test email to be able to go to the user testing. After all they are who are trying to make it work. It can be any email used by anyone and usually to see this feature you are a host or portal administrator so security should not be as much of a concern here. The FROM address will use the HOST email settings if set to HOST mode in DNN for the portal, otherwise it should use the Portal Administrators Email account to send from. This however does not address the issue of using the email address supplied in settings and creating another setting which will be applied next to figure out the correct logic. * Update ServerSettingsSmtpAdminController.cs * Fix build issue to previous changes. PortalSettings.UserInfo.Email is this the current user email? I am still getting my things setup. I ran VS 2019 to build successfully and I am trying to read through the namespaces correctly so I can understand the changes. I noticed a number of files possibly needing updates due to being deprecated in 9.4.2 but after I changed the namespace issue those warnings went away. One last thought is how do you get intellisense working on DNN solution. Is that a dream to have or am I missing something in my setup? * Change to address to the current user's email testing the server. I believe a UI to set the TO address would be nice to help troubleshoot and address issues trying to send to a specific user. * Corrected To address for current user email * TO Current User Email * DNN-29110 Site Assets > Select All is not working (#3251) * DNN-29110 Site Assets > Select All is not working * Code review fix * Moves Remember Login above Login Button: referenced in issue #2471 #3255 (#3256) * Move Remember Login Above Login Button * Update Login.ascx * file copy paste issue... * DNN-34250 Search is not working even after re-Indexing Search for All DNN Portals (#3260) * DNN-32474 - Recursive delete option is added (#3249) * Move Country Above Region in UserProfile.cs * spacing issue resolved. * I keep saving but github is moving it on its own. * Fixed tab for spaces to resolve spacing issues * Add PortalSettings overloads back (#3295) Fixes #3289 * Updates issue templates as per 9.4.2 release * Move Email Above Username in User.ascx #3305 (#3306) * Move Email Above Username in User.ascx * Fixed tab index values in User.ascx * Fix NuGet warning NU5048 (#3304) The iconUrl property of the nuspec file is deprecated, it's recommended to add an icon property, which points to a file within the package, while also retaining the iconUrl property for older clients. See https://docs.microsoft.com/en-us/nuget/reference/nuspec#iconurl * Word-Break added to Journal P (#3307) * Set core object versions to core version (#3287) Ensure skin and radeditorprovider get the core version nr and add SPROC to set the version of all core packages/desktopmodules to the core version Fixes #3239 * Third installment of build reorganization (#3285) * Improve versioning * New approach: keep versions in source control and change upon CI build * Remove previous versioning logic * Cleaning up tasks * Fixes * Ensure UpdateDnnManifests targets just the right manifests * Harmonize naming of zips and nuget packages * Testing webpack building to dev website * Adjust webpack projects in AE to build to dev site * Fixes * Delete duplicate stuff * This is generated by Webpack and shouldn't be tracked * Include missing project from Lerna * Fixes for building SitesListView * Update to packages * Further cleaning * Remove RadEditorProvider * Fix ModuleSettings > Added to Pages grid, pager wasn't working * Allow only alphanumeric characters to be used when forming url from website name (#3316) * Ensure UpdateDnnManifests doesn't run for building dev site (#3314) * Reload page when the code tells you to (#3315) * Reload page when the code tells you to * Optimize code * Google Analytics TrackingID is stored in lowercase #3318 (#3322) * Update sums.resources * Fix casing mismatch errors in security analyzer file compare * Update Dnn.AdminExperience/Extensions/Content/Dnn.PersonaBar.Extensions/Components/Security/Checks/CheckDefaultPage.cs Co-Authored-By: Brian Dukes <bdukes@engagesoftware.com> * Registered IPersonaBarContainer with DI (#3329) * Document approval process and group * Fix typo * Relaxed nuget.cake wildcard (#3331)
This is the first in a series of revisions of the way we build DNN. It follows the merging on AE into the main repository. What this merge does is the following:
Move all content from ./Website to ./DNN Platform/Website and merge with what was there. The goal is to make it clear what is under source control and what is the website being built. ./DNN Platform/Website is source. ./Website is where the build process copies content to so it can start packaging.
Move the DNN packaging from MSBuild to Cake. Cake is better suited for this type of task and the old MSBuild tasks were very convoluted due to legacy code.