Test and release DSH ~~4.0.4~~ 4.1.0 #21
Comments
|
We need to check whether an SHM and SRE can be deployed from the current release-v4.0.4 (a3ff6c0e71e5c4d18d3817ee405b1312ae7f8037) |
@jemrobinson I've updated the todo list to complete this issue above, feel free to add any other steps you think are required |
|
I'll actually do this before #1467 - this way I can use the new SHM I'll have set up to test changes to the Firewall rule |
|
After deploying the SHM and running through the setup steps I got to Configure MFA and then got this error: |
|
Are you running this command on the NPS server? |
Yes |
|
Did you get a prompt (in a web browser) for login credentials? |
|
Yes, I logged in with the aad.admin account I set up for this AAD |
|
OK, if you run the script again (following the troubleshooting steps) does it still fail? Have you gone through the other troubleshooting steps yet? |
|
Have tried a couple of times, but this error seems to be slightly different from the ones in the troubleshooting steps, will have a look at those in case it's the same issue though |
|
Ah, hadn't set up MFA for the aad.admin account apparently - sorted now |
|
@craddm @jemrobinson I've deployed a new test SHM with 4.0.4 called "Orange" and will test building an SRE once I have my new laptop set up. What should we do with regard to the other SHMs set up:
|
|
Also, are there any particular tests that should be run other than the SRE smoke tests? |
|
|
For |
|
Despite the SRD VM image build not raising any problems (see alan-turing-institute/data-safe-haven#1514 (comment)) there seems to be an issue with Full error messages from smoke testsDo we need to update a package list somewhere other than |
|
This is because the test tries to run |
|
@jemrobinson We should now be able to make a release for the |
|
Have you done all the pre-release check? Not sure we've got these written down anywhere (@JimMadge do you know?) but see e.g. release 4.0.0. We need:
|
|
I have the:
I think we should formally decide then which checks should be done for which releases, were all of those things done for each of the previous patch releases e.g. 4.0.3/4.0.2? I would guess no? https://github.com/alan-turing-institute/data-safe-haven/releases Reading through the security checklist just now it felt like if anything this is something we should be doing on a per deployment basis - though maybe we think that's too laborious and so we assume that reproducible IAC principles mean we can do it for a version? Even in this case, should we do it for every patch release or just minor or major releases? |
|
I agree that these weren't done for previous patch releases, but I'm not sure whether this is still a patch release (bug fixes only) or if it's a minor release now. Here's the diff (alan-turing-institute/data-safe-haven@v4.0.3...release-v4.0.4) what do you think @edwardchalstrey1 @JimMadge @craddm ? Can you post a changelog here @edwardchalstrey1 (in the style of previous release changelogs) so we can judge better? |
Looks like @JimMadge made one already: https://github.com/alan-turing-institute/data-safe-haven/releases/tag/untagged-a52657d965ec43f09888 I've updated that with the recent PRs to the release branch and dropped "beta" from the release name |
|
My thoughts are we should decide and document the checks that we want to do for the 3 levels of versioning, then decide what this release should be and if needed change the number (e.g. 4.1.0) - I'm not against doing the full set of checks you describe @jemrobinson even for patch releases, but I think we should decide that first. |
|
Looking at what we've done in the past, it's:
What sounds reasonable to you: @edwardchalstrey1 @JimMadge @craddm for each of these release types? |
|
What does the penetration test involve? Maybe releases that have security fixes (as this does), should be at least a MINOR and subject to all these tests, whereas releases that only have feature additions, non-security related bug fixes and documentation changes can be patches with no testing except perhaps the usual smoke tests on a deployed SRE? Also just thinking now, would it be normal in software to bundle unrelated changes into a release in the way we have been doing? This does seem fine to me, but if we're taking this approach as oppose to doing releases for a set of related features, should we have a pre-defined interval, e.g. make a release once per quarter, and decide the patch/minor/major increment based on whatever is in it? |
|
The penetration tests are carried out by a third-party (iSTORM). We set up an SHM and a couple of SREs and give them user accounts and various admin accounts. They try to find exploits and report back to us. Typically takes around a week or two weeks for an in-depth test. Results of previous penetration tests are in Sharepoint under
We've already said that MINOR means "Adds new functionality, but new SREs can be deployed into existing environments" while PATCH means "Fixes bugs only and new SREs can be deployed into existing environments". I think security fixes can still be a PATCH release.
I guess the real question is: what kind of change can we make where we're ~100% certain that it won't break our SHM or SRE scripts. Documentation changes are in that category, removing MSRDS support is not. Where's the dividing line and is it at the current patch/minor boundary?
For security fixes, it's important to get a patch release out as soon as possible. Major and minor don't have that constraint. The problem is that preparing a release takes away from development time so we generally want to have bundled a few important changes to make it worth doing. |
Meta point, why is this issue not in the DSH repo? |
I think can move it there, in fact, might it be worth having a GH issue template for releases that has a checklist of the above |
|
Let's leave this issue here, but create a new GH issue template in the DSH repo with a checklist of whatever we agree here. @JimMadge How about something like:
|
Although note that the security checklist requires setting up two SREs (t2 and t3) |
|
Is the argument that if we make a security fix to the SHM, that will always be a major release, so we don't need to check SHM deployment? |
|
@jemrobinson question - for the SHM/SRE/SRD logs, where do/should we save these? |
|
Save them in Sharepoint in under |
thanks, I guess we should record that somewhere internally, maybe on the Here's a PR for release template: alan-turing-institute/data-safe-haven#1543 |
|
Closing this issue with the opening of alan-turing-institute/data-safe-haven#1544 |
20.04.2023050900dsgpeakSRE, need to investigaterelease-v4.0.4branch during the DSG, a test SRE should be deployed with the latest version before we can conclude all is wellrelease-v4.0.4branch ("Orange")20.04.2023062000)Orangewith latest 4.0.4 branch commitThe text was updated successfully, but these errors were encountered: