Check answers

[govuk-design-system]

Use this issue to discuss this pattern in the GOV.UK Design System.

[hannalaakso]

We tested Prototype Kit's Check your answers page on NVDA + Firefox + Windows 10). If NVDA is set to read all content from top of page, it reads out "List of 12 items" on encountering the definition list ("Personal items"). As 12 items here includes both the dt tags ("Name") and the Change links, this might not be helpful to the user. NVDA reads out List of x items with both ul and dl lists so it could be it's not clear to the user how many actual "items" there are in a definition list from NVDA's description.

[kr8n3r]

Posting here for visibility

AXe will report an accessibility issue with the Description List that is used in the pattern, with
"<dl> is immediately followed by a <div> and not a <dt>"

This is a bug in aXe and a pull request has been opened to address it.

More details on allowed markup in the Description List element: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/dl under Permitted content section.

[Ash-Wilson]

I've been testing an alternative version to this pattern within the Apply for Direct Rent Payments service. There is no user need to have a change link at each section, as some of the questions are on the same page and grouped to one thing per page e.g Tenant Details. It's tested really well and I wonder if it's worth having something like this as an alternative, depending on what the needs are?

[sdh100shaun]

We have also been looking at a similar pattern
at the DVSA - what are the thoughts on this pattern where each section has a change?

[joelanman]

@Ash-Wilson @sdh100shaun The official guidance is to start with One thing per page, in which case most answers would need their own 'Change' link. However if you find in research that grouping some questions on a page works better for your users, then you should do that.

Be really careful though - one advantage of One thing per page is it makes understanding and interaction simpler for everyone. People who have problems reading and understanding, or interacting with digital services can really benefit from One thing per page. This is normally more important than grouping questions to make it a bit quicker for other people.

[sdh100shaun]

thanks @joelanman , our UX person is going to test, especially as this fits into an established application.

[drewno-design]

We've tested alternative layout of the page - with values in bold and keys/labels in normal font weight. The alternative version performed better for users with sight issues. It was easier for them to read values that were bold.

This might be a case of a few values on a page, where it is not needed to scan a lot.

[jfranciswebdesign]

Every now and then we'll get feedback from users indicating they think "Check your answers" means they've made an error, as in they're interpreting the purpose of the page as 'You've made a mistake, check your answers, and correct accordingly". So a title change might be something to consider?

[MartinDM]

Am I right in saying that 'Check your answers' pattern is not part of the new Design System and is not in the govuk-frontend npm package?
https://govuk-prototype-kit.herokuapp.com/docs/templates/check-your-answers

Is it more a case of copying over the styles from Elements, in order to use the markup in the example? Just wanted to check it's not directly import-able before copying it over/re-implementing.
Thank you :)

[amyhupe]

Dropbox Paper audit

On 16 January 2019 the Design System team reviewed and removed a Dropbox Paper document discussing the Check answers pattern.

The aim was to reduce the number of places containing guidance and code by:

• migrating relevant, useful content into the Design System itself
• recording important research findings in the community backlog
• removing the original Dropbox Paper page

Below is a record of the outcomes of that review.

If you need to, you can see the original Dropbox Paper content in the internet archive.

Review outcomes

Updates to the Design System

The Design System team will carry out the following updates to ensure that relevant, useful content from the Dropbox Paper file is added to the Design System.

• Add guidance on how to name check answers pages to make clear that there is an action required of the user

[ChazUK]

I was wondering how people handle the navigation between this Check Answers page and Question pages, and are there any examples that people can point me in the direction of?

Question Pages are seen as a complete flow navigating through them in order until they are all completed. 1 -> 2 -> 3 -> 4 -> 5 -> Check Answers.
When a user decides to change an answer they are directed to the Question Page, when the user completes the question and clicks Continue are they then returned to the Check Answers page or forced to go through the remaining Questions? Would it be better to change the copy to be more explicit?

[drewno-design]

@ChazUK When user clicks "Change" on "Check answers" page we direct them to the page with the question. When on that page they click "Continue" we direct them straight to "Check answers" page. No need to go through entire flow again. Does that answer your question?

Example: Get MOT reminder: https://www.gov.uk/mot-reminder

[ChazUK]

Thanks @rafaldrewnowski, that definitely helps.

I like the use of "Save and return to review" as the Submit button, but I have noticed that you have used Back and Cancel and return. Attaching screenshots for anyone looking at this in the future.

[drewno-design]

That's a good call out. Thank you!

[jonathaninch]

Every now and then we'll get feedback from users indicating they think "Check your answers" means they've made an error, as in they're interpreting the purpose of the page as 'You've made a mistake, check your answers, and correct accordingly". So a title change might be something to consider?

We've come across this too IIRC. Maybe 'Confirm your answers' would be better?

[terrysimpson99]

Yes. I think 'Summary' fits well. See: https://design-system.service.gov.uk/components/summary-list/

[joelanman]

When we first worked on this pattern on Register to vote we originally had something like 'Confirm your answers' but some users misread it as a confirmation that they had submitted the details on that page - like a receipt. This meant they wouldn't actually complete the process. I would guess 'Summary' might have the same issue. So we changed it to be more active - a step they need to do, and it seemed to solve that problem.

Having said that, people thinking theres an error is a problem too - do you know how common and severe it was? Would people get stuck looking for an error that wasn't there?

[jfranciswebdesign]

When we first worked on this pattern on Register to vote we originally had something like 'Confirm your answers' but some users misread it as a confirmation that they had submitted the details on that page - like a receipt. This meant they wouldn't actually complete the process. I would guess 'Summary' might have the same issue. So we changed it to be more active - a step they need to do, and it seemed to solve that problem.

Having said that, people thinking theres an error is a problem too - do you know how common and severe it was? Would people get stuck looking for an error that wasn't there?

It wasn't that common, or severe - it just happens occassionally. I distinctly remember one person spending ages going back and forth between screens looking for a (non-existent) error. We did try "Check before you submit" which seemed to solve the problem.

[terrysimpson99]

We currently use 'Check your answers before submitting' but we'd like to use the same code in three places (pre-submit web page, pre-submit printed copy, and post-submit printed copy). We either change the title to something that applies in all three places (e.g. 'Summary') or we have different titles. We haven't decided yet so we're interested in this discussion.

[enoranidi]

We have been looking at a slight variation of the 'Check your answers' pattern where our users could potentially add a high amount of goods to their application (e.g. 60 for example).

Originally we had each item (x60) displayed as per this cropped screenshot:

But the page was very long and difficult to scan. We also observed that with this layout our users were unable to check incorrectly added items, duplicates, see missing goods, etc.

For our next iteration of the prototype, we tried using a 'Manage' link where the user would be taken to another page to then check and/or change their response, before returning back to the main 'Check your answers' page.

So far testing has come back with positive feedback with this implementation for our users.

And kudos to GOV.UK Notify team as I got my design inspiration from their service.

[NickColley]

Worth noting that this pattern helps meet WCAG Success Criterion 3.3.4: Error Prevention (Legal, Financial, Data) (Level AA)

Guidance: Providing the ability for the user to review and correct answers before submitting

[jonathaninch]

Not sure 'Change' is always the best word to use on CYA pages.

We have examples of UR where a user needs to check the input of another user before it's submitted to HMRC (pension scheme administrators). In those cases, the user doesn't want to change the input but to view / check / review it. Of course, if they find an error they will want to change it in that instance.

Difficult finding the right word to cover 'reviewing but also potentially changing'...

[terrysimpson99]

Please consider adding something about why this page is two-thirds and indicate an option to go full width. We have at least one, possibly more, services where we believe full width would suit users better.

After asking on slack comments included:
"It reduces line lengths which makes them easier to read. It means the action links arent so far to the right, so people using screen magnifiers are less likely to miss them."

"In many services, the user's answers will tend to be shorter - which is more suited towards 2/3 layout. I don't think the layout is ideal for displaying paragraph like answers though"

"sure, you can go full width if you need to"

Those answers were very helpful. Please can the information in those comments be considered for inclusion?

[yantantether]

I'd like to query the use of a 'govuk-visually-hidden' class mid-sentence.
e.g.

<a class="govuk-link" href="#">
  Change<span class="govuk-visually-hidden"> name</span>
</a>

We have a project that are localising content to different languages, and we had to be sure not to assume English grammar can be re-used in a different language when coding it up. To localise the code above, we might assume we can simply translate 'Change' and 'name' separately and put them together in the same order as English grammar. I don't think we should make that assumption.

Could we consider having more complete sentences as visible and accessible labels?
e.g.

<a class="govuk-link" href="#">
  <span class="govuk-visually-hidden">Change your name</span>
  <span aria-hidden="true">Change</span>
</a>

Is anyone looking at making the localisation of toolkit templates easier btw?

[ChazUK]

Depending on your chosen translation approach you should be able to include HTML in your translatable fields, which is how I've used it in the past.

Although I do like your approach.

[jbuller]

There is an accessibility argument for having distinct text of link/buttons visible, rather than hidden.
Non-keyboard and mouse users with Dragon voice control software currently have to say 'Click Change' or 'Click button' then specify which one of the highlighted matching elements they want.
If the text is unique they can say 'Click change name' and avoid the extra step.
It doesn't do the aesthetic any good though.

[DavDoh]

We tested the Check your answers pattern with a series of GP's who have a lot of text-heavy detailed content to review.

We tweaked the Check your answers template by instead of having a govuk-grid-column-two-thirds layout, to a full govuk-grid-column-full layout. The users responded very well to this layout. I would suggested offering an expanded version in the Design System for text-heavy answers.

[hannalaakso]

@DavDoh Apologies for the delay in replying and thanks for sharing your research 🙌

Your proposal sounds similar to what has been suggested in alphagov/govuk-design-system#1290. I've added a comment to that issue signpost it to your findings here. Feel free to add any further comments to that issue.

[joelanman]

@DavDoh if you have a lot of text it can be helpful to go full width to fit it in, but specifically in your screenshot there isn't a lot of text, and then the danger is that screen-magnifier users don't realise the Change links off on the right, as theres a large gap.

[f-marry]

What's the recommended design for optional question that have been seen but were not answered?
The non answer is meaningful, so I'd show the question, but what to put in the "answer" place? Leave blank or spell out "not answered"?

[frankieroberto]

@f-marry good question! I’ve started to look at what you might do if there’s no answer given but and answer is required (which can happen in a few different contexts) here: alphagov/govuk-design-system#1668

For optional questions though, our service currently uses placeholder text ("Empty") in dark grey. I think that’s probably better than an empty space, but I’m not entirely sure. We also currently add an "(optional)" text to the key, to remind users it is optional.

However I'm increasingly not a fan of optional textfields at all, as users sometimes expect all fields to be required. One alternative is to ask a guard question first, eg "Do you want to provide any additional background?" and then display a textarea if they answer Yes.

[cjforms]

On the 'check your answers' page, I'd recommend:

• saying "you chose not to answer" or similar, I'm not content designing here
• making sure the question also has 'optional'

On the problem of whether or not to have optional question: I have definitely come across plenty of circumstances where users wanted to provide further information even though it wasn't strictly required. A few examples:

• delivery instructions ("If I'm out, leave behind red door")
• times when it is acceptable to contact by phone ("Do not call after 8pm", "I am not allowed to take personal calls in working hours")
• parts of a story that the user considers to be relevant but that have not yet been asked for ("I need this money urgently", "I wasn't sure how to answer question X but I have tried my best", "Please take xxx other thing into consideration when you are dealing with this").

When I was working on the design of the Self Assessment tax return, there was an 'additional information' box that was in fact required in a particular and very rare circumstance that I now forget, but we learned that many taxpayers used it for all sorts of things - most typically "the answer in box xx.xx is an estimate" (even when it had no tax consequences).

There is definite merit in @frankieroberto's suggestion of a guard question (I call these 'filter questions'), because in a sense the user has to ask themselves that guard question ("Do I want to provide additional information?") and it often helps to make those sorts of internal conversations explicit on the page. But in this case, it looks like we might be heading to a yes/no question with no third option - and those always worry me, because the real world is so rarely binary.

[edwardhorsford]

Like @frankieroberto I make sure to always have something in the value column of a summary list - either Not provided or similar.

One other point though - sometimes it's better to combine several fields and show them as a single row in the summary list. This is implied on the design system, but I'm not sure it's really spelt out. Often optional fields fit as part of some other data - eg part of a full name row, part of an address.

The example from the design system has this:

Which presumably should link to an address form with an optional address line 2.

[jonathanbranthwaite]

In earlier iterations of our Attendance Allowance journey, applicants were only able to review their answers at the very end of the application, as per the pattern.

Being overwhelmed and fatigued led users to having a lack of confidence with what they had inputted.

Using smaller intervals of check your answers tested well for Bereavement Support Payment (BSP) who have cognitively overloaded users.

Because of the size of our journey, we believed that it would give our users more manageable sections to review and move on.

Various rounds of usability testing findings have been positive:

• Most users reviewed their answers when they reached each of the check your answers pages
• By breaking sections down throughout the journey, it helped users to progressively review as they completed each section
• Check your answers worked well in giving confidence to applicants as they progressed through the form, as well as switching context from section to section

However, recent end-to-end testing has shown some users thought that they had completed their application after the first CYA page. So this is something we will continue to monitor and look at how we can better manage their expectations.

We have looked at some other teams using progress indicators. BSP, for example, added ‘You have completed section 1 of 4’ for their smaller section CYA pages, so this is something we could explore here too.

[Scipio-AG]

We used a task list to help visualise the journey to users. This gave them more autonomy when completing the application form and also meant they had a better understanding of the journey/ what to expect. This pattern could help with the issue of users believing they have completed the journey at CYA 1.