[Playwright version 4.9.0] / Microsoft.Playwright.PlaywrightException : TypeError: No color space found with id = "oklch"

[evelynmcr]

Which package were you using when you encountered the issue?**

• Deque.AxeCore.Selenium
• Deque.AxeCore.Playwright
• Deque.AxeCore.Commons

Describe the bug

We have a .NET project which uses Playwright Axe for accessibility testing of our site. These tests have been running successfully so far. We just got a dependabot PR in our project to bump the version of Deque.AxeCore.Playwright from 4.8.2 to 4.9.0. However, a lot of our tests have failed with this version change due to an issue with the oklch color space. Error message can be found below.

I tried rerunning our tests on our main branch (where Playwright version is still 4.8.2) and they all pass. It's only on the PR with the new version of Playwright (4.9.0) that they fail. I don't have much knowledge about color spaces; looking at the diff for v4.8.2 vs v4.9.0 I'm unsure what in the new version would have caused this.

We're using .NET version 8.0 and Microsoft.Playwright version 1.42.0, if relevant.

Any guidance appreciated.

Error Message:
   OneTimeSetUp: Microsoft.Playwright.PlaywrightException : TypeError: No color space found with id = "oklch"
    at Function.value (eval at evaluate (:226:30), <anonymous>:12:218661)
    at F1.t.space (eval at evaluate (:226:30), <anonymous>:12:236527)
    at F1 (eval at evaluate (:226:30), <anonymous>:12:237122)
    at A1 (eval at evaluate (:226:30), <anonymous>:12:238922)
    at O (eval at evaluate (:226:30), <anonymous>:12:247417)
    at eval (eval at evaluate (:226:30), <anonymous>:12:559003)
    at UtilityScript.evaluate (<anonymous>:233:19)
    at UtilityScript.<anonymous> (<anonymous>:1:44)
  Stack Trace:
     at Microsoft.Playwright.Transport.Connection.InnerSendMessageToServerAsync[T](ChannelOwner object, String method, Dictionary`2 dictionary, Boolean keepNulls) in /_/src/Playwright/Transport/Connection.cs:line 206
   at Microsoft.Playwright.Transport.Connection.WrapApiCallAsync[T](Func`1 action, Boolean isInternal) in /_/src/Playwright/Transport/Connection.cs:line 526
   at Microsoft.Playwright.Core.Frame.EvaluateAsync(String script, Object arg) in /_/src/Playwright/Core/Frame.cs:line 688
   at Deque.AxeCore.Playwright.AxeContent.DefaultAxeContentEmbedder.EmbedAxeCoreIntoFrame(IFrame frame)
   at Deque.AxeCore.Playwright.AxeCoreWrapper.DefaultAxeCoreWrapper.ConfigureAxe(IFrame frame)
   at Deque.AxeCore.Playwright.AxeCoreWrapper.DefaultAxeCoreWrapper.RunInner(IFrame frame, Object rawContextArg, AxeRunOptions runOptions)
   at Deque.AxeCore.Playwright.AxeCoreWrapper.DefaultAxeCoreWrapper.Run(IPage page, AxeRunContext context, AxeRunOptions options)
   at Deque.AxeCore.Playwright.PageExtensions.RunAxeInner(IPage page, AxeRunContext context, AxeRunOptions options, String axeSource)
   at Deque.AxeCore.Playwright.PageExtensions.RunAxe(IPage page, AxeRunOptions options)
   at Tests.Helpers.AxeHelper.CreateAxeReport(String pageName, IEnumerable`1 rulesToExclude) in D:\a\1\s\src\Tests\Helpers\AxeHelper.cs:line 53

[straker]

Thanks for the issue. Would it be possible to see the test code that generates that error? Specifically the HTML that Deque.AxeCore.Playwright is testing against. I can't think of anything that's changed from 4.8.2 to 4.9.0 that would cause this problem as we didn't do anything to update color spaces. 4.7.1 is where we added support for CSS 4 color spaces such as oklch

[evelynmcr]

Thanks @straker - yes it does seem odd but it's the only thing I could see in the changelog that looked semi-related to color spaces!

The snippet below is the code we're using for our TestHelper. The error in our tests can be traced back to the last line in this snippet ie var axeResult = await _page.RunAxe(options: options);

It's Chromium browser and we have also set the following in our tests:

AssertionOptions.FormattingOptions.MaxDepth = 100;
AssertionOptions.FormattingOptions.MaxLines = 300;

Snippet from TestHelper where the last line is causing the errors:

using AxeCore.HTMLReporter;
using Microsoft.Playwright;
using Deque.AxeCore.Commons;
using Deque.AxeCore.Playwright;
using Tests.Scenarios;

namespace Tests.Helpers;

public static class AxeRules
{
    public const string
        ColorContrast = "color-contrast",
        FrameTitle = "frame-title",
        ImageAlt = "image-alt",
        List = "list",
        ListItem = "listitem";
}

public class AxeHelper
{
    private readonly IPage _page;
    private readonly string _testName;
    private readonly SummaryReportGenerator _summaryReportGenerator;
    public readonly Dictionary<string, AxeResult> Results = new();

    private int _stepNumber = 1;

    public AxeHelper(IPage page, string testName, SummaryReportGenerator summaryReportGenerator)
    {
        _page = page;
        _testName = testName;
        _summaryReportGenerator = summaryReportGenerator;
    }

    public async Task CreateAxeReport(string pageName, IEnumerable<string>? rulesToExclude = null)
    {
        var options = new AxeRunOptions
        {
            RunOnly = RunOnlyOptions.Tags(TestSettingsHelper.Settings.RuleTagsToUse),
            Rules = new Dictionary<string, RuleOptions>
            {
                { "target-size", new RuleOptions{ Enabled = false }},
                { "definition-list", new RuleOptions{ Enabled = false }}
            }
        };

        if (rulesToExclude != null)
        {
            foreach (var rule in rulesToExclude)
                options.Rules.TryAdd(rule, new RuleOptions { Enabled = false });
        }

        var axeResult = await _page.RunAxe(options: options);

        [...]
    }
}

[straker]

Thanks for the code snippet. Do you by chance know what the HTML of the page looks like at the time the error is thrown? Would it be possible to put that HTML either in this issue or somewhere public (like a gist or codepen, etc.)?

[evelynmcr]

Thanks - it's a lot of different pages that had this test failure, here's one example: https://system.spektrix.net/qa_automation_accessibility/website/eventlist.aspx

[Zidious]

Hey @evelynmcr!

I spun up an example using the URL provided and wasn't able to replicate the same error message. The stack trace refers to a possible issue with an iframe, the example URL doesn't appear to have any iframes but I've tested other pages that do contain iframes and have not had that error thrown. Are you passing any specific Chrome options that could affect your tests?

  <ItemGroup>
    <PackageReference Include="Deque.AxeCore.Playwright" Version="4.9.0" />
    <PackageReference Include="Microsoft.Playwright" Version="1.42.0" />
  </ItemGroup>

using Deque.AxeCore.Commons;
using Deque.AxeCore.Playwright;
using Microsoft.Playwright;

public class Example
{
    public static async Task Main(string[] args)
    {

      var playwright = await Playwright.CreateAsync();
      var browser = await playwright.Chromium.LaunchAsync(
        new BrowserTypeLaunchOptions
        {
            Headless = false
        }
      );

      var page = await browser.NewPageAsync();
      await page.GotoAsync("https://system.spektrix.net/qa_automation_accessibility/website/eventlist.aspx");
      AxeResult axeResults = await page!.RunAxe();

      foreach (var violation in axeResults.Violations)
      {
          Console.WriteLine("Rule ID: " + violation.Id);
          foreach (var node in violation.Nodes)
          {
              Console.WriteLine("Node HTML: " + node.Html);
          }
      }  
    }
}

[evelynmcr]

Hi @Zidious , thanks for this. Very odd as we're not using an iframe anywhere other than on our payments page (we have an iframe for the card details / payment processing) but it's still the same error on all the pages that don't have this iframe.

Below are the only additional browser options we have configured:

var playwright = await MicrosoftPlaywright.CreateAsync();
var browserOptions = new BrowserTypeLaunchOptions
{
    Headless = false,
    SlowMo = 50 // Ensure sleep of 50ms between tasks to improve stability
};
Chromium = await playwright.Chromium.LaunchAsync(browserOptions);

[Zidious]

Hmm, interesting ya that shouldn't affect anything. I did notice:

   at Tests.Helpers.AxeHelper.CreateAxeReport(String pageName, IEnumerable`1 rulesToExclude) in D:\a\1\s\src\Tests\Helpers\AxeHelper.cs:line 53

Could you shed a bit more light on AxeHelper and rulesToExclude?

[evelynmcr]

Thanks - I actually posted the AxeHelper code further up here

Re: rulesToExclude, it differs depending on what page is being tested, but is one of the following:

AxeRules.ColorContrast
AxeRules.List, AxeRules.ListItem
AxeRules.ImageAlt
AxeRules.FrameTitle

[Zidious]

My mistake, missed that! Are you able to run my example outside of your existing test suite and do you get any errors relating to color space?

[straker]

@evelynmcr I just ran into this same error message on a different website. The issue was caused by the website loading a script that overrode native JS features (like array functions) which causes our script to work in unexpected ways, thus causing the error. They were using prototype.js. Do you know if your test pages that are erroring load any scripts that might override native JS features?

[evelynmcr]

@straker thanks - we do use prototype.js and some other scripts on some of our pages too so perhaps that's causing the issue. Do you have any idea why it's only with the new version of Playwright that it's interacting like this?

@Zidious - apologies, not had a chance to try running that example yet, can try it tomorrow!

[straker]

@evelynmcr I tried older versions and it looks like axe-core still errors but changes where the error occurs. Axe-core v4.8.4 errors when running color contrast and produces an incomplete for color contrast rule with a javascript stack error message, whereas v4.9.0 the error occurs when axe-core loads on the page and you get the error message about no color space.

[evelynmcr]

@straker thanks - so both errors about color, but still different? Our current version on main is 4.8.2 which runs the tests successfully. So I guess something must have happened between 4.8.2 and 4.8.4 if there's errors happening on the latter

(As an aside, we've upgraded our version of Microsoft.Playwright from 1.42.0 to 1.43.0 and the tests are still failing)

[evelynmcr]

Hi, just wondering if anything else has come of this. Every dependabot PR we get now for our tests repo (for other packages, but includes the Deque.AxeCore.Playwright upgrade too) fails until we remove the Deque.AxeCore.Playwright package upgrade to 4.9.0. I can't see 4.8.4 (to check for diff between 4.8.2 and 4.8.4) so presumably it's not available publicly? Any other thoughts? Thanks!

[straker]

@evelynmcr Unforntuatefnly I don't think there's anything we can do about the issue. Since we've determined it's the prototype.js file overriding native JS features that change their behavior, the fix would be to remove prototype.js. Color contrast fails to run in both the new and the old version, it's just the newer version changes where the error occurs so you see it at run-time rather than in the results object.

[evelynmcr]

As an update, we've upgraded to version 4.9.1 now and our tests are passing again! Seems like it didn't like 4.9.0 but 4.9.1 is good.

[straker]

Glad to hear, going to close this issue then.

As part of 4.9.1 we found a workaround to the prototype.js issue. It's just a workaround though so we still recommend removing prototype.js as it can continue to cause problems in the future by overriding native JS features.

[padmavemulapati]

Customer not seeing issue with the updated dependencies as well my tests are passing successfully with the apis supported.

Environment:

Label	Value
Product	axe-devtools-nuget
Version	4.9.2-pre-64115d6
OS-Details	_MAC- Intel 14.4.1 _
BrowserDetails	_Chrome Version 125.0.3  _
.Net Version	8.0