WebDriver.Execute Error Handling

[MarkJohnstoneGitHub]

It appears could move the error handling in WebDriver.Execute to WebDriver.SendRequest as have access to the HRESULT/HTTP status code in WebDriver.SendRequest . That would tidy up the Execute function.

HTTP status codes
https://learn.microsoft.com/en-us/previous-versions/windows/desktop/ms767681(v=vs.85)
https://www.w3.org/TR/webdriver/#dfn-error-code

In WebDriver.SendRequest

Select Case client.Status
    Case 400, 404, 405, 500 
        Err.raise client.Status, "WebDriver", client.responseText  
End Select

'Or maybe. Note if status codes >= 400 are only errors then change accordingly
Select Case client.Status
Case 200 'OK
Case Else
Err.raise client.Status, "WebDriver", client.responseText
End Select

If client.responseText isn't anappropriate errror message then maybe the IsResponseError(response) and GetResponseErrorMessage(response). Maybe in the calling command eg NavigateTo could Raise a specific error message from the response.

[GCuser99]

will take a look and get back to you - thanks!

[6DiegoDiego9]

@MarkJohnstoneGitHub, the errors handled in the Execute method are related to the web response while the errors in client.Status in the SendRequest method are related to the communication with the local webdriver at 127.0.0.1.
They're two different kind of errors and I believe that Execute is the proper place where the first kind of errors should (continue to) be handled.
The second kind of errors should never happen. If they happen, it would be good to handle them in SendRequest. Do they happen to you?

[MarkJohnstoneGitHub]

@6DiegoDiego9 #Regarding "the errors handled in the Execute method are related to the web response while the errors in client.Status in the SendRequest method are related to the communication with the local webdriver at 127.0.0.1." etc

I don't think that's that case. What is occuring you send request and a HTTP status is returned in the client.Status etc. So you can determine if a error occured from the client.status. Eg below output for when testing SendKeys and argh accidently used the URI Template NavigateTo instead of ElementSendKeys .

So you can determine if an error occured from the client.status and client.statustext. If need more details for the error can be extracted from the responseText. Possible at a future date use a dictionary of errorr messages https://www.w3.org/TR/webdriver/#dfn-error-code . The error in the responseText would be handy to look it up.

Eg. The response returned from an bad request mentioned above

Request Method : POST
Request URL : http://localhost:9515/session/813046ee1ab07ffb161d87af307bedfd/url
Json Request Body : {"text":"Leonardo da VinJci","value":["L","e","o","n","a","r","d","o"," ","d","a"," ","V","i","n","J","c","i"]}
client.Status 400
client.statusText Bad Request
client.responseText
{"value":{"error":"invalid argument","message":"invalid argument: 'url' must be a string\n (Session info: MicrosoftEdge=108.0.1462.54)","stacktrace":"Backtrace:\n\tMicrosoft::Applications::Events::EventProperties::SetProperty [0x00007FF71A7391F2+14306]\n\tMicrosoft::Applications::Events::EventProperty::EventProperty [0x00007FF71A6C09B2+842898]\n\t(No symbol) [0x00007FF71A371D90]\n\t(No symbol) [0x00007FF71A3F044E]\n\t(No symbol) [0x00007FF71A3D5D2A]\n\t(No symbol) [0x00007FF71A3A836E]\n\t(No symbol) [0x00007FF71A3EF70C]\n\t(No symbol) [0x00007FF71A3D5B03]\n\t(No symbol) [0x00007FF71A3A73B8]\n\t(No symbol) [0x00007FF71A3A638E]\n\t(No symbol) [0x00007FF71A3A7AE4]\n\tMicrosoft::Applications::Events::EventProperty::EventProperty [0x00007FF71A60FDF8+119000]\n\tMicrosoft::Applications::Events::EventProperty::EventProperty [0x00007FF71A5FD4D6+42934]\n\tMicrosoft::Applications::Events::EventProperty::EventProperty [0x00007FF71A6006EC+55756]\n\t(No symbol) [0x00007FF71A477923]\n\tMicrosoft::Applications::Events::Eve
ntProperty::EventProperty [0x00007FF71A6C846A+874314]\n\tMicrosoft::Applications::Events::EventProperty::EventProperty [0x00007FF71A6CD564+895044]\n\tMicrosoft::Applications::Events::EventProperty::EventProperty [0x00007FF71A6CD6BC+895388]\n\tMicrosoft::Applications::Events::EventProperty::EventProperty [0x00007FF71A6D67DE+932542]\n\tBaseThreadInitThunk [0x00007FFEA7077614+20]\n\tRtlUserThreadStart [0x00007FFEA89A26A1+33]\n"}}

[MarkJohnstoneGitHub]

I created a WebDriverCommand class to handle setting parameters, sending commands and obtain the required response eg GetText, GetElement, GetElements etc. That simpflies some confusing parameter settings i.e. placing the SesssionId parameter in the Excute function and parsing of the Endpoint URI template etc. It's a bit messy and confusing the way parameters are handled. Simplier to seperate into obtain path from the URI Template and Request Body parameters. Also properties for the Status etc for error handling. So far working fine and just slowly going through commands and updating.

An Example of the calling code for using WebDriverCommand.cls

Public Sub NavigateTo(ByVal url As String)
    Dim driverCommand As WebDriverCommand
    Set driverCommand = WebDriverCommand.Create(Me, EndPoints.NavigateTo)
    driverCommand.AddSessionParam sessionId_
    driverCommand.AddURLParam url
    driverCommand.SendRequest
    
    '@TODO("Add Error handling in WebDriverCommand.SendRequest or in each command??")
    'i.e  If Not driverCommand.Status = 200 Then '"OK"
End Sub

Note. I renamed WebJSonConverter to JsonConverter.
Also used the URI template from https://www.w3.org/TR/webdriver/#dfn-Endpoints using {variable name} instead of $variable name. Currently work in progress and still modifying WebDriverCommand.cls as go through the various commands.
Possibly rename Endpoint and Endpoints, might need a more appropriate name.
Likely place Endpoints into a dictionary to help lookup it's corresponding error in the future. It won't change the interface.

WebDriverCommand.txt
Endpoint.txt
EndPoints.txt

[MarkJohnstoneGitHub]

@6DiegoDiego9
Thanks. Will check later when get around to testing what status codes are sent back from client.Status and the response status, if they correspond etc. Agree appear to be two different kinds of errors maybe could handle any errors raised by client.Status.

It will be a while before get around to testing the errors returned by client status etc.

The Excute function is doing a couple of functions with parse the URI and obtaining the response error message. Probably a good idea to place the URI Template parsing into a separate function at least.

[6DiegoDiego9]

The first kind of errors (remote) cannot be seen in client.Status.
client.Status should always be "200 OK" whatever the remote page returns.
client.Status is just about the communication with the local webdriver at 127.0.0.1.

[MarkJohnstoneGitHub]

@6DiegoDiego9 Doesn't appear to be the case. It doesn't sense that would be always OK. You send a request to the selenium server and it sends back a response i.e. the client.status client.responsetext etc. If you enter in a bad request it shows up in the client.status and responseText etc. Most errors shouldn't occur if sent valid requests controlled by the client app. Only test case can think of is an invalid URL atm. Possible others if you manually closed windows before the client app does. Interesting case the URL redirect must get around to testing that.

[GCuser99]

One solution to the "tidiness" problem is to merge Execute and SendRequest, as SendRequest is only called by Execute and no other procedure. How would this work for both of you?

Friend Function Execute(driverCommand As Variant, Optional parameters As Dictionary = Nothing, Optional ByVal raise As Boolean = True) As Dictionary
    Dim cmdMethod As String
    Dim cmdPath As String
    Dim cmdArgs As New Dictionary
    Dim parmKey As Variant
    Dim response As Dictionary
    Dim client As New MSXML2.ServerXMLHTTP60
    Dim jc As New WebJsonConverter
    
    cmdMethod = driverCommand(0)
    cmdPath = driverCommand(1)
    
    If parameters Is Nothing Then
        Set parameters = New Dictionary
    End If
    
    'Set session id
    If Not parameters.Exists("sessionId") Then
        parameters.Add "sessionId", sessionId_
    End If
    
    For Each parmKey In parameters.keys
        If InStr(cmdPath, "$" & parmKey) > 0 Then 'path parameter
            cmdPath = Replace(cmdPath, "$" & parmKey, parameters(parmKey))
        Else 'non-path parameter
            cmdArgs.Add parmKey, parameters(parmKey)
        End If
    Next parmKey
    
    'there are commands that don't require sessionId in path: tCMD.CMD_STATUS, tCMD.CMD_NEW_SESSION, tCMD.CMD_GET_ALL_SESSIONS, tCMD.CMD_SHUTDOWN
    If cmdArgs.Exists("sessionId") Then cmdArgs.Remove "sessionId"
    
    'Send request to selenium server

    client.Open cmdMethod, driverUrl_ & cmdPath
    
    'set receiveTimeout to infinite to accommodate longer than 30 sec (default) pageloads
    client.setTimeouts 0, 60000, 30000, 0
    
    If cmdMethod = "POST" Or cmdMethod = "PUT" Then
        client.setRequestHeader "Content-Type", "application/json; charset=utf-8"
        client.setRequestHeader "Cache-Control", "no-cache"
        client.send jc.ConvertToJson(cmdArgs)  '
    Else
        client.send
    End If

    Do While client.readyState < 4
        DoEvents
    Loop

    Set response = jc.ParseJson(client.responseText)
    
    If raise And IsResponseError(response) Then 'raise error if occurs
        Debug.Print GetResponseErrorMessage(response)
        Err.raise client.status, "WebDriver", GetResponseErrorMessage(response)
    End If

    Set Execute = response 'Pass the response dictionary object - let caller parse info based on context, including error
End Function

[MarkJohnstoneGitHub]

@GCuser99 I was suggesting KISS principle and separate functions for ParseURI, SendRequest and GetResponseErrorMessage instead of lumping all the code in Excute. Apart from tidying up it makes testing/maintenance easier. And call from Excute.

[6DiegoDiego9]

I find a bit annoying to climb so many subprocedures (SendRequest → Execute → ...) in the call stack when debugging bad FindElements. This is a reason for me to prefer merging SendRequest into Execute.

[MarkJohnstoneGitHub]

Step in the right direction. Yeah annoying transversing the additional subprocedures when debugging.

You're still lumping in parsing parameters and sending request code in one function i.e. doing multiple things. If had functions for GetPath and GetCmdArgs would tidy up the function. The approach been working on I kept the URI template parameters and request body parameters separate. It makes parsing the URI template to get path and the request body simpler as well.

[6DiegoDiego9]

+1 to merge the two methods
-1(!) to remove the "response" error handling, that I used by the unHighlight method to spot ignorable errors (eg. stale elements) and could be generally useful for the user

Your mix of "If [...] IsResponseError(response) Then [...] Err.raise client.status" is actually a bug.
Client.Status is related to the http communication with the local webdriver at 127.0.0.1 while "response" is related to the web page.
Client.Status will always be "200 Ok" unless there's a serious internal problem, that likely never happened to us.

When SeleniumVBA for example opens a web page, if that page is not found, client.Status will still be "200 Ok" while "response" ("response" only) will contain the wanted "404 - no such element" error.

We could just rename lHttpStatus to lRemoteHttpStatus to avoid confusion...

[GCuser99]

@6DiegoDiego9, I think I understand what you are saying, and it makes sense to me, but I cannot replicate. When using the merged Execute that I posted above, every error that I have generated (so far) the client.status equals one of the remote response codes (400-500). I'm only getting 200 when there is no error. What am I missing? Can you provide an example?

Thanks!

[6DiegoDiego9]

Ahhh I've had a better look and I've to apologize! your solution @GCuser99 is actually perfect and the best one for me!

The thing that misled me was the fact that the following line passes with a client.status = 200:
NavigateTo "https://www.google.com/zzzzzzzzzzzzzzzzz.html"
However, I now see that "response" didn't contain the error either! hmmmm

[GCuser99]

Yes, very interesting - browser returns 404 but driver returns 200.... head scratcher...

[GCuser99]

I guess that result is because Google redirects to its own 404 page. If you try

NavigateTo "https://www.gooooooogle.com"

.... then driver returns 500 unknown error
🙂

[GCuser99]

Another thing this discussion exposes - I looked a little closer as to why the error messages seem to have a bunch of escaped characters... I think we need to make this modification:

 Private Function GetResponseErrorMessage(resp As Dictionary) As String
    'Dim jc As New WebJsonConverter 'old way
    GetResponseErrorMessage = vbNullString
    If TypeName(resp("value")) = "Dictionary" Then
        If resp("value").Exists("error") Then
            'GetResponseErrorMessage = jc.ConvertToJson(resp("value")("message"), 4) 'old way
            GetResponseErrorMessage = resp("value")("message") 'new way
        End If
    End If
End Function

[6DiegoDiego9]

Agree 100%.
I don't see a good reason for converting the message to JSON and doing so even seems to make it appear worse by combining multiple lines into a single line with "\r\n" separators.

[MarkJohnstoneGitHub]

@6DiegoDiego9 "the errors handled in the Execute method are related to the web response while the errors in client.Status in the SendRequest method are related to the communication with the local webdriver at 127.0.0.1."

I did some brief testing what was returned from client and response for the NavigateTo with an invalid URL using Chrome.
I got the same results from the client and response objects. Agree they are different but appear return the same regarding errors. At least expected different error messages.

Using GetErrorMessage post above by @GCuser99 and a GetErrorStatus(response) extracted from the Excute function. For testing I created a separate class for handling the response for testing WebDriverResponse.cls. Possibly handy if fleshed out to search the WebDriverResponse dictionary for other WebDriver commands. I might look into that later.

Invalid URL NavigateTo
client.Status : 500
client.statusText : Internal Server Error
client.responseText : {"value":{"error":"unknown error","message":"unknown error: net::ERR_NAME_NOT_RESOLVED\n (Session info: chrome=108.0.5359.125)","stacktrace":"Backtrace ... etc

From
GetErrorStatus(response) : 500
GetErrorMessage(response) : unknown error: net::ERR_NAME_NOT_RESOLVED
(Session info: chrome=108.0.5359.125)

VERSION 1.0 CLASS
BEGIN
  MultiUse = -1  'True
END
Attribute VB_Name = "WebDriverResponse"
Attribute VB_GlobalNameSpace = False
Attribute VB_Creatable = False
Attribute VB_PredeclaredId = True
Attribute VB_Exposed = False
'@Folder("SeleniumVBA")
'@PredeclaredId
Option Explicit

'============================================='
'Types
'============================================='
Private Type TWebDriverResponse
    mResponse As Scripting.Dictionary
    mErrorStatus As Long
    mErrorDescription As String
End Type

'============================================='
'Private Variables
'============================================='
Private this As TWebDriverResponse


'============================================='
'Constructor
'============================================='
Public Function Create(resp As Scripting.Dictionary) As WebDriverResponse
    With New WebDriverResponse
        Set .Response = resp
        Set Create = .Self
    End With
End Function

'============================================='
'Properties
'============================================='
Public Property Get Self() As WebDriverResponse
    Set Self = Me
End Property

Public Property Set Response(resp As Scripting.Dictionary)
    Set this.mResponse = resp
    If Not this.mResponse Is Nothing Then
        If IsResponseError() Then
            this.mErrorDescription = this.mResponse("value")("message")
            this.mErrorStatus = GetErrorStatus(this.mResponse)
        End If
    End If
End Property

Public Property Get ErrorStatus() As Long
    ErrorStatus = this.mErrorStatus
End Property

Public Property Get ErrorDescription() As String
    ErrorDescription = this.mErrorDescription
End Property

'============================================='
'Public Methods
'============================================='

'============================================='
'Private Methods
'============================================='
Private Function IsResponseError() As Boolean
    IsResponseError = False
    If TypeName(this.mResponse("value")) = "Dictionary" Then
        If this.mResponse("value").Exists("error") Then
            IsResponseError = True
        End If
    End If
End Function

Private Function GetErrorMessage(resp As Dictionary) As String
    GetErrorMessage = vbNullString
    If TypeName(resp("value")) = "Dictionary" Then
        If resp("value").Exists("error") Then
            GetErrorMessage = resp("value")("message") 'new way
        End If
    End If
End Function

Private Function GetErrorStatus(resp As Scripting.Dictionary) As Long
    Dim responseStatus As Long
    Select Case resp.Item("value").Item("error")
        'Complete list of error codes & numbers taken from here https://www.w3.org/TR/webdriver/#dfn-error-code as of 18-oct-2022
        Case "element click intercepted", "element not interactable", "insecure certificate", "invalid argument", "invalid cookie domain", "invalid element state", "invalid selector"
            responseStatus = 400 'Bad Request
        Case "invalid session id", "no such alert", "no such cookie", "no such element", "no such frame", "no such window", "no such shadow root", "stale element reference", "detached shadow root", "unknown command"
            responseStatus = 404 'Not Found
        Case "javascript Error", "move target out of bounds", "script timeout error", "session not created", "timeout", "unable to set cookie", "unable to capture screen", "unexpected alert open", "unknown error", "unsupported operation"
            responseStatus = 500 'Internal Server Error
        Case "unknown method"
            responseStatus = 405 'Method Not Allowed
    End Select
    GetErrorStatus = responseStatus
End Function

[6DiegoDiego9]

#48 (comment)