Skip to content
This repository has been archived by the owner on Oct 1, 2018. It is now read-only.

Refactor Progress

Jump to bottom
Z edited this page Aug 8, 2014 · 41 revisions

The purpose of this document is to track the parser architecture refactor works on #114.

Parser Architecture Refactor

The main interest are ***Parser.h files with the exception of the UriParser.h file and the parse() route. Necessary Markdown-AST processing files were already refactored and are not included in this list.

The list is ordered in the way the work should be done (first to last):

  • AssetParser.h
  • HeadersParser.h
  • PayloadParser.h
  • ParameterDefinitionParser.h
  • ParametersParser.h
  • ActionParser.h
  • ResourceParser.h
  • ResourceGroupParser.h
  • BlueprintParser.h
  • snowcrash.cc

Note: Section parsers are now solely a partial specifications of SectionProcessor.

Test Refactor

Tests should be refactored alongside with the respective files mentioned above, as such they are not listed on this Wiki.

Note: Tests should no longer construct Markdown AST, instead parse blueprint snippets using provided SectionParserHelper::parse()

Verification

  • Leak test
  • Performance test
  • Regression test

Performance Check

Add benchmark results and analysis collected using the perf target on a representative collection of blueprints (e.g. blueprints listed at Apiary.io homepage)

Leak Test

Using the perf detect and fix any possible memory leaks.

Regression Test

In coordination with Protagonist – use pre and after refactor versions of protagonist to compare serialized outputs of parsing an input blueprint

Double Content-Type

  • Resolved

Response payload with a Content-Type in signature when referencing a model with a Content-Type in its signature result in 2 Content-Type headers.

Given

# A [/a]
+ model (type)
    
        X

## GET 
+ response 200 (type)
    
    [A][]

Actual (after-refactor)

OK.

_version: 2.0
metadata:
name:
description:
resourceGroups:
- name:
  description:
  resources:
  - name: "A"
    description:
    uriTemplate: "/a"
    model:
      name: "A"
      description:
      headers:
      - name: "Content-Type"
        value: "type"
      body: "X\n"
      schema:
    parameters:
    actions:
    - name:
      description:
      method: "GET"
      parameters:
      examples:
      - name:
        description:
        requests:
        responses:
        - name: "200"
          description:
          headers:
          - name: "Content-Type"
            value: "type"
          body: "X\n"
          schema:

Expected (pre-refactor)

OK.

_version: 2.0
metadata:
name:
description:
resourceGroups:
- name:
  description:
  resources:
  - name: "A"
    description:
    uriTemplate: "/a"
    model:
      name: "A"
      description:
      headers:
      - name: "Content-Type"
        value: "type"
      body: "X\n"
      schema:
    parameters:
    actions:
    - name:
      description:
      method: "GET"
      parameters:
      examples:
      - name:
        description:
        requests:
        responses:
        - name: "200"
          description:
          headers:
          - name: "Content-Type"
            value: "type"
          - name: "Content-Type"
            value: "type"
          body: "X\n"
          schema:

Resolution

New behavior (actual) is OK and not considered a regression issue.

Unrecognized parameter

  • Resolved

Recognize parameter when there is no description on its signature and remaining description is not a new node.

Given:

# GET /{id}

+ Parameters
+ id (required, string)
  This is a description

Actual (after-refactor)

OK.
warning: (5)  ignoring unrecognized block :28:24;56:22
warning: (6)  No parameters defined in parameters section :13:65
warning: (6)  action is missing a response :0:13
warning: (1)  expected API name, e.g. '# <API Name>' :0:13

{
"_version": "2.0",
"metadata": [],
"name": "",
"description": "",
"resourceGroups": [
  {
    "name": "",
    "description": "",
    "resources": [
      {
        "name": "",
        "description": "",
        "uriTemplate": "/{id}",
        "model": {},
        "parameters": [],
        "actions": [
          {
            "name": "",
            "description": "",
            "method": "GET",
            "parameters": [],
            "examples": []
          }
        ]
      }
    ]
  }
]
}

Expected (pre-refactor)

{
  "_version": "2.0",
  "metadata": [],
  "name": "",
  "description": "",
  "resourceGroups": [
      {
          "name": "",
          "description": "",
          "resources": [
              {
                  "name": "",
                  "description": "",
                  "uriTemplate": "/{id}",
                  "model": {},
                  "parameters": [],
                  "actions": [
                      {
                          "name": "",
                          "description": "",
                          "method": "GET",
                          "parameters": [
                              {
                                  "name": "id",
                                  "description": "\nThis is a description\n\n",
                                  "type": "string",
                                  "required": true,
                                  "default": "",
                                  "example": "",
                                  "values": []
                              }
                          ],
                          "examples": []
                      }
                  ]
              }
          ]
      }
  ]
}

Resolution

This is a regression which needs to be fixed.

Unexpected nodes with keywords do not stop the parsing

  • Resolved

Also, having model in list item sentence makes it a ModelSectionType @alikh31 is working on it.

Given

FORMAT: 1A

# S

Hello

+ Response

# GET /

Actual (after-refactor)

OK.

_version: 2.0
metadata:
- name: "FORMAT"
  value: "1A"
name: "S"
description: "Hello\n\n"
resourceGroups:

Expected (pre-refactor)

_version: 2.0
metadata:
- name: "FORMAT"
  value: "1A"
name: "S"
description: "Hello\n\n+ Response\n\n"
resourceGroups:
- name:
  description:
  resources:
  - name:
    description:
    uriTemplate: "/"
    model:
    parameters:
    actions:
    - name:
      description:
      method: "GET"
      parameters:
      examples:

OK.
warning: (6)  no response defined for 'GET /' :36:7

Resolution

Neither actual or expected behavior is correct.

To get to the expect behavior clearly the steps are described here: https://github.com/apiaryio/snowcrash/pull/155#issuecomment-50970869

However the parser should warn when a keyword defined section is swallowed in description (this would be a new behavior). For this a change to isDescriptionNode is needed – see here

Metadata spanning multiple paragraphs

  • Resolved

Metadata can be distributed among any number of paragraph nodes.

Given

meta1: 1

meta2: 2

# API 
Lorem Ipsum

Actual (after-refactor)

OK.
warning: (1)  expected API name, e.g. '# <API Name>' :0:10

{
"_version": "2.0",
"metadata": [
  {
    "name": "meta1",
    "value": "1"
  }
],
"name": "",
"description": "meta2: 2\n\n# API \n\nLorem Ipsum\n",
"resourceGroups": []
}

Expected (pre-refactor)

{
  "_version": "2.0",
  "metadata": [
      {
          "name": "meta1",
          "value": "1"
      },
      {
          "name": "meta2",
          "value": "2"
      }
  ],
  "name": "",
  "description": "# API \nLorem Ipsum\n",
  "resourceGroups": []
}

Resolution

This is a regression which needs to be fixed.

API name with multiple metadata

  • Resolved

Not recognizing api name when metadata spans multiple paragraphs

Given

meta1: 1

meta2: 2

# API 
Lorem Ipsum

Actual:

OK.
{
  "_version": "2.0",
  "metadata": [
    {
      "name": "meta1",
      "value": "1"
    },
    {
      "name": "meta2",
      "value": "2"
    }
  ],
  "name": "API",
  "description": "Lorem Ipsum\n",
  "resourceGroups": []
}

Expected (pre-refactor)

{
  "_version": "2.0",
  "metadata": [
      {
          "name": "meta1",
          "value": "1"
      },
      {
          "name": "meta2",
          "value": "2"
      }
  ],
  "name": "",
  "description": "# API \nLorem Ipsum\n",
  "resourceGroups": []
}

Resolution

New behaviour (actual) is OK and not considered a regression issue.

Responses with excessive indentation

  • Resolved

Wrong indentation for transaction examples in actions.

Given

# API
## GET /a

+ Request

  + Headers

          Accept: application/json

  + Response 202 (application/json)

          A

  + Response 200 (application/json)

      + Body

              B

Actual (after-refactor)

OK.
warning: (6)  action is missing a response for a request :6:11

{
"_version": "2.0",
"metadata": [],
"name": "API",
"description": "",
"resourceGroups": [
  {
    "name": "",
    "description": "",
    "resources": [
      {
        "name": "",
        "description": "",
        "uriTemplate": "/a",
        "model": {},
        "parameters": [],
        "actions": [
          {
            "name": "",
            "description": "",
            "method": "GET",
            "parameters": [],
            "examples": [
              {
                "name": "",
                "description": "",
                "requests": [
                  {
                    "name": "",
                    "description": "",
                    "headers": [
                      {
                        "name": "Accept",
                        "value": "application/json"
                      }
                    ],
                    "body": "",
                    "schema": ""
                  }
                ],
                "responses": []
              }
            ]
          }
        ]
      }
    ]
  }
]
}

Expected (pre-refactor)

{
  "_version": "2.0",
  "metadata": [],
  "name": "API",
  "description": "",
  "resourceGroups": [
      {
          "name": "",
          "description": "",
          "resources": [
              {
                  "name": "",
                  "description": "",
                  "uriTemplate": "/a",
                  "model": {},
                  "parameters": [],
                  "actions": [
                      {
                          "name": "",
                          "description": "",
                          "method": "GET",
                          "parameters": [],
                          "examples": [
                              {
                                  "name": "",
                                  "description": "",
                                  "requests": [
                                      {
                                          "name": "",
                                          "description": "",
                                          "headers": [
                                              {
                                                  "name": "Accept",
                                                  "value": "application/json"
                                              }
                                          ],
                                          "body": "",
                                          "schema": ""
                                      }
                                  ],
                                  "responses": [
                                      {
                                          "name": "202",
                                          "description": "",
                                          "headers": [
                                              {
                                                  "name": "Content-Type",
                                                  "value": "application/json"
                                              }
                                          ],
                                          "body": "A\n",
                                          "schema": ""
                                      },
                                      {
                                          "name": "200",
                                          "description": "",
                                          "headers": [
                                              {
                                                  "name": "Content-Type",
                                                  "value": "application/json"
                                              }
                                          ],
                                          "body": "B\n",
                                          "schema": ""
                                      }
                                  ]
                              }
                          ]
                      }
                  ]
              }
          ]
      }
  ]
}

Resolution

  1. Leave the actual (new) behavior.
  2. Create a new issue for not warning about ingoring excessively nested responses.

Model reference indentation

  • Resolved

When a symbol is referenced in a Payload, parser warns them if it's not indented by 8 spaces.

Given

# Posts [/posts]
+ Model (application/json)

      {...}

## List [GET]

+ Response 200

  [Posts][]

Actual (after-refactor)

OK.
warning: (10)  response is expected to be a pre-formatted code block, every of its line indented by exactly 4 spaces or 1 tabs :0:0
warning: (1)  expected API name, e.g. '# <API Name>' :0:17

{
"_version": "2.0",
"metadata": [],
"name": "",
"description": "",
"resourceGroups": [
  {
    "name": "",
    "description": "",
    "resources": [
      {
        "name": "Posts",
        "description": "",
        "uriTemplate": "/posts",
        "model": {
          "name": "Posts",
          "description": "",
          "headers": [
            {
              "name": "Content-Type",
              "value": "application/json"
            }
          ],
          "body": "{...}\n",
          "schema": ""
        },
        "parameters": [],
        "actions": [
          {
            "name": "List",
            "description": "",
            "method": "GET",
            "parameters": [],
            "examples": [
              {
                "name": "",
                "description": "",
                "requests": [],
                "responses": [
                  {
                    "name": "200",
                    "description": "",
                    "headers": [
                      {
                        "name": "Content-Type",
                        "value": "application/json"
                      }
                    ],
                    "body": "{...}\n",
                    "schema": ""
                  }
                ]
              }
            ]
          }
        ]
      }
    ]
  }
]
}

Expected (pre-refactor)

{
  "_version": "2.0",
  "metadata": [],
  "name": "",
  "description": "",
  "resourceGroups": [
      {
          "name": "",
          "description": "",
          "resources": [
              {
                  "name": "Posts",
                  "description": "",
                  "uriTemplate": "/posts",
                  "model": {
                      "name": "Posts",
                      "description": "",
                      "headers": [
                          {
                              "name": "Content-Type",
                              "value": "application/json"
                          }
                      ],
                      "body": "{...}\n",
                      "schema": ""
                  },
                  "parameters": [],
                  "actions": [
                      {
                          "name": "List",
                          "description": "",
                          "method": "GET",
                          "parameters": [],
                          "examples": [
                              {
                                  "name": "",
                                  "description": "",
                                  "requests": [],
                                  "responses": [
                                      {
                                          "name": "200",
                                          "description": "",
                                          "headers": [
                                              {
                                                  "name": "Content-Type",
                                                  "value": "application/json"
                                              }
                                          ],
                                          "body": "{...}\n",
                                          "schema": ""
                                      }
                                  ]
                              }
                          ]
                      }
                  ]
              }
          ]
      }
  ]
}

Resolution:

  1. when indented by 8 spaces (code block) – reference is parsed as an asset
  2. when indented by 4 spaces (pragraph) – explode the reference

New line parameter description

  • Resolved

Add parameter description when no description on its signature and remaining description is not a new node.

Given

# API

## GET /vehicles/{id}/command/sun_roof_control?state={state}

+ Parameters
  + id (number) ... The ID number of the car
  + state (string)
      This description is not shown in AST
      + Values
          + `open`
          + `close`
          + `comfort`
          + `vent`

+ Response 200

Actual (after-refactor)

OK.
warning: (5)  ignoring unrecognized block :129:17;150:39;193:11;208:15;227:16;247:18;269:15

_version: 2.0
metadata:
name: "API"
description:
resourceGroups:
- name:
  description:
  resources:
  - name:
    description:
    uriTemplate: "/vehicles/{id}/command/sun_roof_control?state={state}"
    model:
    parameters:
    actions:
    - name:
      description:
      method: "GET"
      parameters:
      - name: "id"
        description: "The ID number of the car"
        type: "number"
        required: true
        default:
        example:
        values:
      examples:
      - name:
        description:
        requests:
        responses:
        - name: "200"
          description:
          headers:
          body:
          schema:

Expected (pre-refactor)

_version: 2.0
metadata:
name: "API"
description:
resourceGroups:
- name:
  description:
  resources:
  - name:
    description:
    uriTemplate: "/vehicles/{id}/command/sun_roof_control?state={state}"
    model:
    parameters:
    actions:
    - name:
      description:
      method: "GET"
      parameters:
      - name: "id"
        description: "The ID number of the car"
        type: "number"
        required: true
        default:
        example:
        values:
      - name: "state"
        description:
        type: "string"
        required: true
        default:
        example:
        values:
        - value: "open"
        - value: "close"
        - value: "comfort"
        - value: "vent"
      examples:
      - name:
        description:
        requests:
        responses:
        - name: "200"
          description:
          headers:
          body:
          schema:

OK.

Resolution

The AST should match expected however the state should have its description set.

URI Param example value with ...

  • Resolved

Allows parameter default value to have ....

Given

FORMAT: 1A
HOST: http://www.google.com

# sandbox

## GET /vehicles/{id}/command/sun_roof_control?state={state}

Controls the car's panoramic roof, if installed.

+ Parameters

  + state (required, string) ... State of the roof
  + id (required, string, `wejude77...`) ... The ID number of the car

+ Response 200

Actual (after-refactor)

OK.

_version: 2.0
metadata:
name: "API"
description:
resourceGroups:
- name:
  description:
  resources:
  - name:
    description:
    uriTemplate: "/vehicles/{id}/command/sun_roof_control?state={state}"
    model:
    parameters:
    actions:
    - name:
      description:
      method: "GET"
      parameters:
      - name: "id"
        description: "The ID number of the car"
        type: "number"
        required: true
        default:
        example: "wejude77..."
        values:
      examples:
      - name:
        description:
        requests:
        responses:
        - name: "200"
          description:
          headers:
          body:
          schema:

Expected (pre-refactor)

OK.
warning: (5)  ignoring unrecognized list, expected parameter discussion, e.g. '<parameter name> ... lorem ipsum' :86:52
warning: (3)  no parameters specified, expected a nested list of parameters, one parameter per list item :71:11

_version: 2.0
metadata:
name: "API"
description:
resourceGroups:
- name:
  description:
  resources:
  - name:
    description:
    uriTemplate: "/vehicles/{id}/command/sun_roof_control?state={state}"
    model:
    parameters:
    actions:
    - name:
      description:
      method: "GET"
      parameters:
      examples:
      - name:
        description:
        requests:
        responses:
        - name: "200"
          description:
          headers:
          body:
          schema:

Resolution

The actual behavior is correct. The issue was brought by fixing #125 or #107

Exception during parsing a blueprint with a missing symbol

  • Resolved

Given

# Posts [/posts]
+ Model (application/json)

      {...}

## List [GET]

+ Response 200

  [Post][]

Actual (after-refactor)

error: (1)  parser exception: 'map::at'
warning: (10)  response is expected to be a pre-formatted code block, every of its line indented by exactly 4 spaces or 1 tabs :0:0

_version: 2.0
metadata:
name: "API"
description:
resourceGroups:

Expected (pre-refactor)

error: (3)  undefined symbol 'Post' :102:10

_version: 2.0
metadata:
name: "API"
description:
resourceGroups:

Resolution

The expected behavior is correct which makes this a regression.

API Name is required by default

  • Resolved

Given

# GET /1
+ response 200

Actual (after-refactor)

OK.
warning: (1)  expected API name, e.g. '# <API Name>' :0:9

_version: 2.0
metadata:
name:
description:
resourceGroups:
- name:
  description:
  resources:
  - name:
    description:
    uriTemplate: "/1"
    model:
    parameters:
    actions:
    - name:
      description:
      method: "GET"
      parameters:
      examples:
      - name:
        description:
        requests:
        responses:
        - name: "200"
          description:
          headers:
          body:
          schema:

Expected (pre-refactor)

_version: 2.0
metadata:
name:
description:
resourceGroups:
- name:
  description:
  resources:
  - name:
    description:
    uriTemplate: "/1"
    model:
    parameters:
    actions:
    - name:
      description:
      method: "GET"
      parameters:
      examples:
      - name:
        description:
        requests:
        responses:
        - name: "200"
          description:
          headers:
          body:
          schema:

OK.

Resolution

The API name is optional by default

No Headers Specficied

  • Resolved

Missing warning when there are no headers specified in the headers section.

Given

# API
## GET /1
+ response 200
    + headers
    + body

            A1

Actual (after-refactor)

OK.

_version: 2.0
metadata:
name: "API"
description:
resourceGroups:
- name:
  description:
  resources:
  - name:
    description:
    uriTemplate: "/1"
    model:
    parameters:
    actions:
    - name:
      description:
      method: "GET"
      parameters:
      examples:
      - name:
        description:
        requests:
        responses:
        - name: "200"
          description:
          headers:
          body: "A1\n"
          schema:

Expected (pre-refactor)

OK.
warning: (3)  no headers specified :37:8

_version: 2.0
metadata:
name: "API"
description:
resourceGroups:
- name:
  description:
  resources:
  - name:
    description:
    uriTemplate: "/1"
    model:
    parameters:
    actions:
    - name:
      description:
      method: "GET"
      parameters:
      examples:
      - name:
        description:
        requests:
        responses:
        - name: "200"
          description:
          headers:
          body: "A1\n"
          schema:

Resolution

Add missing warning

Non-recognized parameter without newline after Parameters

  • Resolved

Doesn't recognize the main Parameters section.

Given

# GET /1

+ Parameters
    -_id (string, `1`) ... Desks

+ Response 200

Actual (after-refactor)

_version: 2.0
metadata:
name:
description:
resourceGroups:
- name:
  description:
  resources:
  - name:
    description:
    uriTemplate: "/1"
    model:
    parameters:
    actions:
    - name:
      description: "+ Parameters\n    -_id (string, `1`) ... Desks\n\n"
      method: "GET"
      parameters:
      examples:
      - name:
        description:
        requests:
        responses:
        - name: "200"
          description:
          headers:
          body:
          schema:

Expected (pre-refactor)

OK.
warning: (5)  ignoring additional content after 'parameters' keyword, expected a nested list of parameters, one parameter per list item :12:11;27:29
warning: (3)  no parameters specified, expected a nested list of parameters, one parameter per list item :12:11;27:29

_version: 2.0
metadata:
name:
description:
resourceGroups:
- name:
  description:
  resources:
  - name:
    description:
    uriTemplate: "/1"
    model:
    parameters:
    actions:
    - name:
      description:
      method: "GET"
      parameters:
      examples:
      - name:
        description:
        requests:
        responses:
        - name: "200"
          description:
          headers:
          body:
          schema:

Resolution

Expected is correct, so this is a regession.

Body with content on signature

  • Resolved

Doesn't get recognized as BodySection.

Given

# GET /
+ Response 200
    + Body
            {}

Actual (after-refactor)

_version: 2.0
metadata:
name:
description:
resourceGroups:
- name:
  description:
  resources:
  - name:
    description:
    uriTemplate: "/"
    model:
    parameters:
    actions:
    - name:
      description:
      method: "GET"
      parameters:
      examples:
      - name:
        description:
        requests:
        responses:
        - name: "200"
          description:
          headers:
          body: "+ Body\n        {}\n"
          schema:

Expected (pre-refactor)

_version: 2.0
metadata:
name:
description:
resourceGroups:
- name:
  description:
  resources:
  - name:
    description:
    uriTemplate: "/"
    model:
    parameters:
    actions:
    - name:
      description:
      method: "GET"
      parameters:
      examples:
      - name:
        description:
        requests:
        responses:
        - name: "200"
          description:
          headers:
          body: "    {}\n"
          schema:

OK.
warning: (10)  message-body asset is expected to be a pre-formatted code block, separate it by a newline and indent every of its line by 12 spaces or 3 tabs :29:5;42:7

Resolution

This is a regression.

Missing Source Map

  • Resolved

A source map is missing for poorly nested asset

Given

# API
## GET /1
+ response 200
    + Body

        X

Actual (after-refactor)

OK.
warning: (10)  message-body is expected to be a pre-formatted code block, every of its line indented by exactly 12 spaces or 3 tabs :0:0

_version: 2.0
metadata:
name: "API"
description:
resourceGroups:
- name:
  description:
  resources:
  - name:
    description:
    uriTemplate: "/1"
    model:
    parameters:
    actions:
    - name:
      description:
      method: "GET"
      parameters:
      examples:
      - name:
        description:
        requests:
        responses:
        - name: "200"
          description:
          headers:
          body: "X"
          schema:

Expected (pre-refactor)

OK.
warning: (10)  message-body asset is expected to be a pre-formatted code block, every of its line indented by exactly 12 spaces or 3 tabs :51:1

_version: 2.0
metadata:
name: "API"
description:
resourceGroups:
- name:
  description:
  resources:
  - name:
    description:
    uriTemplate: "/1"
    model:
    parameters:
    actions:
    - name:
      description:
      method: "GET"
      parameters:
      examples:
      - name:
        description:
        requests:
        responses:
        - name: "200"
          description:
          headers:
          body: "X"
          schema:

Resolution

The source map should be :51:1

Clone this wiki locally