[Feature Request]: Allow Playwright Selectors

[cjoecker]

What needs to be done?

Allow the usage of playwright locators for navigating on the browser.

What problem does this solve?

Right now, the suggested way of navigating on the browser is using queries, js, or CSS classes. Doing that, the scrapping logic is done using implementation details. It then make the scrapping logic unreliable since change in the code made by the sites owners will break the scrapping.

Using locators mimmics the user's behavior what makes the scrapping more reliable. E.g. I say with locators "click on the button with a next page label" instead of saying "click on the third button with the .next-page CSS class".

Target users/beneficiaries

Everyone doing scrapping

Current alternatives/workarounds

Use JS playwright directly loosing all the smart logic behind crawl4ai

Proposed approach

See above