Skip to content

Commit

Permalink
Add support for Adblock filters (#5968)
Browse files Browse the repository at this point in the history
Co-authored-by: John Gardner <gardnerjohng@gmail.com>
Co-authored-by: Colin Seymour <colin@github.com>
  • Loading branch information
3 people authored Sep 1, 2022
1 parent 5fb7b70 commit e78ef71
Show file tree
Hide file tree
Showing 13 changed files with 633 additions and 0 deletions.
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,9 @@
[submodule "vendor/grammars/Vala-TMBundle"]
path = vendor/grammars/Vala-TMBundle
url = https://github.com/technosophos/Vala-TMBundle
[submodule "vendor/grammars/VscodeAdblockSyntax"]
path = vendor/grammars/VscodeAdblockSyntax
url = https://github.com/ameshkov/VscodeAdblockSyntax.git
[submodule "vendor/grammars/WhileySyntaxBundle"]
path = vendor/grammars/WhileySyntaxBundle
url = https://github.com/Whiley/WhileySyntaxBundle.git
Expand Down
2 changes: 2 additions & 0 deletions grammars.yml
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,8 @@ vendor/grammars/VBDotNetSyntax:
- source.vbnet
vendor/grammars/Vala-TMBundle:
- source.vala
vendor/grammars/VscodeAdblockSyntax:
- text.adblock
vendor/grammars/WhileySyntaxBundle:
- source.whiley
vendor/grammars/X10:
Expand Down
21 changes: 21 additions & 0 deletions lib/linguist/heuristics.yml
Original file line number Diff line number Diff line change
Expand Up @@ -650,6 +650,27 @@ disambiguations:
# VIM_MODELINE pattern in `./lib/linguist/strategy/modeline.rb`.
- language: Vim Help File
pattern: '(?:(?:^|[ \t])(?:vi|Vi(?=m))(?:m[<=>]?[0-9]+|m)?|[ \t]ex)(?=:(?=[ \t]*set?[ \t][^\r\n:]+:)|:(?![ \t]*set?[ \t]))(?:(?:[ \t]*:[ \t]*|[ \t])\w*(?:[ \t]*=(?:[^\\\s]|\\.)*)?)*[ \t:](?:filetype|ft|syntax)[ \t]*=(help)(?=$|\s|:)'
- language: Adblock Filter List
pattern: |-
(?x)\A

This comment has been minimized.

Copy link
@tristansavina

tristansavina Sep 7, 2022

Hello,
The convert-pcre helper interpret this as a regex flag and throw a SyntaxError exception
SyntaxError: Invalid flags supplied to RegExp constructor 'x'

What was the expected behaviour ?

This comment has been minimized.

Copy link
@lildude

This comment has been minimized.

Copy link
@tristansavina

tristansavina Sep 7, 2022

Oups, sorry, it was a bug on the linguistjs (https://github.com/Nixinova/LinguistJS) module I used
It seems to use a custom logic to parse your heuristics file (Nixinova/LinguistJS@c89a149#diff-8db135deb3995a7fed0c583be339d66cf20f2ae1c54d5027bc1b7f0a76f7346a)
Thanks for you response

\[
(?<version>
(?:
[Aa]d[Bb]lock
(?:[ \t][Pp]lus)?
|
u[Bb]lock
(?:[ \t][Oo]rigin)?
|
[Aa]d[Gg]uard
)
(?:[ \t] \d+(?:\.\d+)*+)?
)
(?:
[ \t]?;[ \t]?
\g<version>
)*+
\]
# HACK: This is a contrived use of heuristics needed to address
# an unusual edge-case. See https://git.io/JULye for discussion.
- language: Text
Expand Down
13 changes: 13 additions & 0 deletions lib/linguist/languages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,19 @@ Ada:
tm_scope: source.ada
ace_mode: ada
language_id: 11
Adblock Filter List:
type: data
color: "#800000"
ace_mode: text
extensions:
- ".txt"
aliases:
- ad block filters
- ad block
- adb
- adblock
tm_scope: text.adblock
language_id: 884614762
Adobe Font Metrics:
type: data
color: "#fa0f00"
Expand Down
20 changes: 20 additions & 0 deletions samples/Adblock Filter List/Imperial Units Remover.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[Adblock Plus 3.13]
! Title: 📏 Imperial Units Remover
! Version: 23May2022v1-Alpha
! Expires: 10 days
! Description: Are you pro-metric and are tired of seeing imperial unites all over the English-language parts of the internet? If so, then here's a list for you.
! Homepage: https://github.com/DandelionSprout/adfilt/blob/master/Wiki/General-info.md#english

! Removing miles from dual-unit distance measurements in Google Maps (uBO and AdGuard only)
google.*##html[lang$=US] #ruler span:first-of-type
google.*##html[lang=en] #ruler span:first-of-type
google.*##html:not([lang$=US], [lang=en]) #ruler span:nth-of-type(2)

! Removes imperial measurements from multi-unit displays
weather.gov###myfcst-tempf
weather.gov#?#p[class^=myforecast-current-]:has-text(°F)
holiday-weather.com#?#.temperature_data_container > :has-text(°F)
holiday-weather.com#?#.distance_data_container > :has-text(miles)
lematin.ma##.meteo-bloc span
sveip.no##.wmsb_f
gismeteo.de##.ifnoie
82 changes: 82 additions & 0 deletions samples/Adblock Filter List/abp-filters-anti-cv.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
[Adblock Plus 3.1]
! Checksum: i7lcTV9/kkiE8g3aav4y/g
! Version: 202208071141
! Title: ABP filters
! Last modified: 07 Aug 2022 11:41 UTC
! Expires: 1 hours (update frequency)
! Homepage: https://github.com/abp-filters/abp-filters-anti-cv
!
! Filter list designed to fight circumvention ads (including, in some cases, their tracking) and fix critical issues for Adblock Plus users
!
! Please report any issues
! on GitHub https://github.com/abp-filters/abp-filters-anti-cv/issues
! or via filters+cv@adblockplus.org
!
! *** abp-filters-anti-cv:arabic.txt ***
! MISC
3sk.io,3sk.co,33sk.tv,esheeq.co#$#abort-on-property-read atob; abort-on-property-write Fingerprint2; abort-on-property-read decodeURIComponent; abort-on-property-read RegExp
3oloum.com,7olm.org,ahladalil.com,syriaforums.net,ahlamontada.com,alamontada.com,arabepro.com,banouta.net,gid3an.com,jordanforum.net,yoo7.com,forumalgerie.net,own0.com##a[onclick^="this.href=atob"]
3oloum.com,7olm.org,ahladalil.com,syriaforums.net,ahlamontada.com,alamontada.com,arabepro.com,banouta.net,gid3an.com,jordanforum.net,yoo7.com,forumalgerie.net,own0.com#$#abort-on-property-read atob
! #CV-685
arabseed.co,arabseed.com,arabseed.me,arabseed.net,arabseed.tv#$#abort-on-property-read atob; abort-on-property-read decodeURI; abort-on-property-read decodeURIComponent
hawak.net,hawak.tv,rotana.video#$#abort-on-property-read atob
! #CV-667
beinconnect.us,coroot.blogspot.com,ktarab.com,shofnow.com,actionz.tv,brstej.com,jaewinter.co,movs4u.live,mvs4u.tv,kora-online.tv,filmey.tv,animetak.net,arabseed.tv,arabseed.co,arabseed.com,arabseed.me,arabseed.net,shahid4u.com,shahid4u.net#$#abort-current-inline-script Math zfgloaded
! Popup/Popunder/Clickunder
egyanime.com,egydead.com,egydead.live#$#abort-on-property-write _pop
actionz.tv,cimaclub.com,cimaclub.in,elmstba.com,hawak.net,hawak.tv,lodynet.co,lodynet.dev,lodynet.ink,lodynet.tv,rotana.video#$#abort-on-property-read decodeURIComponent
egy.best,egybest.asia,egybest.bid,egybest.biz,egybest.cheap,egybest.co,egybest.com,egybest.cool,egybest.fail,egybest.life,egybest.ltd,egybest.ist,egybest.name,egybest.net,egybest.network,egybest.ninja,egybest.nl,egybest.online,egybest.org,egybest.pw,egybest.rocks,egybest.site,egybest.xyz,egybest.zone,egybest2.com,iegy.best#$#abort-on-property-read open
akhbara24.news,anime4up.art,anime4up.com,arabseed.me,asia2tv.cn,asia2tv.com,baramjak.com,cima-club.cc,cima-club.in,cima-club.io,cima-club.lol,cima-club.vip,cima4u.cloud,cima4u.film,cima4u.ink,cima4u.tv,cima4u.ws,cimalina.me,cimalina.net,egy-live.online,egynow.cam,egynow.live,hawak.net,iegybest.co,lodynet.co,lodynet.dev,lodynet.ink,lodynet.tv,manga.ae,moshahda.net,movizland.com,movizland.cyou,movizland.online,egyanime.com,egydead.com,egydead.live,shahed4u.cafe,shahed4u.casa,shahed4u.cloud,shahed4u.mba,shahed4u.red,shahed4u.tips,shahed4u.ws,tuktukcinema.co,tuktukcinema.net,witanime.com,yalla-shoot.today#$#abort-current-inline-script Math /atob|break/; abort-current-inline-script String.fromCharCode /atob|break/; abort-current-inline-script document.documentElement /atob|break|JSON.parse/; abort-on-property-write JSON.parse
arabxd.com#$#abort-current-inline-script document.querySelectorAll popMagic
! #CV-754
akwam.cc,akoam.com,akoam.net,akwam.co,akwam.cx,akwam.net,akwam.org,gateanime.com,gocoo.co#$#abort-on-property-read String.fromCharCode
! *** abp-filters-anti-cv:bulgarian.txt ***
! #CV-667
gledaiseriali.net#$#abort-current-inline-script Math zfgloaded
! *** abp-filters-anti-cv:chinese.txt ***
! #CV-767
ipv6.baidu.com,xueshu.baidu.com,www.baidu.com,www1.baidu.com#$#hide-if-contains 广告 .c-container '.f13 > .m'; hide-if-contains 广告 #content_right>[class] span[data-tuiguang]
1111.baidu.com,bdimg.com,tieba.baidu.com,tieba.com,xingqu.baidu.com#$#abort-on-property-read Object.prototype.loadImage
baidu.com#$#hide-if-contains 广告 #content_right>div .ec-tuiguang
baidu.com#$#hide-if-contains 广告 #content_left>div .ec-tuiguang
! MISC
torrentkitty.tv#$#abort-on-property-read _fupfgk;abort-on-property-read _nyjdy
178.com,nga.cn#$#abort-on-property-write __LOAD_BBS_ADS_12
cn.bing.com#$#hide-if-contains 360 #b_results>li .b_adProvider
2345.com#$#abort-current-inline-script setDefaultTheme
ahri8.top#$#abort-current-inline-script document.querySelectorAll popMagic
ifenpaidy.com#$#abort-on-property-read localStorage
! Popups & Popunders
cocomanga.com#$#abort-on-property-read __cad.cpm_popunder; abort-on-property-read __ad
madouqu.com,madouqu.tk#$#abort-current-inline-script document.querySelectorAll popMagic
editcode.net#$#abort-current-inline-script setTimeout openAd
theporn.cc#$#abort-on-property-read is_show_alert_window
! Video ads
iyingshi9.tv#$#override-property-read YZM.ads.state false
! #CV-667
hdzone.org#$#abort-current-inline-script Math zfgloaded
! *** abp-filters-anti-cv:czech.txt ***
! #CV-655
iprima.cz,www.seznam.cz,novinky.cz,super.cz,ahaonline.cz,expres.cz,kinobox.cz,horoskopy.cz#$#json-override ads ''
kinobox.cz,novinky.cz,expres.cz,fights.cz,tiscali.cz,hudebniskupiny.cz,osobnosti.cz,ahaonline.cz,super.cz#$#prevent-listener beforeunload (); cookie-remover /^_?adb/;
sauto.cz#$#override-property-read sssp undefined
super.cz#$#override-property-read sspPositions null
iprima.cz##.sas_center, .px-ad, .sas_mone, .mone_box, .ifr-passback, [data-sas_status], div[id^="sas_"], div[class^="mas_mone_"], [data-alter_area="leaderboardpremium-1"], [data-alter_area="boardbottom-1"], .ifr-masshack, .px-zone, .sas_megaboard, .ifr-classflak, [data-sas-creative], #cnn_reklama, div[class^="mas_"], ._sasia_fcid, [class^="mas"][class$="_clue"]
iprima.cz#?#div:-abp-has(>.mone_header)
iprima.cz##.px-type-banner
iprima.cz##[class$="_value"], [id^="px-id-"]
iprima.cz##[class$="_fcid"], [data-mashiatus]
iprima.cz##div[data-d-area]
iprima.cz##[class*="sas"]
iprima.cz##a[href*="/area=branding"]
iprima.cz##[id*="-ad-"], .ifr-claassflak, a[href*="/posid=sas_"]
iprima.cz##.dekes_reblika
/area=*/posid=*$subdocument,xmlhttprequest,script,domain=iprima.cz
/keyword*/area=$xmlhttprequest,domain=iprima.cz
/section=adblock_desktop^$domain=iprima.cz
?format=js|$script,domain=iprima.cz
?guci=$subdocument,domain=iprima.cz
?v127|$script,domain=iprima.cz
/area=halfpagead$xmlhttprequest,domain=iprima.cz
/area=branding$xmlhttprequest,domain=iprima.cz
auto.cz,autorevue.cz,maminka.cz,reflex.cz,ahaonline.cz,blesk.cz,dama.cz,e15.cz,info.cz,mojezdravi.cz,onetv.cz,zeny.cz,zive.cz##.inserted_rtb
99 changes: 99 additions & 0 deletions samples/Adblock Filter List/anti-facebook.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
[Adblock Plus 2.0]
! Checksum: Ea7JI8rmiGKaIOR604if2A
! Title: Anti-Facebook List
! Updated: 2021-09-19 08:57 UTC
! Expires: 9 days
! License: http://creativecommons.org/licenses/by/3.0/
! Homepage: http://www.fanboy.co.nz/
! Reporting Issues: https://github.com/ryanbr/fanboy-adblock/issues
!
! Legal stuff (T&C's) .
! In no event shall Fanboy List, or the list author be liable for any indirect, direct, punitive, special, incidental, or consequential damages whatsoever.
! By downloading or viewing, or using this list, you are accepting these terms and the license.
!
!
! Warning, will break on facebook-based comment sites and may also break on some facebook apps or games.
!
! Thirdparty Filters
!
||api-read.facebook.com/restserver.php?api_key=$third-party
||api.facebook.com^$third-party
||badge.facebook.com^$third-party
||connect.facebook.com^$third-party,domain=~facebook.net|~fb.com
||connect.facebook.net^$third-party,domain=~facebook.com|~fb.com
||facebook.com/connect/$third-party
||facebook.com/dialog/oauth?display=popup$popup,domain=humorhub.org
||facebook.com/plugins/activity.php?$third-party
||facebook.com/plugins/comments.php?$third-party
||facebook.com/plugins/facepile.php?$third-party
||facebook.com/plugins/fan.php?$third-party
||facebook.com/plugins/follow.php$third-party
||facebook.com/plugins/like.php?$third-party
||facebook.com/plugins/like_box.php$third-party
||facebook.com/plugins/likebox.php?$third-party
||facebook.com/plugins/post.php?$third-party
||facebook.com/plugins/recommendations.php?$third-party
||facebook.com/plugins/recommendations_bar.php?$third-party
||facebook.com/plugins/send.php?$third-party
||facebook.com/plugins/share_button.php?$third-party
||facebook.com/plugins/subscribe.php$third-party
||facebook.com/plugins/subscribe?$third-party
||facebook.com/restserver.php?*.getStats&$third-party
||facebook.com/whitepages/wpminiprofile.php$third-party
||facebook.com/widgets/activity.php?$third-party
||facebook.com/widgets/fan.php?$third-party
||facebook.com/widgets/like.php?$third-party
||facebook.com/widgets/recommendations.php?$third-party
||fbcdn-profile-a.akamaihd.net^$third-party,domain=~facebook.com
||google.com/js/client:plusone.js$third-party
||google.com/js/plusone.js$third-party
||graph.facebook.com/?id=$third-party
||graph.facebook.com^$third-party,xmlhttprequest
||profile.ak.fbcdn.net^$third-party,domain=~facebook.com
||scontent-a.*.fbcdn.net^$third-party,domain=~facebook.com
||spot.im/embed/scripts/launcher.js$third-party
||static.ak.fbcdn.net^$third-party,domain=~facebook.com
!
! Test Filters (For Gannett sites, which don't work anyways with Facebook blocked)
!
##.util-bar-module-firefly-visible
!
! Whitelist Filters
!
@@/cdn-cgi/pe/bag2?*connect.facebook.net$domain=forwardprogressives.com|onhax.net|opensubtitles.org|viralthread.com|youngcons.com
@@||akamaihd.net/rsrc.php/$domain=facebook.com
@@||channel.facebook.com^$domain=facebook.com
@@||connect.facebook.com^*/AudienceNetworkPrebid.js$domain=vice.com
@@||facebook.com/ajax/browse/$domain=facebook.com
@@||facebook.com/ajax/bz$domain=facebook.com
@@||facebook.com/ajax/chat/buddy_list.php$domain=facebook.com
@@||facebook.com/ajax/chat/hovercard/$domain=facebook.com
@@||facebook.com/ajax/hovercard/$domain=facebook.com
@@||facebook.com/ajax/litestand/$domain=facebook.com
@@||facebook.com/ajax/notifications/$domain=facebook.com
@@||facebook.com/ajax/pagelet/$domain=facebook.com
@@||facebook.com/ajax/photos/$domain=facebook.com
@@||facebook.com/ajax/presence/$domain=facebook.com
@@||facebook.com/ajax/typeahead/$domain=facebook.com
@@||facebook.com/ajax/webstorage/$domain=facebook.com
@@||facebook.com/chat/$domain=facebook.com
@@||facebook.com/images/$domain=facebook.com
@@||fbcdn-photos-*.akamaihd.net^$domain=facebook.com
@@||fbcdn-profile-*.akamaihd.net^$domain=facebook.com
@@||fbcdn-sphotos-*.akamaihd.net^$domain=facebook.com
@@||fbexternal-*.akamaihd.net^$domain=facebook.com
@@||scontent-*.fbcdn.net^$domain=facebook.com
!
! Problematic sites
!
@@||cloudfront.net^$font,domain=magicseaweed.com
@@||connect.facebook.com^*/audiencenetworkprebid.js$script,domain=screenrant.com
@@||connect.facebook.net^$script,domain=id.logi.com|tinder.com|abc.go.com|adultswim.com|contv.com|damnyouautocorrect.com|embedly.com|fitbit.com|instagram.com|interviewmagazine.com|noovo.ca|pogo.com|reddit.com|salon.com|sci2.tv|southpark.cc.com|southpark.de|upi.com
@@||google.com/js/plusone.js$domain=abcnews.go.com|watch.nba.com
@@||graph.facebook.com^$xmlhttprequest,domain=theguardian.com
@@||staticxx.facebook.com/connect/$domain=embedly.com
!
! Cloudflare CDN
!
/cdn-cgi/pe/bag2?*connect.facebook.com
/cdn-cgi/pe/bag2?*connect.facebook.net
45 changes: 45 additions & 0 deletions samples/Adblock Filter List/fake-news.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
[Adblock Plus 2.0]
! Title: Fake-News List
! Updated: 10 Mar 2018
! License: http://creativecommons.org/licenses/by/3.0/
! Homepage: http://www.fanboy.co.nz/
! Reporting Issues: https://github.com/ryanbr/fanboy-adblock/issues
!
! Legal stuff (T&C's) .
! In no event shall Fanboy List, or the list author be liable for any indirect, direct, punitive, special, incidental, or consequential damages whatsoever.
! By downloading or viewing, or using this list, you are accepting these terms and the license.
!
! (Truncated from original list, which is ~30 KBs)

! infowarsstore.com
##a[href^="http://infowarsstore.com"]
##a[href^="http://www.infowarsstore.com"]
##a[href^="https://www.infowarsstore.com"]
infowarsstore.com#@#a[href^="http://infowarsstore.com"]
infowarsstore.com#@#a[href^="http://www.infowarsstore.com"]
infowarsstore.com#@#a[href^="https://www.infowarsstore.com"]
||infowarsstore.com^$popup,third-party

! infowars.com
##a[href^="http://infowars.com"]
##a[href^="http://www.infowars.com"]
infowars.com#@#a[href^="http://www.infowars.com"]
||infowars.com^$popup,third-party

! abcnews.com.co
##a[href^="http://abcnews.com.co"]
##a[href^="http://www.abcnews.com.co"]
abcnews.com.co#@#a[href^="http://abcnews.com.co"]
abcnews.com.co#@#a[href^="http://www.abcnews.com.co"]
||abcnews.com.co^$popup,third-party

! 82.221.129.208
##a[href^="http://82.221.129.208"]
82.221.129.208#@#a[href^="http://82.221.129.208"]

! angrypatriotmovement.com
##a[href^="http://angrypatriotmovement.com"]
##a[href^="http://www.angrypatriotmovement.com"]
angrypatriotmovement.com#@#a[href^="http://angrypatriotmovement.com"]
angrypatriotmovement.com#@#a[href^="http://www.angrypatriotmovement.com"]
||angrypatriotmovement.com^$popup,third-party
Loading

0 comments on commit e78ef71

Please sign in to comment.