You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is what I do:
ruby-head > bla = Robots.new("test")
=> #<Robots:0x000001008743c0 @user_agent="test", @parsed={}>
ruby-head > bla.allowed?("http://lacostarecords.net/")
=> false
This is the robots.txt
# Block a bot that was causing issues by ignoring Disallow lines below
User-Agent: OmniExplorer_Bot
Disallow: /
# Block hotlinking of music files by projectplaylist.com due to perceived user bandwidth theft
User-agent: projectplaylist-directlink
Disallow: /
# Block all bots from the core Homestead site
User-agent: *
Disallow: /~site/Scripts_ElementMailer
Disallow: /~site/Scripts_ExternalRedirect
Disallow: /~site/Scripts_ForSale
Disallow: /~site/Scripts_HitCounter
Disallow: /~site/Scripts_NewGuest
Disallow: /~site/Scripts_RealTracker
Disallow: /~site/Scripts_Track
Disallow: /~site/Scripts_WebPoll
it should return true, shouldn't it?
The text was updated successfully, but these errors were encountered:
This is what I do:
ruby-head > bla = Robots.new("test")
=> #<Robots:0x000001008743c0 @user_agent="test", @parsed={}>
ruby-head > bla.allowed?("http://lacostarecords.net/")
=> false
This is the robots.txt
# Block a bot that was causing issues by ignoring Disallow lines below
User-Agent: OmniExplorer_Bot
Disallow: /
it should return true, shouldn't it?
The text was updated successfully, but these errors were encountered: