foreach.lic

=begin
  Executes a series of commands for each item matching the criteria you specify.

  Some examples:
    ;foreach box in inv; move to locker
    ;foreach box in locker; move to backpack
    ;foreach gem in cloak; get item; appraise item; put item in container
    ;foreach gem in red sack; move to gemcutter; turn gemcutter; move to blue sack
    ;foreach scroll in inv; read item
    ;foreach gem in backpack; get item; prep 704; cast item; put item in container   (Note: won't check mana)
    ;foreach reagent in inv; get item; sell item
    ;foreach gem in backpack; get item; ;my_script item; put item in container
    ;foreach name=*quartz orb in inv; get item; put item in locker

  Basic usage:
    ;foreach [UNIQUE] [FIRST #] [AFTER #] [[ATTRIBUTE]=VALUE] in <CONTAINER>[; command; command; ...]

  Type ;foreach or ;foreach help for more details and information on functionality, or visit:
  https://github.com/dewiniaid/gs4-lich-scripts/blob/master/Foreach.md

  ;foreach is blind to closed containers.

	  author: LostRanger (thisgenericname@gmail.com)
	  game: any
	  tags: utility
	  required: Lich >= 4.6.0.

  version: 1.0.1 (2019-11-21)

  changelog:
    version 1.0.1 (2019-11-21)
      * Fixes to avoid items possibly being missed under certain circumstances due to Lich limitations.  This was
        most likely to happen in `;foreach ... in inv...` on large inventories on certain combinations of hardware
        and internet connectivity.

    version 1.0 (2019-10-21)
      * Now handles collective containers.  Example:
        If you have "You see three mannequins.  Looking at the mannequins, you see a dark grey mannequin, [...]"
        ;foreach on mannequins   will ;foreach every mannequin in the list.

    version 0.10.8 (2019-08-07)
      * Fix WAITFOR being broken since 0.10

    version 0.10.7 (2019-08-07)
      * Fix issue with commands that cause RT (and other statusbar-related updates) causing crashes on non-StormFront
        clients.

    version 0.10.6 (2019-07-26)
      * Fix the new pattern matching syntax being too greedy, which could break some Foreach scripts.

    version 0.10.5 (2019-07-25)
      * Properly unsilence ourselves when we send actual commands.  Previously, people who trusted Foreach could not
        see the commands it sent.  Now, only commands related to determining your initial inventory are squelched.
      * You can now `;foreach q=/pattern/ in ...` to do regular expression matches.

    version 0.10.4 (2019-06-20)
      * Made multiple fixes to 'giveitem'.  It should actually be recognized as a command again.
      * Added a new 'fastinv'/'qinv' target, which is similar to 'inv' but skips the INV FULL step.  This allows for
        slightly faster startup at the cost of being unable to check marked/registered status and being potentially
        unaware of the contents of containers that you have not previously LOOKed in.

    version 0.10.3 (2019-06-13)
      * Overhauled Stormfront status bar updates.  They should no longer significantly slow down the script.
        Additionally, status bar updates are less frequent (except when they are the result of the script pausing
        or resuming)
      * Running ;foreach while ;foreach is already running (i.e. with ;force) no longer clobbers the statusbar in Stormfront.
      * Immediate !commands now delay every 5 consecutive immediate commands rather than on every fifth item.
      * `;foreach ... in backpack on table` should now work correctly.
        - Note that a quirk in GS4's item parsing code means you will frequently need to use 'get item in container'
          syntax when using foreach in this way.

    version 0.10.2 (2019-06-12)
      * Sorting is now case-insensitive.
      * Read, Look, Analyze and Inspect now add an implicit 'item' if needed.
      * Giveitem now searches for a matching PC in the room at startup and autocompletes to that PC's full name.
        - It will error out immediately if no matching PCs are found or more than one matching PC is found.
        - It will no longer fail when you do not type the character's name entirely with proper capitalization.
      * Unspported: Commands can be prefixed with a "!" to ignore various checks on usability in favor of speed.
          Use at your own risk!

    version 0.10.1 (2019-06-11)
      * Fix marked/unmarked/registered/unregistered filters not working in some cases.

    version 0.10 (2019-06-10)
      * Foreach now reports its status to the shortcut bar in Stormfront, and can be paused, unpaused and killed from
          that location.
      * You can now ;foreach ... under|behind ...  instead of just ;foreach ... in   and ;foreach ... on
      * You can now ;foreach over multiple targets (;foreach in backpack,cloak,disk...)
      * Pattern matching is no longer case-sensitive.
          ;foreach q=elan*pack in pack should match "an Elanthian Guilds voucher pack"
      * `;foreach sorted ... in ...` will iterate over items alphabetically within their respective containers.
          Articles a/an/some/the are ignored when sorting.  The order of which container is evaluated first is unchanged.
      * `;foreach reversed ... in ...` will iterate over items backwards within their respective containers.
          This can be used to avoid changing container/room order, or in conjunction with 'sorted' to sort descending.
      * Completely overhauled the item matching engine to make it easier to maintain going forward.
      * `;foreach type=none in ...` will now match items that have no explicit type."

    version 0.9.9 (2019-05-24)
      * Add UNMARK shortcut for MARK <item> REMOVE
      * Improved documentation, including updating the online documentation
      * Convenience shortcuts for in-game verbs should now trigger when abbreviated:
          DRop, PLACe, SELl, APpraise, SELl, and REGister.
      * If the first command is `get`, `take`, `register`, `mark` or `unmark`, an implicit "get item" or
          "remove item" will be added beforehand.  This behavior already existed for drop, place, sell and appraise.
      * If the only command is appraise, register, mark, or unmark, items will be RETURNed
          afterwards in addition to the implicit get/remove item at the start.

    version 0.9.3 (2019-05-24)
      * No longer times out if you `;foreach in empty container` and have FLAG SORTEDVIEW ON
      * Improved error messaging when targetting empty, closed or nonexistant containers.
      * Fix ;foreach quick when using multiple patterns (;foreach q=red,blue in ...)
      * `move to locker` automatically opens your locker just like `foreach in locker` does

    version 0.9.2 (2019-05-22)
      * Now correctly works with locker manifests, even if there's nothing on your armor stand.

    version 0.9.1 (2019-05-21)
      * You can now ;foreach quick=phrase in ... or ;foreach q=phrase in ... as an alternate method for matching items
          This is equivalent to ;foreach fullname=*phrase* in ...   (note wildcards!)

          This feature was documented in 0.9, but was not implemented in that release.

    version 0.9 (2019-05-20)
      * You can now further filter displayed items with ';foreach [un]registered ... in ...` and
          `;foreach [un]marked ... in ...``
      * `;foreach in locker` will now open your locker if it is not already, and will close your locker when it exits
          if it originally opened it.
      * `;foreach in locker` in a premium locker will now try to use your locker manifest to avoid needing to scan
          multiple containers.  This adds a substantial startup speed boost.  If this attempt fails, it is still
          improved in that it will no longer examine the curtain, your disk, or any other non-locker containers in the
          room.
      * Added some more pretty formatting to ;foreach help  (Stormfront only)
      * Make a second attempt at improving compatibility with the Profanity FE

    version 0.8 (2019-05-09)
      * `;foreach in locker` should now work for non-premium lockers.
      * Improve compatibility with the Profanity FE
      * Fix an issue which caused Foreach to stall briefly in some cases.
      * No longer prompts to install xmlpatch on first launch
      * Add waitcastrt, waitcastrt? and waitrt? commands
      * Add sleep X command, where X is a duration.
      * Add UNIQUE, FIRST # and AFTER # modifiers.

    For a full changelog, see https://github.com/dewiniaid/gs4-lich-scripts/blob/master/Foreach.md

    version 0.1 (2017-06-18)
      * Initial release
=end

# [JUST UPDATED] ;foreach 0.10, featuring lots of updates, a few bugfixes, support for multiple targets, sorting, and more.  ;repo info foreach -- ;repo download foreach -- ;foreach help

# [JUST UPDATED] ;foreach 0.9, featuring filtering by marked/registered status, faster scans of premium lockers, and automatic locker opening/closing.  ;repo info foreach -- ;repo download foreach -- ;foreach help

EXPERIMENTAL_EXPIRES = nil
# EXPERIMENTAL_EXPIRES = Time.at(1560644027)

class ItemMatcher
    attr_accessor :first, :after, :unique, :sorted, :reversed, :item_filter
    attr_reader :total_items, :status_filter
    attr_reader :container_names, :container_contents

    HELP_URL = "https://github.com/dewiniaid/gs4-lich-scripts/blob/master/Foreach.md"
    UNREGISTERED = 1 << 0
    REGISTERED = 1 << 1
    UNMARKED = 1 << 2
    MARKED = 1 << 3

    INVFULL_PATTERN = /^( {2,})([^<]*)<a exist="(-?\d+)" noun="([^"]*)">([^<]*)<\/a>(?: ([^(]*)(?:$|(?= \()))?(?: (\(.*\)))?$/
    # Matches: 1 = leading space, 2 = prename, 3 = id, 4 = noun, 5 = name, 6 = postname, 7 = registered/marked attributes
    # You are carrying nothing at this time.  -- no inventory

    # INV_PATTERN = /(?:(?:Peering into|[IO]n) .* <a exist="(-?\d+)"[^>]*>([^<]*)<\/a>,? you see )|(?:[OI]n the <a exist="(-?\d+)"[^>]*>([^<]*)<\/a>:\s*$)|(There is nothing [io]n the|That is closed.|I could not find what you were referring to\.)/
    INV_PATTERN = /(?:(?:Peering into|In|On|Under|Behind) .* <a exist="(-?\d+)"[^>]*>([^<]*)<\/a>,? you see )|(?:[OI]n the <a exist="(-?\d+)"[^>]*>([^<]*)<\/a>:\s*$)|(There is nothing (?:in|on|under|behind) the|That is closed.|I could not find what you were referring to\.)|^You see <a.*?<\/a>\.\s+Looking at the <a.*?<\/a>, you see (.*)\.\s*$/
    LOCKER_PATTERNS = {
        :start => /^(?:(Looking in front of you, you see)|Thinking back, you recall) the contents of your locker/,
        :end => /^Obvious items:|^There are no items in this locker./,
        :container => /^<pushBold\/>[IO]n .* <a .* noun="(.*)">/,
        :item => /^(?:<popBold\/>)? {6}(?! )([^<]*)<a exist="(-?\d+)" noun="([^"]*)">([^<]*)<\/a>(?: ([^(]*)(?:$|(?= \()))?(?: (\(.*\)))?$/,
        :town_locales => /(Landing|Illistim|Vaalor|Mist Harbor|Kharam|Teras|Icemule|Solhaven|River's Rest)/
    }

    def initialize(first: nil, after: nil, unique: false, sorted: false, reversed: false, marked: nil, registered: nil)
        @first = first
        @after = after
        @unique = unique
        @sorted = sorted
        @reversed = reversed

        @item_filter = nil
        @status_filter = 0

        self.marked = marked
        self.registered = registered

        @container_names = {}
        @container_contents = {}
        @item_statuses = {}

        @pending_ingests = {}

        @scanned_inv = false
        @scanned_hands = false
        @scanned_locker  = false

        @seen_ids = Set.new

        @total_items = nil

        @command_cache = {}
    end

    def add_item!(item, container, container_name=nil, status_text=nil)
        return unless @seen_ids.add?(item.id)
        (@container_contents[container] ||= []) << item
        @container_names[container] ||= container_name

        @item_statuses[item.id] = ItemMatcher.status_from_text(status_text) if status_text

        return item
    end

    def add_item(item, container_id, container_name=nil, status_text=nil)
        return nil if @item_filter and not @item_filter.call(item)
        return add_item!(item, container_id, container_name, status_text)
    end

    def marked
        return true if (@status_filter & MARKED) != 0
        return false if (@status_filter & UNMARKED) != 0
        return nil
    end

    def marked=(v)
        @status_filter &= (MARKED|UNMARKED)
        @status_filter |= MARKED if v == true
        @status_filter |= UNMARKED if v == false
    end

    def registered
        return true if (@status_filter & REGISTERED) != 0
        return false if (@status_filter & UNREGISTERED) != 0
        return nil
    end

    def registered=(v)
        @status_filter &= (REGISTERED|UNREGISTERED)
        @status_filter |= REGISTERED if v == true
        @status_filter |= UNREGISTERED if v == false
    end

    def self.status_from_text(text)
        value = 0
        return UNREGISTERED | UNMARKED unless text
        if text =~ /registered/i
            value |= REGISTERED
        else
            value |= UNREGISTERED
        end

        if text =~ /marked/i
            value |= MARKED
        else
            value |= UNMARKED
        end
    end

    def _get_inventory(command)
        return @command_cache[command] unless @command_cache[command].nil?

        lines = ForeachScript.quiet_command(
            command,
            /^(?:You are carrying nothing at this time|You are currently (?:wearing and )?carrying)/,
            /(?:<prompt)|(?:.*items? displayed\.\)$)/,
            # /(?:<prompt)|(?:items displayed.\)\b)/,
            true,
            5
        )
        unless lines
            echo "Inventory request timed out."
            exit
        end
        return (@command_cache[command] = lines)
    end

    def _get_locker_containers
        containers = {}
        GameObj.loot.each do |item|
            containers[item.noun] = item if ForeachScript::PREMIUM_LOCKER_CONTAINERS[item.full_name]
        end
        return containers if containers.length == ForeachScript::PREMIUM_LOCKER_CONTAINERS.length
    end

    def _get_locker_town
        if Room.current and Room.current.id
            loc = Room.current.location
        else
            loc = Map.get_location
        end

        unless loc
            echo "Unable to determine what location this locker is in."
            return nil
        end

        unless loc =~ LOCKER_PATTERNS[:town_locales]
            echo "Location of this room is not familiar.  Unable to determine what town this locker is in."
            return nil
        end

        return $1
    end

    def _get_locker_manifest
        return @command_cache[:locker_manifest] if @command_cache[:locker_manifest]
        @scanned_locker = true
        locker_containers = _get_locker_containers
        return nil unless locker_containers
        return nil unless (town = _get_locker_town)
        manifest_command = "locker manifest #{town}"
        lines = ForeachScript.quiet_command(manifest_command, LOCKER_PATTERNS[:start])
        unless lines
            echo "Locker manifest request timed out."
            exit
        end
        to_return = []
        lines.each do |line|
            if line =~ LOCKER_PATTERNS[:start]
                unless $1   # Remote locker, not local locker
                    echo "Location of this room appears to be incorrect, or your locker is not open."
                    return nil
                end
            elsif line =~ LOCKER_PATTERNS[:end]
                return @command_cache[:locker_manifest] = to_return
            elsif line =~ LOCKER_PATTERNS[:container] or line =~ LOCKER_PATTERNS[:item]
                to_return << line
            end
        end
        echo "Locker manifest read was incomplete"
        return nil
    end

    def ingest_locker_from_manifest
        finished = false
        locker_containers = _get_locker_containers
        container_id = container_name = nil
        lines = _get_locker_manifest
        return false unless lines
        lines.each do |line|
            if line =~ LOCKER_PATTERNS[:container]
                unless (t = locker_containers[$1])
                    echo "Unrecognized container noun: #{$1}"
                    return nil
                end
                container_id = t.id
                container_name = t.full_name
            elsif line =~ LOCKER_PATTERNS[:item]
                unless container_id
                    manifest_command = "locker manifest #{town}"
                    echo "I found an item in a container before I found my first container.  This shouldn't happen."
                    echo "Please save a copy of #{manifest_command.upcase} and contact LostRanger"
                    exit
                end

                add_item(
                    GameObj.new($2.dup, $3.dup, $4.dup, $1.dup, $5.dup),
                    container_id, container_name, $6 || ''
                )
            end
        end
        return true
    end

    def _ingest_inventory(command)
        container_id = container_name = nil
        _get_inventory(command).each do |line|
            next unless line =~ INVFULL_PATTERN
            case $1.length
            when 2
                container_id = $3
                container_name = $5
                @item_statuses[container_id] = ItemMatcher.status_from_text($7)
            when 6
                unless container_id
                    echo "I found an item in a container before I found my first container.  This shouldn't happen."
                    echo "Please save a copy of INV FULL and contact LostRanger"
                    exit
                end
                add_item(
                    GameObj.new($3.dup, $4.dup, $5.dup, $2.dup, $6.dup),
                    container_id, container_name, ($7 || '')
                )
            else
                next
            end
        end
        return true
    end

    def ingest_inventory
        @scanned_inv = true
        return _ingest_inventory('inventory full')
    end

    def ingest_hands
        @scanned_hands = true
        return _ingest_inventory('inventory hands full')
    end

    def _status_from_inventory(command)
        _get_inventory(command).each do |line|
            next unless line =~ INVFULL_PATTERN
            @item_statuses[$3] = ItemMatcher.status_from_text($7 || '')
        end
    end

    def status_from_inventory
        return if @scanned_inv
        _status_from_inventory('inventory full')
    end

    def status_from_hands
        return if @scanned_hands
        _status_from_inventory('inventory hands full')
    end

    def ingest_container(container_id, container_name, items)
        items.each {|item| add_item(item, container_id, container_name)}
    end

    def schedule_container(container, location='in')
        h = (@pending_ingests[container.id] ||= {})
        h[:item] ||= container
        (h[:locations] ||= Set.new).add(location)
        return h
    end

    def ingest_legacy_locker
        GameObj.loot.each{|x|
            if (loc = ForeachScript::PREMIUM_LOCKER_CONTAINERS[x.full_name])
                schedule_container(x, loc)
            end
        }
    end

    def ingest_loot(location='in')
        # @pending_ingests += GameObj.loot.map{|item| item.id}
        GameObj.loot.each{|x| schedule_container(x, location)}
    end

    def ingest_desc
        # @pending_ingests += GameObj.loot.map{|item| item.id}
        GameObj.room_desc.each do |item|
            add_item(item, "_desc", "In the room")
        end
    end

    def ingest_ground
        GameObj.loot.each do |item|
            add_item(item, "_ground", "On the ground")
        end
    end

    def ingest_worn
        GameObj.inv.each do |item|
            add_item(item, "_worn", "On your person")
        end
    end

    def ingest_previous(other)
        other.container_contents.each do |container_id, items|
            container_name = other.container_names[container_id]
            items.each{|item| add_item(item, container_id, container_name)}
        end
    end

    def self.quiet_inv_scan_hook(xml)
        if @in_sorted_view
            @in_sorted_view = false if xml =~ /Total items: \d+$/
            return nil
        end

        return nil if xml =~ /^<prompt|^s*$/
        return xml unless xml =~ INV_PATTERN
        @in_sorted_view = true if $3
        return nil
        #
        # (xml =~ INV_PATTERN or xml =~ /^<prompt/ or xml =~ /^\s*$/) ? nil : xml
    end

    def maybe_fetch_status_data(exist)
        return true if @status_filter == 0
        return true if @item_statuses[exist]
        return fetch_status_data(exist)
    end

    def fetch_status_data(exist)
        if GameObj.right_hand.id == exist or GameObj.left_hand.id == exist
            status_from_hands
        else
            status_from_inventory
            status_from_hands unless @item_statuses[exist]
        end
        return @item_statuses[exist] && true
    end

    def ingest_from_name(target, position='in')
        begin
            ForeachScript.stop_scripts
            xml = dothistimeout("look #{position} #{target}", 15, INV_PATTERN)
            unless xml =~ INV_PATTERN
                echo "Timed out waiting for inventory command."
                exit
            end
            collection = $6
            error = $5
            exist = ($1 or $3)
            name = ($2 or $4)
        ensure
            ForeachScript.resume_scripts
        end
        if error
            if error.start_with?("There")  # is nothing in there
                echo "Container '#{target}' is empty."
                return false  # Not a hard error.
            elsif error.start_with?("That")  # is closed
                echo "Container '#{target}' is closed."
                exit
            else
                echo "Container '#{target}' was not found."
                exit
            end
        end
        if collection
            # Collection of items
            items = collection.split(/(?:\s*,\s*)|(?:\s+and\s+(?=[^,]*<a))/)
            items.each{|item|
                next unless item =~ /^(?:(.+)\s+)?<a exist="(-?\d+)" noun="([^"]*)">([^<]*)<\/a>(?:\s+(.+))?/
                ingest_from_name("##{$2}", position)
            }
            return
        end
        unless exist
            echo "Item ID for container '#{target}' could not be determined."
            exit
        end
        unless GameObj.containers[exist]
            echo "Inventory for container '#{target}' was not available."
            exit
        end
        ingest_container(exist, name, GameObj.containers[exist] || [])
        unless maybe_fetch_status_data(exist)
            echo "The container '#{container_names[exist]}' does not appear to be in your inventory, which means that marked/registered information cannot be obtained from it."
            exit
        end
    end

    def ingest_all_pending
        return if @pending_ingests.length == 0
        begin
            ForeachScript.stop_scripts
            remaining = @pending_ingests.length
            echo "Scanning #{remaining} inventor#{remaining==1 ? 'y' : 'ies'}, please wait..."  # if $SAFE == 0
            hook = ForeachScript.anon_hook
            @in_sorted_view = false
            DownstreamHook.add(hook, proc {|xml| ItemMatcher.quiet_inv_scan_hook(xml)})
            @pending_ingests.each do |id, ingest|
                container = ingest[:item]
                if remaining != @pending_ingests.length and remaining % ForeachScript::CONTAINER_UPDATE_INTERVAL == 0
                    echo "Still scanning #{remaining} inventor#{remaining==1 ? 'y' : 'ies'}, please wait..."  # if $SAFE == 0
                end
                ingest[:locations].each{|location| fput "look #{location} ##{container.id}"}
                remaining -= 1
            end
            waitfor '<prompt'

            @pending_ingests.each do |id, ingest|
                ingest_container(id, ingest[:item].full_name, GameObj.containers[id] || [])
                unless maybe_fetch_status_data(id)
                    echo "The container '#{container_names[id]}' does not appear to be in your inventory, which means that marked/registered information cannot be obtained from it."
                    exit
                end
            end
            @pending_ingests.clear
        ensure
            ForeachScript.resume_scripts
        end
    end

    def finalize
        ingest_all_pending

        seen_names = @unique && Set.new || nil
        skip = @after || 0
        first = @first

        final_contents = {}
        final_names = {}
        total_items = 0

        @container_contents.each do |container, contents|
            # Filter by status/uniqueness
            if @status_filter > 0
                contents.each{|item|
                    unless maybe_fetch_status_data(item.id)
                        echo "Failed to retrieve marked/registered info for '#{item.full_name}'.  Most likely this is because it is in a container not in your inventory."
                        exit
                    end
                }
                if @unique
                    contents.select!{|item| ((@item_statuses[item.id] & @status_filter) == @status_filter) && seen_names.add?(item.full_name)}
                else
                    contents.select!{|item| (@item_statuses[item.id] & @status_filter) == @status_filter}
                end
            elsif @unique
                contents.select!{|item| seen_names.add?(item.full_name)}
            end

            # Sort
            if @sorted == :noun
                contents.sort_by!{|x| "#{x.noun.downcase}#{x.full_name.gsub(/^(a|an|some|the)\s+/, '').downcase}"}
            elsif @sorted
                contents.sort_by!{|x| x.full_name.gsub(/^(a|an|some|the)\s+/, '').downcase} if @sorted
            end

            # Reverse
            contents.reverse! if @reversed

            # Handle skips
            if skip >= contents.length
                skip -= contents.length
                next
            elsif skip > 0
                contents = contents.drop(skip)
                skip = 0
            end

            if first
                if first >= contents.length
                    first -= contents.length
                else
                    contents = contents.first(first)
                    first = 0
                end
            end

            final_contents[container] = contents
            final_names[container] = @container_names[container]
            total_items += contents.length
        end

        @container_contents = final_contents
        @container_names = final_names
        @total_items = total_items
        @command_cache.clear
        @seen_ids.clear
        return total_items > 0
    end
end


class DummyStatusUpdater
    def status_until(status, cond=true, &block)
        block.call
    end

    def set(*a, **kw)
        # Dummy
    end
end

class StatusUpdater
    def initialize(script, level=1)
        @script = script
        @level = level
        @id = "quick-foreach-#{@level}"
        if @level > 1
            @title = "Foreach[#{@level}]: "
            @previous_quickbar = "quick-foreach-#{@level-1}"
        else
            @title = "Foreach: "
            @previous_quickbar = 'quick'
        end

        @unpaused_status = 'Running'

        root = REXML::Element.new("openDialog")
        root.attributes['id'] = @id
        root.attributes['location'] = 'quickBar'
        root.attributes['title'] = 'foreach'
        @xml_root = root

        @xml_data = data = root.add_element(
            "dialogData",
            {
                'id' => @id,
                'clear' => true
            })
        data.add_element('label', {'id' => "foreachLabel-#{@level}", 'value' => @title})

        @xml_status = data.add_element('label', {'id' => "foreachStatusLabel-#{@level}", 'value' => @unpaused_status})

        data.add_element("sep")

        data.add_element("link", {
            'id' => "foreachKill-#{@level}",
            'value' => "stop",
            'cmd' => "#{$lich_char}kill #{@script.name}"
        })

        data.add_element("sep")
        @xml_pause = data.add_element("link", {'id' => "foreachPauseResume-#{@level}" })

        data.add_element("sep")
        @xml_item = data.add_element("sep", {'id' => "foreachItemData-#{@level}"})

        @xml = nil

        @last_was_paused = @script.paused?

        @watchdog_thread = nil
        @updater_thread = nil

        @hook_name = nil
        before_dying { cleanup }
    end

    def cleanup
        UpstreamHook.remove(@hook_name) if @hook_name
        @watchdog_thread.kill if @watchdog_thread
        @updater_thread.kill if @updater_thread
        @watchdog_thread = @updater_thread = nil

        if @xml
            puts "<switchQuickBar id=\"#{@previous_quickbar}\" />"
            puts "<closeDialog id=\"#{@id}\" />"
        end
    end

    def set_hook
        now = Time.now
        @hook_name = "Foreach/QB::#{now.tv_sec}.#{now.tv_usec}-#{Random.rand(10000)}"
        UpstreamHook.add(@hook_name, proc{|line|
            next line unless line =~ /<c>_qlink change (.*)$/
            qb = $1
            if qb == @id
                puts "<switchQuickBar id=\"#{@id}\"/>"
                next nil
            elsif qb =~ /^quick-foreach-(\d+)$/
                level = $1.to_i
                next if level > @level
            end
            @previous_quickbar = qb
            next line
        })
    end

    def start_watchdog
        return if @watchdog_thread
        paused = @script.paused
        @watchdog_thread = Thread.new {
            while true
                sleep 0.25
                new_paused = @script.paused?
                if new_paused != paused
                    paused = new_paused
                    set
                    # @updater_thread.wakeup if @updater_thread
                end
            end
        }
    end

    def start_monitor
        return if @updater_thread
        last_xml = @xml
        @updater_thread = Thread.new {
            while true
                sleep 2
                xml = @xml
                unless  xml == last_xml
                    last_xml = xml
                    _respond xml
                end
            end
        }
    end

    def set(status: nil, item: nil, items_processed: nil, items_total: nil, percent: nil, immediate: false)
        @unpaused_status = status if status
        is_paused = @script.paused || status == 'Paused'
        if is_paused
            status = 'Paused'
        else
            status = @unpaused_status
        end

        @xml_status.attributes['value'] = status if status

        a = @xml_pause.attributes
        if is_paused
            a['value'] = "resume"
            a['cmd'] = "#{$lich_char}unpause #{@script.name}"
        else
            a['value'] = "pause"
            a['cmd'] = "#{$lich_char}pause #{@script.name}"
        end

        if item
            type = item.type
            type = '(no type)' if type == '' || type.nil?
            @xml_item.attributes['value'] = "[#{items_processed+1}/#{items_total}] (#{percent}%) ##{item.id} \"#{item.full_name}\" (#{type})"
        end

        new_xml = @xml_data.to_s
        return if new_xml == @xml
        first_run = @xml.nil?
        @xml = new_xml

        if first_run
            puts "#{@xml_root.to_s}<switchQuickBar id=\"#{@id}\" />"

            set_hook
            start_watchdog
            start_monitor
        elsif (is_paused != @last_was_paused) || immediate
            last_was_paused = is_paused
            @updater_thread.wakeup  # End sleep early to force an immediate update if our paused status changed.
        end
    end

    def status_until(status, cond=true, &block)
        set(status: status) if cond
        block.call
        set(status: 'Running') if cond
    end
end


module ForeachScript
    VERSION = '1.0.1 (2019-11-21)'
    VERSION_INT = 1_000_001
    CONFIG_VERSION = 2
    # FILTER_PATTERN = /^(?:(?:(?:(.+)=)?(.+)\s+(in|on|under|behind)\s+)|(in|on|under|behind)\s+)(.+)$/
    FILTER_PATTERN = /^(?:(in|on|under|behind)|(?:(?:(.+)=)?(.+?)\s+(in|on|under|behind)))\s+(.+)$/

    # INV_PATTERN = /(?:(?:Peering into|In|On|Under|Behind) .* <a exist="(-?\d+)"[^>]*>([^<]*)<\/a>,? you see )|(?:[OI]n the <a exist="(-?\d+)"[^>]*>([^<]*)<\/a>:\s*$)|(There is nothing (?:in|on|under|behind) the|That is closed.|I could not find what you were referring to\.)/
    HOOK = 'foreach_script_temp_hook'
    HELP_URL = "https://github.com/dewiniaid/gs4-lich-scripts/blob/master/Foreach.md"

    # INVFULL_PATTERN = /^( {2,})([^<]*)<a exist="(-?\d+)" noun="([^"]*)">([^<]*)<\/a>(?: ([^(]*)(?:$|(?= \()))?(?: (\(.*\)))?$/
    # # Matches: 1 = leading space, 2 = prename, 3 = id, 4 = noun, 5 = name, 6 = postname, 7 = registered/marked attributes
    # # You are carrying nothing at this time.  -- no inventory
    #
    CONTAINER_UPDATE_INTERVAL = 10  # How often do we notify how many more containers are being scanned
    ITEM_UPDATE_INTERVAL = 10  # How often do we notify how many more items are being processed?

    COMMAND_PATTERNS = {
        :move => /^(f(?:ast)?)?(?:move|mv)\s+(?:(.+)\s+)?to\s+(.*)$/i,
    }

    PREMIUM_LOCKER_CONTAINERS = {
        'armor stand' => 'on',
        'weapon rack' => 'on',
        'magical item bin' => 'in',
        'clothing wardrobe' => 'in',
        'deep chest' => 'in'
    }

    @previous_matcher ||= nil

    def self.puts(msg)
        _respond msg
    end

    def self.zput(what, fast)
        if fast
            put what
        else
            fput what
        end
    end

    def self.anon_hook(prefix = '')
        now = Time.now
        "Foreach::#{prefix}-#{now.tv_sec}.#{now.tv_usec}-#{Random.rand(10000)}"
    end

    def self.cleanup
        [DownstreamHook, UpstreamHook].each{|provider|
            provider.list.find_all{|name| name.start_with?('Foreach::')}.each{|name| provider.remove(name)}
        }
    end

    def self.send_formatted(msg, mono=true)
        if @stormfront
            msg = REXML::Text.new(msg, respect_whitespace: true).to_s
            msg.gsub!(/\*\*(.*?)\*\*/, '<preset id="whisper">\1</preset>')
            msg = "<output class=\"mono\" />\n#{msg}\n<output class=\"\" />" if mono
            puts msg
        else
            msg.gsub!('**', '')
            respond msg
        end
    end

    def self.send_help_formatted(msg)
        if @stormfront
            msg = REXML::Text.new(msg, respect_whitespace: true).to_s
            msg.gsub!(/\*\*(.*?)\*\*/, '<preset id="whisper">\1</preset>')
            msg.gsub!(/`(.*?)`/, '<preset id="speech">\1</preset>')
            msg.gsub!("#{name} help", "</preset><d cmd=\"#{name} help\">#{name} help</d><preset id=\"whisper\">")
            puts "<output class=\"mono\" />\n#{msg}\n<output class=\"\" />"
        else
            msg.gsub!('**', '')
            msg.gsub!('`', '')
            respond msg
        end
    end

    def self.first_time_setup(force = false)
        return if GameSettings['version'] == CONFIG_VERSION and not force
        return if EXPERIMENTAL_EXPIRES

        send = "#{$lich_char}send to #{@script.name}"

        option = proc{|what|
            cmd = "#{send} #{what.upcase}"
            if @stormfront
                puts "    <d cmd=\"#{cmd}\">#{cmd}</d>"
            else
                puts "    #{cmd}"
            end
        }

        unless force
            if GameSettings['version'] == 1
                echo "Migrating settings from a previous version of Foreach..."
                if Script.exists?('GameObjAddMore.lic')
                    unless Script.exists?('xmlpatch.lic')
                        echo "You have GameObjAddMore installed, which is deprecated and will be removed soon."
                        echo "The script 'xmlpatch' replaces it.  I'll go install that for you now."

                        Script.run('repository', 'download xmlpatch.lic')
                        Script.run('xmlpatch')
                    end
                    echo "Done!"
                    puts "\nTo see what's new in this verison and view examples on how to use #{$lich_char}foreach, read the following:"
                    respond "    #{HELP_URL}"
                    GameSettings['version'] = CONFIG_VERSION
                    return
                end
            end
        end

        echo "Starting one time setup..."

        msg = ["**Welcome to ForEach version #{VERSION}**"]
        msg << "**#{'-' * (msg[0].length-4)}**"
        msg << "This one-time setup process will help you get started with ForEach and configure it for future use."
        msg << "Once you've answered a few quick questions, you won't see these messages again."
        msg << ''
        if @stormfront
            msg << "Click on your answers to the questions, or type '#{send} <answer>, where <answer> is your answer."
        else
            msg << "Type '#{send} <answer>' to answer these questions, where <answer> is your answer."
        end
        msg << ''
        send_formatted(msg.join("\n"), false)

        while true
            puts "Would you like Lich to check for updates to this script on startup?\n"
            option.call('yes')
            option.call('no')
            response = unique_get.strip.downcase
            if response =~ /^y(?:es)?$/
                Script.run('repository', "set-updatable #{@script.name}.lic")
                Script.run('autostart', "add --global repository download-updates")
                respond "If you change your mind later, enter #{$lich_char}repository unset-updatable #{@script.name}.lic"
                break
            elsif response =~ /^no?$/
                Script.run('repository', "unset-updatable #{@script.name}.lic")
                respond "If you change your mind later, enter #{$lich_char}repository set-updatable #{@script.name}.lic"
                break
            else
                echo 'Please answer YES or NO'
            end
        end
        puts ''
        sleep 1
        gameobjadd = false
        gameobjaddmore = false
        #
        # while true
        #     puts [
        #         "ForEach has the ability to filter items by type using some functionality built-in to Lich.  However,",
        #         " the data included in Lich is a bit outdated so some items may have the wrong type -- or have no type",
        #         " at all.\n",
        #         "\n",
        #         "I've written a script called 'xmlpatch' that is designed to run on startup and update Lich's data",
        #         " about spells and items.  This script exits afterwards and requires no additional resources beyond that",
        #         " point.\n",
        #         "\n",
        #         "'xmlpatch' replaces and supercedes similar scripts like GameObjAdd and GameObjAddMore.\n",
        #         "\n",
        #         "Would you like me to setup xmlpatch for you?\n"
        #     ].join('')
        #     option.call('yes')
        #     option.call('no')
        #     option.call('info')
        #     response = unique_get.strip.downcase
        #     if response =~ /^y(?:es)?$/
        #         Script.run('repository', "download xmlpatch.lic")
        #         Script.run('xmlpatch')
        #         gameobjadd = true
        #
        #         # respond "If you change your mind later, you can remove GameObjAdd from Autostart with: ${lich_char}autostart remove --global GameObjAdd"
        #         break
        #     elsif response =~ /^no?$/
        #         respond "If you change your mind later, enter the following commands:"
        #         respond "#{$lich_char}repository download xmlpatch.lic"
        #         respond "#{$lich_char}xmlpatch"
        #         break
        #     elsif response =~ /^(?:info|help|details)$/
        #         Script.run('repository', 'info xmlpatch.lic')
        #     else
        #         echo 'Please answer YES, NO or INFO'
        #     end
        # end

        sleep 1
        echo "Just a few more things..."
        puts "You may want to check out some of my other scripts.  Here's a list:"
        Script.run('repository', 'list --author=LostRanger')
        puts ''
        puts [
                 "If ForEach is a trusted script, it will hide the commands it is sending to scan inventories and thus",
                 " reduce a tiny bit of screen spam.  This is entirely optional and doesn't otherwise affect the script at all.",
                 "  You can trust the script with\n",
             ].join('')
        respond "    #{$lich_char}trust foreach"
        puts "(LostRanger recommends that you review the code for scripts before trusting them!)"
        puts "\nTo view information and examples on how to use #{$lich_char}foreach, read the following:"
        respond "    #{HELP_URL}"
        puts "\nTo see a short reminder on how to use #{$lich_char}foreach in game, type:"
        respond "    #{$lich_char}#{@script.name} help"
        puts "\nTo run this setup process again at any time, type:"
        respond "    #{$lich_char}#{@script.name} setup"

        if @script.vars[0].strip.length > 0 and not force
            puts "\nTo run the command you were about to run when Foreach launched its first-time setup, type:"
            respond "    #{$lich_char}#{@script.name} #{@script.vars[0]}"
        end
        GameSettings['version'] = CONFIG_VERSION
        exit
    end

    def self.show_changelog
        prev = GameSettings[:current_version] || 0

        changelog = {
            8_000 => [
                "Major version 0.8 changes:",
                '',
                "  * `;foreach in locker` should now work for non-premium lockers",
                '',
                "  * Fixed an issue which caused Foreach to stall briefly in some cases.",
                '',
                "  * No longer prompts to install XMLpatch on first launch.",
                '',
                "  * Add `sleep X` command, where X is a duration in seconds.",
                '',
                "  * `#{$lich_char}#{@script.name} loud` now toggles loud mode (mainly for debugging) and `#{$lich_char}#{@script.name} quiet` now disables it.  This logic was accidentally reversed earlier.",
                '',
                "  * Add `waitcastrt`, `waitrt?`, and `waitcastrt?` commands:",
                "    `waitrt?` and `waitcastrt?` only wait for roundtime if it is currently present, as opposed to waiting for you to enter roundtime first.",
                '',
                "  * Added `UNIQUE`, `FIRST #`, and `AFTER #` modifiers.",
            ],
            9_000 => [
                "Major version 0.9.x series changes:",
                '',
                "  * Improved documentation, including updating the online documentation.",
                '',
                "  * You can now further filter displayed items with `;foreach [un]registered ... in ...` and `;foreach [un]marked ... in ...`.",
                "    These options will only work for items in your inventory or premium locker.",
                '',
                "  * `;foreach in locker` will now open your locker if it is not already, and will close your locker when it exits if it originally opened it.  `move to locker` will do similar.",
                '',
                "  * `;foreach in locker` in a premium locker will now try to use your locker manifest to avoid needing to scan multiple containers.  This adds a substantial startup speed boost.",
                "    If this attempt fails, it is still improved in that it will no longer examine the curtain, your disk, or any other non-locker containers in the room.",
                '',
                "  * Added some more pretty formatting to ;foreach help  (Stormfront only)",
                '',
                "  * Make a second attempt at improving compatibility with the Profanity FE",
                '',
                "  * Add UNMARK as a shortcut for MARK item REMOVE",
                '',
                "  * Convenience shortcuts for in-game verbs should now trigger when abbreviated.",
                "    This includes: DRop, PLACe, SELl, APpraise, and REGister",
                '',
                "  * If the first command is `get`, `take`, `register`, `mark` or `unmark`, an implicit \"get item\" or \"remove item\" will be added beforehand.",
                "    This behavior already existed for `drop`, `place`, `sell `and `appraise`.",
                '',
                "  * If the only command specified is `appraise`, `register`, `mark` or `unmark`, items will be RETURNed afterwards in addition to the implicit get/remove at the start.",
            ],
            10_000 => [
                "Version 0.10 changes:",
                '',
                "  * `;foreach sorted ... in ...` will iterate over items alphabetically within their respective containers.",
                "    Articles a/an/some/the are ignored when sorting.  The order of which container is evaluated first is unchanged.",
                '',
                "  * `;foreach reversed ... in ...` will iterate over items backwards within their respective containers.",
                "    This can be used to avoid changing container/room order, or in conjunction with 'sorted' to sort descending.",
                '',
                "  * `;foreach in X,Y,Z` can now be used to search in multiple targets at a time.",
                "    This works with containers as well as the 'special' targets",
                "    Try `;foreach in inv,disk;move to locker`",
                '',
                "  * Pattern matching is no longer case-sensitive.",
                "    `;foreach q=elan*pack in sack` should match \"an Elanthian Guilds voucher pack\"",
                '',
                "  * Completely overhauled the item matching engine to make it easier to maintain going forward.",
                '',
                "  * `;foreach type=none in ...` will now match items that have no explicit type.",
                '',
                "  * You can now ;foreach ... under|behind ...  instead of just ;foreach ... in   and ;foreach ... on",
                '',
                "  * Legacy premium locker searching now looks ON armor stands and weapon racks rather than IN them.",
                "    (This only triggers if LOCKER MANIFEST fails for some reason, and only on premium lockers.)",
            ],
            10_001 => [
                "Version 0.10.1 changes:",
                '',
                "  * Fix filters for marked/unmarked/registered/unregistered not working in some cases.",
            ],
            10_002 => [
                "Version 0.10.2 changes:",
                '',
                "  * Sorting is now case-insensitive.",
                "  * Read, Look, Analyze and Inspect now add an implicit 'item' if needed.",
                "  * Giveitem now searches for a matching PC in the room at startup and autocompletes to that PC's full name.",
                "    - It will error out immediately if no matching PCs are found or more than one matching PC is found.",
                "    - It will no longer fail when you do not type the character's name entirely with proper capitalization.",
                "  * Unspported: Commands can be prefixed with a '!' to ignore various checks on usability in favor of speed.",
                "      Use at your own risk!",
            ],
            10_003 => [
                "Version 0.10.3 changes:",
                '',
                "  * Overhauled Stormfront status bar updates.  They should no longer significantly slow down the script.",
                "    Additionally, status bar updates are less frequent (except when they are the result of the script pausing or resuming)",
                '',
                "  * Running ;foreach while ;foreach is already running (i.e. with ;force) no longer clobbers the statusbar in Stormfront.",
                '',
                "  * Immediate !commands now delay every 5 consecutive immediate commands rather than on every fifth item.",
                '',
                "  * `;foreach ... in backpack on table` should now work correctly.",
                "    Note that a quirk in GS4's item parsing code means you will frequently need to use 'get item in container' syntax when using foreach in this way.",
            ],
            10_004 => [
                "Version 0.10.4 changes:",
                '',
                "  * Made multiple fixes to 'giveitem'.  It should actually be recognized as a command again.",
                '',
                "  * Added a new `fastinv`/`qinv` target.",
                "    This is similar to `inv`, but skips the INV FULL step.  This allows for slightly faster startup but with some caveats.  For details, see the documentation.",
            ],
            10_005 => [
                "Version 0.10.5 changes:",
                '',
                "  * Properly unsilence ourselves when we send actual commands.",
                "    Previously, people who trusted Foreach could not see the commands it sent.",
                "    Now, only commands related to determining your initial inventory are squelched.",
                '',
                "  * You can now `;foreach q=/pattern/ in ...` to do regular expression matches.",
            ],
            10_006 => [
                "Version 0.10.6 changes:",
                '',
                "  * Fix the new pattern syntax being too greedy, which could break some Foreach scripts.",
            ],
            10_008 => [
                "Version 0.10.7/0.10.8 changes:",
                '',
                "  * Fix crash involving commands after RT in clients other than Stormfront.",
                "  * Fix WAITFOR being broken since 0.10",

            ],
            1_000_000 => [
                "Version 1.0 changes:",
                '',
                "  * Support unwrapping collective containers",
                "    In short, this means that when you have something like 'You see three mannequins.  Looking at the mannequins...', `;foreach on mannequins` will look at all of the mannequins."
            ],
            1_000_001 => [
                "Version 1.0.1 changes:",
                '',
                "  * Fixes to avoid items possibly being missed under certain circumstances due to Lich limitations.",
                "    This was most likely to happen in `;foreach ... in inv...` on large inventories on certain combinations of hardware and internet connectivity.",
            ]
        }

        if prev != VERSION_INT
            msg = []
            msg << "<preset id='speech'>--------------------</preset>"
            msg << "<preset id='speech'>Welcome to #{$lich_char}#{@script.name}</preset> <preset id='whisper'>#{VERSION}</preset>"
            msg << ''
            changelog.each do |version, notes|
                if version > prev or version == VERSION_INT
                    msg << "<preset id='whisper'>#{notes[0]}</preset>"
                    msg += notes[1..-1]
                    msg << ''
                end
            end
            msg << '<output class="mono" />'
            msg << "For a more detailed changelog: <d>#{$lich_char}repo info #{@script.name}</d>."
            msg << "To enable automatic updates:   <d>#{$lich_char}repo set-updatable #{@script.name}</d>`."
            msg << "For details on new features:   <d>#{$lich_char}#{@script.name} help</d>"
            msg << "                   or visit:   #{HELP_URL}"
            msg << "For more LostRanger scripts:   https://bit.ly/lostranger-scripts"
            msg << '<output class="" />'
            msg << ''
            msg << "This message will only be displayed once unless a new version is downloaded."
            msg << "<preset id='speech'>--------------------</preset>"
            send msg
            GameSettings[:current_version] = VERSION_INT
            sleep 1.5
        end
    end

    def self.send(what)
        if @stormfront
            return _respond(what)
        end

        unless what.is_a?(Array)
            what = [what]
        end

        return _respond what.map{|line|
            if line =~ /^\s*<\/?output[^>]*>$/
                nil
            else
                line = line.gsub(/<preset.*?>(.*?)<\/preset>/, "\\1")
                line.gsub!('<pushBold/>', monsterbold_start)
                line.gsub!('<popBold/>', monsterbold_end)
                line.gsub!(/<\/?d.*?>/, '')
                line
            end
        }.keep_if{|x| x}
    end

    def self.show_help(full)
        name = "#{$lich_char}#{@script.name}"
        msg = []
        msg << "**This is ForEach version #{VERSION}**"
        msg << "**Usage:**"
        msg << "     #{name} [OPTIONS] [[ATTRIBUTE=]VALUE] in/on/under/behind <TARGETS>[; command; command; ...]"
        msg << "  or #{name} [OPTIONS] [[ATTRIBUTE=]VALUE] in/on/under/behind <TARGETS>[/ command/ command/ ...]"
        msg << "  or #{name} [OPTIONS] [[ATTRIBUTE=]VALUE] in/on/under/behind <TARGETS>[| command| command| ...]"
        msg << "  or #{name} [OPTIONS] [ATTRIBUTE=]/PATTERN/ in/on/under/behind <TARGETS>[| command| command| ...]"
        # msg << ''
        # msg << "Type #{name} SET for settings"
        msg << ''
        if full
            msg << "**ATTRIBUTE** can be one of: `noun`, `name`, `fullname`, `quick`, or `type`.  If unspecified, it is type."
            msg << "    These can also be shortned to `n`, `m`, `f`, `q` or `t`."
            msg << "    `quick` is equivalent to `fullname` with automatic wildcards at the ends."
            msg << ''
            msg << "**VALUE** is what the attribute's value must match.  Wildcards (with *) are supported.  Multiple VALUEs can be matched by separating them with commas."
            msg << "**PATTERN** may be used instead, in which case it is a regular expression that the attribute must match."
            msg << "Patterns are denoted by being surrounded by slashes, i.e. `/(blue|white) crystal/`"
            msg << ''
            msg << "**TARGETS** is a comma-separated list of one or more TARGETs, each of which can be a name of a container or one of the following:"
            msg << ''
            msg << "    `INV` or `INVENTORY`  - Contents of all containers in your inventory."
            msg << "    `WORN`              - All worn items in your inventory."
            msg << "    `FLOOR`/`GROUND`/`ROOM` - Items in the room.  (Not their contents)"
            msg << "    `LOOT`              - Contents of items in the room.  (e.g. contents of open boxes on the ground)"
            msg << "    `LOCKER`            - Contents of your locker (including premium locker containers)."
            msg << "    `PREVIOUS` or `LAST`  - All items that matched the previous run of Foreach."
            msg << "    `FASTINV` or `QINV`   - Like INV, but using only loaded inventory data without an INV FULL.  See documentation for caveats and full details."
            msg << ''
            msg << "Each **command** will be executed on the item in sequence.  Within each command, the following replacements will be be made:"
            msg << "    `ITEM`      - An exact reference to the item, i.e. it's item number."
            msg << "    `NAME`      - The full name of the item.  Not suitable for commands, suitable for speech."
            msg << "    `NOUN`      - The item's noun.  Unreliable for commands, suitable for speech."
            msg << "    `CONTAINER` - An exact reference to the container the item was found in.  Not available if using `FLOOR`/`GROUND`"
            msg << "                If using `PREVIOUS`, this is the container the item WAS in, not its current location."
            msg << ''
            msg << "**OPTIONS** can consist of any of the following:"
            msg << "    `UNIQUE`         - Only examines the first occurrence for any given full item name."
            msg << "    `FIRST n`        - Only examines the first `n` matching items.  (The word '`FIRST`' is optional.)"
            msg << "    `AFTER n`        - will skip the first `n` matching items.  (You can also use `SKIP`)"
            msg << "    `[UN]MARKED`     - Only considers marked or unmarked items.  Only available for targets in your inventory or premium locker."
            msg << "    `[UN]REGISTERED` - Only considers registered or unregistered items.  Only available for targets in your inventory or your premium locker."
            msg << "    `SORTED`         - Within each target, sort items alphabetically by full name (ignoring articles a/an/some/the)."
            msg << "    `NOUNSORTED`     - Within each target, sort items alphabetically by noun, then by full name (ignoring articles a/an/some/the)."
            msg << "    `REVERSED`       - Reverse the order in which items are iterated, possibly after sorting."
            msg << ''
            msg << "If no commands are specified, #{name} will show a list of all matching items instead."
            msg << "This list will show item types, so '#{name} in backpack' can help you determine which item type to use."
            msg << ''
            msg << "If no attribute or value are specified, #{name} examines all items in the relevant locations."
            msg << ''
            msg << "Each command can be a game command, a lich script (invoked as if typing it in the client), or one of the following"
            msg << "convenience commands:"
            msg << ''
            msg << "    `UNMARK [<item>]`                - Equivalent to MARK <item> REMOVE, but works with shortcuts."
            msg << "    `ECHO <message>`                 - Echo a line of text to your client."
            msg << "    `GIVEITEM <player>`              - Give an item to a player and wait for them to accept it."
            msg << "    `[FAST]MOVE [<item>] TO <where>` - Convenience shortcut for 'get item; put item in container.  <where> can also be `GROUND` or `FLOOR`"
            msg << "    `STASH [<item>]`                 - Attempts to move the item to your lootsack(s), trying them in order if they are full."
            msg << "    `PAUSE`                          - Pause the script.  #{name} will resume where it left off when unpaused."
            msg << "    `SLEEP <time>`                   - Pause for <time> seconds.  Decimals, e.g. '0.5', are allowed."
            msg << "    `WAITRT` or `WAITRT?`              - Wait for roundtime to finish.  'waitrt?' will not wait if you are not already in RT."
            msg << "    `WAITCASTRT` or `WAITCASTRT?`      - Wait for cast roundtime to finish.  'waitcastrt?' will not wait if you are not already in cast RT."
            msg << "    `WAITFOR <phrase>`               - Waits to see the specified phrase.  Case insensitive."
            msg << "    `WAITRE <pattern>`               - Waits to see the specified regular expression."
            msg << "    `WAITMANA <#>` or `WAITMP <#>`     - Waits until you have at least this much mana."
            msg << "    `WAITHEALTH <#>` or `WAITHP <#>`   - Waits until you have at least this much mana."
            msg << "    `WAITSPIRIT <#>` or `WAITSP <#>`   - Waits until you have at least this much stamina."
            msg << "    `WAITSTAMINA <#>` or `WAITST <#>`  - Waits until you have at least this much spirit."
            msg << "    `WAITFOR <phrase>`               - Waits to see the specified phrase.  Case insensitive."
            msg << "    `WAITRE <pattern>`               - Waits to see the specified regular expression."
            msg << ''
            msg << "There are some other convenience shortcuts with certain regular game commands, details are in the online documentation."
            msg << "#{name} is blind to closed containers (except lockers, which will be automatically opened and closed afterwards.)"
            msg << ''
            msg << "If #{name} is trusted ('#{$lich_char}trust #{@script.name}'), its initial inventory scan output will be silenced."
            msg << ''
            msg << "Item types are based on Lich's definition of item types, also seen in #{$lich_char}sorter.  The current list is:"
            msg << GameObj.type_data.map{|k,v| k}.sort.join(', ')
        else
            msg << "**ATTRIBUTEs** include `type`(default), `noun`, `name`, `fullname`, `quick`, or shorthand equivalents `t`, `n`, `m`, `f` and `q`"
            msg << "**TARGETs** include any inventory container or `INV`, `WORN`, `ROOM`, `LOOT`, `LOCKER`, and `LAST` (and various synonyms listed in the full help)"
            msg << "**COMMANDs** are any game command, or convenience shortcuts `UNMARK`, `ECHO`, `GIVEITEM`, `STASH`, `FASTMOVE`, `MOVE`, `PAUSE`, `SLEEP`, `WAITRT`, `WAITCASTRT`, `WAITFOR`, `WAITRE`, `WAITMANA`, `WAITHEALTH`, `WAITSPIRIT`, and `WAITSTAMINA`"
            msg << "**OPTIONS** include `FIRST n`, `AFTER n`, `UNIQUE`, `[UN]MARKED` and `[UN]REGISTERED`"
            msg << ''
        end

        if full
            msg << "**To view detailed documentation and examples online, visit:**"
        else
            msg << "**This is the short version of the documentation.  For more, see #{name} help or visit:**"
        end
        msg << HELP_URL
        msg << ''
        if full
            msg << "**For information on other scripts by LostRanger, visit:**"
            msg << "https://bitly.com/lostranger-scripts"
        end

        return send_help_formatted(msg.join("\n"))
    end
    #
    # def self.show_settings
    #     msg = []
    #     msg << "**This is ForEach version #{VERSION}**"
    #     msg << "**Usage:**"
    #     msg << "     #{name} SET [name=value [name=value ...]]"
    #     msg << ''
    #     msg << "**Settings (and current values):**"
    #
    #     [
    #         [
    #             "STATUS", GameSettings[:show_status] && 'ON ' || 'OFF',
    #             "Show status in shortcut bar (Stormfront only)"
    #         ],
    #         [
    #             "SAFETY", GameSettings[:show_status] && 'ON ' || 'OFF',
    #             "Show status in shortcut bar (Stormfront only)"
    #         ]
    #
    #     ].each do |k, v, d|
    #         msg << "    #{k.ljust(10)} = `#{v.ljust(10)}` : #{d}"
    #     end
    #
    #     return send_help_formatted(msg.join("\n"))
    # end

    def self.quiet_command(command, start_pattern, end_pattern = /<prompt/, include_end = true, timeout=5)
        result = []
        name = self.anon_hook
        filter = false

        begin
            Timeout::timeout(timeout, Interrupt) {
                DownstreamHook.add(name, proc {|xml|
                    if filter
                        if xml =~ end_pattern
                            DownstreamHook.remove(name)
                            filter = false
                            # result << xml.rstrip if include_end
                            # thread.raise(Interrupt)
                            # next(include_end ? nil : xml)
                        else
                            # result << xml.rstrip
                            next(nil)
                        end
                    elsif xml =~ start_pattern
                        filter = true
                        # result << xml.rstrip
                        next(nil)
                    else
                        xml
                    end
                })
                fput command

                until (xml = get) =~ start_pattern; end
                result << xml.rstrip
                until (xml = get) =~ end_pattern
                    result << xml.rstrip
                end
                if include_end
                    result << xml.rstrip
                end
            }
        rescue Interrupt
            nil
        end
        return result
    end

    def self.stop_scripts
        ['sorter'].each{|s|
            if Script.running?(s)
                Script.kill(s)
                @stopped_scripts << s
            end
        }
    end

    def self.resume_scripts
        @stopped_scripts.each{|s| Script.start(s) }
        @stopped_scripts.clear
    end

    def self.holding?(id)
        id == GameObj.right_hand.id or id == GameObj.left_hand.id
    end

    def self.find_sacks(base = 'lootsack')
        # Adapted from ;loot
        sacks = []
        ix = nil
        while true
            var = "#{base}#{ix}"
            ix = 1 if ix.nil?
            ix += 1
            name = Vars[var].strip
            return sacks unless name.length > 0
            if name =~ /^my\s+(.*)$/i
                name = $1
            end
            pattern = Regexp::escape(name)
            sack = nil
            [ /\b#{pattern}$/i, /\b#{pattern.sub(' ', '.*')}$/i, /\b#{pattern.sub(' ', '.*')}/i ].each{|pattern|
                sack = GameObj.inv.find {|obj| obj.name =~ pattern }
                break if sack
            }
            if sack
                sacks << sack
            else
                echo "warning: failed to find #{var} '#{name}'"
            end
        end
    end

    def self.build_filter(value, attr, target)
        attr = 'type' unless attr
        return proc {|item| true} unless value

        if value =~ /^\/(.*)\/([a-zA-Z]+)?$/
            options = $2
            begin
                pattern = Regexp.new($1, Regexp::IGNORECASE)
            rescue => ex
                echo ex.to_s
                exit
            end
            echo "Note: 'quick' is equivalent to 'full' when using regular expressions." if attr == 'quick' or attr == 'q'
            echo "Warning: Modifiers in regular expressions are not supported.  Patterns are always case-insensitive." if options
        elsif attr == 'quick' or attr == 'q' then
            pattern = /(?:#{value.split(/\s*,\s*/).map{|x| "(?:#{Regexp::escape(x).gsub('\*', '.*')})"}.join('|')})/i
        else
            pattern = /^(?:#{value.split(/\s*,\s*/).map{|x| "(?:#{Regexp::escape(x).gsub('\*', '.*')})"}.join('|')})$/i
        end


        case attr
        when 'type', 't'
            if value =~ /^none|unknown$/i
                return proc {|item| item.type == '' or item.type.nil?}
            end
            unless GameObj.type_data.find{|k, v| k.split(',').find{|t| t =~ pattern}}
                echo "No item types match the pattern '#{value}'.  Did you perhaps mean '#{$lich_char}#{@script.name} #{value =~ /\s/ ? 'name' : 'noun'}=#{value} in #{target};...'?"
                echo "Type '#{$lich_char}#{@script.name}' by itself for more options and a list of item types."
                exit
            end
            return proc {|item| item.type.split(',').find{|t| t =~ pattern}}
        when 'name', 'm'
            return proc {|item| item.name =~ pattern}
        when 'fullname', 'f', 'quick', 'q'
            return proc {|item| item.full_name =~ pattern}
        when 'noun', 'n'
            return proc {|item| item.noun =~ pattern}
        else
            return nil
        end
    end

    def self.wait_block(message, target, valueproc, statusbar)
        if valueproc.call < target
            echo message
            statusbar.status_until(message, immediate: true){ wait_until { valueproc.call >= target } }
        end
    end

    def self.is_premium_locker?
        looking_for = Set.new(PREMIUM_LOCKER_CONTAINERS.keys)
        GameObj.loot.each{|item|
            if looking_for.delete?(item.full_name)
                if looking_for.empty?
                    return true
                end
            end
        }
        return false
    end

    def self.open_locker
        return if @opened_locker
        @opened_locker = true
        put "open locker"
        while (line = get)
            if line =~ /^Your locker is currently holding/
                before_dying { waitrt?; put "close locker" }
                return
            elsif line =~ /^That is already open|What were you referring to/
                return
            end
        end
    end

    def self.autocompletes_to(input, command)
        prefix, rest = command.split('|')
        command = prefix + rest
        return input.length >= prefix.length && command.start_with?(input)
    end

    def self.run(script)
        @opened_locker = false
        @stormfront = ($frontend == 'stormfront')
        @profanity = ($frontend == 'profanity')
        @script = script
        @script.downstream_buffer.instance_variable_set(:@max_size, 1000)
        @script.want_downstream = false
        @script.want_downstream_xml = true
        @script_was_paused = false

        self.first_time_setup(@script.vars[0] =~ /^\s*setup\s*$/i ? true : false)
        self.show_changelog

        @stopped_scripts = []
        before_dying {
            self.cleanup
            resume_scripts
        }

        @script.vars[0] =~ /^\s*(help|(?:.*?\b(?:in|on|under|behind)\s+.+?))\s*(?:([;\/|])\s*(.*?)\s*)?$/i
        filter = $1
        separator = $2
        commands = $3

        matcher = ItemMatcher.new

        # Read first/skip/unique bits.
        while filter =~ /^(?:(unique|reverse|reversed|nsort|nsorted|nounsort|nounsorted|sort|sorted|(?:un)?(?:mark|marked|register|registered))|(?:(?:(first|after|skip)\s+)?(\d+)))\s+(.*)/i
            filter = $4
            if $1
                case $1
                when 'unique'
                    if matcher.unique
                        echo "'unique' specified more than once."
                        exit
                    end
                    matcher.unique = true
                when 'reverse', 'reversed'
                    if matcher.reversed
                        echo "'reversed' specified more than once."
                        exit
                    end
                    matcher.reversed = true
                when 'sort', 'sorted'
                    if matcher.sorted
                        echo "'sorted' specified more than once."
                        exit
                    end
                    matcher.sorted = true
                when 'nsort', 'nsorted', 'nounsort', 'nounsorted'
                    if matcher.sorted
                        echo "'sorted' specified more than once."
                        exit
                    end
                    matcher.sorted = :noun
                else
                    opt = $1
                    value = !opt.start_with?("un")
                    unless value
                        opt = opt[2, opt.length]
                    end

                    case opt
                    when 'mark', 'marked'
                        if matcher.marked.nil?
                            matcher.marked = value
                        elsif matcher.marked == value
                            echo "'#{$1}' specified more than once."
                            exit
                        else
                            echo "Cannot specify both '#{opt}' and 'un#{opt}."
                            exit
                        end
                    when 'register', 'registered'
                        if matcher.registered.nil?
                            matcher.registered = value
                        elsif matcher.registered == value
                            echo "'#{$1}' specified more than once."
                            exit
                        else
                            echo "Cannot specify both '#{opt}' and 'un#{opt}."
                            exit
                        end
                    else
                        echo "Unknown boolean option '#{opt}'"
                        exit
                    end
                end
            elsif $3
                if not $2 or $2.downcase == 'first'
                    if matcher.first
                        echo "'first' specified more than once."
                        exit
                    end
                    matcher.first = $3.to_i
                else
                    if matcher.after
                        echo "'skip' specified more than once."
                        exit
                    end
                    matcher.after = $3.to_i
                end
            end
        end

        # Determine filter and container bits.
        unless filter =~ FILTER_PATTERN
            self.show_help((filter =~ /^help$/i) ? true : false)
            exit
        end

        filter_attr = ($2.downcase or 'type')
        filter_value = $3
        position = ($1 or $4)
        targets = $5
        explicit_all = false  # Safety bypass.

        if filter_attr == 'type' and filter_value =~ /all|any|everything/i
            explicit_all = true
            filter_attr = nil
            filter_value = nil
        end

        filter_proc = self.build_filter(filter_value, filter_attr, targets)
        unless filter_proc
            echo "Unknown attribute '#{filter_attr}'"
            exit
        end

        matcher.item_filter = filter_proc

        # Process commands
        lootsacks = nil

        if separator and commands
            commands = commands.split(/\s*#{Regexp::escape(separator)}\s*/)
            count = commands.length
            next_is_script = false
            new_commands = []
            commands.each_with_index{|command, ix|
                prefix = ''
                if command[0] == '!'
                    prefix = '!'
                    command = command[1..-1]
                end
                if command == ''  # Blank.
                    # Due to the split, a Lich script will appear as a blank command followed by the script invocation
                    # as the next command -- if the separator and $lich_char are the same.  Flag this for the next run
                    next_is_script = ($lich_char == separator)
                    # Continue to the next iteration rather than adding this command, too.
                    count -= 1
                    next
                end
                if next_is_script
                    next_is_script = false
                    # Convert to script invocation
                    new_commands << "#{prefix}#{$lich_char}#{command}"
                    next
                end

                command =~ /^(\w+)(.*)$/
                # Autocomplete command to simplify coding in other shortcuts
                first_word = $1.downcase
                rest = $2
                if first_word
                    %w(dr|op plac|e sel|l ap|praise reg|ister ge|t tak|e r|ead l|ook ana|lyze ins|pect).each{|verb|
                        if self.autocompletes_to(first_word, verb)
                            command = verb.gsub('|', '') + rest
                            break
                        end
                    }
                end

                if first_word == 'giveitem'
                    who = rest.strip.capitalize
                    if who == ''
                        echo "GIVEITEM requires a target character."
                        exit
                    end
                    if GameObj.pcs.find{|x| x.noun == who}
                        command = "giveitem #{who}"
                    else
                        matches = GameObj.pcs.find_all{|x| x.noun.start_with?(who)}
                        if matches.length == 0
                            echo "GIVEITEM: No PCs found matching '#{who}'."
                            exit
                        elsif matches.length > 1
                            echo "GIVEITEM: Multiple PCs found matching '#{who}': #{matches.map{|x| x.noun}.join(', ')}."
                            exit
                        else
                            who = matches[0].noun
                            command = "giveitem #{who}"
                        end
                    end
                end

                if command =~ /^(drop|place|sell|appraise|stash|register|mark|unmark|get|take|read|look|analyze|inspect)$/i
                    # Add implicit ITEM if no modifiers.
                    command = "#{$1} item"
                end
                if command =~ /^stash\s+/i and not lootsacks
                    # Adapted from ;loot, with heavy modifications
                    lootsacks = self.find_sacks('lootsack')
                    unless lootsacks.length > 0  # Already did the search
                        echo "Failed to find any lootsacks, STASH will not work."
                        echo "Configure lootsacks with ;vars set lootsack=container, ;vars set lootsack2=container, etc...."
                        exit
                    end
                end
                if ix == 0  # First command logic.
                    # Add implicit GETs when no modifiers.
                    if command =~ /^drop\s+item$/i
                        command = '_drag item drop'
                    elsif command =~ /^(?:giveitem)|(?:place|sell|(appraise|register|mark|unmark))\s+item$/i
                        implicit_return = $1 && (count == 1)
                        new_commands << prefix + '_remget item'  # Hackery
                        new_commands << prefix + command
                        new_commands << prefix + "return" if implicit_return
                        next
                        #
                        #
                        #
                        # implicit_return = command =~ /^(?:appraise|register|mark|unmark)\s+item/
                        # implicit_get = implicit_return || command =~ /^(giveitem)|^(?:(place|sell|app(?:raise)?)\s+item)$/i
                        # command =~ /^(giveitem)|^(?:(place|sell|app(?:raise)?)\s+item)$/i
                        # implicit_return = count == 1 && !$1
                        # if target =~ /^worn$/i
                        #     new_commands << 'remove item'
                        # else
                        #     new_commands << 'get item'
                        # end
                    end
                end
                new_commands << prefix + command
                # if commands.length == 1 and command =~ /^app(?:raise)?\s+item$/i
                #     new_commands << 'return'
                # end
            }
            commands = new_commands
        end
        commands = nil if commands and commands.length == 0

        # Find possible item IDs
        silence_me if $SAFE == 0

        seen_locker = seen_inv = false

        targets = targets.split(/\s*,\s*/)
        targets.uniq!
        targets.each do |target|
            target.strip!
            next if target == ''
            if target =~ /^(?:locker)$/i
                open_locker
            end

            if target =~ /^(?:fast|q)inv(?:entory)?$/i and matcher.status_filter != 0
                echo "Filtering by status requires a regulary inventory view, changing '#{target}' to 'inv'"
                target = "inv"
            end

            if target =~ /^inv(?:entory)?$/i  # Entire inventory
                next if seen_inv
                seen_inv = true
                # Sanity check
                if commands and filter_value.nil?
                    echo 'WARNING: THIS COMMAND WILL ACT ON EVERY ITEM THAT YOU HAVE IN YOUR ENTIRE INVENTORY!'
                    if explicit_all
                        echo "'#{$lich_char}kill #{@script.name}' to abort!"
                        sleep 1  # A moment before it all scrolls off
                        # echo "'#{$lich_char}unpause #{@script.name}' to continue, or '#{$lich_char}kill #{@script.name}' to abort"
                        # pause_script
                    else
                        echo "If you REALLY want to do this, you must explicitly say so by using '#{$lich_char}#{@script.name} ALL in #{target}';..."
                        exit
                    end
                end
                matcher.ingest_inventory
            elsif target =~ /^(?:fast|q)inv(?:entory)?$/i
                # Sanity check
                if commands and filter_value.nil?
                    echo 'WARNING: THIS COMMAND WILL ACT ON EVERY ITEM THAT YOU HAVE IN YOUR ENTIRE INVENTORY!'
                    if explicit_all
                        echo "'#{$lich_char}kill #{@script.name}' to abort!"
                        sleep 1  # A moment before it all scrolls off
                        # echo "'#{$lich_char}unpause #{@script.name}' to continue, or '#{$lich_char}kill #{@script.name}' to abort"
                        # pause_script
                    else
                        echo "If you REALLY want to do this, you must explicitly say so by using '#{$lich_char}#{@script.name} ALL in #{target}';..."
                        exit
                    end
                end
                GameObj.inv.each{|item|
                    if (items = GameObj.containers[item.id])
                        matcher.ingest_container(item.id, item.full_name, items)
                    end
                }
            elsif target =~ /^(?:prev(?:ious)?|last)$/i  # Previous run
                unless @previous_matcher
                    echo "No previous run data is available."
                    exit
                end
                if matcher.status_filter != 0
                    echo "Target 'previous' does not support filtering by mark or registration status."
                    exit
                end
                matcher.ingest_previous(@previous_matcher)
            elsif target =~ /^(?:loot)$/i  # Loot in room
                if matcher.status_filter != 0
                    echo "Target 'loot' does not support filtering by mark or registration status."
                    exit
                end
                matcher.ingest_loot
            elsif target =~ /^(?:desc)$/i  # Room description
                if matcher.status_filter != 0
                    echo "Target 'desc' does not support filtering by mark or registration status."
                    exit
                end
                matcher.ingest_desc
            elsif target =~ /^(?:locker)$/i and self.is_premium_locker?
                next if seen_locker
                seen_locker = true
                unless matcher.ingest_locker_from_manifest
                    if matcher.status_filter != 0
                        echo "I had to fall back to legacy locker scanning, which means I can't filter by mark or registration status."
                        echo "Most likely, this can be fixed by updating the MapDB for this locker.  Please contact LostRanger with the room number (if known), description, and location for details."
                        exit
                    else
                        echo "Locker manifest scanning failed, falling back to legacy scan."
                        # Fall back to legacy locker scan.
                        matcher.ingest_legacy_locker
                    end
                end
            elsif target =~ /^(?:floor|ground|room)$/i  # Items on ground
                if matcher.status_filter != 0
                    echo "Target '#{target}' does not support filtering by mark or registration status."
                    exit
                end
                matcher.ingest_ground
            elsif target =~ /^worn$/i  # Worn items
                matcher.ingest_worn
            else   # A specific container
                if target =~ /^locker$/i
                    if matcher.status_filter != 0
                        echo "Target 'locker' only supports filtering by mark or registration status when used on a premium locker."
                        exit
                    end
                    open_locker
                end
                matcher.ingest_from_name(target, position)
            end
        end

        unless matcher.finalize
            echo 'No matching items found!'
            exit
        end
        total_items = matcher.total_items

        @previous_matcher = matcher

        statusbar = nil
        if @stormfront && commands
            @running_instances ||= 0
            begin
                statusbar = StatusUpdater.new(script, @running_instances += 1)
                before_dying { @running_instances -= 1 }
            rescue
                @running_instances -= 1
                exit
            end
        else
            statusbar = DummyStatusUpdater.new
        end

        silence_me if $SAFE == 0

        before_next = nil  # Possibly skippable things to do before the next command.
        move_container_cache = {}  # Ids of discovered containers
        processed_items = 0
        immediate = nil
        matcher.container_contents.each{|container, items|
            container_name = matcher.container_names[container]
            # next unless items.length > 0
            # items = items.find_all{|item| filter.call(item)}
            # next unless items.length > 0
            if commands
                items.each{|item|
                    ref = "##{item.id}"
                    percent = (100.0 * processed_items / total_items).to_i
                    if processed_items % ITEM_UPDATE_INTERVAL == 0
                        echo "Item #{processed_items + 1} of #{total_items}.  (#{percent}% complete)"
                    end
                    statusbar.set(
                        status: "Running",
                        item: item,
                        items_processed: processed_items,
                        items_total: total_items,
                        percent: percent
                    ) if statusbar

                    commands.each_with_index{|original, ix|
                        command = (
                        original
                            .gsub(/\bitem\b/i, ref)
                            .gsub(/\bnoun\b/i, item.noun)
                            .gsub(/\bname\b/i, item.name)
                            .gsub(/\bcontainer\b/i, ("##{container}"))
                        )

                        if command[0] == '!'
                            command = command[1..-1]
                            immediate ||= 0
                            immediate += 1
                            if immediate > 5
                                immediate = nil
                            end
                        else
                            immediate = nil
                        end

                        if command =~ /^_remget (.*)/
                            if container == '_worn'
                                command = "remove #{$1}"
                            else
                                command = "get #{$1}"
                            end
                        end

                        if command.start_with?($lich_char)
                            before_next.call if before_next
                            before_next = proc { waitrt? }
                            scriptname, args = command.split(/\s+/, 2)
                            scriptname = scriptname[1..-1]
                            echo "Running #{$lich_char}#{scriptname} #{args}"
                            statusbar.status_until("Waiting on #{$lich_char}#{scriptname}"){ Script.run(scriptname, args) }
                        elsif command =~ COMMAND_PATTERNS[:move]
                            before_next.call if before_next
                            before_next = nil
                            fast = $1 ? true : false
                            what = ($2 or ref)
                            where = $3
                            holding = (what == ref and self.holding?(item.id))
                            floor = (where =~ /^(?:ground|floor)$/i) ? true : false
                            if what[0] == '#'
                                if floor
                                    zput("_drag #{what} drop", immediate)
                                    waitfor '<right>Empty', '<left>Empty' unless immediate
                                    next
                                end
                                open_locker if where =~ /^locker$/i  # Convenience
                                where = move_container_cache[where] if move_container_cache[where]
                                if where[0] == '#'
                                    zput("_drag #{what} #{where}", immediate)
                                    waitfor '<right>Empty', '<left>Empty' unless immediate
                                    next
                                end
                            end
                            get_command = "get #{what}"
                            put_command = floor ? "place #{what}" : "put #{what} in #{where}"

                            unless holding
                                zput(get_command, fast || immediate)
                            end
                            zput(put_command, fast || immediate)
                            waitfor '<right>', '<left>'
                            next if floor

                            unless where[0] == '#' or move_container_cache.include?(where)
                                nil until get =~ /^(?:(?:<.*>)?You (?:put|place|absent-mindedly drop|tuck|pop) (?:[^<]*)<a exist="(-?\d+).*?<\/a>.*<a exist="(-?\d+).*?<\/a)|<prompt/
                                # move_container_cache[where] = ($2 and (GameObj.inv.find{|x| x.id == $2} || GameObj.loot.find{|x| x.id == $2})) ? "##{$2}" : false
                                move_container_cache[where] = ($2 ? "##{$2}" : false)
                            end
                        elsif command =~ /^wait(?:mana|mp) (-?\d+)$/
                            before_next.call if before_next
                            before_next = nil
                            n = $1.to_i
                            self.wait_block("Waiting for #{n} mana", n, proc { XMLData.mana }, statusbar)
                        elsif command =~ /^wait(?:health|hp) (-?\d+)$/
                            before_next.call if before_next
                            before_next = nil
                            n = $1.to_i
                            self.wait_block("Waiting for #{n} HP", n, proc { XMLData.health }, statusbar)
                        elsif command =~ /^wait(?:spirit|sp) (-?\d+)$/
                            before_next.call if before_next
                            before_next = nil
                            n = $1.to_i
                            self.wait_block("Waiting for #{n} spirit", n, proc { XMLData.spirit }, statusbar)
                        elsif command =~ /^wait(?:stamina|st) (-?\d+)$/
                            before_next.call if before_next
                            before_next = nil
                            n = $1.to_i
                            self.wait_block("Waiting for #{n} stamina", n, proc { XMLData.stamina }, statusbar)
                        elsif command =~ /^stash (.*$)/i
                            before_next.call if before_next
                            before_next = nil
                            what = $1
                            success = false
                            lootsacks.length.times{
                                sack = lootsacks[0]
                                next unless GameObj.inv.find{|x| x.id == sack.id}
                                where = "##{sack.id}"
                                if what[0] == '#' and ix == 0
                                    zput("_drag #{what} #{where}", immediate)
                                else
                                    zput("get #{what}", immediate) if ix == 0
                                    immediate = false
                                    fput "put #{what} in #{where}"
                                end
                                nil until get =~ /^Your .* won't fit in .*|<(left|right)>Empty</ unless immediate
                                waitfor '<prompt' unless immediate
                                if $1
                                    success = true
                                    break
                                end
                                lootsacks << lootsacks.shift
                            }
                            next if success
                            echo "All lootsacks are full, pausing script."
                            echo "#{$lich_char}unpause #{@script.name} to continue anyways, or abort with #{$lich_char}kill #{@script.name}"
                            pause_script
                        elsif command =~ /^echo (.*)$/i
                            before_next.call if before_next
                            before_next = nil
                            echo $1
                        elsif command =~ /^waitrt$/i
                            before_next = nil
                            statusbar.status_until("Waiting for RT"){waitrt}
                        elsif command =~ /^waitrt\?$/i
                            before_next = nil
                            statusbar.status_until("Waiting for RT", checkrt > 0){waitrt?}
                        elsif command =~ /^waitcastrt$/i
                            before_next = nil
                            statusbar.status_until("Waiting for Cast RT"){waitcastrt}
                        elsif command =~ /^waitcastrt\?$/i
                            before_next = nil
                            statusbar.status_until("Waiting for Cast RT", checkcastrt){waitcastrt?}
                        elsif command =~ /^waitfor (.*)$/i
                            script.want_downstream = true
                            script.want_downstream_xml = false
                            statusbar.status_until("Waiting"){waitfor $1}
                            script.want_downstream_xml = true
                            script.want_downstream = false
                            before_next = proc { statusbar.status_until("Waiting for RT", checkrt > 0){waitrt?} }
                        elsif command =~ /^waitre (.*)$/i
                            text = $1
                            flags = nil
                            if text =~ /\/(.+)\/(.*)$/
                                text = $1
                                flags = $2 unless $2 == ''
                            end
                            pattern = Regexp::new(text, flags)
                            statusbar.status_until("Waiting"){ nil until get =~ pattern }
                            before_next = proc { statusbar.status_until("Waiting for RT", checkrt > 0){waitrt?} }
                        elsif command =~ /^pause$/i
                            percent = (100.0 * processed_items / total_items).to_i
                            msg = []
                            msg << '<output class="mono" />' if $stormfront
                            msg << "[#{@script.name}: Pausing at #{self.item_detail(item)},  (#{processed_items + 1} of #{total_items}, #{percent}% complete)."
                            msg << '<output class="" />' if $stormfront
                            puts msg.join("\n")
                            statusbar.status_until("Paused"){ pause_script }
                            before_next = proc { waitrt? }
                        elsif command =~ /^giveitem(?:\s+(?:to\s+)?(.+))?$/i
                            before_next.call if before_next
                            before_next = nil
                            unless $1
                                echo 'Give to whom?'
                                exit
                            end
                            zput("give ##{item.id} to #{$1}", immediate)
                            statusbar.status_until("Waiting for #{$1}"){ waitforre /#{$1}<\/a> has accepted your offer/ }
                        elsif command =~ /^return(?:\s+item)?$/i
                            before_next.call if before_next
                            before_next = nil
                            if container == '_worn'
                                zput("wear #{ref}", immediate)
                            elsif container == '_ground'
                                zput("place #{ref}", immediate)
                            else
                                zput("put #{ref} in ##{container}", immediate)
                            end
                        elsif command =~ /^sleep (\d*\.?\d+)$/i
                            statusbar.status_until("Sleeping") { sleep $1.to_f }
                        elsif command =~ /^unmark\s+(.*)/i
                            before_next.call if before_next
                            before_next = nil
                            fput "mark #{$1} remove"
                        else
                            before_next.call if before_next
                            before_next = nil
                            zput(command, immediate)
                            before_next = proc { waitfor '<prompt'; statusbar.status_until("Waiting for RT", checkrt > 0){waitrt?} } unless immediate
                        end
                    }
                    processed_items += 1
                }
            else
                msg = ['']
                header = "[#{container_name}]: "
                if @stormfront
                    header = REXML::Text.new(header).to_s
                    msg << "<preset id=\"whisper\">#{header}</preset>"
                else
                    header = REXML::Text.new(header).to_s if @profanity
                    msg << monsterbold_start + header + monsterbold_end
                end
                items.each{|item|
                    msg << item_detail(item)
                }
                puts msg.join("\n")
            end
        }
        unless commands.length > 0
            puts "\nTotal items: #{total_items}"
        end
    end

    def self.item_detail(item)
        if @stormfront
            noun = REXML::Text.new(item.noun).to_s
            text = REXML::Text.new("#{item.full_name} (#{item.type})").to_s
            return "<a exist=\"#{item.id}\" noun=\"#{noun}\">##{item.id}</a> #{text}"
        else
            text = "##{item.id} #{item.full_name} (#{item.type})"
            return REXML::Text.new(text).to_s if @profanity
            return text
        end
    end
end

if defined?(GameObj.type_data)
    GameObj.load_data if GameObj.type_data.empty? or GameObj.type_data.nil?
    if GameObj.type_data.nil?
        wait_while { running?('updater') }
        wait_while { running?('repository') }
        start_script 'repository', [ 'download', 'gameobj-data.xml' ]
        wait_while { running?('repository') }
        wait_while { running?('updater') }
        start_script 'updater', [ 'add', 'gameobj-data.xml' ]
        wait_while { running?('updater') }
        GameObj.load_data
        exit if GameObj.type_data.nil?
    end
end

if EXPERIMENTAL_EXPIRES
    if EXPERIMENTAL_EXPIRES > Time.now
        echo "** This is an EXPERIMENTAL foreach build.  It should be updated by #{EXPERIMENTAL_EXPIRES}."
    else
        echo "** This is an OUTDATED EXPERIMENTAL foreach build.  It should probably have been updated by now.  It expired at#{EXPERIMENTAL_EXPIRES}."
        echo "You can still use it, but you may encounter bugs and minor annoyances."
        sleep 2  # Minor annoyances.
    end
end

ForeachScript.run(script)