Want to run fully locally using OLLAMA and SEARXNG

[Shyamnath-Sankar]

can anyone please guide me to create this fully locally in my pc

llm as OLLAMA CLIENT
web search searXNG

i will be very must greatful if you help me through

[Yucheng-Jiang]

Hi, have you tried the example script here: https://github.com/stanford-oval/storm/blob/main/examples/storm_examples/run_storm_wiki_ollama_with_searxng.py

[cl93a]

What is exactly the searxng_api_url of Searxng? Is it just the root domain, or /search?q= ? Because mine results in unsuccessful outcomes:

python run_storm_wiki_ollama_with_searxng.py --searxng-api-url http://localhost:MY_PORT --do-research --do-generate-outline
Topic: dragons
root : ERROR : Error occurs when searching query : Expecting value: line 1 column 1 (char 0)

Searxng works elsewhere (e.g. Open-Webui). Any ideas what might be wrong?

[gsoul]

python examples/storm_examples/run_storm_wiki_ollama_with_searxng.py --model=hf.co/bartowski/Llama-3.3-70B-Instruct-GGUF:IQ4_XS --searxng-api-url=http://localhost:MY_PORT/search? --do-research --do-generate-outline --do-generate-article --do-polish-article --retriever=searxng

This seems to work for me ~50% of the time. And gives me errors like this the other half:

Error: CODE 500 - {"error":"POST predict: Post \"http://127.0.0.1:33643/completion\": EOF"}
root : ERROR    : Error occurs when generating answer: can only join an iterable
Error: CODE 500 - {"error":"POST predict: Post \"http://127.0.0.1:33643/completion\": EOF"}
Error: CODE 500 - {"error":"an unknown error was encountered while running the model "}
Error: CODE 500 - {"error":"POST predict: Post \"http://127.0.0.1:45537/completion\": EOF"}
root : ERROR    : Error occurs when generating answer: can only join an iterable
Traceback (most recent call last):
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/urllib3/connectionpool.py", line 534, in _make_request
    response = conn.getresponse()
               ^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/urllib3/connection.py", line 516, in getresponse
    httplib_response = super().getresponse()
                       ^^^^^^^^^^^^^^^^^^^^^
  File ".../anaconda3/lib/python3.11/http/client.py", line 1386, in getresponse
    response.begin()
  File ".../anaconda3/lib/python3.11/http/client.py", line 325, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File ".../anaconda3/lib/python3.11/http/client.py", line 286, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../anaconda3/lib/python3.11/socket.py", line 706, in readinto
    return self._sock.recv_into(b)
           ^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/urllib3/util/retry.py", line 474, in increment
    raise reraise(type(error), error, _stacktrace)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/urllib3/util/util.py", line 39, in reraise
    raise value
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/urllib3/connectionpool.py", line 536, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=11434): Read timed out. (read timeout=120)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../soft/storm/examples/storm_examples/run_storm_wiki_ollama_with_searxng.py", line 154, in <module>
    main(parser.parse_args())
  File ".../soft/storm/examples/storm_examples/run_storm_wiki_ollama_with_searxng.py", line 101, in main
    runner.run(
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/knowledge_storm/storm_wiki/engine.py", line 394, in run
    information_table = self.run_knowledge_curation_module(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/knowledge_storm/interface.py", line 499, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/knowledge_storm/storm_wiki/engine.py", line 219, in run_knowledge_curation_module
    self.storm_knowledge_curation_module.research(
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/knowledge_storm/storm_wiki/modules/knowledge_curation.py", line 379, in research
    conversations = self._run_conversation(
                    ^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/knowledge_storm/storm_wiki/modules/knowledge_curation.py", line 340, in _run_conversation
    conv = future.result()
           ^^^^^^^^^^^^^^^
  File ".../anaconda3/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File ".../anaconda3/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File ".../anaconda3/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/knowledge_storm/storm_wiki/modules/knowledge_curation.py", line 318, in run_conv
    return conv_simulator(
           ^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/dspy/primitives/program.py", line 26, in __call__
    return self.forward(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/knowledge_storm/storm_wiki/modules/knowledge_curation.py", line 61, in forward
    user_utterance = self.wiki_writer(
                     ^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/dspy/primitives/program.py", line 26, in __call__
    return self.forward(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/knowledge_storm/storm_wiki/modules/knowledge_curation.py", line 117, in forward
    question = self.ask_question_with_persona(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/dspy/predict/predict.py", line 61, in __call__
    return self.forward(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/dspy/predict/chain_of_thought.py", line 59, in forward
    return super().forward(signature=signature, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/dspy/predict/predict.py", line 103, in forward
    x, C = dsp.generate(template, **config)(x, stage=self.stage)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/dsp/primitives/predict.py", line 77, in do_generate
    completions: list[dict[str, Any]] = generator(prompt, **kwargs)
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/dsp/modules/ollama.py", line 171, in __call__
    response = self.request(prompt, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/dsp/modules/ollama.py", line 145, in request
    return self.basic_request(prompt, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/dsp/modules/ollama.py", line 97, in basic_request
    response = requests.post(urlstr, json=settings_dict, timeout=self.timeout_s)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/requests/adapters.py", line 713, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='localhost', port=11434): Read timed out. (read timeout=120)

[gsoul]

ok, my bug seem to be fixed by updating ollama. The url for searxng is "http://localhost:MY_PORT/search? "