Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multinode-HA Vespa Setup for Local Testing #1071

Merged
merged 56 commits into from
Feb 25, 2025

Conversation

vicilliar
Copy link
Contributor

@vicilliar vicilliar commented Dec 16, 2024

  • What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
    Testing improvement

  • What is the current behavior? (You can also link to an open issue here)
    Current vespa setup only uses a single node.

  • What is the new behavior (if this is a feature change)?
    We implement a multinode setup for local vespa, so we can simulate cloud shards and replicas.
    vespa_local.py start function now accepts --Shards and --Replicas as parameters. If Shards > 1 or Replicas > 0, multinode vespa setup is used. Multinode vespa setup has 3 config server nodes, max(2, total_content_nodes / 4) API nodes, and shards * (1 + replicas) content nodes.
    Unit test github workflow now accepts shards and replicas as parameters.
    Orchestrator workflow was created, which runs 4 unit tests setups:
    (1) 0 replicas, 1 shard
    (2) 1 replica, 1 shard
    (3) 0 replicas, 2 shards
    (4) 1 replica, 2 shards

Unit tests on multinode vespa will ignore the following directories: tests/integ_tests/core/inference, tests/integ_tests/processing, tests/integ_tests/s2_inference

Multinode vespa tests will use m6i.2xlarge instead of m6i.xlarge due to the higher memory usage from many vespa nodes. Config and API nodes are ~1gb and content nodes are ~500mb. A 9 node system (3 config, 2 API, 4 content) needs roughly 7gb for vespa alone.

  • Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)
    No

  • Have unit tests been run against this PR? (Has there also been any additional testing?)
    In progress

  • Related Python client changes (link commit/PR here)

  • Related documentation changes (link commit/PR here)

  • Other information:

  • Please check if the PR fulfills these requirements

  • The commit message follows our guidelines
  • Tests for the changes have been added (for bug fixes/features)
  • Docs have been added / updated (for bug fixes / features)

papa99do
papa99do previously approved these changes Feb 23, 2025
@vicilliar vicilliar merged commit e556293 into mainline Feb 25, 2025
49 of 57 checks passed
@vicilliar vicilliar deleted the joshua/multi-shard-replica-vespa branch February 25, 2025 02:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants