Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Docker Compose tests for Feast 0.5 #725

Merged
merged 2 commits into from
May 21, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Update Docker Compose tests to Feast 0.5
  • Loading branch information
woop committed May 21, 2020
commit d6faa779d0c30756d6b119b9e40ca4aea754e2e2
3 changes: 2 additions & 1 deletion infra/docker-compose/.env.sample
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
# General
COMPOSE_PROJECT_NAME=feast
FEAST_VERSION=latest
FEAST_REPOSITORY_VERSION=v0.4.7
FEAST_REPOSITORY_VERSION=v0.5-branch

# Feast Core
FEAST_CORE_IMAGE=gcr.io/kf-feast/feast-core
FEAST_CORE_CONFIG=direct-runner.yml
FEAST_CORE_GCP_SERVICE_ACCOUNT_KEY=placeholder.json
FEAST_STREAM_OPTIONS_BOOTSTRAPSERVERS=kafka:9092

# Feast Serving Batch (BigQuery)
FEAST_BATCH_SERVING_CONFIG=batch-serving.yml
Expand Down
3 changes: 0 additions & 3 deletions infra/docker-compose/core/direct-runner.yml

This file was deleted.

1 change: 1 addition & 0 deletions infra/docker-compose/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ services:
environment:
DB_HOST: db
GOOGLE_APPLICATION_CREDENTIALS: /etc/gcloud/service-accounts/key.json
FEAST_STREAM_OPTIONS_BOOTSTRAPSERVERS: ${FEAST_STREAM_OPTIONS_BOOTSTRAPSERVERS}
restart: on-failure
depends_on:
- db
Expand Down
3 changes: 3 additions & 0 deletions infra/docker-compose/jupyter/startup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ set -ex
# Clone Feast repository into Jupyter container
git clone -b ${FEAST_REPOSITORY_VERSION} --single-branch https://github.com/feast-dev/feast.git || true

# Install Python dependencies
make -C feast/ compile-protos-python

# Install CI requirements (only needed for running tests)
pip install -r feast/sdk/python/requirements-ci.txt

Expand Down
21 changes: 10 additions & 11 deletions infra/docker-compose/serving/online-serving.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
feast:
core-host: core
core-grpc-port: 6565
store:
config-path: /etc/feast/store.yml
jobs:
store-type: REDIS
store-options:
host: redis
port: 6379

grpc:
port: 6566
active_store: online
stores:
- name: online
type: REDIS
config:
host: redis
port: 6379
subscriptions:
- name: "*"
project: "*"
17 changes: 14 additions & 3 deletions infra/scripts/test-docker-compose.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,20 @@ Running Docker Compose tests with pytest at 'tests/e2e'
============================================================
"

clean_up () {
ARG=$?

# Shut down docker-compose images
docker-compose -f docker-compose.yml -f docker-compose.online.yml down

# Remove configuration file
rm .env

exit $ARG
}

trap clean_up EXIT

export PROJECT_ROOT_DIR=$(git rev-parse --show-toplevel)
export COMPOSE_INTERACTIVE_NO_CLI=1

Expand All @@ -30,6 +44,3 @@ ${PROJECT_ROOT_DIR}/infra/scripts/wait-for-it.sh ${JUPYTER_DOCKER_CONTAINER_IP_A

# Run e2e tests for Redis
docker exec feast_jupyter_1 bash -c 'cd feast/tests/e2e/ && pytest -s basic-ingest-redis-serving.py --core_url core:6565 --serving_url=online-serving:6566'

# Shut down docker-compose images
docker-compose -f docker-compose.yml -f docker-compose.online.yml down