Skip to content

Commit

Permalink
Accelerate dev.up by disable program caching (openedx#491)
Browse files Browse the repository at this point in the history
By default, program caching is executed every time a user runs `make
dev.up`. This is prohibitive and annoying for most developers who are
not working on programs. This change makes programs caching opt-out by
default. To launch the platform with programs caching, run:

    make dev.up.<service> dev.cache-programs

or:

    make dev.up.with-programs.<service>

or:

    make dev.up.with-programs
  • Loading branch information
regisb authored Mar 11, 2020
1 parent cde3fb0 commit 8ef7ef2
Show file tree
Hide file tree
Showing 3 changed files with 15 additions and 14 deletions.
4 changes: 2 additions & 2 deletions .travis/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ if [[ "$DEVSTACK" == "analytics_pipeline" ]]; then
elif [[ "$DEVSTACK" == "lms" ]]; then
make dev.pull.discovery dev.pull.forum
make dev.provision.services.lms+discovery+forum
make no_cache=True dev.up.lms
make dev.up.lms
sleep 60 # LMS needs like 60 seconds to come up
make healthchecks.lms healthchecks.discovery validate-lms-volume
make up-marketing-detached
Expand All @@ -23,7 +23,7 @@ else
make dev.provision.services.lms
esac
make dev.provision.services."$DEVSTACK"
make no_cache=True dev.up."$DEVSTACK"
make dev.up."$DEVSTACK"
sleep 60
make healthchecks."$DEVSTACK"
fi
17 changes: 7 additions & 10 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,9 @@ dev.provision.services.%: ## Provision specified services with local mounted dir

dev.provision: | check-memory dev.clone.ssh dev.provision.services stop ## Provision dev environment with all services stopped

dev.cache-programs: ## Copy programs from Discovery to Memcached for use in LMS.
$(WINPTY) bash ./programs/provision.sh cache

dev.provision.xqueue: | check-memory dev.provision.xqueue.run stop stop.xqueue # Provision XQueue; run after other services are provisioned

dev.provision.xqueue.run:
Expand All @@ -108,17 +111,13 @@ dev.pull.%: ## Pull latest Docker images for a given service and all its depende

dev.up: | check-memory ## Bring up all services with host volumes
bash -c 'docker-compose $(STANDARD_COMPOSE_FILES) up -d'
@# Comment out this next line if you want to save some time and don't care about catalog programs
$(WINPTY) bash ./programs/provision.sh cache

dev.up.%: | check-memory ## Bring up a specific service and its dependencies with host volumes
bash -c 'docker-compose $(STANDARD_COMPOSE_FILES) up -d $*'
@# Comment out this next line if you want to save some time and don't care about catalog programs
ifdef no_cache
echo "Not runing bash ./programs/provision.sh cache"
else
$(WINPTY) bash ./programs/provision.sh cache
endif

dev.up.with-programs: dev.up dev.cache-programs ## Bring up a all services and cache programs in LMS.

dev.up.with-programs.%: dev.up.$* dev.cache-programs ## Bring up a service and its dependencies and cache programs in LMS.

dev.up.watchers: | check-memory ## Bring up asset watcher containers
bash -c 'docker-compose -f docker-compose-watchers.yml up -d'
Expand All @@ -131,8 +130,6 @@ dev.nfs.up.watchers: | check-memory ## Bring up asset watcher containers

dev.nfs.up: | check-memory ## Bring up all services with host volumes
docker-compose $(NFS_COMPOSE_FILES) up -d
@# Comment out this next line if you want to save some time and don't care about catalog programs
#./programs/provision.sh cache >/dev/null

dev.nfs.up.all: | dev.nfs.up dev.nfs.up.watchers ## Bring up all services with host volumes, including watchers

Expand Down
8 changes: 6 additions & 2 deletions programs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,16 @@ Normally you don't need to manually run these scripts to provision the demo cour

If you have an existing older devstack installation and want to add the demo program to it, simply run:

``./programs/provision.sh``
./programs/provision.sh

And it will set it up for you. This can be run mutiple times safely.

## Recaching

If you have an existing devstack with the demo program, but want to recache the programs inside LMS (something you need to do every time you bring the LMS container back up), simply run:

``./programs/provision.sh cache``
make dev.cache-programs

To launch a service and recache the programs, run instead:

make dev.up.<service> dev.cache-programs

0 comments on commit 8ef7ef2

Please sign in to comment.