Manage history-archive http server with supervisor #507
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tsachiherman
approved these changes
Oct 9, 2023
| @@ -233,6 +233,7 @@ function copy_defaults() { | |||
| cp /opt/stellar-default/$NETWORK/supervisor/etc/supervisord.conf.d/soroban-rpc.conf $SUPHOME/etc/supervisord.conf.d 2>/dev/null || true | |||
| fi | |||
| cp /opt/stellar-default/$NETWORK/supervisor/etc/supervisord.conf.d/friendbot.conf $SUPHOME/etc/supervisord.conf.d 2>/dev/null || true | |||
| cp /opt/stellar-default/$NETWORK/supervisor/etc/supervisord.conf.d/history-archive.conf $SUPHOME/etc/supervisord.conf.d 2>/dev/null || true | |||
There was a problem hiding this comment.
Consider replacing cp with install, as it's working around files in use.
There was a problem hiding this comment.
What do you mean by "install"? I'm not familiar with that.
There was a problem hiding this comment.
Going to merge because I'm not sure how to change to doing an install, and the line I'm adding here is consistent with how existing services confs get added. Let me know how to do an install and happy to follow up in another PR.
There was a problem hiding this comment.
the linux install command is similar to cp, but act as
rm dest- then,
cp src dest
The main advantage in using it is to avoid the potential failure of the cp command in case the file is being used.
( there are other benefits, but this is the important one, imho )
There was a problem hiding this comment.
TIL. It also sounds like the other main reason to use install is that it removes the hard link meaning the copy will only replace the file at that one path, not files at all paths sharing the hardlink.
It's very unlikely the file is in use, and equally unlikely the file would have multiple hard links, so I don't think using install will have a material difference.
A downside of install is similar to sed, it's behavior is different on GNU vs BSD linux, so fun times if we encourage use and it ends up in scripts run on Linux and Mac.
chadoh
pushed a commit
to AhaLabs/stellar-cli
that referenced
this pull request
Oct 16, 2023
What ---- Update the typescript bindings with all the changes that landed in the futurenet-compatible `smartdeploy` branch since `main` stopped working with futurenet. Why --- Lots of good improvements and fixes are not yet in `main`, ~including tests for the generated libraries.~ _CI tests will be included in a future PR; reasonable time-sink exceeded for this one. Closes: stellar#996 Known limitations ----------------- The newly-added tests are not yet being exercised in CI; you can see the commit history for all the details of how we tried to work around this. At this point, we will: - wait for stellar/quickstart#507 to get merged & included in a published quickstart image - retry the `bindings-ts.yml` workflow, which was almost working except that friendbot was not starting properly when quickstart was run _in CI_ (though it does start properly, even if a bit slowly, when quickstart is run locally) - after a basic run-tests-with-published-quickstart is working, the next step is to build a custom quickstart image using _this repository's_ RPC version, and use that custom quickstart to run the integration tests. This might be able to be accomplished [as described by ChatGPT here](https://chat.openai.com/share/0f355fbe-93b9-4c8d-9c12-7f54846134bd).
chadoh
pushed a commit
to AhaLabs/stellar-cli
that referenced
this pull request
Oct 16, 2023
What ---- Update the typescript bindings with all the changes that landed in the futurenet-compatible `smartdeploy` branch since `main` stopped working with futurenet. Why --- Lots of good improvements and fixes are not yet in `main`, ~including tests for the generated libraries.~ _CI tests will be included in a future PR; reasonable time-sink exceeded for this one. Closes: stellar#996 Known limitations ----------------- The newly-added tests are not yet being exercised in CI; you can see the commit history for all the details of how we tried to work around this. At this point, we will: - wait for stellar/quickstart#507 to get merged & included in a published quickstart image - retry the `bindings-ts.yml` workflow, which was almost working except that friendbot was not starting properly when quickstart was run _in CI_ (though it does start properly, even if a bit slowly, when quickstart is run locally) - after a basic run-tests-with-published-quickstart is working, the next step is to build a custom quickstart image using _this repository's_ RPC version, and use that custom quickstart to run the integration tests. This might be able to be accomplished [as described by ChatGPT here](https://chat.openai.com/share/0f355fbe-93b9-4c8d-9c12-7f54846134bd).
chadoh
pushed a commit
to AhaLabs/stellar-cli
that referenced
this pull request
Oct 16, 2023
What ---- Update the typescript bindings with all the changes that landed in the futurenet-compatible `smartdeploy` branch since `main` stopped working with futurenet. Why --- Lots of good improvements and fixes are not yet in `main`, ~including tests for the generated libraries.~ _CI tests will be included in a future PR; reasonable time-sink exceeded for this one. Closes: stellar#996 Known limitations ----------------- The newly-added tests are not yet being exercised in CI; you can see the commit history for all the details of how we tried to work around this. At this point, we will: - wait for stellar/quickstart#507 to get merged & included in a published quickstart image - retry the `bindings-ts.yml` workflow, which was almost working except that friendbot was not starting properly when quickstart was run _in CI_ (though it does start properly, even if a bit slowly, when quickstart is run locally) - after a basic run-tests-with-published-quickstart is working, the next step is to build a custom quickstart image using _this repository's_ RPC version, and use that custom quickstart to run the integration tests. This might be able to be accomplished [as described by ChatGPT here](https://chat.openai.com/share/0f355fbe-93b9-4c8d-9c12-7f54846134bd).
tsachiherman
added a commit
to stellar/stellar-cli
that referenced
this pull request
Oct 17, 2023
* chore: update ts bindings What ---- Update the typescript bindings with all the changes that landed in the futurenet-compatible `smartdeploy` branch since `main` stopped working with futurenet. Why --- Lots of good improvements and fixes are not yet in `main`, ~including tests for the generated libraries.~ _CI tests will be included in a future PR; reasonable time-sink exceeded for this one. Closes: #996 Known limitations ----------------- The newly-added tests are not yet being exercised in CI; you can see the commit history for all the details of how we tried to work around this. At this point, we will: - wait for stellar/quickstart#507 to get merged & included in a published quickstart image - retry the `bindings-ts.yml` workflow, which was almost working except that friendbot was not starting properly when quickstart was run _in CI_ (though it does start properly, even if a bit slowly, when quickstart is run locally) - after a basic run-tests-with-published-quickstart is working, the next step is to build a custom quickstart image using _this repository's_ RPC version, and use that custom quickstart to run the integration tests. This might be able to be accomplished [as described by ChatGPT here](https://chat.openai.com/share/0f355fbe-93b9-4c8d-9c12-7f54846134bd). * build: bump delta for flaky test * build: increase deltas in cli simulation test --------- Co-authored-by: Willem Wyndham <willem@ahalabs.dev> Co-authored-by: Tsachi Herman <24438559+tsachiherman@users.noreply.github.com>
chadoh
added a commit
to AhaLabs/stellar-cli
that referenced
this pull request
Oct 19, 2023
* chore: update ts bindings What ---- Update the typescript bindings with all the changes that landed in the futurenet-compatible `smartdeploy` branch since `main` stopped working with futurenet. Why --- Lots of good improvements and fixes are not yet in `main`, ~including tests for the generated libraries.~ _CI tests will be included in a future PR; reasonable time-sink exceeded for this one. Closes: stellar#996 Known limitations ----------------- The newly-added tests are not yet being exercised in CI; you can see the commit history for all the details of how we tried to work around this. At this point, we will: - wait for stellar/quickstart#507 to get merged & included in a published quickstart image - retry the `bindings-ts.yml` workflow, which was almost working except that friendbot was not starting properly when quickstart was run _in CI_ (though it does start properly, even if a bit slowly, when quickstart is run locally) - after a basic run-tests-with-published-quickstart is working, the next step is to build a custom quickstart image using _this repository's_ RPC version, and use that custom quickstart to run the integration tests. This might be able to be accomplished [as described by ChatGPT here](https://chat.openai.com/share/0f355fbe-93b9-4c8d-9c12-7f54846134bd). * build: bump delta for flaky test * build: increase deltas in cli simulation test --------- Co-authored-by: Willem Wyndham <willem@ahalabs.dev> Co-authored-by: Tsachi Herman <24438559+tsachiherman@users.noreply.github.com>
tsachiherman
added a commit
to stellar/stellar-cli
that referenced
this pull request
Oct 20, 2023
* chore: update ts bindings What ---- Update the typescript bindings with all the changes that landed in the futurenet-compatible `smartdeploy` branch since `main` stopped working with futurenet. Why --- Lots of good improvements and fixes are not yet in `main`, ~including tests for the generated libraries.~ _CI tests will be included in a future PR; reasonable time-sink exceeded for this one. Closes: #996 Known limitations ----------------- The newly-added tests are not yet being exercised in CI; you can see the commit history for all the details of how we tried to work around this. At this point, we will: - wait for stellar/quickstart#507 to get merged & included in a published quickstart image - retry the `bindings-ts.yml` workflow, which was almost working except that friendbot was not starting properly when quickstart was run _in CI_ (though it does start properly, even if a bit slowly, when quickstart is run locally) - after a basic run-tests-with-published-quickstart is working, the next step is to build a custom quickstart image using _this repository's_ RPC version, and use that custom quickstart to run the integration tests. This might be able to be accomplished [as described by ChatGPT here](https://chat.openai.com/share/0f355fbe-93b9-4c8d-9c12-7f54846134bd). * build: bump delta for flaky test * build: increase deltas in cli simulation test --------- Co-authored-by: Willem Wyndham <willem@ahalabs.dev> Co-authored-by: Tsachi Herman <24438559+tsachiherman@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
Remove the python3 http server that is started as a background job of the main process and replace it with running the same http server with supervisor.
Why
The history archive http server is a critical component of the quickstart image when run in local mode as it serves the history archives for horizon, soroban-rpc.
Today the history archive http server is run as a background job from the start script, and its logs are sent to /dev/null. This gives us no visibility into what its doing, if its still running, and if it encountered any issues, what they are.
We use supervisor to run long running services in the quickstart image and there's no reason I can see why we wouldn't also use it for the history archive http server.
The reason I care about this today is there was a case where someone was experiencing issues and the logs appeared as if the history archive wasn't available. Because the history archive wasn't managed by supervisor there was no way to debug if it was still running or what it was doing.