-
Notifications
You must be signed in to change notification settings - Fork 81
WIP Produce the simplest-possible getting started guide for the Bisq HTTP API #54
Conversation
For posterity, here is the Slack conversation between @blabno and me (@cbeams) that led to this iteration of the doc. It describes the history and design motivations behind this kind of 'getting started guide' effort. From https://bisq.slack.com/archives/C75ABAU9J/p1526035993000053:
|
i.e. join artificially separated lines into a single line representing an entire paragraph. This approach is maximally conducive to effective editing and review, especially where `git diff --word-diff` is used (as seen in the output of diffs in the GitHub web UI).
Previously, the instructions would spin up an API instance that would attempt to use the default app name and app data directory, meaning that it would run into filesystem locks with an existing local Bisq app if one is running. We should assume that users interested in using the Bisq HTTP API are already running Bisq locally, and in order to avoid this kind of collision, and also to just generally play it safe, we should instruct the user to spin up their instance in a throwaway data directory. I've used the name 'bisqmon' here as in, a "Bisq monitor bot". This commit does *not* update the `docker run` invocation in the same way, but it should be updated to match.
Previously, the user was instructed to verify the API is up and running with the 'offers' endpoint and was then instructed in the next section to invoke the same endpoint. This change makes it easy to see that the API is up and running by returning a much shorter and easier to read response, and has the added side benefit of informing the user which version of Bisq they're running. The user then moves on in the next step to explore the 'offers' endpoint, which they're now more ready to do.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NACK. This is already getting close to the mark, @blabno. I got excited walking through this guide myself. As I read and worked through it with my 'new user' hat on, I started to get a sense of the power that this API could afford me, and the possibilities started spinning. Good stuff. We can make it even tighter now, so that it's virtually all signal, no noise, gets the reader in and out quickly with working code at the end, and hungry to do more afterward.
You'll see I've done a fair bit of editing already in my review commits. Please do go through each one, read the commit comment and study the diff so we can align on common practices, style, etc.
I've left a number of review comments below as well, with checkboxes for convenience.
Thanks. I'm stoked about getting this out there for people. And what's great about the way we're doing it is that we can publish this doc literally as soon as this editing process is finished. We don't need to integrate anything into the canonical bisq-*
repositories, etc. We can get it out there for people to allow them to try this stuff out right now, with minimum fuss, such that they can provide provide maximum feedback before we attempt to roll it all up into Bisq proper.
http-api-monitor-offers.adoc
Outdated
@@ -0,0 +1,327 @@ | |||
= Monitoring Bisq offers with Http API | |||
|
|||
This guide walks you through the process of creating a bot that monitors available offers and sends email notifications. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd like to drop the email notifications. It makes sense for a script you'd actually want to run in real life, but here in a "let's build the simplest possible bot that could actually work for the purposes of getting our heads around the Bisq HTTP API" guide, all the stuff around GMail and email libraries and usernames and passwords and the code required to bootstrap it all is just noise and makes the whole thing longer and more complicated than it needs to be. As I walked through this guide myself, I punted when I got to this point; I just didn't want to deal. Because: (a) I never use my GMail account, (b) it's set up for 2FA anyway so a simple username / password won't work, (c) I don't want to set up a test account just for the purposes of this guide that told me it would take 15 minutes and is now in fact going to suck up more of my time than I want it to.
As an alternative, just print interesting offers (i.e. those that match the filter) to the command line whenever they show up, or if for some reason that is not idiomatic, then to a file that the user can tail.
http-api-monitor-offers.adoc
Outdated
} | ||
---- | ||
|
||
Where `offers` is an array with individual offers and `total` is number of all offers. The model for each individual `OfferDetail` is defined as follows: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure there's a lot of value in including the whole model here. As a reader, I feel like I'm supposed to do something with this, when in fact it's just for informational purposes. As an alternative, I'd deep-link the user to the swagger API docs that show this same information, and possibly do it in an INFO admonition so that the reader understands this is "just if they care / want to go deeper", and that it's not part of the critical path of the step-by-step guide.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is problematic cause the link looks like http://localhost:8080/swagger#/offers/find3
. That find3
might change into something else if we modify the resource in future by adding/removing GET endpoints.
http-api-monitor-offers.adoc
Outdated
direction: 'SELL', | ||
paymentMethodId: 'SEPA' | ||
} | ||
---- |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- This result and the "we need to filter those offers" language above again make me as a reader feel like I'm supposed to do something with this information. There is actually no "step" here, it's just for informational / context purposes, and that's fine, but it should be made more clear. For example, you can say something like:
We'll need to filter our offers using the static criteria you see below:
[json snippet]
In the sections that follow, we'll do just that with a combination of data returned from Bisq HTTP API endpoint calls and programmatic filtering.
(Very quickly written, but you get the idea)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
List available offers
and Get the market price
parts are just describing how the API looks like.
I wanted to first give API overview and then dive into the javascript.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've introduced API overview
section and those List available offers
and Get the market price
are now subsections so it should be clear for the user that they don't need to do anything there.
http-api-monitor-offers.adoc
Outdated
} | ||
---- | ||
|
||
== The JavaScript part |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- I'd retitle this as something like "Write the monitoring bot code" or "Build the monitoring bot".
"The JavaScript part" sounds boring or like some random detail, whereas "Writing the bot" sounds like what I came here to do.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
http-api-monitor-offers.adoc
Outdated
|
||
npm install lodash http-as-promised nodemailer | ||
|
||
In general our script will look like this: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the sections that follow, I recommend that all source code snippets:
-
get sourced from an actual, e.g. 'bisqmon.js' source file checked into the repository. Or perhaps
http-api-monitor-offers.js
to align with the naming of this file. We'll probably need to think more about how to structure companion sources along with these docs in a flat namespace, but it doesn't matter much at this point: what does matter is that the code below should get extracted as snippets from real source code that actually works. Asciidoctor has great tools to help here, see https://asciidoctor.org/docs/user-manual/#include-partial. -
get labeled such that they show the name of the file they're coming from (it's fine if all three are labeled as coming from the same file. The point is that this guide needs to be an extremely literal, step-by-step process of going from zero to working code. So if there is code that the user should write, you should tell them where to write it (i.e. in a file named
bisqmon.js
) and in which order to write it. Add this statement, add this function, and that function, and you're done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
both done
http-api-monitor-offers.adoc
Outdated
0.25 BTC (-6%) | ||
|
||
[NOTE] | ||
If you would like to use something else that gmail then you will need a bit different mail transport configuration. For reference look at https://nodemailer.com/smtp/. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Can be dropped once email is gone.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Deliver a proper ending to the guide. Right now the ending is a bit abrupt. Wrap things up with a bit of prose (a 'congratulations' is fine), and I often like to do a "next steps" section at this point, where you can direct users to whatever other resources you think they may find most valuable after having just gotten started here. Of course this would include other HTTP API docs and guides as we write them, but it could also include encouraging users to come talk with the team in Slack, to provide feedback on the project via GitHub issues, etc.
index.adoc
Outdated
@@ -3,6 +3,7 @@ | |||
* *_User Docs_* | |||
** <<intro#, Introduction>> — What Bisq is, why it exists and how it works | |||
** <<getting-started#, Getting Started>> — Go from zero to trading in 15 minutes | |||
** <<http-api#, Getting Started - HTTP API>> — How to access Bisq programatically |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Should be updated to link to actual doc path, and should be titled something more like "Getting Started with the Bisq HTTP API" (especially so long as this is the only entry in this list. If it uses the actual title of the doc as it currently exists here ("Monitoring Bisq offers with Http API"), then it will seem too specific, and readers won't know that they should "START HERE" if they're new to the HTTP API.
http-api-monitor-offers.adoc
Outdated
* NodeJS 6+ | ||
* Docker (to run from an image) or Git and Maven (to build from source) | ||
* A GMail account | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Right about here we should do an INFO admonition that lets the user know that the Bisq HTTP API is currently incubating, and that this means things may be a bit rough around the edges, that their feedback is extra appreciated, along with pointers about how to provide that feedback. You may just direct them to the "next steps" section at bottom where they can find out more about providing feedback (see my related comment about 'next steps' below).
http-api-monitor-offers.adoc
Outdated
cd bisq-api | ||
mvn compile exec:java \ | ||
-Dexec.mainClass="io.bisq.api.app.ApiMain" \ | ||
-Dexec.args="--appName=bisqmon" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- As I mentioned in 24ffd06, the
--appName
option and arg should be carried over to thedocker
arrangement as well. I have not done that in my edits.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
http-api-monitor-offers.adoc
Outdated
mvn compile exec:java \ | ||
-Dexec.mainClass="io.bisq.api.app.ApiMain" \ | ||
-Dexec.args="--appName=bisqmon" | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Right about here we should do an INFO admonition that explains why they're having to run a separate instance of Bisq to experiment with the HTTP API. It's going to be confusing as hell for users who have a perfectly good Bisq client already up and running locally to spin up a different, unrelated instance, unless they have that explained to them. I'd just explain it very clearly in the context of this being an "incubating" project, and that when the the project comes out of incubation, this functionality will be integrated directly into the official Bisq client you run locally. In the meantime, you need to spin up a separate instance to experiment with (and provide feedback on!) this incubating project.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
@blabno, I've pushed your commit 911f6ca to the PR branch, and I've also invited you as a collaborator on my fork so you can push any additional commits too. Could you go through my review comments above, check any / all boxes you've addressed and comment on anything that remains outstanding? I've taken a quick look at the changes, and they look good. I'll have more time to get back to this soon. Thanks. |
Being collaborator on your fork @cbeams won't allow me to tick boxes in this PR made against bisq-network's repo. |
I see a few additional commits above, @blabno, thanks. Thanks too for noting "done" everywhere you've already addressed stuff. I'll go through this step by step myself once more with an eye toward merging and publishing it. If there's anything else you'd like to get in or wrap up before that happens, please go ahead. |
@blabno, I've just brought this PR up to date with the latest on @m52go, would you be up for reviewing this doc? I believe we're close to being ready to ship and promote this, but I'd like to have someone else review it first, follow the instructions step by step and provide feedback on anything that trips you up. Of course you're free to suggest improvements to the text as well. I'm granting you collaborator rights on my fork so you can just push additional commits as you like. In any case, the doc should take 15 minutes or so to read and complete from scratch. Even if all you can do is play the role of the uninitiated reader and provide your feedback, that would be very helpful. |
@cbeams sure thing, will do within the next day or two. |
More substantial suggestions to be posted in comments on GitHub.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My commit included copy edits and other minor changes. These are more substantive suggestions.
Overall it was very quick and easy...I think it took me less than 15 minutes! Very exciting stuff.
---- | ||
|
||
[NOTE] | ||
Complete interactive API documentation is available at http://localhost:8080/swagger. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd make this stand out a bit more. I don't know if it was just me, but I glazed over it the first few times I saw it. (I also wasn't aware of Swagger, so I just thought it was a cool pathname).
Maybe take it out of a NOTE and make it something like "*Complete API documentation is available at http://localhost:8080/swagger*. It's interactive, so you can play around with it right in your browser.
"
** Docker (to run from an image) or | ||
** Git, Maven and JDK8 (to build from source) | ||
|
||
NOTE: Using the Bisq HTTP API in no way requires the use of NodeJS on the client side. It is just used for example purposes in this guide. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggest removing the parenthetical note above ("to execute the API client script...") and making this note more detailed.
Specifically, explicitly state:
- Node.js is not a dependency of the HTTP API.
- It's only used here to make the tasks of making HTTP requests and manipulating responses easier for the purposes of this example, but that those tasks can be done with any tools the user prefers.
} | ||
---- | ||
|
||
Next we need to filter those offers by price. There are two types of offers: _market-based price offers_ and _fixed price offers_. They are distinguished by the `useMarketBasedPrice` attribute. In the case of market-based price offers the filtering criteria is easy: the `marketPriceMargin` value must be above 0.1. In the case of fixed price offers we have to fetch the market price of BTC in EUR, calculate whether the price is 1% or less, and then filter offers which have a price _less_ than that calculated price. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"On Bisq, there are two types of offers..." [Suggestion]
"In the case of fixed price offers we have to fetch the market price of BTC in EUR, calculate a threshold price that fits our needs (i.e., the price that's 1% lower than the market price), and then find all offers which have a price less than that threshold price." [Suggestion—I found this wording confusing]
include::http-api-monitor-offers.js[tags=flow] | ||
---- | ||
|
||
So first 2 things to do (concurrently and asynchronously) is to fetch offers and market price of BTC from our API. Then we need to filter those offers and display the results. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure "(concurrently and asynchronously)" adds any value.
|
||
Let's install some dependencies: | ||
|
||
npm install lodash http-as-promised |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might be helpful to mention adding the require()
statements here too.
I built the script as I read along, and ended up with errors because I didn't require()
those modules. It's a super-simple thing that most people will figure out quickly, and I know it's already in the full script at the bottom, but just a quick mention along the lines of "don't forget to require these modules" or something might be nice.
|
||
=== Run the API using Docker | ||
|
||
The easiest way to run the API in headless mode is by using a Docker image: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could 'headless' be clarified here? Knowing virtually nothing about how this API works, my initial impression was that the API would query my local Bisq client for data, and that 'headless' simply meant 'no GUI' but that's clearly not the case.
Basically I wasn't sure if the API needed my Bisq client to be running in the background.
A quick 1 or 2 lines about what these images actually do / how the API works might help?
Also, how does one know when the Docker image is done setting up? Is it when the Bisq ASCII art shows? I watched the terminal for a while until I realized that what I was seeing was actually streaming data, and API calls were already working.
} | ||
//end::getOffers[] | ||
|
||
function notify(offers) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Aside from the require()
statements, this notify()
method was the one other item that wasn't mentioned in the text that I didn't have in my code when I ran the script for the first time.
Just FYI. I guess it's because this is structured as a bottom-up walk-through. If it were reversed (i.e., top-down approach with full script first and then important parts clarified), it would be fine to pick and choose what to cover.
|
||
== What you'll build | ||
|
||
You'll build a NodeJS-based script that connects to Bisq over an HTTP API to get offers and market prices and then displays "interesting" offers on the console if any are found. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's officially 'Node.js'. Just concerned the JS hipster 'ninjas' will get us if we don't get this right...
Closing as stalled since this API effort was superceded by another one. |
This PR is being created from a squashed version of @blabno's original commits in
blabno:feature/http-api
.This is the first of what may become a collection of such guides designed to get new API users up and running as quickly as possible with working code that implements the simplest possible versions of real-world use cases.