Skip to content
This repository has been archived by the owner on Jun 15, 2021. It is now read-only.

WIP Produce the simplest-possible getting started guide for the Bisq HTTP API #54

Closed
wants to merge 16 commits into from
Closed
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
327 changes: 327 additions & 0 deletions http-api-monitor-offers.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,327 @@
= Monitoring Bisq offers with Http API

This guide walks you through the process of creating a bot that monitors available offers and sends email notifications.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to drop the email notifications. It makes sense for a script you'd actually want to run in real life, but here in a "let's build the simplest possible bot that could actually work for the purposes of getting our heads around the Bisq HTTP API" guide, all the stuff around GMail and email libraries and usernames and passwords and the code required to bootstrap it all is just noise and makes the whole thing longer and more complicated than it needs to be. As I walked through this guide myself, I punted when I got to this point; I just didn't want to deal. Because: (a) I never use my GMail account, (b) it's set up for 2FA anyway so a simple username / password won't work, (c) I don't want to set up a test account just for the purposes of this guide that told me it would take 15 minutes and is now in fact going to suck up more of my time than I want it to.

As an alternative, just print interesting offers (i.e. those that match the filter) to the command line whenever they show up, or if for some reason that is not idiomatic, then to a file that the user can tail.


== What you'll build

You'll build a NodeJS-based script that connects to Bisq over an HTTP API to get offers and market prices and sends email notification whenever "interesting" offers are detected.

== What you’ll need

* About 15 minutes
* A favorite text editor or IDE
* NodeJS 6+
* Docker (to run from an image) or Git and Maven (to build from source)
* A GMail account

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Right about here we should do an INFO admonition that lets the user know that the Bisq HTTP API is currently incubating, and that this means things may be a bit rough around the edges, that their feedback is extra appreciated, along with pointers about how to provide that feedback. You may just direct them to the "next steps" section at bottom where they can find out more about providing feedback (see my related comment about 'next steps' below).

== Run the API

There are two alternative ways you can run an instance of the Bisq HTTP API, either from a Docker image or from source.

=== Run the API using Docker

The easiest way to run the API in headless mode is by using a Docker image:
Copy link
Contributor

@m52go m52go Jun 30, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could 'headless' be clarified here? Knowing virtually nothing about how this API works, my initial impression was that the API would query my local Bisq client for data, and that 'headless' simply meant 'no GUI' but that's clearly not the case.

Basically I wasn't sure if the API needed my Bisq client to be running in the background.

A quick 1 or 2 lines about what these images actually do / how the API works might help?

Also, how does one know when the Docker image is done setting up? Is it when the Bisq ASCII art shows? I watched the terminal for a while until I realized that what I was seeing was actually streaming data, and API calls were already working.


docker run -it --rm -p 8080:8080 -e BISQ_API_HOST=0.0.0.0 bisq/api

=== Run the API from source

For more hard-core developers that want to run from source:

git clone https://github.com/mrosseel/bisq-api
cd bisq-api
mvn compile exec:java \
-Dexec.mainClass="io.bisq.api.app.ApiMain" \
-Dexec.args="--appName=bisqmon"
Copy link
Contributor Author

@cbeams cbeams May 17, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • As I mentioned in 24ffd06, the --appName option and arg should be carried over to the docker arrangement as well. I have not done that in my edits.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Right about here we should do an INFO admonition that explains why they're having to run a separate instance of Bisq to experiment with the HTTP API. It's going to be confusing as hell for users who have a perfectly good Bisq client already up and running locally to spin up a different, unrelated instance, unless they have that explained to them. I'd just explain it very clearly in the context of this being an "incubating" project, and that when the the project comes out of incubation, this functionality will be integrated directly into the official Bisq client you run locally. In the meantime, you need to spin up a separate instance to experiment with (and provide feedback on!) this incubating project.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

=== Verify the API is running

In both cases the API should be running on port 8080. You can verify that by executing:

curl http://localhost:8080/api/v1/version

You should receive a response like the following:

[source,json]
----
{
"application": "0.6.7",
"network": 1,
"p2PMessage": 10,
"localDB": 1,
"tradeProtocol": 1
}
----

[NOTE]
Complete interactive API documentation is available at http://localhost:8080/swagger.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd make this stand out a bit more. I don't know if it was just me, but I glazed over it the first few times I saw it. (I also wasn't aware of Swagger, so I just thought it was a cool pathname).

Maybe take it out of a NOTE and make it something like "*Complete API documentation is available at http://localhost:8080/swagger*. It's interactive, so you can play around with it right in your browser."


== List available offers

Open offers are available at http://localhost:8080/api/v1/offers.

The response should look like this:

[source,json]
----
{
"offers": [OfferDetail],
"total": integer
}
----

Where `offers` is an array with individual offers and `total` is number of all offers. The model for each individual `OfferDetail` is defined as follows:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure there's a lot of value in including the whole model here. As a reader, I feel like I'm supposed to do something with this, when in fact it's just for informational purposes. As an alternative, I'd deep-link the user to the swagger API docs that show this same information, and possibly do it in an INFO admonition so that the reader understands this is "just if they care / want to go deeper", and that it's not part of the critical path of the step-by-step guide.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is problematic cause the link looks like http://localhost:8080/swagger#/offers/find3. That find3 might change into something else if we modify the resource in future by adding/removing GET endpoints.


[source,json]
----
{
acceptedBankIds [string]
acceptedCountryCodes [string]
amount integer($int64)
arbitratorNodeAddresses [string]
bankId string
baseCurrencyCode string
blockHeightAtOfferCreation integer($int64)
buyerSecurityDeposit integer($int64)
counterCurrencyCode string
countryCode string
currencyCode string
date string($date-time)
direction string Enum: [ BUY, SELL ]
hashOfChallenge string
id string
isCurrencyForMakerFeeBtc boolean
isPrivateOffer boolean
lowerClosePrice integer($int64)
makerFee integer($int64)
makerPaymentAccountId string
marketPriceMargin number($double)
maxTradeLimit integer($int64)
maxTradePeriod integer($int64)
minAmount integer($int64)
offerFeePaymentTxId string
ownerNodeAddress string
paymentMethodId string
price integer($int64)
protocolVersion integer($int32)
sellerSecurityDeposit integer($int64)
state string Enum: [ UNKNOWN, OFFER_FEE_PAID, AVAILABLE, NOT_AVAILABLE, REMOVED, MAKER_OFFLINE ]
txFee integer($int64)
upperClosePrice integer($int64)
useAutoClose boolean
useMarketBasedPrice boolean
useReOpenAfterAutoClose boolean
versionNr string
}
----

Now let's assume that we want to buy bitcoin, and let's define "interesting" offers as those that:

. sell bitcoin at a discount 1% or more under the current market price, and
. accept payment in Euros to a Polish SEPA account

First we need to filter those offers using following static criteria:

[source,json]
----
{
baseCurrencyCode: 'BTC',
counterCurrencyCode: 'EUR',
direction: 'SELL',
paymentMethodId: 'SEPA'
}
----
Copy link
Contributor Author

@cbeams cbeams May 17, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • This result and the "we need to filter those offers" language above again make me as a reader feel like I'm supposed to do something with this information. There is actually no "step" here, it's just for informational / context purposes, and that's fine, but it should be made more clear. For example, you can say something like:

We'll need to filter our offers using the static criteria you see below:

[json snippet]

In the sections that follow, we'll do just that with a combination of data returned from Bisq HTTP API endpoint calls and programmatic filtering.

(Very quickly written, but you get the idea)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

List available offers and Get the market price parts are just describing how the API looks like.
I wanted to first give API overview and then dive into the javascript.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've introduced API overview section and those List available offers and Get the market price are now subsections so it should be clear for the user that they don't need to do anything there.


Next we need to filter those offers by price. There are two types of offers: _market-based price offers_ and _fixed price offers_. They are distinguished by the `useMarketBasedPrice` attribute. In case of market-based price offers the filtering criteria is easy: the `marketPriceMargin` value must be above 0.1. In case of fixed price offers we have to fetch market price of BTC in EUR, then calculate whether the price is 1% or less, and finally we must filter offers which have a price _less_ than that calculated price.

=== Get the market price

In order to get the market price of BTC in EUR, execute the following query:

curl http://localhost:8080/api/v1/currencies/prices?currencyCodes=EUR

You should receive a response like the following:

[source,json]
----
{
"prices": {
"EUR": 7035.62
}
}
----

== The JavaScript part
Copy link
Contributor Author

@cbeams cbeams May 17, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • I'd retitle this as something like "Write the monitoring bot code" or "Build the monitoring bot".

"The JavaScript part" sounds boring or like some random detail, whereas "Writing the bot" sounds like what I came here to do.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


Let's install some dependencies:

npm install lodash http-as-promised nodemailer

In general our script will look like this:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the sections that follow, I recommend that all source code snippets:

  • get sourced from an actual, e.g. 'bisqmon.js' source file checked into the repository. Or perhaps http-api-monitor-offers.js to align with the naming of this file. We'll probably need to think more about how to structure companion sources along with these docs in a flat namespace, but it doesn't matter much at this point: what does matter is that the code below should get extracted as snippets from real source code that actually works. Asciidoctor has great tools to help here, see https://asciidoctor.org/docs/user-manual/#include-partial.

  • get labeled such that they show the name of the file they're coming from (it's fine if all three are labeled as coming from the same file. The point is that this guide needs to be an extremely literal, step-by-step process of going from zero to working code. So if there is code that the user should write, you should tell them where to write it (i.e. in a file named bisqmon.js) and in which order to write it. Add this statement, add this function, and that function, and you're done.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

both done


[source,javascript]
----
Promise.all([getOffers(), getMarketPrice()])
.then(filterOffers)
.then(notify)
.catch(e => console.error(e));
----

So first 2 things to do (concurrently and asynchronously) is to fetch offers and market price of BTC from our API. Then we need to filter those offers and finally send notification.

Getting offers is a simple call that returns promise with array of offers:

[source,javascript]
----
function getOffers() {
return $http.get('http://localhost:8080/api/v1/offers', {resolve: 'body', json: true})
.then(body => body.offers);
}
----

Similarly with `getMarketPrice`:

[source,javascript]
----
function getMarketPrice() {
return $http.get('http://localhost:8080/api/v1/currencies/prices?currencyCodes=EUR', {resolve: 'body', json: true})
.then(body => _.get(body, 'prices.EUR'))
}
----

Now our `filterOffers` function is ready to receive an array of results from the two functions described above:

[source,javascript]
----
function filterOffers([offers, marketPrice]) {
return _(offers)
.filter({
baseCurrencyCode: 'BTC',
counterCurrencyCode: 'EUR',
direction: 'SELL',
paymentMethodId: 'SEPA'
})
.filter(i => _.includes(i.acceptedCountryCodes, 'PL'))
.filter(getPriceFilter(marketPrice))
.map(i => ({amount: i.amount, margin: i.useMarketBasedPrice ? i.marketPriceMargin : marketPrice / i.price}))
.value();
}
----

This function filters offers to match our criteria. It returns matching offers and maps them to a bit simpler structure that contains as little data as needed for `notify` function. We are using `lodash` library to simplify the filtering.

The `getPriceFilter` function creates the actual filter function and looks like this:

[source,javascript]
----
function getPriceFilter(marketPrice) {
const maxPrice = marketPrice * (1 - threshold) * 10000;
return offer => {
if (offer.useMarketBasedPrice)
return offer.marketPriceMargin >= threshold;
return offer.price < maxPrice;
}
}
----

We are multiplying `marketPrice` by `10000` because that is the format in which the API returns the price.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Somewhere right about here, you should instruct the user to actually run node bisqmon.js (or whatever the correct invocation should be), and then show them what they should expect to see as correct output. This is the big payoff. They've hung with us for 15 minutes, reading our instructions, taking in the context, and following each step faithfully. Now they get what they came for: working code that does exactly what we said it would.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


Here is a full script. You must substitute `EMAIL_ACCOUNT_USERNAME`, `EMAIL_ACCOUNT_PASSWORD`, `EMAIL_FROM_ADDRESS` and `EMAIL_TO_ADDRESS`
with appropriate values.

[source,javascript]
----
const _ = require('lodash');
const $http = require('http-as-promised');
const nodemailer = require('nodemailer');

const threshold = 0.1;

const EMAIL_ACCOUNT_USERNAME = 'email@gmail.com';
const EMAIL_ACCOUNT_PASSWORD = 'secret';
const EMAIL_FROM_ADDRESS = 'from@gmail.com';
const EMAIL_TO_ADDRESS = 'to@gmail.com';

function getPriceFilter(marketPrice) {
const maxPrice = marketPrice * (1 - threshold) * 10000;
return offer => {
if (offer.useMarketBasedPrice)
return offer.marketPriceMargin >= threshold;
return offer.price < maxPrice;
}
}

function getMarketPrice() {
return $http.get('http://localhost:8080/api/v1/currencies/prices?currencyCodes=EUR', {resolve: 'body', json: true})
.then(body => _.get(body, 'prices.EUR'))
}

function getOffers() {
return $http.get('http://localhost:8080/api/v1/offers', {resolve: 'body', json: true}).then(body => body.offers);
}

function notify(offers) {
if (!offers.length) {
console.log('No interesting offers found');
return;
}

const transporter = nodemailer.createTransport({
service: 'gmail',
auth: {
user: EMAIL_ACCOUNT_USERNAME,
pass: EMAIL_ACCOUNT_PASSWORD
}
});

const text = _.map(offers, offer => `${offer.amount / 100000000} BTC (-${_.round(offer.margin * 100, 2)}%)`).join('\n');

const mailOptions = {
from: EMAIL_FROM_ADDRESS,
to: EMAIL_TO_ADDRESS,
subject: `${offers.length} interesting BTC offers from Bisq`,
text: text
};

transporter.sendMail(mailOptions, function (error) {
if (error) {
console.log(error);
} else {
console.log(`Notification about ${offers.length} offers sent`);
}
});
}

function filterOffers([offers, marketPrice]) {
return _(offers)
.filter({
baseCurrencyCode: 'BTC',
counterCurrencyCode: 'EUR',
direction: 'SELL',
paymentMethodId: 'SEPA'
})
.filter(i => _.includes(i.acceptedCountryCodes, 'PL'))
.filter(getPriceFilter(marketPrice))
.map(i => _.pick(i, 'baseCurrencyCode', 'counterCurrencyCode', 'direction', 'paymentMethodId', 'id', 'useMarketBasedPrice', 'price', 'marketPriceMargin', 'amount', 'minAmont'))
.map(i => ({amount: i.amount, margin: i.useMarketBasedPrice ? i.marketPriceMargin : marketPrice / i.price}))
.value();
}

Promise.all([getOffers(), getMarketPrice()])
.then(filterOffers)
.then(notify)
.catch(e => console.error(e));
----
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • As per the comments above re email, I'd yank all of that and just print to the console.

  • It's probably a good idea to recap the whole file here, I agree, so that the reader can take the whole thing in, see that it (a) makes sense, (b) is short, and (c) can easily copy and paste it if they have failed to copy and paste the individual snippets one-by-one in the steps above.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • This is right where you should have them invoke node bisqmon.js (or whatever).

If there are any matching offers you should receive an mail like this:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • If email is out of the picture, they'd now see "something like the following" at the command line.


5 interesting BTC offers from Bisq
0.0625 BTC (-2%)
0.01 BTC (-2%)
0.01 BTC (-5%)
0.033 BTC (-3%)
0.02 BTC (-1.5%)
0.25 BTC (-6%)

[NOTE]
If you would like to use something else that gmail then you will need a bit different mail transport configuration. For reference look at https://nodemailer.com/smtp/.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Can be dropped once email is gone.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Deliver a proper ending to the guide. Right now the ending is a bit abrupt. Wrap things up with a bit of prose (a 'congratulations' is fine), and I often like to do a "next steps" section at this point, where you can direct users to whatever other resources you think they may find most valuable after having just gotten started here. Of course this would include other HTTP API docs and guides as we write them, but it could also include encouraging users to come talk with the team in Slack, to provide feedback on the project via GitHub issues, etc.

1 change: 1 addition & 0 deletions index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
* *_User Docs_*
** <<intro#, Introduction>> — What Bisq is, why it exists and how it works
** <<getting-started#, Getting Started>> — Go from zero to trading in 15 minutes
** <<http-api#, Getting Started - HTTP API>> — How to access Bisq programatically
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Should be updated to link to actual doc path, and should be titled something more like "Getting Started with the Bisq HTTP API" (especially so long as this is the only entry in this list. If it uses the actual title of the doc as it currently exists here ("Monitoring Bisq offers with Http API"), then it will seem too specific, and readers won't know that they should "START HERE" if they're new to the HTTP API.


* *_Contributor Docs_*
** <<contributor-checklist#, New contributor checklist>>
Expand Down