diff --git a/.vitepress/config.mts b/.vitepress/config.mts index 3c41261..62c5464 100644 --- a/.vitepress/config.mts +++ b/.vitepress/config.mts @@ -50,13 +50,13 @@ export default defineConfig({ text: "Accelerating PMTiles", collapsed: true, items: [ - { text: "Server", link: "/deploy/server" }, { text: "Overview", link: "/deploy/" }, { text: "Cost Calculator", link: "/deploy/cost" }, { text: "AWS", link: "/deploy/aws" }, - { text: "Azure", link: "/deploy/azure" }, { text: "Cloudflare", link: "/deploy/cloudflare" }, { text: "Google Cloud", link: "/deploy/google-cloud" }, + { text: "Azure", link: "/deploy/azure" }, + { text: "Server", link: "/deploy/server" }, ], }, { diff --git a/pmtiles/cli.md b/pmtiles/cli.md index 503a6f3..fc50c69 100644 --- a/pmtiles/cli.md +++ b/pmtiles/cli.md @@ -5,24 +5,61 @@ outline: deep # pmtiles CLI +## Installation + +`pmtiles` is a single binary with no external dependencies. + +The source code is on [GitHub at protomaps/go-pmtiles](https://github.com/protomaps/go-pmtiles). + +To download, see [Releases on GitHub](https://github.com/protomaps/go-pmtiles/releases) for your OS and architecture. + ## CLI Overview +### Local archives + +The CLI works with local tilesets on disk, for example: + +```sh +pmtiles show test.pmtiles +``` + ### Remote archives -Remote buckets are specified in the CLI via URLS. Commands are similar to: +However, `pmtiles` can also work with remote HTTP archives and tilesets on cloud storage, even in private buckets. +```sh +pmtiles show https://r2-public.protomaps.com/protomaps-sample-datasets/cb_2018_us_zcta510_500k.pmtiles ``` -pmtiles [COMMAND] [KEY] --bucket=[PROTOCOL]://[BUCKET_NAME] + +`pmtiles` uses the [go-cloud](https://gocloud.dev/howto/blob/) library for connecting and authenticating to cloud storage. + +Commands for S3, Azure Blob and Google Cloud Storage: + +:::info +Commands that uses URL characters like `?` and `&`, should be escaped by a backslash `\` in your shell. +::: + +```sh +pmtiles show test.pmtiles --bucket=s3://BUCKET_NAME +pmtiles show test.pmtiles --bucket=azblob://CONTAINER_NAME?storage_account=ACCOUNT +pmtiles show test.pmtiles --bucket=gs://BUCKET_NAME ``` -The bucket URL can contain query parameters like: +For S3-compatible blob storage (Minio, Cloudflare R2, etc) outside of AWS: -* `endpoint`: If not using AWS, an S3-compatible HTTPS endpoint. -* `region`: a provider-specific region string, such as `us-west-2` for AWS, and `auto` for all Cloudflare regions. +```sh +pmtiles show test.pmtiles --bucket=s3://BUCKET_NAME?endpoint=https://example.com®ion=auto +``` -Example of reading from a private Cloudflare R2 bucket: +:::info +Some S3-compatible storage servers like Minio, Ceph and SeaweedFS may require [additional URL options](https://gocloud.dev/howto/blob/#s3-compatible) like `s3ForcePathStyle=true`. +::: -Since this command uses URL characters like `?` and `&`, those **must be escaped** by a backslash `\`. +### Private buckets + +`pmtiles` uses [go-cloud's](https://gocloud.dev/howto/blob/) default authentication methods for each cloud provider. + +For example, the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables are used to sign requests to private S3-compatible buckets: ```sh export AWS_ACCESS_KEY_ID=MY_KEY @@ -30,14 +67,13 @@ export AWS_SECRET_ACCESS_KEY=MY_SECRET pmtiles show NAME.pmtiles --bucket=s3://R2_BUCKET_NAME\?endpoint=https://R2_ACCOUNT_ID.r2.cloudflarestorage.com\®ion=auto ``` -Note that S3-compatible storage servers like Minio, Ceph and SeaweedFS may require [additional URL options](https://gocloud.dev/howto/blob/#s3-compatible) like `s3ForcePathStyle=true`. - ## Commands ### show ```bash pmtiles show INPUT.pmtiles +pmtiles show INPUT.pmtiles --bucket=s3://BUCKET_NAME ``` Print an archive's header data and metadata. @@ -48,7 +84,7 @@ Print an archive's header data and metadata. pmtiles tile INPUT.pmtiles 0 0 0 ``` -Output a single tile to stdout. +Output a single tile's contents to stdout. ### verify @@ -63,6 +99,8 @@ Check that an archive is ordered correctly and has correct header information. ```bash pmtiles extract INPUT.pmtiles OUTPUT.pmtiles --bbox=MIN_LON,MIN_LAT,MAX_LON,MAX_LAT pmtiles extract INPUT.pmtiles OUTPUT.pmtiles --region=REGION.geojson +pmtiles extract https://example.com/INPUT.pmtiles OUTPUT.pmtiles --maxzoom=MAXZOOM +pmtiles extract INPUT.pmtiles OUTPUT.pmtiles --maxzoom=MAXZOOM --bucket=s3://BUCKET_NAME ``` Create a smaller archive from a larger archive. The source archive may be local or remote. The source archive must be clustered. @@ -71,26 +109,73 @@ Options: * `--maxzoom`: Extract only a subset of zoom levels. Extracting a full sub-pyramid from 0 to `maxzoom` is always an efficient operation that makes minimal I/O or network requests to the source archive. * `--minzoom`: Extract only a partial sub-pyramid. This may require many more requests than leaving the default `--minzoom=0`. Because this removes overview zoom levels, it should only be used in specific situations. +* `--region`: a [GeoJSON](https://geojson.org) Polygon, Multipolygon, Feature, or FeatureCollection. +* `--download-threads` Number of parallel requests to speed up downloads. +* `--overfetch` extra data to download to batch small requests: 0.05 is 5%. ### serve +The simplest way to consume PMTiles on the web is directly in the browser with [pmtiles.js along with a renderer-specific client](/pmtiles/maplibre). However, decoding PMTiles on the server and exposing a ZXY API works with more clients and can result in lower latency. A ZXY API is directly supported by web and native renderers such as [MapLibre](https://maplibre.org), without needing the PMTiles client library. Using `pmtiles serve` also allows serving a public API from a private storage bucket. + +:::info +When using `pmtiles serve`, requests for the raw file like `/test.pmtiles`, either whole or partial range requests, are not supported. A standard web server like Apache, Nginx or Caddy can serve those. +::: + +Serve a directory or bucket of tilesets (like TILESET.pmtiles) from local or cloud storage as a ZXY endpoint: + ```bash pmtiles serve . +# serves this directory at http://localhost:8080/TILESET/{z}/{x}/{y}.mvt +# the .pmtiles extension is added automatically +# TileJSON at http://localhost:8080/TILESET.json pmtiles serve . --bucket=https://example.com pmtiles serve / --bucket=s3://BUCKET_NAME +pmtiles serve PREFIX --bucket=s3://BUCKET_NAME ``` -Expose [TileJSON](https://github.com/mapbox/tilejson-spec/tree/master/3.0.0), e.g. `/mymap.json`, as well as Z/X/Y tile URLs, e.g. `/mymap/{z}/{x}/{y}.mvt`, for a directory or bucket of archives. **Requests for the raw file e.g. `mymap.pmtiles` will not work.** +For ZXY URLs, the extension must match the type of the tiles in the archive, for example `mvt`, `png`, `jpg`, `webp`, `avif`. + +Flags: + +* `--cors=ORIGIN` set the value of the Access-Control-Allow-Origin. * is a valid value but must be escaped in your shell. Appropriate for development use. +* `--cache-size=SIZE_MB` set the global size of the header and directory LRU cache, shared across all archives. Default is 64 MB. +* `--port=PORT` specify the HTTP port. Defaults to 8080. +* `--public-url`: Required for serving [TileJSON](https://github.com/mapbox/tilejson-spec/tree/master/3.0.0). Specify the full URL as it should appear to the browser client like `http://localhost:8080` or `https://example.com`. + +For production usage, it's recommended to run behind a CDN or reverse proxy like Caddy to handle SSL and CORS. See the guide on [Accelerating PMTiles](/deploy/). -A Z/X/Y URL like is directly supported by web and native clients such as [MapLibre](http://maplibre.org), without needing the PMTiles client library. Using `pmtiles serve` this way also allows serving public Z/X/Y traffic from private storage buckets. ### convert +Convert an [MBTiles](https://github.com/mapbox/mbtiles-spec/tree/master/1.3) archive to PMTiles. + ```bash pmtiles convert INPUT.mbtiles OUTPUT.pmtiles ``` -Convert from MBTiles. +### upload + +Upload an archive to cloud storage. + +```bash +# requires environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY credentials +pmtiles upload INPUT.pmtiles REMOTE.pmtiles --bucket=s3://BUCKET_NAME +``` + +You will need write permissions to the bucket, for example this AWS IAM policy: + +```json + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": "s3:*", + "Resource": "arn:aws:s3:::my-bucket-name/*" + } + ] + } +``` ### version