Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use bilinear interpolation for rasters #114

Open
dnomadb opened this issue Jan 7, 2016 · 18 comments
Open

Use bilinear interpolation for rasters #114

dnomadb opened this issue Jan 7, 2016 · 18 comments

Comments

@dnomadb
Copy link

dnomadb commented Jan 7, 2016

Currently we are using nearest neighbor (the mapnik default) interpolation for the raster symbolizer:
https://github.com/mapbox/tilelive-vector/blob/master/templates/layer.xml#L21

Changing it to scaling="bilinear" will give better looking output tiles, which is especially important if we move towards resolution rounding as detailed here: mapbox/mapnik-omnivore#133

cc @yhahn @springmeyer @perrygeo

@springmeyer
Copy link
Contributor

👍

Also, while you are at it can you remove the <ElseFilter></ElseFilter> line? That is junk and not doing anything helpful. Just drop this: https://github.com/mapbox/tilelive-vector/blob/master/templates/layer.xml#L20

@perrygeo
Copy link

perrygeo commented Jan 7, 2016

@yhahn brought up the fact that this might have some negative impact on performance. Has anyone done the benchmarks to see how much impact bilinear vs nearest has?

Clarifying question: is this RasterSymbolizer only put to use for overzoomed rasters?

@springmeyer
Copy link
Contributor

Yes, bilinear scaling is more expensive than nearest as we have an optimized path in Mapnik for nearest: https://github.com/mapnik/mapnik/blob/master/src/image_scaling.cpp#L148-L166. So it is worth benchmarking. However I would not be surprised if benchmarking would not pick up a measurable difference.

Clarifying question: is this RasterSymbolizer only put to use for overzoomed rasters?

Best answered by @yhahn. Another question to keep in mind is "What is the impact when raster layers are composited serverside into a new vector tile with other layers". In this case we default to bilinear scaling when overzooming: https://github.com/mapnik/node-mapnik/blob/master/src/mapnik_vector_tile.cpp#L922. When compositing without overzooming (all z/x/y match) then the original image is preserved untouched.

@celoyd
Copy link

celoyd commented Jan 7, 2016

👍 for bilinear on aesthetics grounds. The perceived quality is considerably higher. People don’t get as mad at blur as they get at pixely-ness.

@perrygeo
Copy link

Based on @yhahn's test-fillzoom branch, I was able to test bilinear resampling in overzooms:

  • bilinear makes the images less blocky, more blurry as expected
  • It does not make any difference in terms of spatial alignment.

Here's the results for the current default
xray-fillzoom-2-1-1 original
and for bilinear
xray-fillzoom-2-1-1
and together in a gif (no shift here)
xray

Here's the diff in case we decide to apply this (in map.xml, not layer.xml as written above)

diff --git a/templates/map.xml b/templates/map.xml
index 1158f3d..4ebc9e6 100644
--- a/templates/map.xml
+++ b/templates/map.xml
@@ -15,7 +15,7 @@

 <Style name="_image" filter-mode="first">
   <Rule>
-    <RasterSymbolizer opacity="1" />
+    <RasterSymbolizer opacity="1" scaling="bilinear" />
   </Rule>
 </Style>

@sgillies
Copy link

@dnomadb this smells like it's related what's going on in https://github.com/mapbox/unpacker/issues/752: pixelated overview tiles.

@dnomadb
Copy link
Author

dnomadb commented Feb 4, 2016

I am going to test this out for performance + quality increase. Here are what I see are the steps:

  • Branch off and integrate bilinear resampling here (branch name: bilinear)
  • publish this tilelive-vector branch as a -dev release
  • Integrate this into a branch of mapbox-maps
  • Publish this mapbox-maps branch as a -dev release
  • Bring this release into a branch of api-maps
  • All tests passing on ^^^
  • Deploy this api-maps branch to staging
  • Benchmark this via bench
  • Inspect visual quality and weigh against results of ^ to decide whether this is 👍 / 👎

I'll update this hitlist as I find all the parts I missed :D

@perrygeo
Copy link

Some node version problems with tilelive-vector tests: https://travis-ci.org/mapbox/tilelive-vector/builds/109968314

@springmeyer
Copy link
Contributor

@perrygeo I've also seen those and they are a mystery still (#119). They are therefore very unlikely related to your work and rather a problem we currently have in master.

@dnomadb
Copy link
Author

dnomadb commented Feb 18, 2016

Move on and try it on staging regardless.

@perrygeo 👍

@perrygeo
Copy link

perrygeo commented Mar 1, 2016

Tested on api-maps@bilinear - no bueno.

Trying round 2 by adding bilinear to templates.layer.xml as well as map.xml...

@perrygeo
Copy link

perrygeo commented Mar 1, 2016

Alright, so all the tests are broken and I'm confused about the distinction between layer.xml and map.xml but we've got some preliminary results

Looking at the mapbox.satellite tilejson we see a maxzoom: 19

So a standard 256 z19 tile should be the same, no overzoom. ✔️ while the 2x versions should differ. ✔️

$ diff 203882_1x_bilinear.png 203882_1x_default.png
$ diff 203882_2x_bilinear.png 203882_2x_default.png
Binary files 203882_2x_bilinear.png and 203882_2x_default.png differ

The difference is due to the bilinear interpolation.

The current default

203882_2x_default

With bilinear

203882_2x_bilinear

So the question: is bilinear interpolation worth it from the visual perspective?

@perrygeo
Copy link

perrygeo commented Mar 1, 2016

In terms of benchmarking for speed, I'm not sure bench is going to be very helpful as it's more likely to pick up differences in infrastructure than in the actual tile generation process. We need to reduce extraneous variables, not add them.

EDIT URL needed an access token.

Still, bench makes it easy enough to run them so I started two stacks and ran bench:

  • api-maps@master: results show 79.3/s
  • api-maps@bilinear: results show 74.1/s

It's hard to say how much is due to difference in network/infrastructure/availability-zones, etc but they're pretty close. Next step we may want to force z19 @2x as a next step?

Here's the invocations:

mbx watchbot queue '{"urls":["http://api-maps-bilinear-VPC-632541875.us-east-1.elb.amazonaws.com/v4/mapbox.satellite/{z}/{x}/{y}.png?access_token=pk.eyJ1IjoicGVycnlnZW8iLCJhIjoiY2lsOGd3dGRxMDAwYXN6bTBxemhnNm5lcyJ9.OJbdkgZOGRPRJZ0RZUpPCw"]}' production
mbx watchbot queue '{"urls":["http://api-maps-notbilinear-VPC-1856557177.us-east-1.elb.amazonaws.com/v4/mapbox.satellite/{z}/{x}/{y}.png?access_token=pk.eyJ1IjoicGVycnlnZW8iLCJhIjoiY2lsOGd3dGRxMDAwYXN6bTBxemhnNm5lcyJ9.OJbdkgZOGRPRJZ0RZUpPCw"]}' production

@yhahn
Copy link
Member

yhahn commented Mar 1, 2016

Per chat, ^^ the benchmarks above are benching the endpoint when it returns a 401 unauthorized response. Try rerunning the benchmarks with ?access_token=<token> in the URL and maybe check a 0/0/0 tile before running the bench as well to make sure the stack will serve images for the URLs in question.

@perrygeo
Copy link

perrygeo commented Mar 1, 2016

Thanks @yhahn @dnomadb - updated results ^^^ - pretty close timings. I'm gonna try with 2x z19 tiles only next.

@perrygeo
Copy link

perrygeo commented Mar 1, 2016

Forcing z19 2x pngs

mbx watchbot queue '{"minzoom":19,"urls":["http://api-maps-bilinear-VPC-632541875.us-east-1.elb.amazonaws.com/v4/mapbox.satellite/{z}/{x}/{y}@2x.png?access_token=pk.eyJ1IjoicGVycnlnZW8iLCJhIjoiY2lsOGd3dGRxMDAwYXN6bTBxemhnNm5lcyJ9.OJbdkgZOGRPRJZ0RZUpPCw"]}' production
mbx watchbot queue '{"minzoom":19,"urls":["http://api-maps-notbilinear-VPC-1856557177.us-east-1.elb.amazonaws.com/v4/mapbox.satellite/{z}/{x}/{y}@2x.png?access_token=pk.eyJ1IjoicGVycnlnZW8iLCJhIjoiY2lsOGd3dGRxMDAwYXN6bTBxemhnNm5lcyJ9.OJbdkgZOGRPRJZ0RZUpPCw"]}' production

So, discounting extraneous variables, it looks like bilinear adds about 5% to the response time. Not horrible. We could go deeper to try to really isolate the rendering times sans network but let's stick with that rough assumption for now.

@camillacaros
Copy link

Liking what we're seeing so far with bilinear! Smoother edges on buildings, and an easing of the columning effect in color elsewhere.

Default on left // Bilinear on right:
screen shot 2016-03-01 at 11 42 11 am

@perrygeo
Copy link

perrygeo commented Mar 2, 2016

like buttah

Y, I think it's an improvement and it certainly smooths out a lot of the artifacts of compression and overzooming.

The 💰 question: Is the visual improvement big enough to justify moving forward?

Pros:

  • less grainy, more smoothy
  • less noticable artifacts

Cons:

  • the lift to merge, update test suites, cut releases and deploy.
  • the slight 5% performance hit
  • only improves tiles that are actually overzoomed (IOW zero difference for mapbox.satellite at 256px z19 tiles)
  • a bit of an unknown how/if this will affect other renders like user-uploaded rasters and other tilesets (this is mostly reflecting my own ignorance of the system)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants