Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(*): bump of sharp for Node v12 compatibility #13646

Merged
merged 18 commits into from
May 3, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 2 additions & 5 deletions packages/gatsby-plugin-manifest/.gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,2 @@
/gatsby-node.js
/gatsby-ssr.js
/gatsby-browser.js
/common.js
/app-shell.js
/*.js
!index.js
31 changes: 31 additions & 0 deletions packages/gatsby-plugin-manifest/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -368,3 +368,34 @@ This article from the Chrome DevRel team is a good intro to the web app
manifest—https://developers.google.com/web/fundamentals/engage-and-retain/web-app-manifest/

For more information see the w3 spec https://www.w3.org/TR/appmanifest/ or Mozilla docs https://developer.mozilla.org/en-US/docs/Web/Manifest.

## Troubleshooting

### Incompatible library version: sharp.node requires version X or later, but Z provides version Y

This means that there are multiple incompatible versions of the `sharp` package installed in `node_modules`. The complete error typically looks like this:

```
Something went wrong installing the "sharp" module

dlopen(/Users/misiek/dev/gatsby-starter-blog/node_modules/sharp/build/Release/sharp.node, 1): Library not loaded: @rpath/libglib-2.0.dylib
Referenced from: /Users/misiek/dev/gatsby-starter-blog/node_modules/sharp/build/Release/sharp.node
Reason: Incompatible library version: sharp.node requires version 6001.0.0 or later, but libglib-2.0.dylib provides version 5801.0.0
```

To fix this, you'll need to update all Gatsby plugins in the current project that depend on the `sharp` package. Here's a list of official plugins that you might need to update in case your projects uses them:

- `gatsby-plugin-sharp`
- `gatsby-plugin-manifest`
- `gatsby-remark-images-contentful`
- `gatsby-source-contentful`
- `gatsby-transformer-sharp`
- `gatsby-transformer-sqip`

To update these packages, run:

```sh
npm install gatsby-plugin-sharp gatsby-plugin-manifest gatsby-remark-images-contentful gatsby-source-contentful gatsby-transformer-sharp gatsby-transformer-sqip
```

If updating these doesn't fix the issue, your project probably uses other plugins from the community that depend on a different version of `sharp`. Try running `npm list sharp` or `yarn why sharp` to see all packages in the current project that use `sharp` and try updating them as well.
3 changes: 2 additions & 1 deletion packages/gatsby-plugin-manifest/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@
},
"dependencies": {
"@babel/runtime": "^7.0.0",
"sharp": "^0.21.3"
"semver": "^5.6.0",
"sharp": "^0.22.1"
},
"devDependencies": {
"@babel/cli": "^7.0.0",
Expand Down
2 changes: 1 addition & 1 deletion packages/gatsby-plugin-manifest/src/gatsby-node.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import fs from "fs"
import path from "path"
import sharp from "sharp"
import sharp from "./safe-sharp"
import createContentDigest from "gatsby/dist/utils/create-content-digest"
import { defaultIcons, doesIconExist, addDigestToPath } from "./common"

Expand Down
168 changes: 168 additions & 0 deletions packages/gatsby-plugin-manifest/src/safe-sharp.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
// This is very hacky, but this should be temporary hack
// until we make "sharp" a peerDependency and can be removed afterwards.
// It's not done yet because it would cause too much friction.
// Image processing is important part of Gatsby ecosystem
// and there's lot of guides and tutorials that we don't control
// that would be outdated. Moving "sharp" to be peerDependency
// is scheduled for gatsby@3.

// This file is duplicated in multiple location in this repository,
// So make sure to apply same changes every "safe-sharp" file.

const childProcess = require(`child_process`)
const path = require(`path`)
const fs = require(`fs`)
const semver = require(`semver`)

const originalConsoleError = console.error
const restoreConsoleError = () => {
console.error = originalConsoleError
}

const getDetailedMessage = () => {
try {
// `npm list` seems to work in yarn installed projects as long
// as there is no package-lock.json, so let's bail out
// if both lock files exist

if (
fs.existsSync(path.join(process.cwd(), `package-lock.json`)) &&
fs.existsSync(path.join(process.cwd(), `yarn.lock`))
) {
return null
}

let msg = []
const { dependencies } = JSON.parse(
childProcess.execSync(`npm list sharp --json`, {
encoding: `utf-8`,
})
)

const findSharpVersion = dependency => {
if (dependency.dependencies.sharp) {
return dependency.dependencies.sharp.version
}

for (let depName of Object.keys(dependency.dependencies)) {
const v = findSharpVersion(dependency.dependencies[depName])
if (v) {
return v
}
}

return null
}

const { latestVersion, topLevelPackages } = Object.keys(
dependencies
).reduce(
(acc, depName) => {
const sharpVersion = findSharpVersion(dependencies[depName])
if (sharpVersion) {
acc.topLevelPackages[depName] = sharpVersion

if (
!acc.latestVersion ||
semver.gt(sharpVersion, acc.latestVersion)
) {
acc.latestVersion = sharpVersion
}
}

return acc
},
{
latestVersion: undefined,
topLevelPackages: {},
}
)

let packagesToUpdate = []
// list top level dependencies
msg = msg.concat([
`List of installed packages that depend on sharp:`,
...Object.keys(topLevelPackages).map(depName => {
const sharpVersion = topLevelPackages[depName]
if (sharpVersion !== latestVersion) {
packagesToUpdate.push(depName)
}
return ` - ${depName}${
sharpVersion
? ` (${sharpVersion})${
sharpVersion !== latestVersion ? ` - needs update` : ``
}`
: ``
}`
}),
])

if (packagesToUpdate.length > 0) {
msg = msg.concat([
``,
`If you are using npm, run:`,
``,
`npm install ${packagesToUpdate.join(` `)}`,
``,
`If you are using yarn, run:`,
``,
`yarn add ${packagesToUpdate.join(` `)}`,
])
}

return msg
// eslint-disable-next-line no-empty
} catch {
return null
}
}

const handleMessage = msg => {
if (msg.includes(`Incompatible library version: sharp.node requires`)) {
restoreConsoleError()

let msg = [
`It looks like there are multiple versions of "sharp" module installed.`,
`Please update any packages that depend on "sharp".`,
``,
]

const detailedMessage = getDetailedMessage()
if (!detailedMessage) {
msg = msg.concat([
`To get a list of installed packages that depend on "sharp" try running:`,
` - npm list sharp (if you use npm)`,
` - yarn why sharp (if you use yarn)`,
` and update packages that depend on version older than latest listed in output of above command.`,
])
} else {
msg = msg.concat(detailedMessage)
}

msg = msg.concat([
``,
`If an older version of "sharp" still persists and this error is displayed after updating your packages, open an issue in the package's repository and request them to update the "sharp" dependency.`,
])

console.error(msg.join(`\n`))
}
}

let sharp
try {
// sharp@0.22.1 uses console.error and then process.exit and doesn't throw
// so to capture error and provide meaningful troubleshooting guide
// we intercept console.error calls and add special handling.
console.error = (msg, ...args) => {
originalConsoleError(msg, ...args)
handleMessage(msg)
}
sharp = require(`sharp`)
} catch (e) {
handleMessage(e.toString())
throw e
} finally {
restoreConsoleError()
}

module.exports = sharp
31 changes: 31 additions & 0 deletions packages/gatsby-plugin-sharp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,37 @@ limited to the latitude/longitude information of where the picture was taken
can either leave `stripMetadata` to its default of `true`, or manually
pre-process your images with a tool such as [ExifTool][17].

## Troubleshooting

### Incompatible library version: sharp.node requires version X or later, but Z provides version Y

This means that there are multiple incompatible versions of the `sharp` package installed in `node_modules`. The complete error typically looks like this:

```
Something went wrong installing the "sharp" module

dlopen(/Users/misiek/dev/gatsby-starter-blog/node_modules/sharp/build/Release/sharp.node, 1): Library not loaded: @rpath/libglib-2.0.dylib
Referenced from: /Users/misiek/dev/gatsby-starter-blog/node_modules/sharp/build/Release/sharp.node
Reason: Incompatible library version: sharp.node requires version 6001.0.0 or later, but libglib-2.0.dylib provides version 5801.0.0
```

To fix this, you'll need to update all Gatsby plugins in the current project that depend on the `sharp` package. Here's a list of official plugins that you might need to update in case your projects uses them:

- `gatsby-plugin-sharp`
- `gatsby-plugin-manifest`
- `gatsby-remark-images-contentful`
- `gatsby-source-contentful`
- `gatsby-transformer-sharp`
- `gatsby-transformer-sqip`

To update these packages, run:

```sh
npm install gatsby-plugin-sharp gatsby-plugin-manifest gatsby-remark-images-contentful gatsby-source-contentful gatsby-transformer-sharp gatsby-transformer-sqip
```

If updating these doesn't fix the issue, your project probably uses other plugins from the community that depend on a different version of `sharp`. Try running `npm list sharp` or `yarn why sharp` to see all packages in the current project that use `sharp` and try updating them as well.

[1]: https://alistapart.com/article/finessing-fecolormatrix
[2]: http://blog.72lions.com/blog/2015/7/7/duotone-in-js
[3]: https://ines.io/blog/dynamic-duotone-svg-jade
Expand Down
3 changes: 2 additions & 1 deletion packages/gatsby-plugin-sharp/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@
"potrace": "^2.1.1",
"probe-image-size": "^4.0.0",
"progress": "^1.1.8",
"sharp": "^0.21.3",
"semver": "^5.6.0",
"sharp": "^0.22.1",
"svgo": "^1.2.0"
},
"devDependencies": {
Expand Down
2 changes: 1 addition & 1 deletion packages/gatsby-plugin-sharp/src/duotone.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
const sharp = require(`sharp`)
const sharp = require(`./safe-sharp`)

module.exports = async function duotone(duotone, format, pipeline) {
const duotoneGradient = createDuotoneGradient(
Expand Down
21 changes: 1 addition & 20 deletions packages/gatsby-plugin-sharp/src/index.js
Original file line number Diff line number Diff line change
@@ -1,23 +1,4 @@
try {
require(`sharp`)
} catch (error) {
// Bail early if sharp isn't available
console.error(
`
The dependency "sharp" does not seem to have been built or installed correctly.

- Try to reinstall packages and look for errors during installation
- Consult "sharp" installation page at http://sharp.pixelplumbing.com/en/stable/install/

If neither of the above work, please open an issue in https://github.com/gatsbyjs/gatsby/issues
`
)
console.log()
console.error(error)
process.exit(1)
}

const sharp = require(`sharp`)
const sharp = require(`./safe-sharp`)

const imageSize = require(`probe-image-size`)

Expand Down
2 changes: 1 addition & 1 deletion packages/gatsby-plugin-sharp/src/process-file.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
const sharp = require(`sharp`)
const sharp = require(`./safe-sharp`)
const fs = require(`fs-extra`)
const debug = require(`debug`)(`gatsby:gatsby-plugin-sharp`)
const duotone = require(`./duotone`)
Expand Down
Loading