Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gatsby-source-wordpress fails to build site #5371

Closed
scruffydan opened this issue May 11, 2018 · 21 comments · Fixed by #5375
Closed

gatsby-source-wordpress fails to build site #5371

scruffydan opened this issue May 11, 2018 · 21 comments · Fixed by #5375
Labels
type: bug An issue or pull request relating to a bug in Gatsby

Comments

@scruffydan
Copy link

I am trying to build my first gatsby powered site, using my current WordPress site (1000 posts and almost 3000 pictures) as a source for gatsby and I cannot get it to build. So far I have just the basic site generated by gatsby new and the gatsby-source-wordpress plugin. My gatsby site is published here https://github.com/scruffydan/MindofDan

Description

gatsby develop doesn't complete

This is what I get:

=END PLUGIN=====================================: 33804.250ms
error Plugin gatsby-source-wordpress returned an error


  TypeError: Cannot read property 'id' of undefined

  - normalize.js:275
    [MindofDan]/[gatsby-source-wordpress]/normalize.js:275:9

  - Array.map

  - normalize.js:270 Object.exports.mapElementsToParent
    [MindofDan]/[gatsby-source-wordpress]/normalize.js:270:19

  - gatsby-node.js:142 _callee$
    [MindofDan]/[gatsby-source-wordpress]/gatsby-node.js:142:34

  - next_tick.js:61 process._tickCallback
    internal/process/next_tick.js:61:11


⠠ source and transform nodes

I am using verboseOutput: true and concurrentRequests: 10.

It just hangs at source and transform nodes and never completes. I have left it running for 12 hours.

However I don't always get the TypeError: Cannot read property 'id' of undefined error, sometimes it just hangs at source and transform nodes.

Steps to reproduce

Clone the repo https://github.com/scruffydan/MindofDan
run npm install
run gatsby develop

Expected result

The site should build

Actual result

gatsby hangs and doesn'tcomplete

Environment

  • Gatsby version (npm list gatsby): gatsby@1.9.259
  • gatsby-cli version (gatsby --version): 1.1.52
  • Node.js version: 10.1.0
  • Operating System: macOS 10.13.4

File contents (if changed)

gatsby-config.js:

module.exports = {
  siteMetadata: {
    title: 'Gatsby Default Starter',
  },
  plugins: [
	'gatsby-plugin-react-helmet',
  	{
		resolve: 'gatsby-source-wordpress',
		options: {
			baseUrl: 'mind.ofdan.ca',
			protocol: 'https',
			concurrentRequests: 10,
			hostingWPCOM: false,
			useACF: false,
			verboseOutput: true
		}
	}
  ]
}

package.json:

{
  "name": "mind.of.dan",
  "description": "Mind of Dan built with Gatsby",
  "version": "0.1.0",
  "author": "Dan Moutal",
  "dependencies": {
    "gatsby": "^1.9.259",
    "gatsby-link": "^1.6.44",
    "gatsby-plugin-react-helmet": "^2.0.11",
    "gatsby-source-wordpress": "^2.0.84",
    "react-helmet": "^5.2.0"
  },
  "keywords": [
    "gatsby",
    "WordPress"
  ],
  "license": "MIT",
  "scripts": {
    "build": "gatsby build",
    "develop": "gatsby develop",
    "format": "prettier --write 'src/**/*.js'",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "devDependencies": {
    "prettier": "^1.12.1"
  }
}

gatsby-node.js: N/A
gatsby-browser.js: N/A
gatsby-ssr.js: N/A

Thanks in advance

@KyleAMathews
Copy link
Contributor

Can you try excluding some of the WP API routes?

@pieh
Copy link
Contributor

pieh commented May 11, 2018

After adding some debug output:

Parent element not found: wordpress__wp_comments id: 6688 parent: 6681
Parent element not found: wordpress__wp_comments id: 6687 parent: 6681
Parent element not found: wordpress__wp_comments id: 6686 parent: 6681
Parent element not found: wordpress__wp_comments id: 6685 parent: 6681
Parent element not found: wordpress__wp_comments id: 6684 parent: 6681
Parent element not found: wordpress__wp_comments id: 6683 parent: 6681
Parent element not found: wordpress__wp_comments id: 6682 parent: 6681

I will add handling of that soonish.

Getting stuck without output would almost certainly be happening while pulling images, some kind of error that we don't handle (or handle correctly), will need to investigate more

@scruffydan
Copy link
Author

Thanks for looking into my issues

What is the best/easiest way to try out the changes in these new commits?

@pieh
Copy link
Contributor

pieh commented May 14, 2018

You'll have to wait for next npm publish - so probably few days. I'll notify here when it's done, so you could update your packages

Alternatively You could try to apply changes to your local node_modules packages, but be prepared for hardly readable transpiled scripts there.

@scruffydan
Copy link
Author

I'll wait.

Thanks!

@scruffydan
Copy link
Author

Not sure of a new version has been published but I was able to run gatsby develop by excluding the comments end point. So the problem is likely there.

I also looked at my WordPress database and confirmed that there is a comment with an ID of 6681 Not sure how to square that with the debug output, but comment 6681 is a ping back so maybe thats it

@scruffydan
Copy link
Author

scruffydan commented May 16, 2018

As test I removed the comment type field for comment 6681 so Wordpress would no longer thinks its a pingback and now even with the comments API endpoint re-enabled it works.

Looks like Gatsby doesn't like pingbacks as parents. Not sure if the issue lies with the WordPress API or gatsby, but the end result isn't a pleasant experience

Regardless it works now. Time for me to figure out how to build something nice:)

@maxbaun
Copy link

maxbaun commented May 22, 2018

I was able to fix this issue by adjusting the amount of concurrent items were in the queue in the gatsby-source-filesystem plugin:

It would be nice to have an option in the gatsby-source-filesystem to adjust the amount of concurrent requests that are in the queue. That way on a lesser powerful machine, so the node process doesn't lock up.

Changing it between 1-50 concurrent requests fixed this for me.

@scruffydan
Copy link
Author

I thought adding concurrentRequests: 10 to the gatsby-source-wordpress config limited the concurrent requests?

@pieh
Copy link
Contributor

pieh commented May 28, 2018

@scruffydan there couple request queues running under the hood - wordpress plugin has queue for REST endpoints (controlled by concurrentRequests) and filesystem has separate queue for file downloading (which is currently hardcoded to 200)

@jonathanbell
Copy link

@KyleAMathews + @scruffydan How do we exclude some of the WP API routes? I've looked at not including them on the WP server side of things, but is there a way inside Gatsby?

@lightstrike
Copy link
Contributor

@jonathanbell In case you're still wondering, see the excludedRoutes option 😄

@jonathanbell
Copy link

@lightstrike Thank you!

@pmarxbraun
Copy link

Hey,
Can't build my site, I got this error

error Plugin gatsby-source-wordpress returned an error
  TypeError: Cannot read property 'id' of undefined
  - normalize.js:240 e.tags___NODE.e.tags.map.t
    [wp-tutorial-site]/[gatsby-source-wordpress]/normalize.js:240:122
  - Array.map
  - normalize.js:240 entities.map.e
    [wp-tutorial-site]/[gatsby-source-wordpress]/normalize.js:240:30
  - Array.map
  - normalize.js:235 Object.exports.mapPostsToTagsCategories.entities [as mapPostsToTagsCategories]
    [wp-tutorial-site]/[gatsby-source-wordpress]/normalize.js:235:19
  - gatsby-node.js:121 Object.<anonymous>
    [wp-tutorial-site]/[gatsby-source-wordpress]/gatsby-node.js:121:26
  - Generator.next
  - next_tick.js:77 process.internalTickCallback
    internal/process/next_tick.js:77:7

whereas it works with other base URL. Any ideas ? thanks

@jonniebigodes
Copy link

@pmarxbraun check your posts if they have tags, at least one tag and categories. based on the error you posted it looks like that's the case. Check if that's the case and once the posts are updated, issue gatsby clean to purge the data that is cached, so that you can get a "clean slate" and gatsby develop and see if the issue happens again. Sounds good?

@pmarxbraun
Copy link

@jonniebigodes thanks for your help. It does not seem to work :-( All posts have categories and tags.

error Plugin gatsby-source-wordpress returned an error


  TypeError: Cannot read property 'id' of undefined
  
  - normalize.js:240 e.tags___NODE.e.tags.map.t
    [wp-tutorial-site]/[gatsby-source-wordpress]/normalize.js:240:122
  
  - Array.map
  
  - normalize.js:240 entities.map.e
    [wp-tutorial-site]/[gatsby-source-wordpress]/normalize.js:240:30
  
  - Array.map
  
  - normalize.js:235 Object.exports.mapPostsToTagsCategories.entities [as mapPostsToTagsCategories]
    [wp-tutorial-site]/[gatsby-source-wordpress]/normalize.js:235:19
  
  - gatsby-node.js:121 Object.<anonymous>
    [wp-tutorial-site]/[gatsby-source-wordpress]/gatsby-node.js:121:26
  
  - Generator.next
  
  - next_tick.js:77 process.internalTickCallback
    internal/process/next_tick.js:77:7
  

warn The gatsby-source-wordpress plugin has generated no Gatsby nodes. Do you need it?
success source and transform nodes - 24.197 s
success building schema - 0.451 s
[ { GraphQLError: Cannot query field "allWordpressPost" on type "Query".
      at Object.Field (/Users/philippebraun/Documents/Code/Gatsby/wp-tutorial-site/node_modules/graphql/validation/rules/FieldsOnCorrectType.js:64:31)
      at Object.enter (/Users/philippebraun/Documents/Code/Gatsby/wp-tutorial-site/node_modules/graphql/language/visitor.js:332:29)
      at Object.enter (/Users/philippebraun/Documents/Code/Gatsby/wp-tutorial-site/node_modules/graphql/language/visitor.js:383:25)
      at visit (/Users/philippebraun/Documents/Code/Gatsby/wp-tutorial-site/node_modules/graphql/language/visitor.js:250:26)
      at validate (/Users/philippebraun/Documents/Code/Gatsby/wp-tutorial-site/node_modules/graphql/validation/validate.js:63:22)
      at graphqlImpl (/Users/philippebraun/Documents/Code/Gatsby/wp-tutorial-site/node_modules/graphql/graphql.js:81:50)
      at /Users/philippebraun/Documents/Code/Gatsby/wp-tutorial-site/node_modules/graphql/graphql.js:36:250
      at new Promise (<anonymous>)
      at graphql (/Users/philippebraun/Documents/Code/Gatsby/wp-tutorial-site/node_modules/graphql/graphql.js:34:10)
      at graphqlRunner (/Users/philippebraun/Documents/Code/Gatsby/wp-tutorial-site/node_modules/gatsby/dist/bootstrap/index.js:359:14)
      at Promise (/Users/philippebraun/Documents/Code/Gatsby/wp-tutorial-site/gatsby-node.js:34:5)
      at Promise._execute (/Users/philippebraun/Documents/Code/Gatsby/wp-tutorial-site/node_modules/bluebird/js/release/debuggability.js:313:9)
      at Promise._resolveFromExecutor (/Users/philippebraun/Documents/Code/Gatsby/wp-tutorial-site/node_modules/bluebird/js/release/promise.js:483:18)
      at new Promise (/Users/philippebraun/Documents/Code/Gatsby/wp-tutorial-site/node_modules/bluebird/js/release/promise.js:79:10)
      at Object.exports.createPages (/Users/philippebraun/Documents/Code/Gatsby/wp-tutorial-site/gatsby-node.js:27:10)
      at runAPI (/Users/philippebraun/Documents/Code/Gatsby/wp-tutorial-site/node_modules/gatsby/dist/utils/api-runner-node.js:219:37)
    message: 'Cannot query field "allWordpressPost" on type "Query".',
    locations: [ [Object] ] } ]
⠋ createPages
error UNHANDLED REJECTION


  TypeError: Cannot read property 'allWordpressPost' of undefined
  
  - gatsby-node.js:59 graphql.then
    /Users/philippebraun/Documents/Code/Gatsby/wp-tutorial-site/gatsby-node.js:59:26

Thanks again for you help 👍

@pmarxbraun
Copy link

@jonniebigodes Any ideas about this ? thanks again for your help

@jonniebigodes
Copy link

@pmarxbraun can you share your gatsby-config.js or at least the portion relative to gatsby-source-wordpress? I won't mind taking a look at it. I have already a Gatsby website that sources from it, that i used to tackle some issues related to some issues with wordpress and see what is happening.

@pmarxbraun
Copy link

@jonniebigodes here it is :-)

module.exports = {
  siteMetadata: {
    title: `Wordpress with Gatsby Tutorial`,
    description: `Kick off your next, great Gatsby project with this default starter. This barebones starter ships with the main Gatsby configuration files you might need.`,
    author: `@pmarxbraun`,
  },
  plugins: [
    `gatsby-plugin-react-helmet`,
    {
      resolve: `gatsby-source-wordpress`,
      options: {
        /*
         * The base URL of the WordPress site without the trailingslash and the protocol. This is required.
         * Example : 'dev-gatbsyjswp.pantheonsite.io' or 'www.example-site.com'
         */
        baseUrl: `we-chain.com`,
        // The protocol. This can be http or https.
        protocol: `https`,
        // Indicates whether the site is hosted on wordpress.com.
        // If false, then the asumption is made that the site is self hosted.
        // If true, then the plugin will source its content on wordpress.com using the JSON REST API V2.
        // If your site is hosted on wordpress.org, then set this to false.
        hostingWPCOM: false,
        // If useACF is true, then the source plugin will try to import the WordPress ACF Plugin contents.
        // This feature is untested for sites hosted on WordPress.com
        useACF: false,
      },
    },
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        name: `images`,
        path: `${__dirname}/src/images`,
      },
    },
    `gatsby-transformer-sharp`,
    `gatsby-plugin-sharp`,
    {
      resolve: `gatsby-plugin-manifest`,
      options: {
        name: `gatsby-starter-default`,
        short_name: `starter`,
        start_url: `/`,
        background_color: `#663399`,
        theme_color: `#663399`,
        display: `minimal-ui`,
        icon: `src/images/gatsby-icon.png`, // This path is relative to the root of the site.
      },
    },
    // this (optional) plugin enables Progressive Web App + Offline functionality
    // To learn more, visit: https://gatsby.dev/offline
    // `gatsby-plugin-offline`,
  ],
}

@jonniebigodes
Copy link

@pmarxbraun sorry for the wait. And i think i have a solution for your case.

What i did was:

  • Modify my gatsby-config.js to match yours and as soon as i issue gatsby develop i'm presented with the following:

braun1

braun2

With that it led me to believe that you followed probably this and this. Which is correct, but not accurate entirely. With this you'll get into some errors like you did.
So with that in mind, going back to the plugin's documentation page i adjusted the options.

  • Modified gatsby-config.js once more to allow the specific routes to be fetched, transforming it into:
module.exports = {
  /* Your site config here */
  plugins:[
    {
      resolve: `gatsby-source-wordpress`,
      options: {
        /*
         * The base URL of the WordPress site without the trailingslash and the protocol. This is required.
         * Example : 'dev-gatbsyjswp.pantheonsite.io' or 'www.example-site.com'
         */
        baseUrl: `we-chain.com`,
        // The protocol. This can be http or https.
        protocol: `https`,
        // Indicates whether the site is hosted on wordpress.com.
        // If false, then the asumption is made that the site is self hosted.
        // If true, then the plugin will source its content on wordpress.com using the JSON REST API V2.
        // If your site is hosted on wordpress.org, then set this to false.
        hostingWPCOM: false,
        // If useACF is true, then the source plugin will try to import the WordPress ACF Plugin contents.
        // This feature is untested for sites hosted on WordPress.com
        useACF: false,
        includedRoutes: [
          "**/posts",
          "**/pages",
          "**/media",
          "**/categories",
          "**/tags",
        ],
      },
    },
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        name: `images`,
        path: `${__dirname}/src/images`,
      },
    },
    `gatsby-transformer-sharp`,
    `gatsby-plugin-sharp`,
  ]
}
  • Issued gatsby clean and gatsby develop , to get a "clean slate" build and the error still. As you can see bellow:
    braun3

  • Commented out the tags endpoint, yelded the following:
    braun4

As you can see, the error was fixed.

@pmarxbraun
Copy link

@jonniebigodes Thanks a lot ! I works now :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug An issue or pull request relating to a bug in Gatsby
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants