Not refreshing access token automatically

[calculuschild]

Environment details

• OS: Windows 10 64x
• Node.js version: 12.16.3
• npm version: 6.14.4
• googleapis version: 59.0.0

Steps to reproduce

I expect the API to automatically refresh the access token on the next API call after it expires, but I only get an "Invalid Client" error. I can successfully access the API and upload/download google drive files until the access token expires, so I know the tokens I received from the API on first signin are valid. I have that same refresh token assigned to the oauth2client immediately before any request. I can verify that they are set via console.log. Is there some missing step to get the API to "automatically refresh the access token" as it says in the documentation?

	oAuth2Client.setCredentials({
		access_token: googleAccessToken,
		refresh_token: googleRefreshToken
  });

[JustinBeckwith]

👋 The invalid client error sounds suspect. Can you share:

• The code you're using to instantiate the GoogleAuth client or OAuth2Client
• The exact stack trace you're seeing

[calculuschild]

Sure thing. Let me know if you need more details:

The code for my OAuth2Client:

let oAuth2Client = new google.auth.OAuth2(
	config.googleClientId,
	config.googleClientSecret,
	'/auth/google/redirect'
);

oAuth2Client.on('tokens', (tokens) => {
	console.log("ON TOKENS"); //<-- This is never reached
	if (tokens.refresh_token) {
		// store the refresh_token in my database!
		console.log("Refresh Token: " + tokens.refresh_token);
	}
	console.log("New Access Token: " + tokens.access_token);
});

.... later in the same function....

oAuth2Client.setCredentials({
	access_token: account.googleAccessToken,
	refresh_token: account.googleRefreshToken
});

console.log("Access Token: " + account.googleAccountToken); //Works fine
console.log("Refresh Token: " + account.googleRefreshToken); //Works fine

const drive = google.drive({version: 'v3', auth: oAuth2Client});

fileMetadata = {
	'name': brew.title + '.txt',
	'appProperties': {
		'shareId': brew.shareId,
		'editId' : brew.editId,
		'title'  : brew.title,
	}
};

media = {
	mimeType: 'text/plain',
	body: brew.text
};

drive.files.create({
	resource: fileMetadata,
	media: media
})
.then(res => {
	console.log("Success!"); // Only reaches here on a fresh login
	console.log(res.data.id);
	return res.status(200).send(res.data);
})
.catch(err => {
	console.log("WHYYYYYY"); // Reaches here after 1 hour. Refresh token not working????!?!
	console.error(err);
	//return res.status(500).send('Error while saving');
});

Here's the full error I get:

WHYYYYYYY
GaxiosError: Invalid Credentials
    at Gaxios._request (C:\Users\thato\Documents\homebrewery\node_modules\gaxios\build\src\gaxios.js:89:23)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async OAuth2Client.requestAsync (C:\Users\thato\Documents\homebrewery\node_modules\google-auth-library\build\src\auth\oauth2client.js:343:18) {
  response: {
    config: {
      url: 'https://www.googleapis.com/upload/drive/v3/files?uploadType=multipart',
      method: 'POST',
      userAgentDirectives: [Array],
      paramsSerializer: [Function],
      data: [PassThrough],
      headers: [Object],
      params: [Object: null prototype],
      validateStatus: [Function],
      retry: true,
      body: [PassThrough],
      responseType: 'json',
      retryConfig: [Object]
    },
    data: { error: [Object] },
    headers: {
      'alt-svc': 'h3-29=":443"; ma=2592000,h3-27=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-T050=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"',
      connection: 'close',
      'content-length': '249',
      'content-type': 'application/json; charset=UTF-8',
      date: 'Fri, 04 Sep 2020 01:46:29 GMT',
      server: 'UploadServer',
      vary: 'Origin, X-Origin',
      'www-authenticate': 'Bearer realm="https://accounts.google.com/", error=invalid_token',
      'x-guploader-uploadid': 'ABg5-UxxOru2I9OyxWCFXebrqdW8XnV-FOL2kXiyEUdhUuEMY9mlCiQfOB19355LDnUVa1EkMdHWgj4X60LS0IX-ylk'
    },
    status: 401,
    statusText: 'Unauthorized',
    request: {
      responseURL: 'https://www.googleapis.com/upload/drive/v3/files?uploadType=multipart'
    }
  },
  config: {
    url: 'https://www.googleapis.com/upload/drive/v3/files?uploadType=multipart',
    method: 'POST',
    userAgentDirectives: [ [Object] ],
    paramsSerializer: [Function],
    data: PassThrough {
      _readableState: [ReadableState],
      readable: false,
      _events: [Object: null prototype],
      _eventsCount: 2,
      _maxListeners: undefined,
      _writableState: [WritableState],
      writable: true,
      allowHalfOpen: true,
      _transformState: [Object],
      _flush: [Function: flush],
      [Symbol(kCapture)]: false
    },
    headers: {
      'x-goog-api-client': 'gdcl/4.4.0 gl-node/12.16.3 auth/6.0.6',
      'content-type': 'multipart/related; boundary=b6356565-0b2d-49cb-96c2-2bd8d442797a',
      'Accept-Encoding': 'gzip',
      'User-Agent': 'google-api-nodejs-client/4.4.0 (gzip)',
      Authorization: 'Bearer ya29.a0AfH6SMDIwG_SasM8QlX3zqGHf8X9Tj71pfvRN7L0jemdUxb4JGZv4TuHhDRwEJr5sVQZGQ4nnibEKpL4xBjjWEVSuLXSIyD5E_vBcOK18NQzR_SYXylQ36WoIj83ZqPXkUjsPEL3cWQiQ-j5dQ7qNx9965KUSu4xa7s',
      Accept: 'application/json'
    },
    params: [Object: null prototype] { uploadType: 'multipart' },
    validateStatus: [Function],
    retry: true,
    body: PassThrough {
      _readableState: [ReadableState],
      readable: false,
      _events: [Object: null prototype],
      _eventsCount: 2,
      _maxListeners: undefined,
      _writableState: [WritableState],
      writable: true,
      allowHalfOpen: true,
      _transformState: [Object],
      _flush: [Function: flush],
      [Symbol(kCapture)]: false
    },
    responseType: 'json',
    retryConfig: {
      currentRetryAttempt: 0,
      retry: 3,
      httpMethodsToRetry: [Array],
      noResponseRetries: 2,
      statusCodesToRetry: [Array]
    }
  },
  code: 401,
  errors: [
    {
      domain: 'global',
      reason: 'authError',
      message: 'Invalid Credentials',
      locationType: 'header',
      location: 'Authorization'
    }
  ]
}

[calculuschild]

Followup:

Oddly enough, if I deliberately do NOT include the access token in the oAuth2Client, the refresh appears to happen just fine. A bug? Got the idea from this stack exchange thread but the guy seems just as confused as I am.

oAuth2Client.setCredentials({
	// COMMENT OUT access_token: account.googleAccessToken,
	refresh_token: account.googleRefreshToken
});

ON TOKENS
New Access Token: y**********************************************

But... sending only the refresh token and refreshing the access token with every single request seems contrary to the intended design? I could probably check the clock before each request and not send the access_token when I expect it is about to expire, but doesn't this defeat the purpose of the "automatic refresh"?

"This library will automatically use a refresh token to obtain a new access token if it is about to expire."
"Once the client has a refresh token, access tokens will be acquired and refreshed automatically in the next call to the API."

[calculuschild]

Is there a preferred workaround until this is fixed?

[JustinBeckwith]

Hey folks, we haven't had a chance to dig in and see what's happening here. I don't think anything interesting has changed in the auth stack recently, so it requires some sleuthing.

[calculuschild]

Further update: I seem to only run into this issue using the google drive.files.create() function, and the oauth2Client.on('tokens'... is never called when I do. Other calls such as files.get() and files.list() refresh the access token just fine however.

[sandinmyjoints]

@calculuschild From https://github.com/googleapis/google-auth-library-nodejs#oauth2

This library comes with an OAuth2 client that allows you to retrieve an access token and refreshes the token and retry the request seamlessly if you also provide an expiry_date and the token is expired.

(emphasis mine)

Looking at the code you pasted originally:

oAuth2Client.setCredentials({
		access_token: googleAccessToken,
		refresh_token: googleRefreshToken
  });

You provided access_token but not expiry_date (which is available on the tokens argument pass to the tokens event handler). If you provide expiry_date, does it refresh as expected?

[tinyCoder32]

We have the same issue here, we've been struggling for days. oauth2Client.on('tokens'... is never called. We also tried passing expiry_date and didn't work.

[rizwanwalayat]

Having the same problem. Commenting out the access_token doesn't work as well.
Digging further, it gives an error of "invalid_client" & "Oauth2 client not found" when its apparently attempting to get a new accessToken by calling "https://oauth2.googleapis.com/token" and passing the refreshToken.
Can someone please help or fix this issue?

[rizwanwalayat]

I'm trying to get the expiry_date and pass it with the tokens as someone suggested. But using PassportJS, I'm not getting that with the tokens. I'm trying to decode the AccessToken to get the expiry_date using some jwt libraries (e.g. jsonwebtokens) but its not working. Can someone point me in the right direction on how to achieve this?

[HarrisonJackson]

We are seeing this issue with calendar.events.insert, too. Our temporary workaround is to call the deprecated refreshAccessTokens method each time.

oAuth2Client.refreshAccessToken((error, tokens) => {
  console.log('refreshAccessToken', tokens);
})

[Zircoz]

Apparently we are also having this issue here https://www.github.com/shriyamadan/youtube-dashboard

[kazuooooo]

In my case, refresh token is expired in 1 week, because my app is in testing status

https://developers.google.com/identity/protocols/oauth2#expiration

[viniferrareze]

someone found a solution to this problem?

[bcoe]

@calculuschild could you please provide a snippet of the code you're writing that's failing to refresh?

I just tested with the following example:

const {google} = require('googleapis');
const people = google.people('v1');

/**
 * To use OAuth2 authentication, we need access to a a CLIENT_ID, CLIENT_SECRET, AND REDIRECT_URI.  To get these credentials for your application, visit https://console.cloud.google.com/apis/credentials.
 */
const keyPath = path.join(__dirname, 'oauth2.keys.json');
let keys = {redirect_uris: ['']};
if (fs.existsSync(keyPath)) {
  keys = require(keyPath).web;
}

/**
 * Create a new OAuth2 client with the configured keys.
 */
const oauth2Client = new google.auth.OAuth2(
  keys.client_id,
  keys.client_secret,
  keys.redirect_uris[0]
);

oauth2Client.on('tokens', (tokens) => {
  console.info('token event:');
  console.info(tokens);
});

oauth2Client.credentials = {
  refresh_token
}

/**
 * This is one of the many ways you can configure googleapis to use authentication credentials.  In this method, we're setting a global reference for all APIs.  Any other API you use here, like google.drive('v3'), will now use this auth client. You can also override the auth client at the service and method call levels.
 */
google.options({auth: oauth2Client});

async function runSample() {
  // retrieve user profile
  const res = await people.people.get({
    resourceName: 'people/me',
    personFields: 'emailAddresses'
  });
  console.log(res.data);
}

const scopes = [
  'https://www.googleapis.com/auth/contacts.readonly',
  'https://www.googleapis.com/auth/user.emails.read',
  'profile'
];
setInterval(() => {
  runSample().catch((err) => {
    console.info(err);
    process.exit(1);
  });
}, 10000);

[bcoe]

@viniferrareze @kazuooooo @calculuschild if the issue is that your app is in Testing status, follow the steps to make the app production:

Or, while testing you may need to perform a new login when testing, to get a new refresh token periodically.

[sofisl]

@calculuschild, I've attempted to recreate your steps here. I haven't been able to reproduce your issue. See my code below:

const keyPath = 'KEYPATH';
let keys = {redirect_uris: ['']};
if (fs.existsSync(keyPath)) {
  keys = require(keyPath).web;
}

/**
 * Create a new OAuth2 client with the configured keys.
 */
const oauth2Client = new google.auth.OAuth2(
  keys.client_id,
  keys.client_secret,
  keys.redirect_uris[0]
);

google.options({auth: oauth2Client});

async function authenticate(scopes) {
  return new Promise((resolve, reject) => {
    // grab the url that will be used for authorization
    const authorizeUrl = oauth2Client.generateAuthUrl({
      access_type: 'offline',
      scope: scopes.join(' '),
    });

    oauth2Client.on('tokens', (tokens) => {
      console.log(tokens);
      if (tokens.refresh_token) {
        // store the refresh_token in my database!
        console.log(`REFRESH TOKEN: ${tokens.refresh_token}`);
      }
      console.log(`ACCESS TOKEN: ${tokens.access_token}`);
    });
    const server = http
      .createServer(async (req, res) => {
        try {
          if (req.url.indexOf('/oauth2callback') > -1) {
            const qs = new url.URL(req.url, 'http://localhost:3000')
              .searchParams;
            res.end('Authentication successful! Please return to the console.');
            server.destroy();
            const {tokens} = await oauth2Client.getToken(qs.get('code'));
            oauth2Client.credentials = tokens; // eslint-disable-line require-atomic-updates
            resolve(oauth2Client);
          }
        } catch (e) {
          reject(e);
        }
      })
      .listen(3000, () => {
        // open the browser to the authorize url to start the workflow
        opn(authorizeUrl, {wait: false}).then(cp => cp.unref());
      });
    destroyer(server);
  });
}

async function runSample() {
  const drive = google.drive({version: 'v3', auth: oauth2Client});

  const fileMetadata = {
    'name': 'name'
  };
  
  const media = {
    mimeType: 'text/plain',
  };
  
  drive.files.create({
    resource: fileMetadata,
    media: media
  })
  .then(res => {
    console.log("Success!"); // Only reaches here on a fresh login
    console.log(res.data.id);
  })
  .catch(err => {
    console.error(err);
    //return res.status(500).send('Error while saving');
  });
}

const scopes = [
  'https://www.googleapis.com/auth/drive',
  'https://www.googleapis.com/auth/drive.file',
  'https://www.googleapis.com/auth/drive.appdata',
];
 authenticate(scopes)
   .then(client => runSample(client))
   .catch(console.error);

Once I get the access token and refresh token, I can call the sample again just fine:

async function main() {
  oauth2Client.setCredentials({
    access_token: //access token I printed out earlier
    refresh_token: //refresh token I printed out earlier
  });
  
  runSample();

};

 main().catch(console.error);

A couple of points of friction I had which might be useful to this thread:

1. You may not be hitting the tokens event because it only appears in the first authorization (I had to revoke access my app already had to get the refresh token again)
2. You need to set access type to offline when generating the Auth URL:

const authorizeUrl = oauth2Client.generateAuthUrl({
      access_type: 'offline',
      scope: scopes.join(' '),
    });

3. Make sure your scopes are set correctly, the ones required for creating a file are:

const scopes = [
  'https://www.googleapis.com/auth/drive',
  'https://www.googleapis.com/auth/drive.file',
  'https://www.googleapis.com/auth/drive.appdata',
];

I'm going to be closing for now, but feel free to reopen if you keep running into issues!

[madaher-dev]

How does this solve the issue if you did not change anything from the code he wrote?

[localnerve]

If you are reading this thread b/c you are getting a 403 sporadically, it's likely not an access token issue, but a service policy issue you are running into.
In my case, using Google Drive API, I ran into 403s making too many requests (too quickly) at once, and was solved by serially streaming the files. Example: https://github.com/localnerve/google-drive-folder

[madaher-dev]

Thank you. Actually it is more of the API refusing to refresh if the token was binded to the oAuthClient. If i only send the refresh token it works fine. Exactly the scenario happening with the poster. I am not adding the expiry date so that might be the issue i am assuming. Calling the api with a refresh token suits my usecase as anyway my API calls are not as frequent as the token life but was also having same concerns as the poster. Very weird why this is not well documented with the package.

[JustinBeckwith]

@madaher-dev this stuff can be a little more complicated than we'd all like. If you'd help, can I trouble you to create a new github issue, with an example of the code that you're trying, and the results?

[localnerve]

Thank you. Actually it is more of the API refusing to refresh if the token was binded to the oAuthClient. If i only send the refresh token it works fine. Exactly the scenario happening with the poster. I am not adding the expiry date so that might be the issue i am assuming. Calling the api with a refresh token suits my usecase as anyway my API calls are not as frequent as the token life but was also having same concerns as the poster. Very weird why this is not well documented with the package.

Two GoogleAuth clientOptions you might benefit from:

• forceRefreshOnFailure
• eagerRefreshThresholdMillis

[madaher-dev]

I can verify that forceRefreshOnFailure fixed the issue
Thank you guys

oauth2Client.setCredentials({
access_token: process.env.GOOGLE_ACCESS_TOKEN,
refresh_token: process.env.GOOGLE_REFRESH_TOKEN,
forceRefreshOnFailure: true
});

[rubek-joshi]

I am on typescript and I cannot put the forceRefreshOnFailure: true in the setCredentials method. Am I suppose to get around this by ignoring ts checks?

[madaher-dev]

Try sending only the refresh token without the actual token.
oAuth2Client.setCredentials({
//access_token: googleAccessToken, -->Remove this
refresh_token: googleRefreshToken
});
It worked with me as mentioned by the author.

[onuriltan]

Try sending only the refresh token without the actual token. oAuth2Client.setCredentials({ //access_token: googleAccessToken, -->Remove this refresh_token: googleRefreshToken }); It worked with me as mentioned by the author.

This one gives me unauthorized_client error

[aberba]

I notice because my consent screen isn't yet verified, the token tend to expire unexpectedly sometimes. It's quiet unpredictable.

With that said, here are some options to debug:

• Set on refresh_token in oAuth2Client.setCredentials as mentioned above. The access_token is not meant to be stored anyways, only store the refresh token.

oauth2Client.setCredentials({
    // remove this
    //access_token: googleAccessToken, 
    
    refresh_token: googleRefreshToken
});

• Google API is failing..happens sometimes.

• Get your OAuth screen verified

• user has revoked their access or user has been inactive for 6 month per Google policy.