Strapi warning 'uploadStream function' and upload error #43

brunoshine · 2022-05-14T09:09:23Z

Hi,

I'm starting a fresh build of a strapi instance and have added the strapi-provider-upload-azure-storage and made the changes to the plugins.js file.
When starting strapi with npm run develop I get the following warning:
Warning: The upload provider "strapi-provider-upload-azure-storage" doesn't implement the uploadStream function. Strapi will fallback on the upload method. Some performance issues may occur.

When I try to upload a file I get:
error: <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <Error> <Code>InvalidOperation</Code> <Message>Invalid storage account. RequestId:11864a50-8c43-49bf-b6a9-1a42fdd24d6f Time:2022-05-14T09:01:37.323Z</Message> </Error> Error: <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <Error> <Code>InvalidOperation</Code> <Message>Invalid storage account. RequestId:11864a50-8c43-49bf-b6a9-1a42fdd24d6f Time:2022-05-14T09:01:37.323Z</Message> </Error> at new RestError (/home/bfigueir/_work/afs/strapi_bo/afs/node_modules/@azure/ms-rest-js/dist/msRest.node.js:2480:28) at /home/bfigueir/_work/afs/strapi_bo/afs/node_modules/@azure/ms-rest-js/dist/msRest.node.js:3687:37 at processTicksAndRejections (node:internal/process/task_queues:96:5)

I've tried with both local account (via azurite) and azure account. I've tested the credentials (for both) using Azure Storage Explorer with success.

Any thoughts? Thanks

strapi-provider-upload-azure-storage: ^2.0.0

Strapi:
Version │ 4.1.9 (node v16.10.0) │
Edition │ Community

The text was updated successfully, but these errors were encountered:

jakeFeldman · 2022-05-16T04:16:25Z

Hello @brunoshine I haven't been actively following Strapi changes recently. It looks like there might have been some updates that require a change.

Though the error you have posted makes me think that there might be a config issue. Specifically the Message field's value Invalid storage account. Are you getting the same error message when using Azure storage vs Azurite?

t3custom · 2022-05-24T04:46:41Z

Hi @jakeFeldman - I'm getting the same error as @brunoshine.

(node:72272) Warning: The upload provider "strapi-provider-upload-azure-storage" doesn't implement the uploadStream function. Strapi will fallback on the upload method. Some performance issues may occur. at createProvider (/Users/johnjarrard/Desktop/t3-strapi/node_modules/@strapi/plugin-upload/server/register.js:58:13) at Object.module.exports [as register] (/Users/johnjarrard/Desktop/t3-strapi/node_modules/@strapi/plugin-upload/server/register.js:11:38) at Object.register (/Users/johnjarrard/Desktop/t3-strapi/node_modules/@strapi/strapi/lib/core/domain/module/index.js:47:46) at Object.register (/Users/johnjarrard/Desktop/t3-strapi/node_modules/@strapi/strapi/lib/core/registries/modules.js:33:19) at async Strapi.runLifecyclesFunctions (/Users/johnjarrard/Desktop/t3-strapi/node_modules/@strapi/strapi/lib/Strapi.js:479:5) at async Strapi.register (/Users/johnjarrard/Desktop/t3-strapi/node_modules/@strapi/strapi/lib/Strapi.js:350:5) at async Strapi.load (/Users/johnjarrard/Desktop/t3-strapi/node_modules/@strapi/strapi/lib/Strapi.js:425:5) at async Strapi.start (/Users/johnjarrard/Desktop/t3-strapi/node_modules/@strapi/strapi/lib/Strapi.js:169:9)

Here's my configuration:

module.exports = ({ env }) => ({ upload: { config: { provider: 'strapi-provider-upload-azure-storage', providerOptions: { account: env('STORAGE_ACCOUNT'), accountKey: env('STORAGE_ACCOUNT_KEY'), containerName: env('STORAGE_CONTAINER_NAME'), defaultPath: 'assets', maxConcurrent: 10 } } } });

Versions:
@strapi/strapi: 4.1.11
strapi-provider-upload-azure-storage: ^2.0.0

Please let me know if my configuration looks wrong.

jakeFeldman · 2022-05-25T00:54:51Z

@t3custom Hi thanks for reporting. The config looks correct.

Can you confirm that images are/are not being uploaded?

The console output shows this as a warning. Warning: The upload provider "strapi-provider-upload-azure-storage" doesn't implement the uploadStream function. and from the strapi docs upload should still be working.

💡 TIP
For performance reasons, the upload plugin will only use the uploadStream function if it exists, otherwise it will fallback on the upload function.

To be transparent I haven't tested with the newest version of strapi. I would welcome a PR to implement the uploadStream method.

t3custom · 2022-05-26T19:20:55Z

Hi @jakeFeldman - Yes I can confirm that the images are being uploaded. I'll mess with implementing when I get through this project.

jakeFeldman · 2022-06-02T03:18:28Z

Awesome! Thanks @t3custom!

moeidsaleem · 2022-06-20T15:39:04Z

Hi - Any update regarding this? i am also getting broken thumbnails. I think it has to do with the path of thumbnail

Francescoliverio · 2022-06-21T17:19:55Z

Dear @jakeFeldman,

I'm using your plugin with the new version of Strapi v4.2.0 that allows you to update a custom logo in the Strapi Dashboard and I was struggling with a response error 500 from Strapi.

Please take a look inside the Chagelog ( https://strapi.io/blog/announcing-strapi-v4.2 )

At this point I found a workaround for fix this issue but could be nice if you can please update the library with the function uploadStream and implement the logic to convert a Stream to Buffer.

By the way this is what I did for fixing the error and to be capable to Update my custom logo on Strapi v4.2.0, let's start! 🚀

This is the Strapi Console error:

At this point I implemented this code inside the file located in this path ./src/index.js:

I created a function for converting a Stream to a Buffer with this code:

function stream2buffer(stream) {
  return new Promise((resolve, reject) => {
    const _buf = [];

    stream.on('data', (chunk) => _buf.push(chunk));
    stream.on('end', () => resolve(Buffer.concat(_buf)));
    stream.on('error', (err) => reject(err));
  });
}

I implemented this code before runs the application inside the bootstrap:

bootstrap({ strapi }) {
    /**
     * Temporary workaround for strapi-provider-upload-azure-storage
     */
    strapi.plugins.upload.provider.uploadStream = async (file) => {
      file.buffer = await stream2buffer(file.stream);

      return strapi.plugins.upload.provider.upload(file);
    };
  },

This is the all code in the context:

'use strict';

function stream2buffer(stream) {
  return new Promise((resolve, reject) => {
    const _buf = [];

    stream.on('data', (chunk) => _buf.push(chunk));
    stream.on('end', () => resolve(Buffer.concat(_buf)));
    stream.on('error', (err) => reject(err));
  });
}

module.exports = {
  /**
   * An asynchronous register function that runs before
   * your application is initialized.
   *
   * This gives you an opportunity to extend code.
   */
  register(/*{ strapi }*/) {},

  /**
   * An asynchronous bootstrap function that runs before
   * your application gets started.
   *
   * This gives you an opportunity to set up your data model,
   * run jobs, or perform some special logic.
   *
   */
  bootstrap({ strapi }) {
    /**
     * Temporary workaround for strapi-provider-upload-azure-storage
     */
    strapi.plugins.upload.provider.uploadStream = async (file) => {
      file.buffer = await stream2buffer(file.stream);

      return strapi.plugins.upload.provider.upload(file);
    };
  },
};

Of course this is a workaround but still I fixed my problem with the new version of Strapi v4.2.0

I hope to help someone that has the same issue but please @jakeFeldman 🙏 update the library for removing the Warning and fix the issue because this uploadStream has become mandatory if you want to run Strapi v4.2.0 in Azure Cloud

Best wishes and thank you for did this integration!

Regards
Francesco 🇮🇹

Brocy · 2022-06-22T11:32:57Z

Dear @jakeFeldman

I reinforce Francesco's request, thanks in advance, and thanks for the plugin.

Regards
Brocy

SteveScott · 2022-06-22T14:17:40Z

But that is just a warning? It recovers? You can still use the plugin right?

…

On Tue, May 24, 2022 at 12:46 AM t3custom ***@***.***> wrote: Hi @jakeFeldman <https://github.com/jakeFeldman> - I'm getting the same error as @brunoshine <https://github.com/brunoshine>. (node:72272) Warning: The upload provider "strapi-provider-upload-azure-storage" doesn't implement the uploadStream function. Strapi will fallback on the upload method. Some performance issues may occur. at createProvider ***@***.***/plugin-upload/server/register.js:58:13) at Object.module.exports [as register] ***@***.***/plugin-upload/server/register.js:11:38) at Object.register ***@***.***/strapi/lib/core/domain/module/index.js:47:46) at Object.register ***@***.***/strapi/lib/core/registries/modules.js:33:19) at async Strapi.runLifecyclesFunctions ***@***.***/strapi/lib/Strapi.js:479:5) at async Strapi.register ***@***.***/strapi/lib/Strapi.js:350:5) at async Strapi.load ***@***.***/strapi/lib/Strapi.js:425:5) at async Strapi.start ***@***.***/strapi/lib/Strapi.js:169:9) Here's my configuration: module.exports = ({ env }) => ({ upload: { config: { provider: 'strapi-provider-upload-azure-storage', providerOptions: { account: env('STORAGE_ACCOUNT'), accountKey: env('STORAGE_ACCOUNT_KEY'), containerName: env('STORAGE_CONTAINER_NAME'), defaultPath: 'assets', maxConcurrent: 10 } } } }); Versions: @strapi/strapi: 4.1.11 strapi-provider-upload-azure-storage: ^2.0.0 Please let me know if my configuration looks wrong. — Reply to this email directly, view it on GitHub <#43 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABHLFSKREZWVOZ2XAM5MP6LVLRNLXANCNFSM5V5JPQKA> . You are receiving this because you are subscribed to this thread.Message ID: <jakeFeldman/strapi-provider-upload-azure-storage/issues/43/1135402981 @github.com>

-- From the personal email account of Steve Scott.

Francescoliverio · 2022-06-22T15:21:40Z

But that is just a warning? It recovers? You can still use the plugin right?
…
On Tue, May 24, 2022 at 12:46 AM t3custom @.> wrote: Hi @jakeFeldman https://github.com/jakeFeldman - I'm getting the same error as @brunoshine https://github.com/brunoshine. (node:72272) Warning: The upload provider "strapi-provider-upload-azure-storage" doesn't implement the uploadStream function. Strapi will fallback on the upload method. Some performance issues may occur. at createProvider @./plugin-upload/server/register.js:58:13) at Object.module.exports [as register] @./plugin-upload/server/register.js:11:38) at Object.register @./strapi/lib/core/domain/module/index.js:47:46) at Object.register @./strapi/lib/core/registries/modules.js:33:19) at async Strapi.runLifecyclesFunctions @./strapi/lib/Strapi.js:479:5) at async Strapi.register @./strapi/lib/Strapi.js:350:5) at async Strapi.load @./strapi/lib/Strapi.js:425:5) at async Strapi.start @.***/strapi/lib/Strapi.js:169:9) Here's my configuration: module.exports = ({ env }) => ({ upload: { config: { provider: 'strapi-provider-upload-azure-storage', providerOptions: { account: env('STORAGE_ACCOUNT'), accountKey: env('STORAGE_ACCOUNT_KEY'), containerName: env('STORAGE_CONTAINER_NAME'), defaultPath: 'assets', maxConcurrent: 10 } } } }); Versions: @strapi/strapi: 4.1.11 strapi-provider-upload-azure-storage: ^2.0.0 Please let me know if my configuration looks wrong. — Reply to this email directly, view it on GitHub <#43 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHLFSKREZWVOZ2XAM5MP6LVLRNLXANCNFSM5V5JPQKA . You are receiving this because you are subscribed to this thread.Message ID: </issues/43/1135402981 @github.com>
-- From the personal email account of Steve Scott.

Hey @SteveScott!
Yes of course you can use it!

The Media logic upload work perfectly!

I only had some problems with the new version of Strapi 4.2.0 that has inside the release a cool feature for changing the Logo inside the Dashboard and for doing it it use a uploadStream function that is not present inside the library of @jakeFeldman

This is the context from Strapi Dashboard:

Please take a look here:

https://github.com/jakeFeldman/strapi-provider-upload-azure-storage/blob/master/lib/index.js

in this point of the code, from line 57:

return {
            upload: file => new Promise((resolve, reject) => {
                const fileName = file.hash + file.ext;
                const containerWithPath = Object.assign({}, containerURL);
                containerWithPath.url += file.path ? `/${file.path}` : `/${config.defaultPath}`;

                const blobURL = BlobURL.fromContainerURL(containerWithPath, fileName);
                const blockBlobURL = BlockBlobURL.fromBlobURL(blobURL);

                file.url = cdnBaseURL 
                    ? blobURL.url.replace(serviceBaseURL, cdnBaseURL)
                    : blobURL.url;

                return uploadStreamToBlockBlob(
                    Aborter.timeout(60 * 60 * 1000),
                    new BufferStream(file.buffer), blockBlobURL,
                    4 * 1024 * 1024,
                    ~~(config.maxConcurent) || 20,
                    {
                        blobHTTPHeaders: {
                            blobContentType: file.mime
                        }
                    }
                ).then(resolve, reject);
            }),
            delete: file => new Promise((resolve, reject) => {
                let fileUrl = file.url;
                if (cdnBaseURL) {
                    fileUrl = fileUrl.replace(cdnBaseURL, serviceBaseURL);
                }
                const _temp = fileUrl.replace(containerURL.url, '');
                const pathParts = _temp.split('/').filter(x => x.length > 0);
                const fileName = pathParts.splice(pathParts.length - 1, 1);
                const containerWithPath = Object.assign({}, containerURL);
                containerWithPath.url += '/' + pathParts.join('/');

                const blobURL = BlobURL.fromContainerURL(containerWithPath, fileName);
                const blockBlobURL = BlockBlobURL.fromBlobURL(blobURL);

                return blockBlobURL.delete().then(resolve, err => reject(err));
            }),
        };

The function uploadStream is not defined.
And could be nice to have it for not having problems and warning anymore.

I hope I have been as clear as possible!

Regards
Francesco 🇮🇹

jakeFeldman · 2022-06-22T18:09:47Z

Version 2.1.0 has been published w/ support for uploadStream Thanks @hahabsw for the PR!

jakeFeldman added the enhancement New feature or request label Jun 2, 2022

Francescoliverio mentioned this issue Jun 22, 2022

Azure Cloud - strapi-provider-upload-azure-storage - error: strapi.plugin(...).provider.uploadStream is not a function - SOLUTIONS strapi/strapi#13613

Closed

hahabsw mentioned this issue Jun 22, 2022

Add uploadStream method. #50

Merged

jakeFeldman closed this as completed Jun 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Strapi warning 'uploadStream function' and upload error #43

Strapi warning 'uploadStream function' and upload error #43

brunoshine commented May 14, 2022

jakeFeldman commented May 16, 2022

t3custom commented May 24, 2022

jakeFeldman commented May 25, 2022

t3custom commented May 26, 2022

jakeFeldman commented Jun 2, 2022

moeidsaleem commented Jun 20, 2022

Francescoliverio commented Jun 21, 2022 •

edited

Loading

Brocy commented Jun 22, 2022

SteveScott commented Jun 22, 2022 via email

Francescoliverio commented Jun 22, 2022

jakeFeldman commented Jun 22, 2022

Strapi warning 'uploadStream function' and upload error #43

Strapi warning 'uploadStream function' and upload error #43

Comments

brunoshine commented May 14, 2022

jakeFeldman commented May 16, 2022

t3custom commented May 24, 2022

jakeFeldman commented May 25, 2022

t3custom commented May 26, 2022

jakeFeldman commented Jun 2, 2022

moeidsaleem commented Jun 20, 2022

Francescoliverio commented Jun 21, 2022 • edited Loading

Brocy commented Jun 22, 2022

SteveScott commented Jun 22, 2022 via email

Francescoliverio commented Jun 22, 2022

jakeFeldman commented Jun 22, 2022

Francescoliverio commented Jun 21, 2022 •

edited

Loading