Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ImageMagick: use separate bucket for blurred images. #1346

Merged
merged 11 commits into from
Jun 12, 2019
14 changes: 11 additions & 3 deletions functions/imagemagick/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,23 @@ Functions for your project.

1. Create a Cloud Storage Bucket:

gsutil mb gs://YOUR_BUCKET_NAME
gsutil mb gs://YOUR_INPUT_BUCKET_NAME

This storage bucket is used to upload images for the function to check.

1. Create a second Cloud Storage Bucket:

gsutil mb gs://YOUR_OUTPUT_BUCKET_NAME

This second storage bucket is used to store blurred images. (Un-blurred images will not be saved to this bucket.)

This is necessary because saving the blurred image to the input bucket would cause your function to be invoked a second time with the blurred image itself.

1. Deploy the `blurOffensiveImages` function with a Storage trigger:

gcloud functions deploy blurOffensiveImages --trigger-bucket=YOUR_BUCKET_NAME
gcloud functions deploy blurOffensiveImages --trigger-bucket=YOUR_INPUT_BUCKET_NAME --set-env-vars BLURRED_BUCKET_NAME=YOUR_OUTPUT_BUCKET_NAME

* Replace `YOUR_BUCKET_NAME` with the name of the Cloud Storage Bucket you created earlier.
* Replace `YOUR_INPUT_BUCKET_NAME` and `YOUR_OUTPUT_BUCKET_NAME` with the names of the respective Cloud Storage Buckets you created earlier.

1. Upload an offensive image to the Storage bucket, such as this image of
a flesh-eating zombie: https://cdn.pixabay.com/photo/2015/09/21/14/24/zombie-949916_1280.jpg
Expand Down
22 changes: 14 additions & 8 deletions functions/imagemagick/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ const storage = new Storage();
const vision = require('@google-cloud/vision').v1p1beta1;

const client = new vision.ImageAnnotatorClient();

const {BLURRED_BUCKET_NAME} = process.env;
// [END functions_imagemagick_setup]

// [START functions_imagemagick_analyze]
Expand Down Expand Up @@ -67,7 +69,7 @@ exports.blurOffensiveImages = event => {
console.log(
`The image ${file.name} has been detected as inappropriate.`
);
return blurImage(file);
return blurImage(file, BLURRED_BUCKET_NAME);
} else {
console.log(`The image ${file.name} has been detected as OK.`);
}
Expand All @@ -76,8 +78,8 @@ exports.blurOffensiveImages = event => {
// [END functions_imagemagick_analyze]

// [START functions_imagemagick_blur]
// Blurs the given file using ImageMagick.
function blurImage(file) {
// Blurs the given file using ImageMagick, and uploads it to another bucket.
function blurImage(file, blurredBucketName) {
const tempLocalPath = `/tmp/${path.parse(file.name).base}`;

// Download file from bucket.
Expand Down Expand Up @@ -109,19 +111,23 @@ function blurImage(file) {
.then(() => {
console.log(`Image ${file.name} has been blurred.`);

// Mark result as blurred, to avoid re-triggering this function.
const newName = `blurred-${file.name}`;
// Upload result to a different bucket, to avoid re-triggering this function.
// You can also re-upload it to the same bucket + tell your Cloud Function to
// ignore files marked as blurred (e.g. those with a "blurred" prefix)
const blurredBucket = storage.bucket(blurredBucketName);

// Upload the Blurred image back into the bucket.
return file.bucket
.upload(tempLocalPath, {destination: newName})
return blurredBucket
.upload(tempLocalPath, {destination: file.name})
.catch(err => {
console.error('Failed to upload blurred image.', err);
return Promise.reject(err);
});
})
.then(() => {
console.log(`Blurred image has been uploaded to ${file.name}.`);
console.log(
`Blurred image has been uploaded to: gs://${blurredBucketName}/${file.name}`
);

// Delete the temporary file.
return new Promise((resolve, reject) => {
Expand Down
19 changes: 6 additions & 13 deletions functions/imagemagick/test/index.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,11 @@ const tools = require('@google-cloud/nodejs-repo-tools');
const vision = require('@google-cloud/vision').v1p1beta1;

const bucketName = 'my-bucket';
const blurredBucketName = 'my-blurred-bucket';
const defaultFileName = 'image.jpg';

process.env.BLURRED_BUCKET_NAME = blurredBucketName;

let VisionStub = sinon.stub(vision, 'ImageAnnotatorClient');
VisionStub.returns({
safeSearchDetection: sinon.stub().returns(
Expand Down Expand Up @@ -122,12 +125,13 @@ it('blurOffensiveImages blurs unblurred images (Node 6 syntax)', async () => {
`Image ${sample.mocks.file.name} has been blurred.`,
]);
assert.deepStrictEqual(console.log.getCall(4).args, [
`Blurred image has been uploaded to ${sample.mocks.file.name}.`,
`Blurred image has been uploaded to: gs://${blurredBucketName}/${sample.mocks.file.name}`,
]);
});

it('blurOffensiveImages blurs unblurred images (Node 8 syntax)', async () => {
const sample = getSample(defaultFileName);

await sample.program.blurOffensiveImages({
bucket: bucketName,
name: defaultFileName,
Expand All @@ -146,18 +150,7 @@ it('blurOffensiveImages blurs unblurred images (Node 8 syntax)', async () => {
`Image ${sample.mocks.file.name} has been blurred.`,
]);
assert.deepStrictEqual(console.log.getCall(4).args, [
`Blurred image has been uploaded to ${sample.mocks.file.name}.`,
]);
});

it('blurOffensiveImages ignores already-blurred images', async () => {
const sample = getSample('blurred-${defaultFileName}');
await sample.program.blurOffensiveImages({
data: {bucket: bucketName, name: `blurred-${defaultFileName}`},
});
assert.strictEqual(console.log.callCount, 1);
assert.deepStrictEqual(console.log.getCall(0).args, [
`The image ${sample.mocks.file.name} is already blurred.`,
`Blurred image has been uploaded to: gs://${blurredBucketName}/${sample.mocks.file.name}`,
]);
});

Expand Down
5 changes: 3 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,9 @@
"eslint": "^5.9.0",
"eslint-config-prettier": "^4.0.0",
"eslint-plugin-node": "^9.0.0",
"eslint-plugin-prettier": "^3.0.0",
"eslint-plugin-prettier": "^3.1.0",
"eslint-plugin-promise": "^4.1.1",
"prettier": "^1.15.2"
"prettier": "^1.18.2",
"requestretry": "^4.0.0"
}
}