Skip to content
This repository was archived by the owner on Aug 30, 2021. It is now read-only.

Save profile images to Amazon S3 #1857

Merged
merged 4 commits into from
Sep 19, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,15 @@ $ docker run -p 27017:27017 -d --name db mongo
$ docker run -p 3000:3000 --link db:db_1 mean
```

### Amazon S3 configuration

To save the profile images to S3, simply set those environment variables:
UPLOADS_STORAGE: s3
S3_BUCKET: the name of the bucket where the images will be saved
S3_ACCESS_KEY_ID: Your S3 access key
S3_SECRET_ACCESS_KEY: Your S3 access key password


## Getting Started With MEAN.JS
You have your application running, but there is a lot of stuff to understand. We recommend you go over the [Official Documentation](http://meanjs.org/docs.html).
In the docs we'll try to explain both general concepts of MEAN components and give you some guidelines to help you improve your development process. We tried covering as many aspects as possible, and will keep it updated by your request. You can also help us develop and improve the documentation by checking out the *gh-pages* branch of this repository.
Expand Down
9 changes: 9 additions & 0 deletions config/env/default.js
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,16 @@ module.exports = {
illegalUsernames: ['meanjs', 'administrator', 'password', 'admin', 'user',
'unknown', 'anonymous', 'null', 'undefined', 'api'
],
aws: {
s3: {
accessKeyId: process.env.S3_ACCESS_KEY_ID,
secretAccessKey: process.env.S3_SECRET_ACCESS_KEY,
bucket: process.env.S3_BUCKET
}
},
uploads: {
// Storage can be 'local' or 's3'
storage: process.env.UPLOADS_STORAGE || 'local',
profile: {
image: {
dest: './modules/users/client/img/profile/uploads/',
Expand Down
2 changes: 1 addition & 1 deletion modules/core/client/views/header.client.view.html
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
<ul class="nav navbar-nav navbar-right" ng-show="vm.authentication.user">
<li class="dropdown" uib-dropdown>
<a class="dropdown-toggle user-header-dropdown-toggle" uib-dropdown-toggle role="button">
<img ng-src="/{{vm.authentication.user.profileImageURL}}" alt="{{vm.authentication.user.displayName}}" class="header-profile-image" />
<img ng-src="{{vm.authentication.user.profileImageURL}}" alt="{{vm.authentication.user.displayName}}" class="header-profile-image" />
<span ng-bind="vm.authentication.user.displayName"></span> <b class="caret"></b>
</a>
<ul class="dropdown-menu" role="menu">
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
<form class="signin form-horizontal">
<fieldset>
<div class="form-group text-center">
<img ngf-src="vm.fileSelected ? picFile : '/' + vm.user.profileImageURL" alt="{{vm.user.displayName}}" class="img-thumbnail user-profile-picture" ngf-drop>
<img ngf-src="vm.fileSelected ? picFile : vm.user.profileImageURL" alt="{{vm.user.displayName}}" class="img-thumbnail user-profile-picture" ngf-drop>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can add a config setting to the Shared config that would allow you to check whether the profile should be retrieved using local storage.

Then you wouldn't need remove the "/", thus solving path issues on the backend.

shared: {
    profileImageIsExternal: process.env.UPLOADS_STORAGE ? true : false
    ...
}

</div>
<div ng-show="vm.loading" class="form-group text-center">
<img ng-src="/modules/core/client/img/loaders/loader.gif" height="50" width="50" alt="Loading image...">
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,27 @@ var _ = require('lodash'),
errorHandler = require(path.resolve('./modules/core/server/controllers/errors.server.controller')),
mongoose = require('mongoose'),
multer = require('multer'),
multerS3 = require('multer-s3'),
aws = require('aws-sdk'),
amazonS3URI = require('amazon-s3-uri'),
config = require(path.resolve('./config/config')),
User = mongoose.model('User'),
validator = require('validator');

var whitelistedFields = ['firstName', 'lastName', 'email', 'username'];

var useS3Storage = config.uploads.storage === 's3' && config.aws.s3;
var s3;

if (useS3Storage) {
aws.config.update({
accessKeyId: config.aws.s3.accessKeyId,
secretAccessKey: config.aws.s3.secretAccessKey
});

s3 = new aws.S3();
}

/**
* Update user details
*/
Expand Down Expand Up @@ -57,10 +72,24 @@ exports.update = function (req, res) {
exports.changeProfilePicture = function (req, res) {
var user = req.user;
var existingImageUrl;
var multerConfig;


if (useS3Storage) {
multerConfig = {
storage: multerS3({
s3: s3,
bucket: config.aws.s3.bucket,
acl: 'public-read'
})
};
} else {
multerConfig = config.uploads.profile.image;
}

// Filtering to upload only images
var multerConfig = config.uploads.profile.image;
multerConfig.fileFilter = require(path.resolve('./config/lib/multer')).imageFileFilter;

var upload = multer(multerConfig).single('newProfilePicture');

if (user) {
Expand Down Expand Up @@ -95,7 +124,9 @@ exports.changeProfilePicture = function (req, res) {

function updateUser() {
return new Promise(function (resolve, reject) {
user.profileImageURL = config.uploads.profile.image.dest + req.file.filename;
user.profileImageURL = config.uploads.storage === 's3' && config.aws.s3 ?
req.file.location :
'/' + req.file.path;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be changed back to what it was before: config.uploads.profile.image.dest + req.file.filename, if we can still use the "/" on the front-end when using local storage.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not fan of the '/' in the front end. I prefer having the final URL in the database. Otherwise if we switch from local to s3, all old URL would be bad formed. And that add more logic to the front where it shouldn't be.

user.save(function (err, theuser) {
if (err) {
reject(err);
Expand All @@ -109,24 +140,47 @@ exports.changeProfilePicture = function (req, res) {
function deleteOldImage() {
return new Promise(function (resolve, reject) {
if (existingImageUrl !== User.schema.path('profileImageURL').defaultValue) {
fs.unlink(existingImageUrl, function (unlinkError) {
if (unlinkError) {

// If file didn't exist, no need to reject promise
if (unlinkError.code === 'ENOENT') {
console.log('Removing profile image failed because file did not exist.');
return resolve();
}

console.error(unlinkError);

reject({
message: 'Error occurred while deleting old profile picture'
if (useS3Storage) {
try {
var { region, bucket, key } = amazonS3URI(existingImageUrl);
var params = {
Bucket: config.aws.s3.bucket,
Key: key
};

s3.deleteObject(params, function (err) {
if (err) {
console.log('Error occurred while deleting old profile picture.');
console.log('Check if you have sufficient permissions : ' + err);
}

resolve();
});
} else {
resolve();
} catch (err) {
console.warn(`${existingImageUrl} is not a valid S3 uri`);

return resolve();
}
});
} else {
fs.unlink(path.resolve('.' + existingImageUrl), function (unlinkError) {
if (unlinkError) {

// If file didn't exist, no need to reject promise
if (unlinkError.code === 'ENOENT') {
console.log('Removing profile image failed because file did not exist.');
return resolve();
}

console.error(unlinkError);

reject({
message: 'Error occurred while deleting old profile picture'
});
} else {
resolve();
}
});
}
} else {
resolve();
}
Expand Down
3 changes: 3 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,9 @@
},
"dependencies": {
"acl": "~0.4.10",
"amazon-s3-uri": "0.0.3",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This library is rather new, and doesn't have many stars. However, the implementation looks fine and isn't too complicated. We'll probably have to keep an eye on this library though.

Copy link

@Isabeljason Isabeljason Sep 6, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even we follow the library updates there are few minor changes that need to be update to it. As i am a new bee to MEAN.js need support from others to get into and complicated issues like path, switching, controllers, pre & post updates need to be checked very carefully.
Update me on this if any wrong.

"async": "~2.5.0",
"aws-sdk": "^2.104.0",
"body-parser": "~1.17.1",
"bower": "~1.8.0",
"chalk": "~2.1.0",
Expand All @@ -59,6 +61,7 @@
"mongoose": "~4.11.3",
"morgan": "~1.8.1",
"multer": "~1.3.0",
"multer-s3": "^2.7.0",
"nodemailer": "~4.0.1",
"owasp-password-strength-test": "~1.3.0",
"passport": "~0.3.2",
Expand Down