Skip to content

CoolDogee/cap-that-pic

Repository files navigation

cap-that-pic

Code Chrysalis X Mercari Greenfield Project


Cap That Pic generates suitable captions for your images. It uses the technology of Microsoft Azure's Computer Vision SDK, inhouse dataset of lyrics and custom build algorithm to generate intelligent and artistic captions that best describe a users image. Give it a try ;)

What is it?

Just think how many times you struggle coming up with good captions for you images, this application helps you to come up with artistic captions that best suits the attributes of the image.

How do you use it?

Generate Caption in just 3 steps

  1. User enters an image URL in the 'Please Enter Image URL' input box.
  2. Then click on Generate Caption Button to generate the caption that best matches your image. Yayy <3
  3. You can then share your image to Instagram or Facebook with the generated caption using the Facebook Share or Instagram Share button.

Demo Time!

Try out our Application.

Want to Contribute?

Please create a new issue and make a Pull Request corresponding to that issue :)

Unique Selling Proposition (USP) for the Product

After extracting the tags from Azure (Computer Vision) SDK and lyrics from MongoDB then we execute our custom build algorithm to find the best matched caption for the picture which is the key feature or USP of our product.

Minimal Viable Product (MVP) [Using a User Story]

  • I am John Doe
  • I recently clicked some images on a hiking trip.
  • I want to share theses images on social media to impress my friends by using artistic captions for images.
  • I need a product that can generate an artistic caption for my images.
  • I upload an image on ‘Cap That Pic’ and yeah!!!! I can now easily get amazing captions for my images and I can share these on Facebook or Instagram ;)

Essential Features

  • Structured User Interface for user to enter an Image URL.
  • Backend architecture (routes and handling requests) to communicate with Azure SDK & lyrics dataset (stored in MongoDB).
  • Custom build algorithm to generate a best matching caption based on the tags and lyrics fetched from MongoDB dataset.

Technologies Used

  • Cap That Pic uses ReactJS as the Frontend structural framework.
  • Backend architecture is implemented using Golang
  • Azure (Computer Vision) SDK to extract tags from user's image.
  • MongoDB contains a dataset of song lyrics corresponding to several artists.
  • CircleCI for continuous integration and delivery with automated testing.
  • Docker Containers to package and deploy the application as one package.
  • Heroku as a cloud platform for continuous deployment and managing the application.

Engineering Challenges

  1. What if tags extraced from image are less?
  2. Generating one package of the application (connecting backend with frontend) using Docker and deploying on Heroku
  3. % matching of tags (extracted from azure SDK) with the lyrics (extracted from MongoDB)
  4. Connect our application with Instagram/Facebook/Twitter to share the image with caption directly without leaving our application.
  5. Integrating ReactJs frontend with Backend implemented in GoLang.
  6. Poor Microsoft Azure (Computer Vision) support for GoLang

Future Goals

  1. Upload images from local computer.
  2. Improving response time for generating captions
  3. Share on Facebook/Twitter
  4. A URL that generates caption on its own (so that friends can share it among themselves) (Example: https://cap-that-pic.herokuapp.com/?fileName=https://cms.hostelworld.com/hwblog/wp-content/uploads/sites/2/2017/08/lovelyforliving.jpg).
  5. Show tags generated from the image in UI
  6. Improve the UI/UX of the application.
  7. Implementation of reCAPTCHA.

Deployment Guildline

  1. Build the image and push it to your docker hub. {your_image} format like username/imagename
docker build . --tag  {your_image} 
docker push {your_image} 
  1. Create a kubernetes cluster in your google cloud console and export the env variables following:
export GKE_CLUSTER={your_cluster_name}
export GKE_CLUSTER_ZONE={your_cluster_name_zone} 
  1. In mongo_deployment.yml and web_deployment.yml, fill kouzoh-p-{your_gcp_project_id}. In web_deployment.yml, fill {your_image}, {COMPUTER_VISION_KEY}. your_gcp_project_id is the one you register in https://github.com/kouzoh/kouzoh-p-terraform.

  2. Run the deploy script

chmod 755 deploy.sh
./deploy.sh

About

Generate suitable captions for images

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published