Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add matplotlib image support #1218

Closed
brollb opened this issue Mar 26, 2019 · 3 comments
Closed

Add matplotlib image support #1218

brollb opened this issue Mar 26, 2019 · 3 comments
Assignees
Labels
enhancement job feedback Graphs, images, and other feedback generated during job execution.

Comments

@brollb
Copy link
Contributor

brollb commented Mar 26, 2019

Currently, deepforge supports using matplotlib for generating 2D plots. It would be great if this could be extended to support displaying images as well (particularly useful for image classification problems!).

@brollb brollb added enhancement job feedback Graphs, images, and other feedback generated during job execution. labels Mar 26, 2019
@brollb
Copy link
Contributor Author

brollb commented Mar 26, 2019

Job feedback is currently encoded in the stdout for a job, decoded by the server, and then the project is updated to contain the actual corresponding objects. As matplotlib supports custom backends, we have a custom backend defined for use within deepforge which prints a JSON representation of the graph when using it for plotting.

To add support for images, we would need to:

@umesh-timalsina
Copy link
Contributor

Now that #1329 is merged, Incremental support to Image trace with updated meta-model can be provided.

@umesh-timalsina
Copy link
Contributor

umesh-timalsina commented Nov 27, 2019

Steps for Image Support is quite different for plotly than #1218 (comment).

  1. Custom Matplotlib Backend Should be updated to get Image Information(The best strategy now is the JSON Dump a numpy Matrix although this could be changed to support any other form of encoding).
  2. Update The MetaModel such that node Image is contained within the SubGraph.
  3. So, the image information will now be available upon plots creation.
  4. Update the ExecuteJob.Metadata.js to create Image Nodes upon getting the information.
  5. Update PlotlyDescExtractor.js to get the plotly JSON from the image Node.
  6. Update ExecutionIndexControl.js to send updates regarding Images to the Widget.
  1. Custom Matplotlib Backend Should be updated to get Image Information(The best strategy now is the JSON Dump a numpy Matrix although this could be changed to support any other form of encoding).

This strategy works but since the dumped matrix is very large, stdout for a execution is almost full of the Matrix. Rather than dumping this array, this should be converted to png and uploaded to the file system borrowing some ideas from #1218 (comment)

Another Alternative to this strategy is to Base64Encode the matplotlib Image numpy Array.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement job feedback Graphs, images, and other feedback generated during job execution.
Projects
None yet
Development

No branches or pull requests

2 participants