diff --git a/README.md b/README.md index a729240..74319ae 100644 --- a/README.md +++ b/README.md @@ -7,33 +7,40 @@ So if you like to see the kotlin, you can go through the repo! ## Example apps -### Digit classifier +### Depth Estimation + +An Android app which uses the MiDaS model to perform monocular depth estimation on RGB images directly. The app displays a depth map over the live camera feed and works for both the front and the rear cameras. + + +Contributed from: https://github.com/shubham0204/Realtime_MiDaS_Depth_Estimation_Android + +### Digit Classifier *End-to-end sample of a digit classifier model built with TensorFlow 2.0 (Keras API), and trained on MNIST dataset.* Added from: https://github.com/tensorflow/examples/tree/master/lite/examples -### Image segmentaion +### Image Segmentation The used model, [DeepLab](https://ai.googleblog.com/2018/03/semantic-image-segmentation-with.html) is a state-of-art deep learning model for semantic image segmentation, where the goal is to assign semantic labels (e.g. person, dog, cat) to every pixel in the input image. Added from: https://github.com/tensorflow/examples/tree/master/lite/examples -### Optical character recognition +### Optical Character Recognition OCR is the process of recognizing characters from images using computer vision and machine learning techniques. This reference app demos how to use TensorFlow Lite to do OCR. It uses a text detection model and a text recognition model as a pipeline to recognize texts. Added from: https://github.com/tensorflow/examples/tree/master/lite/examples -### Pose esimation +### Pose Estimation This is an app that continuously detects the body parts in the frames seen by your device's camera. These instructions walk you through building and running the demo on an Android device. Camera captures are discarded immediately after use, nothing is stored or saved. Added from: https://github.com/tensorflow/examples/tree/master/lite/examples -### Pose net +### PoseNet This is an app that continuously detects the body parts in the frames seen by your device's camera. These instructions walk you through building and running the demo on an Android device. Camera captures are discarded immediately after use, nothing is stored or saved. Added from: https://github.com/tensorflow/examples/tree/master/lite/examples -### Sound classification +### Sound Classification This Android application demonstrates how to classify sound on-device. It uses: - TFLite Task Library @@ -41,14 +48,15 @@ This Android application demonstrates how to classify sound on-device. It uses: Added from: https://github.com/tensorflow/examples/tree/master/lite/examples -### Style transfer +### Style Transfer Artistic style transfer is an optimization technique used to take two images: a content image and a style reference image (such as an artwork by a famous painter) and blend them together so the output image looks like the content image, but “painted” in the style of the style reference image. Added from: https://github.com/tensorflow/examples/tree/master/lite/examples -### Coming Soon! 🚀 -**BERT qa** +## Coming Soon! 🚀 + +**BERT Q&A** **Text classification** @@ -78,3 +86,4 @@ Added from: https://github.com/tensorflow/examples/tree/master/lite/examples ## Contribute ❤️ You can just create new a [new issue](https://github.com/SunitRoy2703/Tensorflow-lite-samples-kotlin/issues/new) or mail me at iamsunitroy03@gmail.com + diff --git a/depth_estimation/.gitignore b/depth_estimation/.gitignore new file mode 100644 index 0000000..0c05a4c --- /dev/null +++ b/depth_estimation/.gitignore @@ -0,0 +1,18 @@ +*.iml +.gradle +/local.properties +/.idea +/app/src/androidTest +/app/src/test +/.idea/caches +/.idea/libraries +/.idea/modules.xml +/.idea/workspace.xml +/.idea/navEditor.xml +/.idea/assetWizardSettings.xml +.DS_Store +/build +/captures +.externalNativeBuild +.cxx +local.properties diff --git a/depth_estimation/LICENSE.txt b/depth_estimation/LICENSE.txt new file mode 100644 index 0000000..29f81d8 --- /dev/null +++ b/depth_estimation/LICENSE.txt @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/depth_estimation/README.md b/depth_estimation/README.md new file mode 100644 index 0000000..8f9e655 --- /dev/null +++ b/depth_estimation/README.md @@ -0,0 +1,73 @@ +
+

Realtime Depth Estimation In Android With MiDaS

+
+ +![depth image sample](repo_images/depth_sample.jpg) +![camera sample](repo_images/rgb_sample.jpg) + +* [Official MiDaS GitHub repo](https://github.com/isl-org/MiDaS) +* [TFLite model on TensorFlow Hub](https://tfhub.dev/intel/midas/v2_1_small/1) +* [APK for this project](https://github.com/shubham0204/Realtime_MiDaS_Depth_Estimation_Android/raw/master/app.apk) + +The repository contains an Android project which uses the [MiDaS](https://github.com/isl-org/MiDaS) model to perform monocular +depth estimation. You can find the official Android example here -> https://github.com/isl-org/MiDaS/tree/master/mobile/android + +This project uses the TFLite model from the [MiDaS's TensorFlow Hub repo](https://tfhub.dev/intel/midas/v2_1_small/1). +The following features are included in the project, + +1. Well documented code with links to SO answers wherever required. +2. Uses latest APIs like [CameraX](https://developer.android.com/training/camerax) and [Kotlin Coroutines](https://developer.android.com/kotlin/coroutines). +3. No use of heavy packages like OpenCV to process and display the depth map. The application is coded in a +*Android* friendly manner. +4. Works for both front and rear camera and also in portrait and landscape orientations. + +### Project info: + +``` +compileSdk 30 +applicationId "com.shubham0204.ml.depthestimation" +minSdk 23 +targetSdk 30 +androidGradlePluginVersion 7.0.0 +gradlePluginVersion 7.0.2 +``` + + +## Citation + +``` +@article{DBLP:journals/corr/abs-1907-01341, + author = {Katrin Lasinger and + Ren{\'{e}} Ranftl and + Konrad Schindler and + Vladlen Koltun}, + title = {Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot + Cross-Dataset Transfer}, + journal = {CoRR}, + volume = {abs/1907.01341}, + year = {2019}, + url = {http://arxiv.org/abs/1907.01341}, + archivePrefix = {arXiv}, + eprint = {1907.01341}, + timestamp = {Mon, 08 Jul 2019 14:12:33 +0200}, + biburl = {https://dblp.org/rec/journals/corr/abs-1907-01341.bib}, + bibsource = {dblp computer science bibliography, https://dblp.org} +} +``` + +## License + +``` +Copyright 2021 Shubham Panchal +Licensed under the Apache License, Version 2.0 (the "License"); +You may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +``` diff --git a/depth_estimation/app/.gitignore b/depth_estimation/app/.gitignore new file mode 100644 index 0000000..42afabf --- /dev/null +++ b/depth_estimation/app/.gitignore @@ -0,0 +1 @@ +/build \ No newline at end of file diff --git a/depth_estimation/app/build.gradle b/depth_estimation/app/build.gradle new file mode 100644 index 0000000..fa479eb --- /dev/null +++ b/depth_estimation/app/build.gradle @@ -0,0 +1,67 @@ +plugins { + id 'com.android.application' + id 'kotlin-android' +} + +android { + compileSdk 30 + + defaultConfig { + applicationId "com.shubham0204.ml.depthestimation" + minSdk 23 + targetSdk 30 + versionCode 1 + versionName "1.0" + + testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner" + } + aaptOptions { + noCompress "tflite" + } + buildTypes { + release { + minifyEnabled false + proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro' + } + } + compileOptions { + sourceCompatibility JavaVersion.VERSION_1_8 + targetCompatibility JavaVersion.VERSION_1_8 + } + kotlinOptions { + jvmTarget = '1.8' + } +} + +// App assets folder +project.ext.ASSET_DIR = projectDir.toString() + '/src/main/assets' + +// Download TF Lite model. +apply from: 'download.gradle' + + +dependencies { + implementation 'androidx.core:core-ktx:1.6.0' + implementation 'androidx.appcompat:appcompat:1.3.1' + implementation 'com.google.android.material:material:1.4.0' + implementation 'androidx.constraintlayout:constraintlayout:2.1.0' + + // CameraX dependencies + implementation "androidx.camera:camera-camera2:1.0.0" + implementation "androidx.camera:camera-lifecycle:1.0.0" + implementation "androidx.camera:camera-view:1.0.0-alpha26" + implementation "androidx.camera:camera-extensions:1.0.0-alpha26" + + // TensorFlow Lite dependencies + implementation 'org.tensorflow:tensorflow-lite:2.4.0' + implementation 'org.tensorflow:tensorflow-lite-gpu:2.4.0' + implementation 'org.tensorflow:tensorflow-lite-support:0.1.0' + + // Kotlin Coroutines + implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.9' + implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-android:1.3.9' + + testImplementation 'junit:junit:4.+' + androidTestImplementation 'androidx.test.ext:junit:1.1.3' + androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0' +} \ No newline at end of file diff --git a/depth_estimation/app/download.gradle b/depth_estimation/app/download.gradle new file mode 100644 index 0000000..f40efe8 --- /dev/null +++ b/depth_estimation/app/download.gradle @@ -0,0 +1,10 @@ +apply plugin: 'de.undercouch.download' + +task downloadModelFile(type: Download) { + src 'https://tfhub.dev/intel/lite-model/midas/v2_1_small/1/lite/1?lite-format=tflite' + dest project.ext.ASSET_DIR + '/depth_model.tflite' + overwrite false +} + + +preBuild.dependsOn downloadModelFile \ No newline at end of file diff --git a/depth_estimation/app/proguard-rules.pro b/depth_estimation/app/proguard-rules.pro new file mode 100644 index 0000000..481bb43 --- /dev/null +++ b/depth_estimation/app/proguard-rules.pro @@ -0,0 +1,21 @@ +# Add project specific ProGuard rules here. +# You can control the set of applied configuration files using the +# proguardFiles setting in build.gradle. +# +# For more details, see +# http://developer.android.com/guide/developing/tools/proguard.html + +# If your project uses WebView with JS, uncomment the following +# and specify the fully qualified class name to the JavaScript interface +# class: +#-keepclassmembers class fqcn.of.javascript.interface.for.webview { +# public *; +#} + +# Uncomment this to preserve the line number information for +# debugging stack traces. +#-keepattributes SourceFile,LineNumberTable + +# If you keep the line number information, uncomment this to +# hide the original source file name. +#-renamesourcefileattribute SourceFile \ No newline at end of file diff --git a/depth_estimation/app/src/main/AndroidManifest.xml b/depth_estimation/app/src/main/AndroidManifest.xml new file mode 100644 index 0000000..633dd88 --- /dev/null +++ b/depth_estimation/app/src/main/AndroidManifest.xml @@ -0,0 +1,26 @@ + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/depth_estimation/app/src/main/java/com/shubham0204/ml/depthestimation/BitmapUtils.kt b/depth_estimation/app/src/main/java/com/shubham0204/ml/depthestimation/BitmapUtils.kt new file mode 100644 index 0000000..85512fe --- /dev/null +++ b/depth_estimation/app/src/main/java/com/shubham0204/ml/depthestimation/BitmapUtils.kt @@ -0,0 +1,97 @@ +/* + * Copyright 2021 Shubham Panchal + * Licensed under the Apache License, Version 2.0 (the "License"); + * You may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.shubham0204.ml.depthestimation + +import android.content.Context +import android.graphics.* +import android.media.Image +import java.io.ByteArrayOutputStream +import java.io.File +import java.io.FileOutputStream + +// Helper class for operations on Bitmaps +class BitmapUtils { + + companion object { + + // Rotate the given `source` by `degrees`. + // See this SO answer -> https://stackoverflow.com/a/16219591/10878733 + fun rotateBitmap( source: Bitmap , degrees : Float ): Bitmap { + val matrix = Matrix() + matrix.postRotate( degrees ) + return Bitmap.createBitmap(source, 0, 0, source.width, source.height, matrix , false ) + } + + + // Use this method to save a Bitmap to the internal storage ( app-specific storage ) of your device. + // To see the image, go to "Device File Explorer" -> "data" -> "data" -> "com.ml.quaterion.facenetdetection" -> "files" + fun saveBitmap(context: Context, image: Bitmap, name: String) { + val fileOutputStream = FileOutputStream(File( context.filesDir.absolutePath + "/$name.png")) + image.compress(Bitmap.CompressFormat.PNG, 100, fileOutputStream) + } + + + // Resize the given Bitmap with given `targetWidth` and `targetHeight`. + // See this SO answer -> https://stackoverflow.com/a/65574102/10878733 + fun resizeBitmap( src : Bitmap , targetWidth : Int , targetHeight : Int ) : Bitmap { + return Bitmap.createScaledBitmap( src , targetWidth , targetHeight , true ) + } + + + // Flip the given `Bitmap` vertically. + // See this SO answer -> https://stackoverflow.com/a/36494192/10878733 + fun flipBitmap( source: Bitmap ): Bitmap { + val matrix = Matrix() + matrix.postScale(-1f, 1f, source.width / 2f, source.height / 2f) + return Bitmap.createBitmap(source, 0, 0, source.width, source.height, matrix, true) + } + + + fun byteBufferToBitmap( imageArray : FloatArray , imageDim : Int ) : Bitmap { + val pixels = imageArray.map { it.toInt() }.toIntArray() + val bitmap = Bitmap.createBitmap(imageDim, imageDim, Bitmap.Config.RGB_565 ); + for ( i in 0 until imageDim ) { + for ( j in 0 until imageDim ) { + val p = pixels[ i * imageDim + j ] + bitmap.setPixel( j , i , Color.rgb( p , p , p )) + } + } + return bitmap + } + + + // Convert android.media.Image to android.graphics.Bitmap and rotate it by `rotationDegrees` + // See the SO answer -> https://stackoverflow.com/a/44486294/10878733 + fun imageToBitmap( image : Image , rotationDegrees : Int ): Bitmap { + val yBuffer = image.planes[0].buffer + val uBuffer = image.planes[1].buffer + val vBuffer = image.planes[2].buffer + val ySize = yBuffer.remaining() + val uSize = uBuffer.remaining() + val vSize = vBuffer.remaining() + val nv21 = ByteArray(ySize + uSize + vSize) + yBuffer.get(nv21, 0, ySize) + vBuffer.get(nv21, ySize, vSize) + uBuffer.get(nv21, ySize + vSize, uSize) + val yuvImage = YuvImage(nv21, ImageFormat.NV21, image.width, image.height, null) + val out = ByteArrayOutputStream() + yuvImage.compressToJpeg(Rect(0, 0, yuvImage.width, yuvImage.height), 100, out) + val yuv = out.toByteArray() + return rotateBitmap( BitmapFactory.decodeByteArray(yuv, 0, yuv.size) , rotationDegrees.toFloat() ) + } + + } + +} \ No newline at end of file diff --git a/depth_estimation/app/src/main/java/com/shubham0204/ml/depthestimation/DrawingOverlay.kt b/depth_estimation/app/src/main/java/com/shubham0204/ml/depthestimation/DrawingOverlay.kt new file mode 100644 index 0000000..0c601bb --- /dev/null +++ b/depth_estimation/app/src/main/java/com/shubham0204/ml/depthestimation/DrawingOverlay.kt @@ -0,0 +1,59 @@ +/* + * Copyright 2021 Shubham Panchal + * Licensed under the Apache License, Version 2.0 (the "License"); + * You may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.shubham0204.ml.depthestimation + +import android.content.Context +import android.graphics.Bitmap +import android.graphics.Canvas +import android.util.AttributeSet +import android.view.SurfaceHolder +import android.view.SurfaceView + +// Overlay to display the depth map over the `PreviewView`. +// See activity_main.xml +class DrawingOverlay(context : Context, attributeSet : AttributeSet) : SurfaceView( context , attributeSet ) , SurfaceHolder.Callback { + + // This variable is assigned in FrameAnalyser.kt + var depthMaskBitmap : Bitmap? = null + + // These variables are assigned in MainActivity.kt + var isFrontCameraOn = true + var isShowingDepthMap = false + + + + override fun surfaceCreated(holder: SurfaceHolder) { + } + + + override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) { + } + + + override fun surfaceDestroyed(holder: SurfaceHolder) { + } + + + // Will be called when `drawingOverlay.invalidate()` is called in FrameAnalyser.kt. + override fun onDraw(canvas: Canvas?) { + if ( depthMaskBitmap != null && isShowingDepthMap ) { + // If the front camera is on, then flip the Bitmap vertically ( or produce a mirror image of it ). + canvas?.drawBitmap( + if ( isFrontCameraOn ) { BitmapUtils.flipBitmap( depthMaskBitmap!! ) } else { depthMaskBitmap!! }, + 0f , 0f , null ) + } + } + +} \ No newline at end of file diff --git a/depth_estimation/app/src/main/java/com/shubham0204/ml/depthestimation/FrameAnalyser.kt b/depth_estimation/app/src/main/java/com/shubham0204/ml/depthestimation/FrameAnalyser.kt new file mode 100644 index 0000000..c4a3ed4 --- /dev/null +++ b/depth_estimation/app/src/main/java/com/shubham0204/ml/depthestimation/FrameAnalyser.kt @@ -0,0 +1,78 @@ +/* + * Copyright 2021 Shubham Panchal + * Licensed under the Apache License, Version 2.0 (the "License"); + * You may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.shubham0204.ml.depthestimation + +import android.graphics.Bitmap +import android.view.View +import androidx.camera.core.ImageAnalysis +import androidx.camera.core.ImageProxy +import kotlinx.coroutines.CoroutineScope +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.launch +import kotlinx.coroutines.withContext + +// Image Analyser for performing depth estimation on camera frames. +class FrameAnalyser( + private var depthEstimationModel : MiDASModel , + private var drawingOverlay: DrawingOverlay ) : ImageAnalysis.Analyzer { + + private var frameBitmap : Bitmap? = null + private var isFrameProcessing = false + var isComputingDepthMap = false + + + + override fun analyze(image: ImageProxy) { + // Return if depth map computation is turned off. See MainActivity.kt + if ( !isComputingDepthMap ) { + image.close() + return + } + // If a frame is being processed, drop the current frame. + if ( isFrameProcessing ) { + image.close() + return + } + isFrameProcessing = true + if ( image.image != null ) { + // Get the `Bitmap` of the current frame ( with corrected rotation ). + frameBitmap = BitmapUtils.imageToBitmap( image.image!! , image.imageInfo.rotationDegrees ) + image.close() + CoroutineScope( Dispatchers.Main ).launch { + runModel( frameBitmap!! ) + } + } + } + + + private suspend fun runModel( inputImage : Bitmap ) = withContext( Dispatchers.Default ) { + // Compute the depth given the frame Bitmap. + val output = depthEstimationModel.getDepthMap( inputImage ) + withContext( Dispatchers.Main ) { + // Notify that the current frame is processed and the pipeline is + // ready for the next frame. + isFrameProcessing = false + if ( drawingOverlay.visibility == View.GONE ) { + drawingOverlay.visibility = View.VISIBLE + } + // Submit the depth Bitmap to the DrawingOverlay and update it. + // Note, calling `drawingOverlay.invalidate()` here will call `onDraw()` in DrawingOverlay.kt. + drawingOverlay.depthMaskBitmap = BitmapUtils.resizeBitmap( output , + frameBitmap!!.width , frameBitmap!!.height) + drawingOverlay.invalidate() + } + } + +} \ No newline at end of file diff --git a/depth_estimation/app/src/main/java/com/shubham0204/ml/depthestimation/Logger.kt b/depth_estimation/app/src/main/java/com/shubham0204/ml/depthestimation/Logger.kt new file mode 100644 index 0000000..9dc15ae --- /dev/null +++ b/depth_estimation/app/src/main/java/com/shubham0204/ml/depthestimation/Logger.kt @@ -0,0 +1,36 @@ +/* + * Copyright 2021 Shubham Panchal + * Licensed under the Apache License, Version 2.0 (the "License"); + * You may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.shubham0204.ml.depthestimation + +import android.util.Log + +// Utility class to log messages +class Logger { + + companion object { + + private val APP_LOG_TAG = "Depth_Estimation_App" + + fun logError( message : String ) { + Log.e( APP_LOG_TAG , message ) + } + + fun logInfo( message : String ) { + Log.i( APP_LOG_TAG , message ) + } + + } + +} \ No newline at end of file diff --git a/depth_estimation/app/src/main/java/com/shubham0204/ml/depthestimation/MainActivity.kt b/depth_estimation/app/src/main/java/com/shubham0204/ml/depthestimation/MainActivity.kt new file mode 100644 index 0000000..1197aca --- /dev/null +++ b/depth_estimation/app/src/main/java/com/shubham0204/ml/depthestimation/MainActivity.kt @@ -0,0 +1,208 @@ +/* + * Copyright 2021 Shubham Panchal + * Licensed under the Apache License, Version 2.0 (the "License"); + * You may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.shubham0204.ml.depthestimation + +import android.Manifest +import android.content.pm.PackageManager +import android.content.res.ColorStateList +import android.graphics.Color +import android.os.Bundle +import android.util.Size +import android.view.View +import android.widget.Toast +import androidx.activity.result.contract.ActivityResultContracts +import androidx.appcompat.app.AlertDialog +import androidx.appcompat.app.AppCompatActivity +import androidx.camera.core.CameraSelector +import androidx.camera.core.ImageAnalysis +import androidx.camera.core.Preview +import androidx.camera.lifecycle.ProcessCameraProvider +import androidx.camera.view.PreviewView +import androidx.core.app.ActivityCompat +import androidx.core.content.ContextCompat +import androidx.lifecycle.LifecycleOwner +import com.google.android.material.floatingactionbutton.FloatingActionButton +import com.google.common.util.concurrent.ListenableFuture +import java.util.concurrent.ExecutionException +import java.util.concurrent.Executors + +class MainActivity : AppCompatActivity() { + + private lateinit var previewView : PreviewView + private lateinit var drawingOverlay: DrawingOverlay + + private var preview: Preview? = null + private lateinit var cameraProviderListenableFuture : ListenableFuture + private lateinit var frameAnalyser : FrameAnalyser + private var frameAnalysis: ImageAnalysis? = null + + private var isFrontCameraOn = true + private var isDepthMapDisplayed = false + + + + override fun onCreate(savedInstanceState: Bundle?) { + super.onCreate(savedInstanceState) + + // Remove the status bar to have a full screen experience + // See this answer on SO -> https://stackoverflow.com/a/68152688/10878733 + window.decorView.systemUiVisibility = View.SYSTEM_UI_FLAG_FULLSCREEN + setContentView(R.layout.activity_main) + + previewView = findViewById( R.id.camera_preview_view ) + + // Make sure that the DrawingOverlay remains on top + // See this SO answer -> https://stackoverflow.com/a/28883273/10878733 + drawingOverlay = findViewById( R.id.camera_drawing_overlay ) + drawingOverlay.setWillNotDraw(false) + drawingOverlay.setZOrderOnTop(true) + + val depthEstimationModel = MiDASModel( this ) + frameAnalyser = FrameAnalyser( depthEstimationModel , drawingOverlay ) + + val flipCameraFAB = findViewById( R.id.flip_camera_fab ) + flipCameraFAB.setOnClickListener { + when( isFrontCameraOn ) { + true -> setupCameraProvider( CameraSelector.LENS_FACING_BACK ) + false -> setupCameraProvider( CameraSelector.LENS_FACING_FRONT ) + } + // Alert the DrawingOverlay regarding the change in lens facing. + // This is important as for the front camera, we need to flip ( vertically ) the frames to + // draw them on the overlay. + drawingOverlay.isFrontCameraOn = !isFrontCameraOn + isFrontCameraOn = !isFrontCameraOn + } + + val depthMapDisplayFAB = findViewById( R.id.depth_map_toggle_fab ) + depthMapDisplayFAB.setOnClickListener { + if ( isDepthMapDisplayed ) { + drawingOverlay.visibility = View.GONE + } + // Alert the FrameAnalyser to stop computing the depth maps. + frameAnalyser.isComputingDepthMap = !isDepthMapDisplayed + // Alert the DrawingOverlay to stop drawing any depth maps ( if they were to be drawn ). + drawingOverlay.isShowingDepthMap = !isDepthMapDisplayed + isDepthMapDisplayed = !isDepthMapDisplayed + depthMapDisplayFAB.backgroundTintList = + ColorStateList.valueOf( if ( isDepthMapDisplayed ) { Color.GREEN } else { Color.WHITE } ) + val message = if ( isDepthMapDisplayed ) { "Displaying Depth map..." } else { "Displaying Camera Feed..." } + Toast.makeText( this , message , Toast.LENGTH_LONG ).show() + } + depthMapDisplayFAB.backgroundTintList = ColorStateList.valueOf( Color.WHITE ) + + // Request the CAMERA permission as we'll require it for displaying the camera preview. + // See https://developer.android.com/training/permissions/requesting#allow-system-manage-request-code + if (ActivityCompat.checkSelfPermission( this , Manifest.permission.CAMERA) + != PackageManager.PERMISSION_GRANTED ) { + requestCameraPermission() + } + else { + setupCameraProvider( CameraSelector.LENS_FACING_FRONT ) + } + + } + + + private fun requestCameraPermission() { + Logger.logInfo( "Requesting camera permission" ) + requestCameraPermissionLauncher.launch( Manifest.permission.CAMERA ) + } + + + private val requestCameraPermissionLauncher = registerForActivityResult( + ActivityResultContracts.RequestPermission() ) { + isGranted : Boolean -> + if ( isGranted ) { + Logger.logInfo( "Camera permission granted by user." ) + setupCameraProvider( CameraSelector.LENS_FACING_FRONT ) + } + else { + Logger.logInfo( "Camera permission denied by user." ) + val alertDialog = AlertDialog.Builder( this ).apply { + setTitle( "Permissions" ) + setMessage( "The app requires the camera permission to function." ) + setPositiveButton( "GRANT") { dialog, _ -> + dialog.dismiss() + requestCameraPermission() + } + setNegativeButton( "CLOSE" ) { dialog, _ -> + dialog.dismiss() + finish() + } + setCancelable( false ) + create() + } + alertDialog.show() + } + } + + + // Setup the PreviewView for live camera feed. + // See the docs -> https://developer.android.com/training/camerax/preview + // and https://developer.android.com/training/camerax/analyze + private fun setupCameraProvider( cameraFacing : Int ) { + cameraProviderListenableFuture = ProcessCameraProvider.getInstance( this ) + cameraProviderListenableFuture.addListener({ + try { + val cameraProvider: ProcessCameraProvider = cameraProviderListenableFuture.get() + bindPreview( cameraProvider , cameraFacing ) + } + catch (e: ExecutionException) { + Logger.logError( e.message!! ) + } + catch (e: InterruptedException) { + Logger.logError( e.message!! ) + } + }, ContextCompat.getMainExecutor( this )) + } + + + private fun bindPreview( cameraProvider: ProcessCameraProvider , lensFacing : Int ) { + // Unbind any previous use-cases as we'll attach them once again. + if ( preview != null && frameAnalysis != null ) { + cameraProvider.unbind( preview , frameAnalysis ) + } + + Logger.logInfo( "Setting camera with ${ + if ( lensFacing == CameraSelector.LENS_FACING_FRONT ) { "front" } + else { "rear" } + } lens facing" ) + + preview = Preview.Builder().build() + val cameraSelector = CameraSelector.Builder() + .requireLensFacing( lensFacing ) + .build() + preview!!.setSurfaceProvider(previewView.surfaceProvider) + + // Set the resolution which is the closest to the screen size. + val displayMetrics = resources.displayMetrics + val screenSize = Size( displayMetrics.widthPixels, displayMetrics.heightPixels ) + Logger.logInfo( "Screen size is $screenSize" ) + + frameAnalysis = ImageAnalysis.Builder() + .setTargetResolution( screenSize ) + .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST) + .build() + + frameAnalysis!!.setAnalyzer( Executors.newSingleThreadExecutor() , frameAnalyser ) + cameraProvider.bindToLifecycle( + (this as LifecycleOwner), + cameraSelector, + frameAnalysis, + preview + ) + } + +} \ No newline at end of file diff --git a/depth_estimation/app/src/main/java/com/shubham0204/ml/depthestimation/MiDASModel.kt b/depth_estimation/app/src/main/java/com/shubham0204/ml/depthestimation/MiDASModel.kt new file mode 100644 index 0000000..09e9f97 --- /dev/null +++ b/depth_estimation/app/src/main/java/com/shubham0204/ml/depthestimation/MiDASModel.kt @@ -0,0 +1,143 @@ +/* + * Copyright 2021 Shubham Panchal + * Licensed under the Apache License, Version 2.0 (the "License"); + * You may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.shubham0204.ml.depthestimation + +import android.content.Context +import android.graphics.Bitmap +import android.os.Build +import org.tensorflow.lite.DataType +import org.tensorflow.lite.Interpreter +import org.tensorflow.lite.gpu.CompatibilityList +import org.tensorflow.lite.gpu.GpuDelegate +import org.tensorflow.lite.nnapi.NnApiDelegate +import org.tensorflow.lite.support.common.FileUtil +import org.tensorflow.lite.support.common.TensorOperator +import org.tensorflow.lite.support.common.TensorProcessor +import org.tensorflow.lite.support.common.ops.NormalizeOp +import org.tensorflow.lite.support.image.ImageProcessor +import org.tensorflow.lite.support.image.TensorImage +import org.tensorflow.lite.support.image.ops.ResizeOp +import org.tensorflow.lite.support.tensorbuffer.TensorBuffer +import org.tensorflow.lite.support.tensorbuffer.TensorBufferFloat + +// Helper class for the MiDAS TFlite model +class MiDASModel( context: Context ) { + + // See the `app/src/main/assets` folder for the TFLite model + private val modelFileName = "depth_model.tflite" + private var interpreter : Interpreter + private val NUM_THREADS = 4 + + // These values are taken from the Python file -> + // https://github.com/isl-org/MiDaS/blob/master/mobile/android/models/src/main/assets/run_tflite.py + private val inputImageDim = 256 + private val mean = floatArrayOf( 123.675f , 116.28f , 103.53f ) + private val std = floatArrayOf( 58.395f , 57.12f , 57.375f ) + + // Input tensor processor for MiDAS + // 1. Resize the image to ( 256 , 256 ) + // 2. Normalize using the given mean and std for each channel. + private val inputTensorProcessor = ImageProcessor.Builder() + .add( ResizeOp( inputImageDim , inputImageDim , ResizeOp.ResizeMethod.BILINEAR ) ) + .add( NormalizeOp( mean , std ) ) + .build() + + // Output tensor processor for MiDAS + // Perform min-max scaling for the outputs. See `MinMaxScalingOp` class. + private val outputTensorProcessor = TensorProcessor.Builder() + .add( MinMaxScalingOp() ) + .build() + + + + init { + // Initialize TFLite Interpreter + val interpreterOptions = Interpreter.Options().apply { + // Add the GPU Delegate if supported. + // See -> https://www.tensorflow.org/lite/performance/gpu#android + if ( CompatibilityList().isDelegateSupportedOnThisDevice ) { + Logger.logInfo( "GPU Delegate is supported on this device." ) + addDelegate( GpuDelegate( CompatibilityList().bestOptionsForThisDevice )) + } + else { + // Number of threads for computation + setNumThreads( NUM_THREADS ) + } + + interpreter = Interpreter(FileUtil.loadMappedFile( context, modelFileName ) , interpreterOptions ) + Logger.logInfo( "TFLite interpreter created." ) + } + + + fun getDepthMap( inputImage : Bitmap ) : Bitmap { + return run( inputImage ) + } + + + private fun run( inputImage : Bitmap ): Bitmap { + // Note: The model takes in a RGB image ( of shape ( 256 , 256 , 3 ) ) and + // outputs a depth map of shape ( 256 , 256 , 1 ) + // Create a tensor of shape ( 1 , inputImageDim , inputImageDim , 3 ) from the given Bitmap. + // Then perform operations on the tensor as described by `inputTensorProcessor`. + var inputTensor = TensorImage.fromBitmap( inputImage ) + + val t1 = System.currentTimeMillis() + inputTensor = inputTensorProcessor.process( inputTensor ) + + // Output tensor of shape ( 256 , 256 , 1 ) and data type float32 + var outputTensor = TensorBufferFloat.createFixedSize( + intArrayOf( inputImageDim , inputImageDim , 1 ) , DataType.FLOAT32 ) + + // Perform inference on the MiDAS model + interpreter.run( inputTensor.buffer, outputTensor.buffer ) + + // Perform operations on the output tensor as described by `outputTensorProcessor`. + outputTensor = outputTensorProcessor.process( outputTensor ) + + Logger.logInfo( "MiDaS inference speed: ${System.currentTimeMillis() - t1}") + + // Create a Bitmap from the depth map which will be displayed on the screen. + return BitmapUtils.byteBufferToBitmap( outputTensor.floatArray , inputImageDim ) + } + + + // Post processing operation for MiDAS + // Apply min-max scaling to the outputs of the model and bring them in the range [ 0 , 255 ]. + // Also, we apply a transformation which changes the data type from `int` to `uint` in Python. + // As unsigned integers aren't supported in Java, we add 255 + pixel if pixel < 0 + class MinMaxScalingOp : TensorOperator { + + override fun apply( input : TensorBuffer?): TensorBuffer { + val values = input!!.floatArray + // Compute min and max of the output + val max = values.maxOrNull()!! + val min = values.minOrNull()!! + for ( i in values.indices ) { + // Normalize the values and scale them by a factor of 255 + var p = ((( values[ i ] - min ) / ( max - min )) * 255).toInt() + if ( p < 0 ) { + p += 255 + } + values[ i ] = p.toFloat() + } + // Convert the normalized values to the TensorBuffer and load the values in it. + val output = TensorBufferFloat.createFixedSize( input.shape , DataType.FLOAT32 ) + output.loadArray( values ) + return output + } + + } + +} diff --git a/depth_estimation/app/src/main/res/drawable-v24/ic_launcher_foreground.xml b/depth_estimation/app/src/main/res/drawable-v24/ic_launcher_foreground.xml new file mode 100644 index 0000000..2b068d1 --- /dev/null +++ b/depth_estimation/app/src/main/res/drawable-v24/ic_launcher_foreground.xml @@ -0,0 +1,30 @@ + + + + + + + + + + + \ No newline at end of file diff --git a/depth_estimation/app/src/main/res/drawable/depth_map_toggle_icon.xml b/depth_estimation/app/src/main/res/drawable/depth_map_toggle_icon.xml new file mode 100644 index 0000000..b4d0af2 --- /dev/null +++ b/depth_estimation/app/src/main/res/drawable/depth_map_toggle_icon.xml @@ -0,0 +1,5 @@ + + + diff --git a/depth_estimation/app/src/main/res/drawable/flip_camera_icon.xml b/depth_estimation/app/src/main/res/drawable/flip_camera_icon.xml new file mode 100644 index 0000000..6f479a6 --- /dev/null +++ b/depth_estimation/app/src/main/res/drawable/flip_camera_icon.xml @@ -0,0 +1,5 @@ + + + diff --git a/depth_estimation/app/src/main/res/drawable/ic_launcher_background.xml b/depth_estimation/app/src/main/res/drawable/ic_launcher_background.xml new file mode 100644 index 0000000..07d5da9 --- /dev/null +++ b/depth_estimation/app/src/main/res/drawable/ic_launcher_background.xml @@ -0,0 +1,170 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/depth_estimation/app/src/main/res/layout/activity_main.xml b/depth_estimation/app/src/main/res/layout/activity_main.xml new file mode 100644 index 0000000..7571ef1 --- /dev/null +++ b/depth_estimation/app/src/main/res/layout/activity_main.xml @@ -0,0 +1,53 @@ + + + + + + + + + + + + + + \ No newline at end of file diff --git a/depth_estimation/app/src/main/res/mipmap-anydpi-v26/ic_launcher.xml b/depth_estimation/app/src/main/res/mipmap-anydpi-v26/ic_launcher.xml new file mode 100644 index 0000000..eca70cf --- /dev/null +++ b/depth_estimation/app/src/main/res/mipmap-anydpi-v26/ic_launcher.xml @@ -0,0 +1,5 @@ + + + + + \ No newline at end of file diff --git a/depth_estimation/app/src/main/res/mipmap-anydpi-v26/ic_launcher_round.xml b/depth_estimation/app/src/main/res/mipmap-anydpi-v26/ic_launcher_round.xml new file mode 100644 index 0000000..eca70cf --- /dev/null +++ b/depth_estimation/app/src/main/res/mipmap-anydpi-v26/ic_launcher_round.xml @@ -0,0 +1,5 @@ + + + + + \ No newline at end of file diff --git a/depth_estimation/app/src/main/res/mipmap-hdpi/ic_launcher.webp b/depth_estimation/app/src/main/res/mipmap-hdpi/ic_launcher.webp new file mode 100644 index 0000000..c209e78 Binary files /dev/null and b/depth_estimation/app/src/main/res/mipmap-hdpi/ic_launcher.webp differ diff --git a/depth_estimation/app/src/main/res/mipmap-hdpi/ic_launcher_round.webp b/depth_estimation/app/src/main/res/mipmap-hdpi/ic_launcher_round.webp new file mode 100644 index 0000000..b2dfe3d Binary files /dev/null and b/depth_estimation/app/src/main/res/mipmap-hdpi/ic_launcher_round.webp differ diff --git a/depth_estimation/app/src/main/res/mipmap-mdpi/ic_launcher.webp b/depth_estimation/app/src/main/res/mipmap-mdpi/ic_launcher.webp new file mode 100644 index 0000000..4f0f1d6 Binary files /dev/null and b/depth_estimation/app/src/main/res/mipmap-mdpi/ic_launcher.webp differ diff --git a/depth_estimation/app/src/main/res/mipmap-mdpi/ic_launcher_round.webp b/depth_estimation/app/src/main/res/mipmap-mdpi/ic_launcher_round.webp new file mode 100644 index 0000000..62b611d Binary files /dev/null and b/depth_estimation/app/src/main/res/mipmap-mdpi/ic_launcher_round.webp differ diff --git a/depth_estimation/app/src/main/res/mipmap-xhdpi/ic_launcher.webp b/depth_estimation/app/src/main/res/mipmap-xhdpi/ic_launcher.webp new file mode 100644 index 0000000..948a307 Binary files /dev/null and b/depth_estimation/app/src/main/res/mipmap-xhdpi/ic_launcher.webp differ diff --git a/depth_estimation/app/src/main/res/mipmap-xhdpi/ic_launcher_round.webp b/depth_estimation/app/src/main/res/mipmap-xhdpi/ic_launcher_round.webp new file mode 100644 index 0000000..1b9a695 Binary files /dev/null and b/depth_estimation/app/src/main/res/mipmap-xhdpi/ic_launcher_round.webp differ diff --git a/depth_estimation/app/src/main/res/mipmap-xxhdpi/ic_launcher.webp b/depth_estimation/app/src/main/res/mipmap-xxhdpi/ic_launcher.webp new file mode 100644 index 0000000..28d4b77 Binary files /dev/null and b/depth_estimation/app/src/main/res/mipmap-xxhdpi/ic_launcher.webp differ diff --git a/depth_estimation/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.webp b/depth_estimation/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.webp new file mode 100644 index 0000000..9287f50 Binary files /dev/null and b/depth_estimation/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.webp differ diff --git a/depth_estimation/app/src/main/res/mipmap-xxxhdpi/ic_launcher.webp b/depth_estimation/app/src/main/res/mipmap-xxxhdpi/ic_launcher.webp new file mode 100644 index 0000000..aa7d642 Binary files /dev/null and b/depth_estimation/app/src/main/res/mipmap-xxxhdpi/ic_launcher.webp differ diff --git a/depth_estimation/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.webp b/depth_estimation/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.webp new file mode 100644 index 0000000..9126ae3 Binary files /dev/null and b/depth_estimation/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.webp differ diff --git a/depth_estimation/app/src/main/res/values-night/themes.xml b/depth_estimation/app/src/main/res/values-night/themes.xml new file mode 100644 index 0000000..f0cbf28 --- /dev/null +++ b/depth_estimation/app/src/main/res/values-night/themes.xml @@ -0,0 +1,16 @@ + + + + \ No newline at end of file diff --git a/depth_estimation/app/src/main/res/values/colors.xml b/depth_estimation/app/src/main/res/values/colors.xml new file mode 100644 index 0000000..f8c6127 --- /dev/null +++ b/depth_estimation/app/src/main/res/values/colors.xml @@ -0,0 +1,10 @@ + + + #FFBB86FC + #FF6200EE + #FF3700B3 + #FF03DAC5 + #FF018786 + #FF000000 + #FFFFFFFF + \ No newline at end of file diff --git a/depth_estimation/app/src/main/res/values/strings.xml b/depth_estimation/app/src/main/res/values/strings.xml new file mode 100644 index 0000000..8d2b39a --- /dev/null +++ b/depth_estimation/app/src/main/res/values/strings.xml @@ -0,0 +1,3 @@ + + Depth Estimation + \ No newline at end of file diff --git a/depth_estimation/app/src/main/res/values/themes.xml b/depth_estimation/app/src/main/res/values/themes.xml new file mode 100644 index 0000000..964604c --- /dev/null +++ b/depth_estimation/app/src/main/res/values/themes.xml @@ -0,0 +1,16 @@ + + + + \ No newline at end of file diff --git a/depth_estimation/build.gradle b/depth_estimation/build.gradle new file mode 100644 index 0000000..486cbf4 --- /dev/null +++ b/depth_estimation/build.gradle @@ -0,0 +1,19 @@ +// Top-level build file where you can add configuration options common to all sub-projects/modules. +buildscript { + repositories { + google() + mavenCentral() + } + dependencies { + classpath "com.android.tools.build:gradle:7.0.0" + classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:1.5.20" + classpath 'de.undercouch:gradle-download-task:4.0.2' + + // NOTE: Do not place your application dependencies here; they belong + // in the individual module build.gradle files + } +} + +task clean(type: Delete) { + delete rootProject.buildDir +} \ No newline at end of file diff --git a/depth_estimation/gradle.properties b/depth_estimation/gradle.properties new file mode 100644 index 0000000..98bed16 --- /dev/null +++ b/depth_estimation/gradle.properties @@ -0,0 +1,21 @@ +# Project-wide Gradle settings. +# IDE (e.g. Android Studio) users: +# Gradle settings configured through the IDE *will override* +# any settings specified in this file. +# For more details on how to configure your build environment visit +# http://www.gradle.org/docs/current/userguide/build_environment.html +# Specifies the JVM arguments used for the daemon process. +# The setting is particularly useful for tweaking memory settings. +org.gradle.jvmargs=-Xmx2048m -Dfile.encoding=UTF-8 +# When configured, Gradle will run in incubating parallel mode. +# This option should only be used with decoupled projects. More details, visit +# http://www.gradle.org/docs/current/userguide/multi_project_builds.html#sec:decoupled_projects +# org.gradle.parallel=true +# AndroidX package structure to make it clearer which packages are bundled with the +# Android operating system, and which are packaged with your app"s APK +# https://developer.android.com/topic/libraries/support-library/androidx-rn +android.useAndroidX=true +# Automatically convert third-party libraries to use AndroidX +android.enableJetifier=true +# Kotlin code style for this project: "official" or "obsolete": +kotlin.code.style=official \ No newline at end of file diff --git a/depth_estimation/gradle/wrapper/gradle-wrapper.jar b/depth_estimation/gradle/wrapper/gradle-wrapper.jar new file mode 100644 index 0000000..e708b1c Binary files /dev/null and b/depth_estimation/gradle/wrapper/gradle-wrapper.jar differ diff --git a/depth_estimation/gradle/wrapper/gradle-wrapper.properties b/depth_estimation/gradle/wrapper/gradle-wrapper.properties new file mode 100644 index 0000000..2a4f313 --- /dev/null +++ b/depth_estimation/gradle/wrapper/gradle-wrapper.properties @@ -0,0 +1,6 @@ +#Wed Aug 25 07:50:30 IST 2021 +distributionBase=GRADLE_USER_HOME +distributionUrl=https\://services.gradle.org/distributions/gradle-7.0.2-bin.zip +distributionPath=wrapper/dists +zipStorePath=wrapper/dists +zipStoreBase=GRADLE_USER_HOME diff --git a/depth_estimation/gradlew b/depth_estimation/gradlew new file mode 100644 index 0000000..4f906e0 --- /dev/null +++ b/depth_estimation/gradlew @@ -0,0 +1,185 @@ +#!/usr/bin/env sh + +# +# Copyright 2015 the original author or authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +############################################################################## +## +## Gradle start up script for UN*X +## +############################################################################## + +# Attempt to set APP_HOME +# Resolve links: $0 may be a link +PRG="$0" +# Need this for relative symlinks. +while [ -h "$PRG" ] ; do + ls=`ls -ld "$PRG"` + link=`expr "$ls" : '.*-> \(.*\)$'` + if expr "$link" : '/.*' > /dev/null; then + PRG="$link" + else + PRG=`dirname "$PRG"`"/$link" + fi +done +SAVED="`pwd`" +cd "`dirname \"$PRG\"`/" >/dev/null +APP_HOME="`pwd -P`" +cd "$SAVED" >/dev/null + +APP_NAME="Gradle" +APP_BASE_NAME=`basename "$0"` + +# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"' + +# Use the maximum available, or set MAX_FD != -1 to use that value. +MAX_FD="maximum" + +warn () { + echo "$*" +} + +die () { + echo + echo "$*" + echo + exit 1 +} + +# OS specific support (must be 'true' or 'false'). +cygwin=false +msys=false +darwin=false +nonstop=false +case "`uname`" in + CYGWIN* ) + cygwin=true + ;; + Darwin* ) + darwin=true + ;; + MINGW* ) + msys=true + ;; + NONSTOP* ) + nonstop=true + ;; +esac + +CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar + + +# Determine the Java command to use to start the JVM. +if [ -n "$JAVA_HOME" ] ; then + if [ -x "$JAVA_HOME/jre/sh/java" ] ; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD="$JAVA_HOME/jre/sh/java" + else + JAVACMD="$JAVA_HOME/bin/java" + fi + if [ ! -x "$JAVACMD" ] ; then + die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +else + JAVACMD="java" + which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." +fi + +# Increase the maximum file descriptors if we can. +if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then + MAX_FD_LIMIT=`ulimit -H -n` + if [ $? -eq 0 ] ; then + if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then + MAX_FD="$MAX_FD_LIMIT" + fi + ulimit -n $MAX_FD + if [ $? -ne 0 ] ; then + warn "Could not set maximum file descriptor limit: $MAX_FD" + fi + else + warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT" + fi +fi + +# For Darwin, add options to specify how the application appears in the dock +if $darwin; then + GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\"" +fi + +# For Cygwin or MSYS, switch paths to Windows format before running java +if [ "$cygwin" = "true" -o "$msys" = "true" ] ; then + APP_HOME=`cygpath --path --mixed "$APP_HOME"` + CLASSPATH=`cygpath --path --mixed "$CLASSPATH"` + + JAVACMD=`cygpath --unix "$JAVACMD"` + + # We build the pattern for arguments to be converted via cygpath + ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null` + SEP="" + for dir in $ROOTDIRSRAW ; do + ROOTDIRS="$ROOTDIRS$SEP$dir" + SEP="|" + done + OURCYGPATTERN="(^($ROOTDIRS))" + # Add a user-defined pattern to the cygpath arguments + if [ "$GRADLE_CYGPATTERN" != "" ] ; then + OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)" + fi + # Now convert the arguments - kludge to limit ourselves to /bin/sh + i=0 + for arg in "$@" ; do + CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -` + CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option + + if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition + eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"` + else + eval `echo args$i`="\"$arg\"" + fi + i=`expr $i + 1` + done + case $i in + 0) set -- ;; + 1) set -- "$args0" ;; + 2) set -- "$args0" "$args1" ;; + 3) set -- "$args0" "$args1" "$args2" ;; + 4) set -- "$args0" "$args1" "$args2" "$args3" ;; + 5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;; + 6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;; + 7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;; + 8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;; + 9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;; + esac +fi + +# Escape application args +save () { + for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done + echo " " +} +APP_ARGS=`save "$@"` + +# Collect all arguments for the java command, following the shell quoting and substitution rules +eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS" + +exec "$JAVACMD" "$@" diff --git a/depth_estimation/gradlew.bat b/depth_estimation/gradlew.bat new file mode 100644 index 0000000..ac1b06f --- /dev/null +++ b/depth_estimation/gradlew.bat @@ -0,0 +1,89 @@ +@rem +@rem Copyright 2015 the original author or authors. +@rem +@rem Licensed under the Apache License, Version 2.0 (the "License"); +@rem you may not use this file except in compliance with the License. +@rem You may obtain a copy of the License at +@rem +@rem https://www.apache.org/licenses/LICENSE-2.0 +@rem +@rem Unless required by applicable law or agreed to in writing, software +@rem distributed under the License is distributed on an "AS IS" BASIS, +@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +@rem See the License for the specific language governing permissions and +@rem limitations under the License. +@rem + +@if "%DEBUG%" == "" @echo off +@rem ########################################################################## +@rem +@rem Gradle startup script for Windows +@rem +@rem ########################################################################## + +@rem Set local scope for the variables with windows NT shell +if "%OS%"=="Windows_NT" setlocal + +set DIRNAME=%~dp0 +if "%DIRNAME%" == "" set DIRNAME=. +set APP_BASE_NAME=%~n0 +set APP_HOME=%DIRNAME% + +@rem Resolve any "." and ".." in APP_HOME to make it shorter. +for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi + +@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m" + +@rem Find java.exe +if defined JAVA_HOME goto findJavaFromJavaHome + +set JAVA_EXE=java.exe +%JAVA_EXE% -version >NUL 2>&1 +if "%ERRORLEVEL%" == "0" goto execute + +echo. +echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. +echo. +echo Please set the JAVA_HOME variable in your environment to match the +echo location of your Java installation. + +goto fail + +:findJavaFromJavaHome +set JAVA_HOME=%JAVA_HOME:"=% +set JAVA_EXE=%JAVA_HOME%/bin/java.exe + +if exist "%JAVA_EXE%" goto execute + +echo. +echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% +echo. +echo Please set the JAVA_HOME variable in your environment to match the +echo location of your Java installation. + +goto fail + +:execute +@rem Setup the command line + +set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar + + +@rem Execute Gradle +"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %* + +:end +@rem End local scope for the variables with windows NT shell +if "%ERRORLEVEL%"=="0" goto mainEnd + +:fail +rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of +rem the _cmd.exe /c_ return code! +if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1 +exit /b 1 + +:mainEnd +if "%OS%"=="Windows_NT" endlocal + +:omega diff --git a/depth_estimation/repo_images/depth_sample.jpg b/depth_estimation/repo_images/depth_sample.jpg new file mode 100644 index 0000000..1df12c5 Binary files /dev/null and b/depth_estimation/repo_images/depth_sample.jpg differ diff --git a/depth_estimation/repo_images/rgb_sample.jpg b/depth_estimation/repo_images/rgb_sample.jpg new file mode 100644 index 0000000..a68868c Binary files /dev/null and b/depth_estimation/repo_images/rgb_sample.jpg differ diff --git a/depth_estimation/settings.gradle b/depth_estimation/settings.gradle new file mode 100644 index 0000000..3deee5b --- /dev/null +++ b/depth_estimation/settings.gradle @@ -0,0 +1,10 @@ +dependencyResolutionManagement { + repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS) + repositories { + google() + mavenCentral() + jcenter() // Warning: this repository is going to shut down soon + } +} +rootProject.name = "Depth Estimation" +include ':app'