Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hud10837/import render code #580

Merged
merged 10 commits into from
Sep 24, 2024

Conversation

hud10837
Copy link
Collaborator

@hud10837 hud10837 commented Sep 19, 2024

Related to issue: https://devtopia.esri.com/runtime/kotlin/issues/4639

Description:

Adds rendering code to render the camera feed for Tabletop AR. This will be used by the TableTopSceneView.

Summary of changes:

  • Adds a new internal.rendering directory to the module
  • Adds several classes related to rendering the camera feed
  • Adds a texture and shaders for rendering the camera feed and AR planes
  • Adds the ARCore dependency to the ar module

Pre-merge Checklist

@hud10837 hud10837 marked this pull request as ready for review September 20, 2024 09:39
// Shader names.
private const val VERTEX_SHADER_NAME = "shaders/plane.vert"
private const val FRAGMENT_SHADER_NAME = "shaders/plane.frag"
private const val TEXTURE_NAME = "models/trigrid.png"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this path correct? Becuase it's under "textures" in the assets folder.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, I changed the path to textures as we are not using any models

@hud10837
Copy link
Collaborator Author

Thanks @gunt0001 I have addressed your comments

Copy link
Collaborator

@gunt0001 gunt0001 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hud10837 - looks good 👍

@hud10837
Copy link
Collaborator Author

Second vtest run looks good, merging now

@hud10837 hud10837 merged commit c584bd1 into feature-branches/tabletop-ar Sep 24, 2024
@hud10837 hud10837 deleted the hud10837/import-render-code branch September 24, 2024 13:14
hud10837 added a commit that referenced this pull request Oct 4, 2024
* import necessary files for rendering camera feed

* copyright

* rm obj dependency

* make kotlin classes internal

* make java classes non-public

* centralize logging and tag

* add doc links to hello ar

* make planerenderer companion internal

* Fix texture name

* add newline
hud10837 added a commit that referenced this pull request Oct 14, 2024
* merge `Feature branches/forms` into v.next (#566)

* `Forms`: Add `TextFormElement` (#542)

* `Forms`: Add `TextFormElement` tests (#551)

* add tests

* updated feature form doc

* bump sdk version

* `Forms` : Add SubTypeFeatureLayer support (#559)

* `Forms`: Fix stale `LaunchedEffect`s (#563)

* fix stale launched effects

* use rememberupdatedstate

* update feature form doc (#565)

* Remove unnecessary creating the viewmodel in the main activity (#571)

* Set up Tabletop AR project (#545)

* prototype design options

* prototype tabletopSceneView

* remove unused implementations

* create readme

* add tabletopsceneviewproxy

* update microapp to use tabletop proxy

* add TableTopSceneViewScope

* apply Compose gradle plugin

* fix warnings

* rename microapp

* delete tests for microapp

* add copyright

* fix since years

* use swift doc on tabletopsceneview

* add proxy doc

* add scope doc

* fix references in proxy doc

* fix references in scope doc

* fix references in scope doc

* newlines

* rm unit test

* fix doc and imports

* fix doc and imports

* revert authentication changes

---------

Co-authored-by: Gunther Heppner <gheppner@esri.com>

* Hud10837/import render code (#580)

* import necessary files for rendering camera feed

* copyright

* rm obj dependency

* make kotlin classes internal

* make java classes non-public

* centralize logging and tag

* add doc links to hello ar

* make planerenderer companion internal

* Fix texture name

* add newline

* Tabletop AR: Add ArSurfaceView (#590)

* bring in existing implementation of ArSurfaceView

* bring in changes to TabletopSceneView and TableTopSceneViewState

* pare down changes to just implement ARSurfaceView and add lifecycle management wrappers for AR Session and GlSurfaceView

* rm camera controller line

* mv call to box

* mv ArSessionWrapper to internal

* fix since tags

* rename ARSurfaceView -> ArCameraFeed

* rename localLifecycleOwner -> lifecycleOwner

* mv initialization of sceneViewProxy to TableTopSceneViewProxy

* Hud10837/session init and permissions (#595)

* mv assets folder to correct location

* out of the box permissions request

* rename camera permission function

* make request permission optional

* add initialization status

* add availability check

* update microapp

* extract string resources

* rm unnecessary changes

* don't use stateflow for microapp

* rename string res

* rename string resource

* don't require camera permission before checking arcore visibility

* mv box call

* add local function for update status

* simplify availability check

* rename status

* make status constructors internal

* use updateState for initial status

* mv status doc to member objects

* add remaining param doc

* fix remaining param doc

* add parameter for callback to lambda

* revert whitespace in manifest

* remove debug delay

* add factory rememberTableTopSceneViewStatus

* add extension fun to mutable state

* add extension fun to mutable state

* rename camera feed file

* use launchedeffect to ensure initializing status is only sent once

* Don't send callback on first status

* use side effects for callback

* use collectAsStateWithLifecycle

* add doc to rememberTableTopSceneViewStatus()

* add new DetectingPlanes status

* create callback and set status for first plane detected

* add in scene placement logic

* add translation factor, anchor point, clipping distance

* rm initial viewpoint on scene

* display simple helper text while detecting planes

* use string res

* use string res for lat lon callout content

* enhance microapp with status messages

* use better data for microapp

* rm debug code

* use val instead of var for status

* cleanup

* use display rotation for lens intrinsics

* checkout unexpected files from feature branch

* checkout unexpected files from feature branch

* add doc to TextWithScrim

* use when statements and be exhaustive

* mv declarations of cameraController and arCoreAnchor

* add doc Pose.transformationMatrix

* use when (val status = ...

* use explicit api mode

* fix build warnings

---------

Co-authored-by: Kaushik Meesala <kaushik.pulagara@gmail.com>
Co-authored-by: Puneet Prakash <puneet_prakash@esri.com>
Co-authored-by: Gunther Heppner <gheppner@esri.com>
gunt0001 added a commit that referenced this pull request Nov 13, 2024
* Set up Tabletop AR project (#545)

* prototype design options

* prototype tabletopSceneView

* remove unused implementations

* create readme

* add tabletopsceneviewproxy

* update microapp to use tabletop proxy

* add TableTopSceneViewScope

* apply Compose gradle plugin

* fix warnings

* rename microapp

* delete tests for microapp

* add copyright

* fix since years

* use swift doc on tabletopsceneview

* add proxy doc

* add scope doc

* fix references in proxy doc

* fix references in scope doc

* fix references in scope doc

* newlines

* rm unit test

* fix doc and imports

* fix doc and imports

* revert authentication changes

---------

Co-authored-by: Gunther Heppner <gheppner@esri.com>

* Hud10837/import render code (#580)

* import necessary files for rendering camera feed

* copyright

* rm obj dependency

* make kotlin classes internal

* make java classes non-public

* centralize logging and tag

* add doc links to hello ar

* make planerenderer companion internal

* Fix texture name

* add newline

* Tabletop AR: Add ArSurfaceView (#590)

* bring in existing implementation of ArSurfaceView

* bring in changes to TabletopSceneView and TableTopSceneViewState

* pare down changes to just implement ARSurfaceView and add lifecycle management wrappers for AR Session and GlSurfaceView

* rm camera controller line

* mv call to box

* mv ArSessionWrapper to internal

* fix since tags

* rename ARSurfaceView -> ArCameraFeed

* rename localLifecycleOwner -> lifecycleOwner

* mv initialization of sceneViewProxy to TableTopSceneViewProxy

* Hud10837/session init and permissions (#595)

* mv assets folder to correct location

* out of the box permissions request

* rename camera permission function

* make request permission optional

* add initialization status

* add availability check

* update microapp

* extract string resources

* rm unnecessary changes

* don't use stateflow for microapp

* rename string res

* rename string resource

* don't require camera permission before checking arcore visibility

* mv box call

* add local function for update status

* simplify availability check

* rename status

* make status constructors internal

* use updateState for initial status

* mv status doc to member objects

* add remaining param doc

* fix remaining param doc

* add parameter for callback to lambda

* revert whitespace in manifest

* remove debug delay

* add factory rememberTableTopSceneViewStatus

* add extension fun to mutable state

* add extension fun to mutable state

* rename camera feed file

* use launchedeffect to ensure initializing status is only sent once

* Don't send callback on first status

* use side effects for callback

* use collectAsStateWithLifecycle

* add doc to rememberTableTopSceneViewStatus()

* Tabletop AR: scene placement (#602)

* merge `Feature branches/forms` into v.next (#566)

* `Forms`: Add `TextFormElement` (#542)

* `Forms`: Add `TextFormElement` tests (#551)

* add tests

* updated feature form doc

* bump sdk version

* `Forms` : Add SubTypeFeatureLayer support (#559)

* `Forms`: Fix stale `LaunchedEffect`s (#563)

* fix stale launched effects

* use rememberupdatedstate

* update feature form doc (#565)

* Remove unnecessary creating the viewmodel in the main activity (#571)

* Set up Tabletop AR project (#545)

* prototype design options

* prototype tabletopSceneView

* remove unused implementations

* create readme

* add tabletopsceneviewproxy

* update microapp to use tabletop proxy

* add TableTopSceneViewScope

* apply Compose gradle plugin

* fix warnings

* rename microapp

* delete tests for microapp

* add copyright

* fix since years

* use swift doc on tabletopsceneview

* add proxy doc

* add scope doc

* fix references in proxy doc

* fix references in scope doc

* fix references in scope doc

* newlines

* rm unit test

* fix doc and imports

* fix doc and imports

* revert authentication changes

---------

Co-authored-by: Gunther Heppner <gheppner@esri.com>

* Hud10837/import render code (#580)

* import necessary files for rendering camera feed

* copyright

* rm obj dependency

* make kotlin classes internal

* make java classes non-public

* centralize logging and tag

* add doc links to hello ar

* make planerenderer companion internal

* Fix texture name

* add newline

* Tabletop AR: Add ArSurfaceView (#590)

* bring in existing implementation of ArSurfaceView

* bring in changes to TabletopSceneView and TableTopSceneViewState

* pare down changes to just implement ARSurfaceView and add lifecycle management wrappers for AR Session and GlSurfaceView

* rm camera controller line

* mv call to box

* mv ArSessionWrapper to internal

* fix since tags

* rename ARSurfaceView -> ArCameraFeed

* rename localLifecycleOwner -> lifecycleOwner

* mv initialization of sceneViewProxy to TableTopSceneViewProxy

* Hud10837/session init and permissions (#595)

* mv assets folder to correct location

* out of the box permissions request

* rename camera permission function

* make request permission optional

* add initialization status

* add availability check

* update microapp

* extract string resources

* rm unnecessary changes

* don't use stateflow for microapp

* rename string res

* rename string resource

* don't require camera permission before checking arcore visibility

* mv box call

* add local function for update status

* simplify availability check

* rename status

* make status constructors internal

* use updateState for initial status

* mv status doc to member objects

* add remaining param doc

* fix remaining param doc

* add parameter for callback to lambda

* revert whitespace in manifest

* remove debug delay

* add factory rememberTableTopSceneViewStatus

* add extension fun to mutable state

* add extension fun to mutable state

* rename camera feed file

* use launchedeffect to ensure initializing status is only sent once

* Don't send callback on first status

* use side effects for callback

* use collectAsStateWithLifecycle

* add doc to rememberTableTopSceneViewStatus()

* add new DetectingPlanes status

* create callback and set status for first plane detected

* add in scene placement logic

* add translation factor, anchor point, clipping distance

* rm initial viewpoint on scene

* display simple helper text while detecting planes

* use string res

* use string res for lat lon callout content

* enhance microapp with status messages

* use better data for microapp

* rm debug code

* use val instead of var for status

* cleanup

* use display rotation for lens intrinsics

* checkout unexpected files from feature branch

* checkout unexpected files from feature branch

* add doc to TextWithScrim

* use when statements and be exhaustive

* mv declarations of cameraController and arCoreAnchor

* add doc Pose.transformationMatrix

* use when (val status = ...

* use explicit api mode

* fix build warnings

---------

Co-authored-by: Kaushik Meesala <kaushik.pulagara@gmail.com>
Co-authored-by: Puneet Prakash <puneet_prakash@esri.com>
Co-authored-by: Gunther Heppner <gheppner@esri.com>

* TableTop improvements (#607)

* stop rendering planes as soon as the anchor point is determined

* configure ARCore Session to only detect horizontal planes

* disable pan/zoom/rotate interaction

* make clippingDistance an optional parameter and ensure modifier is first optional param

* enhance comment

Co-authored-by: hud10837 <hmiears@esri.com>

---------

Co-authored-by: hud10837 <hmiears@esri.com>

* TableTop doc (#644)

* expand AR readme

* explain translationFactor

* update micro app to use simplified dataset

* cache arcGISSceneAnchor in the composition

* add prerequisites and usage sections

* doc fixes

* add AR to top level readme

* remove basemap and make surface opaque

* add screenshot

* update README

* improve doc on TableTopSceneView parameters

* add README for ARTableTop micro app

* improve doc on translationFactor

Co-authored-by: hud10837 <hmiears@esri.com>

* refine doc for prerequisites

* add snippet for AR required manifest

* fix spelling

Co-authored-by: hud10837 <hmiears@esri.com>

---------

Co-authored-by: hud10837 <hmiears@esri.com>

* Tabletop - disable navigation (#662)

* remove interaction options from TableTopSceneView

* remove setViewpoint functions from TableTopSceneViewProxy

* remove unused imports

* Tabletop callout fix & Session config (#663)

* cache _isManualRenderingEnabled and sync with SceneView when set

* invoke content lambda when SceneView ready or in manual rendering mode

* configure AR Session in onResume instead of upon construction

* change ArCameraFeed constructor to take a Session instead of ArSessionWrapper

* create Session in onResume, expose as StateFlow

* observe session state flow in order to render the ArCameraFeed

* lock the screen of the TableTop micro app

* roll back screen orientation lock

* sync manual rendering flag in sceneView property setter

* remove `onDrawStatusChanged` from TableTopSceneView

* fix link

* fix link

* fix link

---------

Co-authored-by: hud10837 <hmiears@esri.com>
Co-authored-by: Kaushik Meesala <kaushik.pulagara@gmail.com>
Co-authored-by: Puneet Prakash <puneet_prakash@esri.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants