Brainstorming & Research

21/11/22

Mystery posters app

Ideas

App that scans mystery posters and shows (3D?) AR image on/out of poster (mystery posters that show surprise if you scan them)
- react native
- https://medium.com/simform-engineering/what-is-augmented-reality-how-to-implement-ar-using-react-native-2340bdba9a8d
  - ViroReact
    - based on 2 APIs (ARKit for iOS and ARCore for Android)
    - open source
- Viro
  - https://github.com/viromedia/viro
- Aframe : web framework
  - https://aframe.io/
  - https://aframe.io/blog/arjs3/
- Digitally decorate your room
- hidden messages
- put yourself on a billboard
- secret message for someone → send it to someone
- put it in a place that means something to you → only you can see what it is
- ‘for someone you love’, ‘to make you laugh’, ‘to make you cry’, ‘something scary’, ‘I want to tell you a secret’
- look inside the building → becomes a window
- send location + secret message →
- own printable template that you can hang anywhere
- could also work for ‘enemies’ → send them very far to see a pointless message
- wireframes:
  - account?
  - map with locations
  - send/create
  - received messages/map
  - notifications
Looking on creative belgium for inspiration
- https://www.creativebelgium.be/new-talent-award-briefings
  - game for mums
  - find interesting way to teach maths

22/11/22

_Feedback moment_1 See feedback page for details.

Researching and brainstorming

AR game → when scanning area → see the game play
https://www.8thwall.com/?gclid=Cj0KCQiA4OybBhCzARIsAIcfn9ne3fRDjHydzZS3blPMjTWBK7rRX3IbZSAVHUfZXBC0_5siulQh6fcaAt__EALw_wcB
- image targets: https://www.8thwall.com/products-web#image-target
- support for three.js
- a-frame
AR posters
- https://educationaltechnology.wooster.edu/ar-posters/
- https://medium.com/@devdevcharlie/how-to-create-augmented-reality-posters-with-unity-vuforia-ec80a82e6d51 → unity + vuforia
  - target image
Comparing options
- https://answers.unity.com/questions/1456462/arkit-vs-arcore-vs-vuforia-vs-8th-wall.html
- vuforia: runs on both iOS and android + older phones
  - paid (or watermark)
  - uses arKit or arCore if phone supports it, otherwise own AR tech
  - has image tracking
  - well-documented
  - baked into unity
- 8th wall
  - doesn’t have image tracking
  - but free
  - great documentation
  - same as vuforia (both iOS and android + older devices)
- EasyAR (OpenVC)
  - free image tracking
- ARKit (iOS) and ARCore (android)
  - more stable while building apps
Do I need an app?
- https://blog.cws.net/do-we-need-an-app-5-things-to-consider
  - cons of apps
    - no live update
    - website: as many updates as you want, live immediately
    - bad SEO
    - need Google/Apple approval for distribution
      - need different versions for the different platforms
    - asking for permissions: discouraging for users
    - uses phone storage
- https://vwo.com/blog/10-reasons-mobile-apps-are-better/
  - apps: more personalisation
  - sending notifications (in-app or push notifications) (→ but there are web push notifications)
  - using device features (camera, contact list, …)
  - can work offline
  - websites depend on browsers for styling
    - app: easily use gestures
  - apps can work faster (locally stored data)
- https://www.crowdbotics.com/blog/how-to-know-if-you-really-do-or-dont-need-a-mobile-app
  - https://clearbridgemobile.com/mobile-app-vs-mobile-website-which-is-the-better-option/
react native
- https://mmcgbl.com/top-react-native-features/
  - live reloading
AR
- https://en.yeeply.com/blog/frameworks-sdks-augmented-reality-app/
  - marker based AR
    - https://digitalpromise.org/initiative/360-story-lab/360-production-guide/investigate/augmented-reality/getting-started-with-ar/types-of-ar/.
- AR on the web
  - https://ar-js-org.github.io/AR.js-Docs/
    - image tracking
    - marker tracking
    - works with webgl and webrtc
    - marker-based: light weight, image tracking: CPU consuming
    - problem in chrome with phones with multiple cameras
    - need https
    - image tracking documentation
      - https://ar-js-org.github.io/AR.js-Docs/image-tracking/
    - https://www.unitear.com/WebAR
Idea: send AR cards to people
other cross platform app frameworks than react native
- https://www.aalpha.net/articles/cross-platform-app-developent-frameworks/
https://www.intelivita.com/blog/augmented-reality-frameworks/.

Feedback moment 2 See feedback page for details.

Further research

web AR vs native AR
- https://rockpaperreality.com/insights/web-ar/why-web-based-ar-outperforms-apps/.
  - Browsers now allow Web AR to use the camera, gyroscope, magnetometer, and accelerometer of devices, giving developers the tools needed to produce creative experiences.
  - key differences
    - app:
      - more features available
      - better-quality computer vision that ARKit and ARCore
      - better performance (more memory)
      - con: user needs to download the app
      - con: needs to be compatible with multiple devices & operating systems
    - web
      - accessible through QR code or link
        
        greater reach/engagement
      - reaches more mobile devices
      - less power
      - instant accessibility across multiple devices and platforms.
- https://rockpaperreality.com/insights/web-ar/is-webar-better-than-ar-apps/
  - web
    - via NFC tag, QR, link
    - Due to the constraints on hardware and network resources, WebAR experiences are generally slightly shorter and less detailed than traditional AR app experiences. For most use cases, however, WebAR more than compensates due to its accessibility on all smartphones. For extremely detailed scenes, however, one may prefer to develop a more expensive AR app.

→ these 2 articles were written by company that uses WebAR to create AR experiences, so they might be less positive about AR apps

28/11/22

Brainstorming about the functionalities that I want

Scan a card/poster
- Need camera access/permissions
Create new content for a card or poster
- some text
- some (3D) figures/animations?
- AR
Send card to someone
- notifications
Receive/open card
- Saved cards that people sent to them

Comparing Progressive Web Apps with Apps

camera access
- PWA
  - permissions on iOS
sending card
- PWA:
  - via link/QR code
- App: need an account already? need to download the app already
notifications
- PWA
  - https://binarapps.com/push-notifications-in-pwa-everything-you-need-to-know/ (4/2/22)
    - Service Worker:
      - JS file that runs separately from the main browser thread
      - Notifications API and Push API
      - Service Workers and HTTPS, as web push notifications can only be implemented on HTTPS enabled domains to prevent potential malicious attacks.
      - push notifications on iOS?
        
        in practice, no official support for push notifications from Apple
        
        work around
        
        requires application ID to be registered with the Apple Developer Portal with the approval of the relevant service
        
        via Apple Wallet
        
        not simple or practical
- PWA push notifications for iOS
  - https://www.izooto.com/blog/ios-safari-push-notifications
    - starting from 2023
    - no Apple Developer Account needed anymore
    - current ‘solution’: facebook messenger
saving cards
- if just a link or QR, what if they lose it?
create card
- add text
- choose from figures (3D)
- add pictures from your phone?
offline access
AR
- iOS
  - https://developer.apple.com/augmented-reality/
PWA vs App
- https://www.teknicks.com/blog/pwa-progressive-web-app-vs-native-apps-which-is-better/
  - PWA:
    - pro:
      - cheaper to develop
      - only one version (responsive)
      - update and deploy changes without needing approval
      - more control over SEO
      - more secure than normal websites (https)
      - don’t require installation
      - can add pwa to homescreen
      - pwa can be shared with others via url
      - can be used offline when downloaded to homescreen (via service workers)
    - con
      - people trust security of links less
      - more latency and battery use
      - some restrictions to accessing your device features (especially iOS)
  - App
    - con:
      - need different version for different devices
      - needs updates to maintain in each app store (updates need to be approved by app store)
      - need to submit to different stores
      - app needs to be easy to notice in app store (imagery, reviews, name, keywords)
      - a lot of commitment to download an app
      - uses local phone memory
      - user needs to update app as well
    - pro
      - requirements of app stores increases your reliability
      - web stores can promote your app
      - people trust security of apps more, as they need to pass app store requirements
      - hardware can deliver better experience
      - faster
      - more powerful
      - can leverage platform specific tools/ take advantage of operating system features
      - offline usage
      - geofencing → combined with push notifications, can give real time updates
      - faster loading time (does not depend on browser or internet connection, but on operating system of phone)
      - credibility
      - low latency
      - better at exploiting device features like geofencing, sensor detection, and more.

app: better for complex product that needs increased control of the device, when security is vital, speed and UI, low latency

send it to someone who doesn’t know it

would you trust a random link?

would you download an app just to see this?

FLOW/FEATURE BRAINSTORM

Sing up/ Log in
- how long can you keep logged in on a website vs app?
Only need to be logged in when you want to save or send a card/poster?
- sing in to save/send your creation
Create a new card/poster
- send it to someone
sent folder
inbox
template print out
- choose color, but a little text
scanner
- when card is recognised → asks which one you want to see
  - saved cards
  - received ones
- take a snapshot (real picture + AR)
favouriting creations
push notifications
- you received a card
- your card is delivered
ppsst someone sent you a secret message
a message only you two can see

If you send it to someone who doesn’t know the product yet?

→ better via a link?

Look at it offline

camera access/permissions
push notifications
save creations
send creations to someone else
view creations and received cards offline
sign up
staying logged in

Further research

PWA: pros and cons

https://www.d-tt.nl/en/articles/pwa-progressive-web-apps-pros-cons

all devices
also via computer
push notifications only on android
can access camera
no installation required
can be opened directly via URL
can be added to phone home screen
user always has the most up to date version
uses less storage space
partial offline usage
SEO
cheaper

CONS

user needs to visit the website itself to be able to save it to home screen (not accessible via app store)
fewer functionalities → can’t access contacts, calendar, bluetooth…
apple doesn’t support all functionalities yet (e.g. push notifications)
lower performance than apps

https://www.adjust.com/blog/native-app-vs-progressive-web-app/

PWA
- pros
  - less storage space
  - loading speed (service workers)
  - unified user experience for all devices
  - lower cost
  - more online visibility
- cons
  - need to write native wrapper to put PWA on app store, much harder process + need valid proof that you are legal, registered business
  - needs own security certification
  - limited offline capabilities
  - uses more battery power
native app
- pros
  - integration:
    - can interact with other native apps
    - e.g. connect with Facebook
  - can integrate payment processing of app store
  - app store visibility
  - security
    - easier to implement robust security features (app has access to necessary device information)
  - more efficient on battery consumption
- cons
  - more complex development
  - maintenance
  - not multi platform

Camera access/asking permissions

https://code-boxx.com/javascript-camera-pwa/

PWA with camera access that works offline

https://stackoverflow.com/questions/61366147/how-to-store-camera-permission-in-ios-pwa

iOS problem with needing to keep reconfirming camera permissions

https://firt.dev/notes/pwa-ios/

https://docs.mendix.com/refguide/mobile/introduction-to-mobile-technologies/progressive-web-app/

PWAs have the following limitations on iOS:

Push notifications are not supported for PWAs on iOS
Offline data is not supported for PWAs on iOS

https://medium.com/arjs/why-web-apps-are-the-future-of-augmented-reality-c503e796a0c5

it becomes harder to make people install apps on their phone
discoverability for apps is not that good
competition for screen time with the big apps
people delete apps quickly
WebAssembly
WebRTC (also iOS?)

https://scandiweb.com/blog/wp-content/uploads/2020/09/pwa-nativeApps_feature-comparison.pdf

https://love2dev.com/pwa/ios/

iOS does not support push notifications for PWA

https://web.dev/file-system-access/

https://developer.mozilla.org/en-US/docs/web/api/window/showopenfilepicker#browser_compatibility

access file system via web app → limited browser compatibility

AR

https://program-ace.com/blog/augmented-reality-sdk/

Software developer kits (SDK)

vuforia (unity)
wikitude (apps)
ARKit → apple
ARCore → google (android), compatible with vuforia
ARToolkit → open source, free, image tracking

https://www.g2.com/categories/ar-sdk

https://developer.vuforia.com/

https://matthiashamann.work/en/blog/article/ar-frameworks-in-comparison

Making choices

USER JOURNEY

MYSTERY MAIL

Receiver
- person receives mystery post card via mail
- QR code (and general link + unique code as alternative)
- goes to site via one of the links
- gets welcome message that says some mystery mail is waiting for them
- needs to give permission to access camera
- instructions on top of camera view to scan the post card
- scans post card with camera
- mystery message is shown with AR
- options
  - take picture with your phone to save the message
  - make account to save online for later
  - send a card back/ create your own card
Sender
- create a card with text and some 3D figurine choices
- can download (and print) card template if they want to try it out
- can save concept and edit later on
- can save
- can send to a friend
  - enter address
  - enter your own email address to get confirmation
  - make account if they want to save their card
  - can choose physical copy or email digital version (print out or just scan computer screen)

Final conclusion:

As the user journey consists for a big part of users receiving a card, who do not know the product, it is less cumbersome to just scan a code or go to a site.

Maybe in a later stage → use this site as platform to encourage the app (more power for AR and allows for push notifications on iOS)

Only limitations for PWA are notifications, repeated camera permissions (iOS) and staying logged in (but log in is not required for MVP)

→ can opt to just send email to themselves w

PWA: React vs Vue vs Angular

https://www.simicart.com/blog/pwa-frameworks/

https://bsscommerce.com/blog/best-pwa-framework-angular-vue-react/

PWA with React

https://create-react-app.dev/docs/making-a-progressive-web-app/

https://blog.logrocket.com/building-pwa-react/

Service Workers

https://web.dev/learn/pwa/service-workers/