iOS ARKit

Augmented reality (AR) describes user experiences that add 2D or 3D elements to the live view from a device's camera in a way that makes those elements appear to inhabit the real world. ARKit combines device motion tracking, camera scene capture, advanced scene processing, and display conveniences to simplify the task of building an AR experience.

ARKit tracks the real-world the user inhabits, and matches it with a coordinate space for you to place virtual content. World tracking also offers features to make AR experiences more immersive, like the ability to recognize objects and images in the user's environment and respond to real-world lighting conditions.

For iOS devices that have a TrueDepth camera, ARFaceTrackingConfiguration enables you to augment the front-camera feed, while providing you with real-time tracking for the pose and expression of faces. With that information, that you might, for example, choose to overlay realistic virtual masks. Or, you might omit the camera view and use facial expression data to animate virtual characters, as in the Animoji app for iMessage.

World-tracking AR sessions use a technique called visual-inertial odometry. This process combines motion sensor data with computer vision analysis of camera imagery to track the device's position and orientation in real-world space, also known as pose, which is expressed in the ARCamera transform property. For best results, world tracking needs consistent sensor data and camera imagery with visual complexity or recognizable features.

Basic Lifecycle of an AR Session

The figure below shows the changes in tracking state when you start running an AR session.

Immediately after you run a new session, the tracking state for provided frames is ARCamera.TrackingState.notAvailable, indicating that ARKit has not yet gathered enough information to estimate the device’s pose.

A few frames later, the tracking state changes to ARCamera.TrackingState.limited(_:), indicating that a device pose is available but its accuracy is uncertain. A limited state always includes a reason for reduced tracking quality; in this case, the session is still ARCamera.TrackingState.Reason.initializing.

After a short time, the tracking state changes to ARCamera.TrackingState.normal, indicating that the device pose is accurate and all ARKit features are available.

Tie

References:

https://developer.apple.com/documentation/arkit