ARCore Document

 

 

ARCore Document

ARCore Overview ARCore is Google’s platorm for building augmened realiy experiences. Using dieren APIs, ARCore enables your phone o sense is environmen, unders undersand and he world and inerac wih informaon. Some of he APIs are available across Android and iOS o enable e nable shared AR experiences. ARCore uses hree key capabilies o inegrae virual conen wih he real world as seen hrough your phone's camera: Moon racking allows he phone o undersand and rack is posion relave o he world. Environmenal Environmen al undersandin undersanding g allows he phone o deec he size and locaon of all ype of surfaces: horizonal, horizonal, vercal and angled surfaces like he ground, a coee able or walls. Ligh esmaon allows he phone o esmae he environmen' environmen'ss curren lighng condions. Supported Devices

ARCore is designed o work on a wide variey of qualied Android phones running Android 7.0 (Nouga) and laer. A full lis of all suppored devices is available here. How does ARCore work?

Fundamenally, ARCore is doing wo hings: racking he posion of he mobile device as i moves, and building is own undersanding of he real world. ARCore's moon racking echnology uses he phone's camera o idenfy ineresng poins, cal called led feaure feaures, s, and racks racks how hose poins poins move over over me. me. Wi Wih h a com combin binao aon n of he movemen of hese poins and readings from he phone's ineral sensors, ARCore deermines boh bo h he posion posion and oriena orienaon on of he phone as i mov moves es hr hroug ough h spa space. ce.In In add addio ion n o idenfying key poins, ARCore can deec a surfaces, like a able or he oor, and can also esmae he average lighng in he area around i. These capabilies combine o enable ARCore o build is own undersanding of he world around i.ARCore's undersanding of he real world les you place objecs, annoaons, or oher informaon in a way ha inegraes seamlessly wih he real world. You can place a napping kien on he corner of your coee able, or annoae a painng wih biographical informaon abou he ars. Moon racking means ha you can move around and view hese objecs from any angle, and even if you urn around and leave he room, when you come back, he kien or annoaon will be righ w where here you le i.

Fundamental Concepts

 

Before diving ino ARCore, i's helpful o undersand a few fundamenal conceps. Togeher, hese conceps illusrae how ARCore enables experiences ha can make virual conen appear o res on real surfaces or be aached o real world locaons.

Moon racking As your phone moves hrough he world, ARCore uses a process called concurren odomery and mapping, or COM, o undersand where he phone is relave o he world around i. ARCore deecs visually disnc feaures in he capured camera image called feaure poins and uses hese poins o compue is change in locaon. The visual informaon is combined wih ineral measuremens from he device's IMU o esmae he pose (posion and orienaon) of he camera relave o he world over me. By aligning he pose of he virual camera ha renders your 3D conen wih he pose of he device's camera provided by ARCore, developers are able o render virual conen from he correc perspecve. The rendered virual image can be overlayed on op of he image obained from he device's camera, making i appear as if he virual conen is par of he real world.

Environmenal Environmen al undersanding ARCore is consanly improving is undersanding of he real world environmen by deecng feaure poins and planes.ARCore looks for clusers of feaure poins ha appear o lie on common horizonal or vercal surfaces, like ables or walls, and makes hese surfaces available o yo your ur app as planes planes.. ARC ARCor ore e can also deerm deermine ine each plane' plane'ss bou bound ndary ary and make ha informaon available available o your app. You can use his informaon o place virual objecs resng on a surfaces.Because ARCore uses feaure poins o deec planes, a surfaces wihou exure, such as a whie wall, may no be deeced properly.

Ligh esmaon ARCore can deec informaon abou he lighng of is environmen and provide you wih he average inensiy and color correcon of a given camera image. This informaon les you ligh your virual objecs under he same condions as he environmen around hem, increasing he sense of realism.

User inerac ineracon on ARCore ARC ore use usess hi es esng ng o ak ake e an (x, (x,y) y) coord coordina inae e cor corres respon pondin ding g o he ph phone one's 's scr screen een (provided by a ap or whaever oher ineracon you wan your app o suppor) and projecs a ray ino he camera's view of he world, reurning any planes or feaure poins ha he ray inersecs, along wih he pose of ha inersecon in world space. This allows users o selec or oherwise inerac wih objecs in he environmen.

 

Oriened Poins Oriened poins les you place virual objecs on angled surfaces. When you perform a hi es ha reurns a feaure poin, ARCore will look a nearby feaure poins and use hose o aemp o esmae he angle of he surface a he given feaure poin. ARCore will hen reurn a pose ha akes ha angle ino accoun.Because ARCore uses clusers of feaure poins o deec he surface's angle, surfaces wihou exure, such as a whie wall, may no be deeced properly.

Anchors and Trackables Poses can change as ARCore improves is undersanding of is own posion and is environmen environmen. . When you wan o place a virual objec, you need o dene an anchor o ensure ha ARCore racks he objec's posion over me. Oen mes you creae an anchor based on he pose reurned by a hi es, as described in user ineracon. ineracon.The The fac ha poses can change means ha ARCore may updae he posion of environmenal objecs like planes and feaure poins over me. Planes and poins are a special ype of objec called a rackable. Like he name suggess, hese are objecs ha ARCore will rack over me. You can anchor virual objecs o specic rackables o ensure ha he relaonship beween your virual objec and he rackable remains sable even as he device moves around. This means ha if you place a virual Android gurine on your desk, if ARCore laer adjuss he pose of he plane associaed wih he desk, he Android gurine will sll appear o say on op of he able.

Augmened Images Augmened Images allows you o build AR apps ha can respond o specic 2D images such as produc packaging or movie posers. Users can rigger AR experiences when hey poin heir phone's camera a specic images - for insance, hey could poin heir phone's camera a a movie poser and have a characer pop ou and enac a scene.Images can be compiled oine o creae an image daabase, or individual images can be added in real me from he device. Once regisered, ARCore will deec hese images, he images boundaries, and reurn a corresponding pose.

Sharing ARCore Cloud Anchor API les you creae collaborave or mulplayer apps for Android and iOS devices.Wih Cloud Anchors, one device sends an anchor and nearby feaure poins o he cloud for hosng. These anchors can be shared wih oher users on eiher Android or iOS devices in he same environmen. This enables apps o render he same 3D objecs aached o hese anchors, leng users have he same AR experience simulaneously simulaneously.