project soli seminar report.docx

MAHANT BACHITTER SINGH COLLEGE OF ENGINEERING & TECHNOLOGY

SEMINAR REPORT ON “PROJECT SOLI” Submitted in the partial fulfillment of requirement for the award of degree of Bachelor of Engineering in Computer Science Engineering.

Submitted By: Asmita Gautam Roll No. : 272/13

DEPARTMENT OF COMPUTER SCIENCE & ENGG M.B.S.COLLEGE OF ENGINEERING & TECHNOLOGY BABLIANA, JAMMU-181101

Mahant Bachittar Singh College of Engineering & Technology Jammu

CERTIFICATE

It is to certify that the Seminar Report titled “ Project Soli” is presented by Asmita Gautam (roll no. – 272/13) is partial fulfillmentof the requirement for the award of the degreeof Bachelor of Engineering in ComputerScience & Engineering from Mahant Bachittar Singh College of Engineering & Technology, Jammu.

Amiteshwar Singh (HOD, CSE)

Babliana, Jeewan Nagar Road, P.O. Miran Sahib, Jammu-181 101, J&K State,(India) Tele: 0191-2262897 # Fax: 0191 – 2262896 # E.mail: [email protected]

ACKNOWLEDGEMENT It is not possible to prepare any seminar report without the assistance and encouragement of other people. This one is no exception.A Dissertation is created with a blend of ideas, views, suggestions and contributions. I express my thankfulness to all those people who helped me and encouraged me while preparing this report. It is indeed gratifying to have the privilege to express my deep sense of gratitude and appreciation to Mr. Amiteshwar Singh, HoD, Department of Computer Science and Engineering, MBSCET, for his incessant encouragement, inspiring supervision and valuable guidance during the course of this seminar. For all his kind consideration I’m beholden him in my feelings of respect of regards for him. I would also like to acknowledge all my teachers who helped me in successfully completing this seminar. At last, I wish to thank my parents for constantly encouraging me to learn and for supporting every decision of mine. Their personal sacrifice in providing this opportunity is gratefully acknowledged.

Name: ___Asmita Gautam__ Roll No. : ___ 272/13___

3

ABSTRACT Project Soli is a new technology that uses radar to enable new types of touch less interactions. This technology considers the design of a human gesture recognition system based on pattern recognition of signatures from a portable smart radar sensor. The movements of gestures from a human can be captured using a radar sensor, and by detection of theses gestures, some special task on a device can be done. The project is under research by Google ATAP, and it is termed as Project Soli. In this technology, a Radar sensor along with a capturing system is made into a small chip and this chip can be connected to any device like Computer, Smartphone etc. The different functions in these devices like Call, Volume control, Zoom etc. can be done using specific gesture without having to touch or use another interaction method.

4

TABLE OF CONTENTS

CHAPTER NO. PAGE NO. CHAPTER 1 1. INTRODUCTION 1 1.1 GOOGLE 1 1.2 GOOGLE ATAP 1 1.3 PROJECT SOLI 3 CHAPTER 2 2. RADAR TECHNOLOGY 5 2.1 HOW IT WORKS? 5 2.2 BASIC DESIGN OF RADAR SYSYTEM 6 2.3 ADVANTAGES OF RADAR TECHNOLOGY 7 CHAPTER 3 3. THE SOLI SENSOR 8 3.1 HOW DOES IT WORK? 9 3.2 VIRTUAL TOOL GESTURE 11 CHAPTER 4 4. ALPHA DEVELOPMENT KIT 17 4.1 RESULTS OF ALPHA DEVELOPMENT KIT 22 5

CHAPTER 5 5. APPLICATIONS OF PROJECT SOLI 26 CHAPTER 6 6. ADVANTAGES AND DISADVANTAGES OF PROJECT SOLI 30 CONCLUSION 31 REFERENCES 32 LIST OF FIGURES Fig. No.

Description

Page No.

1.1

Logo for Google

1

1. 2

Google’s ATAP division

2

1.3

Project Soli

3

2.1

Radar

5

3.1

Initial prototype

8

3.2

Soli Chip

9

3.3(a)

Working of Radar

10

3.3(b)

Working of Radar

10

3.3(c)

Working of Radar

11

3.4(a)

Pinching motion can indicate press of a button

12

3.4(b)

Pinching motion can indicate press of a button

12

3.5(a)

Rubbing a finger along the length of a thumb

13

3.5(b)

Rubbing a finger along the length of a thumb

13

3.6(a)

Pulling your thumb across your pointer knuckle

14

3.6(b)

Pulling your thumb across your pointer knuckle

14

3.7(a)

Rubbing the tip of your finger and thumb together

15

3.7(b)

Rubbing the tip of your finger and thumb together

15

4.1

Soli Development Kit

17 6

4.2

Smart Watch

18

4.3

Soli Chip Embedded in Smart Watch

19

4.4

Beta Development Kit

21

4.5

Object Detection

22

4.6

Visualization Interface

23

4.7

Gesture Security Password

24

4.8

Predictive Drawing

25

5.1

Gaming with Soli Chip

26

5.2

Smartphone with Embedded Soli Chip

27

5.3

Android Wear with Embedded Soli Chip

28

5.4

For viewing Patient Reports

29

7

PROJECT SOLI

CHAPTER-1 INTRODUCTION 1.1 GOOGLE Google is an American multinational technology company specializing in Internet-related services and products that include online advertising technologies, search, cloud computing and software.Google wasfounded by LarryPage andSergey Brin while they were still Ph.D. Students at StanfordUniversity, California.

Fig. 1.1 Logo for Google

It runs one of the largest search engines on the World Wide Web (WWW). Every day 200 million (200,000,000) people use it. Google's main office ("Googleplex") is in Mountain View, California, United States.[1] 1.2 GOOGLE ATAP Google’s Advanced Technology and Projects group (ATAP) is a skunk works team and inhouse technology incubator, created by former DARPA director Regina Dugan. ATAP is similar to X, but works on shorter projects, granting project leaders only two years in which to move a project from concept to proven product. According to Dugan, the ideal ATAP Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [1 ]

PROJECT SOLI

project combines technology and science, requires a certain amount of novel research, and creates a marketable product within the two-year time frame. Historically, the ATAP team was born at Motorola and kept when Google sold Motorola to Lenovo; for this reason, ATAP ideas have tended to involve mobile hardware technology.

Fig. 1.2 Google’s ATAP division

Some of the announced projects by Google ATAP to date are described below: 

Project Tango is a computer-vision technology that allows mobile devices to detect their position relative to the world around them, without requiring GPS or other



external signals. Project Jacquard is a new system for weaving technology into fabric, transforming everyday objects like clothes into interactive surfaces. It will allow designers and



developers to build connected, touch sensitive textiles into their own products. Project Soli is a new interaction sensor using radar technology. The sensor can track sub-millimetre motions with high speed and accuracy. It fits onto a chip, can be produced at scale and build into small devices and everyday objects.[2] Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [2 ]

PROJECT SOLI



Project Vault, a secure computing environment on a Micro SD card.

Regina Dugan created and leads ATAP. Dugan is the former director of DARPA, the independent research branch of the U.S. Department of Defence (Do D). The group has just 75 full-time employees, more former DARPA workers among them. ATAP has about a dozen projects in the works at any given time, each of them in development over a 2-year span.[3]

1.3. PROJECT SOLI Project Soli is a new way of touch less interactions onewherethe human hand becomes a natural, intuitive interface for our devices. It is a sensing technology that uses miniature radar to detect for motion tracking of the human hand.The founder of Project Soli is Ivan Poupyrev.Project Soli was revealed during Google I/O 2015 on May 29, 2015.

Fig. 1.3. Project Soli

Google and Infineon are working together on project Soli. Infineon will provide hardware and application support while Google will provide software documentation and algorithms. The RADAR technology is used in project Soli. Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [3 ]

PROJECT SOLI

CHAPTER-2 RADAR TECHNOLOGY Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [4 ]

PROJECT SOLI

Radar is an acronym for RAdio Detecting And Ranging. The name itself suggests that the radars are used to detect the presence of object and determine its range, i.e., distance and bearing, using radio frequency waves.

Fig. 2.1. Radar

2.1 HOW IT WORKS? The electronic principle on which radar operates is very similar to the principle of soundwave reflection. If you shout in the direction of a sound-reflecting object (like a rocky canyon or cave), you will hear an echo. If you know the speed of sound in air, you can then estimate the distance and general direction of the object. The time required for an echo to return can be roughly converted to distance if the speed of sound is known. Radar uses electromagnetic energy pulses in the same way. The radio-frequency (RF) energy is transmitted to and reflected from the reflecting object. A small portion of the reflected energy returns to the radar set. This returned energy is called an ECHO, just as it is in sound terminology. Radar sets use the echo to determine the direction and distance of the reflecting object.

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [5 ]

PROJECT SOLI

2.3. BASIC DESIGN OF RADAR SYSTEM The following figure shows the operating principle of a primary radar set. The radar antenna illuminates the target with a microwave signal, which is then reflected and picked up by a receiving device. The electrical signal picked up by the receiving antenna is called echo or return. The radar signal is generated by a powerful transmitter and received by a highly sensitive receiver.

Fig . 2.2 Design of Radar

All targets produce a diffuse reflection i.e. it is reflected in a wide number of directions. The reflected signal is also called scattering. Backscatter is the term given to reflections in the opposite direction to the incident rays. Radar signals can be displayed on the traditional plan position indicator (PPI) or other more advanced radar display systems. A PPI has a rotating vector with the radar at the origin, which indicates the pointing direction of the antenna and hence the bearing of targets. 

Transmitter The radar transmitter produces the short duration high-power rf pulses of energy that are into space by the antenna. Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [6 ]

PROJECT SOLI



Duplexer The duplexer alternately switches the antenna between the transmitter and receiver so that only one antenna need be used. This switching is necessary because the highpower pulses of the transmitter would destroy the receiver if energy were allowed to enter the receiver.



Receiver The receivers amplify and demodulate the received RF-signals. The receiver provides video signals on the output.



Radar Antenna The Antenna transfers the transmitter energy to signals in space with the required distribution and efficiency. This process is applied in an identical way on reception.[4]

2.3. ADVANTAGES OF RADAR  Radar is able to operate day or night, in lightness or darkness over a long range;  Radar is able to operate in all weathers, in fog and rain, it can even penetrate walls or layers of snow;  Radar has very broad coverage; it is possible to observe the whole hemisphere;  Radar detects and tracks moving objects, a high resolution imaging is possible, that results in an object recognition;  Radar can operate unmanned, 24 hours a day, 7 days a week.

CHAPTER-3 SOLI SENSOR

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [7 ]

PROJECT SOLI

Soli’s radar sensor is a marvel in many respects. For one thing, it solves a long-lived issue when it comes to gesture-recognition technology. Previous forays into the topic yielded almost-answers such as stereo cameras (which have difficulty understanding the overlap of fingers, e.g.) and capacitive touch sensing (which struggles to interpret motion in a 3D context).Google ATAP’s answer is radar. Radar is capable of interpreting objects’ position and motion even through other objects, making it perfect for developing a sensor that can be embedded in different kinds of devices like smartphones. The difficulty was that radar hardware is too large for wearable applications. Way too large. Even the scaled-down early prototypes ATAP developed were about the size of a briefcase.

Fig. 3.1 Initial Prototype

However, after several iterations, the current model is only 8mm x 10mm: smaller than a dime. And that’s including the antenna array.

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [8 ]

PROJECT SOLI

Fig. 3.2 Soli Chip

By the by, this radar tech went from the size of a briefcase to the size of a dime in a span of ten months.The Soli sensor is a fully integrated, low-power radar operating at 60-GHz. The Soli embedded system, developed in partnership with Infineon, is also a large stride forward. For comparison, evaluating normal radar information often requires the use of a supercomputer. [5]

3.1. HOW DOES IT WORK? Soli sensor technology works by emitting electromagnetic waves in a broad beam. Objects within the beam scatter this energy, reflecting some portion back towards the radar antenna. Properties of the reflected signal, such as energy, time delay, and frequency shift capture rich information about the object’s characteristics and dynamics, including size, shape, orientation, material, distance, and velocity.

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [9 ]

PROJECT SOLI

Fig. 3.3(a) Working of Radar

Fig. 3.3(b) Working of Radar

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [1 0]

PROJECT SOLI

Fig. 3.3(c) Working of Radar

Soli tracks and recognizes dynamic gestures expressed by fine motions of the fingers and hand. In order to accomplish this with a single chip sensor, a novel radar sensing paradigm with tailored hardware, software, and algorithms is developed. Unlike traditional radar sensors, Soli does not require large bandwidth and high spatial resolution; in fact, Soli’s spatial resolution is coarser than the scale of most fine finger gestures. Instead, our fundamental sensing principles rely on motion resolution by extracting subtle changes in the received signal over time. By processing these temporal signal variations, Soli can distinguish complex finger movements and deforming hand shapes within its field.

3.3. VIRTUAL TOOL GESTURE The concept of Virtual Tools is a key to Soli interactions. Virtual Tools are gestures that mimic familiar interactions with physical tools. This metaphor makes it easier to communicate, learn, and remember Soli interactions.

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [1 1]

PROJECT SOLI



Imagine an invisible button between your thumb and index fingers – you can press it by tapping your fingers together.

Fig. 3.4(a)Pinching motion can indicate press of a button

Fig. 3.4(b) Pinching motion can indicate press of a button



A Virtual Dial that you turn by rubbing thumb against index finger. Imagine grabbing and pulling a Virtual Slider in thin air. Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [1 2]

PROJECT SOLI

Fig. 3.5(a) Rubbing a finger along the length of a thumb

Fig. 3.5(b) Rubbing a finger along the length of a thumb



Imagine grabbing and pulling a Virtual Slider in thin air.

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [1 3]

PROJECT SOLI

Fig. 3.6(a) Pulling your thumb across your pointer knuckle

Fig. 3.6(b) Pulling your thumb across your pointer knuckle

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [1 4]

PROJECT SOLI



Feedback is generated by the haptic sensation of fingers touching each other. Without the constraints of physical controls.

Fig. 3.7(a) Rubbing the tip of your finger and thumb together

Fig. 3.7(b) Rubbing the tip of your finger and thumb together

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [1 5]

PROJECT SOLI

Even though these controls are virtual, the interactions feel physical and responsive. Without the constraints of physical controls, these virtual tools can take on the fluidity and precision of our natural human hand motion.

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [1 6]

PROJECT SOLI

CHAPTER-4 ALPHA DEVELOPMENT KIT Project Soli is Google’s Post-Touch experiment and its Alpha Development Kit was released by Google in October of 2015. It first shipped a developer kit to about 60 developers last year. The team said it was encouraged by how those developers used the sensor; they built object recognition tools, musical instruments and more.

Fig.4.1 Soli Development Kit

The early kit, though, was really only usable in a controlled environment. It used too much power and while the sensor was small, it did need a fully powered desktop or laptop to run it.

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [1 7]

PROJECT SOLI

That’s obviously not useful, so the team challenged itself to run on a smart watch. To do that, the team redesigned the chip with Infineon to reduce power consumption 22 times — down from 1.2 to 0.054 W now. The footprint is now also 3x smaller.

Fig.4.2Smart watch

Radar typically takes a lot of computational power, but this new version is now 256x more efficient and can still run at 18,000 frames per second. With all of this work, the team was able to build Soli into a smartwatch. This means you can interact with the watch without touching it — and you can use basic gestures to interact with it. In a demo, the ATAP team showed how you can scroll across messages with pretty amazing precision by just holding your hand over the watch and move closer and further away. You can even use a virtual dial gesture to interact with the watch face — basically just like you would use an analog watch. Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [1 8]

PROJECT SOLI

The ATAP team noted that this means you can show more information on the screen, simply because you don’t have to account for the finger covering up the watch face. The team noted that Soli isn’t just meant for smartwatches, though. Working with Harman’s JBL brand, the team worked on building prototype speakers with a built-in Soli radar that allows you to control the speaker. The sensor can see up to a distance of 15 meters, so you could control your speaker from a distance, too.

Fig.4.3. Soli Chip embedded in Smart Watch

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [1 9]

PROJECT SOLI

To further seed the development community, ATAP is going to launch a new beta-quality dev kit for developers. It’s unclear when we’ll see the first products with the Project Soli technology will arrive in stores. But just like ATAP worked with Levi’s on bringing its Project Jacquard smart fabrics to market, it will likely do the same with Soli, as well.[7]

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [2 0]

PROJECT SOLI

Fig.4.4 Soli Beta DevKit

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [2 1]

PROJECT SOLI

4.1. RESULTS OF ALPHA DEVELOPMENT KIT Here are a couple of highlights from the results: 

In the UK, the University of St. Andrews harnessed Soli for object recognition, using it to accurately predict what's placed on the sensor, such as copper vs. steel and milk vs. water.

Fig.4.5 Object Detection

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [2 2]

PROJECT SOLI



Also in the UK, Goldsmiths University of London was used to develop a visualization interface (that looks excitingly similar to Tony Stark's gesture-based visualization in Iron Man).

Fig.4.6. Visualization Interface

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [2 3]

PROJECT SOLI



In Spain, B-Reel used Soli to create a gesture-based security system, where a custom gesture serves as a password.

Fig. 4.7. Gesture Security System

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [2 4]

PROJECT SOLI



And Japan's Semitransparent Design used Soli for predictive drawing.

Fig.4.8. Predictive Drawing

As you can see, there are many doors that can be opened when Soli is released to developers in general. According to Poupyrev, he's looking forward to seeing what the development community can do with Soli, saying "I really want them to be excited and motivated to do something cool with it." For the beta release, ATAP intends to also release an API with the representation, feature, and gesture label data they've built. [5]

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [2 5]

PROJECT SOLI

CHAPTER-5 APPLICATIONS OF PROJECT SOLI The potential applications of Project Soli are as below:

For playing games: The soli chip can be embedded in the computer devices and gaming devices and then you play games by providing input through gestures to machines.

Fig.5.1. Gaming with Embedded Soli

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [2 6]

PROJECT SOLI



Smartphone can be controlled by using soli: Smartphone can be controlled by just embedding the soli chip in your phone. No such product is launched till now, but a smartphone with embedded soli chip is expected to be released in future.

Fig.5.2. Smartphone with Embedded Soli Chip

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [2 7]

PROJECT SOLI



In smart watch: A demo of Android Wear with embedded soli chip was showcased at Google I/O conference 2016. It is expected to be released soon.

Fig.5.3. Android Wear with Embedded Soli Chip

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [2 8]

PROJECT SOLI



In a field of medical: For viewing the patients reports, X-rays the Soli Chip can be used.

Fig.5.3. For Viewing Patient Reports

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [2 9]

PROJECT SOLI

CHAPTER-6 ADVANTAGES AND DISADVANTAGES OF PROJECT SOLI ADVANTAGES  Allows to control Gadgets with gestures.  Allows free hand typing.  Good Accuracy over control.

 Need not to carry gadgets while using them.

DISADVANTAGES  It has a very small radar range.  Multiple gestures could not be possible.  Since this technology is fairly new to the industry it will be very expensive but as time goes on its price will eventually reduce.

 Most of the gestures may require training before use this may get very time consuming.

Co mp ut e r Scie nce Eng ine e ring Dep t t . , MBSC ET [3 0]

PROJECT SOLI

CONCLUSION

Project Soli is a new technology that uses radars to enable new type of touchless interactions. This technology considers the design of a human gesture recognition system based on pattern recognition of signatures from a portable smart radar sensor One of the big problems with wearable devices right now is inputs - there's no simple way to control these devices. Therefore gestures will be used by individuals to carry out certain functions with electronic machines. In this technology, a Radar sensor along with a capturing system is made into a small chip and this chip can be connected to any device like computer, Smartphone etc. The different functions in these devices like call, volume control, zoom etc. can be done using specific gesture without having to touch or use another interaction method.

Co mp ute r Scie nce Eng ine e ring De p tt . , M BSCET

[ 31 ]

PROJECT SOLI

REFERENCES 1. https://en.wikipedia.org/wiki/Google 2. https://en.wikipedia.org/wiki/Google_ATAP 3. http://whatis.techtarget.com/definition/Google-ATAP-AdvancedTechnologies-and-Products 4. http://www.radartutorial.eu/01.basics/Radar %20Principle.en.html 5. http://www.allaboutcircuits.com/news/radar-chiprevolutionizing-gesture-recognition-google-atap-project-soli/ 6. https://atap.google.com/soli/ 7. https://techcrunch.com/2016/05/20/googles-atap-is-bringingits-project-soli-radar-sensor-to-smartwatches-and-speakers/

Co mp ute r Scie nce Eng ine e ring De p tt . , M BSCET

[ 32 ]