Photo Realism: An Eye for Detail - Research Thesis

Photorealism: An Eye for Detail

Photorealism: An Eye for Detail BSc (Hons) Animation Technology and Visual Effects Photorealism is an artistic style characterized by the highly detailed depiction of ordinary life with the impersonality of a photograph; creating an accurate representation using traditional art practices such as painting or using computer visual effects. Throughout this research project I plan to analyse and document a variety of design techniques that aid the development of computer generated images that appear photorealistic to the human eye. Evolving from the pop-art and art movements of the 1960’s and 70’s in east coast America, photorealism has expanded into a variety of different sub-genres such as hyper and super realism and with the advance of computer technology and media production, photorealism has become a focal area within visual effects.

Fig 1.1 - The Door To… - Aleks Braz (2007)

To understand the advanced techniques used to compose a photorealistic scene a basic understanding of colour, light and physical form must be developed as they are the base attributes when creating a computer generated environment. James Roberts

1

Photorealism: An Eye for Detail

Colour Colour is a characteristic of light; the difference between light and other types of radiation is defined by their wavelength, energy and frequency. Radiation is energy that travels and dissipates as it moves, whether it is x-ray or light waves, together they all form the electromagnetic spectrum.

Fig 2.1 - Electromagnetic Spectrum, NASA (300 BC)

The visible section of the spectrum consists of violet, blue, green, yellow, orange and red before encountering infrared that we interpret as heat. Humans have the ability to see objects surrounding them by light being emitted from a source, these waves are absorbed by the object and the remaining waves are reflected into the viewer’s eyes and processed by our brain to construct an image, this is a known subtractive colour. It is not practical to digitally reproduce ‘nature’s way’ of creating colours instead we use the additive colour method, a process using the primary colours of light consisting of red, green and blue and when mixed together they make white, similar to the reverse of a light hitting a prism.

Fig 2.2 - The Additive Colour System (Circa 1860)

In the 19th century, physicist Lord William Thompson Kelvin discovered the Kelvin scale of precise temperature measurement; carbon would emit different colours depending on the heat applied to it. Based on this research the modern day colour temperature scale was established which is often used in real world lighting architecture. Source Candle Flame Sunlight: Sunset or Sunrise 100-Watt Household Bulb Tungsten Lamp (500W – 1k) Fluorescent Lights Tungsten Lamp (2k – 10k) Sunlight: Early Morning / Late Afternoon Sunlight: Noon Daylight Overcast Sky Summer Sunlight Skylight

°K 1,900 2,000 2,865 3,200 3,200 – 7,500 3,275 – 3,400 4,300 5,000 5,600 6,000 – 7,000 6,500 12,000 – 20,000

Fig 2.3 Common Colour Temperature Scale (1848)

Colour temperatures applied to different lights are not the actual physical temperature of the source but the description of the light’s colour when being compared to the heated carbon. The Kelvin Colour Temperature scale is very beneficial to the development of photorealistic image as it allows digital artists to illuminate a scene with accurate real-world lighting colours and alter the effect with detailed precision. James Roberts

2

Photorealism: An Eye for Detail

Light Light is as essential to computer graphics as it is to reality. We are accustomed to light and the ambient properties that help illuminate our environment but it controls many atmospheric conditions with the ability to adjust our mood and alter how we observe world. The exact physical description of the interaction of light with an object is very complicated and physicists have spent centuries studying the topic. A lot of the real-world physics do not apply in a digital scene but they are still important to understand how light behaves. As computers do not use subtractive lighting, arrays of mathematical equations are used to substitute the properties of natural light.

Fig 3.1 - Digital Lighting Model

1. Diffuse Light represents the colour of any light that is cast after leaving an illuminated object; this property controls parameters like tint, intensity and texture of the model’s surface. 2. Specular Light refers to how “shiny” the surface is, often perceived by a white highlight. Specularity in real-world physics explains how reflective a surface is but in computer imagery specularity only characterises brightness. 3. Reflectivity in the digital scene specifically handles reflection of the surrounding environment in the object’s surface. As well as being one of the major aesthetic considerations, shadows are considered one of the most important technical aspects of lighting. Many people perceive shadows as space for objects to get lost in, but they provide cues for such attributes as location of the light source, how far away an object is positioned and an assurance that objects share the same space. There are two main categories when discussing lighting algorithms; direct illumination and global illumination. Direct Illumination is best defined as a scene composed of lights and objects. It calculates the light cast directly onto the objects from the primary light source and not the light that bounces off the floor and other surfaces in the scene. If a secondary object is present then algorithm will cast shadows, but these will be solid black shadows as there is no ambient light. Global Illumination models the direct light produced but also attempts to interpret the indirect light emitted from the objects in a scene. It processes this information from the ambient and diffuse light components, adding colour and depth to shadows by calculating the bounces of light around the scene from the different light sources but does not affect the intensity of light within the scene. James Roberts

3

Photorealism: An Eye for Detail

Physical Form Around 300 BC, Greek mathematician Euclid studied the relationships between distances and angles documenting the notion of “Euclidean Space” or “Cartesian Space”, the theory that defines the geometry occupied by an object via calculating the flatness of its mesh; this is the basis of polygon modelling and the x, y and z values in found in 3D design applications. In modern day mathematics and computer graphics, this poses some disadvantages. If all objects are constructed of straight lines then it becomes impossible to precisely represent a curve, meaning a large number of polygons are needed to calculate these shapes in an appealing aesthetic manner, this process in return increases the rendering time and reduces real-time speed within the engine.

Fig 4.1 - Polygon Modelling - “The Da Vinci Code” - The Senate VFX (May 2006)

To create a computer generated scene, extremeley powerful software is used known as a graphics engine such as 3D Studio Max, an industry standard software package used in games development, film and television. Within these engines artists and designers have the ability to construct detailed scenes and perform advanced calculations with minimal effort. One of the preferred methods of creating objects within a graphics engine is known as polygon modelling. A polygon is a n-sided shape constructed of vertices, a vertex is a independent coordinate that defines a three-dimensional point within the evironment. This is a preferred method as it is faster to implement and render than other techniques, a benefit for real-time computer graphics. Although there is a lot of mathematics involved with photorealism in computer arts, traditional art practices are still important. Rules such as, an average mature human is the equivalent height to 7 ½ heads and the distance from head to toe is equivalent to the space between fingertip to fingertip. These ratios are very important when attempting to imitate a real-world subject, as humans have an extraordinary ability to notice if something is incorrect although unaware of what specific element is absent.

Fig 4.2 - Da Vinci, The Vitruvian Man (Circa 1492)

James Roberts

4

Photorealism: An Eye for Detail

Uncanny Valley The Uncanny Valley is the hypothesis referring to the emotional response between humans and non-human entities, originally conceived by Ernst Jentsch, a German Psychologist in the early 1900’s. “Among all the psychical uncertainties that can become an original cause of the uncanny feeling, there is one in particular that is able to develop a fairly regular, powerful and very general effect: namely, doubt as to whether an apparently living being is animate and, conversely, doubt as to whether a lifeless object may not in fact be animate - and more precisely, when this doubt only makes itself felt obscurely in one's consciousness. The mood lasts until these doubts are resolved and then usually makes way for another kind of feeling.” Ernst Jentsch, On the Psychology of the Uncanny (1906) This theory was interpreted for modern day application by Japanese Roboticist Masahiro Mori in 1970. Mori states that the more human looking an object becomes; the more recognition and empathy will be portrayed by a human, up to a certain point where the response becomes negative, this dip of familiarity is the property that gives the uncanny valley its name. These premises are based on the idea that non-human entities’ movements and characteristics will be easily noticed and appear correct as they are not imitating the real-world subjects, but the more “almost human” or “photorealistic” an entity becomes the feeling of doubt arises, unless the audience does not realize that the scene is artificial.

Fig 5.1 - Familiarity vs. Appearance Graph - Uncanny Valley - Masahiro Mori (1970)

Computer generated characters that portray humans or living beings are extremely hard to create and convince an audience as realistic. Conversely environments and digital landscapes have a greater verisimilitude. Due to environments being virtually static and defining characteristics can be reproduced using computer applications, they are predominately used as backgrounds. As an audience is focused on the lead character they are less likely to spend time speculating the authenticity of the scene. The composite of real-life video footage combined with digital environments is a good technique for creating the perception of the actual existence of an artificial scene. This is beneficial for visual effects in films as it provides the ability to create a realistic scene which can be constructed and altered at anytime with minimal cost. James Roberts

5

Photorealism: An Eye for Detail

Rendering Rendering is the final process in the computer graphics pipeline. Using a dedicated graphics engine it is the catalyst that generates the final composition of all the elements within the scene.

Fig 6.1 - The Rendering Equation, Pat Hanrahan, Stanford University, USA (2000)

This extremely complicated formula is a mathematical representation of the calculations made during the rendering process, developed by Pat Hanrahan at Stanford University. Although it looks very complex it can be translated into a simple explanation. The equation calculates the energy leaving a light source in correlation with light being processed by the camera or point-of-view, this combined energy must be equal to the light being absorbed and emitted from the geometry of the object. (See fig 3.1 - Digital Lighting Model) A rendering engine is computer software that calculates detailed algorithms specifying information about light, geometry and many more advanced mathematical equations. These engines can be split into 3 categories: Scanline, Radiosity and Raytracing. Scanline or Rasterisation is the technique of generating an image from a collection of primitives pixel-by-pixel. Rasterisation is similar and uses pixel-by-pixel representation. It also uses a technique similar to back face cull, by ignoring polygons and primitives that are not in view of the camera and calculating specific objects within the environment. This reduces render and production time, making it a popular method for real-time applications like computer games and motion capture. Radiosity is a rendering technique that replicates global illumination and is often linked with the Scanline renderer to produce indirect illumination. The radiosity equation calculates the geometry and light within the scene, using this information it computes the equivalent of light bouncing around the scene producing artificial ambient light. Raytracing is an algorithm that simulates the optical properties of light as it bounces around a scene. It processes this information by tracking individual rays of lights from the original source being reflected or refracted off objects and finally entering the perspective point-of-view e.g. a camera. This technique is known as Photon Mapping, photon meaning a small particle of light. This equation was designed to re-create principles of physical lighting and achieve Fig 6.2 - Realistic Rendering with Mental Ray unsurpassed realism. It is a popular rendering solution as it is remarkably efficient at representing real-world light and also has a vast array of variables such as caustics, shaders and sub-surface scatter allowing an artist to fine-tune their design to the smallest detail. There are a variety of different companies who have published raytracing render engines, one of the most common is a product developed by Autodesk called mental ray®, a high-performance rendering engine that is used throughout the digital art industry in areas like movies, advertising, architectural design and is also incorporated into many of the top 3D design applications. James Roberts

6

Photorealism: An Eye for Detail

mental ray® As mentioned previously mental ray® is a high-performance rendering engine used to generate realistic digital imagery by calculating the individual rays of light used to illuminate the scene. There is a combination of procedures allowing designers to create detailed realistic images within mental ray®. They consist of: • • • • •

Global Illumination Caustics Shaders Sub Surface Scatter Final Gather

Global Illumination is the process of calculating indirect light within a 3-dimensional environment. (See Pg. 3 - Light) Caustics are the result of specular light being reflected or refracted off a curved surface, processing caustics via a user-defined light source and geometry. This effect is too complicated for a standard engine and the Scanline renderer does not even support this characteristic. This property is represented physically in the real-world by the intense white highlight from a magnifying glass when focused with the sun or the shimmering ripple effect seen projected in swimming pools and on the sea floor. Shaders are small computer programs that process a specific property within the scene and mental ray® is extremely flexible at managing a range of different shaders at once. These individual applications can be applied to many different objects within the scene from geometry to lights and cameras and are usually applied via the material channel. If you imagine when a standard raytraced image is rendered the photon is cast onto the objects mesh and then directed to the camera, on the initial collision the shader becomes active and defines the characteristics of the object before completing the render. Sub Surface Scatter is an advanced shader associated with the geometry section of mental ray®, this process emulates the interaction of light penetrating geometry and being scattered by materials composed of many layers throughout the object and finally emitted out at a variety of different angles. This technique is extremely useful for representing light passing through materials like skin and wax. Final Gather is the reverse method of photon mapping and raytracing; they all have the same underlying equations but when final gather is active the rays are mapped first from the 3dimensional mesh and then the angles are calculated for the light source and point-of-view. This technique is beneficial when used in conjunction with global illumination as the latter is ideal for providing depth and detail to shadows and atmospheric conditions whereas final gather renders small incremental details, smoothing shadows and eliminating any unwanted artefacts.

Fig7.1 – mental ray®, is a perfect way of producing global illumination (left) but when final gather (right) is active the scene becomes a lot clearer and balanced. (2006)

James Roberts

7

Photorealism: An Eye for Detail

Conclusion This research has looked at a number of technical theories that contribute affects which enhance the production of a photorealistic image developed with 3D graphics applications. When designing an image that appears photorealistic and achieves the resemblance of real life, two properties must be accounted for, render-time and final application. If the artwork is used in a pre-rendered application, for example a motion picture or a still render, then factors such as level of detail and poly count do not need to be restricted. Equally if the product is used in a real-time application then these aspects need to be controlled because the graphics engine must render each frame of the scene instantly. When it comes to technology and the implementation of a photorealistic image the most authentic way to replicate these characteristics is a single frame render of an environment or scene. Keeping the render as a single frame removes the problem of characters movements appearing artificial and allows detailed effects such as raytrace rendering, volumetric effects and post production to be incorporated to enhance the realism of the image. To overcome this issue the composition of real-life characters combined with digital environments provides the illusion that a scene actually exists, a process already used in many visual effects films and goes virtually unnoticed. Using a raytracing render algorithm, like Autodesk’s mental ray®, adds a significant amount of realism to a scene, accurately re-producing the physical properties of light. This provides a detailed atmospheric effect like Global Illumination and caustics that cannot be represented in real-time applications and pixel-by-pixel representation. However, if the technical theories are considered in the context of Jentsch’s and Mori’s hypothesis of the uncanny valley with respect to the psychology of human perception, where subjects only appear acceptable when not trying to emulate human’s movements and actions it may be relevant to address the vital relationship between artistic input alongside the technical developments when producing photorealistic images. Auguste Rodin known for his life work of studying the figure and its movements in space forewarned from the beginning of the previous century: “If the artist only reproduces superficial features as photography does, if he copies the lineaments of a face exactly, without reference to character, he deserves no admiration. The resemblance which he ought to obtain is that of the soul.” Auguste Rodin, Artist & Sculptor (1840 – 1917)

James Roberts

8

Photorealism: An Eye for Detail

References Fig 1.1 - The Door To… - Aleks Braz (2007) CG Society http://features.cgsociety.org/story_custom.php?story_id=4321 Fig 2.1 - Electromagnetic Spectrum - 300 BC http://science.hq.nasa.gov/kids/imagers/ems/waves3.html Fig 2.2 - The Additive Colour System (Circa 1860) 3D Graphics: A Visual Approach - R.J. Wolfe Fig 2.3 - Common Colour Temperatures - Kelvin Scale (1848) CG Lighting Techniques - Darren Brooker - Focal Press Fig 3.1 - Digital Lighting Model (September 1999) 3D Graphics: A Visual Approach - R.J. Wolfe Fig 3.2 & 3.3 - Direct & Global Illumination (April 2006) Rendering with Mental Ray - Joop Van Der Steen - Focal Press Fig 4.1 - Polygon Modelling Courtesy of Sony Pictures - The Da Vinci Code - The Senate VFX - (19th May 2006) http://www.pinewoodgroup.com/gen/default_film_production.aspx Fig 4.2 - The Vitruvian Man, Leonardo Da Vinci (Circa 1492) Fig 5.1 - Psychology of the Uncanny (1906) Ernst Jentsch - Zur Psychologie der Unheimlichen Translated By: Roy Sellars (1995) Fig 5.2 - The Uncanny Valley Thesis – Masahiro Mori (1970) Simplified version of the Familiarity vs. Appearance Graph Energy, Pg 33 - 35 Translated By: Karl F. MacDorman and Takeshi Minato http://www.androidscience.com/theuncannyvalley/proceedings2005/uncannyvalley.htm Fig 6.1 - The Rendering Equation, Pat Hanrahan, Stanford University, USA (Spring 2000) https://graphics.stanford.edu/courses/cs348b-00/lectures/lecture12/walk001.html Fig 6.2 - Realistic Rendering with mental ray®, Chen Qingfeng, INDOOR (2005) http://www.chen3d.com/environment/large.asp?id=128 Fig 7.1 - Demonstration of Global Illumination and Final Gather within mental ray®, Autodesk’s Media and Entertainment, 3D Studio Max 9 (October 2006) Help Guide, Topic: Final Gathering (mental ray® renderer)

James Roberts

9

Photorealism: An Eye for Detail

Bibliography 3D Graphics: A Visual Approach (20th September 1999) Publisher: Oxford University Press, Author: R.J. Wolfe ISBN: 0-19-511395-0 Computer Graphics and Virtual Environments: Realism to Real-Time (8th October 2001) Publisher: Addison Wesley, Author: Mel Slater, Anthony Steed, Yiorgos Chrysanthou, ISBN: 0-201- 62420-6 3DS Max Lighting (25th January 2005) Publisher: Wordware Publishing Inc, Author: Nicholas Boughen ISBN: 1-55622-401-X Atlas of Human Anatomy for the Artist (8th February 1982) Publisher: Oxford University Press, Author: Stephen Rogers Peck ISBN: 0-19-503095-8 Essential CG Lighting Techniques with 3DS Max (18th April 2006) Publisher: Focal Press, Autodesk Media & Entertainment, Author: Darren Brooker ISBN: 0-240-52022-X Digital Lighting & Rendering - Second Edition (11th May 2006) Publisher: New Riders Publications, Author: Jeremy Birn ISBN: 0-321-31631-2

James Roberts

10