computer animation lecture notes

CM50245 Week 19 Computer Animation and Games II It is an extension of the previous unit.

What We Have Done Computer animation and game are closely related. Of course there are differences. Games are about real-time running. Animation - you can do offline work. For animation - everything is pre-design. For games - you need artificial intelligence to control characters in real-time.

Computer Graphics Sub-areas Related Graphics Techniques Course Plan Last semester we spoke about skeleton deformation. How deformation of the skeleton affects the deformation of the figure. This semester we are going to talk about non-skeleton deformation.

Lecture 1 Geometric Modelling One the four sub-areas of computer graphics. How to represent shapes, and how to acquire shapes from physical nature, how to analyse them.

Where is Modelling? Once we acquired digital model through 3D scanner, we can process this model using various techniques. Rendering studies how to specify a particular image from a digital figure, how to apply different materials etc. Imaging is a counter-product of modelling. It cares about how to capture 2D digital images. And how to further process 2D images through different applications. Computer visions - how to reconstruct 3D shapes from 2D figures.

Modelling

Shape Acquisition 3D scanner. Scanning geometry and constructing the shape. Hand-made shapes. Also a popular way. These two techniques are the most popular. Other ways are - medical use, CT and MRI technologies to capture geometry of human organ. Seismic exploration - reconstruction of geometry under the Earth. But the two first are the most popular.

3D Scanning is Easy Now Was very expensive before, but now is very cheap. The quality isn’t as great as laser scanner, but still OK. Microsoft has a Kinet device. You can put it on your tablet and use the app to do 3D scanning now. KinetFusion: developed by Microsoft research in Cambridge. Gives you an opportunity to create 3D objects in real time. [clip] Digital Geometry The King Kong character is scanned and reconstructed from the sculpture. First there was a sculpture and then it was scanned and processed into the animation. Medical applications use a lot of digital geometry. For new doctors its best to first work on digital models than on real ones. 3D city modelling is another scenario. City planning.

Shape Representation Parametric representation. Representation of the shape using two parameters. The sphere longitude and latitude to represent the sphere. Implicit representation. Using single function to represent the shape. Mesh representation. Mathematically is very simple. Mesh consists of set of vertexes and a set of faces. Different representations have different advantages and disadvantages. For parametric - we can easily iterate all the points on the surface. For implicit - we can easily check whether a point is inside or outside of the surface. For mesh - the storage is simple. It is easy to render. You just need to render all the faces. Mesh can easily represent shapes with different topologies.

Processing

3D Geometry Processing A lot of techniques in Photoshop are reproduced in model processing. Literature “Polygon Mesh Processing”. In this unit we will cover Reconstruction, Simplification and Editing.

Geometry Processing Pipeline 1. How to acquire the digital model. The output is raw 3D data. The quality of the data is not so good. 2. Surface reconstruction. Based on the raw data you reconstruct the mesh model of the good quality from the raw data. Mesh contains more geometric information than raw data. It has connectivity between the points. 3. Analysis of surface quality. For instance, the noise level of the reconstructed mesh. Which part is smooth, which has nods? 4. Surface smoothing for noise removal. Improving the geometry of the model. 5. Parameterisation. Once we have a mesh of the reasonable quality we can performed parameterisation. To map the 3D shapes onto a 2D plan. If you have this mapping it can help you a lot if you want to assign textures to the 3D surface. To have a particular colour or surface effects you need to map 3D to 2D to assign it. 6. Simplification for complexity reduction. You need to reduce the resolution of the input mesh. So that it can help you to efficiently render the object for computer game. 7. Remeshing for improving mesh quality. Resample the input mesh and have a better connectivity. If the quality of triangulation is not good. The area of triangles is not evenly distributed. For some applications this type of mesh isn’t good. You want to have a mesh with uniform size of triangles. Here you perform remeshing into a higher quality mesh. 8. Freeform and multiresolution modelling. If you want to stretch the model, for example. Mesh editing will be covered in week 2.

[break] Lecture 2 3D Data Acquisition Surface Reconstruction Digital Michelangelo Project

After scanning the team had to develop software to do the reconstruction to get the complete mesh model. 1GB sample point from scanning. 8M triangles. 3D scanning techniques Can be classified hierarchically according to the scheme.

Contact Scanners Contact between the scanner and the scanned surface. You need to be sure that the scanner is in touch with the surface. It is a mechanical arm. This is similar to the skeleton structure. The principle of contact scanner is based on the forward kinematics. It contains angle sensors which detect movement of angling bones. The problem - it is very slow. The user needs to place the needle on every surface. So it is not very popular. The ant example. The artist first manually draws the mesh on the model. Then the designer manually places the prone of the scanner at each vertex. Based on this information he can finally reconstruct the digital 3D shape. This process is very time consuming. He needs to predeterming the locations he wants to scan. Then he needs to move the probe at each of these locations. Only then the shape is acquired. For non-contact scanners the process is much easier.

Time-of-Flight Scanners First the scanner amids some light to the object. The light hits the object and gets reflected. The scanner also has a sensor which can receive the reflected light. It can also record the time difference between releasing and receiving the light. We know the speed of the light, divided by 2 to compute the distance. So if you amid light at different points of the object you get different shapes and depth. Laser LIDAR. The scanning range of LIDAR is much bigger than Kinet. For LIDAR it is more than 100 meters, for Kinet is 2-3 meters max. LIDAR is good to scan outdoor things. The accuracy is also better. But the underlying principle is the same.

Triangulation Principle Passive Stereo or Active Stereo

Passive Stereo Matching You need to establish correspondence between two cameras. For this you need to establish some features. Because the correspondence is established afterwards it is called passive, because it is not the scanner which does the correspondence, but you - afterwards.

We don’t need to project light or laser. Active Stereo Matching We need to project some kind of light to the object. Structured Light Scanner You need to have a structured, usually black and white pattern, light and the camera which captures the projection. The camera captures pattern distortion. The key technology is that it can identify 3D position of the configuration together with the distorted pattern. It can then reconstruct the information of the captured object. [demo] Time-Coded Light Patterns We need to project black and white pattern several times on the same object. We increase frequency. Every time it is doubled. The reason for doing this is to identify a point with a certain resolution. Lets look at the red dot. For the first pattern you can see that it lies in the black region. But also a lot of neighbouring points. For the next round it is lit by white colour. Then it gets black again, then white etc. We can see a colour sequence. If we lay the red dot in another position we get another colour sequence. Problems: this methods isn’t particularly good in working with textured surfaces. It is also quite slow because the scanner needs to compute all the patterns constantly.

Laser Scanning For each sweeping steps the laser produces a fan. The camera computes the fan. Based on the triangulation principle we know the line. If we use the laser fan to sweep the object you’ll get many points to compute. Minolta and Cyberware are examples of laser scanner. Minolta is about 5 kg heavy. So it is not easy to use on the tripod, for instance. It is very accurate but rather slow because of the sweeping approach. It needs to sweep the object from top to the bottom. It also doesn’t work well with reflective and diffusive surfaces. With reflective surfaces the light will miss the camera. With diffusive - it will be dissolved before it gets to the camera.

Triangulation Problems You need to move camera around to avoid the occlusions. One scan is not enough.

[break]

Lecture 3

3D Registration 3D Model Acquisition Pipeline 3D Scanner. We can get the point-cloud from this view point. One scan is not enough. You need to take several scans from different views. We need to change the viewpoint and scan another frame. Now we have two scans. But if we are using a single scanner these two scans are in the same coordinate system. So we need to merge them into the one system. This is called 3D registration. Alignment. It is based on strong assumption that these two scans have a lot of overlapping regions. Meaning that some regions are scanned by both scans. Next step is merging. We combine two scans into one scan. Then we can check again whether we have the complete scan. If not - we can repeat the procedure until we have a complete shape.

Aligning 3D Data Red curve is one scan and green curve is another. We can fix the red scan and perform some rotation and translation to the green one. How to do it? Because we know it there must be some overlapping regions. Suppose all these correspondencies are even, then it is easy to solve. We just need to formulate this as an optimisation problem. In practice often we don’t know the correspondences. The possible way to do this in this situation is to compute some unique features on the surface. To introduce some features to establish such correspondences. Another way to do it is based on simple heuristics. We use closest points as correspondence points. To avoid flows we need a very good initial position. If it is not good - it will not convert.

Basic ICP Algorithm Simple heuristics will give you a lot of wrong correspondences. Therefore you need to do several heuristics. The single iteration will not align the two curves very well, so we need multiple iterations.

Point Selection All points will be too complex. The simplest way to do is to perform a uniform or random sampling. But this sampling ignores the geometry of the whole line patch. The best way to do is to involve geometry analysis. Which is to distribute points according to the normal distribution. Stable distribution will give us a better performance. Closest Point Search Brute force is the simplest way. But it involves a lot of complexity. N is a number or points. If they are a lot it will be very slow. BSP tree or other partitioning is used instead.

BSP Closest Point

With the example, we partition the space into two subspaces. Then we keep on dividing sub-spaces. ABC stores the splitting plan at different levels. Suppose we are given the red point and we want to find the closest point within the other points. Within the BSP tree first we check the relative location of the red point: A. We need to substitution the coordinates of the point to the creation of the plan. Positive - on the left, negative - on the right. We ignore the irrelevant regions and keep on traversing the relevant areas. Our point is in region F so we discard other regions. Rejection of “Bad” Pairs There are several rules. You can use colour as an additional property to charge the alignment. Find rotation R and translation t to minimise This formulation if very easy to understand. Shape Matching: Translation Idea of solving: f(r,t) pf(r,t)It=0 In objective function we don’t have t anymore, we have r. Shape Matching: Rotation Shape Matching: Quaternions A very complicated way of solving. Multiview Registration What if you have more than two patches? The simplest way is the one we’ve discussed before. Suppose we have 6 scans. First we align and merge first two scans, then we add the third scan to the ones which are already merged. Then we add the forth to the merge of first three. Finally we align the last one with the previous ones. But it has a problem with the initial condition. If it is not good the alignment will be affected. It will have some errors. So for each step you introduce some error which is propagating along the sequence. When you align the last patch you find out that it cannot be aligned perfectly. The way to avoid it. The simplest way is to simultaneously transform all the patches instead of transforming one. In this case for each of the scan you have rotation and translation to solve.

Multi-Match Registration You don’t do the merging too early. It helps us to avoid the error.