Blending 3D and Photography - Chapter 1

lut_456

IntroductionIn this tutorial I will be sharing the techniques used to create an image composed of 3D and photography. I have divided the process into five stages: concepts, photography, 3D modeling, 3D layout, and rendering and compositing. These stages will overlap quite a bit as I will be doing shader tests whilst modeling and bits of compositing at the same time. Nevertheless separating the different processes will make it easier to organize the information.

ConceptsMy starting point was to think of a cybernetic addition to the hand. All kinds of images start to come into your mind when thinking about this kind of thing, such as the robotic hand of the Terminator, which was covered in bloody tissue, Luke Skywalker's burned prosthetic and Robocop's face (it's the only place where you see the flesh/metal boundary). Most of the time, the fusion of human tissue and metal looks brutal. I wanted something slightly different.

I did some research on modern prosthetics, which are getting rapidly amazing. There are mind-controlled limbs that provide tactile feedback, a bit like things you see in science-fiction. One thing that modern prosthetics have in common is the material. They are rarely made of metal and are usually made of some kind of composite polymer. Although modern prosthetics can do amazing things, they don't look that great yet.

This is where the idea started to hatch. How will these prosthetics look in a few years when the technical issues are solved and aesthetics start to become important? Carbon fiber is already in use because of its practical, mechanical qualities, but it also has quite cool aesthetic qualities. Just take a look at modern supercars.

I decided to create a hand with a couple of new, high-end carbon fiber fingers, which looked mildly futuristic, somewhat elegant and potentially possible at the same time. I wanted to emphasize the precision and dexterity of the prosthetic fingers, which is why I chose to have the hand holding a scalpel.

PhotographyWith the help of my brother I set up a very simple shoot in a dark basement (to get the almost black background I wanted). The main light was behind and above my hand, the fill was provided by an overhead fluorescent light (pretty weak), and a silver reflector on the right side (Fig.01). That was basically it for the photography setup.

Fig.01

The next step was to go through some trial and error to find a nice pose and position for the scalpel etc. Once we had played with this for a while I had some usable photos, of which I chose the best one and flipped it horizontally. Later in the process I cropped and recomposed the photo very slightly, cleaned up the scalpel (using the Healing Spot brush in photoshop) and enhanced the reflections a bit.

I used an old, manual 50mm lens attached to a Nikon DSLR, giving it approx 75mm focal length. While I could have created some HDRs for reflections and lighting, I decided to skip that stage as the light setup was quite simple and should be easy to recreate by eye. However, if I had been shooting outdoors I would have tried to capture as much environmental information as possible.

3D ModelingThis stage happened simultaneously with planning the shoot. It's pretty straightforward. I used box modeling in Wings 3D. Fig.02 shows the progress of the model. It needed some slight tweaking afterward to match the real fingers. The tweaking that was done was mainly the shortening of the middle segment.

Fig.02

At the end of the modeling phase I imported the model into 3ds max, set up a very simple shader and started to throw some lights at it. It's a great way to see if there is any additional detail needed, if the surface curvature is right and if the reflections are working nicely.

3D Layout and RenderingI used a combination of 3ds Max and V-Ray for the layout and rendering. The first important step was to match the camera and object positions. For this image that was done by eye. This was mainly because there wasn't much for camera matching software to work with. There are no three-edged corners and no planes, so this was the only option. Of course this wouldn't work if it was animated, but for a single still image, it was good enough.

I did know some of the parameters, like the camera focal length, which I typed in. I then tried to place the camera roughly in the right position in relation to the finger. I tweaked the finger orientation and rotated the segments to match the pose from the photo. To make this job easier, the finger segments had pivots aligned with the real pivot locations and were parented into a simple FK setup.

From this point onwards it was an iterative process; tweak the camera, tweak the fingers then tweak the camera again. When in this situation you will need to go through this process until the tweaks get smaller and smaller and it starts to look good. For a simple scene like this it shouldn't take that much time. Having the camera and fingers positioned, I created some simple geometry to represent the real hand and scalpel, and to provide some shadows and reflections on the CG fingers. It's all invisible to the camera, of course.

Simultaneously I was working on shaders, textures and lights. One shader worth mentioning is the carbon fiber material. I built it the way real carbon is made, with a base layer of carbon weave and strong bumps to bring out the fabric detail. There was also a shiny, reflective, clear coat on top, with another layer on the very top for the text and markings (Fig.03).

Fig.03

A happy accident happened when doing the test renders. I just used simple box mapping for the texture. As it turns out, if you resize and rotate the gizmo a bit, the texture placement actually looks very similar to real-life carbon fibre. It's one of the very, very few cases when UV seams and texture stretching are of any use (Fig.04).

Fig. 04

Fig.05

I used GI and a diffusely-convoluted HDRI map to provide additional fill lighting, and a high-resolution HDRI to generate nice reflections. The reflection map has been rotated, so the reflections show up in the right places. Color mapping was set to Exponential, so it doesn't blow out the highlights.

To see the progress, let's take a look at the sequence of WIP images I saved during the project (Fig.06). See if you can spot the changes between the images.

Fig. 06

When I was happy with the preview renderings I started working on a high resolution render. I had a bit of a setback here because in full-size a slight lack of surface detail became apparent. This was fixed in two ways. First I added another layer of dirt to most of the materials, with a fine texture of spots and scratches. I also added to or increased the strength of the bump mapping in places. The second way was another render pass, containing three various textures in the R, G and B channels, occlusion, spots and anisotropic specular, modulated by tiled scratches, all to be added in compositing. All in all these were subtle changes, but important when you look at the image when it is large and at full size.

CompositingThere were many little steps involved with compositing this image. The first one is masking. I used a layer mask to hide parts of the CG fingers that should be occluded by real elements. For a still image it's the fastest way, but this is not the case for an animation. I also used the Clone Stamp tool to remove the portions of the real fingers visible behind the CG ones.

The next crucial step was matching the brightness and contrast levels in both the photo and CG elements, and matching the colors. This was done using Curves adjustment layers. I added some contrast to the photo and boosted the shadows on the CG a bit.

Another important thing is matching the depth of field. Although subtle, blurring was present and needed to be taken into account. I rendered a ZDepth pass and fed it to the Lens Blur filter. It took a few tries to get the blur amount right, but in the end it matched. The same treatment was necessary for the vrayReflections pass, which I added on top to make the reflections "pop" more.

Then came the polishing tweaks, like adding some blurred smoke in the background, adding a bit of glare in the top-right corner and removing skin-toned finger reflections from the scalpel. I also added an orange logo to the handle (to tie the photo and CG in better, since now both had strong orange accents). I added the additional rendered dirt and specular passes to the CG fingers and painted in some shadows (mostly, the shadow of the scalpel on the CG fingers and darkening the bottom finger behind the scalpel). Finally I painted in greenish fringes/glows on the edges of the CG parts.

To tie both parts better, I took a look at the Channels palette and noticed there was quite a bit of noise in the blue channel of the photo. Needless to say, I added something similar to the blue channel of the CG part. You don't even notice it unless you look into Channels, but it's there. I also added some grain over all the channels later on.

The boundary between flesh and prosthetic is always a challenge. I didn't want anything gross or ugly, just a clean, painless joint (it was very important for me that the cyborg addition looked comfortable). Fortunately the boundary was quite small in relation to the rest of the image. Still, the best way to make it work was finding an existing feature that I could use to "anchor" the prosthetic; in this case a skin fold in the middle of the first finger segment. Then it was just a case of darkening the skin in the whole area and painting in some shadows and highlights, both on the skin and on the CG part. In the end the boundary didn't draw too much attention to itself, which is what I wanted. This method is (again) almost completely useless for animation, but very fast and efficient for still images.

A final touch was the global color correction. This is a subtle modification of Curves, with the blue curve being shaped into a very shallow S shape, which adds a hint of color to the highlights and shadows separately. I also added some vignetting to darken the corners of the image (Fig.07).

Fig.07

ZackD

有点意思的教程！

烟雨

这篇文章不错呀！感谢分享！

雅雅

感谢楼主分享新的文章

kikkik7000

谢谢分享