Whats the best way to do lip-sync?

[Kayoda_Zender]

I know you need Papagayo's for Lip-syncing but do i have to get lots of Layers and draw different mouth posistions? what is the best way to do it?

[dueyftw]

Take all your layers and place them in a Switch Layer renaming them to match the phonemes of Papagayo.

Open Papagayo,s tutorial and read it. The tutorial explains how to use the program better than I could do here. Papagayo is simple and takes only a short time to learn how to use it.


Dale

[Kayoda_Zender]

Take all your layers and place them in a Switch Layer renaming them to match the phonemes of Papagayo.

Open Papagayo,s tutorial and read it. The tutorial explains how to use the program better than I could do here. Papagayo is simple and takes only a short time to learn how to use it.


Dale

When you say all the Layers does that mean i have to draw all the different posistions of the mouth movements? and if so whats the best way to do this?
all so what do you mean by Phonemes and if i read the turorial which i have have opend does that explean what that means?

[Rasheed]

You should, however, realize that the lipsync with Papagayo and switch layers in AS is best suited for cartoon animation. If your animation is more realistic, you should also consider advanced forms of lip syncing, e.g. with switch layer interpolation. Here is an example of that by user heyvern.

[Rasheed]

When you say all the Layers does that mean i have to draw all the different posistions of the mouth movements? and if so whats the best way to do this?
all so what do you mean by Phonemes and if i read the turorial which i have have opend does that explean what that means?

The tutorials assume you have some basic knowledge of animation.

One of the things you should already know is, that lip synchronization is --like anything in animation-- a stylization of real life. You want to tell something with your art, something that can't be told in life action.

One of these things that is unique to drawing (and animation) is called "reading an image". By succinctly drawing an image, by leaving out everything that isn't necessary, you draw attention to what is important. That is the essence of animation drawing.

This practice of expressing yourself through limitation carries over to lip synchronization. You don't want to show every possible mouth shape, only a few mouth shapes. These mouth shapes can each stand for different sounds, uttered by humans while speaking (which sounds are called "phonemes"). It not only simplifies animation (always a good thing), but it also makes it possible for the animation to be dubbed with other languages.

OTOH if you have a close-up of the lips, you would probably be more accurate (better: more like real life), or perhaps even hyper-accurate (bigger than real life).

You can find a lot of tutorials on lip syncing online, and even more in animation books.

[Kayoda_Zender]

When you say all the Layers does that mean i have to draw all the different posistions of the mouth movements? and if so whats the best way to do this?
all so what do you mean by Phonemes and if i read the turorial which i have have opend does that explean what that means?

The tutorials assume you have some basic knowledge of animation.

One of the things you should already know is, that lip synchronization is --like anything in animation-- a stylization of real life. You want to tell something with your art, something that can't be told in life action.

One of these things that is unique to drawing (and animation) is called "reading an image". By succinctly drawing an image, by leaving out everything that isn't necessary, you draw attention to what is important. That is the essence of animation drawing.

This practice of expressing yourself through limitation carries over to lip synchronization. You don't want to show every possible mouth shape, only a few mouth shapes. These mouth shapes can each stand for different sounds, uttered by humans while speaking (which sounds are called "phonemes"). It not only simplifies animation (always a good thing), but it also makes it possible for the animation to be dubbed with other languages.

OTOH if you have a close-up of the lips, you would probably be more accurate (better: more like real life), or perhaps even hyper-accurate (bigger than real life).

You can find a lot of tutorials on lip syncing online, and even more in animation books.

Thanks so Much Rasheed i'll get on it and do some reseach this is my fisrt cartoon that i'm working so thanks for the help.

[mkelley]

You should, however, realize that the lipsync with Papagayo and switch layers in AS is best suited for cartoon animation. If your animation is more realistic, you should also consider advanced forms of lip syncing, e.g. with switch layer interpolation.

Rasheed,

I'm not quite sure why you think you can't do switch layer interpolation with Papagayo.

All P does is give you a tool to read the track and then provides the data that you can use in a switch layer. But your switch layers can be interpolated using that data -- and there is even an example mouth which does that.

Vern's technique, as I understand it, uses bones rather than switch layers, but you can achieve equally smooth interpolation (I do all the time) with layers and the big P.

[heyvern]

Vern's technique, as I understand it, uses bones rather than switch layers, but you can achieve equally smooth interpolation (I do all the time) with layers and the big P.

No bones in that sample.
I did it using switch layers with interpolation to demonstrate changing shape layer order inside a switch layer for creating better phonemes more easily. In the past I would painstakingly squish and shrink shapes to "hide" them. Then I discovered that shape order changes don't effect interpolation.

Notice that the teeth and tongue go ABOVE other shapes on some of the key frames.

I only created about 3 or 4 of the phonemes for that sample but if I did them all it could be used with Papagayo.

-vern

[Rasheed]

You should, however, realize that the lipsync with Papagayo and switch layers in AS is best suited for cartoon animation. If your animation is more realistic, you should also consider advanced forms of lip syncing, e.g. with switch layer interpolation.

Rasheed,

I'm not quite sure why you think you can't do switch layer interpolation with Papagayo.

I didn't write that, but if that was the impression you got, I'm sorry.

I only wanted to convey that in some cases step animating between mouth shapes isn't very appropriate, especially if your character designs are more realistic. You might want to include switch layer interpolation, because it is so much more fluid than without interpolation. Of course, you probably will need to be more careful about your setup, because switch layer interpolation requires that the number of points in each sublayer is the same.

So, lip syncing with switch layer interpolation is more advanced than without the interpolation. That was my point, and I'm sticking with it.

Of course, that was before I realized that the original poster didn't know enough of the basics of animation and lip syncing.

[slowtiger]

Hm, it would be nice to be able to switch switch layer interpolation on and off over time ... (how I waited to be able to write this sentence!)

One must decide wether to use it in a scene or not, and this should depend on the dialogue. IMO in most cases I'd switch it off, because with fast paced dialogue and all phonemes available it makes for more lively action. If I imagine a scene where I want to have slowly interpolated mouth shapes, I also would like to have more exaggerated mouth shapes - like the examples in "Illusion of Life". But these are always key phrases, something not very common within dialogue.

Maybe sometimes it can be helpful to have two sets of mouth shapes: one normal for average dialogue, and one with exaggerated shapes for certain moments of speech. But the story and character must ask for that, otherwise it would be a waste of time.

[Rasheed]

When you want to apply the principles of believable animation, you might run into problems, because the lips don't change shape independent of the rest of the face. The opening and closing of the mouth is more or less related to the opening and closing of the lower jaw, which has effect on the shape of the lower part of the face.

You could consider including the parts of the lower face in the mouth shapes. However, in believable animation everything doesn't move in sync, because the lips have less mass than the lower jaw and cheeks. As explained in "Illusion of Life", different parts move at different rates (Follow Through and Overlapping Actions).

Another consideration is that not every phoneme might involve (as much) movement of the lower jaw. Some phonemes, I'm sure, can be made by just changing the shape of the lips. This will have effect on the position of the cheeks, but not of the lower jaw.

I'm sure, it is possible to design a set of master/slave layer scripts that uses the keys in the mouth switch layer to create keys on the fly in the slave (bone) layers, so that follow through and overlap is done automatically. I guess it would require tailoring of the layer scripts for every character, because of individual mannerisms, or perhaps use bone dynamics in the mix as well.

You would still be using a tool like Papagayo to sync the voices with the animation, but it wouldn't just be mouth shapes anymore, but a whole lot of automated follow through animation as a consequence of which phoneme is uttered, what comes before and what comes next.

One could even consider to use the switch layer not as switch layer at all, but, instead, use the phonemes to drive bone animation in a predetermined manner, and leave the sublayers completely empty.

Now, that would be advanced lip syncing.

[slowtiger]

I could imagine a set of scripts doing everything what's necessary to create a sophisticated lipsync like the one you describe - but I wouldn't want to use it.

If I want to create complex animation like Disney, I'd drew it by hand. If I work in AS, I adapt my style to what's easily achieveable within the program. Automated lipsync is just a production tool to create lengthy passages of dialogue for not-so-first-class animation. If a client wants the kind of life-like facial features like you described, I'd suggest to use 3D software tailored to that aim - and adjust the budget to that.

[Rasheed]

I guess you're right, but it can't hurt exploring this sophisticated way to do lip syncing, and perhaps find something that meets limited and full motion animation somewhere in between.

[slowtiger]

Of course it's always worth to explore things. But I have noticed just recently that I can't allow for the luxury of unproven solutions when I need to get jobs done ... *g*

[Rasheed]

I've tried some bone rigging, but it is still too complicated. I first need to simplify matters, to see how to get the thing working.

I guess after doing a convincing head turn, the next holy grail is to simplify convincing secondary animation with the least amount of fuss.