Lip Sync Tips + Best Practices

General Moho topics.

Moderators: Víctor Paredes, Belgarath, slowtiger

User avatar
Gibble
Posts: 131
Joined: Mon Aug 20, 2018 7:01 pm

Lip Sync Tips + Best Practices

Post by Gibble »

Hey Gang,
I want to crank out lots of keyframes per day and I am looking to automate lip-syncing as much as possible so I can focus on other tasks.
I have used the Auto Lip Sync feature- and added a text version of the audio to the layer so it could more accurately predict the phonemes [see attached].

Image
image upload

Overall though, this is a pretty poor solution. The phonemes it chooses are often wrong, at least 70% of the time. It seems like it'd be easier just to hand key all this but I am looking fo make at least 1000 frames/day and need every shortcut, automation I can to help get me there. Do you have any suggestions how I can improve the auto- lip sync feature?

Notes: the voicetrack is isolated. It is very clear. My mouths are set up correctly.
User avatar
J. Baker
Posts: 1085
Joined: Wed Mar 23, 2005 7:22 pm
Location: USA
Contact:

Re: Lip Sync Tips + Best Practices

Post by J. Baker »

Have you used Papagayo with Moho?
https://www.lostmarble.com/papagayo/
User avatar
slowtiger
Posts: 6237
Joined: Thu Feb 16, 2006 6:53 pm
Location: Berlin, Germany
Contact:

Re: Lip Sync Tips + Best Practices

Post by slowtiger »

Depending on your style you might be happy with this solution: viewtopic.php?p=207600&hilit=lip+sync#p207600.
AS 9.5 MacPro Quadcore 3GHz 16GB OS 10.6.8 Quicktime 7.6.6
AS 11 MacPro 12core 3GHz 32GB OS 10.11 Quicktime 10.7.3
Moho 13.5 iMac Quadcore 2,9GHz 16GB OS 10.15

Moho 14.1 Mac Mini Plus OS 13.5
User avatar
Gibble
Posts: 131
Joined: Mon Aug 20, 2018 7:01 pm

Re: Lip Sync Tips + Best Practices

Post by Gibble »

J. Baker wrote: Tue Dec 10, 2024 5:14 am Have you used Papagayo with Moho?
https://www.lostmarble.com/papagayo/
Thanks for the suggestion of Papagayo, @J. Baker - but it seems that the download link isn't working any longer. That certainly looks like a great tool though- what OS are you running it on? I have Windows 11 but it says its only compatible with Windows 7 and 8 supported, XP and Vista or Mac. But do you find that tool more accurate then the default lip sync tools built into Moho??
User avatar
Gibble
Posts: 131
Joined: Mon Aug 20, 2018 7:01 pm

Re: Lip Sync Tips + Best Practices

Post by Gibble »

slowtiger wrote: Tue Dec 10, 2024 11:46 am Depending on your style you might be happy with this solution: viewtopic.php?p=207600&hilit=lip+sync#p207600.
Thank you for the great suggestion, SlowTiger. I read through that twice now- and I think I understand your workflow and it certainly sounds like a good option. Some of your reference images have since been removed which is why it was a bit tricky to fully grasp. But your explanation was excellent.
"Snappy/Poppy" animated mouths is what I am going for- not too fluid or smooth- so would your method be suitable for that??
Last edited by Gibble on Tue Dec 10, 2024 6:39 pm, edited 1 time in total.
User avatar
GCharb
Posts: 2270
Joined: Mon Oct 10, 2005 2:31 am
Location: Montreal, Quebec, Canada
Contact:

Re: Lip Sync Tips + Best Practices

Post by GCharb »

The Papayago links are working for me, at least the V2 beta do for both Windows and Mac...

There is also Magpie Pro that might be useful, but both are pretty old, so I have no idea if they work on newer OS!
User avatar
Gibble
Posts: 131
Joined: Mon Aug 20, 2018 7:01 pm

Re: Lip Sync Tips + Best Practices

Post by Gibble »

GCharb wrote: Tue Dec 10, 2024 6:30 pm The Papayago links are working for me, at least the V2 beta do for both Windows and Mac...

There is also Magpie Pro that might be useful, but both are pretty old, so I have no idea if they work on newer OS!
Thank you for the confirmation @GCarb. Are you using the download link from here? ***UPDATE**I can download now- my antivirus was blocking the download due to it being unrecognized. And I could install it on Windows 11***
Thanks for the tip!
Last edited by Gibble on Tue Dec 10, 2024 6:58 pm, edited 2 times in total.
User avatar
MrMiracle77
Posts: 189
Joined: Mon Jun 24, 2019 2:30 am

Re: Lip Sync Tips + Best Practices

Post by MrMiracle77 »

There's a "Switch Layer Icon Script" that got a lot of attention a while back:

viewtopic.php?t=34078

What it does is it creates small preview images for your switch layer. You can then simply click on the icon to switch to the indicated shape.

The script also allows you to advance/dial back the timeline by holding the center mouse button and dragging left/right.
- Dave

(As Your GM)
User avatar
J. Baker
Posts: 1085
Joined: Wed Mar 23, 2005 7:22 pm
Location: USA
Contact:

Re: Lip Sync Tips + Best Practices

Post by J. Baker »

I never use "automatic" for anything. At least nothing I can think of at the moment. So I haven't tried that feature. I just suggested Papagayo as it may be faster than scrubbing the audio in Moho and manually editing the switch layers. If you do something enough, your method may be faster than another's method because your use to that method.

Quick tip: Some people like switching their mouth shapes on cue but I found that if the mouth shape happens right before the sound, it looks best when played back. So leave a second (little bit) of silence at the beginning of your wav file if you plan to use Papagayo. ;)

Oh, I use Windows 10.
User avatar
Gibble
Posts: 131
Joined: Mon Aug 20, 2018 7:01 pm

Re: Lip Sync Tips + Best Practices

Post by Gibble »

MrMiracle77 wrote: Tue Dec 10, 2024 6:50 pm There's a "Switch Layer Icon Script" that got a lot of attention a while back:

viewtopic.php?t=34078

What it does is it creates small preview images for your switch layer. You can then simply click on the icon to switch to the indicated shape.

The script also allows you to advance/dial back the timeline by holding the center mouse button and dragging left/right.
That looks like a great plug-in! Thx @MrMiracle77.
User avatar
Gibble
Posts: 131
Joined: Mon Aug 20, 2018 7:01 pm

Re: Lip Sync Tips + Best Practices

Post by Gibble »

J. Baker wrote: Tue Dec 10, 2024 7:09 pm I never use "automatic" for anything. At least nothing I can think of at the moment. So I haven't tried that feature. I just suggested Papagayo as it may be faster than scrubbing the audio in Moho and manually editing the switch layers. If you do something enough, your method may be faster than another's method because your use to that method.

Quick tip: Some people like switching their mouth shapes on cue but I found that if the mouth shape happens right before the sound, it looks best when played back. So leave a second (little bit) of silence at the beginning of your wav file if you plan to use Papagayo. ;)

Oh, I use Windows 10.
Thanks for the added tips, @ J.Baker! I didn't plan to use a fully automated system- but if there are some tools/methods that got me even 50% to the finish line, then I want to know about them as it would save lots of time overall.
And thanks for the tips for syncing on cue. I 100% agree with you on that .
I'll animate on cue- then once I am completed- I'll shift everything 2 frames ahead b/c it looks more accurate.
We 'see' before we 'hear'. Light[seeing] travels faster than sound :)
BTW- Papagayo seems like a very handy tool! Although I am only 30 mins into testing it, it seems like a much more intuitive approach to lip syncing than in the default Moho app.
Thx again @J.Baker!!
User avatar
Greenlaw
Posts: 10382
Joined: Mon Jun 19, 2006 5:45 pm
Location: Los Angeles
Contact:

Re: Lip Sync Tips + Best Practices

Post by Greenlaw »

Something else to look into: Moho has built-in Rhubarb support. Victor mentions it here:

https://lostmarble.com/forum/viewtopic. ... 03#p216203

I was going to use this feature for the 'artist' character in our Moho tutorial series, but I couldn't get it to work properly. From what I recall, it ignored the standard Rhubarb naming convention and instead defaulted to Moho's 'generic' volume-activated lipsync. That was a while back, though, and before 14.2 was released. If I have time this week, I'll take another look at it.

But if anyone else tries it out, let me know how it works for you. There's some Moho-related info about Rhubarb support at the Rhubarb link, but the page describes outputting a .dat file for Moho, and not using the built-in support. Victor's post explains a little more about the built-in support.

For now, I'm using the Switch Selection Window to keyframe my lip-sync. I used this method exclusively in all our tutorial videos. It only takes me a few minutes to get through each scene, and it's way more accurate than any audio-based lip-sync system I've tried. To see examples of this method, I think the Let's Draw Vector Art In Moho! Part 1 tutorial had the most lipsynced 'artist' footage to date, and I animated all of that footage in about a day (not just the lipsync.)

Another good example is Blooper, which is pretty much all-talking, and the entire short was animated in a day. (I may have spent more time writing the dialog around the original blooper audio, and planning/recording the VO session.)

To me, lip-sync is the easy part--there are a few simple rules I follow, and I repeat them until I reach the end of the timeline. The challenging part is selling the performance, which is everything else happening with the character.
Last edited by Greenlaw on Tue Dec 10, 2024 10:31 pm, edited 1 time in total.
User avatar
slowtiger
Posts: 6237
Joined: Thu Feb 16, 2006 6:53 pm
Location: Berlin, Germany
Contact:

Re: Lip Sync Tips + Best Practices

Post by slowtiger »

Thought I forgot about it, but I posted this summer's work already: viewtopic.php?t=37435
The lip sync uses a very limited approach: A E I O U M F x is all I used, where x is an open mouth at rest.
AS 9.5 MacPro Quadcore 3GHz 16GB OS 10.6.8 Quicktime 7.6.6
AS 11 MacPro 12core 3GHz 32GB OS 10.11 Quicktime 10.7.3
Moho 13.5 iMac Quadcore 2,9GHz 16GB OS 10.15

Moho 14.1 Mac Mini Plus OS 13.5
User avatar
Gibble
Posts: 131
Joined: Mon Aug 20, 2018 7:01 pm

Re: Lip Sync Tips + Best Practices

Post by Gibble »

slowtiger wrote: Tue Dec 10, 2024 9:47 pm Thought I forgot about it, but I posted this summer's work already: viewtopic.php?t=37435
The lip sync uses a very limited approach: A E I O U M F x is all I used, where x is an open mouth at rest.
Thanks so much for the suggestion and the links, @SlowTiger. That script looks like one heckuva tool! I can see why everyone is singing it's praises. I can't wait to dig into it and see if it will be a good option- which it seems like it could be perfect on first glance!
User avatar
Gibble
Posts: 131
Joined: Mon Aug 20, 2018 7:01 pm

Re: Lip Sync Tips + Best Practices

Post by Gibble »

Greenlaw wrote: Tue Dec 10, 2024 8:47 pm Something else to look into: Moho has built-in Rhubarb support. Victor mentions it here:

https://lostmarble.com/forum/viewtopic. ... 03#p216203

I was going to use this feature for the 'artist' character in our Moho tutorial series, but I couldn't get it to work properly. From what I recall, it ignored the standard Rhubarb naming convention and instead defaulted to Moho's 'generic' volume-activated lipsync. That was a while back, though, and before 14.2 was released. If I have time this week, I'll take another look at it.

But if anyone else tries it out, let me know how it works for you. There's some Moho-related info about Rhubarb support at the Rhubarb link, but the page describes outputting a .dat file for Moho, and not using the built-in support. Victor's post explains a little more about the built-in support.

For now, I'm using the Switch Selection Window to keyframe my lip-sync. I used this method exclusively in all our tutorial videos. It only takes me a few minutes to get through each scene, and it's way more accurate than any audio-based lip-sync system I've tried. To see examples of this method, I think the Let's Draw Vector Art In Moho! Part 1 tutorial had the most lipsynced 'artist' footage to date, and I animated all of that footage in about a day (not just the lipsync.)

Another good example is Blooper, which is pretty much all-talking, and the entire short was animated in a day. (I may have spent more time writing the dialog around the original blooper audio, and planning/recording the VO session.)

To me, lip-sync is the easy part--there are a few basic rules I follow, and I repeat them until I reach the end of the timeline. The challenging part is selling the performance, which is everything else happening with the character.
Thanks for all the great feedback and suggestion, DR. I love your tutorial series BTW. I have gone through much of it and I have learned a ton! I really appreciate how you start right form the very beginning- including things like layouts, preferences, etc. I immediately benefitted. Thanks for all the hard work and effort you put into that. I am certain it'll help so many people as it helped me.
Post Reply