Script for lip sync I'm working on

[lehtiniemi]

Oh btw, did you ever update the script with the consonant-length-thingy you mentioned somewhere earlier?

[strider2000]

Not yet, because I have not yet studied enough to know if there are other similar changes I'd like to make. The consonant length is a trivial change, but there may be other changes I'd like to make. Plus, I've been spending most of my time rigging (actually working on the smart bone actions) of some new characters. It's taken up most of my free time lately

[lehtiniemi]

Looking forward to! Also if you ever get time, please consider the idea I wrote some messages ago about some help for filling long vowels.

In a nutshell, if there's vowel A that is over X frames long so that it would simply appear to be "still", especially when using switch frames, then the script would start going through layers named A 1, A 2, A 3 etc. These would be all version of A, modified by the user to be little different than the original to create movement.

Another great add would be symbols you could use instead of perioid (.) so that instead of ending to "rest", you could end to, say, four custom slots named #1, #2, #3 and #4 or so. Or something similar. This way you could choose the ending expression when lipsyncing, this would be a huge timesaver. I constantly found myself going to mouths-timeline and switching to different ending position.

And once more - what an awesome script. Saves years of my life!!!!!!!!!! I find it working as it is for switch layers, there's almost no post-work needed. Sometimes I need to shift frames here and there and then there is the switching the ending expression. But compared to everything else there, whoa.

Oh, one quick question! Is there somewhere a table of phonetic alphabets that you used to program the script? Any tips where to look so that I use correct format when I switch to them?

[strider2000]

Here's the map. It's listed in the readme. It's kept in "phonemes" in the utility directory.

Code: Select all

msPhonemes.phonemeMap = {
    a = { "AI", "v" },
    b = { "MBP", "c" },
    c = { "etc", "c" },
    d = { "etc", "c" },
    e = { "E", "v" },
    f = { "FV", "v" },
    g = { "etc", "c" },
    h = { "etc", "c" },
    i = { "E", "v" },
    j = { "etc", "c" },
    k = { "etc", "c" },
    l = { "L", "v" },
    m = { "MBP", "v" },
    n = { "etc", "v" },
    o = { "O", "v" },
    p = { "MBP", "c" },
    q = { "WQ", "c" },
    r = { "etc", "v" },
    s = { "etc", "v" },
    t = { "etc", "c" },
    u = { "U", "v" },
    v = { "FV", "v" },
    w = { "WQ", "v" },
    x = { "etc", "v" },
    y = { "etc", "v" },
    z = { "etc", "v" },
    oo = { "U", "v" },
    oa = { "O", "v" },
    ee = { "E", "v" },
    ea = { "E", "v" },
    ch = { "etc", "c" },
    th = { "etc", "v" },
    gh = { "FV", "v" },
    ou = { "U", "v" },
    wh = { "WQ", "v" },
    igh = { "AI", "v" },
    eigh = { "AI", "v" },
    A = { "AI", "v" },
    E = { "E", "v" },
    I = { "AI", "v" },
    U = { "U", "v" },
    O = { "O", "v" },
    R = { "O", "v" },
	["-"] = { "rest","c"}
}

I've thought of having that customizable by the user. Of course, whoever can write scripts can customize it even if I don't make something in the GUI

There are some good ideas that I'll try to get to when I can (I wish I had hours ) In the mean time, maybe you realize this, but you can end on any letter you'd like and if you don't add a - it won't rest. So, you don't actually have to spell things correctly. You just spell them the way you want it to work.

What I need to add is the ability to include special mouth positions.

[lehtiniemi]

Here's the map. It's listed in the readme. It's kept in "phonemes" in the utility directory.

Code: Select all

msPhonemes.phonemeMap = {
    a = { "AI", "v" },
    b = { "MBP", "c" },
    c = { "etc", "c" },
    d = { "etc", "c" },
    e = { "E", "v" },
    f = { "FV", "v" },
    g = { "etc", "c" },
    h = { "etc", "c" },
    i = { "E", "v" },
    j = { "etc", "c" },
    k = { "etc", "c" },
    l = { "L", "v" },
    m = { "MBP", "v" },
    n = { "etc", "v" },
    o = { "O", "v" },
    p = { "MBP", "c" },
    q = { "WQ", "c" },
    r = { "etc", "v" },
    s = { "etc", "v" },
    t = { "etc", "c" },
    u = { "U", "v" },
    v = { "FV", "v" },
    w = { "WQ", "v" },
    x = { "etc", "v" },
    y = { "etc", "v" },
    z = { "etc", "v" },
    oo = { "U", "v" },
    oa = { "O", "v" },
    ee = { "E", "v" },
    ea = { "E", "v" },
    ch = { "etc", "c" },
    th = { "etc", "v" },
    gh = { "FV", "v" },
    ou = { "U", "v" },
    wh = { "WQ", "v" },
    igh = { "AI", "v" },
    eigh = { "AI", "v" },
    A = { "AI", "v" },
    E = { "E", "v" },
    I = { "AI", "v" },
    U = { "U", "v" },
    O = { "O", "v" },
    R = { "O", "v" },
	["-"] = { "rest","c"}
}

I've thought of having that customizable by the user. Of course, whoever can write scripts can customize it even if I don't make something in the GUI

There are some good ideas that I'll try to get to when I can (I wish I had hours ) In the mean time, maybe you realize this, but you can end on any letter you'd like and if you don't add a - it won't rest. So, you don't actually have to spell things correctly. You just spell them the way you want it to work.

What I need to add is the ability to include special mouth positions.

Thanks for the map!

Yea, actually I've written "hello" like "hieyllou." to get longer vowels so that's easy to do as well. Maybe the automatic function would be really hard to make work. Special mouths would be good indeed! I'll also see if I come up with some good idea and can write it myself into the script, one day, if I need it desperately enough.

[RichardU]

Awesome! Awesome! Awesome!

I just tried your script. Lip sync is now fun! I can't wait to delete Papagayo from my hard drive. If Papagayo had a physical existence, I would shoot it, blow it up, burn the remainder, and pour the ashes into a cold dark hole. Your script would be placed on a velvet pillow atop a golden column, with a spinning dicso light to forever illuminate its magnificence.

Thanks.

Richard

[lehtiniemi]

Awesome! Awesome! Awesome!

I just tried your script, lip sync is now fun! I can't wait to delete Papagayo from my hard drive. If Papagayo had a physical existence, I would shoot it, blow it up, burn the remainder, and pour the ashes into a cold dark hole. Your script would be placed on a velvet pillow atop a golden column, with a spinning dicso light to forever illuminate it's magnificence.

Thanks.

Richard

I agree, it's totally fun to use and you almost wish you get more dialogue in your animation. It's so fast to do the speech and then you already have so much done in the animation. I never got Papagayo to work on mac so I'm so relieved there's finally a lipsync solution that's fast and will also work on other languages than English. Kudos!

[strider2000]

Thanks RichardU and lehtiniemi. I'm humbled by your kind words. I'm just glad it's helpful.

lehtiniemi, I've made the changes to update the UI and added a control to the dialog allowing you to specify the number of frames for consonants.

I've also been thinking about your slow speech issue. If you use vector layers for the mouth shapes, and construct them by just moving points between the different switch layers, you can set the switch layer to interpolate between layers. I believe that will provide the best approach to what you want, ie slow speech, where the mouth moves to slowly from one to another, rather than step interpolation between each layer. I was just testing it with very slow speech and it seems to work well. Granted I did have the consonant sizing change in.

It's true that you can't interpolate between bitmap layers, but that poses it's own challenges. In general, you'd want to be able to have in betweens from any one mouth position to any other. Technically it's doable, but I think it could be much more complex set up than the vector layer approach.

I'll upload the changes next week. Unfortunately I'm away from my home computer so I can't upload the changes yet.

[strider2000]

Just uploaded the changes here
http://www.mediafire.com/download/qljnv ... pSync3.zip

It's just a couple of simple changes
* The timeline now updates the view without you having to refresh (thanks for the note lehtiniemi)
* You can now specify the length of consonants. Default is 1 frame to match the old approach

I do plan to continue making improvements, based on Richard Williams notes. But since this is not my job I only do it when burning the midnight oil , so I can't promise any time frame. I will however, post updates here.

If you have ideas, please feel free to let me know. If there are programmers that have an idea to add, I do have this in github.

Enjoy.

[RichardU]

Wonderful. You might want to add "msLipSync.png" to the list of files in the readme.txt.

Richard

[strider2000]

Ok. Thanks.

[lehtiniemi]

Great, thank you so much again!!!

[RichardU]

Strider. I've been using your script, and have two suggestions. When I drag across the image, and the Lip Sync Dialog Box shows up, it would be great if the contents of Text String were already the focus and highlighted. Then I could just start typing to enter the next string.

Also, if I cancel out of the Dialog Box, it still adds the contents of Text String to the switch layer. If I delete the contents of Text String and hit okay, it effectively cancels.

Thanks again!

[strider2000]

Hmmm, the cancel works on my system. Are you on a Mac, by any chance (I run on Windows 10). The focus would be great, but as of yet I don't know how to set the focus for the dialogs

[RichardU]

I'm Win10 as well. So cancel works for you when there's something in Text String? I just confirmed again that it does not work for me.

As for the focus, I've learned that I can hit the tab key three times and I'm right where I need to be. FWIW.