Really, one of the loudest, most ear-piercing things I ever heard. Think about it.
Sorry, I just wanted to tell that little bit o' story. But that's where the name comes from. Myna's are generally pretty good at interpreting and running back what they hear. So I got a name.
1st, it requires this bit. The Linguistic Information Sound Editor. It's a handy-dandy, freely available chunk of the Agent Developers Toolkit. And it's cool beans, I tell ya. I'm not just lazy but I'm sure I couldn't have done the work otherwise. So it's easy to lean on the Redmond folks. The program allows you to bring in a .WAV file, decipher it's "Linguistic Information" (all the phonemes in there) automatically. Well, you have to type in the text in the LISE. Then you save the file as a .LWV file (a "Linguistically" enhanced WaV). It'll still read fine as a .WAV in pretty much any editor, it's just got some good thing tacked on the end. If you wan't, yoy can usually (haven't found an exception yet) rename the file back to .WAV. And it does a bangup job of detecting all the proper phonemes without any input at all (except the typing part).
PREPPING THE AUDIO FILE
As with just about every automatic phoneme detection scheme, you first need to type in the text of what the character is saying as a guide for the detection (I've had some success entering rough phonetic spellings for non-verbal sounds. Also, if the person drops their h's, or ending g's, try spelling it like that. It can help). Then in this case, you need to get the editor to do the detection. Either use the Edit>Generate Linguistic Info command, or the icon that has a little yellow lightning bolt on it. A little progress bar pops up, and in a few seconds you should be seeing yellow bars with the words above and the phonemes below displayed in the window that shows the waveform. You can then play back the wave, and check that it's working about right by watching the mouth on the right side of the editor. If something doesn't look right, you can either move the phonemes around by click-dragging their edges, or right clicking on them to change the phoneme entirely. Anyway, after you've done those couple of things, then you should save the LWV.
It does a better job if the files aren't too long. If the files are really long, or particularly noisy or muffled, it may not be able to detect the phonemes at all. Just keep that in mind.
Here's a sample (and here)of what it does in 1 minute of work(a mouth switch imported form the Moho>Object Library> mouths.moho file, tweaked to use the scripts acceptable names. A file run through the LISE untouched (the Reagan sound file from the tutorials)
HOW TO WORK WITH MYNA
What did I do? I started thinking about this almost 3 years ago. But Moho didn't have any scripting then. Tried to do it in visual basic, and failed to get as far as I liked. Now, after many hours in front of a hot hex editor, I nailed down the structure of a file format that isn't documented anywhere.
So what the script does. Choose a switch layer that has the switch layers named properly. Run the script, and choose a properly encoded audio file you've already run through the Linguistic Sound Editor(generally .LWV file, but you can rename it back to .WAV without any problems I've found). The script will tell you if there is anything wrong with it. So, it'll check the file, see if it's stereo or mono, how many bits per sample, what the sample rate is, how mant frames per second Moho is running at, and just drop those Switch keyframes in there.
I love the sound of "totally automated".
There are a few dialogs that will pop up as you go.
1. The first is, of course, the one asking you which file you want to use.
2. Whether or not you would like a full report of every phoneme being imported sent to the Lua Console window. Not something you will often need, but helpful if something is screwy.
3. Whether you would like to Export a switch data file (which will prompt you for the file you would like to export to), just Add the keyframes to the project, or do Both. If you export a switch data file and then decide to change the Frame Rate of your project, just re-export it from the LWV file, Myna automatically compensates for the new Frame Rate.
4. If you'd like to skip any of the keyframes. Sometimes you can end up with an awful lot of keyframes which can make the mouth movement a little too busy. You can always cull them by hand, but I've tried to automate it for you a bit, you can see if it works for you. The keyframes can be set to FULL (every last one of them), TWOS (every other keyframe), or THREES (every third keyframe). It automatically detects if, because of this culling, you would now have the same keyframe twice in a row, and won't add the second one. Also, "rest" keyframes are never skipped. This isn't a very intelligent method, but improvement is for another day.
I'll eventually put all this into a single dialog, but I just didn't have the time.
SETTING UP YOUR SWITCH LAYER
It's only in the naming of the sublayers in the switch file. The naming is based on the Preston Blair phoneme set (a good description on what the mouth shapes shout be like at http://www.garycmartin.com/mouth_shapes.html), with the addition of a separate shape for "th". The Lost Marble mouths are set up with sublayers for every letter of the alphabet, but the actual drawings are mostly duplicates and really follow this convention. So what I did in this case is find the first letter that existed that would be in one of the phoneme sets and rename it, e.g. I renamed the "A" layer to "ai", the "B" layer to "mbp", "Closed" to "rest", etc. Then I deleted the layers that I hadn't renamed. You can use any set of mouths you like, as long as the layers are named correctly. The ones you need are:
ai
e
mbp
cdgjknrsyz --might just rename this "other"
fv
l
th
o
u
wq
rest -- the "closed" shape
If you use these for your sublayer names it's all a go (note - it's important that the letters be lower case and typed exactly as you see, or it won't work properly). I intend to add more flexibility as I go, so you can set things up as you please, but for now this is a common convention that should get the job done in most circumstances.
The raw phonemes that the LISE puts in there are based on the IPA (International Phonetics Association) designations. Potentially, if you really wanted to you could make up a switch layer with sublayers named after all of the IPA phonemes the editor properly recognizes (close to 90 of them) if you really needed to get every sound exactly proper to lip, teeth and tongue position. It might make sense in hyper-real 3D character work, but seems like overkill for most cartoons. I will set it up so you can do that in the future, but this is a first pass at it.
THE FILE
This is GPL. Play with it. I'm proud of this little bit o'work. I don't think there's ever been a freely available solution to automate Lipsync chores. Have at it. Anyone have a 6 or 7 year old to try it out with? If you find any bugs, let me know. Have fun. Sorry it's only Windows, I can only do what I can do. EDIT If someone with a Mac could try processing a WAV file in Windows or Linux and see if Myna can read the file on a Mac, I'd appreciate it. Might have to tweak the byte order for Macs, but that wouldn't be very hard.
***UPDATED 4/17/05 1:10am GMT***
--added ability to add keyframes starting from current frame
Download the MYNA package here.
Bandwidth limit insurance copy
The .ZIP included the sf_myna. lua file, which goes somewhere in the Moho>Scripts>Menu directory (I have it in the Sound directory). The library (sf_utilities.lua - it's the latest version, no conflict with previous versions but it needs this one) goes in the Moho>Scripts>Utilities directory. The MynaMouths.moho file has all the mouths, and they are all set up now. Set up some different sound files and try it out.
TIPS
-- 1st, this only works for uncompressed files. No MP3's (including MP3's with .WAV headers and .WAV extensions), no ADPCM. But any style of uncompressed .WAV should be fine. If the original .WAV is compressed, just make sure when you save the LWV that the Format is set to PCM.
-- some files that have been overly compressed (audio level control style, not as above) seem to give the LISE a problem. The same for files with a lot of clipping (meaning it was recorded at far too high a level). Good clean sound helps the tracking a lot, but theres some tolerance.
