Question about Masking

GettinWetFeet · Post by **GettinWetFeet** » Fri Jun 13, 2008 7:38 pm

Sorry if this has been covered. Been searching the forum for a month, and not seen a solution that works for me.

I'm trying to get an animated head from CrazyTalk Pro, into Anime Studio with transparency. Because I've been trying
unsuccessfully for a month, I've done my own face animation without the use of CrazyTalk because of deadline reasons.
But I really want to figure this out because it would make my work go a million times faster.

I've tried exporting my original AVI from CT using the Huffyuv v2.1 with transparency codec, then importing to AS and clicking the "avi has alpha" box, but it would make the entire image invisible.

I've also tried using the Indeo Video 5.11 codec with alpha with the same problem. Completely transparent.

I have also tried to export my original scene in sequenced TGAs and then import with the "sequenced image" script.
This provided no masking at all.

I've tried using QTP (quicktime pro) to import either of these avi's to conver to mov, but they do not recognize the format
saved with those codec's I guess, ad I can't get them to open.

Then I saw Vern's (I believe it was Vern) post about masking layers, and had this brain fart that I thought would work.

I have a group set up.
in the group I have 2 imported AVI's
The top one is full color
The second one is greyscale.

I've set the group Masking section to "hide all"
I've set the full color section to "mask this layer"
and I've set the Greyscale masking section to "add to mask"

I THOUGHT (should learn not to think) that this would mask out the background area in the color AVI.

Obviously it's not working, or I would be bothering you all.

Can anyone help me or point me in the right direction?

heyvern · Post by **heyvern** » Fri Jun 13, 2008 7:52 pm

The only masking in AS for images that works is an alpha channel. Color doesn't work. It has to have an alpha.

If you have a QT movie with "Millions+" color setting you will get an alpha channel that is supported by AS. The "+" is what gives you an alpha channel. I don't know what settings for AVI would give that alpha channel. I know that AS uses QT for it's image support. Rather than using AVI, if you can use QT with an alpha channel (millions+) it will work.

Hopefully someday, AS will have support for masking based on the actual color of the layer... for now you have to have a working alpha channel.

And don't forget to RENDER to see the results. Yes the preview does show masking but to make sure it's working properly just do a quick render.

----

If none of these alpha channel masking options work, there is another possibility. Use the layer blending modes in the layer properties (multiply, add, screen etc).

For example, if you have an animated image layer that is a black or dark shape on white and use the multiply blending mode, the dark mouth shape will appear to be "masked" over a lighter color under it. This is a quick way to "fake" a masking effect.

-vern

GettinWetFeet · Post by **GettinWetFeet** » Fri Jun 13, 2008 8:13 pm

Thanks for the quick reply.

Unfortunately, I'm not able to export to MOV. Even though I can convert to MOV using my AVS conversion program, it doesn't have the "+" option in the profile setup.

I'm going to try and rerender the original AVI with a white background and see how this works.

I'll probably be back. I'm obsessed with figuring this out.

GettinWetFeet · Post by **GettinWetFeet** » Fri Jun 13, 2008 8:52 pm

Just a quick reply...it's a no go with the color layers options.

Great tip though for other things I need to accomplish. Thanks Vern!

Does anyone know if there is any software that might help me?

The original file formats I have to choose from are:
avi, wmv, sequenced tga, and sequenced bmp.

heyvern · Post by **heyvern** » Fri Jun 13, 2008 8:59 pm

Sequenced TGA supports an alpha channel. I just imported a TGA with an alpha into AS and it worked perfectly.

I don't know if this will help you. You would still have to import all those frames as separate image layers.

If you buy QT pro ($30?) you can import the TGA sequence and export to a mov file that would have the alpha.

----

You said the layer blending mode in AS didn't work? Was it that it didn't look right or, that the AVI format didn't work?

-vern

heyvern · Post by **heyvern** » Fri Jun 13, 2008 9:04 pm

I just took a quick look at CrazyTalk Pro. It could be that the alpha is working but the animation you are creating doesn't actually have a transparent background.

Does CrazyTalk have layers? Is there something in it to define the alpha channel itself before exporting?

I'm just trying to eliminate something unrelated to Anime Studio.

-vern

heyvern · Post by **heyvern** » Fri Jun 13, 2008 9:09 pm

I think I got it now.

You need to use the TGA export. According to the feature description AVI export uses a "key color" for compositing, it doesn't have the alpha channel. However the TGA export can use the mask that is defined in CrazyTalk for the alpha channel.

You still need to figure out an easy way to import that into AS (image layers per frame or QT export).

-vern

GettinWetFeet · Post by **GettinWetFeet** » Fri Jun 13, 2008 9:21 pm

Great Vern! I have a ray of hope now. I do have the CTPro software, and have done the sequenced TGA's...

but you know what's weird, when I open a tga in my graphics software, it opens with the selection already highlighted (like they're supposed to), but yet when I imported it into AS, it's like it lost that.

But I'm going to try again. I'm going to start fresh and see what happens.

CT does not have layers...well it "kind of" does. There's a background/static layer, and the masked face layer that sits on top.

but that's how the program functions, you don't actually have any control over them.

Ok, I'm going to reread your post again with my "pay close attention" cap on and see if I can't get these tga's to import with transparency.

GettinWetFeet · Post by **GettinWetFeet** » Fri Jun 13, 2008 9:22 pm

p.s. thanks for all your help with this. I'd put off asking on the forum for some time, cuz I hate to be a pain. Now I wish i'd asked sooner.

GettinWetFeet · Post by **GettinWetFeet** » Fri Jun 13, 2008 9:38 pm

Sigh....I don't understand. Maybe it's the targa's that the program generates.
Here's one of the tga's that it generated.
http://www.espiritglen.com/images/DazzyTGATest00000.tga
hopefully that can be right clicked and saved.

Do you see anything wrong with that?

When I import them as a series, or do just one of them, I still can't grab the transparency.

heyvern · Post by **heyvern** » Sat Jun 14, 2008 10:30 am

I'm a total idiot! Please forgive me.

I zoned out. AS doesn't support TGA alpha channels. I am so sorry. I don't know what I was thinking. I had multiple copies in different formats of a test image and I must have loaded a PNG version by mistake. I am so sorry I sent you off on a wild goose chase.

I think you might be.... stuck. If you can't export in a format like PNG or QT with millions+ you are probably out of luck.

$30 for a copy of QT pro would solve your problems though. It could open the TGAs and export the millions+ alpha channel movie file in a snap.

There is one more thought on this... do yo have Photoshop? Or an image application that can "batch process" files? If so you could batch process the TGAs so it loads the alpha, creates a new layer with just the image and exports as PNG with transparency for use in AS.

I have a Photoshop "action" that I use all the time that does this. I use it to replace the background on animated sequences. I just point it at the folder of TGAs and go... watch a movie... get some coffee... wash the car.

Once again, sorry for the misinformation... I should know better.

-vern

GettinWetFeet · Post by **GettinWetFeet** » Mon Jun 16, 2008 11:11 pm

Thank you so much for your help Vern. I'm sorry I didn't get back sooner. I did just what you said. I exported to tga, and then imported as an image series into Quicktime pro and it works great!

I'm very happy to be able to do this now. A whole new set of ideas have opened up to me now.