Cannot follow Tutorial 2.6 on masking

baltner · Post by **baltner** » Sun Aug 30, 2009 8:45 pm

Greetings:

I have been reading the tutorials in the Anime Studio Debut User's Guide. Tutorial 2.6 on page 31 suggests that "everything in the group is hidden" upon setting the group mask to "Hide All" in the Masking Group layer. However, the figure shown is not what I see when I use Tutorial 2.6.moho.

Masking is an important (and confusing) part of using Anime and I want to get it right. I need to understand why I cannot reproduce what is in the user guide.

Thanks,
Bruce

madrobot · Post by **madrobot** » Tue Sep 01, 2009 10:01 pm

I'm assuming you rendered?
As it won't show up in the work window

b15fliptop · Post by **b15fliptop** » Wed Sep 02, 2009 12:56 am

I think, if I understand the situation here, that this has something to do with the way masking is handled in AS 6 versus earlier versions. In versions prior to 6, selecting "Hide All" would indeed hide everything. However, it seems that in AS 6 that the program automatically assumes that you want to add the bottom layer of your masking group to the mask, which is usually the case anyway. The strange part is that when you look at the masking setting for the bottom layer, it doesn't say that it has been added to the mask when it clearly has. So the tutorial was likely written before this new "functionality" was added, hence the incongruity between what you are expecting to see and what actually happens. Does that make sense?

baltner · Post by **baltner** » Wed Sep 02, 2009 2:26 am

Thanks for the replies, but neither makes sense.
madrobot: I tried rendering (Preview is a rendering of a single frame, right?) but all that does is make the spotlights blurry, it has no effect on the masking.

b15fliptop: The bottom layer of the group in this case is called "Wall" and when I look at its properties it says "Mask this layer," which doesn't agree with what you said ("The strange part is that when you look at the masking setting for the bottom layer, it doesn't say that it has been added to the mask when it clearly has").

I wrote to Smith Micro tech support about this (incident 090830-000186) and their explanation is that masking is not supported in Debut and that the user guide was written for the Pro version so everything in it doesn't necessarily apply to Debut. They also added other hand-waving arguments about it being too complicated to explain but they offer training videos if I want to understand it. Not much help there.

Blue · Post by **Blue** » Wed Sep 02, 2009 3:28 am

b15fliptop, yeah that totally freaks me out each time. While it potentially could be more efficient to have the software do this automatically, masking is confusing enough without AS taking any liberties on my behalf. I think I'd like it NOT to automatically mask or at least give us the option to toggle this in the preferences. I always end up adjusting the mask properties on the bottom layer regardless.

b15fliptop · Post by **b15fliptop** » Wed Sep 02, 2009 10:01 am

baltner wrote:b15fliptop: The bottom layer of the group in this case is called "Wall" and when I look at its properties it says "Mask this layer," which doesn't agree with what you said ("The strange part is that when you look at the masking setting for the bottom layer, it doesn't say that it has been added to the mask when it clearly has").

Masking a layer and adding it to a mask are two different things. Masking it means that it's visibility will be determined by another layer, which is the mask. Adding it to the mask means that it is functioning as the mask, and therefore determining the visibility of the other layers being masked.

And Blue, I agree. They could have at least let us know that things were changing. Unless it's a bug or something...

baltner · Post by **baltner** » Sat Sep 05, 2009 4:21 am

OK, I get it now. That really helps. Now I understand why I wasn't seeing what the tutorial said I should, and why adding the bottom layer is something I'd want to do (the tutorial has this as a necessary second step).

Now when I read the tutorial it makes sense. I guess I thought of "mask" as simply blocking something, but the group mask adds an additional twist: "No other layers in the same group will be allowed to draw outside this visible region (the mask)." In this case the wall is masking everything not within it, instead of the reverse.

Thanks for the help. Much better than I got from Smith Micro (useless).