I have a problem with masking. I want to do lightning in a scene. What I do is basically this:
I make a totaly black layer above all other layers for the darkness and mask it. Then I subtract all lights from it. So only the lighted spots are visible and the rest is black. To this point there is no problem. The problem are the lightsources itself. They consist of multiple layers organised in a group. That allows me to move them as a unit. To visualize that:
Put everything in a bone layer. Bind light sources and lights to the same bone. Move bones around.
AS 9.5 MacPro Quadcore 3GHz 16GB OS 10.6.8 Quicktime 7.6.6
AS 11 MacPro 12core 3GHz 32GB OS 10.11 Quicktime 10.7.3
Moho 13.5 iMac Quadcore 2,9GHz 16GB OS 10.15
Moho 14.1 Mac Mini Plus OS 13.5
Lightsource Group
+ Lamp
+ Light
+ Shadow with a hole
Rest of the scene
To generate the shadow with a hole:
Make sure autofill is off
Draw a rectangle and a circle inside the rectangle
Select everything with Ctrl+A
Create Shape
Oh yes, I forgot about this one. You don't need masking at all, just create the black layer with a hole in it. Set it to Multiply and adjust opacity to your like.
You can even create a light with soft edges on one part and hard edges at another by combining two layers with different blur settings, put them into a group layer and control blend mode and opacity there.
AS 9.5 MacPro Quadcore 3GHz 16GB OS 10.6.8 Quicktime 7.6.6
AS 11 MacPro 12core 3GHz 32GB OS 10.11 Quicktime 10.7.3
Moho 13.5 iMac Quadcore 2,9GHz 16GB OS 10.15
Moho 14.1 Mac Mini Plus OS 13.5
Well, it works for 2 lights, but possibly not more.
AS 9.5 MacPro Quadcore 3GHz 16GB OS 10.6.8 Quicktime 7.6.6
AS 11 MacPro 12core 3GHz 32GB OS 10.11 Quicktime 10.7.3
Moho 13.5 iMac Quadcore 2,9GHz 16GB OS 10.15
Moho 14.1 Mac Mini Plus OS 13.5
Binding the points of multiple holes to bones will fail wenn the holes intersect (I think, I have not tried it).
For now I used layer blending modes: overlay for the lights and multiply for the darkness. Works ok.
For the future I will use masking with light layers and bones I think. In this scene this would have meant to remake too much animation.