My setup is one layer more complicated -- I'm doing a switch set for the mouths (so I can interpolate the sub layers), and each of the sub layers is a group (probably doesn't have to be -- I'm coming from After Effects so there's some logic to the program that I keep getting tripped up on) so the setup is:
Mouth Group (Mask inside bottom layer)
>>>>Mouth Switch
>>>>>>Mouth Shape Group (mask inside bottom layer)
>>>>>>>>>>Lips (stroke, no fill) (don't mask this layer)
>>>>>>>>>>Teeth and Tongue
>>>>>>>>>>Mouth Fill
>>>>>>>>>>Mask for inside of mouth (reference of Lips layer, fill and no stroke) (add to mask but keep invisible)
>>>>Head Shape Mask (add to mask but keep invisible)
So the Head Shape Mask is masking the mouth switch group, which has masked groups inside of it. Now that I'm looking at it, I think it's because the lips are set to "don't mask this layer" -- that affects the mask of the group it's in like it's supposed to, but the canvas view is carrying that setting through to the outer group masking, where the render is masking correctly (or, at least, as I'm intending).
I think there's a problem with the logic that I'm using to set this whole mouth up, it's a mess -- but that's a whole separate issue
