Switch layers in masks

[ggoblin]

Apologies if this is a stupid question..

How do I ensure the mask excludes strokes for a vector layer inside a switch layer?

Please refer to image: https://i.postimg.cc/JzQWp8Vh/moho-masks.png

The middle image shows the mask with no switch layer, the green rectangle with option "exclude strokes" masks the pink circle perfectly, leaving intact the green rectangles stroke border. I want to achieve this, but where the green rectangle is inside a switch layer.

The left image show the mask as a switch layer, the green rectangle masks the pink circle but there is no option on the switch layer masking panel to select "exclude strokes" so the stroke border is destroyed. (The switch layer is set to "add to mask", the green layer inside it has no masking tab available.)

In the right image, I placed the green rectangle mask inside a group within the switch layer, so that the green rectangle vector layer inside the group now gives me the option "exclude strokes", but unfortunately it is ignored with the same results. (switch layer set to "add to mask", group q2 inside switch layer set to "hide all", and green layer inside this group set to "add to mask" and "exclude strokes".)

Apologies if I haven't described it coherently.

Thank you for your help.

[Greenlaw]

The switch layer will treat each group as a single image, so you need to place a copy of the mask inside each group, and then enable masking for each group to mask its contents.

This is how you create complex drawings for each switch layer drawing. For example if you're using the Switch Layer for mouth drawings, each group in the Switch Layer can have an upper teeth layer, a lower teeth layer and a tongue layer all masked by the mouth layer. Since each group has the mouth shape, it can be masked by it. (You can even enable Exclude Strokes for each group.)

Advanced Tip: if you copy and paste the same mouth layer into each group, you can also use Interpolate Sublayers to morph between the grouped drawings for smoother mouth transitions. For this to work correctly, the mouth shape drawing needs to have the same points and you cannot add or remove points; otherwise, the morph effect will be ignored for that layer and it will be treated as a normal switch drawing. (You can fix this by simply replacing the 'broken' layer with a copy of one of the working ones.) Note: This is not something I bother with for most rigs, but it can be very effective for certain situations, like when characters are talking v-e-r-y s-l-o-w-l-y for example.

If you're trying to mask the Switch Layer, you need to place the Switch Layer inside a group and then place the mask under the Switch Layer. Note that this may not display correctly in the Workspace but it should render correctly.

In general, I try to avoid nesting complicated masks through multiple groups...this usually doesn't display correctly and it can behave unpredictably or be difficult to edit if you decide to change something. It's better to break down complex masks into smaller groups with their own simpler masking applied even if it means creating many duplicate mask layers. If you're concerned about having too many mask layers to edit, you can minimize the task by using reference layers for the mask copies (the 'morphing' mouth shape described above for example).

Hope this helps.

[ggoblin]

Thank you for such a swift and considerate response. It really helps.

Just to clarify if I understood correctly, while Switch Layers can be masked, they should not be used as the masking layer?

As an exercise, I was creating a generic eye rig where the eyeball group layer was being masked by an eye shape layer (masking layer). I had two eye shapes (male & female) in a switch layer so I could easily switch which mask to use. Whilst the masking worked, the problem I hit was that the eyeball was drawing over the stroke of the eye shape (the eye lashes). The option to exclude strokes from the mask doesn't seem to be available under switch layers.

So a solution would be to create a switch layer with male and female groups and then duplicate the entire eye with its masking in each of these two groups (the only thing differing between the groups being the shape of the eye).. probably better to just create two rigs?

Thank you for the excellent suggestion of using reference layers, I haven't used reference layers before, I will investigate.

Thank you.

[Greenlaw]

You can use the Switch Layer as a Mask. In some cases, it might not display correctly in the workplace but this setup should render correctly. It may depend on the specific situation and how complex the masking is. Just something to be aware of.

[synthsin75]

As an exercise, I was creating a generic eye rig where the eyeball group layer was being masked by an eye shape layer (masking layer). I had two eye shapes (male & female) in a switch layer so I could easily switch which mask to use. Whilst the masking worked, the problem I hit was that the eyeball was drawing over the stroke of the eye shape (the eye lashes). The option to exclude strokes from the mask doesn't seem to be available under switch layers.

This is how you'd generally do a masked eye switch:
https://www.mediafire.com/file/ovadj4e0 ... .moho/file

You just make a reference of the male/female switch, place it above the pupil, and remove the fills from the reference switch layers. This way you can animate the eyes from the original switch, use the same pupil for both, and the reference automatically keeps the stroke overlay synced.

[ggoblin]

Thank you Wes for your concise yet precise example, it really helped me understand how to use reference layers in this case. Much appreciated.

Thank you Greenlaw for your advice on display issues with masks.. I encountered them very quickly.. good advice to keep masks simple so they display correctly whilst you are modelling with them.. its a headache to keep on rendering.

It is interesting that excluding of strokes when masking switch layers isn't available, hence the need to display the stroke above the mask. I wonder if there is a technical reason for this?

[synthsin75]

It is interesting that excluding of strokes when masking switch layers isn't available, hence the need to display the stroke above the mask. I wonder if there is a technical reason for this?

It has to do with the order in which layers are composited together when it's rendered. Basically, the renderer can't composite an effect, like exclude stroke, after it's already composited the layer with that stroke. The info has already been lost in the process.

Might be something they can address in the future, but the method I showed is how we did it before we had exclude stroke. Except now we have the benefit of reference layers, when we use to need a layerscript.

[Greenlaw]

...I haven't used reference layers before, I will investigate.

A few things to be aware of:

Normally you want to edit only the original layer (the one with the target icon.) If you edit or animate a reference layer, you break its link with the original. Unfortunately, this is really easy to do in Moho.

Fortunately, this is really easy to fix.

If you do break the link with the original layer, you can fix it by r-clicking on the layer and choosing Sync All Channels To Original. Any edits or animation you made to this layer will be lost but you may be able to copy the keyframes from the Reference layer and paste them to the original before syncing the layers.

There are a couple of things you can do to prevent accidentally editing a Reference layer.

1. Use Synthsin75's Lock Layer script. This tool locks a selected layer so you can't make edits to it.

...or...

2. Highlight the Reference layers in a way you can't miss what they are. Yes, there's already a unique 'swoosh' icon for the Reference layer but I still don't notice it sometimes so I like to prefix the layer name with REF_ to make it more obvious. I might also color code the layer red or another bright color.

Hope this helps.

[ggoblin]

Wes, thank you for the explanation. It helps to understand how your tools work so that you can work with them, rather than against them.

[ggoblin]

Thank you Greenlaw for the tips, I will bear them in mind when I explore reference layers. The concept seems very powerful.

[synthsin75]

If you use my lock layer tool, this is the one you want: http://lostmarble.com/forum/viewtopic.p ... 36#p167236

[HOWLING_ABANDON]

This is helpful!

[ggoblin]

Thank you Wes. Having used locking layers in photoshop I'm sure this is going to be very useful in many circumstances.