Storing masks in RGB channels

Storing masks in RGB channels
Base image for the examples in this article

Finally returning to posting the original manuscripts of the articles I've written for 3D World magazine in 2014. This one was first published in issue 186 under the title "The Mighty Multimatte".


In an earlier piece we've been looking at raster images as data containers which may be used for storing various supplementary information as well as the pictures themselves. One of the most straightforward usage of these capabilities is rendering masks.

A lot can be done to an image after it has been rendered, in fact contemporary compositing packages even allow us to move a good deal of classical shading into post, often offering a much more interactive workflow. But even if you prefer to polish your beauty renders inside the 3D software till they come out with no need for extra touches, there still can be an urgent feedback from the client or the last little color tweak you'd need to apply under the time pressure – and compositing becomes a life savior again.


The perfect matte

However, the success of most compositing operations depends on how many elements you can isolate procedurally (that is without tracing them manually). And, no less important, with what precision (which refers to antialiasing first of all).

What we are looking for is the antialiased matte with the value of exactly 1.0 (white) for the pixels completely occupied by the object of interest, exactly 0 (black) for the rest of the image,* and antialising identical to that of the beauty render.

*Mask values above one and below zero cause no less problems than the gray ones.

Storing masks in RGB channels

Here are the results of a color-correction through the corresponding mattes. Left-to-right: with proper antialiasing, aliased, and with an attempt to fix aliasing through blurring. Note the corrupted edges in the middle example and dark halos in the right.

The power of three

It is easy to notice, that all this data requires only one raster channel to be stored. Thus a regular RGB image can naturally preserve quality mattes for 3 objects at a time. It only takes applying Constant shaders of pure Red (1,0,0), Green (0,1,0) and Blue (0,0,1) colors to the objects of interest and a black (0,0,0) Constant shader to the rest of the scene. Render layers functionality (implemented in every contemporary 3D package I can think of) comes very handy here. You might want to turn off slower features like shadows and GI for just the masks element, although typically setting all the shaders in the scene to Constant is already enough for the render engine to optimize itself sufficiently.*

*Due to smart sampling, antialiasing of the matte element might not be exactly the same as in beauty pass, still this is normally the closest one can practically get.

Alternatively, some renderers offer a pre-designed output variable (like MultiMatte in V-Ray) to render masks in the similar way. More channels (like Alpha, ZDepth, Motion Vectors or any arbitrary extra image channels) could be used for storing more mattes in the same image file of course, but typically it is not worth the time/inconvenience to set up first and extract later, compared to simply adding more RGB outputs to isolate more objects. Compositing applications and image editors naturally provide the tools to easily use any of the RGBA channels as a mask for an operation, which is another reason to stick with those. (In Photoshop, for instance, it only takes a Ctrl-click in the Channels panel to turn one into a selection).

Storing masks in RGB channels

An example of 2 objects isolated into the Red and Blue channels with Constant shaders.

What to mask?

Unless we're isolating parts of the scene with the specific purpose in mind, the main generic principle here is what will most likely require adjustments? Those are either the parts of the scene in doubt or simply the most important ones. Thus by default foreground/hero object is worth of its own matte (a channel). Grouping objects into mask channels based upon their materials or surface qualities is useful as well, since it allows for adjusting in terms of “highlights on metals” or “the color of liquids”.

But the most demanded usually are masks to separate meaningful objects and their distinct parts, especially those of the similar color, since it is tricky to isolate them with keying.

When working on a sequence of animated shots, consistency in using the same colors for the same objects from one shot to another becomes a very useful habit. This way, the same compositing setup can be propagated to the new shots most painlessly. It is generally better to add more MultiMatte outputs to the scene but stay consistent, rather than to try fitting only the masks needed for each shot into one image every time, so that in one shot a (let's say Green) channel would isolate the character, and in another – a prop.

Storing masks in RGB channels

When masking both an object and its parts in the same matte – think in terms of channels. For instance, if we want to utilize the Green channel in our example for the parts of the main object, we might want to use yellow (1,1,0) for the shader color to avoid holes in the Red channel.

The pitfalls

The world is imperfect though, and sometimes in a crunch there is simply no time to render the proper additional elements (or AOVs – Arbitrary Output Variables). That is the time to manipulate the existing data in comping in order to compensate for the missing. Individual mattes can be added, subtracted, inverted and intersected to get the new ones. Every AOV can be useful in compositing in one way or another, and any non-uniform pass can provide some masking information to be extracted – it only takes thinking of them as the source of masking data and understanding what exactly is encoded within each particular element (which we are going to touch upon in the following few issues).

And right now let's look at some dangers hidden along the way. The biggest pitfall is corrupting the edges of the matte (due to over-processing in post or the way it was rendered). 3D applications often offer some fast way of rendering object IDs (mattes) out of the box like assigning a random color or a discrete floating point luminance value to each. Though it might be faster to set up than the proper MultiMatte-like pass, the temptation should be generally avoided. With totally random colors per object the only way to procedurally separate one mask from another is keying, which will be often limited by close colors of other objects and thus quite crude.


Storing masks in RGB channels

The illustration above shows that you can not isolate one of the mattes stored sequentially by luminance while preserving the same antialiasing over different objects.

Even when assigning “the extended matte colors” (Red, Green, Blue, Yellow, Cyan, Magenta, Black, White, Alpha) instead of the totally random ones in order to store more mattes in a single image and separate them with various channel combinations rather than color keying later, the quality of the edges still gets endangered, although the results are much better typically. *This method should not be confused with the aforementioned usage of yellow, when it was still within a “one channel – one object” paradigm.

And no need to mention that any method of rendering masks/IDs without antialiasing is a no-go.


Divide and conquer

However, if going towards really heavy post-processing, it often becomes safer to render an object and its background separately. The trick in this case is not to forget the shadows and reflections, which means utilizing matte/shadow/ghost objects and visibility for primary (camera) rays functionality rather than just hiding the parts of the scene. 

13 comments:

  1. 우리카지노는 대한민국의 바카라 업계를 장악하고 있는 카지노사이트 입니다. 우리카지노가 대한 민국에서 장악한 바카라 시장점유율이 50%가 넘고 10년 넘게 온라인 바카라 시장을 장악해왔기 때문에 대한민국에서는 우리카지노를 모르는 사람은 드뭅니다. 이런 바카라 업계의 독보적인 입지 때문에 늘 유명하거나 최고만을 찾는 사람들이 카지노사이트를 찾을때는 늘
    우리카지노를 찾습니다.바카라를 처음 시작하시는 초보자분들에게도 우리카지노에서 카지노사이트를 시작하시기 좋은 환경입니다. 우리카지노사이트에서는 신규가입시 3만쿠폰을 지급 해주기 때문입니다. 사람들이 늘 1등만을 찾는 이유는 분명 있습니다. 다른 카지노사이트와는 달리 우리카지노를 이용하실시 에이전트를 끼고 게임을 하신다면 본사 이외에 활동쿠폰 및 오링쿠폰을 별도로 제공해주고 있기 때문입니다. 이러한 이유들 때문에 카지노사이트 업계에서 바카라를 즐기신다면 다들 우리카지노를 선호 하십니다. 카지노사이트에서 바카라를 이기기 물론 어렵습니다. 하지만 우리카지노의 에이전트를 끼고 바카라를 즐기신다면 승산이 있다고 봅니다. 우리카지노 에이전트의 연락처는 홈페이지로 연락하시면 언제든지 부담없이 소통가능 합니다. 카지노사이트를 선정할때는 바카라를 다른곳보다 유리하게 즐길 수 있는 카지노를 선택해야한다고 생각합니다. 그것이 바로 우리카지노 입니다. 이상으로 우리카지노와
    바카라 카지노사이트 사이의 상관관계를 알아보았습니다 바카라사이트.

    ReplyDelete
  2. Hello, thanks for sharing this content with us, I am fully indebted to you for your help. Are you interested in traveling to Ethiopia? The Ethiopia visa application can be completed online in less time. Processing of an eVisa usually takes up to 3 days. Then you can travel to Ethiopia.

    ReplyDelete
  3. Im Immo Cashflow Booster erfahren Sie, wie Sie durch smarte Immobilienanlagestrategien passiven Cashflow generieren. Sie müssen keine Bankgeschäfte tätigen, Geld leihen oder etwas kaufen.

    Eric erklärte 6 verschiedene Strategien, die verwendet werden können, um durch Leasing und Leasing einen Cashflow zu generieren. Diese Methode ist völlig neu und eigentlich eine interne Strategie.

    Anfangs war ich dieser Strategie etwas skeptisch gegenüber, denn wenn man die vermietete Wohnung wieder untervermieten möchte, wie überzeugt man dann den Vermieter.

    Aber ich habe eine bessere Ausbildung und Eric hat klare Anweisungen, wie man den Vermieter leicht überreden kann. Der Immo Cashflow Booster bietet dir Top aktuelle Strategien, welche auch jahrelang funktionieren. Und in diesem Erfahrungsbericht, werde ich dir meine persönlichen Immo Cashflow Booster Erfahrungen mitteilen.

    ReplyDelete
  4. Just 99 Web Design offers affordable web design packages for every business, from basic one-page websites to full eCommerce website design services. affordable web design services

    ReplyDelete
  5. Murphy Bed NYC was Founded in New York in 2014, our company grew from a desire to modernize the space saving furniture industry. Focusing on quality and contemporary design, we offer sophisticated home space to help create your inspired lifestyle. Based out of New York, we understand how every square inch counts in the big cities. We are determined to deliver the best whether it’s space, style or durability. Make the most out of your
    home or office with our Murphy beds. Frustrated by compromises between quality, affordability, and convenience? Murphy Bed NYC started with a new approach to space saving furniture.
    By adopting an innovative business model, We could replace the confusing, exhausting environment of the murphy beds store with the best customer service.
    Murphy Bed NYC Offers quality transforming Wall Beds to make your home functional. Free Shipping - High Quality - Warranty, Murphy bed with desk, Murphy Bed with Couch, Wall bed with couch, Kids murphy bed, Queen Size Murphy Bed, Full Size Murphy Bed, Space saving furniture, Horizontal Murphy Bed, Vertical Murphy Bed & More.

    ReplyDelete
  6. Avantajlý ve kaliteli bir dünyaya superbetin ile giriþ yapýn. Süperbetin global ölçekte bahis ve casino hizmet veren bir þirkettir. Kullanýcýlar kaliteli bir hizmet sunmakta. Bahsedeceklerimizden ziyade sizlerde Superbetin giriþ yaparak hizmetleri görebilirsiniz. Makalemizde sizlere Superbetin güvenilir mi? Süperbetin giriþ bilgileri gibi konularda bahsedeceðiz. Ayrýca Superbetin kayýt ol sayesinde özel bonuslara ulaþýn. Sizlerde süperbetin sayesinde avantajlý bir bahis ve casino hizmeti ulaþýn.
    Süperbetin sunduðu hizmetler hakkýnda detaylý bilgiler vereceðiz. Lakin öncelikle sitenin güvenilirliðinden bahsedelim. Ýlk olarak güvenilir olmayan bir site sektörde uzun yýllar hizmet veremez. Dolandýrýcý siteler zaten en fazla 3 ay hizmet verebilmiþtir. Superbetin ise sektörde 10 yýlý aþkýn zamandýr hizmet vermekte. Lisansýný bahis ve casino sektörünün en göze lisans kurumu olan çuraçao hükümetinden almýþtýr. Superbetin oyunlarý bahis ve casino oyunlarýndan saðladýklarý kazançlarýný sorun yaþamadan çekebilmekte.
    Superbetin
    Süperbetin giriþ yaparak sizlerde avantajlý bir bahis ve casino hizmetine ulaþabilirsiniz. Türkiye’deki kullanýcýlar güncel giriþ adresine sayfamýzda her zaman ulaþabilmekte. Avantajlý bir bahis ve casino hizmetine doðru siteyi tercih ederek ulaþýn. Hesabý olmayan kullanýcýlar Süperbetin kayýt iþlemini kolay bir þekilde yapabilmekte. Hiçbir ek belge göndermeden ve ayrýca kimlik bilgisi vermeden kolay bir þekilde hesabýnýzý oluþturun. Ayrýca yeni üyelere özel bonuslar bulunmakta. Deneme bonusu ve hoþgeldin bonusu yeni üyelere özeldir.
    Süperbetin bahis bölümünde geniþ bir bahis bülteni canlý olarak iddaa oynayabilirsiniz. Ayrýca tek maç üzerinden canlý bahisler oynanabilmekte. Binlere farklý müsabakaya yüksek oranlardan canlý bahis oynayabileceðiniz bir hizmet sizleri beklemekte. Sadece spor bahisleri de deðil. E-sports bahisleri, sanal bahisleri gibi farklý bahis seçenekleri de bulunmakta.
    Superbetin sayesinde gerçek bir casino hizmetine de ulaþabilirsiniz. Gerçek casino salonlarýndan canlý kurpiyerlerin sunumlarýyla 4K görüntü kalitesi üzerinden hizmet alýn. Canlý casino oyunlarý ve binlerce farklý slot oyunlarý sizleri beklemekte.
    Süperbetin bahis ve casino oyunlarýndan saðladýðýnýz kazançlarýnýzý sorun yaþamadan çekebilirsiniz. Sonuç olarak istediðiniz finansal yöntem üzerinden para yatýrma ve çekme iþlemlerinizi kolay bir þekilde yapýn.
    Sizlerde Süperbetin giriþ yaparak avantajlý bahis ve casino dünyasýna giriþ yapabilirsiniz.
    Süperbetin kullanýcýlarý para yatýrma ve çekme iþlemlerinizi yapabilirsiniz. Sonuç olarak BTC, usdt gibi coinler üzerinden finansal iþlem sunan sitelerden birisidir.
    Geniþ bir iddaa bülteni üzerinden binlerce farklý müsabakaya ayrýca yüksek oranlardan canlý iddaa oynayýn.
    Gerçek casino salonlarýndan özel olarak bir hizmete ulaþýn. Canlý casino oyunlarý ve slot oyunlarý superbetin dünyasýnda sizleri beklemekte.

    ReplyDelete
  7. tipobet global ölçekte hizmet veren sektörün en köklü ve kaliteli
    sitelerinden birisidir. Kullanýcýlarýna geniþ bir bahis bülteni ve gerçek bir casino dünyasý sunmakta. Tipobet365 kullanýcýlarý avantajlý bir hizmete ulaþabilmekte. Sizlerde Tipobet kayýt olarak geniþ bir canlý bahis bültenine ve gerçek casino hizmetine ulaþabilirsiniz. Ayrýca yeni kayýt olanlar 50 TL yatýrým þartsýz deneme bonusu alabilmekte. Tipobet giriþ adresine her zaman sayfamýzdan ulaþabilirsiniz. Doðru siteleri tercih ederek kaliteli bir hizmete ulaþýn. Bahis ve casino sitesi hakkýnda hizmetleri hakkýnda detaylý bilgi verelim.
    Tipobet
    Öncelikle sitesinin güvenilirliði hakkýnda bilgi verelim. Makalemizin devamýnda sunduðu hizmetlerden bahsedebilirsiniz. Tipobet güvenilir mi? Sorusu oldukça fazla gelmekte. Çuraçao lisansý üzerinde hizmet veren þirket lisans veren kurum tarafýndan da sürekli olarak denetlenmekte. Ayrýca güvenilir olmayan bir site sektörde uzun yýllar hizmet veremez. Tipobet365 dünyasýndan saðladýðýnýz kazançlarý kolay bir þekilde çekebilirsiniz. Finansal araçlarýna ayrýca deðineceðiz. Lakin oyuncular kazançlarýný sorun yaþamdan çekebilmekte.
    Tipobet bahis bölümünde bahis severleri spor bahisleri, sanal bahisler, E-Sports bahisleri gibi farklý bahis oyunlarý beklemekte. Spor bahislerinden futbol baþta olmak üzere onlarca farklý spor branþý bulunmakta. Tipobet365 en geniþ canlý bahis bülteni sunan þirkettir. Oyuncular tüm müsabakalara canlý bahis oynayabilmekte. Ayrýca bahis oranlarý yüksek seçenekleri ise fazladýr. Tipobet365 giriþ yaparak sizlerde sunduðu hizmeti görebilirsiniz.
    Tipobet casino bölümünde ise gerçek casino salonlarýndan hizmet sunulmakta. Casino masa oyunlarý gerçek casino salonlarýnda 4K görüntü kaliteli üzerinden aktarýlmakta.
    Oyuncular canlý kurpiyerlerin sunumlarý ve gerçek oyuncularla beraber poker, blackjack, rulet gibi masa oyunlarýný oynayabilmekte. Gerçek paralar üzerinden gerçek bir casino hizmeti Tipobet365 dünyasýnda. Casino masa oyunlarý yanýnda Tipobet slot oyna hizmeti de vermekte. Binlerce farklý slot oyunlarý Tipobet365 dünyasýnda bulunmakta. Tüm slot saðlayýcý oyunlarýna ulaþabilirsiniz.
    Tipobet para yatýrma ve çekme konusuna gelirsek. Oyuncularýn kazançlarýný çekmekte bir sorun yaþamadýðýndan bahsettik. Ayrýca finansal iþlemleriniz için onlarca farklý finansal araç sunulmakta. Kullanýcýlar istedikleri yöntemi seçerek iþlemlerini yapabilmekte. Kripto paralar, Papara ve havale / eft yöntemleri baþta olmak üzere istediðiniz yöntem üzerinden iþlemlerinizi yapabilirsiniz. Ayrýca Tipobet365 her para yatýrma iþleminde kullanýcýlar %15 Yatýrým bonusu vermekte. Her yatýrýma özel çevrim þartsýz %15 bonus alýn.
    Tipobet bonus konusunu gelirsek. Sektördeki siteler arasýnda ne fazla bonus veren site Tipobet365’dir. Bünyesinde onlarca farklý bonus farklý bonus bulunmakta. Tipobet kayýt olan kullanýcýlar yatýrým yapmadan 50 TL deneme bonusu alabilmekte. Ayrýca ilk yatýrýmlarýna özel %100 hoþgeldin bonusu da bulunmakta. Her yatýrýmlarýna özel %15 yatýrým bonusu da alabilirsiniz. Yatýrým bonusu çevrim þartsýz olarak verilmekte. Þansýz gününüzde kayýp bonusu alarak geri dönüþ yapabilirsiniz. Tipobet365 bonus bölümünde tüm bonuslarý ve kurallarý görebilirsiniz. Ayrýca bonus iþlemleri canlý destek ekibi üzerinden yapýlmakta.
    Sizlerde Tipobet giriþ yaparak avantajlý bahis ve casino hizmetine ulaþabilirsiniz. Tipobet365 güncel giriþ adresi sayfamýzda paylaþýlmakta tipobet kayıt.

    ReplyDelete
  8. Ghostwriters are basically the professionals who write the content for you but does not put their name on the work or takes any credit for their writing. Ghost writing simply means professional who are hired to write your memories, blogs or speeches and many more such scripts for you and your brand but doesn’t tag themselves on the work or you can say they don’t take the credit for their work and you are not allowed to share your work anywhere else. Ghostwriting agency

    ReplyDelete
  9. You can increase your Player Level by incomes experience factors. Experience Points are earned every time that you just make a spin in a Slot or Table Game. They’ve carved out a distinct segment producing some true game-changers latest years|in recent times|lately}, but Evolution initially discovered their footing churning out more familiar staples of the on 코인카지노 line casino floor. Simply play and Hoff’s Extra Wheel will seem at random moments. Please enter the cellphone quantity which is registered in your Wheelz account and we'll ship a affirmation code to vary your password. If you struggle with some indicators of problematic gambling, we encourage you to seek help from an expert who can offer you more tips and instruments to beat compulsive gambling behaviour.

    ReplyDelete
  10. This blog post was highly insightful. You broke down the topic into clear and understandable points perfectly. The personal stories really brought your ideas to life and kept me engaged. I'm eager to share some information with you. Curious about the democratic republic of the Congo visa for the United Arab Emirates UAE residents must apply through the DRC Required documents include a valid passport, completed application form, passport size photo, and visa fee. Processing time varies, so apply well in advance of your travel dates.

    ReplyDelete