Procedural Creature Generator from Denis Kozlov on Vimeo.
Procedural systems I’ve been developing during the recent years served different purposes, not the least one being exploration of how far one can go in formalizing the visual art, expressing its language in machine-readable terms. “Quite far” is the answer I’ve got, and today I’d like to share my vision of the next generation of artistic tools which could empower anyone to render their imaginations with almost the ease of thought. But first, let’s take a look at how these procedural systems are made.
Procedural Aircraft Design Demo from Denis Kozlov on Vimeo.
There’s no magic involved; essentially the work is about translating an artist’s professional expertise into a form that computer understands. In a way, each digital artist is doing this daily. Big difference from classical CG content creation is that you draw not a single instance of a subject, but all its types, versions and modifications at once. This requires understanding the subject and its creation process so clearly, as to be able to write down an explicit algorithm – explain it to a computer. This approach is analytical as opposed to statistical methods of deep learning, so the system is extremely transparent – its inner workings are human-readable by design.
After clearly formulating the full scope of the task, step one is to recognize the patterns. Both the patterns in the artistic process which can become the spine of a future system, and the patterns associated with the subject: What makes up the subject structurally? Where the diversity is coming from? What has the biggest impact on the viewer? This is a very exciting part requiring tons of research. For Kozinarium, the concept of Hox genes has been extremely influential, even though it did not make it into the final toolset directly. Project Aero got much of its insights from reading up on how the actual airplanes are designed, especially the constraints imposed on production.
Based on these patterns the actual system is designed. I guess grown-ups call it ‘software architecture’. The most creative yet challenging step. This is when you decide what to keep, what to leave out, what is the skeleton and what are the details. These are quite important decisions – the essence of the final system – what can and cannot be changed afterwards. For instance both Aero and Kozinarium utilize bilateral symmetry, which in Aero can be turned off manually; Kozinarium reserves some space for adding second symmetry plane and possibly even radial one.
Step 3 is to express the resulting design with the actual CG tools, which requires some thorough knowledge of computer graphics. To create something seriously new you cannot just be looking up techniques as you go, since that would restrict you only to the paths already taken by others. Instead you have to have a good overview of possibilities beforehand, get a reliable “feeling of the material” – predict the results of operations and often envision them in precise numerical terms. Diverse professional experience comes very handy here, and there’s a reason why I’ve dared to develop my first notable procedural system only after 16 years of practice as a CG|VFX artist. For the particular techniques I’ve used please refer to image captions throughout this article and the original 2015 Aero post.
Then finally comes the implementation stage. The work definitely requiring much skill and deserving proper recognition on its own, it’s not in a primary focus for this piece and is covered in many other sources. My main choice is Houdini for 3D work and Fusion for 2D (I find both tools just absolutely fantastic). C++ is arguably the most versatile, yet quite low-level solution; Java and Processing seem to be popular within the procedural circles; Python is an industry standard in commercial CGI.
This is the process in a nutshell. It can be boiled down to a basic analysis-synthesis-implementation chain, but does require certain expertise. Erudition is important, but pattern recognition and ability to see/translate between the structural, the verbal and the visual are probably key. The rewarding part is that once the system is in place, life usually becomes notably easier. Of course each new project involves tons of research, but with experience comes the vision, seeing through patterns and approaches. And eventually pretty much anything can be expressed.
Now imagine a system where you can create any visuals or 3D objects by merely describing them. Doesn’t matter whether with words or parametric sliders, existing or totally made-up. What’s important is that you don’t need to understand the technicalities in order to create, unless you want to. This is high-level graphics creation, as opposed to directly manipulating pixels or polygons at the low level.
|Flight Immunity is the art project I’ve created with Project Aero. It includes over 50 original aircraft designs and would have hardly be possible with traditional low-level tools.|
Such ability wouldn’t turn one into an artist on its own (same as merely owning a camera doesn’t) and in fact it could even alter the meaning we prescribe to the word ‘artist’ (again, just like photography did), yet it would make visual arts much more accessible to everyone. Furthermore, creating and customizing such systems could become a form of art in its own right, and we would encounter not only generic, universal content creation tools, but also individual, one-of-a-kind systems tailored for a particular person, reflecting and complementing her. Not to say there are many other areas like architecture or engineering where these new content creation systems would have a very practical use.
I am absolutely sure that such software will inevitably appear because
a) it is better than the current tools,
b) it can be created.
CGI tools have gone a long way since their introduction merely few decades ago. At first they’ve provided little more real automation than perfect perspective projections, later with the advent of unbiased rendering came realistic tonal distributions – more realistic out of the box than if constructed by an average person and possibly even an average artist. The ways we define digital form has evolved too from tedious number-crunching to natural drawing/sculpting. All these and many other advances are implacably moving towards more and more high-level creation methods.
Still current DCC systems are inefficient in many ways. Typically they’re just designed in very technical terms with little cushioning between the gears and the user – seemingly an atavism from the times when even simple representation of visual data was challenging. Polygons, UVs, interpolation, concatenation – the very language is wrong for high-level work. These terms are neither natural for a human artist, nor they carry much semantics for the machine to make a better use of them than bluntly storing and rendering.
At the same time there are at least several approaches to building a universal high-level content creation system. The first is to build a network of specialized modules, each one tailored to its specific task. As a matter of fact, for many popular tasks like landscape generation or digital humans such modules exist for quite a while – think Vue, SpeedTree or MakeHuman; Project Aero and Kozinarium would fall into this category too. The trick is to make those modules compatible, let them talk to each other.
Invention of a true general AI could be another solution of course, but before that happens contemporary machine learning methods already show very impressive results in CGI applications and look like another viable approach to the next generation of graphics software. But if only my understanding is correct, the problem is somewhat similar – we can train the modules for specific tasks, but are missing the glue to assemble them together.
Developing such a glue – a generalized framework that could connect individual procedural modules – is the third approach that I see.
Visual art is formalizable and can be expressed algorithmically to a much higher degree than it’s commonly considered. This is the message going throughout all my procedural work. It takes a lot of knowledge, it takes a lot of effort and devotion, it does take a vision, but it is formalizable. I found that while designing procedural systems I think in altogether different terms than ‘pixels’, ‘polygons’, or ‘UVs’. Translation into this technical language of CGI happens quite late in the process, but is already being done to some extent for each project. What’s needed is to generalize it.
Basic notions like point, curve and surface can all be redefined in primary terms like corners, contrasts, rhythms and junctions. Particular CG implementations like NURBS or polygons, raster or vector should become secondary and be derived from them. Multiple parametrizations should be available in a semantic space of concepts like visual weight, isolation, accent, saturation, etc. There’s nothing mystical about it, just existing artistic knowledge base expressed in mathematical terms. From where I currently stand developing such framework looks as realistic as creating a procedural creature generator or an aircraft design toolkit had looked.
This system could become a base for a brand new high-level DCC software, a language to connect individual modules of different nature together and possibly even a guide for more context-aware machine learning systems. I’ve surely got a vision for such language and hope to get an opportunity to design/develop it.
P.S. Kozinarium has been created as a part of my upcoming art project and is not meant for sharing or distribution.
P.P.S. I’m going to attend FMX 2018 in about two weeks – please feel free to intersect and say hello!