SideFX Procedural Dependency Graph

I think it is huge! 
I try to imagine how it will affect on the VFX intustry. I have to look closer, but I think it is a step toward a procedural pipeline and task management which is basically project-asset management from a bird's eye view.


All the details are here:
Side FX PDG

(updated on 2023-06-20 the link was broken so it turned out that PDG is now under Houdini which soft of a admission of that PDG is not software agnostic solution sadly)

Why do we invest in pipeline? - Psyop & Shotgun Software & Pipeline Toolkit

Awsome!


Tony Barbieri and Don Parker; The Shotgun Pipeline at PSYOP from Autodesk Media and Entertainment on Vimeo.

Cloud VFX/CG Studio only for you

The was the major 3D stereo hype around 2010. The recent years was about VR hype based on my experience.

But there is another sign of "revolution". I tend to call it Cloud VFX Studio. There are more and more VFX/CG solutions are available online (for rent). They offer not just the softwares or tools, but also offer their pipeline and project management stuff.

Some of these cloud stuff I've encountered during the past years:

Nimble Collective
is a cloud animation platform as you can read on their site. One of the key member of the crew is Jason Schleifer who was (is) one of the greatest Maya guy ever. He recently started to use Blender because everybody who has some kind of cleverness :) recognize Blender sooner or later.

Clara IO
is an online free! 3D creation package which is developed by the Exocortex team. I don't know and see anybody who used their tool but there are bunch of tutorials on the net.

Metapipe
One of my best collegue called my attantion to Metapipe. They offer complete VFX pipeline on the cloud. Contemporary prome video: happy music, motion-graphics, what else do you need?

Elara
is a scalable VFX pipeline using the elastic power of the cloud - as they said. I watched their webinar. I looks promising, but there is no website so it hasn't lanched yet I guess.

Autodesk Vision Series 2015

I think it is time to relax and let Autodesk be our entertainer. Okay, you might think I`m biased (or paid by Autodesk which is not the case unfortunately) I spend  most of my time using Maya. Of course I`m biased. I wish I had more time to do cool stuff with other softwares like Modo, Houdini, ZBrush, Dynamo, Krita and learn more about  Blender, Substance Painter and Designer, Photoshop, AfterEffects, Photoscan, PFTrack, FinalCut, Flix, Mari, Nuke, Hiero, Katana, Arnold, Renderman, OpenColorIO, ShotGun, Tactic, fTrack, USD, OSL, MU, Python, C++, JavaScript, never ending story...

Anyway, just relax and watch these videos:

Virtual production

Rendering and the cloud

The topic is Procedural Content Creation and the software is 3DS Max. I have to admit it is a bit disappointing for me because we waiting soo long ago a system like that in Maya. Okay, there is SOuP so Autodesk bosses think we don`t need proceduralism any more because we already have it.

And last but not least: Editorial

Digital Production Pipeline

Literally the best video on youtube. I think don't have to append anything to this.

Naming Convention 4. - Numbers

We use numbers to count things. Avoid to use numbers on distinguish things.
Typical example to count versions like:
v001
v002
...
But not the best practice to distinguish different things like:
Tree_01
Tree_02
I usually use letters to distinguish variations.
Tree_A
Tree_B
...
Tree_Z
Using one letter can refer to 26 variations. We can extend it with double letters like:
Tree_AA
Tree_AB
...
We can combine variation with counter like this:
Tree_A_01
Tree_A_02
Tree_B_01
Tree_A_02
...

Anyway, this topic is about using numbers. We are talking about CG and VFX (as TidyVFX is about that) but this knowledge is important for the whole universe of course :)

To establish a naming convention means we have to specify for what to use numbers.

1. Count things
As I wrote above

2. Arrange things
For files and folders usually have to use numbers to specify a certain order because file systems use alphabetical order by default. For eg. if we have IN, OUT, PROD it would look like this in the file system:

But if you want a certain order you have to use numbers like this:

2.1 Padding series
Because of the file systems another thing to deal with is the padding. It is quite simple.
One character can produce ordered sequence from 1 to 9, two character from 1 to 99, etc.
For image sequences in VFX we usually use 4 digits and that means correct order 1 to 9999.
Also have to mention never start a sequence from 1 because it is a big issue when we have to extent it with 80 frame at the beginning for some reason. It would start from -80 and we really don't want that.

2.2 Padding shots (insert shots)
There is another function of padding in shot numbers for eg. We have to consider there could be new/extra shots during the production. So if we use 001, 002, 003, we can't insert a shot between 001 and 002. We have to extend the name with zeros. Same rule. If we use one character that means we can insert 9 extra shot.
So the shot numbers should be like this:
0010
0020
0030
The rule for inserting extra shots is to halving the numbers. To insert shot between 0010 and 0020 would be 0015. To insert a shot between 0010 and 0015 would be 0013 (or 0012).
I would not recommend to use 001A, 001B for shot names.

Naming Convention 3. - Files and Folders 1.

Let's pretend we have a project. A good starting point is to count how many things we have to deal with during a project. Obviously it could be tons of files and data. There are a lot of ways to deal with them but eternal question is "what is what?". So we have something and we have to give a name for that. Considering this we have to define what this something is. What is the reason for this something? For what we are going to use it? Etc.
To answer these questions we have to have a look at our project. That something can be an image file of a concept art or a 3d model or it can be a feedback that our client sent us.
Of course every project is different so we can only talk about it in general. This blog is about VFX-CG so I try to collect ideas about that kind of projects.

First of all we know there are inputs and outputs.
Inputs can be any kind of material which is not created during the project in-house.
Outputs can be any kind of material which is sent to the client, co-workers, off-site studios, etc.

If we are talking about folder structure it starts like this:

IN
OUT

One step back. There should be a solution to identify projects (or shows).
Date is a simple organizer with a project ID.

Definition:
[yyyy][counter][separator][ProjectID]
[yyyy] - year like 2014
[counter] - two digit like 01
[separator] - underscore
[ProjectID] - abbreviated name of the project - exactly 3 capital letters like OBP

For eg.:
201401_OBP (Obviously Basic Project)

So we have:

201401_OBP/
   IN/
   OUT/

Of course we shoud have something called production. We can extend that concept with pre-production and/or post-production.

That is the (first) point where we have to consider importance or priority because we have options to organize the folder hierarchy and there is no ultimate solution as I wrote before. One really important thing is to try to avoid redundancy. For example we can arrange folders like this:

201401_OBP/
    IN/
    PRODUCTION/
        PREPROD/
        PROD/
        POSTPROD/
    OUT/

As you can see PRODUCTION or PROD basically the same but there are two folders with almost the same name. So we can avoid that like this:

201401_OBP/
    IN/
    PREPROD/
    POSTPROD/
    PROD/
    OUT/

Usually this is also called flat vs deep directory stucture.

A typical file system arranges folders like above therefore we should use numbers to add a precise order of the folders like this:

201401_OBP/
    010_IN/
    020_PREPROD/
    030_PROD/
    040_POSTPROD/
    050_OUT/

About using numbers you can read this: Naming Convention 4. - Numbers

To be continued...

Autodesk acquired Shotgun

Okay, it is not recenty news but for me it is because I do an extensive research on asset and project managment topic.
I found this a couple of month ago: Redcurrant. It is in connection with the Goosbery Project as I wrote before. As the open source community push forward the industry (not just CGI and VFX) it could be a viable idea to create an open source asset and project management solution. But as we know it is a tough-tough topic.
So what I expect from that acquisition? Is it all about the big fish eats the small one? Maybe. Because Autodesk provides lots of tools for VFX an CGI (but certainly not all of them) I think there is a need to establish a clear terminology for the industry. There is a global need of generalization I think. Example for that is OpenExr and Alembic.
What do I mean generalization?
There are lot of things which are the same for eg. in Maya, Houdini, etc. and whole the industry use these  concepts. Polygon, shader, render, etc. for example.
But lot of concepts are conceived at different vfx companies, universities and software developers, etc. which I think should be the same. Asset is a typical example for that. Is it a file? Or is it a set of file versions? Or it is a lot of small components which build a 3d character?
Another example is a status of a task. Task is a complex thing itself. About it statuses: there are ready to start, WIP, hold, pending, approved, completed, on feedback, daily, commented, reviewed, has revision, omitted, etc. In this list reviewed, has revision, commented basically the same. Certainly should be an exact definition for for statuses and certainly should be a clear picture how many status necessary for the project management.

There should be a standard definition for a lot of concept in connection with VFX-CGI.
I think a major player on the asset and project management field or an open source movement can affect the generalization of the industry. It might be Shotgun.

About Shotgun you can read here.
About the acquisition you can read here and here.

Naming Convention 1.

Simply it is the most important weather you are an artist or technician (so called TD) in the VFX industry. Furthermore it is the most important thing for mankind (or the universe). Naming convention is essential for everything.
Unique projects has unique naming convention. But let's pretend you won't do anything that nobody did before meaning of the process. So there are guidelines.
Naming convention is extremely important but there is another principle. As Richard Williams wrote in his book: "Keep it stupid simple." aka KISS.
Working with computer means you use operation systems like Windows and you create files which could be organized in directories. But you will find out sooner or later there are at least 2 big problem here.

Big problem 1.
One file can only belong to one directory.

Big problem 2.
More data we have that we can simply put into names, filenames for eg.

But these problems raise a lot of other issues. And this blog is more or less about to discuss those issues. Right now I simply use the KISS principle which also means (for me) do it as you can.

So we need a general solution of file naming. Before I tell you the truth I gather some ideas.

1. Importance of the order

Usually files are arranged by their names (alphabetical order).  So by giving names we can describe the order of the files.

2. Parts (tags)

We can of course combine different information usually called tags. So the question is how many tags should be there to collect the most important information?

3. Compressing information

We can say abbreviation but is more than that. Question is how can we compress more and more information into alphanumeric character. So for eg. we can use abbreviations but in that case we should establish a dictionary for those, because BG does not necessarily mean background for everybody.

4. Documentation

If you are a one person studio you might think documentation is not important. But it is. I think it is for every scale of facilities. So the naming convention should be documented. It can change over time from project to project and that means the naming convention document can have versions or can be attached to projects. It could be a simple text file.

4.1 Documentation template

Usual form to template naming convention is:
[tag1][separator][tag2][tag3][separator][tag4]

It can describe a file rules as well.

We can use explicit characters and variable names to be more clear.
[name]_[versionLetter][versionNumber]_[comment]

It could be necessary to explain the rule of the variables. We can call it file rules.
[name] - Starts with capital alphabetical character and contains any alphanumeric character
[versionLetter] - Any non-capital alphabetical character

This example above use the "_" (underscore) as a constant character or tag. That means you can't change it. Usually when we describe a file path there are constant directories so they are constants as well:
e:/ Projects / [ProjectID] / 2D / ConceptArt / [AssetID] / 
[AssetID]_[versionLetter][versionNumber]_[comment]

In this example above the "Projects", "2D" and the "ConceptArt" are constant tags.

More complex when we use expressions like "AssetID", because this is not a simple variable but a concept plus and abbreviation (ID = identification). We should describe things like Asset and/or AssetID before we use it. But it is not the part of the naming convention documentation rather a documentation of the pipeline or system we utilize.

5. Character table
It is always recommended not to use special characters. That means file names contain only alphanumeric character, underscore and hyphen. We can express with regular expressions: [A-Za-z0-9_-] (see below)

So this would be my general template for file naming:
[name]_[vatriationLetter]_[versionLetter][versionNumber]_[comment]

And the rule for this pattern:
[name] - Starts with capital alphabetical character and contains any alphanumeric character
[variantLetter] - Only one capital alphabetical character
[versionLetter] - Only one non-capital alphabetical character
[versionNumber] - Two digit starts with 01
[comment] - Any alphanumeric character


Below we can see a series of examples how it looks in practice. So it is more like the explanation of the concept behind this naming convention.
Let's say I'm working on a landscape concept art. So the name part would be "Landscape". I separate the process stages like sketch, details, colors. So the version letters represent these stages (a, b, c). I can also comment where the stage starts to be more clarify.
Usually when we work on something we don't consider at the beginning there might be another variant for the subject. So if we use "A" as a base variant we won't ruin the naming convention later with adding an extra tag.

Landscape_A_a01_Sketch
Landscape_A_a02
Landscape_A_a03_Background
Landscape_A_a04
Landscape_A_a05_FGTrees
Landscape_A_b01_Details
Landscape_A_b02
Landscape_A_b03_BGMountains
Landscape_A_b04
Landscape_A_b05
Landscape_A_c01_Colors
Landscape_A_c01_SkyAndClouds

You might find the "a", "b", "c" a little bit confusion to use for versions. You are not the only one. Most people use a "v" letter like  "v01", "v02", etc. But using only one constant version letter we can't embed more information with that.


Footnotes

Regular Expressions aka Regex

Regex is a tool to analyze names. Usually it used to verify and search. If you have a basic idea how regex works you might want to establish naming convention that can be easily handled with regex.
For eg. the naming convention above can be described as regex pattern like this:
[A-Z] [A-Za-z0-9]+_[A-Z]_[a-z][0-9]{2}_[A-Za-z0-9]+

So the regular expression tells to the engine the name:
Starts with one capital alphabetical character [A-Z], followed by alphanumerical character [A-Za-z0-9], it could be one or more +, followed by an underscore _ (constant letter), followed by a capital alphabetical character [A-Z], followed by an underscore, followed by a non-capital alphabetical character [a-z], followed by any number [0-9], it has to be exactly two digit {2}, followed by underscore, followed by any alphanumerical character [A-Za-z0-9] and it could be one or more +.

The real benefit to use regex comes when you or your studio use programming languages or other tools to verify or match paths, file names, or any kind of text, etc.