Computer Graphics, Linux and Everything…

NukeProcess

by on Apr.26, 2009, under Nuke

One thing I liked about Shake and miss in Nuke was its ability to process images easily from the command line. When working on automation tools for a pipeline, this was a very useful feature and a easy one to learn. After realizing that Nuke wasn’t built from a command line software and that such a feature surely won’t come anytime soon, I decided to spend some time writing my own tool to use Nuke from the exterior. It’s not a command line API, but it’s an API that can be used by any Python script anywhere in the pipeline.

Oblique, my employer, has allowed me to release it publicly. So here it is… nukeprocess.py.
Save this inside your custom python library or inside your Python’s site-packages directory. Since this is a tool for controlling an application via programming, I assume you know where to save a python file to use it. You will have to modify some parts to make sure it points to the right Nuke installation and to your own Python library.

It creates a Python script that can be run inside Nuke. It then starts Nuke and pipe the script to the STDIN, executes the script and exits Nuke. At the time of this writing, it works with Nuke 5.1v4.

If you find any bugs or have any comments about this tool, don’t hesitate to tell me. I’ve been using it in production for a few weeks now and I still find a few bugs from time to time. I would like it to be as solid as possible since Oblique pipeline will rely on it. Also, feel free to modify it to your own needs. I would appreciate if you told me what you did. In the Oblique internal version, I added a method to process on the render farm, but since our render manager API is custom made, I felt it was unnecessary for a public release.

Its usage is simple. You first create an instance of the NukeProcess class by giving it the name of the sequence you want to work with as well as its first frame, last frame and step. You then add nodes with the createNode() method by setting all the knob values with arguments. And execute the whole thing with the executeVerbose() method. The knob names you pass as args must be the scripting name. You can find them by hovering the mouse over any knob in Nuke, the tooltip is the script name.

Usage examples

#
# To resize a sequence of images, color correct it, and save to jpeg:
#
from nukeprocess import NukeProcess
np = NukeProcess("/someplace/someImageSequence.%04d.exr", 1, 100, 1)
np.createNode("Reformat", type='to box', box_width=1920, box_height=1080,
              box_fixed=True, resize='fit', black_outside=True)
np.createNode('Grade', black=.00537, white=.83)
np.createNode("Write", file="/someplace/someImageSequence_HD.%04d.jpg",
              file_type='jpeg', _jpeg_quality=1)
np.executeVerbose()

#
# To resize a sequence, offset it to frame 1 and put gray bars:
#
from nukeprocess import NukeProcess

# When declaring a empty string, it creates a Constant node instead of a Read.
np = nukeprocess.NukeProcess("", 1, sequence.getEnd()-sequence.getStart()+1,
                             color="{from_byte(64)}") 

# Use the addCode() method when you need to do something special.
np.addCode("f = nuke.addFormat('512 288 PreviewMovie')")
np.addCode("n.knob('format').setValue(f)")

# We save the node in a variable to use it later in the connectInput() method
np.saveNodeInVariable('constant') 

# sequence.getStart() and sequence.getEnd() come from another API,
# in this case a custom file browser.
np.createNode("Read", file="/somePlace/sequence.%04d.exr",
              first=sequence.getStart(), last=sequence.getEnd(),
              frame="frame+%d" % (sequence.getStart()+1))
np.createNode('Shuffle', alpha=6)
np.createNode("Reformat", type="to box", box_width=512, box_height=288,
              resize='fit', black_outside=True, box_fixed=True)

# The connectInput() method works on the previously added node.
np.createNode("Merge")
np.connectInput('constant')
np.createNode("Write", file="/somePlace/sequence_fit.%04d.exr",
              file_type="exr", compression='ZIP')

# to debug the script, you can print it and paste it in an interactive Nuke.
print np.getScript()
np.executeVerbose()

#
# To monitor the process:
#
from nukeprocess import NukeProcess
np = NukeProcess("/someplace/someImageSequence.%04d.exr", 1, 100, 1)
np.createNode("Write", file="/someplace/someImageSequence_HD.%04d.jpg",
              file_type='jpeg', _jpeg_quality=1)
oProcess = np.execute(1, 100, 1)
sOutLine = "."
while sOutLine != "":
    sOutLine = oProcess.stdout.readline()
    # Update process progress
    # Writing /someplace/someImageSequence_HD.0001.jpg took 1.08 seconds
    if sOutLine.startswith('Writing'):
        # Extract frame number from sOutLine
        try:
            iFrame = int(sOutLine.split()[1].rsplit('.')[-2])
        except:
            pass
        else:
            print("Processing frame: %d" % iFrame)

oProcess.wait()
result = oProcess.returncode

Documentation:

NukeProcess methods:

  • __init__(InputSequence, StartFrame, EndFrame, [Step], **kwargs)
    When instancing the NukeProcess, a createNode(“Read”) is done and
    **kwargs are passed.
  • createNode(NodeName, **kwargs)
    Adds a node in the script. Knobs values can be set via
    keyword arguments.
    example:
    np = createNode(‘Write’, file=’C:/patate.####.exr’, premultiplied=True)

    Internally, createNode() returns n, and uses n.knob().setValue() to
    set the parameters. If you need to create some branches, you might want
    to use saveNodeInVariable([variable]) to keep the last added node in a safe variable
    since n will always be overwritten on the next createNode(). You can then later use
    connectInput([variable]) to connect the second input of a merge node for example.

  • addCode(Code)
    Adds a line of code to the script.
  • saveNodeInVariable(Variable)
    Saves the previously added node in a variable to be used by connectInput().
  • connectInput(Variable)
    Allows branching by connecting the secondary input of the previously added node
    to be connected to the node inside Variable.
    equivalent to: addCode(“n.connectInput(0, %s)” % sVariable)
  • execute([StartFrame], [EndFrame], [Step])
    Launches the render and returns a subprocess.Popen instance.
    stdios are piped so you can use oProcess.stdin.readline().
    Optional args:
    iStartFrame, iEndFrame, iStep
    If args are not present, start and end frames are taken from the input sequence.
  • executeVerbose([StartFrame], [EndFrame], [Step])
    Launches the render and prints the output of the process.
    Optional args:
    iStartFrame, iEndFrame, iStep
    If args are not present, start and end frames are taken from the input sequence.
  • getScript()
    Returns the script in its ready-to-use form,
    with the right indentation.
  • :, , ,
    19 comments for this entry:
    1. Nick

      Cool, thanks.

    2. Jorx

      That’s brilliant!
      I’m a student of 3D animation learning Nuke + Python…
      although I don’t have a need for it yet- it is very interesting, and very educational for me. Thank you for posting your script!

    3. xavier

      le code sous PC me donne:
      ‘C:\Program’ n’est pas reconnu en tant que commande interne’
      il semble que la variable “self._nukeExe” soit mal comprise… je lense les commades python par IDLE
      quelque chose que je n’ai pas compris?

    4. flord

      Hi Xavier. You might need to change the code to point this variable to the right location, annd to make sure it’s in quotes so space in ‘program files’ is kept intact. I haven’t tested it much on Windows and my nuke path is not the standard one.

    5. Jon

      Hi Francois, thanks for putting this up. I’m a TD at a VFX company too, PRPVFX (although i work remotely). PRP are transitioning to Nuke as the main comp app and Python sort of comes with the country so it’s useful to see this sort of thing.

    6. knish Singhai

      great help.

    7. Bastian

      Hey, In nuke 6 I get the following error:

      # Result: Traceback (most recent call last):
      File “”, line 16, in
      File “P:/custompath/customPath/nuke\nukeprocess.py”, line 119, in execute
      stdout = subprocess.PIPE, stderr = subprocess.STDOUT, shell=True, universal_newlines=True)
      File “C:\Program Files\Nuke6.0v5\lib\subprocess.py”, line 615, in __init__
      self.stdout = os.fdopen(c2pread, ‘rU’, bufsize)
      OSError: [Errno 22] Invalid argument

      (I changed the path due to confidence problems).
      It would be awesome if you could point me to a direction.. seems your script could be quiet useful.

      thanks
      Bastian

    8. flord

      From that error message, it seems you’re using nukeprocess.py from within Nuke. That’s not how it’s used. You should call nukeprocess.py from a standard python interpreter, and it will call Nuke for you.
      Am I right?

    9. Vincent

      Thanks Francois, will try this!!

    10. Tim

      Hi Francois,
      Thank you for this!!! I also have a problem like
      xavier .I get this:

      ‘C:\Program’ is not recognized as an internal or external command,
      operable program or batch file.

      I tried everything , all of my path is in quotes .And there is no big difference from your original path.But still get this error.I am using win7 x64 python 2.7 and Nuke 6.0v5
      I tried to run your first example but didn’t work.
      Please let me know what wrong I am doing.
      Thanks

    11. Michal

      Hi
      I wan’t to use your script to check for corrupted frames from renderfarm – so I build a simple comp reading a frame and trying to save it as JPG.
      For non-corrupted input frames it works OK, but with damaged one, script fails with following error, and, what more importand, freezes:

      Nuke 6.1v1, 64 bit, built Jul 28 2010.

      Copyright (c) 2010 The Foundry Visionmongers Ltd. All Rights Reserved.

      >>> >>> >>> … … … … … … >>> >>> >>> >>> >>> >>> False

      >>> False

      >>> … … >>> False

      >>> True

      >>> >>> >>> >>> True

      >>> >>> >>> [13:55.37] ERROR: Read1: Error reading pixel data from image file “//hal/RenderfarmConf/Apps/nukeprocess/partial.exr”. Scan line 432 is missing.

      Writing //hal/RenderfarmConf/Apps/nukeprocess/partial.exr.jpg took 0.42 seconds

      Total render time: 0.42 seconds

      .3Traceback (most recent call last):

      File “stdin”, line 1, in

      RuntimeError: Read1: Error reading pixel data from image file “//hal/RenderfarmConf/Apps/nukeprocess/partial.exr”. Scan line 432 is missing.

      >>> >>> Traceback (most recent call last):

      File “stdin”, line 1, in

      NameError: name ‘result’ is not defined

      Any hints? Thanks

    12. flord

      Yeah, Nuke isn’t too helpful when frames are corrupted. The problem is that nukeprocess runs outside of Nuke, not inside it. So once Nuke is launched, my script can’t change it’s behavior. I should check if anything changed in 6.2, it seems to handle problem frames better. But that won’t happen anytime soon, I don’t even have time to install the latest version at the office.
      Cheers.

    13. danieli903

      Hi Francois,
      I using nuke6.2 and python3.0. I try to make test for your first example (resize and color correct image). However when i run script occur following error:

      Traceback (most recent call last):
      File “testNuke.py”, line 1, in
      from nukeprocess import NukeProcess
      File “D:\users\daniel\setting\Downloads\nuke\nukeprocess.py”, line 133
      print sOutLine
      ^
      SyntaxError: invalid syntax

      Then i make comment this print line. i receive other error while run it again:

      Traceback (most recent call last):
      File “testNuke.py”, line 1, in
      from nukeprocess import NukeProcess
      File “D:\users\daniel\setting\Downloads\nuke\nukeprocess.py”, line 195
      raise TypeError, “Unrecognized knob variable type: %s\n%s: %s” % (type(dKwar
      gs[sKey], sKey, str(dKwargs[sKey])))
      ^
      SyntaxError: invalid syntax

      Any suggestion ? thanks you

    14. James

      How exactly do you set this up with python? I am on a mac running python 2.5.

      -J

    15. James

      Sorry, but I have no idea where to install this python file. Any help would be great! Im running python 2.5 on Snow Leopard

      -J

    16. James

      Awesome script! Thanks for putting this together Francois!!! There isn’t any way to make the script save out a .nk file is there?! Was hoping to use this to automate creation of .nk files with read/write nodes already prepared for artists.

      -James

    17. flord

      Yes, I guess it could be modified to output a .nk file. It would be just a matter to add a function that tells Nuke to save. I don’t have time now to do it and I don’t know when I will have some time, but anyone with some python and Nuke knowledge could do it.

    18. James

      Sorry about all these comments, but I just figured out a round-about way to do this and wanted to share it in case anybody wanted the feature (its extremely round-about though). If you create a script using Francois’ code, then cat the getScript() to somefile.py, you will need to write an extra line at the end of somefile.py:
      nuke.scriptSaveAs(‘path.nk’).

      Then if you run:
      /Applications/nuke/Nuke6.2v4.app/Nuke6.2v4 -nukex -V -t /PathTo/somefile.py

      You get the automated .nk script, which you can create from shotgun, which will have all of your read/write nodes already in it!

      I am both too busy and too new to both Nuke and Python to do this in a simpler/better fashion, but I wanted to contribute what I could. Again thanks soo much Francois for writing/sharing that amazing script!

      -James

    19. Jon Peel

      Just to point it out; this is at least the third tool which I’ve seen do this sort of thing and possibly not the friendliest solution. If you are looking at it, do compare it to Hugh McDonald’s external control module – it answers more or less the same problem.

      http://forums.thefoundry.co.uk/phpBB2/viewtopic.php?t=4203&postdays=0&postorder=asc&start=0

      https://github.com/Nvizible/NukeExternalControl

    Looking for something?

    Use the form below to search the site:

    Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!

    Related Links

    Related blogs I follow, or sites I like.