Tutorial 10: Python scripting

To access the tutorials in CityEngine, click Help > Download Tutorials and Examples. When you choose a tutorial or example, the project is automatically downloaded and added to your workspace.

The Python scripting interface enhances the possibilities of CityEngine. This tutorial explains the basic usage of the Python console and the editor and provides several examples of the automatization of CityEngine tasks.

See the CityEngine Python reference for more information.

Use the Python console and editor

To get started, complete the following steps:

1. Import the Tutorial_10_Python_Scripting project into your CityEngine workspace.
2. Open the Tutorial_10_Python_Scripting/scenes/01_PythonScripting.cej scene.

Open a Python console

Complete the following steps to open a new Python console:

1. To open the console window, click Window > Console.
2. Click the drop-down arrow on the right side of the toolbar to open a Python console.

   

Your first CityEngine Python command allows you to quickly select scene elements with a specific name.
3. Type ce.setSelection.
4. Press Ctrl+Space to show the command completion pop-up if it isn't already open.
5. Type the ce.setSelection(ce.getObjectsFrom(ce.scene, ce.withName("*Broadway*"))) command.
6. Press Enter.
This selects all scene elements with names that contain the word "Broadway."

Use the Python editor

As soon as you plan to use longer and more advanced Python commands, or a set of commands, it's helpful to use the Python editor in CityEngine.

1. To create a Python script, click File > New > Python Module.

   The Create a new Python module dialog box appears.

2. On the Create a new Python module dialog box, for Source Folder, browse to the scripts folder for your project.
3. Type myHelpers for the name of your new Python module.
4. Click Finish.

   

5. Select the Module: Main template.

   

6. Click OK.
The new Python module myHelpers opens in the Python editor in CityEngine.

Add the new selectByAttribute(attr, value) function after the line ce = CE(). Ensure that you insert the code above the main function.

def selectByAttribute(attr, value):
    objects = ce.getObjectsFrom(ce.scene)
    selection = []
    for o in objects:
        attrvalue = ce.getAttribute(o, attr)
        if attrvalue  ==  value:
            selection.append(o)
        
    ce.setSelection(selection)

Call it with specific parameters in the main clause of the script. Ensure that the main block is at the end of the file.

if __name__ == '__main__':
    selectByAttribute("connectionStart","JUNCTION")

To run the script, press F9 while in the Python editor.

Run scripts from the console

Alternatively, you can call helper scripts via the Python console by completing the following steps:

1. In the Python console, add the path to the module to the system path.
2. Import the module.

   >>> sys.path.append(ce.toFSPath("scripts"))
   >>> import myHelpers

3. Call the helper function in the console as follows, with arbitrary parameters:

   myHelpers.selectByAttribute("connectionEnd", "JUNCTION")

   

Extend the scripting.py script

To extend scripting.py, complete the following steps:

1. Create a file named scripting.py in your CityEngine workspace using the file browser of your operating system.
2. Add the following lines to automatically map the helper script at startup:

   import sys
    
   sys.path.append(ce.toFSPath("/Tutorial_10_Python_Scripting__2020_0/scripts"))
   import myHelpers

After a restart of CityEngine, the myHelpers module is loaded automatically. You can call the selection function in the console as follows:

>>> myHelpers.selectByAttribute("connectionEnd", "JUNCTION")

Note:

You can add arbitrary code to the scripting.py file. The scripting module runs automatically when a new Python console is opened or a script is run from the Python editor.

Ensure that your scripting.py file is valid and runs correctly; otherwise, Python code in CityEngine cannot be run. Open a Python console in CityEngine after you create or modify a scripting.py file; problems with running the scripting file are displayed there.

The scripting.py file is read only once on CityEngine startup. If you modify the file, be sure to restart CityEngine.

If the script is not correctly updated on CityEngine startup, delete the Python cache directory $USER_DIR/.cityengine/$CEVERSION_DIR/pythonCache/.

 

Change street widths

Often, you may want to increment the street width attribute of many segments. In situations in which this cannot be accomplished quickly in the GUI, a Python script can help.

Open the Tutorial_10_Python_Scripting/scenes/02_PythonScripting.cej scene.

Create the Python script

1. Create a rule file by clicking File > New > Python > Python Module.
2. Choose the project's script folder, name it setStreetWidths, and choose the Module: Main template.

incrementStreetWidths() function

This function increments the streetWidths attribute of all the selected street segments with a value specified by the user.

First, the function definition:

def incrementStreetWidths(increment):

You need to get all selected segments and loop over them.

selectedSegments = ce.getObjectsFrom(ce.selection, ce.isGraphSegment)
segment in selectedSegments:

To calculate the new street width, get the current value first using the ce.getAttribute() command. Note the syntax of the attribute name with the prefix /ce/street/; this accesses the user attributes of the object.

oldWidth = ce.getAttribute(segment, "/ce/street/streetWidth")

Finally, calculate the new street width by adding the user-provided parameter increment and assigning the new value to the segment.

newWidth = oldWidth+increment
ce.setAttribute(segment, "/ce/street/streetWidth", newWidth)

The entire function is as follows:

# increment the street width parameter of all selected street segments
def incrementStreetWidths(increment):
    selectedSegments = ce.getObjectsFrom(ce.selection, ce.isGraphSegment)
    for segment in selectedSegments:
        oldWidth = ce.getAttribute(segment, "/ce/street/streetWidth")
        newWidth = oldWidth+increment
        ce.setAttribute(segment, "/ce/street/streetWidth", newWidth)

• In the main block of the script, add the function call, and choose an increment.

if __name__ == '__main__':
   incrementStreetWidths(10)

Select a set of street segments.

Run the Python script by pressing F9 while in the Python editor.

Use the @noUIupdate marker

Running the previous script may take some time if many segments are selected. This is because scripts in CityEngine run in a separate thread and update the GUI and the 3D viewport after every command. In this case, after every setAttribute() call, the street network is updated, and the 3D viewport is redrawn.

While this is convenient for this example, normal run time needs to be faster. You can accomplish this by adding the @noUIupdate marker above the function definition.

@noUIupdate
def incrementStreetWidths(increment):

Functions marked this way will block GUI update during execution, and depending on what they do, will run faster by factors.

Caution:

Some combination of scripting commands with the @noUIupdate marker may freeze the user interface.

If you encounter a UI freeze or other unexpected behavior when using @noUIupdate, modify your scripts so that @noUIupdate only marks a small, specific function rather than marking your whole script.

multiplySegmentWidths() function

This function sets several attributes at the same time, specifically, streetWidth, sidewalkWidthLeft, and sidewalkWidthRight. You can specify a factor by which to multiply the widths.

@noUIupdate
def multiplySegmentWidths(factor):
    selectedSegments = ce.getObjectsFrom(ce.selection, ce.isGraphSegment)
    for segment in selectedSegments:

The helper function multiplyAttribute performs the multiplication for the attributes.

      multiplyAttribute(segment, "/ce/street/streetWidth", factor)
multiplyAttribute(segment, "/ce/street/sidewalkWidthLeft", factor)
multiplyAttribute(segment, "/ce/street/sidewalkWidthRight", factor)

def multiplyAttribute(object, attrname, factor):
    oldval = ce.getAttribute(object, attrname)
    newval = oldval*factor
    ce.setAttribute(object, attrname, newval)

Complete Python functions

# multiply street and sidewalk widths of all selected street segments by factor
@noUIupdate
def multiplySegmentWidths(factor):
    selectedSegments = ce.getObjectsFrom(ce.selection, ce.isGraphSegment)
    for segment in selectedSegments:
        multiplyAttribute(segment, "/ce/street/streetWidth", factor)
        multiplyAttribute(segment, "/ce/street/sidewalkWidthLeft", factor)
        multiplyAttribute(segment, "/ce/street/sidewalkWidthRight", factor)

# multiply attribute of object by factor
def multiplyAttribute(object, attrname, factor):
    oldval = ce.getAttribute(object, attrname)
    newval = oldval*factor
    ce.setAttribute(object, attrname, newval)

In the main block of the script, add the function call, and choose a multiplication factor.

if __name__ == '__main__':
   multiplySegmentWidths(1.5)

Select a set of street segments.

Run the Python script by pressing F9 while in the Python editor.

Run from console

Rather than setting the function arguments in the Python editor, you can call the functions described above from the Python console after importing the script module.

>> scriptpath = ce.toFSPath("scripts")
>> sys.path.append(scriptpath)
>> import setStreetWidths
>> setStreetWidths.multiplySegmentWidths(0.5)

 

Set camera from the FBX file

Next, you'll import static camera data into CityEngine via FBX export from Maya.

Open the Tutorial_10_Python_Scripting/scenes/02_PythonScripting.cej scene.

Export the camera to FBX (Maya)

If you do not have Maya, you can skip the following steps, and use the existing data/camera.fbx file.

1. In Maya, select the camera you want to export.
2. Click File > Export Selection.

On the export dialog box, ensure that the settings are set as in the following screen capture:

Camera import script

1. To create a rule file, click File > New > Python > Python Module.
2. Choose the project's script folder, name it importFBXCamera, and choose the Module: Main template.

Parse the FBX file

1. Parse lines and look for the ID.
2. Prepare the camera data in array.
3. Parse lines from the .fbx file that stores camera data.

def findCameraLine(lines):
    lineId = 0
    for line in lines:
        if 'Model::camera' in line:
            return lineId
        else:
            lineId = lineId + 1
    return 0

def parseLine(lines, id):
    data = False
    for line in lines:
        if line.find(id) >=0 :
            data = line.partition(id)[2]
            break
    if data:
        data = data[:len(data)-1] # strip \n
        data = data.split(",")        
    return data

def parseFbxCam(filename):
    f=open(filename)
    lines = f.readlines()
    cnt = 0
    cameraLine = findCameraLine(lines)
    loc = parseLine(lines[cameraLine:], 'P: "Lcl Translation", "Lcl Translation", "", "A",')
    rot = parseLine(lines[cameraLine:], 'P: "Lcl Rotation", "Lcl Rotation", "", "A",')
    return [loc,rot]

Set the CityEngine camera

Open the Viewport window, and call the position and rotation set functions.

def setCamData(data):
    viewport = ce.get3DViews()[0]
    setCamPosV(viewport, data[0])
    setCamRotV(viewport, data[1])

def setCamPosV(v, vec):
    v.setCameraPosition(vec[0], vec[1], vec[2])
    
def setCamRotV(v, vec):
    v.setCameraRotation(vec[0], vec[1], vec[2])

Primary function

def importFbxCamera(fbxfile):
   
    data = parseFbxCam(fbxfile)
    if(data[0] and data[1]) :
        setCamData(data)
        print "Camera set to "+str(data)
    else:
        print "No camera data found in file "+fbxfile

Call in the main block

if __name__ == '__main__':
    camfile = ce.toFSPath("data/camera.fbx")
    importFbxCamera(camfile)

Run the Python script by pressing F9 while in the Python editor.

Your camera should be positioned as in the following image:

Note:

Animation curves are not read; only the transformation camera at the frame of exporting is read.

The camera must be exported as a single object.

 

Animation: Grow the building

You can use Python scripts to automate generation or export processes. This workflow describes how to generate a building animation by setting the building attributes and exporting the set of resulting models.

Open the Tutorial_10_Python_Scripting/scenes/03_PythonScripting.cej scene.

Generate the building

1. Select the lot in the scene.
2. If the building is not generated, click Generate on the toolbar.

   

The rule file includes attributes to change the dimensions of the building. Rather than manually setting these values, you will write a script that changes the values and batch generates the different versions of the model.

Animation script

Create a Python main module named my_grow_building.py.

def growBuilding

This function provides a time line that loops over two ranges and calls the setAttribute function.

def growBuilding():
    for i in range(1,14):
        height = 20+i
        doStep(i,height,1)

    for i in range(15,35):
        height = 34
        width = i-14
        doStep(i,height,width)

def doStep

On the lot object, the two attributes' height and width are modified.

def doStep(i,height,width):    
    object = ce.getObjectsFrom(ce.scene, ce.withName("'Lot1'"))
    ce.setAttributeSource(object, "height", "OBJECT")
    ce.setAttributeSource(object, "width", "OBJECT")
    ce.setAttribute(object, "height", height)
    ce.setAttribute(object, "width", width)
    
    Generate(object)

def Generate

The following generates the building:

def Generate(object):
    ce.generateModels(object)

main

growBuilding is called in the main clause of the script.

if __name__ == '__main__':
   growBuilding()

Batch generate the building

1. Press F9 in the Python editor to run the script.

Batch export

Once you're confident with the generated models, add an additional function named Export and ensure that it is before the main clause.

def Export(i, object):
    dir = ce.toFSPath("models")
    file = "building_merge_" + str(i)
    # prepare export settings 
    settings = OBJExportModelSettings()
    settings.setBaseName(file)
    settings.setOutputPath(dir)
    # do export 
    ce.export(object, settings)

Replace the Generate call in doStep().

#Generate(object)
Export(i, object)

You can find the exported models in the /models/ folder.

 

Write an asset library rule file

If you have a large number of assets, it may be helpful to look at all of them. This section shows how you can generate a CGA rule file automatically, which displays the project's assets.

Open the Tutorial_10_Python_Scripting/scenes/03_PythonScripting.cej scene.

The Python script will write the rule file with the following structure:

Lot -->  Geometries Textures

Geometries --> 
	 Geometry(assetpath)
	 Geometry(assetpath)
	 ...

Geometry(asset) --> i(asset)

This is for the geometry assets and the texture images.

• Create a Python main module asset_lib.py.
• Add the new function writeCGALib.

def writeCGAlib():

Write header information, the starting rule Lot, and the Geometries rule.

cga = "/*Asset Library Loader : Generated by asset_lib.py*/\n version \"2020.0\"\n\n"

# write start rule
cga += "Lot -->  Geometries Textures"

# write rule showing geometries
cga += "\n\nGeometries --> "

Iterate over all .obj files in the asset folder, and prepare the rule call Geometry(assetpath) for each asset.

# get all .obj files from asset directory, and call their loader
for obj in ce.getObjectsFrom("/", ce.isFile, ce.withName("/Tutorial_10*/assets/*.obj")):   
# and write
cga += "\n\t t(2,0,0)  Geometry(\""+obj+"\")"

Write similar rules for the texture assets.

# write rule showing jpg textures
cga+="\n\nTextures -->
\n\ts(1,0,0) set(scope.ty,3) set(scope.tz,0) i(\"facades/xy-plane.obj\")"   

# get all .jpg files from asset directory, and call their loader
for jpg in ce.getObjectsFrom("/", ce.isFile, ce.withName("/Tutorial_10*/assets/*.jpg")):
    cga += "\n\tt(2,0,0)  Texture(\""+jpg+"\")"

Write the asset loader rules.

# write geometry loader rule
cga += "\n\n Geometry(asset) --> 
s(1,0,0) i(asset) set(scope.ty,0) set(scope.tz,0)"

# write texture loader rule
cga += "\n\n Texture(asset) --> set(material.colormap, asset)"

Open a file handle for the .cga file, and write the CGA content.

cgafile = ce.toFSPath("rules/asset_lib.cga")
CGA = open(cgafile, "w")
CGA.write(cga)
CGA.close()
print "written file "+cgafile

Add the new assignAndGenerateLib() function. It assigns the generated .cga file to a scene lot and generates the model.

def assignAndGenerateLib():
    object = ce.getObjectsFrom(ce.scene, ce.withName("'Lot2'"))
    ce.refreshWorkspace()
    ce.setRuleFile(object, "asset_lib.cga")
    ce.setStartRule(object, "Lot")
    ce.generateModels(object)

Finally, call the two functions in the main clause:

if __name__ == '__main__':
    writeCGAlib() 
    assignAndGenerateLib()

Generate the library model

In the Python editor, with the asset_lib.py file open, press F9.

 

Automate CityEngine tasks with startup.py

You can use Python to automate larger or repetitive tasks. For example, you may want to automate model generation from parcel information across the county.

To automate tasks, do the following:

1. Create a project or use an existing one. For this example, use the existing Tutorial_10_Python_Scripting__2020_0 tutorial project.
2. Create a Python main module in the project's scripts folder. The tutorial contains a minimal job named automationJob.py.
3. Insert a function with the tasks for the automated job. For testing, add a call to this function in the '__main__' section. The provided fgdbToKml example imports shapes from a fileGDB to generate models and writes them out to KML:
   automationJob.py

   from scripting import *
   # get a CityEngine instance
   ce = CE()
    
   def fgdbToKml(pathFGDB,layerName,ruleName,startRule = "Generate"):
       # open scene in the automation project
       ce.newFile('/scenes/emptyScene.cej')
        
       # load a database
       importSettings = FGDBImportSettings()
       importSettings.setDatasetFilter(['/'+layerName])
       ce.importFile(ce.toFSPath(pathFGDB), importSettings)
        
       # assign rule file based on the layer name
       layer = ce.getObjectsFrom(ce.scene, ce.isShapeLayer, ce.withName(layerName))[0]
       shapes = ce.getObjectsFrom(layer, ce.isShape)
       ce.setRuleFile(shapes, ruleName)
       ce.setStartRule(shapes, startRule)
        
       # export models to KML
       exportSettings = KMLExportModelSettings()
       exportSettings.setOutputPath(ce.toFSPath("models"))
       exportSettings.setBaseName(layerName)
       exportSettings.setCompression(True)
       ce.export(shapes, exportSettings)
        
       # close Scene
       ce.waitForUIIdle()
       ce.closeFile()
   
   if __name__ == '__main__':
       fgdbToKml("data/CityData.gdb", "NewShapes", 
                 "/ESRI.lib/rules/Buildings/Building_From_Footprint.cga", "Generate")
       pass

4. Create a configuration file to define the job parameters. The tutorial contains an example located in \data\jobConfig.cfg.
   jobConfig.cfg

   [config]
   pathFGDB=data/CityData.gdb
   layerName=NewShapes
   ruleName=/ESRI.lib/rules/Buildings/Building_From_Footprint.cga
   startRule=Generate

5. Add the functions run(cfg) and getCfgValue(cfg,name) to automationJob.py to run the automation job with parameters stored in the configuration file.
   automationJob.py

   ...
   def getCfgValue(cfg,name):
       for c in cfg:
           if  c[0] == name: return c[1]
       return None
    
   def run(cfg):
       pathFGDB = getCfgValue(cfg,'pathfgdb')
       layerName = getCfgValue(cfg,'layername')
       ruleName = getCfgValue(cfg,'rulename')
       startRule = getCfgValue(cfg,'startrule')
        
       fgdbToKml(pathFGDB, layerName, ruleName, startRule)

6. It is recommended that you use a separate CityEngine workspace for automation. Create a folder on your system named C:\Automation Workspace.
7. Copy the /scripts/startup.py file from the tutorial project to the new C:\Automation Workspace root directory.

   Commands in the '__startup__' section of this Python script are automatically run at startup of CityEngine. The first startup argument defines the CityEngine project containing the automation job. It will be linked into the automation workspace. The second argument contains the config file. It is parsed and handed over to the automation job as a list of (name,value) pairs. After the job finishes, CityEngine is safely shut down.

   startup.py

   from scripting import *
   from java import lang
   import ConfigParser, sys
    
   if __name__ == '__startup__':
       # get a CityEngine instance
       ce = CE()
        
       # get startup arguments
       projectFolder = lang.System.getProperty("projectFolder")
       configFilePath = lang.System.getProperty("configFilePath")
        
       # link the automation project into automation workspace
       if "automationProject" in ce.listProjects(): ce.removeProject("automationProject")
       ce.importProject(projectFolder, False, "automationProject")
        
       # read configuration file
       cp = ConfigParser.ConfigParser()
       cp.read(configFilePath)
       cfg = cp.items('config') # list of (name,value) pairs
        
       # run automation job
       sys.path.append(ce.toFSPath("/automationProject/scripts"))
       import automationJob
       automationJob.run(cfg)
        
       # safely shut down CityEngine
       ce.exit()

8. Open the command line and start CityEngine in the automation workspace, and hand over the job definition and parameters:
   Command

   <Path_to_CityEngine.exe> -data <Workspace_Folder> -vmargs 
   -DprojectFolder=<Project_Folder> -DconfigFilePath=<Configuration_FilePath>
    
   @ Example:
   > "C:\Program Files\Esri\CityEngine2020.0\CityEngine.exe" -data 
   "C:\Automation Workspace" -vmargs 
   -DprojectFolder="C:\CE_Workspace\Tutorial_10_Python_Scripting__2020_0" 
   -DconfigFilePath="C:\CE_Workspace\Tutorial_10_Python_Scripting__2020_0\data\jobConfig.cfg"

9. After the job has finished, the \models folder contains the Footprints.kmz output file shown below in ArcGIS Earth.

   

Note:

The automationJob.py file contains a minimal job. Use the CityEnginePython reference to adapt it to your needs.