Tutorial 10: Python scripting

To access the tutorial projects in ArcGIS CityEngine, open CityEngine and click Help > Download Tutorials and Examples in the main menu. After choosing a tutorial or example, the project is automatically downloaded and added to your CityEngine workspace.

In this tutorial, you'll learn the basic usage of the Python console and the editor and explore several examples of the automatization of CityEngine tasks.

The Python scripting interface enhances the possibilities of CityEngine.

Use the Python console and editor

In this section, you'll run a simple selection using the Python console and editor.

Open a Python console

Complete the following steps to open a new Python console:

1. Expand the Tutorial_10_Python_Scripting tutorial folder in the Navigator window.
2. Open the 01_PythonScripting.cej scene in the scenes folder.
3. Click Window > Console in the main menu to open the Console window.
4. Click the drop-down arrow on the right and click Python Console.

   

5. Type ce.setS :

   

   You'll notice as you start typing, a list of commands appear that you can choose from: Your first CityEngine Python command allows you to quickly select scene objects with a specific name:

   Press Ctrl+Space to show the command completion pop-up if it isn't already open.

6. Double-click ce.setSelection in the list to complete the entry in the console.
7. Extend ce.setSelection to the following full command:

   >>> ce.setSelection(ce.getObjectsFrom(ce.scene, ce.withName("*Broadway*")))

8. Press Enter to select all the objects in the scene that contain the word "Broadway" in their name.

   

Create a Python module in the editor

When you want to use longer and more advanced Python commands or a set of commands, it's helpful to write a script using the Python editor in CityEngine.

To create a new Python module, do the following

1. Select File > New > Python > Python Module and click Next to open the Create a new Python module dialog box.
2. On the Create a new Python module dialog box, for Source Folder, browse to the \Tutorial_10_Python_Scripting\scripts\ folder and click OK.
3. in the Name box, type myHelpers.

   

4. Click Finish to open the Template window.
5. Select the Module: Main template.

   

6. Click OK.
   The new myHelpers Python module is created as the myHelpers.py file in the scripts folder and opens in the Python editor:

   

7. Add the new selectByAttribute(attr, value) function after the line ce = CE(). Ensure that you insert the code above the main function.

   def selectByAttribute(attr, value):
       objects = ce.getObjectsFrom(ce.scene)
       selection = []
       for o in objects:
           attrvalue = ce.getAttribute(o, attr)
           if attrvalue  ==  value:
               selection.append(o)
           
       ce.setSelection(selection)

   This function loops through all objects in the scene and reads the value of the attribute passed as the first parameter. It then compares the value to the value passed as the second parameter, and if they match, appends the object to the selection.

8. Select a street shape in the scene and click Object Attributes in the Inspector window to explore the available attributes:

   

9. Since you want the script to select all streets that are connected to a junction, add the function call with specific parameters to the main clause of the script:

   if __name__ == '__main__':
       selectByAttribute("connectionStart","JUNCTION")

10. Press Ctrl+S to save the module.
11. To run the script, press F9 while in the Python editor:

    The streets with Junction intersections are selected:

    

Run scripts from the console

Alternatively, you can call functions in scripts directly in the Python console.

1. In the Python console, add the path containing the module you just created to the system path.

   >>> sys.path.append(ce.toFSPath("scripts"))

2. Then, import the myHelpers module.

   >>> import myHelpers

3. Finally, call the helper function with arbitrary parameters:

   >>> myHelpers.selectByAttribute("connectionEnd", "JUNCTION")

   

Load modules automatically using scripting.py

To create and load the scripting.py script at the startup of CityEngine, do the following:

1. Create a file named scripting.py in your CityEngine workspace using the file browser of your operating system.

   Save the scripting.py file in the CityEngine workspace root folder, such as \{CityEngine Workspace}\scripting.py.

   Tip:

   You can find the path under File > Workspace > Other or right-click a project in the Navigator window and select Show in File Manager.

2. Add the following lines to automatically map the helper script at startup:

   from scripting import *
   
   # get a CityEngine instance
   ce = CE()
   
   import sys
    
   sys.path.append(ce.toFSPath("/Tutorial_10_Python_Scripting__2021_1/scripts"))
   import myHelpers

3. Restart CityEngine and open the 01_PythonScripting.cej scene to automatically load the myHelpers module.
4. Directly call the helper function from the Python console to verify that the automatic loading is working:

   >>> myHelpers.selectByAttribute("sidewalkWidthLeft", 4)

   All streets in which the sidewalkWidthLeft object attribute has a value of 4 are selected.

Note:

You can add arbitrary code to the scripting.py file. The scripting module runs automatically when a new Python console is opened or a script is run from the Python editor.

Ensure that your scripting.py file is valid and runs correctly; otherwise, Python code in CityEngine cannot be run. Open a Python console in CityEngine after you create or modify a scripting.py file; problems with running the scripting file are displayed there.

The scripting.py file is read only once on CityEngine startup. If you modify the file, be sure to restart CityEngine.

If the script is not correctly updated on CityEngine startup, delete the Python cache directory $USER_DIR/.cityengine/$CEVERSION_DIR/pythonCache/.

Change street widths

Often you need to increment the street width attribute of many segments. When you cannot do this efficiently in the GUI, using a Python script can be helpful.

incrementStreetWidths() function

In this section, you'll create a function to increment the streetWidths attribute of all the selected street segments from a value specified by the user.

1. Open the 02_PythonScripting.cej scene.
2. Create a new Python module called setStreetWidths.

   To create a new module, follow Steps 1 through 6 in the Create a Python module in the editor section above.

3. First, add the function definition:

   def incrementStreetWidths(increment):

4. In the function, get all selected segments and loop over them:

   selectedSegments = ce.getObjectsFrom(ce.selection, ce.isGraphSegment)
   for segment in selectedSegments:

5. In the for loop, in order to calculate the new street width, first get the current value using the ce.getAttribute() command:

   oldWidth = ce.getAttribute(segment, "/ce/street/streetWidth")

   Notice, that the syntax of the attribute name with the prefix /ce/street/ accesses the user attributes of the object. See Parameters and attributes for more information.

6. Finally, calculate the new street width by adding the user-provided parameter increment and assigning the new value to the segment:

   newWidth = oldWidth+increment
   ce.setAttribute(segment, "/ce/street/streetWidth", newWidth)

   The entire function is as follows:

   def incrementStreetWidths(increment):
       selectedSegments = ce.getObjectsFrom(ce.selection, ce.isGraphSegment)
       for segment in selectedSegments:
           oldWidth = ce.getAttribute(segment, "/ce/street/streetWidth")
           newWidth = oldWidth+increment
           ce.setAttribute(segment, "/ce/street/streetWidth", newWidth)

7. In the main block of the script, add the function call and choose an increment:

   if __name__ == '__main__':
      incrementStreetWidths(10)

8. Save the file.
9. Select some street segments.
10. Press F9 to run the Python script.

    

    Street widths before increment

    

    Street widths incremented by 10

Use the @noUIupdate decorator

Running the previous script may take some time if there are many segments selected. This is because scripts in CityEngine run in a separate thread and update the GUI and the Viewport window after every command. In this case, the street network is updated after every setAttribute() call and the viewport is redrawn.

To make run time faster, add the @noUIupdate decorator above the function definition:

@noUIupdate
def incrementStreetWidths(increment):

Functions marked this way block GUI update during execution, and depending on what they do, run faster by factors.

Caution:

Some combination of scripting commands with the @noUIupdate decorator may freeze the user interface.

If you encounter a UI freeze or other unexpected behavior when using @noUIupdate, modify your scripts so that @noUIupdate only marks a small specific function rather than marking your whole script.

multiplySegmentWidths() function

The multiplySegmentWidths() function sets several attributes at the same.time, specifically, the streetWidth, sidewalkWidthLeft, and sidewalkWidthRight attributes.

1. Add the following functions to the setStreetWidths script:

   def multiplySegmentWidths(factor):
       selectedSegments = ce.getObjectsFrom(ce.selection, ce.isGraphSegment)
       for segment in selectedSegments:
           multiplyAttribute(segment, "/ce/street/streetWidth", factor)
           multiplyAttribute(segment, "/ce/street/sidewalkWidthLeft", factor)
           multiplyAttribute(segment, "/ce/street/sidewalkWidthRight", factor)
   
   def multiplyAttribute(object, attrname, factor):
       oldval = ce.getAttribute(object, attrname)
       newval = oldval*factor
       ce.setAttribute(object, attrname, newval)

2. Replace the function call in the main block with the following:

   multiplySegmentWidths(1.5)

3. Save the module.
4. Select some street segments.
5. PressF9 to run the script.

   

   Segment widths before multiplied

   

   Segment widths multiplied by 1.5

   Notice that the street widths change much faster because the UI isn't refreshed after every change.

Animation: Grow the building

You can use Python scripts to automate generation or export processes. This workflow describes how to generate a building animation by setting the building attributes and exporting the set of resulting models.

Generate the building

To generate the building, do the following:

1. Open the 03_PythonScripting.cej scene.

   Click Yes when asked "Would you like to regenerate these models?"

2. Select the building.

   If the building is not generated, select the lot and click Generate (Ctrl+G) on the toolbar.

   

   The rule file includes attributes to change the dimensions of the building. Rather than manually setting these values in the Inspector window, you'll write a script that changes the values and batch generates the different versions of the model.

Animation script

To write the animation script, do the following:

1. Create a new Python module called growBuilding.
2. Add the growBuilding function to provide a time line that loops over two ranges and calls the setAttribute function:

   def growBuilding():
       for i in range(1,14):
           height = 20+i
           doStep(i,height,1)
   
       for i in range(15,35):
           height = 34
           width = i-14
           doStep(i,height,width)

3. Use the doStep function to modify the height and width attributes on the lot object:

   def doStep(i,height,width):    
       object = ce.getObjectsFrom(ce.scene, ce.withName("'Lot1'"))
       ce.setAttributeSource(object, "height", "OBJECT")
       ce.setAttributeSource(object, "width", "OBJECT")
       ce.setAttribute(object, "height", height)
       ce.setAttribute(object, "width", width)
       
       Generate(object)

4. Add the Generate function to generate the building:

   def Generate(object):
       ce.generateModels(object)

5. In the main part of the script, call the growBuilding function:

   if __name__ == '__main__':
      growBuilding()

6. Save the module.
7. Press F9 to run the script to batch generate the building:

   

8. To batch export the models. add the Export function. Ensure that it is before the main clause:

   def Export(i, object):
       dir = ce.toFSPath("models")
       file = "building_merge_" + str(i)
       # prepare export settings 
       settings = OBJExportModelSettings()
       settings.setBaseName(file)
       settings.setOutputPath(dir)
       # do export 
       ce.export(object, settings)

9. Replace the Generate call in doStep() function:

   #Generate(object)
   Export(i, object)

10. Save the module.
11. Press F9 to run the script.

    The models are batch exported to the models folder in the project.

Write an asset library rule file

If you have a large number of assets, it may be helpful to display all of them at once. This section shows you how to write a Python script that automatically generates a CGA rule file displaying the project's assets.

1. Open the 03_PythonScripting.cej scene if it's not already open.

   The Python script writes the rule file with the following structure:

   Lot -->  Geometries Textures
   
   Geometries --> 
   	 Geometry(assetpath)
   	 Geometry(assetpath)
   	 ...
   
   Geometry(asset) --> i(asset)

   This is for the geometry assets and the texture images.

2. Create a new Python module called asset_lib.
3. Add the new function writeCGALib:

   def writeCGAlib():

4. Write the header information and the Lot starting rule:

   cga = "/*Asset Library Loader : Generated by asset_lib.py*/\n version \"2023.0\"\n\n"
   
   cga += "Lot -->  Geometries Textures"

5. Write the Geometries rule and get all .obj files in the asset folder; prepare the Geometry(assetpath) rule call for each asset:

   cga += "\n\nGeometries --> "
   for obj in ce.getObjectsFrom("/", ce.isFile, ce.withName("/Tutorial_10*/assets/*.obj")):   
       cga += "\n\t t(2,0,0)  Geometry(\""+obj+"\")"

6. Write a similar rule for the texture assets:

   cga+="\n\nTextures --> \n\ts(1,0,0) set(scope.ty,3) set(scope.tz,0) i(\"facades/xy-plane.obj\")"   
   for jpg in ce.getObjectsFrom("/", ce.isFile, ce.withName("/Tutorial_10*/assets/*.jpg")):
       cga += "\n\tt(2,0,0)  Texture(\""+jpg+"\")"

7. Write the asset loader rules:

   cga += "\n\n Geometry(asset) --> s(1,0,0) i(asset) set(scope.ty,0) set(scope.tz,0)"
   
   cga += "\n\n Texture(asset) --> set(material.colormap, asset)"

8. Open a file handle for the .cga file and write the CGA content:

   cgafile = ce.toFSPath("rules/asset_lib.cga")
   CGA = open(cgafile, "w")
   CGA.write(cga)
   CGA.close()
   print "written file "+cgafile

9. Add the new assignAndGenerateLib() function:

   def assignAndGenerateLib():
       object = ce.getObjectsFrom(ce.scene, ce.withName("'Lot2'"))
       ce.refreshWorkspace()
       ce.setRuleFile(object, "asset_lib.cga")
       ce.setStartRule(object, "Lot")
       ce.generateModels(object)

   The function assigns the generated .cga file to a scene lot and generates the model.

10. Finally, call the two functions in the main clause:

    if __name__ == '__main__':
        writeCGAlib() 
        assignAndGenerateLib()

11. Save the module.
12. Press F9 to generate the library model.

    

Automate CityEngine tasks with startup.py

You can use Python to automate larger or repetitive tasks. For example, you may want to automate model generation from parcel information across the county.

To automate tasks, do the following:

1. Create a new Python module called automationJob.
2. Insert a function with the tasks for the automated job:

   def fgdbToKml(pathFGDB,layerName,ruleName,startRule = "Generate"):
       # open scene in the automation project
       ce.newFile('/scenes/emptyScene.cej')
        
       # load a database
       importSettings = FGDBImportSettings()
       importSettings.setDatasetFilter(['/'+layerName])
       ce.importFile(ce.toFSPath(pathFGDB), importSettings)
        
       # assign rule file based on the layer name
       layer = ce.getObjectsFrom(ce.scene, ce.isShapeLayer, ce.withName(layerName))[0]
       shapes = ce.getObjectsFrom(layer, ce.isShape)
       ce.setRuleFile(shapes, ruleName)
       ce.setStartRule(shapes, startRule)
        
       # export models to KML
       exportSettings = KMLExportModelSettings()
       exportSettings.setOutputPath(ce.toFSPath("models"))
       exportSettings.setBaseName(layerName)
       exportSettings.setCompression(True)
       ce.export(shapes, exportSettings)
        
       # close Scene
       ce.waitForUIIdle()
       ce.closeFile()

   For testing, add a call to this function in the '__main__' section. The provided fgdbToKml example imports shapes from a fileGDB to generate models and writes them out to KML:

   if __name__ == '__main__':
       fgdbToKml("data/CityData.gdb", "Footprints", 
                 "/ESRI.lib/rules/Buildings/Building_From_Footprint.cga", "Generate")
       pass
3. Save and press F9 to test whether everything is working properly. After completion, the models folder contains the Footprints.kmz file.

   Delete the Footprints.kmz file, as you'll create this again in a later step.

4. To automate this process, you'll need to put all the function parameters, used for testing the function above, into a configuration file.

   The tutorial project includes an example jobConfig.cfg file located in the data folder:

   [config]
   pathFGDB=data/CityData.gdb
   layerName=Footprints
   ruleName=/ESRI.lib/rules/Buildings/Building_From_Footprint.cga
   startRule=Generate

5. In the automationJob.py script, add the run(cfg) and getCfgValue(cfg,name) functions below the fgdbToKml function in order to run the automation job with the parameters stored in the configuration file:
   def getCfgValue(cfg,name):
       for c in cfg:
           if  c[0] == name: return c[1]
       return None
    
   def run(cfg):
       pathFGDB = getCfgValue(cfg,'pathfgdb')
       layerName = getCfgValue(cfg,'layername')
       ruleName = getCfgValue(cfg,'rulename')
       startRule = getCfgValue(cfg,'startrule')
        
       fgdbToKml(pathFGDB, layerName, ruleName, startRule)

   For your reference, you can open the automationJob_dist.py file in the scripts folder.

6. Next, create a separate CityEngine workspace for automation by clicking File > Switch Workspace > Other.

   Name the new workspace C:\Automation Workspace\.

7. Copy the \scripts\startup.py script from the tutorial project to the new C:\Automation Workspace\ root directory:

   from scripting import *
   from java import lang
   import ConfigParser, sys
    
   if __name__ == '__startup__':
       # get a CityEngine instance
       ce = CE()
        
       # get startup arguments
       projectFolder = lang.System.getProperty("projectFolder")
       configFilePath = lang.System.getProperty("configFilePath")
        
       # link the automation project into automation workspace
       if "automationProject" in ce.listProjects(): ce.removeProject("automationProject")
       ce.importProject(projectFolder, False, "automationProject")
        
       # read configuration file
       cp = ConfigParser.ConfigParser()
       cp.read(configFilePath)
       cfg = cp.items('config') # list of (name,value) pairs
        
       # run automation job
       sys.path.append(ce.toFSPath("/automationProject/scripts"))
       import automationJob
       automationJob.run(cfg)
        
       # safely shut down CityEngine
       ce.exit()

   Commands in the '__startup__' section of this Python script are automatically run at startup of CityEngine. The first startup argument defines the CityEngine project containing the automation job. It is linked into the automation workspace. The second argument contains the config file. It is parsed and handed over to the automation job as a list of (name,value) pairs. After the job finishes, CityEngine is safely shut down.

8. Open the command line and start CityEngine in the automation workspace, and hand over the job definition and parameters:

   <Path_to_CityEngine.exe> -data <Workspace_Folder> -vmargs 
   -DprojectFolder=<Project_Folder> -DconfigFilePath=<Configuration_FilePath>
    
   @ Example:
   > "C:\Program Files\Esri\CityEngine2023.0\CityEngine.exe" -data 
   "C:\Automation Workspace" -vmargs 
   -DprojectFolder="C:\CE_Workspace\Tutorial_10_Python_Scripting__2021_1" 
   -DconfigFilePath="C:\CE_Workspace\Tutorial_10_Python_Scripting__2021_1\data\jobConfig.cfg"

   After the job has finished, the models folder contains the Footprints.kmz output file which you can then open if you like in ArcGIS Earth:

   

Note:

The automationJob.py file contains a minimal job. Use the CityEnginePython reference to adapt it to your needs.

In this tutorial, you did the following:

• Learned the basic usage of the Python console and editor.
• Explored several examples of the automatization of CityEngine tasks.

See the CityEngine Python reference for more information.

To continue your learning with CityEngine, see the complete CityEngine tutorial catalog.