原文：https://zhuanlan.zhihu.com/p/561152870
目录
收起
1. 准备工作：制作 MetaHuman 角色
1.1 创建 MetaHuman 角色
1.2 Quixel Bridge 下载 MetaHuman
1.3 导出 MetaHuman 到 UE4
2. UE 渲染 MetaHuman 的多视点图片
2.1 如何在 UE 中手动渲染视频？
2.2 Python 自动化渲染
2.3 全部代码
3. 渲染结果
随着 ECCV 2020 NeRF 的出现，在计算机视觉、计算机图形学、三维重建等领域砸开了无数的坑位，一篇 NeRF 养活了多少科研工作者、大厂员工。但是要想训练这样一个 Neural Radiance Filed 却不是一件容易的事情，首先从数据来看，训练一个 NeRF 需要该场景的多视点图片，同时还需要知道每张图片所对应的相机参数，要想得到这样的数据，可谓是耗时耗力。那么，有没有一种方式，可以虚拟的仿真出一组 NeRF 的训练数据？
答案当然是有的，而且不止一种方式。下面本文将分享一种使用 UE4 来渲染带有相机参数的多视点图片的方法。在正式开始之前，首先来介绍一下 MetaHuman，它就是我们要渲染的对象。在计算机视觉、三维重建领域，人脸方向的研究一直是一个热点，而 MetaHuman 则是一个高保真的数字人开源框架，我们可以利用 MetaHuman 来设计出自己想要的数字人形象，然后渲染出其多视点图片，用作进一步的研究。
在 UE4 中渲染 MetaHuman 之前，我们首先需要创造一个 Metahuman 角色，然后并将其导入到 UE4 中进行渲染，整体可分为以下几个步骤
下面分别对这几步进行说明：
登入 MetaHuman Creator 官网：https://metahuman.unrealengine.com/，会看到如下界面，左侧的 CREATE METAHUMAN 是系统默认给出的 Metahuman 角色，可以直接在其基础上进行修改，创作一个新的 Metahuman，这会存在左侧的 MY METAHUMAN 目录中。
具体的操作界面如下，整体分为三部分：脸部、毛发、身体。我们可以对这三部分中的各个细节进行操作，大家可以自行尝试~
我们刚刚创作好的 MetaHuman 角色是存在于官方的服务器上的，在对它进行操作之前，还需要借助 Quixel Bridge 工具将其下载下来。
Quixel Bridge 下载链接：https://quixel.com/bridge
在下载时候，我们使用 Epic Games 账号，后续用的也一直是这个账号。下载完成并登录之后，我们会看到如下界面
左侧导航栏中的 MetaHuman Presets 和 My MetaHuman 分别表示默认角色和我们前一步创建的角色。点击右上角绿色的下载按钮即可下载到本地。
在导出 MetaHuman 角色到 UE4 之前，我们需要配置：导出的目标引擎是什么、引擎的版本号、引擎的位置。如下图所示
之后，点击对应 MetaHuman 角色右上角的 Export 按钮即可将其导出到 UE 引擎中（注意：在导出时，要保证 UE 是打开的）。
在利用 Python 自动化渲染符合要求的多视点图片之前，我们首先要搞明白的一件事情是：如何手动的在 UE4 中渲染视频序列（视频和图片本质上是一样的，图片就是视频一帧一帧截出来的）？
step1：将 MetaHuman 导入到 UE4 的场景中，并将其定位到合适的位置。在场景中添加电影摄像机 CineCameraActor， 如下：
step2：创建过场动画，并添加关卡序列，如下
step3：依次点击，轨道 — Actor到Sequencer — CineCameraActor，创建 CineCameraActor 的轨道序列，如下
step4：在创建好的 CineCameraActor 轨道序列中添加关键帧（下图添加了 5 个关键帧，分别指定了相机的位置和旋转），使得相机按照既定方式运动，如下
step5：至此，视频序列创建完毕，点击 将影片渲染为视频序列或图片序列，完成导出
上述就是在 UE4 中渲染 MetaHuman 的整个过程，只需要鼠标点点就行了，比较简单。但是有一个严重的问题是：如果要渲染一个多视点 MetaHuman 数据集用作 NeRF 训练的话，我们是必须要知道相机的姿态的；而在上述过程中，我们只能准确的知道关键帧的相机姿态，其他帧的相机姿态都是被插值出来的。如果要手动定义数百上千个关键帧，又是相当繁琐的一项工作，因此我们下面考虑利用 python 来自动化的执行上述过程，自动化的定义关键帧，指定相机的运动。
首先，总结一下前一章节手动渲染的过程，整体上可以分为如下 4 个步骤：
上述 4 个步骤中，除了步骤一还需要我们手动操作以外，其余步骤全部都可以利用 python 脚本来实现。在 UE4 中使用 python 接口还需要进行一些简单的配置，具体操作可以参考：https://www.bilibili.com/video/BV1PE411d7z8
步骤一：角色导入以及人脸定位
由于我只想渲染 MetaHuman 的人脸区域，因此我需要将其余身体部分移动到相机视野之外。需要移动的身体部位有：Torso、Legs、Feet。
接着进行 MetaHuman 的定位，为了方便，我们可以将其 Face 位置定位在坐标原点（0,0,0），这是为了方便后面指定相机的位置与旋转。
步骤二：指定相机运动方式：位置和旋转
在这一步，我希望相机能够以 MetaHuman 的人脸（也就是坐标原点）为中心，做球形旋转运动。关于相机的外参，做如下设置：
渲染的相关的设置如下：
相机内参的设置见：步骤三具体的渲染代码。
现在，我们将相机的运动数据（相机位置和旋转）、渲染的相关设置，存为一个 .json 文件，在步骤三渲染的时候直接读取这些数据即可。代码如下：
import json
import numpy as np
# 得到相机的运动数据（位置、旋转）
def get_camera_transform():
radius = 35
camara_transform = []
for y_pitch in range(-30, 31, 15):
for z_yaw in range(-180, 181, 10):
location_x = radius * np.cos(z_yaw / 180 * np.pi) * np.cos(y_pitch / 180 * np.pi)
location_y = radius * np.sin(z_yaw / 180 * np.pi) * np.cos(y_pitch / 180 * np.pi)
location_z = -radius * np.sin(y_pitch / 180 * np.pi)
print([-location_x, -location_y, location_z, 0, y_pitch, z_yaw])
camara_transform.append([-location_x, -location_y, location_z, 0, y_pitch, z_yaw])
return camara_transform
# 将基础数据存为一个 .json 文件
def make_json():
camara_transform = get_camera_transform()
data_dict = {
f"film_fov": 100,
"film_resolution": 512,
"delay_every_frame": 1.5,
"output_image_path": "F:/projects/Unreal projects/MetaHumanRender/RenderImages/Danielle/face_hair",
"camera_data": {
"camera_qRrq9DqY": {
"type": "Camera",
"camera_transform": camara_transform
}
}
}
data_json = json.dumps(data_dict, indent=4)
with open("camera_data.json", 'w') as json_file:
json_file.write(data_json)
if __name__ == '__main__':
make_json()步骤三：读取相机运动数据，创建关卡序列添加关键帧
首先，读取上一节生成的.json文件，获取相机运动数据
def read_json_file(path):
file = open(path, "rb")
file_json = json.load(file)
export_path = file_json['output_image_path'] # image export path
film_fov = file_json.get('film_fov', None) # camera film fov
film_resolution = file_json.get("film_resolution", 512) # camera file resolution
delay_every_frame = file_json.get("delay_every_frame", 3.0) # sequence run delay every frame
camera_json = file_json['camera_data'] # camera data
camera_transform_array = []
camera_rotation_array = []
camera_name_array = []
for camera in camera_json:
camera_transform = camera_json[camera]["camera_transform"]
for index in range(len(camera_transform)):
camera_transform_array.append(unreal.Transform(
location=[camera_transform[index][0], camera_transform[index][1], camera_transform[index][2]],
rotation=[camera_transform[index][3], camera_transform[index][4], camera_transform[index][5]],
scale=[1.0, 1.0, 1.0]
))
camera_rotation_array.append(unreal.Rotator(
roll=camera_transform[index][3],
pitch=camera_transform[index][4],
yaw=camera_transform[index][5]
))
camera_name_array.append(camera)
return camera_transform_array, camera_rotation_array, camera_name_array, export_path, film_fov, film_resolution, delay_every_frame接着创建关卡序列，并添加关键帧，每一个相机姿态对应一个关键帧。在这一部分，做要做了如下几个事情，具体细节可参考代码
代码
def create_sequence(asset_name, camera_transform_array, camera_rotation_array, camera_name_array, film_fov,
package_path='/Game/'):
# create a sequence
sequence = unreal.AssetToolsHelpers.get_asset_tools().create_asset(asset_name, package_path, unreal.LevelSequence,
unreal.LevelSequenceFactoryNew())
sequence.set_display_rate(unreal.FrameRate(numerator=1, denominator=1))
sequence.set_playback_start_seconds(0)
sequence.set_playback_end_seconds(len(camera_transform_array))
# add a camera cut track
camera_cut_track = sequence.add_master_track(unreal.MovieSceneCameraCutTrack)
print(camera_transform_array[0], '=======================================================================')
print(camera_rotation_array[0].get_editor_property("roll"))
# 添加关键帧
for index in range(0, len(camera_name_array)):
camera_cut_section = camera_cut_track.add_section()
camera_cut_section.set_start_frame_bounded(6 * index)
camera_cut_section.set_end_frame_bounded(6 * index + len(camera_transform_array))
camera_cut_section.set_start_frame_seconds(6 * index)
camera_cut_section.set_end_frame_seconds(6 * index + len(camera_transform_array))
# Create a cine camera actor
# 在世界编辑器中创建一个电影摄像机CineCameraActor,并指定位置和旋转
camera_actor = unreal.EditorLevelLibrary().spawn_actor_from_class(unreal.CineCameraActor,
unreal.Vector(0, 0, 0),
unreal.Rotator(0, 0, 0))
# 指定电影摄像机CineCameraActor的名称
camera_actor.set_actor_label(camera_name_array[index])
# 返回CineCamera组件
camera_component = camera_actor.get_cine_camera_component()
ratio = math.tan(film_fov / 360.0 * math.pi) * 2
# step1：胶片背板设置
filmback = camera_component.get_editor_property("filmback")
# sensor_height：胶片或数字传感器的垂直尺寸，以毫米为单位
filmback.set_editor_property("sensor_height", 60)
# sensor_width：胶片或数字传感器的水平尺寸，以毫米为单位
filmback.set_editor_property("sensor_width", 60)
# step2：镜头设置
lens_settings = camera_component.get_editor_property("lens_settings")
lens_settings.set_editor_property("min_focal_length", 4.0)
lens_settings.set_editor_property("max_focal_length", 1000)
lens_settings.set_editor_property("min_f_stop", 1.2)
lens_settings.set_editor_property("max_f_stop", 500)
print(lens_settings)
# step3：聚焦设置
focal_length = 45 # 焦距
current_aperture = 500 # 光圈
camera_component.set_editor_property("current_focal_length", focal_length) # 设置当前焦距
camera_component.set_editor_property("current_aperture", current_aperture) # 设置当前光圈
fov = camera_component.get_editor_property("field_of_view")
print(camera_component.get_editor_property("current_focal_length"))
print(camera_component.get_editor_property("current_aperture"))
print(f"fov - {fov}")
# add a binding for the camera
camera_binding = sequence.add_possessable(camera_actor)
transform_track = camera_binding.add_track(unreal.MovieScene3DTransformTrack)
transform_section = transform_track.add_section()
transform_section.set_start_frame_bounded(6 * index)
transform_section.set_end_frame_bounded(6 * index + len(camera_transform_array))
transform_section.set_start_frame_seconds(6 * index)
transform_section.set_end_frame_seconds(6 * index + len(camera_transform_array))
# get channel for location_x location_y location_z
channel_location_x = transform_section.get_channels()[0]
channel_location_y = transform_section.get_channels()[1]
channel_location_z = transform_section.get_channels()[2]
# get key for rotation_y rotation_z
channel_rotation_x = transform_section.get_channels()[3]
channel_rotation_y = transform_section.get_channels()[4]
channel_rotation_z = transform_section.get_channels()[5]
# camera_transform = camera_transform_array[index]
# camera_location_x = camera_transform.get_editor_property("translation").get_editor_property("x")
# camera_location_y = camera_transform.get_editor_property("translation").get_editor_property("y")
# camera_location_z = camera_transform.get_editor_property("translation").get_editor_property("z")
for i in range(len(camera_transform_array)):
camera_transform = camera_transform_array[i]
camera_location_x = camera_transform.get_editor_property("translation").get_editor_property("x")
camera_location_y = camera_transform.get_editor_property("translation").get_editor_property("y")
camera_location_z = camera_transform.get_editor_property("translation").get_editor_property("z")
camera_rotation = camera_rotation_array[i]
camera_rotate_roll = camera_rotation.get_editor_property("roll")
camera_rotate_pitch = camera_rotation.get_editor_property("pitch")
camera_rotate_yaw = camera_rotation.get_editor_property("yaw")
new_time = unreal.FrameNumber(value=i)
channel_location_x.add_key(new_time, camera_location_x, 0.0)
channel_location_y.add_key(new_time, camera_location_y, 0.0)
channel_location_z.add_key(new_time, camera_location_z, 0.0)
channel_rotation_x.add_key(new_time, camera_rotate_roll, 0.0)
channel_rotation_y.add_key(new_time, camera_rotate_pitch, 0.0)
channel_rotation_z.add_key(new_time, camera_rotate_yaw, 0.0)
# if i % 5 == 0:
# print(new_time, [camera_location_x, camera_location_y, camera_location_z, camera_rotate_roll, camera_rotate_pitch, camera_rotate_yaw])
# add the binding for the camera cut section
camera_binding_id = sequence.make_binding_id(camera_binding, unreal.MovieSceneObjectBindingSpace.LOCAL)
camera_cut_section.set_camera_binding_id(camera_binding_id)
# save sequence asset
unreal.EditorAssetLibrary.save_loaded_asset(sequence, False)
return sequence步骤四：将关卡序列渲染为视频、图片序列
将创建的关卡序列渲染为电影序列，这部分主要就是设置一些渲染参数，对应手动渲染的 step5，具体参数的含义可参考 unreal 官方文档。代码如下：
# sequence movie
def render_sequence_to_movie(export_path, film_resolution, delay_every_frame, on_finished_callback):
# 1) Create an instance of our UAutomatedLevelSequenceCapture and override all of the settings on it. This class is currently
# set as a config class so settings will leak between the Unreal Sequencer Render-to-Movie UI and this object. To work around
# this, we set every setting via the script so that no changes the user has made via the UI will affect the script version.
# The users UI settings will be reset as an unfortunate side effect of this.
capture_settings = unreal.AutomatedLevelSequenceCapture()
# Set all POD settings on the UMovieSceneCapture
# capture_settings.settings.output_directory = unreal.DirectoryPath("../../../unreal-render-release/Saved/VideoCaptures1/")
capture_settings.settings.output_directory = unreal.DirectoryPath(export_path)
# If you game mode is implemented in Blueprint, load_asset(...) is going to return you the C++ type ('Blueprint') and not what the BP says it inherits from.
# Instead, because game_mode_override is a TSubclassOf<AGameModeBase> we can use unreal.load_class to get the UClass which is implicitly convertable.
# ie: capture_settings.settings.game_mode_override = unreal.load_class(None, "/Game/AI/TestingSupport/AITestingGameMode.AITestingGameMode_C")
capture_settings.settings.game_mode_override = None
capture_settings.settings.output_format = "{camera}" + "_{frame}"
capture_settings.settings.overwrite_existing = False
capture_settings.settings.use_relative_frame_numbers = False
capture_settings.settings.handle_frames = 0
capture_settings.settings.zero_pad_frame_numbers = 4
# If you wish to override the output framerate you can use these two lines, otherwise the framerate will be derived from the sequence being rendered
capture_settings.settings.use_custom_frame_rate = False
# capture_settings.settings.custom_frame_rate = unreal.FrameRate(24,1)
capture_settings.settings.resolution.res_x = film_resolution
capture_settings.settings.resolution.res_y = film_resolution
capture_settings.settings.enable_texture_streaming = False
capture_settings.settings.cinematic_engine_scalability = True
capture_settings.settings.cinematic_mode = True
capture_settings.settings.allow_movement = False # Requires cinematic_mode = True
capture_settings.settings.allow_turning = False # Requires cinematic_mode = True
capture_settings.settings.show_player = False # Requires cinematic_mode = True
capture_settings.settings.show_hud = False # Requires cinematic_mode = True
capture_settings.use_separate_process = False
capture_settings.close_editor_when_capture_starts = False # Requires use_separate_process = True
capture_settings.additional_command_line_arguments = "-NOSCREENMESSAGES" # Requires use_separate_process = True
capture_settings.inherited_command_line_arguments = "" # Requires use_separate_process = True
# Set all the POD settings on UAutomatedLevelSequenceCapture
capture_settings.use_custom_start_frame = False # If False, the system will automatically calculate the start based on sequence content
capture_settings.use_custom_end_frame = False # If False, the system will automatically calculate the end based on sequence content
capture_settings.custom_start_frame = unreal.FrameNumber(0) # Requires use_custom_start_frame = True
capture_settings.custom_end_frame = unreal.FrameNumber(0) # Requires use_custom_end_frame = True
capture_settings.warm_up_frame_count = 0
capture_settings.delay_before_warm_up = 0
capture_settings.delay_before_shot_warm_up = 0
capture_settings.delay_every_frame = delay_every_frame
capture_settings.write_edit_decision_list = True
# Tell the capture settings which level sequence to render with these settings. The asset does not need to be loaded,
# as we're only capturing the path to it and when the PIE instance is created it will load the specified asset.
# If you only had a reference to the level sequence, you could use "unreal.SoftObjectPath(mysequence.get_path_name())"
capture_settings.level_sequence_asset = unreal.SoftObjectPath(sequence_asset_path)
# To configure the video output we need to tell the capture settings which capture protocol to use. The various supported
# capture protocols can be found by setting the Unreal Content Browser to "Engine C++ Classes" and filtering for "Protocol"
# ie: CompositionGraphCaptureProtocol, ImageSequenceProtocol_PNG, etc. Do note that some of the listed protocols are not intended
# to be used directly.
# Right click on a Protocol and use "Copy Reference" and then remove the extra formatting around it. ie:
# Class'/Script/MovieSceneCapture.ImageSequenceProtocol_PNG' gets transformed into "/Script/MovieSceneCapture.ImageSequenceProtocol_PNG"
capture_settings.set_image_capture_protocol_type(
unreal.load_class(None, "/Script/MovieSceneCapture.ImageSequenceProtocol_PNG"))
# After we have set the capture protocol to a soft class path we can start editing the settings for the instance of the protocol that is internallyc reated.
capture_settings.get_image_capture_protocol().compression_quality = 85
# Finally invoke Sequencer's Render to Movie functionality. This will examine the specified settings object and either construct a new PIE instance to render in,
# or create and launch a new process (optionally shutting down your editor).
unreal.SequencerTools.render_movie(capture_settings, on_finished_callback)import sys
import unreal, os, json, math
# sequence asset path
sequence_asset_path = '/Game/Render_Sequence.Render_Sequence'
# read json file
def read_json_file(path):
file = open(path, "rb")
file_json = json.load(file)
export_path = file_json['output_image_path'] # image export path
film_fov = file_json.get('film_fov', None) # camera film fov
film_resolution = file_json.get("film_resolution", 512) # camera file resolution
delay_every_frame = file_json.get("delay_every_frame", 3.0) # sequence run delay every frame
camera_json = file_json['camera_data'] # camera data
camera_transform_array = []
camera_rotation_array = []
camera_name_array = []
for camera in camera_json:
camera_transform = camera_json[camera]["camera_transform"]
for index in range(len(camera_transform)):
camera_transform_array.append(unreal.Transform(
location=[camera_transform[index][0], camera_transform[index][1], camera_transform[index][2]],
rotation=[camera_transform[index][3], camera_transform[index][4], camera_transform[index][5]],
scale=[1.0, 1.0, 1.0]
))
camera_rotation_array.append(unreal.Rotator(
roll=camera_transform[index][3],
pitch=camera_transform[index][4],
yaw=camera_transform[index][5]
))
camera_name_array.append(camera)
return camera_transform_array, camera_rotation_array, camera_name_array, export_path, film_fov, film_resolution, delay_every_frame
def create_sequence(asset_name, camera_transform_array, camera_rotation_array, camera_name_array, film_fov,
package_path='/Game/'):
# create a sequence
sequence = unreal.AssetToolsHelpers.get_asset_tools().create_asset(asset_name, package_path, unreal.LevelSequence,
unreal.LevelSequenceFactoryNew())
sequence.set_display_rate(unreal.FrameRate(numerator=1, denominator=1))
sequence.set_playback_start_seconds(0)
sequence.set_playback_end_seconds(len(camera_transform_array))
# add a camera cut track
camera_cut_track = sequence.add_master_track(unreal.MovieSceneCameraCutTrack)
print(camera_transform_array[0], '=======================================================================')
print(camera_rotation_array[0].get_editor_property("roll"))
# 添加关键帧
for index in range(0, len(camera_name_array)):
camera_cut_section = camera_cut_track.add_section()
camera_cut_section.set_start_frame_bounded(6 * index)
camera_cut_section.set_end_frame_bounded(6 * index + len(camera_transform_array))
camera_cut_section.set_start_frame_seconds(6 * index)
camera_cut_section.set_end_frame_seconds(6 * index + len(camera_transform_array))
# Create a cine camera actor
# 在世界编辑器中创建一个电影摄像机CineCameraActor,并指定位置和旋转
camera_actor = unreal.EditorLevelLibrary().spawn_actor_from_class(unreal.CineCameraActor,
unreal.Vector(0, 0, 0),
unreal.Rotator(0, 0, 0))
# 指定电影摄像机CineCameraActor的名称
camera_actor.set_actor_label(camera_name_array[index])
# 返回CineCamera组件
camera_component = camera_actor.get_cine_camera_component()
ratio = math.tan(film_fov / 360.0 * math.pi) * 2
# step1：胶片背板设置
filmback = camera_component.get_editor_property("filmback")
# sensor_height：胶片或数字传感器的垂直尺寸，以毫米为单位
filmback.set_editor_property("sensor_height", 60)
# sensor_width：胶片或数字传感器的水平尺寸，以毫米为单位
filmback.set_editor_property("sensor_width", 60)
# step2：镜头设置
lens_settings = camera_component.get_editor_property("lens_settings")
lens_settings.set_editor_property("min_focal_length", 4.0)
lens_settings.set_editor_property("max_focal_length", 1000)
lens_settings.set_editor_property("min_f_stop", 1.2)
lens_settings.set_editor_property("max_f_stop", 500)
print(lens_settings)
# step3：聚焦设置
focal_length = 45 # 焦距
current_aperture = 500 # 光圈
camera_component.set_editor_property("current_focal_length", focal_length) # 设置当前焦距
camera_component.set_editor_property("current_aperture", current_aperture) # 设置当前光圈
fov = camera_component.get_editor_property("field_of_view")
print(camera_component.get_editor_property("current_focal_length"))
print(camera_component.get_editor_property("current_aperture"))
print(f"fov - {fov}")
# add a binding for the camera
camera_binding = sequence.add_possessable(camera_actor)
transform_track = camera_binding.add_track(unreal.MovieScene3DTransformTrack)
transform_section = transform_track.add_section()
transform_section.set_start_frame_bounded(6 * index)
transform_section.set_end_frame_bounded(6 * index + len(camera_transform_array))
transform_section.set_start_frame_seconds(6 * index)
transform_section.set_end_frame_seconds(6 * index + len(camera_transform_array))
# get channel for location_x location_y location_z
channel_location_x = transform_section.get_channels()[0]
channel_location_y = transform_section.get_channels()[1]
channel_location_z = transform_section.get_channels()[2]
# get key for rotation_y rotation_z
channel_rotation_x = transform_section.get_channels()[3]
channel_rotation_y = transform_section.get_channels()[4]
channel_rotation_z = transform_section.get_channels()[5]
# camera_transform = camera_transform_array[index]
# camera_location_x = camera_transform.get_editor_property("translation").get_editor_property("x")
# camera_location_y = camera_transform.get_editor_property("translation").get_editor_property("y")
# camera_location_z = camera_transform.get_editor_property("translation").get_editor_property("z")
for i in range(len(camera_transform_array)):
camera_transform = camera_transform_array[i]
camera_location_x = camera_transform.get_editor_property("translation").get_editor_property("x")
camera_location_y = camera_transform.get_editor_property("translation").get_editor_property("y")
camera_location_z = camera_transform.get_editor_property("translation").get_editor_property("z")
camera_rotation = camera_rotation_array[i]
camera_rotate_roll = camera_rotation.get_editor_property("roll")
camera_rotate_pitch = camera_rotation.get_editor_property("pitch")
camera_rotate_yaw = camera_rotation.get_editor_property("yaw")
new_time = unreal.FrameNumber(value=i)
channel_location_x.add_key(new_time, camera_location_x, 0.0)
channel_location_y.add_key(new_time, camera_location_y, 0.0)
channel_location_z.add_key(new_time, camera_location_z, 0.0)
channel_rotation_x.add_key(new_time, camera_rotate_roll, 0.0)
channel_rotation_y.add_key(new_time, camera_rotate_pitch, 0.0)
channel_rotation_z.add_key(new_time, camera_rotate_yaw, 0.0)
# if i % 5 == 0:
# print(new_time, [camera_location_x, camera_location_y, camera_location_z, camera_rotate_roll, camera_rotate_pitch, camera_rotate_yaw])
# add the binding for the camera cut section
camera_binding_id = sequence.make_binding_id(camera_binding, unreal.MovieSceneObjectBindingSpace.LOCAL)
camera_cut_section.set_camera_binding_id(camera_binding_id)
# save sequence asset
unreal.EditorAssetLibrary.save_loaded_asset(sequence, False)
return sequence
# sequence movie
def render_sequence_to_movie(export_path, film_resolution, delay_every_frame, on_finished_callback):
# 1) Create an instance of our UAutomatedLevelSequenceCapture and override all of the settings on it. This class is currently
# set as a config class so settings will leak between the Unreal Sequencer Render-to-Movie UI and this object. To work around
# this, we set every setting via the script so that no changes the user has made via the UI will affect the script version.
# The users UI settings will be reset as an unfortunate side effect of this.
capture_settings = unreal.AutomatedLevelSequenceCapture()
# Set all POD settings on the UMovieSceneCapture
# capture_settings.settings.output_directory = unreal.DirectoryPath("../../../unreal-render-release/Saved/VideoCaptures1/")
capture_settings.settings.output_directory = unreal.DirectoryPath(export_path)
# If you game mode is implemented in Blueprint, load_asset(...) is going to return you the C++ type ('Blueprint') and not what the BP says it inherits from.
# Instead, because game_mode_override is a TSubclassOf<AGameModeBase> we can use unreal.load_class to get the UClass which is implicitly convertable.
# ie: capture_settings.settings.game_mode_override = unreal.load_class(None, "/Game/AI/TestingSupport/AITestingGameMode.AITestingGameMode_C")
capture_settings.settings.game_mode_override = None
capture_settings.settings.output_format = "{camera}" + "_{frame}"
capture_settings.settings.overwrite_existing = False
capture_settings.settings.use_relative_frame_numbers = False
capture_settings.settings.handle_frames = 0
capture_settings.settings.zero_pad_frame_numbers = 4
# If you wish to override the output framerate you can use these two lines, otherwise the framerate will be derived from the sequence being rendered
capture_settings.settings.use_custom_frame_rate = False
# capture_settings.settings.custom_frame_rate = unreal.FrameRate(24,1)
capture_settings.settings.resolution.res_x = film_resolution
capture_settings.settings.resolution.res_y = film_resolution
capture_settings.settings.enable_texture_streaming = False
capture_settings.settings.cinematic_engine_scalability = True
capture_settings.settings.cinematic_mode = True
capture_settings.settings.allow_movement = False # Requires cinematic_mode = True
capture_settings.settings.allow_turning = False # Requires cinematic_mode = True
capture_settings.settings.show_player = False # Requires cinematic_mode = True
capture_settings.settings.show_hud = False # Requires cinematic_mode = True
capture_settings.use_separate_process = False
capture_settings.close_editor_when_capture_starts = False # Requires use_separate_process = True
capture_settings.additional_command_line_arguments = "-NOSCREENMESSAGES" # Requires use_separate_process = True
capture_settings.inherited_command_line_arguments = "" # Requires use_separate_process = True
# Set all the POD settings on UAutomatedLevelSequenceCapture
capture_settings.use_custom_start_frame = False # If False, the system will automatically calculate the start based on sequence content
capture_settings.use_custom_end_frame = False # If False, the system will automatically calculate the end based on sequence content
capture_settings.custom_start_frame = unreal.FrameNumber(0) # Requires use_custom_start_frame = True
capture_settings.custom_end_frame = unreal.FrameNumber(0) # Requires use_custom_end_frame = True
capture_settings.warm_up_frame_count = 0
capture_settings.delay_before_warm_up = 0
capture_settings.delay_before_shot_warm_up = 0
capture_settings.delay_every_frame = delay_every_frame
capture_settings.write_edit_decision_list = True
# Tell the capture settings which level sequence to render with these settings. The asset does not need to be loaded,
# as we're only capturing the path to it and when the PIE instance is created it will load the specified asset.
# If you only had a reference to the level sequence, you could use "unreal.SoftObjectPath(mysequence.get_path_name())"
capture_settings.level_sequence_asset = unreal.SoftObjectPath(sequence_asset_path)
# To configure the video output we need to tell the capture settings which capture protocol to use. The various supported
# capture protocols can be found by setting the Unreal Content Browser to "Engine C++ Classes" and filtering for "Protocol"
# ie: CompositionGraphCaptureProtocol, ImageSequenceProtocol_PNG, etc. Do note that some of the listed protocols are not intended
# to be used directly.
# Right click on a Protocol and use "Copy Reference" and then remove the extra formatting around it. ie:
# Class'/Script/MovieSceneCapture.ImageSequenceProtocol_PNG' gets transformed into "/Script/MovieSceneCapture.ImageSequenceProtocol_PNG"
capture_settings.set_image_capture_protocol_type(
unreal.load_class(None, "/Script/MovieSceneCapture.ImageSequenceProtocol_PNG"))
# After we have set the capture protocol to a soft class path we can start editing the settings for the instance of the protocol that is internallyc reated.
capture_settings.get_image_capture_protocol().compression_quality = 85
# Finally invoke Sequencer's Render to Movie functionality. This will examine the specified settings object and either construct a new PIE instance to render in,
# or create and launch a new process (optionally shutting down your editor).
unreal.SequencerTools.render_movie(capture_settings, on_finished_callback)
# sequence end call back
def on_render_movie_finished(success):
unreal.log('on_render_movie_finished is called')
if success:
unreal.log('---end LevelSequenceTask')
# check asset is exist
def check_sequence_asset_exist(root_dir):
sequence_path = os.path.join(root_dir, 'Content', 'Render_Sequence.uasset')
if (os.path.exists(sequence_path)):
unreal.log('---sequence is exist os path---')
os.remove(sequence_path)
else:
unreal.log('---sequence is not exist os path---')
if unreal.EditorAssetLibrary.does_asset_exist(sequence_asset_path):
unreal.log('---sequence dose asset exist---')
unreal.EditorAssetLibrary.delete_asset(sequence_asset_path)
else:
unreal.log('---sequence does not asset exist---')
sequence_asset_data = unreal.AssetRegistryHelpers.get_asset_registry().get_asset_by_object_path(sequence_asset_path)
if sequence_asset_data:
unreal.log('---sequence is exist on content---')
sequence_asset = unreal.AssetData.get_asset(sequence_asset_data)
unreal.EditorAssetLibrary.delete_loaded_asset(sequence_asset)
else:
unreal.log('---sequence is not exit on content---')
def main():
# bind event
on_finished_callback = unreal.OnRenderMovieStopped()
on_finished_callback.bind_callable(on_render_movie_finished)
# get json file
root_dir = unreal.SystemLibrary.get_project_directory()
json_path = os.path.join(root_dir, 'RawData', 'camera_data.json')
# get json data
camera_transform_array, camera_rotation_array, camera_name_array, export_path, film_fov, film_resolution, delay_every_frame = read_json_file(
json_path)
# check sequence is asset
check_sequence_asset_exist(root_dir)
# create sequence
create_sequence('Render_Sequence', camera_transform_array, camera_rotation_array, camera_name_array, film_fov)
# run sequence
render_sequence_to_movie(export_path, film_resolution, delay_every_frame, on_finished_callback)
上图即为渲染的多视点图片，每张图片对应的相机姿态也是确定的。
至此，在 UE4 中利用 python 实现自动化渲染的整个过程就结束了。在这里留下一个小问题：
UE4 中的相机姿态数据我们是无法直接使用的，如何将其转化为 NeRF 中的相机姿态数据呢？大家可以自行探索一下，后续文章再对这一部分的内容进行讲解。