System.img是如何打包的

快速链接:
.
👉👉👉 个人博客笔记导读目录(全部) 👈👈👈

• 付费专栏-付费课程 【购买须知】:
• ARMv8/ARMv9架构精选系列–目录 👈👈👈
• ARMV8/ARMV9/Trustzone/TEE安全课程👈👈👈

当敲击make命令时，会找到第一个目标droid（在build/core/main.mk中），droid依赖droid_targets，droid_targets依赖droidcore和dist_files，droidcore的依赖关系如下：

.PHONY: droidcore
droidcore: files \
	systemimage \
	$(INSTALLED_BOOTIMAGE_TARGET) \
	$(INSTALLED_RECOVERYIMAGE_TARGET) \
	$(INSTALLED_VBMETAIMAGE_TARGET) \
	$(INSTALLED_USERDATAIMAGE_TARGET) \
	$(INSTALLED_CACHEIMAGE_TARGET) \
	$(INSTALLED_BPTIMAGE_TARGET) \
	$(INSTALLED_VENDORIMAGE_TARGET) \
	$(INSTALLED_PRODUCTIMAGE_TARGET) \
	$(INSTALLED_SYSTEMOTHERIMAGE_TARGET) \
	$(INSTALLED_FILES_FILE) \
	$(INSTALLED_FILES_FILE_VENDOR) \
	$(INSTALLED_FILES_FILE_PRODUCT) \
	$(INSTALLED_FILES_FILE_SYSTEMOTHER) \
	soong_docs

切到build/core/Makefile中，我们看到systemimage又依赖INSTALLED_SYSTEMIMAGE，而INSTALLED_SYSTEMIMAGE又依赖BUILT_SYSTEMIMAGE，BUILT_SYSTEMIMAGE其实就是system.img

systemimage: $(INSTALLED_SYSTEMIMAGE)
$(INSTALLED_SYSTEMIMAGE): $(BUILT_SYSTEMIMAGE) $(RECOVERY_FROM_BOOT_PATCH)

BUILT_SYSTEMIMAGE := $(systemimage_intermediates)/system.img

我们再来看看BUILT_SYSTEMIMAGE的依赖关系：

$(BUILT_SYSTEMIMAGE): $(FULL_SYSTEMIMAGE_DEPS) $(INSTALLED_FILES_FILE) $(BUILD_IMAGE_SRCS)
	$(call build-systemimage-target,$@)

BUILT_SYSTEMIMAGE依赖的一些文件生成后，会调用makefile中的build-systemimage-target函数，我们接着去看build-systemimage-target的函数原型:

define build-systemimage-target
  @echo "Target system fs image: $(1)"
  $(call create-system-vendor-symlink)
  @mkdir -p $(dir $(1)) $(systemimage_intermediates) && rm -rf $(systemimage_intermediates)/system_image_info.txt
  $(call generate-userimage-prop-dictionary, $(systemimage_intermediates)/system_image_info.txt, \
      skip_fsck=true)
  $(hide) PATH=$(foreach p,$(INTERNAL_USERIMAGES_BINARY_PATHS),$(p):)$$PATH \
      ./build/tools/releasetools/build_image.py \
      $(TARGET_OUT) $(systemimage_intermediates)/system_image_info.txt $(1) $(TARGET_OUT) \
      || ( echo "Out of space? the tree size of $(TARGET_OUT) is (MB): " 1>&2 ;\
           du -sm $(TARGET_OUT) 1>&2;\
           if [ "$(INTERNAL_USERIMAGES_EXT_VARIANT)" == "ext4" ]; then \
               maxsize=$(BOARD_SYSTEMIMAGE_PARTITION_SIZE); \
               if [ "$(BOARD_HAS_EXT4_RESERVED_BLOCKS)" == "true" ]; then \
                   maxsize=$$((maxsize - 4096 * 4096)); \
               fi; \
               echo "The max is $$(( maxsize / 1048576 )) MB." 1>&2 ;\
           else \
               echo "The max is $$(( $(BOARD_SYSTEMIMAGE_PARTITION_SIZE) / 1048576 )) MB." 1>&2 ;\
           fi; \
           mkdir -p $(DIST_DIR); cp $(INSTALLED_FILES_FILE) $(DIST_DIR)/installed-files-rescued.txt; \
           exit 1 )
endef

看起来挺复杂吧，其实都是些shell语句，仔细看还是能看懂的。看不懂也没关系，我们只关心其中的关键语句 ./build/tools/releasetools/build_image.py，其后跟随了一堆的参数，我们不必关心这些参数，只要 明白，这里调用了build_image.py
打开build_image.py，main函数如下，其实并不复杂，这里随后调用了BuildImage()

def main(argv):
  if len(argv) != 4:
    print __doc__
    sys.exit(1)

  in_dir = argv[0]
  glob_dict_file = argv[1]
  out_file = argv[2]
  target_out = argv[3]

  glob_dict = LoadGlobalDict(glob_dict_file)
  if "mount_point" in glob_dict:
    # The caller knows the mount point and provides a dictionay needed by
    # BuildImage().
    image_properties = glob_dict
  else:
    image_filename = os.path.basename(out_file)
    mount_point = ""
    if image_filename == "system.img":
      mount_point = "system"
    elif image_filename == "system_other.img":
      mount_point = "system_other"
    elif image_filename == "userdata.img":
      mount_point = "data"
    elif image_filename == "cache.img":
      mount_point = "cache"
    elif image_filename == "vendor.img":
      mount_point = "vendor"
    elif image_filename == "oem.img":
      mount_point = "oem"
    else:
      print >> sys.stderr, "error: unknown image file name ", image_filename
      exit(1)

    image_properties = ImagePropFromGlobalDict(glob_dict, mount_point)

  if not BuildImage(in_dir, image_properties, out_file, target_out):
    print >> sys.stderr, "error: failed to build %s from %s" % (out_file,
                                                                in_dir)
    exit(1)

继续查看BuildImage()，原型如下：

def BuildImage(in_dir, prop_dict, out_file, target_out=None):
  """Build an image to out_file from in_dir with property prop_dict.

  Args:
    in_dir: path of input directory.
    prop_dict: property dictionary.
    out_file: path of the output image file.
    target_out: path of the product out directory to read device specific FS config files.

  Returns:
    True iff the image is built successfully.
  """
  # system_root_image=true: build a system.img that combines the contents of
  # /system and the ramdisk, and can be mounted at the root of the file system.
  origin_in = in_dir
  fs_config = prop_dict.get("fs_config")
  base_fs_file = None
  if (prop_dict.get("system_root_image") == "true"
      and prop_dict["mount_point"] == "system"):
    in_dir = tempfile.mkdtemp()
    # Change the mount point to "/"
    prop_dict["mount_point"] = "/"
    if fs_config:
      # We need to merge the fs_config files of system and ramdisk.
      fd, merged_fs_config = tempfile.mkstemp(prefix="root_fs_config",
                                              suffix=".txt")
      os.close(fd)
      with open(merged_fs_config, "w") as fw:
        if "ramdisk_fs_config" in prop_dict:
          with open(prop_dict["ramdisk_fs_config"]) as fr:
            fw.writelines(fr.readlines())
        with open(fs_config) as fr:
          fw.writelines(fr.readlines())
      fs_config = merged_fs_config

  build_command = []
  fs_type = prop_dict.get("fs_type", "")
  run_fsck = False

  fs_spans_partition = True
  if fs_type.startswith("squash"):
    fs_spans_partition = False

  is_verity_partition = "verity_block_device" in prop_dict
  verity_supported = prop_dict.get("verity") == "true"
  verity_fec_supported = prop_dict.get("verity_fec") == "true"

  # Adjust the partition size to make room for the hashes if this is to be
  # verified.
  if verity_supported and is_verity_partition:
    partition_size = int(prop_dict.get("partition_size"))
    (adjusted_size, verity_size) = AdjustPartitionSizeForVerity(partition_size,
                                                                verity_fec_supported)
    if not adjusted_size:
      return False
    prop_dict["partition_size"] = str(adjusted_size)
    prop_dict["original_partition_size"] = str(partition_size)
    prop_dict["verity_size"] = str(verity_size)

  # Adjust partition size for AVB hash footer or AVB hashtree footer.
  avb_footer_type = ''
  if prop_dict.get("avb_hash_enable") == "true":
    avb_footer_type = 'hash'
  elif prop_dict.get("avb_hashtree_enable") == "true":
    avb_footer_type = 'hashtree'

  if avb_footer_type:
    avbtool = prop_dict["avb_avbtool"]
    partition_size = prop_dict["partition_size"]
    # avb_add_hash_footer_args or avb_add_hashtree_footer_args.
    additional_args = prop_dict["avb_add_" + avb_footer_type + "_footer_args"]
    max_image_size = AVBCalcMaxImageSize(avbtool, avb_footer_type, partition_size,
                                         additional_args)
    if max_image_size == 0:
      return False
    prop_dict["partition_size"] = str(max_image_size)
    prop_dict["original_partition_size"] = partition_size

  if fs_type.startswith("ext"):
    build_command = [prop_dict["ext_mkuserimg"]]
    if "extfs_sparse_flag" in prop_dict:
      build_command.append(prop_dict["extfs_sparse_flag"])
      run_fsck = True
    build_command.extend([in_dir, out_file, fs_type,
                          prop_dict["mount_point"]])
    build_command.append(prop_dict["partition_size"])
    if "journal_size" in prop_dict:
      build_command.extend(["-j", prop_dict["journal_size"]])
    if "timestamp" in prop_dict:
      build_command.extend(["-T", str(prop_dict["timestamp"])])
    if fs_config:
      build_command.extend(["-C", fs_config])
    if target_out:
      build_command.extend(["-D", target_out])
    if "block_list" in prop_dict:
      build_command.extend(["-B", prop_dict["block_list"]])
    if "base_fs_file" in prop_dict:
      base_fs_file = ConvertBlockMapToBaseFs(prop_dict["base_fs_file"])
      if base_fs_file is None:
        return False
      build_command.extend(["-d", base_fs_file])
    build_command.extend(["-L", prop_dict["mount_point"]])
    if "extfs_inode_count" in prop_dict:
      build_command.extend(["-i", prop_dict["extfs_inode_count"]])
    if "flash_erase_block_size" in prop_dict:
      build_command.extend(["-e", prop_dict["flash_erase_block_size"]])
    if "flash_logical_block_size" in prop_dict:
      build_command.extend(["-o", prop_dict["flash_logical_block_size"]])
    # Specify UUID and hash_seed if using mke2fs.
    if prop_dict["ext_mkuserimg"] == "mkuserimg_mke2fs.sh":
      if "uuid" in prop_dict:
        build_command.extend(["-U", prop_dict["uuid"]])
      if "hash_seed" in prop_dict:
        build_command.extend(["-S", prop_dict["hash_seed"]])
    if "selinux_fc" in prop_dict:
      build_command.append(prop_dict["selinux_fc"])
  elif fs_type.startswith("squash"):
    build_command = ["mksquashfsimage.sh"]
    build_command.extend([in_dir, out_file])
    if "squashfs_sparse_flag" in prop_dict:
      build_command.extend([prop_dict["squashfs_sparse_flag"]])
    build_command.extend(["-m", prop_dict["mount_point"]])
    if target_out:
      build_command.extend(["-d", target_out])
    if fs_config:
      build_command.extend(["-C", fs_config])
    if "selinux_fc" in prop_dict:
      build_command.extend(["-c", prop_dict["selinux_fc"]])
    if "block_list" in prop_dict:
      build_command.extend(["-B", prop_dict["block_list"]])
    if "squashfs_compressor" in prop_dict:
      build_command.extend(["-z", prop_dict["squashfs_compressor"]])
    if "squashfs_compressor_opt" in prop_dict:
      build_command.extend(["-zo", prop_dict["squashfs_compressor_opt"]])
    if "squashfs_block_size" in prop_dict:
      build_command.extend(["-b", prop_dict["squashfs_block_size"]])
    if "squashfs_disable_4k_align" in prop_dict and prop_dict.get("squashfs_disable_4k_align") == "true":
      build_command.extend(["-a"])
  elif fs_type.startswith("f2fs"):
    build_command = ["mkf2fsuserimg.sh"]
    build_command.extend([out_file, prop_dict["partition_size"]])
  else:
    print("Error: unknown filesystem type '%s'" % (fs_type))
    return False

  if in_dir != origin_in:
    # Construct a staging directory of the root file system.
    ramdisk_dir = prop_dict.get("ramdisk_dir")
    if ramdisk_dir:
      shutil.rmtree(in_dir)
      shutil.copytree(ramdisk_dir, in_dir, symlinks=True)
    staging_system = os.path.join(in_dir, "system")
    shutil.rmtree(staging_system, ignore_errors=True)
    shutil.copytree(origin_in, staging_system, symlinks=True)

  has_reserved_blocks = prop_dict.get("has_ext4_reserved_blocks") == "true"
  ext4fs_output = None

  try:
    if fs_type.startswith("ext4"):
      (ext4fs_output, exit_code) = RunCommand(build_command)
    else:
      (_, exit_code) = RunCommand(build_command)
  finally:
    if in_dir != origin_in:
      # Clean up temporary directories and files.
      shutil.rmtree(in_dir, ignore_errors=True)
      if fs_config:
        os.remove(fs_config)
    if base_fs_file is not None:
      os.remove(base_fs_file)
  if exit_code != 0:
    return False

  # Bug: 21522719, 22023465
  # There are some reserved blocks on ext4 FS (lesser of 4096 blocks and 2%).
  # We need to deduct those blocks from the available space, since they are
  # not writable even with root privilege. It only affects devices using
  # file-based OTA and a kernel version of 3.10 or greater (currently just
  # sprout).
  # Separately, check if there's enough headroom space available. This is useful for
  # devices with low disk space that have system image variation between builds.
  if (has_reserved_blocks or "partition_headroom" in prop_dict) and fs_type.startswith("ext4"):
    assert ext4fs_output is not None
    ext4fs_stats = re.compile(
        r'Created filesystem with .* (?P<used_blocks>[0-9]+)/'
        r'(?P<total_blocks>[0-9]+) blocks')
    m = ext4fs_stats.match(ext4fs_output.strip().split('\n')[-1])
    used_blocks = int(m.groupdict().get('used_blocks'))
    total_blocks = int(m.groupdict().get('total_blocks'))
    reserved_blocks = 0
    headroom_blocks = 0
    adjusted_blocks = total_blocks
    if has_reserved_blocks:
      reserved_blocks = min(4096, int(total_blocks * 0.02))
      adjusted_blocks -= reserved_blocks
    if "partition_headroom" in prop_dict:
      headroom_blocks = int(prop_dict.get('partition_headroom')) / BLOCK_SIZE
      adjusted_blocks -= headroom_blocks
    if used_blocks > adjusted_blocks:
      mount_point = prop_dict.get("mount_point")
      print("Error: Not enough room on %s (total: %d blocks, used: %d blocks, "
            "reserved: %d blocks, headroom: %d blocks, available: %d blocks)" % (
                mount_point, total_blocks, used_blocks, reserved_blocks,
                headroom_blocks, adjusted_blocks))
      return False

  if not fs_spans_partition:
    mount_point = prop_dict.get("mount_point")
    partition_size = int(prop_dict.get("partition_size"))
    image_size = GetSimgSize(out_file)
    if image_size > partition_size:
      print("Error: %s image size of %d is larger than partition size of "
            "%d" % (mount_point, image_size, partition_size))
      return False
    if verity_supported and is_verity_partition:
      ZeroPadSimg(out_file, partition_size - image_size)

  # create the verified image if this is to be verified
  if verity_supported and is_verity_partition:
    if not MakeVerityEnabledImage(out_file, verity_fec_supported, prop_dict):
      return False

  # Add AVB HASH or HASHTREE footer (metadata).
  if avb_footer_type:
    avbtool = prop_dict["avb_avbtool"]
    original_partition_size = prop_dict["original_partition_size"]
    partition_name = prop_dict["partition_name"]
    # key_path and algorithm are only available when chain partition is used.
    key_path = prop_dict.get("avb_key_path")
    algorithm = prop_dict.get("avb_algorithm")
    salt = prop_dict.get("avb_salt")
    # avb_add_hash_footer_args or avb_add_hashtree_footer_args
    additional_args = prop_dict["avb_add_" + avb_footer_type + "_footer_args"]
    if not AVBAddFooter(out_file, avbtool, avb_footer_type, original_partition_size,
                        partition_name, key_path, algorithm, salt, additional_args):
      return False

  if run_fsck and prop_dict.get("skip_fsck") != "true":
    success, unsparse_image = UnsparseImage(out_file, replace=False)
    if not success:
      return False

    # Run e2fsck on the inflated image file
    e2fsck_command = ["e2fsck", "-f", "-n", unsparse_image]
    (_, exit_code) = RunCommand(e2fsck_command)

    os.remove(unsparse_image)

  return exit_code == 0

这里面有一堆的参数，其实都非常有深意后，我们在学习阶段可以不必考虑，当我们遇到相关问题时，那时可能就会对照这些参数，了解其功能和作用。 而我们这里重点查看的，如下：

 if prop_dict["ext_mkuserimg"] == "mkuserimg_mke2fs.sh":
      if "uuid" in prop_dict:
        build_command.extend(["-U", prop_dict["uuid"]])
      if "hash_seed" in prop_dict:
        build_command.extend(["-S", prop_dict["hash_seed"]])

这里是执行了mkuserimg_mke2fs.sh脚本，我们再去看看这个脚本吧
路径：system/extras/ext4_utils/mkuserimg_mke2fs.sh
同样，打开该文件后发现又是一堆的参数命令，看也看不懂，我们依然只是关心其核心的部分，截取如下：

MAKE_EXT4FS_CMD="mke2fs $MKE2FS_OPTS -t $EXT_VARIANT -b $BLOCKSIZE $OUTPUT_FILE $SIZE"
echo $MAKE_EXT4FS_ENV $MAKE_EXT4FS_CMD
env $MAKE_EXT4FS_ENV $MAKE_EXT4FS_CMD
if [ $? -ne 0 ]; then
  exit 4
fi

if [[ $E2FSPROGS_FAKE_TIME ]]; then
  E2FSDROID_ENV="E2FSPROGS_FAKE_TIME=$E2FSPROGS_FAKE_TIME"
fi

E2FSDROID_CMD="e2fsdroid $E2FSDROID_OPTS -f $SRC_DIR -a $MOUNT_POINT $OUTPUT_FILE"
echo $E2FSDROID_ENV $E2FSDROID_CMD
env $E2FSDROID_ENV $E2FSDROID_CMD
if [ $? -ne 0 ]; then
  rm -f $OUTPUT_FILE
  exit 4
fi

这里做了两件事：
（1）mke2fs打包生成了system.img
（2）e2fsdroid制作了system.map
mke2fs对应的源码：external/e2fsprogs/misc/mke2fs.c
e2fsdroid对应的源码 ： external/e2fsprogs/contrib/android/e2fsdroid.c

深思：
其实呢，在正常的android编译，并不会生成system.map，因为最后一个参数$OUTPUT_FILE为空. 在android的设计中system.map是在制作OTA的target-files文件时生成的，然后在制作block差分别时候，会用到system.map，一个block一个block的去求diff。
在制作OTA的target-files时，会重新打包system.img，其流程：
add_img_to_target_files直接调用了build_image.py中的CreateImage（）函数，与正常的打包system.img参数稍微不同，这里增加了一个block_list="system.map"参数，所以其打包结束后，生成了system.map

Q & A:
1、system.img是怎样制作的？
答：使用mke2fs工具制作的
2、system.map是做什么用的？先有的system.img，还是先用的system.map？
答：system.map是制作OTA的target-files文件时，重新打包system.img后，根据system.img做出的system.map，制作差分别时候，会用到system.map文件

不足之处，欢迎指正！

《ARMv8/ARMv9架构学习系列课程》全系列，共计52节课，超16h的视频课程