如何从 pytorch 模块获取子模块序列？

对于火炬module https://pytorch.org/docs/master/generated/torch.nn.Module.html，我想我可以用.named_children, .named_modules等来获取子模块的列表。但是，我想该列表不是按顺序给出的，对吧？一个例子：

In [19]: import transformers

In [20]: model = transformers.DistilBertForSequenceClassification.from_pretrained('distilb
    ...: ert-base-cased')

In [21]: [name for name, _ in model.named_children()]
Out[21]: ['distilbert', 'pre_classifier', 'classifier', 'dropout']

的顺序.named_children()在上面的模型中，给出了 distilbert、pre_classifier、classifier 和 dropout。但是，如果您检查code https://github.com/huggingface/transformers/blob/9931f817b75ecb2c8bb08b6e9d4cbec4b0933935/src/transformers/modeling_distilbert.py#L641，显然dropout发生在之前classifier。那么如何获得这些子模块的顺序呢？

在 Pytorch 中，结果为print(model) or .named_children()等根据声明顺序列出__init__模型的类别，例如

Case 1

class Model(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(1, 10, kernel_size=5)
        self.conv2 = nn.Conv2d(10, 20, kernel_size=5)
        self.fc1 = nn.Linear(320, 50)
        self.fc2 = nn.Linear(50, 10)
        self.conv2_drop = nn.Dropout2d()

    def forward(self, x):
        x = F.relu(F.max_pool2d(self.conv1(x), 2))
        x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
        x = x.view(-1, 320)
        x = F.relu(self.fc1(x))
        x = F.dropout(x, p=0.6)
        x = self.fc2(x)
        return F.log_softmax(x, dim=1)

model = Model()
print(model)
[name for name, _ in model.named_children()]
# output
['conv1', 'conv2', 'fc1', 'fc2', 'conv2_drop']

Case 2

更改了顺序fc1 and fc2构造函数中的层。

class Model(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(1, 10, kernel_size=5)
        self.conv2 = nn.Conv2d(10, 20, kernel_size=5)
        self.fc2 = nn.Linear(50, 10)
        self.fc1 = nn.Linear(320, 50)
        self.conv2_drop = nn.Dropout2d()

    def forward(self, x):
        x = F.relu(F.max_pool2d(self.conv1(x), 2))
        x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
        x = x.view(-1, 320)
        x = F.relu(self.fc1(x))
        x = F.dropout(x, p=0.6)
        x = self.fc2(x)
        return F.log_softmax(x, dim=1)

model = Model()
print(model)
[name for name, _ in model.named_children()]
# output
['conv1', 'conv2', 'fc2', 'fc1', 'conv2_drop']

这就是为什么classifier之前打印过dropout正如它在构造函数中声明的那样：

class DistilBertForSequenceClassification(DistilBertPreTrainedModel):
        ...
        self.distilbert = DistilBertModel(config)
        self.pre_classifier = nn.Linear(config.dim, config.dim)
        self.classifier = nn.Linear(config.dim, config.num_labels)
        self.dropout = nn.Dropout(config.seq_classif_dropout)

尽管如此，您可以使用模型的子模块.modules()等，但它们只会按照声明的顺序列出__init__。如果您只想打印基于的结构forward方法，您可以尝试使用pytorch 摘要 https://github.com/Fangyh09/pytorch-summary.