带过滤器的 Mongo 日期范围索引

2024-02-16

我们有以下查询

db.Comment.find(
    {
        $and: [ 
            { reportCount: { $gt: 0 } },
            { assignee: { $exists: false } }, 
            { creationDate: { $gt: new Date(1507831097809) } },
            { creationDate: { $lt: new Date(1508522297966) } },  
            { siteId: 'MAIN' }, 
            { parent: { $exists: false } }, 
            { status: 'ACTIVE' }
        ]
    })
    .sort({ creationDate: 1 })

我们有一个索引

 {
    "v" : 2,
    "key" : {
        "creationDate" : 1,
        "reportCount" : 1,
        "label" : 1
    }
}

这里有explain结果:

{
"queryPlanner" : {
    "plannerVersion" : 1,
    "namespace" : "myNameSpace",
    "indexFilterSet" : false,
    "parsedQuery" : {
        "$and" : [ 
            {
                "siteId" : {
                    "$eq" : "MAIN"
                }
            }, 
            {
                "status" : {
                    "$eq" : "ACTIVE"
                }
            }, 
            {
                "creationDate" : {
                    "$lt" : ISODate("2017-10-20T17:58:17.966Z")
                }
            }, 
            {
                "creationDate" : {
                    "$gt" : ISODate("2017-10-12T17:58:17.809Z")
                }
            }, 
            {
                "reportCount" : {
                    "$gt" : 0.0
                }
            }, 
            {
                "$nor" : [ 
                    {
                        "assignee" : {
                            "$exists" : true
                        }
                    }
                ]
            }, 
            {
                "$nor" : [ 
                    {
                        "parent" : {
                            "$exists" : true
                        }
                    }
                ]
            }
        ]
    },
    "winningPlan" : {
        "stage" : "FETCH",
        "filter" : {
            "$and" : [ 
                {
                    "siteId" : {
                        "$eq" : "MAIN"
                    }
                }, 
                {
                    "status" : {
                        "$eq" : "ACTIVE"
                    }
                }, 
                {
                    "$nor" : [ 
                        {
                            "assignee" : {
                                "$exists" : true
                            }
                        }
                    ]
                }, 
                {
                    "$nor" : [ 
                        {
                            "parent" : {
                                "$exists" : true
                            }
                        }
                    ]
                }
            ]
        },
        "inputStage" : {
            "stage" : "IXSCAN",
            "keyPattern" : {
                "creationDate" : 1.0,
                "reportCount" : 1.0,
                "label" : 1.0
            },
            "indexName" : "creationDate_1_reportCount_1_label_1",
            "isMultiKey" : false,
            "multiKeyPaths" : {
                "creationDate" : [],
                "reportCount" : [],
                "label" : []
            },
            "isUnique" : false,
            "isSparse" : false,
            "isPartial" : false,
            "indexVersion" : 2,
            "direction" : "forward",
            "indexBounds" : {
                "creationDate" : [ 
                    "(new Date(1507831097809), new Date(1508522297966))"
                ],
                "reportCount" : [ 
                    "(0.0, inf.0]"
                ],
                "label" : [ 
                    "[MinKey, MaxKey]"
                ]
            }
        }
    },
    "rejectedPlans" : [ 
        {
            "stage" : "SORT",
            "sortPattern" : {
                "creationDate" : 1.0
            },
            "inputStage" : {
                "stage" : "SORT_KEY_GENERATOR",
                "inputStage" : {
                    "stage" : "FETCH",
                    "filter" : {
                        "$and" : [ 
                            {
                                "$nor" : [ 
                                    {
                                        "parent" : {
                                            "$exists" : true
                                        }
                                    }
                                ]
                            }, 
                            {
                                "siteId" : {
                                    "$eq" : "MAIN"
                                }
                            }, 
                            {
                                "status" : {
                                    "$eq" : "ACTIVE"
                                }
                            }, 
                            {
                                "creationDate" : {
                                    "$lt" : ISODate("2017-10-20T17:58:17.966Z")
                                }
                            }, 
                            {
                                "creationDate" : {
                                    "$gt" : ISODate("2017-10-12T17:58:17.809Z")
                                }
                            }, 
                            {
                                "reportCount" : {
                                    "$gt" : 0.0
                                }
                            }, 
                            {
                                "$nor" : [ 
                                    {
                                        "assignee" : {
                                            "$exists" : true
                                        }
                                    }
                                ]
                            }
                        ]
                    },
                    "inputStage" : {
                        "stage" : "IXSCAN",
                        "keyPattern" : {
                            "parent" : 1.0
                        },
                        "indexName" : "parent_1",
                        "isMultiKey" : false,
                        "multiKeyPaths" : {
                            "parent" : []
                        },
                        "isUnique" : false,
                        "isSparse" : false,
                        "isPartial" : false,
                        "indexVersion" : 2,
                        "direction" : "forward",
                        "indexBounds" : {
                            "parent" : [ 
                                "[null, null]"
                            ]
                        }
                    }
                }
            }
        }, 
        {
            "stage" : "SORT",
            "sortPattern" : {
                "creationDate" : 1.0
            },
            "inputStage" : {
                "stage" : "SORT_KEY_GENERATOR",
                "inputStage" : {
                    "stage" : "FETCH",
                    "filter" : {
                        "$and" : [ 
                            {
                                "$nor" : [ 
                                    {
                                        "assignee" : {
                                            "$exists" : true
                                        }
                                    }
                                ]
                            }, 
                            {
                                "siteId" : {
                                    "$eq" : "MAIN"
                                }
                            }, 
                            {
                                "status" : {
                                    "$eq" : "ACTIVE"
                                }
                            }, 
                            {
                                "creationDate" : {
                                    "$lt" : ISODate("2017-10-20T17:58:17.966Z")
                                }
                            }, 
                            {
                                "creationDate" : {
                                    "$gt" : ISODate("2017-10-12T17:58:17.809Z")
                                }
                            }, 
                            {
                                "reportCount" : {
                                    "$gt" : 0.0
                                }
                            }, 
                            {
                                "$nor" : [ 
                                    {
                                        "parent" : {
                                            "$exists" : true
                                        }
                                    }
                                ]
                            }
                        ]
                    },
                    "inputStage" : {
                        "stage" : "IXSCAN",
                        "keyPattern" : {
                            "assignee" : 1.0
                        },
                        "indexName" : "assignee_1",
                        "isMultiKey" : false,
                        "multiKeyPaths" : {
                            "assignee" : []
                        },
                        "isUnique" : false,
                        "isSparse" : false,
                        "isPartial" : false,
                        "indexVersion" : 2,
                        "direction" : "forward",
                        "indexBounds" : {
                            "assignee" : [ 
                                "[null, null]"
                            ]
                        }
                    }
                }
            }
        }, 
        {
            "stage" : "SORT",
            "sortPattern" : {
                "creationDate" : 1.0
            },
            "inputStage" : {
                "stage" : "SORT_KEY_GENERATOR",
                "inputStage" : {
                    "stage" : "FETCH",
                    "filter" : {
                        "$and" : [ 
                            {
                                "status" : {
                                    "$eq" : "ACTIVE"
                                }
                            }, 
                            {
                                "creationDate" : {
                                    "$lt" : ISODate("2017-10-20T17:58:17.966Z")
                                }
                            }, 
                            {
                                "creationDate" : {
                                    "$gt" : ISODate("2017-10-12T17:58:17.809Z")
                                }
                            }, 
                            {
                                "reportCount" : {
                                    "$gt" : 0.0
                                }
                            }, 
                            {
                                "$nor" : [ 
                                    {
                                        "assignee" : {
                                            "$exists" : true
                                        }
                                    }
                                ]
                            }, 
                            {
                                "$nor" : [ 
                                    {
                                        "parent" : {
                                            "$exists" : true
                                        }
                                    }
                                ]
                            }
                        ]
                    },
                    "inputStage" : {
                        "stage" : "IXSCAN",
                        "keyPattern" : {
                            "siteId" : 1.0,
                            "updatedDate" : 1.0,
                            "label" : 1.0
                        },
                        "indexName" : "siteId_1_updatedDate_1_label_1",
                        "isMultiKey" : false,
                        "multiKeyPaths" : {
                            "siteId" : [],
                            "updatedDate" : [],
                            "label" : []
                        },
                        "isUnique" : false,
                        "isSparse" : false,
                        "isPartial" : false,
                        "indexVersion" : 2,
                        "direction" : "forward",
                        "indexBounds" : {
                            "siteId" : [ 
                                "[\"MAIN\", \"MAIN\"]"
                            ],
                            "updatedDate" : [ 
                                "[MinKey, MaxKey]"
                            ],
                            "label" : [ 
                                "[MinKey, MaxKey]"
                            ]
                        }
                    }
                }
            }
        }, 
        {
            "stage" : "SORT",
            "sortPattern" : {
                "creationDate" : 1.0
            },
            "inputStage" : {
                "stage" : "SORT_KEY_GENERATOR",
                "inputStage" : {
                    "stage" : "FETCH",
                    "filter" : {
                        "$and" : [ 
                            {
                                "$nor" : [ 
                                    {
                                        "parent" : {
                                            "$exists" : true
                                        }
                                    }
                                ]
                            }, 
                            {
                                "$nor" : [ 
                                    {
                                        "assignee" : {
                                            "$exists" : true
                                        }
                                    }
                                ]
                            }, 
                            {
                                "siteId" : {
                                    "$eq" : "MAIN"
                                }
                            }, 
                            {
                                "status" : {
                                    "$eq" : "ACTIVE"
                                }
                            }, 
                            {
                                "creationDate" : {
                                    "$lt" : ISODate("2017-10-20T17:58:17.966Z")
                                }
                            }, 
                            {
                                "creationDate" : {
                                    "$gt" : ISODate("2017-10-12T17:58:17.809Z")
                                }
                            }, 
                            {
                                "reportCount" : {
                                    "$gt" : 0.0
                                }
                            }
                        ]
                    },
                    "inputStage" : {
                        "stage" : "AND_SORTED",
                        "inputStages" : [ 
                            {
                                "stage" : "IXSCAN",
                                "keyPattern" : {
                                    "parent" : 1.0
                                },
                                "indexName" : "parent_1",
                                "isMultiKey" : false,
                                "multiKeyPaths" : {
                                    "parent" : []
                                },
                                "isUnique" : false,
                                "isSparse" : false,
                                "isPartial" : false,
                                "indexVersion" : 2,
                                "direction" : "forward",
                                "indexBounds" : {
                                    "parent" : [ 
                                        "[null, null]"
                                    ]
                                }
                            }, 
                            {
                                "stage" : "IXSCAN",
                                "keyPattern" : {
                                    "assignee" : 1.0
                                },
                                "indexName" : "assignee_1",
                                "isMultiKey" : false,
                                "multiKeyPaths" : {
                                    "assignee" : []
                                },
                                "isUnique" : false,
                                "isSparse" : false,
                                "isPartial" : false,
                                "indexVersion" : 2,
                                "direction" : "forward",
                                "indexBounds" : {
                                    "assignee" : [ 
                                        "[null, null]"
                                    ]
                                }
                            }
                        ]
                    }
                }
            }
        }
    ]
},
"executionStats" : {
    "executionSuccess" : true,
    "nReturned" : 19,
    "executionTimeMillis" : 8,
    "totalKeysExamined" : 533,
    "totalDocsExamined" : 56,
    "executionStages" : {
        "stage" : "FETCH",
        "filter" : {
            "$and" : [ 
                {
                    "siteId" : {
                        "$eq" : "MAIN"
                    }
                }, 
                {
                    "status" : {
                        "$eq" : "ACTIVE"
                    }
                }, 
                {
                    "$nor" : [ 
                        {
                            "assignee" : {
                                "$exists" : true
                            }
                        }
                    ]
                }, 
                {
                    "$nor" : [ 
                        {
                            "parent" : {
                                "$exists" : true
                            }
                        }
                    ]
                }
            ]
        },
        "nReturned" : 19,
        "executionTimeMillisEstimate" : 0,
        "works" : 534,
        "advanced" : 19,
        "needTime" : 513,
        "needYield" : 0,
        "saveState" : 20,
        "restoreState" : 20,
        "isEOF" : 1,
        "invalidates" : 0,
        "docsExamined" : 56,
        "alreadyHasObj" : 0,
        "inputStage" : {
            "stage" : "IXSCAN",
            "nReturned" : 56,
            "executionTimeMillisEstimate" : 0,
            "works" : 533,
            "advanced" : 56,
            "needTime" : 476,
            "needYield" : 0,
            "saveState" : 20,
            "restoreState" : 20,
            "isEOF" : 1,
            "invalidates" : 0,
            "keyPattern" : {
                "creationDate" : 1.0,
                "reportCount" : 1.0,
                "label" : 1.0
            },
            "indexName" : "creationDate_1_reportCount_1_label_1",
            "isMultiKey" : false,
            "multiKeyPaths" : {
                "creationDate" : [],
                "reportCount" : [],
                "label" : []
            },
            "isUnique" : false,
            "isSparse" : false,
            "isPartial" : false,
            "indexVersion" : 2,
            "direction" : "forward",
            "indexBounds" : {
                "creationDate" : [ 
                    "(new Date(1507831097809), new Date(1508522297966))"
                ],
                "reportCount" : [ 
                    "(0.0, inf.0]"
                ],
                "label" : [ 
                    "[MinKey, MaxKey]"
                ]
            },
            "keysExamined" : 533,
            "seeks" : 477,
            "dupsTested" : 0,
            "dupsDropped" : 0,
            "seenInvalidated" : 0
        }
    }
},

"ok" : 1.0
}

查询仍需要 700-800 毫秒才能返回数据。如何更改索引以使查询运行得更快?不考虑"keysExamined" : 533, "seeks" : 477,这个数据。这只是测试数据。

看起来它使用了索引,但只使用了索引中的第一个字段?还multuKey是假的吗?


解释计划输出的几个要点:

  • 该查询涉及以下属性:siteId, status, creationDate, reportCount, assignee, parent
  • The winning plan has two stages:
    • IX_SCAN 使用creationDate_1_reportCount_1_label_1,这使用索引查找creationDate and reportCount识别 56 个文档,然后将其转发到 FETCH 阶段
    • FETCH 从 IX_SCAN 阶段接收 56 个文档,然后询问这些文档以应用siteId, status, assignee and parent过滤器。此次询问导致 37 份文件被丢弃,其中 19 份文件被退回。

因此,您的索引仅涵盖查询中 6 个属性中的 2 个,查询中的其余 4 个属性通过检查文件不是index。如果您希望此查询完全被索引覆盖,则创建以下索引:

db.collection.createIndex(
    {siteId: 1, status: 1, creationDate: 1, reportCount: 1, assignee: 1, parent: 1}
) 

如果您使用此索引重新运行,那么您应该会发现 (a) MongoDB 选择此索引,并且 (b) IX_SCAN 阶段转发的文档数量与 find 调用返回的文档数量相同。

I say “应该找到”因为这里还有其他方面可能导致 MongoDB 选择不同的索引,例如用于$nor和排序阶段(creationDate: 1)。我建议调整索引并在每次调整后运行解释“on”,并在executionStats子文件:

  • “n返回”
  • “已检查的总密钥”
  • “检查的文档总数”

一个简单的经验法则是:距离越近totalKeysExamined is to nReturned并且越接近totalDocsExamined为零……您的指数覆盖范围就越好。

还有索引成本的问题(就对写入时间和索引存储的影响而言),因此我建议考虑您的非功能性需求 - 在没有完整索引覆盖的情况下是否可以实现您期望的运行时间?如果没有,那么您应该继续进行实证测试,但要准备好根据实际情况调整您的选择。explain()输出告诉你。

本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)

带过滤器的 Mongo 日期范围索引 的相关文章

随机推荐