使用 Selenium Firefox Webdriver 在 Python3 中的第一次请求之前添加 cookie

我有一个 PKI 身份验证网站，因此打开页面时的第一件事就是要求输入 PIN 码，这样我就无法在加载页面之前设置已保存的 cookie。

我见过 chrome 驱动程序的类似解决方案here https://stackoverflow.com/questions/63220248/how-to-preload-cookies-before-first-request-with-python3-selenium-chrome-webdri但 Firefox Webdriver 对象没有execute_cdp_cmd。另外，当尝试仅加载 cookie 而不加载页面，并且根本不尝试使用execute_cdp_cmd 时，我收到以下错误，正如预期的那样，只是引用了 chrome 而不是 firefox。

selenium.common.exceptions.InvalidCookieDomainException: Message: Document is cookie-averse
Stacktrace:
WebDriverError@chrome://remote/content/shared/webdriver/Errors.jsm:186:5
InvalidCookieDomainError@chrome://remote/content/shared/webdriver/Errors.jsm:326:5
GeckoDriver.prototype.addCookie@chrome://remote/content/marionette/driver.js:1931:11

我的代码如下：

import time
import os
import pickle
from selenium import webdriver
from selenium.webdriver.firefox.options import Options

driver = webdriver.Firefox("C:\\Users\\myuser\\PycharmProjects\\myproject\\")
cookie_path = "cookies.pkl"


def get_page():
    # driver.get(f"https://mypkiaythenticationpage.com")

    if not os.path.isfile(cookie_path):
        print("Cookie File not found, generating new one")
        pickle.dump(driver.get_cookies(), open("cookies.pkl","wb"))
    else:
        # driver.delete_all_cookies()
        # driver.execute_cdp_cmd('Network.enable', {})
        cookies = pickle.load(open("cookies.pkl", "rb"))
        for cookie in cookies:
            driver.add_cookie(cookie)
        print("Loading cookie ...")
    driver.get(f"https://mypkiaythenticationpage.com")

def save_last_cookie_state():
    print("Saving last cookie state ...")
    pickle.dump(driver.get_cookies(), open("cookies.pkl", "wb"))


#
if __name__ == '__main__':
    get_page()
    driver.get(f"https://mypkiaythenticationpage.com")
    save_last_cookie_state()

这可以通过cookies.sqlite数据库 http://fileformats.archiveteam.org/wiki/Firefox_cookie_database在分配给的配置文件中webdriver. cookies.sqlite可以使用以下命令从 Selenium cookie 生成：

from collections.abc import Sequence
from contextlib import closing
from typing import Any, Mapping
import os
import pickle
import sqlite3

# Generated via `sqlite3 -readonly path/to/fxprofile/cookies.sqlite ".schema"`.
# See:
# - http://fileformats.archiveteam.org/wiki/Firefox_cookie_database
# - https://kb.mozillazine.org/Profile_folder_-_Firefox#Finding_the_profile_folderSQL_STATEMENT_CREATE_TABLE
SQL_STATEMENT_CREATE_TABLE = r"""
CREATE TABLE
moz_cookies (
    id INTEGER PRIMARY KEY,
    originAttributes TEXT NOT NULL DEFAULT '',
    name TEXT,
    value TEXT,
    host TEXT,
    path TEXT,
    expiry INTEGER,
    lastAccessed INTEGER,
    creationTime INTEGER,
    isSecure INTEGER,
    isHttpOnly INTEGER,
    inBrowserElement INTEGER DEFAULT 0,
    sameSite INTEGER DEFAULT 0,
    rawSameSite INTEGER DEFAULT 0,
    schemeMap INTEGER DEFAULT 0,
    CONSTRAINT moz_uniqueid UNIQUE (name, host, path, originAttributes)
)
"""

SQL_STATEMENT_INSERT_COOKIE = r"""
INSERT INTO
  moz_cookies (
    name,
    value,
    host,
    path,
    expiry,
    lastAccessed,
    creationTime,
    isSecure,
    isHttpOnly
  )
VALUES (
    :name,
    :value,
    :domain,
    :path,
    :expiry,
    :access_time,
    :creation_time,
    :secure,
    :httponly
  );
"""


def generate_cookies_db(
    cookies: Sequence[Mapping[str, Any]],
    access_time: int,
    creation_time: int,
    db_path: str | os.PathLike,
) -> None:
    with closing(sqlite3.connect(db_path)) as connection:
        with closing(connection.cursor()) as cursor:
            cursor.execute(SQL_STATEMENT_CREATE_TABLE)
            for cookie in cookies:
                values = {
                    "access_time": access_time,
                    "creation_time": creation_time,
                    "domain": cookie.get("domain"),
                    "expiry": cookie.get("expiry"),
                    "httponly": cookie.get("httpOnly", False),
                    "name": cookie["name"],
                    "path": cookie.get("path"),
                    "secure": cookie.get("secure", False),
                    "value": cookie["value"],
                }
                cursor.execute(SQL_STATEMENT_INSERT_COOKIE, values)
        connection.commit()


def main():
    cookies_path = "cookies.pkl"
    with open("cookies.pkl", "rb") as fp:
        cookies = pickle.load(fp)
        access_time_us = int(os.path.getmtime(cookies_path) * 1_000_000)
        creation_time_us = int(os.path.getctime(cookies_path) * 1_000_000)
        generate_cookies_db(
            cookies, access_time_us, creation_time_us, "./cookies.sqlite"
        )


if __name__ == "__main__":
    main()

或者，您可以从现有的个人资料 https://kb.mozillazine.org/Profile_folder_-_Firefox#Finding_the_profile_folder直接与sqlite3:

$ sqlite3 -readonly /path/to/fxprofile/cookies.sqlite ".mode insert moz_cookies" ".schema" "SELECT * FROM moz_cookies WHERE host LIKE '%.example.com'" | sqlite3 ./cookies.sqlite

注意cookies.sqliteFirefox 运行时将被锁定。

然后复制生成的cookies.sqlite到使用的配置文件webdriver:

$ mkdir ./profile_template
$ mv ./cookies.sqlite ./profile_template

from selenium import webdriver
from selenium.webdriver.firefox.options import Options

options = Options()
options.profile = webdriver.FirefoxProfile("./profiletemplate")

with webdriver.Firefox(options=options) as session:
    session.get("https://example.com")

该目录将被 Selenium 视为只读，因为webdriver.FirefoxProfile没有坚持下去 https://selenium-python.readthedocs.io/api.html#module-selenium.webdriver.firefox.firefox_profile.