将文件上传到 Google Cloud Storage：500 后端错误

我们尝试将文件上传到 Google Cloud Storage，然后再将其移至 BigQuery，但在某些上传过程中，我们经常遇到“500 内部服务器错误”或“410 消失”（原始消息如下）。

我们使用的是官方 SDK，并添加了指数退避重试，但错误始终存在。请问您有什么建议吗？

这是我们上传的方式（scala）：

val credential = new GoogleCredential().setAccessToken(accessToken)

val requestInitializer = new HttpRequestInitializer() {
    def initialize(request: HttpRequest): Unit = {
        credential.initialize(request)

        // to avoid read timed out exception
        request.setConnectTimeout(200000)
        request.setReadTimeout(200000)

        request.setIOExceptionHandler(new 
            HttpBackOffIOExceptionHandler(new ExponentialBackOff()))
        request.setUnsuccessfulResponseHandler(new 
            HttpBackOffUnsuccessfulResponseHandler(new ExponentialBackOff()))
    }
}

val storage = new Storage.Builder(
    new NetHttpTransport,
    JacksonFactory.getDefaultInstance,
    requestInitializer
).setApplicationName("MyAppHere").build

val objectMetadata = new StorageObject()
    .setBucket(bucketName)
    .setName(distantFileName)

val isc = new InputStreamContent("binary/octet-stream", fis)
val length = isc.getLength
val insertObject = storage.objects().insert(bucketName, objectMetadata, isc)
// For small files, you may wish to call setDirectUploadEnabled(true), to
// reduce the number of HTTP requests made to the server.
if (length > 0 && length <= 2 * 1000 * 1000 /* 2MB */ ) {
    insertObject.getMediaHttpUploader.setDirectUploadEnabled(true)
}
insertObject.execute()

我们的 scala 依赖项：

"com.google.api-client" % "google-api-client" % "1.18.0-rc",
"com.google.api-client" % "google-api-client-jackson2" % "1.18.0-rc",
"com.google.apis" % "google-api-services-bigquery" % "v2-rev142-1.18.0-rc",
"com.google.apis" % "google-api-services-storage" % "v1-rev1-1.18.0-rc",
"com.google.http-client" % "google-http-client" % "1.18.0-rc",
"com.google.oauth-client" % "google-oauth-client" % "1.18.0-rc"

原始 SDK 错误响应：

500 Internal Server Error
    {
        "code" : 500,
        "errors" : [ {
        "domain" : "global",
        "message" : "Backend Error",
        "reason" : "backendError"
    } ],
        "message" : "Backend Error"
    }

410 Gone
    {
        "code" : 500,
        "errors" : [ {
        "domain" : "global",
        "message" : "Backend Error",
        "reason" : "backendError"
    } ],
        "message" : "Backend Error"
    }

Every Backend error应通过指数重试来处理，因为可能会出现服务问题。

如果 10 小时后错误仍然存​​在，那么您应该联系支持人员，以便为您的问题提供 1:1 帮助。