随着parallel
R 包,我可以像这样并行运行:
library(parallel)
cl <- makeCluster(2) # Create a cluster with 2 workers
... # do some parallel stuff
stopCluster(cl)
但是,那cl
引用集群的变量可能会丢失,例如从失败的函数运行时:
do.something <- function() {
library(parallel)
cl <- makeCluster(detectCores())
parLapply(cl, 1:10, function(x) {
stop("An error occured")
})
stopCluster(cl)
}
do.something()
here, stopCluster
尚未被执行。当这种情况发生时,我只剩下工作人员在运行,如下所示ps
:
501 53300 9225 0 2:16PM ttys003 0:00.27 /opt/local/Library/Frameworks/R.framework/Resources/bin/exec/R
501 53390 1 0 2:19PM ttys003 0:00.16 /opt/local/Library/Frameworks/R.framework/Resources/bin/exec/R --slave --no-restore -e parallel:::.slaveRSOCK() --args MASTER=localhost PORT=11099 OUT=/dev/null TIMEOUT=2592000 XDR=TRUE
501 53399 1 0 2:19PM ttys003 0:00.16 /opt/local/Library/Frameworks/R.framework/Resources/bin/exec/R --slave --no-restore -e parallel:::.slaveRSOCK() --args MASTER=localhost PORT=11099 OUT=/dev/null TIMEOUT=2592000 XDR=TRUE
501 53408 1 0 2:19PM ttys003 0:00.16 /opt/local/Library/Frameworks/R.framework/Resources/bin/exec/R --slave --no-restore -e parallel:::.slaveRSOCK() --args MASTER=localhost PORT=11099 OUT=/dev/null TIMEOUT=2592000 XDR=TRUE
501 53417 1 0 2:19PM ttys003 0:00.16 /opt/local/Library/Frameworks/R.framework/Resources/bin/exec/R --slave --no-restore -e parallel:::.slaveRSOCK() --args MASTER=localhost PORT=11099 OUT=/dev/null TIMEOUT=2592000 XDR=TRUE
当然我可以手动kill
但是有时这可能不切实际,例如 R 的多个实例正在运行自己的池。有没有办法在 R 中阻止它们cl
已经丢失?人们通常如何处理这种情况?
有几种机制可以让代码始终运行,即使出现错误:
try
将容易出错的部分包裹在try
or tryCatch
堵塞。然后您可以检查结果以查看是否存在错误。
do.something <- function() {
library(parallel)
cl <- makeCluster(detectCores())
result <- try({
parLapply(cl, 1:10, function(x) {
stop("An error occured")
})
})
if(inherits(result, "try-error"))
print("there was an error!")
stopCluster(cl)
result
}
on.exit
里面的代码on.exit
当函数结束时,无论是干净地还是由于错误,调用总是会运行。
do.something <- function() {
library(parallel)
cl <- makeCluster(detectCores())
on.exit(stopCluster(cl))
parLapply(cl, 1:10, function(x) {
stop("An error occured")
})
}
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)