我正在编写一个多线程Python程序,其中主线程和它生成的其他线程作为守护进程运行(但不是Thread.daemon=True),它们在某些目录中查找某些文件,并在它们存在时对其执行操作。一个/任何线程中可能会发生错误,这将需要整个程序退出。但是,我需要其他线程在退出之前完成当前的工作。
据我了解,如果我为生成的线程设置 myThread.daemon=True ,它们将在主线程退出时立即自动退出。但是,我希望其他线程在退出之前完成当前的工作(除非错误是某种灾难性故障,在这种情况下,我可能会退出所有内容,无论安全与否)。因此,我没有将线程的 daemon 属性设置为 True。
查看线程模块文档和可用的各种对象(例如事件、信号量、条件和锁),我不确定处理我的情况的最佳方法。此外,当程序因 SIGTERM/SIGINT 信号而需要终止时,我不确定如何处理这种情况。
一些代码说明了我的程序结构的简化版本:
import threading
import signals
import glob
import time
class MyThread1( threading.thread ):
def __init__( self, name='MyThread1' ):
threading.Thread.__init__( self )
self.name = name
return
def run( self ):
while True:
filePathList = glob.glob( thisThreadDir + '/*.txt' )
for file in filePathList:
try:
doSomeProcessing( file )
# Then move the file to another thread's dir
# or potentially create a new file that will
# be picked up by another thread
except:
# Need to potentially tell all other threads
# to finish task and exit depending on error
# I assume this would be the place to check for some kind of
# flag or other indication to terminate the thread?
time.sleep( 30 )
# Now imagine a few more custom threads with the same basic structure,
# except that what is happening in doSomeProcessing() will obviously vary
# Main Thread/Script
def sigintHandler( SIGINT, frame ):
# How do I handle telling all threads to finish their current loop
# and then exit safely when I encounter this signal?
sys.exit( 1 )
def sigtermHandler( SIGTERM, frame ):
# Same question for this signal handler
sys.exit( 1 )
signal.signal( signal.SIGINT, sigintHandler )
signal.signal( signal.SIGTERM, sigtermHandler )
myOtherThread1 = MyThread1()
myOtherThreadN = MyThreadN()
myOtherThread1.start()
myOtherThreadN.start()
while True:
filePathList = glob.glob( mainDir + '/*.txt' )
for file in filePathList:
try:
doMainProcessing( file )
# Move file or write a new one in another thread's dir
except:
# Again, potentially need to exit the whole program, but want
# the other threads to finish their current loop first
# Check if another thread told us we need to exit?
time.sleep( 30 )
我会用一个Event
向线程发出信号表明它应该退出:
- 创建一个事件
__init__
- 使用事件的
wait()
in run()
for sleep
并检查何时退出
- 从外部设置事件以停止线程
处理线程内的异常,我会有一个try
/ except
阻止它所做的一切。当捕获到某些内容时,存储异常(和/或您需要的任何其他信息),清理并退出线程。
在外部,在主线程中,检查所有线程中的存储异常,如果发现任何异常,则向所有线程发出信号,告知它们应该退出。
在主线程中处理异常(其中还包括SIGINT
), 有一个try
/except
阻塞在那里并向所有线程发出停止信号。
全部在一起,带有虚拟异常和调试打印:
import threading
import time
class MyThread(threading.Thread):
def __init__(self):
super().__init__()
self.stop_requested = threading.Event()
self.exception = None
def run(self):
try:
# sleep for 1 second, or until stop is requested, stop if stop is requested
while not self.stop_requested.wait(1):
# do your thread thing here
print('in thread {}'.format(self))
# simulate a random exception:
import random
if random.randint(0, 50) == 42:
1 / 0
except Exception as e:
self.exception = e
# clean up here
print('clean up thread {}'.format(self))
def stop(self):
# set the event to signal stop
self.stop_requested.set()
# create and start some threads
threads = [MyThread(), MyThread(), MyThread(), MyThread()]
for t in threads:
t.start()
# main thread looks at the status of all threads
try:
while True:
for t in threads:
if t.exception:
# there was an error in a thread - raise it in main thread too
# this will stop the loop
raise t.exception
time.sleep(0.2)
except Exception as e:
# handle exceptions any way you like, or don't
# This includes exceptions in main thread as well as those in other threads
# (because of "raise t.exception" above)
print(e)
finally:
print('clan up everything')
for t in threads:
# threads will know how to clean up when stopped
t.stop()
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)