Ubuntu18环境下 python 多进程与多线程测试
测试说明
测试的对象为多次求平方的程序。
def task_powers(limit: int) -> float:
"""return powers of arrays from 1 to limit
Args:
limit (int): max num
Returns:
float: list of powers
"""
return [x**2 for x in range(limit)]
单一线程单一进程测试函数
def test_normal(test_func: FunctionType) -> None:
""" test TEST_TIMES times test_func with args = (TASK_PARAM)
Args:
test_func (FunctionType): test function
"""
# 测试多次运行时间
time_s = time.time()
for i in range(TEST_TIMES):
task_powers(TASK_PARAM)
time_e = time.time()
time_used = time_e - time_s
print("Total time used with normal type: \t", time_used)
多线程下测试函数
def test_multiThreading(test_func: FunctionType) -> None:
""" test TEST_TIMES times test_func with args = TASK_PARAM by multithrading
Args:
test_func (FunctionType): _description_
"""
# 测试多次运行时间
time_s = time.time()
ts = []
for i in range(TEST_TIMES):
ts.append(Thread(target=test_func, args=(TASK_PARAM,)))
ts[i].start()
for i in range(TEST_TIMES):
ts[i].join()
time_e = time.time()
time_used = time_e - time_s
print("Total time used with multithrading type: \t", time_used)
多进程下测试函数
def test_multiProposing(test_func: FunctionType) -> None:
""" test TEST_TIMES times test_func with args = TASK_PARAM by multiprocessing
Args:
test_func (FunctionType): test function
"""
# 测试多次运行时间
time_s = time.time()
ts = []
for i in range(TEST_TIMES):
ts.append(Process(target=test_func, args=(TASK_PARAM,)))
ts[i].start()
for i in range(TEST_TIMES):
ts[i].join()
time_e = time.time()
time_used = time_e - time_s
print("Total time used with multiprocessing type: \t", time_used)
主函数与测试结果
TASK_FUNC = task_powers
TASK_PARAM = 5000000 # 测试用函数的参数
TEST_TIMES = 8
if __name__ == '__main__':
test_normal(TASK_FUNC) # test normal
test_multiThreading(TASK_FUNC) # test multithrading
test_multiProposing(TASK_FUNC) # test multiprocessing
查看程序输出:
Total time used with normal type: 9.709944009780884
Total time used with multithrading type: 9.602959871292114
Total time used with multiprocessing type: 1.7592921257019043
打开Ubuntu的系统监视器看各个CPU与内存的使用情况:
可以发现,单一进程单一线程的情况下会把CPU单核落满;多线程情况下会把多个核心都调度起来,但每个核心的利用率不高,平均每个核心利用率都在12.5%(本次测试开了8个线程,也就是说多个核心加起来的利用率约等于拉满一个核心的效果);最后多进程程序运行时几乎所有核心都拉满了利用率,程序运行速度大幅下降,但并不是只有单进程的1/8,推测多进程设置与调度花费了一定时间。