参考:http://www.pixeldonor.com/2013/oct/12/concurrent-zip-compression-java-nio/ http://www.pixeldonor.com/2013/oct/12/concurrent-zip-compression-java-nio/
我正在尝试解压缩 5GB 的压缩文件,平均需要大约 30 分钟,这对于我们的应用程序来说是很多时间,我正在努力减少时间。
我尝试了很多组合,更改了缓冲区大小(默认情况下我的写入块是 4096 字节),更改了 NIO 方法、库,所有结果都非常相同。
仍然没有尝试的一件事是将压缩文件按块分割,因此按多线程块读取它。
片段代码是:
private static ExecutorService e = Executors.newFixedThreadPool(20);
public static void main(String argv[]) {
try {
String selectedZipFile = "/Users/xx/Documents/test123/large.zip";
String selectedDirectory = "/Users/xx/Documents/test2";
long st = System.currentTimeMillis();
unzip(selectedDirectory, selectedZipFile);
System.out.println(System.currentTimeMillis() - st);
} catch (Exception e) {
e.printStackTrace();
}
}
public static void unzip(String targetDir, String zipFilename) {
ZipInputStream archive;
try {
List<ZipEntry> list = new ArrayList<>();
archive = new ZipInputStream(new BufferedInputStream(new FileInputStream(zipFilename)));
ZipEntry entry;
while ((entry = archive.getNextEntry()) != null) {
list.add(entry);
}
for (List<ZipEntry> partition : Lists.partition(list, 1000)) {
e.submit(new Multi(targetDir, partition, archive));
}
} catch (Exception e){
e.printStackTrace();
}
}
可运行的是:
static class Multi implements Runnable {
private List<ZipEntry> partition;
private ZipInputStream zipInputStream;
private String targetDir;
public Multi(String targetDir, List<ZipEntry> partition, ZipInputStream zipInputStream) {
this.partition = partition;
this.zipInputStream = zipInputStream;
this.targetDir = targetDir;
}
@Override
public void run() {
for (ZipEntry entry : partition) {
File entryDestination = new File(targetDir, entry.getName());
if (entry.isDirectory()) {
entryDestination.mkdirs();
} else {
entryDestination.getParentFile().mkdirs();
BufferedOutputStream output = null;
try {
int n;
byte buf[] = new byte[BUFSIZE];
output = new BufferedOutputStream(new FileOutputStream(entryDestination), BUFSIZE);
while ((n = zipInputStream.read(buf, 0, BUFSIZE)) != -1) {
output.write(buf, 0, n);
}
output.flush();
} catch (FileNotFoundException e1) {
e1.printStackTrace();
} catch (IOException e1) {
e1.printStackTrace();
} finally {
try {
output.close();
} catch (IOException e1) {
e1.printStackTrace();
}
}
}
}
}
}
但出于某种原因,它只存储没有文件内容的目录......
我的问题是:关于上述“压缩”文章的方式,在大型 zip 文件上使用多线程制作块的正确方法是什么?