@CrisLuengo 的想法要快得多:本质上,对数据进行分块,加载每个块,然后将其拆分为单独的通道文件以节省 RAM。
下面是一些加载部分的代码,速度很快,不到 1 分钟:
% fake raw data
disp('building... ');
nChan = 256;
nSampsTotal = 10e6;
tic; DATA = rand(nChan,nSampsTotal); toc;
fid = fopen('rawData.dat','w');
disp('writing flat binary file... ');
tic; fwrite(fid,DATA(:),'int16'); toc;
fclose(fid);
% compute the number of samples and chunks
chunkSize = 1e6;
nChunksTotal = ceil(nSampsTotal/chunkSize);
%% load by chunks
t1 = tic;
fid = fopen('rawData.dat','r');
dat = zeros(nChan,chunkSize,'int16');
chunkCnt = 1;
while 1
tic
if chunkCnt <= nChunksTotal
% load the data
fprintf('Chunk %02d/%02d: loading... ',chunkCnt,nChunksTotal);
dat = fread(fid,[nChan,chunkSize],'*int16');
else
break;
end
toc;
chunkCnt = chunkCnt + 1;
end
t = toc(t1); fprintf('Total time: %4.2f secs.\n\n\n',t);
% Total time: 55.07 secs.
fclose(fid);
另一方面,通过跳过文件按通道加载大约需要 20 倍的时间,略多于 20 分钟:
%% load by channels (slow)
t1 = tic;
fid = fopen('rawData.dat','r');
dat = zeros(1,nSampsTotal);
for i = 1:nChan
tic;
fprintf('Channel %03d/%03d: loading... ');
offset = i-1;
fseek(fid,offset*2,'bof');
dat = fread(fid,[1,nSampsTotal],'*int16',(nChan-1)*2);
toc;
end
t = toc(t1); fprintf('Total time: %4.2f secs.\n\n\n',t);
% Total time: 1133.48 secs.
fclose(fid);
我还要感谢 Matlab 论坛上的 OCDER 提供的帮助:link https://www.mathworks.com/matlabcentral/answers/415545-loading-large-binary-files-in-matlab-quickly