I am using the NI DLLs to save as TDM files, and my main issue is that it basically just writes the raw data on the disk, without any sort of compression.
My code:
struct ScopeData {
std::vector<__int16> channel1;
std::vector<__int16> channel2;
int byteDepth;
double maxMVCh1;
double maxMVCh2;
double maxMVCh3;
double maxMVCh4;
};
void WriteNIFile(Scope::ScopeData* sdDataNI,
std::string fileName,
std::string filePath) {
int ddcError = 0;
DDCFileHandle file = 0;
DDCChannelGroupHandle group;
DDCChannelHandle channels[2];
if (!DirectoryExists(filePath.c_str())) {
CreateDirectory(filePath.c_str(), NULL);
}
std::string saveFile = filePath + "\\" + fileName;
ddcError = DDC_CreateFile(saveFile.c_str(),
"TDM",
"OphthaMetrics data",
"OphthaMetrics data acquisition",
"",
"",&file);
ddcError = DDC_AddChannelGroup (file, "RAW Data",
"Raw scope data from two channels", &group);
ddcError = DDC_AddChannel (group, DDC_Int16, "Channel 1",
"Transducer to preamp", "Millivolts", &channels[0]);
ddcError = DDC_AddChannel (group, DDC_Int16, "Channel 2",
"Transducer to preamp", "Millivolts", &channels[1]);
ddcError = DDC_SetChannelProperty(channels[0],
DDC_CHANNEL_MAXIMUM, sdDataNI->maxMVCh1);
ddcError = DDC_SetChannelProperty(channels[0],
DDC_CHANNEL_MINIMUM, -sdDataNI->maxMVCh1);
ddcError = DDC_SetChannelProperty(channels[1],
DDC_CHANNEL_MAXIMUM, sdDataNI->maxMVCh2);
ddcError = DDC_SetChannelProperty(channels[1],
DDC_CHANNEL_MINIMUM, -sdDataNI->maxMVCh2);
ddcError = DDC_SetDataValues(channels[0], &sdDataNI->channel1[0],
sdDataNI->channel1.size());
ddcError = DDC_SetDataValues(channels[1], &sdDataNI->channel2[0],
sdDataNI->channel2.size());
ddcError = DDC_SaveFile(file);
ddcError = DDC_CloseFile(file);
return;
}Only channel 1 and 2 are being used in our tests, which is why only 2 are saved. The DLL saves them just subsequently I guess, and this results in 500 MB files (each channel is around 136 million 16-bit unsigned integers, resulting in a combined 500 MB). When I use zip on the files, it will decrease drastically in size (from 511 MB to 144 MB - which is expected, only 12 bits are set, the dataset is a lot of values ranging from -100 to +100, and only occassionally going for outside that range, etc).
How can I decrease the data streams in size, but still have the files usable in LabView or similar tools? Can I use zip compression on the data before saving it?
