|
I have about 60 sds dataset files that I'm reading into separate sds dataset variables that I store in a dictionary (like shown below). All together they are about 100MB in file size (*.nc files).
Public Class DataSetCollectionClass
Inherits Dictionary(Of String, sds.DataSet)
End Class
The first operation I do is go through all the files and add them to the dictionary, something like this
For each f in files
dictionary.add(f,sds.DataSet.Open(buildURI(f))
Next
At this point, the memory use is minimal. Then I go through the files and call GetData for each of the 8 arrays. Each file contains about 8 1D arrays of 750 points. This causes the memory use to increase dramatically (about 4 GB in this
case). After I have transitioned the data from the dataset to my program I would like to somehow drop the memory. The only way I have found to do this is to purge my dictionary object of the datasets and recreate them. So my questions are:
1. Is there a way to release memory from the dataset variable after using GetData? This way I don't have to purge my dictionary and recreate it.
2. Any reason why the dataset is using so much memory? Anyway to make the memory use more efficient?
|