Optimize data-writing

Aug 16, 2011 at 4:09 PM

I am expecting the new version of SDS come to true. However, it could make sds better if u can consider about the data writing performance.

I find the append method could be the main reason of decreasing performance. I would like to have high-speed writing, then I turned off the auto-commit, expecting that commit could execute very fast, while sds checks every change made by append and commit change to netcdf file one by one, which affects writing performance.

Normally, we need only one unlimited dimension for one variable. When I do append, the data should be cached in someplace as array and all data is directly  written to netcdf file when I do commit. If only Sds can open a interface to turn off 'save change' behavior when I have a variable including only one unlimited dimension.