reading and writing simple netCDFs

Jun 15, 2011 at 1:08 AM
Edited Jun 15, 2011 at 1:36 AM

First of all, I want to apologize, since unknowingly i posted the same problem in issue tracker. i am sorry for that.

Anyways, I am very new in C# and SDS world. So, i have a simple problem. Can anyone plesae tell me how can i see the dimnentions and variables after reading a netCDF file in C#. I don't want to use command line for this purpose. I am also wondering if I could get some help with example codes of writing a simple netCDF file. 

Thanks in advance.

Jun 15, 2011 at 2:44 PM

Hi,

DataSet consists of Variables; each Variable contains data and metadata; the latter is a dictionary of attributes as pairs name/value. 

To open a DataSet, use DataSet.Open() method; e.g. DataSet.Open("myfile.nc").

Collection of variables is exposed as DataSet.Variable property; dimensions are exposed as DataSet.Dimensions. Description with some examples you may find here. Detailed walkthroughs with recommendations, what you should do prior to use SDS, are available in the Getting Started document.

Sample of creating and writing simple NetCDF:

    float[,] grid = new float[360,720];
    double[] x = new double[360];
    double[] y = new double[720];
    // ... compute grid, x and y values
    DataSet ds = DataSet.Open("filepath.nc?openMode=create");
    ds.Add("grid", grid, "x", "y");
    ds.Add("x", x, "x");
    ds.Add("y", y, "y");
    ds.PutAttr("grid", "units", "m/sec2");

See API reference text here. Also SDS documentation can be found after the SDS installation is complete in the Start / Programs / Scientific DataSet 1.2 / Scientific DataSet Reference.

 

Good luck!

If you have any further questions, please reply.

 

Regards, Dmitry.

 

 

 

Jun 16, 2011 at 6:59 PM

Hi Dmitry,

Thanks for your help on my prevous problem on redaing the simple netCDFs. Here I have some more problems.

I have a simple netcdf file with three variables lat, lon and temp. I know Rank of a variable is seen from the schema: lat and lon depends on two dimensions and temp depends on 3 dimensions lat, lon and time. So when I am reading those file, I can simply do this,

float[, ,] termperature = ds.GetData<float[, ,]>(ds.Variables[0].Name); // Gets all data of temperature

               float[,] latitude = ds.GetData<float[,]>(ds.Variables[1].Name); // Gets all data of latitude

               float[,] longitude = ds.GetData<float[,]>(ds.Variables[2].Name); // Gets all data of latitude

 

But can I read this even without reading the schema. I mean wrong dimensionality sends error messages. Is it possible to have some generic codes that will read the schema and will make the datagrid variables according to its dimension? I am not sure if it is making sense. If not, I can explain more. Please help. Thanks once again.

Jun 16, 2011 at 7:17 PM

It is not clear what exactly you need as an output of this "generic reader". What do you want to do next with it?

 

The following sample gets data of all variables (of any rank and data type) of any input DataSet:

using(DataSet ds = new DataSet.Open("sample.nc"))
{
     foreach (Variable v in ds.Variables)
     {
         Console.WriteLine ("Variable with ID={0} has name={1}, rank {2} and type of data {3}",
           v.ID, v.Name, v.Rank, v.TypeOfData); // prints information about the variable

         Array data = v.GetData(); // gets all data of the variable as an array
        // Somehow handle the data
        ...
     }
}

Jun 16, 2011 at 8:19 PM

My previous codes look something like this:

 

               float[, ,] termperature = ds.GetData<float[, ,]>(ds.Variables[0].Name); // Gets all data of temperature

               float[,] latitude = ds.GetData<float[,]>(ds.Variables[1].Name); // Gets all data of latitude

                float[,] longitude = ds.GetData<float[,]>(ds.Variables[2].Name); // Gets all data of latitude

 

               // write a line of text to the file

               TextWriter tw = new StreamWriter(@"C:\Users\avi\Desktop\ExperiemntData\lat.csv");

               tw.WriteLine("latitude" + "," + "longitude" + "," + "temperature");

 

               //// putting all the values of lat into "tepperature"

               for (int i = 0; i < termperature.GetLength(0); i++)

                   for (int j = 0; j < termperature.GetLength(1); j++)

                       for (int k = 0; k < termperature.GetLength(2); k++)

                       {

                           {

                               tw.WriteLine(latitude[j, k] + "," + longitude[j, k] + "," + termperature[i, j, k]);

                           }

                       }

               tw.Close();

               Console.WriteLine("Your file has been written");

So, i am reading the netcdf, then putting them in temperature, lat and long variables and finally output a csv file where a temperature is associated with a particular a lat and a lon (example below). But I want to do the similar for any netCDF that has same format (or additional date as another variable). And at the end I will put them into a relational SQL database (which is haven’t started yet). So, “generic reader” will read the netCDF and output as a csv file (example below). May be an additional column could be a date. By my netcdf has only one time dimension. So I didn’t need to care about it for this example.

latitude longitude temperature

16.72673 -152.411 297.1223

16.72673 -151.892 297.173

16.72673 -151.373 297.2391 

.............................................

Jun 16, 2011 at 11:59 PM

 

Hi Dmitry,

I was reading the same file in R and just saw that it has also attributes (metadata) about the variable. Could you please tell me how can i see the same metadata using sds in C#? 

Jun 17, 2011 at 6:50 AM

Hi,

About metadata reading. Metadata is a collection of attributes, those are pair of name/value; name is a string, value is an arbitrary simple type (numeric or string), or 1d-array of simple types.

There are global metadata and variable metadata; first is attached to the dataset itself, second are attached to each variable of the dataset.

When you know name of an attribute, the simplest way to get its values is to use GetAttr methods:

// Opens a data set:
var ds = DataSet.Open("test.csv?openMode=open");        
// Gets attribute "range" of type int[] from variable "str":
int[] range = ds.GetAttr<int[]>("str", "range"); 
int min = range[0];
int max = range[1];

 

To find out which attributes are presented, use a MetadataDictionary class available through the Variable.Metadata or DataSet.Metadata properties (the latter represents global metadata).

For example, if you want to print out all global attributes and attribute of first variable, do the following:

foreach(var attr in dataSet.Metadata)
{
   Console.WriteLine("{0} : {1}", attr.Key, attr.Value);
}

Variable v = dataSet[0];
foreach(var attr in v.Metadata)
{
   Console.WriteLine("{0} : {1}", attr.Key, attr.Value);
}

The MetadataDictionary also allows to get/set values by names:

Variable v = dataSet["var"];
v.Metadata["Units"] = "m/s";
. . .
string units = (string)v.Metadata["Units"];

Jun 17, 2011 at 7:00 AM
Stupid_Avi wrote:

...But I want to do the similar for any netCDF that has same format (or additional date as another variable)...

 Still not clear, what means "same format"? Maybe, three or four variables, one of which is 3d and others are 2d? Having data of any numeric type or just float? Is their order fixed or not?