Exploring netCDF using sds

Jul 11, 2011 at 10:36 PM

Hi, Thanks for your reply. I this I need more help on this matter. The following are my codes to read a netCDF file and the outputs are also given.

 

Codes:

using (sds.DataSet ds = sds.DataSet.Open(DesiredLoc))

           {

               Console.WriteLine("netCDF DataSet " + ds.Name + "(" + ds.DataSetGuid + ")" + " contents: ");

 

               Console.WriteLine(" Variables:");

               foreach (sds.Variable v in ds.Variables)

               {

                   Console.WriteLine(v.ToString());

               }

 

               Console.WriteLine(" dimensions:");

 

               foreach (sds.Dimension v in ds.Dimensions)

               {

                   Console.WriteLine(v.ToString());

               }

               Console.ReadLine();

 

           }

 

Output:

Variables

[7] Lon of type double (y: 277) (x: 349)

[6] Lon of type double (y: 277) (x: 349)

[5] lambert_Conformal of type S byte

[4] x of type double (x: 349)

[3] y of type double (y: 277)

[2] time of type (time: 1)

[1] total_precipitation of type single (time: 1) (y: 277) (x: 349)

 

Dimensions:

(y: 277)

(x: 349)

(time: 1)

 

Now I have the following questions:

What method/codes I should use to visualize the total_precipitation. If possible, I also want to add an USA map on the background.

I want to export this netCDF file in a csv where the three columns will represent lon, lat and total_precipitation respectively. Can I do that using sds? If so, how?


Thanks in advance.

Avi


Developer
Jul 13, 2011 at 11:11 AM

Hi Avi,

(1) Visualization of a DataSet. You can use a WPF control named DataSetViewControl to visualize a dataset in different ways. This control is a part of the full installation package. There are a number of visualization tutorials demonstrating how to use the control. The source codes for the tutorials can be downloaded here; open the solution Main/sln/CodePlexTutorials/CodePlexTutorials.sln

The total_precipitation variable can be visualized as a color map. For this, you should first reduce its dimensionality, since only 2d data can be visualized as a color map. For this, add a computational variable for the total_precipitation variable, that contains the same data but reduced for the given time index; use DataSet.StrideVariable() method (see also Scientific DataSet Reference in the Start/Programs menu after the SDS is installed):

Variable sprec = ds.StrideVariable(ds.Variables["total_precipitation"], 
    new int[,] { 0, 0, 0 }, // "origin": from the beggining of the array
    new int[] { 0, 1, 1 },  // "stride": first dimension is reduced; others - taking every index
    new int[] { 1, 0, 0 }, // "count": first is reduced; others - take all
    "total_precipitation_stridden"); // name of the new stridden variable

After this, the ds contains new "virtual" variable named "total_precipitation_stridden" (it will not be added in the source file). This variable is a 2d-slice of the original total_precipitation for fixed index 0 along the first dimension "time". 

Now, total_precipitation_stridden can be visualized as a 2d-variable.

 

(2) Yes, you can; but you should "linearize" the data manually. The DataSet allows you (i) to read the data from the original dataset; (ii) create an output CSV dataset and (iii) put the properly transformed arrays in the CSV dataset. The following sample shows how it can be done using SDS:

using Microsoft.Research.Science.Data;
using Microsoft.Research.Science.Data.Imperative; /* don't forget to add a reference to this assembly, too */
...

using(DataSet src = DataSet.Open("source.nc"))
using(DataSet dst = DataSet.Open("destination.csv?openMode=create")
{
   double[] lats = src.GetData<double[]>("Lat");
   double[] lons = src.GetData<double[]>("Lon");
   float[,] prec = src.GetData<float[,]>("total_precipitation",
         DataSet.ReduceDim(0), /* removing first dimension from data*/
         DataSet.FromToEnd(0),
         DataSet.FromToEnd(0));

   int n = lats.Length * lons.Length;
   dst.Add<double[]>("lat", "i"); // variable lat depends on i
   dst.Add<double[]>("lon", "i");
   dst.Add<double[]>("total_precipitation", "i");
   // All new variables depend on same single dimensions,
   // i.e. we get a table as an output
  
   for(int ilat = 0; ilat < lats.Length; ilat++)
        for(int ilon = 0; ilon < lons.Length; ilon++)
        {
            dst.Append("lat", lats[ilat, ilon]);
            dst.Append("lon", lons[ilon, ilon]);
            dst.Append("total_precipitation", prec[ilat, ilon]);
        }
}

In fact, the table lat,lon,precipitation can be visualized as a point set using DataSetViewerControl (visualization style "PointSet").

 

I hope this will help,

Dmitry.

 

Jul 13, 2011 at 7:31 PM

Hi Dmitry, 

Thank you so much for your reply. It seems relly helpful. I have few more question/problem with this.

1. while implementing the following code it got error messages.

Variable sprec = ds.StrideVariable(ds.Variables["total_precipitation"], 
    new int[,] { 0, 0, 0 }, // "origin": from the beggining of the array
    new int[] { 0, 1, 1 },  // "stride": first dimension is reduced; others - taking every index
    new int[] { 1, 0, 0 }, // "count": first is reduced; others - take all
    "total_precipitation_stridden"); // name of the new stridden variable

 

error 1-4: for this line

new int[,] {0,0,0}, // "origin": from the beginning of the array

               Invalid rank specifier: expected ',' or ']'    

error 5: The type arguments for method 'Microsoft.Research.Science.Data.DataSet.StrideVariable<DataType>

(Microsoft.Research.Science.Data.Variable<DataType>, int[], int[], int[], string)' cannot be inferred from the usage. Try specifying the type arguments explicitly.

 

actuaaly after getting errors, i also played a little bit with codes using the reference guide. Bt didn't make any improvements. Could you please tell me what is wrong here?

 

 2. here you showed a way to reduce the dimensionality (3-d to 2-d variable). In this case it was easy, because time had only one dimension (time: 1). 

But if had 12 dimensions (one for each month, for example), could i implement the same concept? If so, how?


3. in the given link (visualization style)

i found a interesting idea: http://sds.codeplex.com/wikipage?title=Visualization%20styles&referringTitle=Visual%20hints

example 5. Points style requires three variables: value, x, and y defined on the same dimension. The visualization shows points at x[i],y[i] with color defined by value[i]

I want to implement the same concept here. I want to plot lat and lon and show colors as the value of precipitation (blue for high, red for low). Could you please send me sample codes for this style.

I know i am asking too much silly questions and help. Actually i am very new in C# and don't know much about other languages as well. Please take my apologies. I very much look forward to hearing from you.

Thanks once again for all your help.

Avi

 


Developer
Jul 14, 2011 at 8:19 AM

Hi Avi,

To fix the mistypes, do the following:

1a. Instead of

 

new int[,] {0,0,0}, // "origin": from the beginning of the array

 

use

 

new int[] {0,0,0}, // "origin": from the beginning of the array

 

1b. Instead of

Variable sprec = ds.StrideVariable(ds.Variables["total_precipitation"], 

use

Variable sprec = ds.StrideVariable((Variable<float>)ds.Variables["total_precipitation"], 

 

2. Yes, it is possible. For example, if you need time with index timeIndex, use such code:

int timeIndex = 11; // December
Variable sprec = ds.StrideVariable((Variable<float>)ds.Variables["total_precipitation"], 
    new int[] { timeIndex, 0, 0 }, // "origin"
    new int[] { 0, 1, 1 },  // "stride": first dimension is reduced; others - taking every index
    new int[] { 1, 0, 0 }, // "count": first is reduced; others - take all
    "total_precipitation_stridden"); // name of the new stridden variable
 

 

Even more, for once created stridden variable (i.e. sprec), you at any time can change the reduced index. For example, to make sprec correspond to timeIndex=1 (February), do the following:

sprec.Metadata[StriddenVariableKeys.KeyForOrigin] = new int[] { 1 };

After this line, sprec "contains" data for February.

3. The sample for visualizing data using PointSet visualization is in the Visualization tutorial 4. Download source codes here; open the solution Main/sln/CodePlexTutorials/CodePlexTutorials.sln

 

Good luck,

Dmitry.