This project is read-only.
sds is a command line program that can perform serveral basic operations on a DataSet. Scientific DataSet installer adds a path to sds to the environment variable PATH, so you can run it from a command line just by typing a command sds.
Structure of a DataSet
sds or sds list commands take an SDS file path or DataSet URI and display a list of variables in the DataSet.

Example
File air.mon.mean.nc can be downloaded (126 MB) from ftp://ftp.cdc.noaa.gov/Datasets/ncep.reanalysis2.derived/pressure/air.mon.mean.nc.
>sds air.mon.mean.nc
[6] air of type Int16 (time:372) (level:17) (lat:73) (lon:144)
[5] time_bnds of type Double (time:372) (nbnds:2)
[4] time of type Double (time:372)
[3] lon of type Single (lon:144)
[2] lat of type Single (lat:73)
[1] level of type Single (level:17)


The tool displays a list of variables in the DataSet opened from the URI. Each variable summary shows the Variable ID, Name, TypeOfData, and shape. Variable shape includes one dimension for vectors, two dimensions for matrices, and so on. Variable summary shows both name and length for each of the dimensions.
Reading metadata
Command sds meta prints metadata of a DataSet. In particular,

sds meta <DataSet URI> prints the global metadata for a DataSet.

sds meta <DataSet URI> <var> ... prints the metadata for individual variables. Each <var> is either a name of a variable or ID.

Example
>sds meta air.mon.mean.nc
                Name = air.mon.mean.nc
         Conventions = CF-1.0
               title = Monthly NCEP/DOE Reanalysis 2
             history = created 2002/03 by Hoop (netCDF2.3)
            comments = Data is from
NCEP/DOE AMIP-II Reanalysis (Reanalysis-2)
(4x/day).  It consists of most variables interpolated to
pressure surfaces from model (sigma) surfaces.
            platform = Model
              source = NCEP/DOE AMIP-II Reanalysis (Reanalysis-2) Model
         institution = National Centers for Environmental Prediction
          references = http://wesley.wwb.noaa.gov/reanalysis2/
http://www.cdc.noaa.gov/cdc/data.reanalysis2.html

>sds meta air.mon.mean.nc air
[6] air of type Int16 (time:372) (level:17) (lat:73) (lon:144)
                Name = air
           long_name = Monthly Air Temperature on Pressure Levels
         valid_range = -32765 -10260
unpacked_valid_range = 137.5 362.5
        actual_range = 179.4077 315.7219
               units = degK
          add_offset = 465.15
        scale_factor = 0.01
       missing_value = 32766
          _FillValue = -32767
           precision = 2
least_significant_digit = 1
             GRIB_id = 11
           GRIB_name = TMP
            var_desc = Air temperature
             dataset = NCEP/DOE AMIP-II Reanalysis (Reanalysis-2) Monthly Averages
          level_desc = Pressure Levels
           statistic = Individual Obs
         parent_stat = Other
       standard_name = air_temperature
        cell_methods = time: mean (monthly from 6-hourly values)

Updating metadata
Command sds update lets you update the metadata for a DataSet:

sds update <DataSet URI> [/g|<var>] [/t:<Type>] <key> <value> [<value> ...]
  • You must specify either the /g flag or <var> after update. The /g flag updates global metadata; <var> is the name or ID of a variable to update.
  • The optional /t flag specifies the type of the attribute. Possible values are the names of any .NET types that the MetadataDictionary supports (see Supported Types). Namespace System can be omited. Examples: Int32, Double, DateTime[]. If the flag is missing, sdsutil tries to infer one of the following types from the actual <value> string: Double, DateTime, Boolean, String, Double[], DateTime[], Boolean[], String[].
  • <key> is a name of an attribute to set.
  • <value> [<value> ...] is a single value or an array of values to set for the attribute.

Example
Sets the attribute "avg" of variable "air" from DataSet "air.nc" equal to 12.5. Inferred type is "Double".
sds update air.nc air avg 12.5

Sets the attribute "descr" of variable "air" from DataSet "air.nc" equal to string "100.0".
sds update air.nc air /t:String descr 100.0

Sets the attribute "range" of variable "air" from DataSet "air.nc" equal to array { -50, 50 }. Inferred type is "Double[]".
sds update air.nc air range -50 50

Sets the global attribute "range" of DataSet "air.nc" equal to array of Int32 { -50, 50 }.
sds update air.nc /g /t:Int32[] range -50 50

Reading data
Command sds data displays the data of a variable or its subset.

sds data <DataSet URI> [ <varstride> ... ], where
<varstride> == <var>[<range>,...]{<cellsize>:<dataformat>}
  • Each <varstride> specifies a variable and its data subset to display. If <varstride> is omitted, sdsutil displays data for every variable in the DataSet.
  • <var> is either a variable name or ID.
  • Optional [<range>,...] element specifies a subset of data. If you omit the [<range>,...], sdsutil displays all data for the variable. The number of <range> elements must be equal to the rank of the variable. Thus, each <range> is start:stop or start:step:stop, where all start,step,stop are indices by the related dimension and also can be omitted. If start is omitted, the subset starts with zero; if step is missing, it is 1; if stop is omitted, the subset ends with the last element in that dimension.
  • Optional {<cellsize>:<dataformat>} defines the ouput format for every data element. Both <cellsize> and <dataformat> are optional. For details about the syntax, seethe MSDN Library page Composite Formatting.

Example
>sds data air.mon.mean.nc level
[1] level of type Single (level:17)
                Name = level
               units = millibar
        actual_range = 1000 10
           long_name = Level
            positive = down
             GRIB_id = 100
           GRIB_name = hPa
                axis = z
  coordinate_defines = point

[0]      1000      925      850      700      600      500      400      300
[8]       250      200      150      100       70       50       30       20
[16]       10

>sds data air.mon.mean.nc level{:E1}
[1] level of type Single (level:17)
                Name = level
               units = millibar
        actual_range = 1000 10
           long_name = Level
            positive = down
             GRIB_id = 100
           GRIB_name = hPa
                axis = z
  coordinate_defines = point

[0]  1.0E+003 9.3E+002 8.5E+002 7.0E+002 6.0E+002 5.0E+002 4.0E+002 3.0E+002
[8]  2.5E+002 2.0E+002 1.5E+002 1.0E+002 7.0E+001 5.0E+001 3.0E+001 2.0E+001
[16] 1.0E+001

>sds data air.mon.mean.nc air[0,1,:10:,:20:]
[6] air of type Int16 (time:372) (level:17) (lat:73) (lon:144)
                Name = air
           long_name = Monthly Air Temperature on Pressure Levels
         valid_range = -32765 -10260
unpacked_valid_range = 137.5 362.5
        actual_range = 179.4077 315.7219
               units = degK
          add_offset = 465.15
        scale_factor = 0.01
       missing_value = 32766
          _FillValue = -32767
           precision = 2
least_significant_digit = 1
             GRIB_id = 11
           GRIB_name = TMP
            var_desc = Air temperature
             dataset = NCEP/DOE AMIP-II Reanalysis (Reanalysis-2) Monthly Averag
es
          level_desc = Pressure Levels
           statistic = Individual Obs
         parent_stat = Other
       standard_name = air_temperature
        cell_methods = time: mean (monthly from 6-hourly values)

[0,1,0,0]         -21660   -21660   -21660   -21660   -21660   -21660   -21660
[0,1,0,140]       -21660

[0,1,10,0]        -19522   -20852   -23004   -21973   -19889   -21282   -20365
[0,1,10,140]      -19773

[0,1,20,0]        -18500   -18964   -19141   -19464   -18560   -19163   -18683
[0,1,20,140]      -18500

[0,1,30,0]        -16681   -17362   -16848   -17181   -17433   -17239   -17335
[0,1,30,140]      -16768

[0,1,40,0]        -17312   -17059   -17065   -16824   -16989   -17404   -16984
[0,1,40,140]      -17427

[0,1,50,0]        -17906   -17604   -18021   -17105   -17743   -17845   -17180
[0,1,50,140]      -17847

[0,1,60,0]        -19466   -19302   -19369   -19257   -19105   -19132   -19026
[0,1,60,140]      -19420

[0,1,70,0]        -19884   -20185   -20376   -20062   -19860   -19730   -19706
[0,1,70,140]      -19832

Copying DataSet
Command sds copy copies DataSet into another DataSet.

sds copy <DataSet URI 1> <DataSet URI 2>
  • First <DataSet URI 1> defines the DataSet to copy, and the second defines the output DataSet.
  • If an output DataSet must be new and it supports "openMode" parameter, specify the extra URI parameter "openMode=create" or "openMode=createNew".
  • Providers for the input and output DataSets can be different. Thus, the command can convert a DataSet from one format to another.
  • If the output DataSet is a NetCDFDataSet, specifiy the extra URI parameter "enableRollback=false" to increase performance.
  • If the input DataSet is too large to fit in memory, sdsutil copies it by parts.

Example
>sds copy air.mon.mean.nc?openMode=open air.mon.mean.csv?openMode=create
Copying . . .

Creating structure and copying global metadata and scalar variables... Done.

Total memory capacity: 200.00 Mb
Deltas for the dimensions adjusted (max iteration capacity: 126.81 Mb):
 Dimension time: 372
 Dimension level: 17
 Dimension lat: 73
 Dimension lon: 144
 Dimension nbnds: 2


Copying data... 100%
(00:01:55.5712749) Done.

Printing information
Command sds info displays DataSet Core version, used by sdsutil, and list of available providers.

>sds info 
DataSet Core version 1.2.5657.0

Provider as: class ASGridDataSet [1.2.5657.0]
Provider csv: class CsvDataSet [1.2.5657.0]
Provider asciigrid: class EsriGridDataSet [1.2.5657.0]
Provider memory: class MemoryDataSet [1.2.5657.0]
Provider memory2: class MemoryDataSet [1.2.5657.0]
Provider nc: class NetCDFDataSet [1.2.5657.0]
Provider wcf: class WcfDataSetFactory [1.2.5657.0]
Provider remoting: class RemotingDataSetFactory [1.2.5657.0]

Extension .csv: CsvDataSet
Extension .tsv: CsvDataSet
Extension .asc: EsriGridDataSet
Extension .nc: NetCDFDataSet

Last edited Apr 11, 2011 at 2:03 PM by dvoits, version 25

Comments

No comments yet.