How to get precip value

Jun 3, 2011 at 10:26 PM


How do I get the actual precipitation values in the array (it's 2-dimensional, I guess).  Right now, in the csv file, precipitation occupies multiple columns.

Thanks for any help.

Jun 5, 2011 at 6:12 PM


you can get actual values of a variable using DataSet.GetData methods; it is possible to get a single value by its indices or data subset:

double val1d = ds.GetData<double>("amountofprecip", 10, 10); // Gets single value for indices 10,10

double[,] val2d = ds.GetData<double[,,]>("amountofprecip"); // Gets all data.

double[,] val2d_subset = ds.GetData<double[,]>("amountofprecip", 
     DataSet.Range(0, 3, 100), // Takes every third element by 2nd dimension
     DataSet.FromToEnd(100)); // Takes all elements starting with index 100

All available methods are given here.




Jun 6, 2011 at 4:12 PM


Thanks for your reply.  I think what I really need is to create a 2-dimensional table of 813 x 1051 size (latitude for 1051 columns x longitude for 813 columns) and the value of each cell is represented by precipitation.  I think similar to the DataViewer's Table By Values but as a CSV file; and instead of rows/columns numbered sequentially, the values for lat/long are used.

Not sure if I'm making sense.

Jun 6, 2011 at 4:20 PM

You mean that you need a CSV file, containing single 2d-array with precipitation?

If it's true, you just should create new DataSet for an output CSV file and add single 2d-variable into it. The data will be saved exactly as 2d-table with rows and columns.

using(DataSet ds = DataSet.Open("output.csv?openMode=create&appendMetadata=false")
    ds.Add("precip", array2d, "lat", "lon"); // adds new variable named "precip" with data array2d depending on dimensions "lat", "lon"   

In the output file you will see something like this:

precip,, ...
10.2,23.4, ...
32.3,23.3, ...

Or you need something else?

Jun 6, 2011 at 5:52 PM


Sorry... What I need is to print out the values for hrapy, hrapx for each precip value like so:

hrap_xor  hrap_yor     amountofprecip
14           10                -1
15           10                -1
16           10                -1
...            ...                 ...

So, instead of "amountofprecip" showing as 2-dimensional array, it's converted to a single array depending on hrap_xor and hrap_yor. 

It would be a huge file consisting of 813 x 1051 rows (854,463 rows).  Is it possible to do this?

Thanks for your patience.

Jun 10, 2011 at 11:20 AM


Sorry...I've been approaching this project the wrong way.   The goal is to insert the precip values into another csv file that has the latitude and longitude fields. 
The netcdf files do not have the cell by cell latitude and longitude values.  But Advanced Hydrologic Prediction System (AHPS) has a polygon containing the
latitude and longitude corresponding to each precip values.  So I exported the attribute table of this polygon into dbf and converted it into a csv file.  I then
used ArcGIS tool for creating a NetCDF table to pull out the precip values for each hrapx and hrapy cell in each netcdf file.  I again converted this into a csv file.

Then I added a field in both csv files (the lat/long csv and the precip csv) using hrapx and hrapy: hrapXY field, in order to create a unique field that I can use
to insert data from the precip csv into the lat/long csv.

Now, what I needed to do was loop through the precip csv, pull out each precip value, and insert them into the lat/long csv file.  Here is where I got
stuck.  I can't manipulate both csv files inside of Microsoft Access because of column limitations (maximum is about 264; can't do it in SQL Server (maximum is
1024); can't do it in Excel 2010 (because ultimately I'll have to export this into ArcGIS; but ArcGIS does not read .xlsx files, only .xls); but can't do it in Excel 2003
either, because its columns maxes out at 264 and rows maxes out at 65,565.   There were also all other kinds of issues getting the csv files into the various databases.
The next thing is to use in-memory DataTables; but I have thousands of netcdf files to process, and I'm concerned about performance (854,653 rows and about 5 or 6
columns for the lat/long csv file).

So...I'm back at sds.  I've managed to create a csv file containing only the 2-dimensional precip values.  But this does not have hrapx and hrapy values.  Its metadata contains
only the starting number for hrapx (14) and starting number for hrapy (10).  I don't know how to loop through the precip csv file to get data from each cell so I can insert the
values into the lat/long csv file.

It's been a long...long week.   Any assistance would be truly appreciated.


Jun 11, 2011 at 12:21 PM

Update:  Created a csv file containing the lat/long values that sds can now read.  Added a new column named "Precip".  I need to update the values of this column but I think I'll start a new thread.

Cheers to y'all!