Requested variable does not exist in the data set

Aug 28, 2012 at 3:48 PM

Hello, I am trying to output some arrays to CSV. It is failing on the last line (see comments). Does anyone know what is going on?

using (DataSet ws = DataSet.Open(outputDir +  "\\" + outputFilename, ResourceOpenMode.Create))
{
	
	//add the days field
	ws.Add<double[]>("DateTime");
	
	//THIS WORKS..fails later when I try again (see below)
	ws.PutData<double[]>("DateTime", days); //days was grabbed from the first fileName earlier. It is the same across all fileNames.

	foreach (string fileName in fileNames)
	{
		using (DataSet nc = DataSet.Open(fileName, ResourceOpenMode.ReadOnly))
		{
			//we need IDs to pull the appropriate variables
			//from the file. 'Name' do not work for some reason.
			//Always says does not exist when trying to use GetData with
			//the variable name in my .nc file.
			int iD = GetVariableID(nc, variable);

			foreach (int[] coord in coords)
			{
				//get an array of data for this variable at this coordinate
				float[] variableData = nc.GetData<float[]>(iD, DataSet.FromToEnd(0), 
					DataSet.ReduceDim(coord[0]), DataSet.ReduceDim(coord[1]));

				double[] daysCompare;//this is actually not used...for testing.
				float[] dailyValues;

				if (variable == "tas")
				{
					//convert variableData to Celsius
					ConvertArrayToCelsius(variableData);
					ComputeDailyAverages(times, variableData, out daysCompare, out dailyValues);
				}
				else
				{
					//convert from kg/m^2/s
					ConvertArrayToHundredthsOfAnInch(variableData);
					ComputeDailySums(times, variableData, out daysCompare, out dailyValues);
				}

				string fieldName = variable + "_" + coord[0].ToString() + "_" + coord[1].ToString();
				
				
				ws.Add<float[]>(fieldName); //works, but has asterisk next to it in debugger? 
				
				//THIS FAILS with 'Requested variable does not exist in the data set'
				ws.PutData<float[]>(fieldName, dailyValues);
				
			}
		}
	}
}
Aug 30, 2012 at 10:12 PM
Edited Aug 30, 2012 at 10:13 PM

OK, so I changed the code to

ws.AddVariable(typeof(System.Single), fieldName, variableData, new string[1] { "x" });
...instead of ws.Add(). This seems to work. However, in the following code, the added float variables both have the same data after they are added to the dataset:
using (DataSet ws = DataSet.Open(outputFilePath, ResourceOpenMode.Create))
{
	//add the days field.
	ws.Add<double[]>("DateTime");
	//populate days
	ws.PutData<double[]>("DateTime", times);

	string fieldName = variable + "_" + coord[0].ToString() + "_" + coord[1].ToString();

	ws.AddVariable(typeof(System.Single), fieldName, variableData, new string[1] { "x" });
	ws.PutData<float[]>(fieldName, variableData); //at this point data is different than what is finally in the output file
    
	//rolling my own array copy to make sure I am not crazy
	float[] convVariableData = new float[variableData.Length];
	for (int i = 0; i < convVariableData.Length; i++)
	{
		convVariableData[i] = variableData[i];
	}

	if (variable == "tas")
	{
		//convert variableData to Celsius
		ConvertArrayToCelsius(convVariableData);
	}
	else
	{
		//convert from kg/m^2/s
		ConvertArrayToHundredthsOfAnInch(convVariableData);
	}

	//convVariableData is different than variableData in debugger at this point
	
	
	ws.AddVariable(typeof(System.Single), fieldName + "_conv", convVariableData, new string[1] { "y" }); 
	ws.PutData<float[]>(fieldName, convVariableData);
	
	//program ends here. Fields should be <times>,<variableData>,<convVariableData>. 
	//Instead it is <times>,<convVariableData>,<convVariableData>

}
I suspect there is some issue with adding data that I am not aware of. I could just write my own CSV's, but this way would be much easier and more concise if I use SDS.
Sep 3, 2012 at 12:51 PM

Hi,

The first question reply:

Here is a pseudo code of your example:

 

ws.Add("DateTime"); 
// ws has variable "DateTime" depending on a default dimension, say, "i", with length "0"

ws.PutData("DateTime", days);
// the variable "DateTime" now contains data, 
// length of the dimension "i" is equal to length of "days" (say, n elements)


...


ws.Add(fieldName); // let fieldName = "x"
// ws has new variable "x" depending on "i" (i.e. default dimension) as "DateTime",
// so both "x" and "DateTime" MUST have same length (n) for the dataset ws to be committed (i.e. to become consistent and be saved)
// The asterisk '*' means that the variable is not committed yet.


ws.PutData(fieldName, data);
// this method doesn't find the variable, since it wasn't committed yet and cannot be found by name.
// There 2 simple ways to put the data:
// (1) Use overload of Add which accepts data as its argument (length of data must be equal n) and remove PutData
// (2) Get ID of the added variable and then pass it to PutData instead of name:
// int id = ws.Add(fieldName).ID;
// ws.PutData(id, data)

 

The second question reply:

You have a mistype: second ws.PutData puts data into the same variable as first ("fieldName" instead of "fieldName + "_conv"").

 

 

Regards, Dmitry.

 

Sep 4, 2012 at 4:54 PM

Ah, it now makes sense. Also, can't believe I let that bug through in my second question...thanks for pointing that out :)