# Time Series Analysis in C#.NET Part I

By , 8 May 2012

## Introduction

These days my company at TCW is focused on developing linear and nonlinear time series applications in .NET languages for middleware applications for neuroscience research. Specifically, I am building hybrid applications in the R statistical and MATLAB languages for prototyping and then translating to C#.NET and/or C++.NET to analyze and predict fMRI/EEG data from BCI gaming applications.

This is the first of a five part series that examines the use of open source libraries to build .NET based time series applications that can be used for both neuroscientific research and commodity/stock price forecasting. The goal is to make available open source nonlinear time series algorithms as part of a statistical engine, i.e. TIME JAQUAR, for both the time and frequency domain that can run on multi-CPU/multi-GPU platforms for handling real-time modeling and prediction from BCI devices.  All code and releases from this series of articles can be found on CodePlex at modelmaker3.codeplex.com.

Because there are many good tutorials on the web for doing time series analysis along with what I have written, I refrain from doing that here.

## Background

There are several good libraries to use as both namespaces and prototyping templates to help prototype linear and nonlinear time series applications in .NET. Here are a couple that are used in my applications:

Library 1: Cronos
Library 2: MathNet

Please consult the documentation on these libraries for more information. Of course, the scalability of the linear algebra and optimization methods are the foundation stones and the subject of additional articles. I have provide the basics in Figure 1 to illustrate some of the features of both libraries and posting the code on CodePlex so that you can follow the project.

Currently, I am modifying these as well for GPU performance.

## Using the code

The following steps are to be performed for doing the estimation:

Step 1: Specify the parameters of the ARMA Model
Step 2: Generate the data for the simulation
Step 3: Fit the model by using an estimation method like Maximum Likelihood Estimation (MLE)
Step 4: Obtain the Model Data, Predictions, and Residual Output

Figure 1. Code listing for steps 1-4

```using ABMath.ModelFramework.Models;
using ABMath.ModelFramework.Data;
using MathNet.Numerics.Distributions;
using MathNet.Numerics.RandomSources;
using MathNet.Numerics;
namespace TCW_ModelMaker3
{
class Program
{
static void Main(string[] args)
{
runARMATest();
}
public static void ConsoleWrite()
{
Console.WriteLine("Welcome to ModelMaker 3");
}
public static void runARMATest()
{
//Specify the AR and MA paramaters
ARMAModel model1 = new ARMAModel(4, 3);
ARMAModel model2 = new ARMAModel(0, 1);
//Select an random distribution for the data
var sd = new StandardDistribution();
//Make new time series variable
TimeSeries ts1 = new TimeSeries();
var dt = new DateTime(2001, 1, 1);
var  normalSim = new StandardDistribution();
var current = new DateTime(2001, 1, 1);
//Create the data
for (int t = 0; t < 100; ++t)
{
double s1 = normalSim.NextDouble();
}

model1.SetInput(0, ts1, null);
//Maximum Likelihood Estimation
model1.FitByMLE(10, 10, 10,null);
//Compute the residuals
model1.ComputeResidualsAndOutputs();
//Get the predicted values
var preds = model1.GetOutput(3) as TimeSeries;
var predName = model1.GetOutputName(3);
//Write the model description with parameter values
Console.WriteLine(model1.Description);
}
}
}```

The most interesting method above is the the `FitByMLE` method. This function samples from the parameter space with a Halton sequence and obtains the best model by optimizing the log-likelihood. Parameters values have three states: `ParameterState.Locked`, `ParameterState.Free`, or `ParameterState.Consequential`. The locked parameters are held at their current values in optimization method, free parameters are optimized, and consequential parameters are computed as a function of other parameters and the data.  See the library on Cronos for more information.

Figure 2. Code listing for the Maximum Likelihood Method
```public virtual void FitByMLE(int numIterationsLDS, int numIterationsOpt,
double consistencyPenalty,
Optimizer.OptimizationCallback optCallback)
{
thisAsMLEEstimable = this as IMLEEstimable;
if (thisAsMLEEstimable == null)
throw new ApplicationException("MLE not supported for this model.");

int optDimension = NumParametersOfType(ParameterState.Free);
int numConsequential = NumParametersOfType(ParameterState.Consequential);
int numIterations = numIterationsLDS + numIterationsOpt;

var trialParameterList = new Vector[numIterationsLDS];
var trialCubeList = new Vector[numIterationsLDS];

var hsequence = new HaltonSequence(optDimension);

for (int i = 0; i < numIterationsLDS; ++i)
{
Vector smallCube = hsequence.GetNext();
Vector cube = CubeInsert(smallCube);
trialParameterList[i] = thisAsMLEEstimable.CubeToParameter(cube);
trialCubeList[i] = cube;
}

var logLikes = new double[numIterationsLDS];
for (int i = 0; i < numIterationsLDS; ++i)
{
Vector tparms = trialParameterList[i];
if (numConsequential > 0)
{
tparms = ComputeConsequentialParameters(tparms);
trialParameterList[i] = tparms;
}

double ll = LogLikelihood(tparms, consistencyPenalty);
logLikes[i] = ll;

if (optCallback != null)
lock (logLikes)
optCallback(tparms, ll, i*100/numIterations, false);
}

// Step 1: Just take the best value.
Array.Sort(logLikes, trialParameterList);
Parameters = trialParameterList[numIterationsLDS - 1];

// Step 2: Take some of the top values and use them to create a simplex, then optimize
// further in natural parameter space with the Nelder Mead algorithm.
// Here we optimize in cube space, reflecting the cube when necessary to make parameters valid.
var simplex = new List<Vector>();
for (int i = 0; i <= optDimension; ++i)
trialParameterList[numIterationsLDS - 1 - i])));
var nmOptimizer = new NelderMead {Callback = optCallback, StartIteration = numIterationsLDS};
currentPenalty = consistencyPenalty;
nmOptimizer.Minimize(NegativeLogLikelihood, simplex, numIterationsOpt);
Parameters = ComputeConsequentialParameters(
thisAsMLEEstimable.CubeToParameter(CubeFix(CubeInsert(nmOptimizer.ArgMin))));

ComputeResidualsAndOutputs();
}```

Of course, this is the main algorithm of interest based on Nelder Mead and the subject of a future article. However, this should be enough to get you started.

## Points of Interest

Fragments of the code in Figure 1. can be placed in the form of a snippet into Visual Studio 2010 using a Snipped Editor. Visit the Workshop to see the video for this.

## History

• 5-8-2012: Code placed on CodePlex.

 Jeff B. Cromwell CEO The Cromwell Workshop United States Member
Dr. Jeff B. Cromwell is the CEO/Neuroeconomist at The Cromwell Workshop. Please visit www.neuronalarchitects.com for more information or his profile on LinkedIn for networking.

Votes of 3 or less require a comment

 Search this forum Profile popups    Spacing RelaxedCompactTight   Noise Very HighHighMediumLowVery Low   Layout Open AllThread ViewNo JavascriptPreview   Per page 102550
 First Prev Next
 Code Not available.....Link is not working.. AVI - The Gr8 30 Nov '12 - 7:17
 Re: Code Not available.....Link is not working.. Jeff B. Cromwell 30 Nov '12 - 8:13
 My vote of 5 Kanasz Robert 6 Nov '12 - 0:05
 Re: My vote of 5 Jeff B. Cromwell 30 Nov '12 - 8:16
 Five and question Jerome Vibert 18 May '12 - 23:00
 You've got my five. Statistical C# project are so rare on this site ! I've downloaded you're project. Everything is compiling fine except this line of code : `model1.ComputeResidualsAndOutputs();` This method does not exists. Could you explain what you usually put in this method ?   Thank you for sharing. I'm waiting for the next part ! Sign In·View Thread·Permalink
 Re: Five and question Jeff B. Cromwell 21 May '12 - 4:01
 Thanks for the vote and pumping me up for the next article that I am working on today. My ABMath.ModelFramework.Models.ARMAModel namespace has the method in the Runtime Version v4.0.30319. If you break on the line after that method look at the values for the model1.Residuals. You can see 100 values in the value field of base and you can use this for model diagnostics and testing. Hope this helps-I will add your question to the book. All the best-Jeff Sign In·View Thread·Permalink
 Re: Five and question Teodor Bildea 12 Jun '12 - 2:20
 Hello,   I am trying to use the code. While the ComputeResidualsAndOutputs method is missing, you mentioned it is available in the runtime version. To break and see model1.Residuals do I have to get Runtime Version v4.0.30319 and include some dll in the project? Where could I get this? I tried on the Cronos site but could use some directions.   Thank you Sign In·View Thread·Permalink
 Re: Five and question Jeff B. Cromwell 30 Nov '12 - 8:15
 I have not been able to recreate your error on my platform and I have changed the API from Cronos for my next book. Please see the above comment.   Thanks,   Jeff B. Cromwell Sign In·View Thread·Permalink
 Last Visit: 31 Dec '99 - 18:00     Last Update: 25 May '13 - 7:57 Refresh 1