13,828,588 members
alternative version

#### Stats

6.1K views
4 bookmarked
Posted 5 Jul 2018
Licenced CPOL

# Process Simulation with the Free DARL Online Service

, 5 Jul 2018
You can simulate with DARL too! This demonstrates a financial trading simulation.

## Introduction

This is the 4th in a sequence of articles about DARL A language and online REST interface and SaaS service that enables you to build and employ fuzzy logic expert systems, and even create them using Machine Learning.

As I've described the uses of DARL so far, they have been concerned only with one point in time, where all the information required for an inference is available, or can be obtained through a set of questions. This ignores the effect of time or the dependency an inference might have on a sequence of events having occurred previously. DARL has a variant called DASL (Doctor Andy's Simulation Language) that can handle time based inference. The only difference between DARL and DASL is one new element, the Delay operator, and a runtime engine that has a temporal database built in.

Code for this example can be found here.

The free REST interface we are using is described here.

The previous articles are:

## Background

The example I'm going to use to demonstrate how DASL/DARL simulation might be useful is drawn from financial trading. Obviously, financial trading is an enormous subject. However, there is one fundamental principle that is very simple: "Buy cheap, sell dear".

Trading is about transferring funds from one financial instrument to another and back again, in such a way as to have more of the first financial instrument when you've finished. In this case, the two financial instruments we will use are the pound sterling and the dollar. I'm going to start with a simulated £10,000 and trade in and out of the dollar.

To do this, I've collected two years' historic data of daily trading between the two in the file you can find in the repository entitled GBP_USD.csv.

This data contains a date and the open, close, high and low exchange rates for each day. We're going to simulate trading at the close. In the file, this value is called "`price`".

When you trade anything through an exchange, or use a high street foreign exchange, there are two sources of cost. There's normally a transaction fee and there's a "spread" which is an offset to the central rate. These are the sources of guaranteed profit to the brokers. Our simulation will have values for both of these.

Simulating trading will require us to respond to a trading signal and to buy whichever currency we are told, to calculate and subtract charges, and to keep track of the value of our holding.

In trading parlance, in currency trading, you can be "long" or "short" a particular currency. If we are holding sterling, we are long sterling and short the dollar, if we are holding dollars we are short sterling and long dollar.

We'll set the terminology relative to sterling, so a long trading signal means buy sterling, and a short means buy dollar. In this simple trading simulation, we will switch all of our cash between one currency and the other.

So we have a history of exchange rates and a simulation of trading that takes trading signals. How do we decide what trading signals to generate? This, of course is the million (billion/trillion?) dollar question. If only we knew!

For simplicity, I'm going to use an old trading strategy that is a kind of momentum trading. It uses two moving averages, one with a small length and one with a large length, and when one crosses the other, a trading signal is generated. Most trading strategies use some analysis of the past history of the instrument to generate trading signals. I've selected lengths of 3 and 9 for these moving averages. You may well want to experiment.

A warning. Financial time series have been shown (by myself and others) to be Chaotic. This means that they are unpredictable long term. What this means in practice is that, even if you find a trading system that works over a period of time, it is very unlikely to continue to work as time goes by. I don't advise you to actually trade with these ideas. In fact, the example I've given loses money. But no doubt some combination of moving averages could be found that didn't.

## The DARL Code

Our simulation consists of the exchange rate history, DARL ruleset code, the REST interface on the DARL site which will perform the simulation, and some C# code to load the data, orchestrate the simulation and report the results.

All the examples so far of DARL code have had only a single ruleset in the code. In this example, there are two rulesets. The first is called "`tradingrules`":

```ruleset tradingrules
{
input numeric price;
input numeric price_1;
input numeric price_2;
input numeric price_3;
input numeric price_4;
input numeric price_5;
input numeric price_6;
input numeric price_7;
input numeric price_8;

output numeric average3;
output numeric average9;

if anything then average3 will be sum(price, price_1, price_2) / 3;
if anything then average9 will be sum
(price, price_1, price_2, price_3, price_4, price_5, price_6, price_7, price_8) / 9;
if average3 is >= average9 then trade will be long;
if average3 is < average9 then trade will be short;
}```

This takes in 9 inputs, the `price `and 8 delayed versions thereof. It creates two moving averages and a single trading signal.

By now, how this works should be obvious. `price_1` is the price delayed by one sample time - in this case one day, and `price_8` is delayed by 8 days.

I'll describe how we get the delayed values later.

The trading simulation looks like this:

```ruleset tradingsim
{
input numeric price;
input numeric balance;

constant transaction_fee 10.00;

output categorical transact {true,false};
output numeric newbalance;
output numeric sterling;

then transact will be true;
otherwise if anything then transact will be false;

if transact is true and trade is long then newbalance will be balance / (price + spread) -
transaction_fee;
if transact is true and trade is short then newbalance will be balance * (price - spread) -
transaction_fee * price;
otherwise if anything then newbalance will be balance;

if trade is short then sterling will be newbalance / price;
otherwise if anything then sterling will be newbalance;
}```

The inputs are the trading signal "`trade`" from the `tradingrules `ruleset, `oldtrade `which is the last trading signal, so `trade `delayed by one cycle, the current `price `and the current `balance`.

The outputs are `transact`, which responds to a change in trading signal, `newbalance`, which is the value of the pot after the transaction, and `sterling`, which is the sterling value of the pot.

So, `transact `is true whenever `trade `is not the same as `oldtrade`, or in the special case of the first trade after the moving averages fill up, when `trade `is known, but `oldtrade `is unknown.

Whenever `transact `fires, we switch the funds from one currency to the other, incurring charges as we go. The charges are set by the constants. So `newbalance `and `balance `are in alternately dollars and pounds sterling.

Finally, in order to keep track of things, `sterling `shows you the value of the pot in pounds sterling by converting `newbalance `whenever it contains dollars.

The last bit of DARL creates the delays and wires everything together. So DARL uses a circuit diagram/schematic paradigm where rulesets are like electronic components, mapinputs and mapoutputs are like edge connectors, and wires and delays wire everything up.

```mapinput price;
mapinput balance;
mapoutput sterling;
mapoutput newbalance;

Delays behave like wires, but they delay the signal. The signal offset values in the curly brackets are perhaps confusing. `{0}` offset means the result of a processing cycle fed back, so the previous value. A delay of `{1}` means the previous value delayed a further cycle, and so on.

## The DASL Runtime

The DASL runtime's job is to take a set of data values arranged in time and sequence them through the inference engine and through the rulesets.

Because DARL uses a fuzzy inference engine, any input or output can be in a known or unknown state and have degrees of certainty attached. If a data item is present in the current sequence of the data used, then it's marked as known. if not, unknown.

In a simulation, you often have data that runs out after a while. For instance, if you were simulating a factory using historical data right up to the present, but you would expect the simulation to continue into the future.

DASL is arranged so that, if you have an output with the same name as a data value in the historical data, it will use the historical value until it runs out, and then switch seamlessly to the simulated value.

This is what happens with the `balance `value. This is only set in the first set of data to set the initial value of the pot at £10,000, and thereafter the value is provided by the simulation.

## Input Data

The time series data input to the simulation is required to be in a particular format.

```public class DaslData
{
/// <summary>
/// The code
/// </summary>
public string code { get; set; }

/// <summary>
/// The time series history
/// </summary>
public DaslSet history { get; set; }
}
```

A `DaslSet` looks like this:

```public class DaslSet
{
/// <summary>
/// Gets or sets the events.
/// </summary>
/// <value>
/// The events.
/// </value>
[Required]
[Display(Name = "The sequence of events",
Description = "A sequence of time-tagged sets of values")]
public List<DaslState> events { get; set; } = new List<DaslState>();

/// <summary>
/// Gets or sets the sample time.
/// </summary>
/// <value>
/// The sample time.
/// </value>
[Required]
[Display(Name = "The sample time",
Description = "Will be used to set up the sample time of the simulation")]
public TimeSpan sampleTime { get; set; }

/// <summary>
/// Gets or sets the description.
/// </summary>
/// <value>
/// The description.
/// </value>
[Display(Name = "The description",
Description = "Description of the contained sampled events")]
public string description { get; set; }
}
```

The `sampleTime `determines the clock rate of the simulation. `events `contains the time series data as time tagged time slices.

`DaslState` looks like this:

```public class DaslState
{
/// <summary>
/// Gets or sets the time stamp.
/// </summary>
/// <value>The time stamp.</value>
[Required]
[Display(Name = "The time stamp",
Description = "The moment these values changed or became valid")]
public DateTime timeStamp { get; set; }

/// <summary>
/// Gets or sets the values.
/// </summary>
/// <value>The values.</value>
[Required]
[Display(Name = "The values",
Description = "A set of values that changed or became valid at the given time")]
public List<DarlVar> values { get; set; }
}
```

Finally, `DarlVar` classes hold individual data values, annotated with a name, uncertainty, etc.

```[Serializable]
public partial class DarlVar
{
/// <summary>
/// The type of data stored in the DarlVar
/// </summary>
public enum DataType
{
/// <summary>
/// Numeric including fuzzy
/// </summary>
numeric,
/// <summary>
/// One or more categories with confidences
/// </summary>
categorical,
/// <summary>
/// Textual
/// </summary>
textual,

}

/// <summary>
/// Gets or sets the name.
/// </summary>
/// <value>The name.</value>
public string name { get; set; }

/// <summary>
/// This result is unknown if true.
/// </summary>
/// <value><c>true</c> if unknown; otherwise, <c>false</c>.</value>
public bool unknown { get; set; } = false;
/// <summary>
/// The confidence placed in this result
/// </summary>
/// <value>The weight.</value>
public double weight { get; set; } = 1.0;

/// <summary>
/// The array containing the up to 4 values representing the fuzzy number.
/// </summary>
/// <value>The values.</value>
/// <remarks>Since all fuzzy numbers used by DARL are convex,
/// i.e., their envelope doesn't have any in-folding
/// sections, the user can specify numbers with a simple sequence of doubles.
/// So 1 double represents a crisp or singleton value.
/// 2 doubles represent an interval,
/// 3 a triangular fuzzy set,
/// 4 a trapezoidal fuzzy set.
/// The values must be ordered in ascending value,
/// but it is permissible for two or more to hold the same value.</remarks>
public List<double> values { get; set; }

/// <summary>
/// list of categories, each indexed against a truth value.
/// </summary>
/// <value>The categories.</value>
public Dictionary<string, double> categories { get; set; }

public List<DateTime> times { get; set; }

/// <summary>
/// Indicates approximation has taken place in calculating the values.
/// </summary>
/// <value><c>true</c> if approximate; otherwise, <c>false</c>.</value>
/// <remarks>Under some circumstances the coordinates of the fuzzy number
/// in "values" may not exactly represent the "cuts" values.</remarks>
public bool approximate { get; set; }

/// <summary>
/// Gets or sets the type of the data.
/// </summary>
/// <value>The type of the data.</value>
public DataType dataType { get; set; }

/// <summary>
/// Gets or sets the sequence.
/// </summary>
/// <value>The sequence.</value>
public List<List<string>> sequence { get; set; }

/// <summary>
/// Single central or most confident value, expressed as a string.
/// </summary>
/// <value>The value.</value>
public string Value { get; set; } = string.Empty;

}
```

## The C# Code

The purpose of the console app is to read the historical data, put it into the right format, send it to the REST service along with the DARL code, and to take the generated simulation data and output it.

The historical data is in CSV format, so we may as well output it the same way. For CSV handling, we'll use the `CsvHelper` NuGet package.

```var initial_balance = "10000";
//get the data and ruleset from the exe
var ddata = new DaslData();
//convert the csv records to a DaslSet.
ddata.history = new DaslSet();
ddata.history.sampleTime = new TimeSpan(1, 0, 0, 0); // 1 day
ddata.history.events = new List<DaslState>();
foreach (var r in records)
{
if (r == records.Last()) //records are in reverse order,
//set the initial value of the balance and add the first price
ddata.history.events.Add(new DaslState { timeStamp = DateTime.Parse(r.Date),
values = new List<DarlVar> { new DarlVar { name = "price",
Value = r.Price, dataType = DarlVar.DataType.numeric },
new DarlVar { name = "balance",
dataType = DarlVar.DataType.numeric, Value = initial_balance } } });
else // add the day's price
ddata.history.events.Add(new DaslState { timeStamp = DateTime.Parse(r.Date),
values = new List<DarlVar> { new DarlVar { name = "price",
Value = r.Price, dataType = DarlVar.DataType.numeric } } });
}```

The first part of the code reads the embedded historical data and ruleset.

We then create a `DaslData` class and populate the code and the history sections.

Note that the records in the historical data are in reverse order, so we add the initial balance as a data item to the last of them. Events can be presented in any order. The DASL runtime's temporal database will make sense of them.

```//send the data off to the simulator
var valueString = JsonConvert.SerializeObject(ddata);
var client = new HttpClient();
var response = await client.PostAsync("https://darl.ai/api/Linter/DaslSimulate",
new StringContent(valueString, Encoding.UTF8, "application/json"));
var returnedData =  JsonConvert.DeserializeObject<DaslSet>(resp);```

Next, we send of the `DaslSet` object to the free REST interface at `darl.ai`.

```//now write out the results as a csv file
var outlist = new List<OutputRecord>();
foreach(var d in returnedData.events)
{
var sterling = d.values.Where(a => a.name == "sterling").First();
var ave3 = d.values.Where(a => a.name == "tradingrules.average3").First();
var ave9 = d.values.Where(a => a.name == "tradingrules.average9").First();
var price = d.values.Where(a => a.name == "price").First();
var newbalance = d.values.Where(a => a.name == "newbalance").First();
price = price.values[0],
date = d.timeStamp,
sterling = sterling.unknown ? 0.0 : sterling.values[0],
ave3 = ave3.unknown ? 0.0 : ave3.values[0],
ave9 = ave9.unknown ? 0.0 : ave9.values[0],
newbalance = newbalance.unknown ? 0.0 : newbalance.values[0],
transact = d.values.Where(a => a.name == "tradingsim.transact").First().Value
});
}
var csvout = new CsvWriter(new StreamWriter("results.csv"));
csvout.WriteRecords(outlist);```

Finally, we write out all the simulated data to a CSV file.

```date,price,trade,sterling,ave3,ave9,transact,newbalance
30/06/2016 00:00:00,1.3311,,10000,0,0,false,10000
01/07/2016 00:00:00,1.3262,,10000,0,0,false,10000
04/07/2016 00:00:00,1.329,,10000,1.32876666666667,0,false,10000
05/07/2016 00:00:00,1.3023,,10000,1.31916666666667,0,false,10000
06/07/2016 00:00:00,1.2931,,10000,1.30813333333333,0,false,10000
07/07/2016 00:00:00,1.291,,10000,1.29546666666667,0,false,10000
08/07/2016 00:00:00,1.2956,,10000,1.29323333333333,0,false,10000
11/07/2016 00:00:00,1.2994,,10000,1.29533333333333,0,false,10000
12/07/2016 00:00:00,1.3243,short,9982.44884089708,1.30643333333333,1.31022222222222,true,13219.757
13/07/2016 00:00:00,1.3147,long,10037.6985635023,1.3128,1.3084,true,10037.6985635023
14/07/2016 00:00:00,1.3342,long,10037.6985635023,1.3244,1.30928888888889,false,10037.6985635023
15/07/2016 00:00:00,1.3193,long,10037.6985635023,1.32273333333333,
1.30821111111111,false,10037.6985635023```

Note that the output `DaslSet` contains all the data items in the simulation that are available, i.e., in a known state, for each step of the simulation. You can access ruleset values using the `<ruleset>.<ioname>` naming convention.

This is the performance of the trading system.

## Things You Can Experiment With

You can easily change the source data. Historical data is freely available from sites like Yahoo finance. The trading algorithm is based round the choice of two moving averages. If you want a different selection, like 4 and 17, you would need to add more delays and inputs and change the darl that calculated the averages.

## Future Work

I created this example and split the darl into two rulesets in order to create an application I could use with Genetic Programming. This is a form of Machine learning that can optimise an element of a system. It is extremely expensive in processing power, so I shan't be providing a free version! However, fuzzy rules are uniquely suited to GP, see this now very old paper. The idea is that GP would modify the `TradingRules` ruleset, initially seeding a large number of randomly generated rules and improving them using simulated evolution.

Watch out for the next article.

## History

• 07/05/2018: Initial version

## Share

 United Kingdom
No Biography provided

## You may also be interested in...

 Pro Pro

 First Prev Next
 Message Closed 20-Jul-18 22:43 Member 13920551 20-Jul-18 22:43
 Message Closed 20-Jul-18 1:21 Member 13919210 20-Jul-18 1:21
 update garmin nuvi Member 1391921020-Jul-18 1:08 Member 13919210 20-Jul-18 1:08
 Message Closed 20-Jul-18 0:19 Member 13919210 20-Jul-18 0:19
 Message Closed 20-Jul-18 0:18 Member 13919210 20-Jul-18 0:18
 Message Closed 19-Jul-18 22:12 Member 13919210 19-Jul-18 22:12
 Last Visit: 15-Jan-19 17:50     Last Update: 15-Jan-19 17:50 Refresh 1