Click here to Skip to main content
13,707,322 members
Click here to Skip to main content
Add your own
alternative version

Tagged as

Stats

4.3K views
4 bookmarked
Posted 5 Jul 2018
Licenced CPOL

Process Simulation with the Free DARL Online Service

, 5 Jul 2018
Rate this:
Please Sign up or sign in to vote.
You can simulate with DARL too! This demonstrates a financial trading simulation.

Introduction

This is the 4th in a sequence of articles about DARL A language and online REST interface and SaaS service that enables you to build and employ fuzzy logic expert systems, and even create them using Machine Learning.

As I've described the uses of DARL so far, they have been concerned only with one point in time, where all the information required for an inference is available, or can be obtained through a set of questions. This ignores the effect of time or the dependency an inference might have on a sequence of events having occurred previously. DARL has a variant called DASL (Doctor Andy's Simulation Language) that can handle time based inference. The only difference between DARL and DASL is one new element, the Delay operator, and a runtime engine that has a temporal database built in.

Code for this example can be found here.

The free REST interface we are using is described here.

The previous articles are:

Background

The example I'm going to use to demonstrate how DASL/DARL simulation might be useful is drawn from financial trading. Obviously, financial trading is an enormous subject. However, there is one fundamental principle that is very simple: "Buy cheap, sell dear".

Trading is about transferring funds from one financial instrument to another and back again, in such a way as to have more of the first financial instrument when you've finished. In this case, the two financial instruments we will use are the pound sterling and the dollar. I'm going to start with a simulated £10,000 and trade in and out of the dollar.

To do this, I've collected two years' historic data of daily trading between the two in the file you can find in the repository entitled GBP_USD.csv.

This data contains a date and the open, close, high and low exchange rates for each day. We're going to simulate trading at the close. In the file, this value is called "price".

When you trade anything through an exchange, or use a high street foreign exchange, there are two sources of cost. There's normally a transaction fee and there's a "spread" which is an offset to the central rate. These are the sources of guaranteed profit to the brokers. Our simulation will have values for both of these.

Simulating trading will require us to respond to a trading signal and to buy whichever currency we are told, to calculate and subtract charges, and to keep track of the value of our holding.

In trading parlance, in currency trading, you can be "long" or "short" a particular currency. If we are holding sterling, we are long sterling and short the dollar, if we are holding dollars we are short sterling and long dollar.

We'll set the terminology relative to sterling, so a long trading signal means buy sterling, and a short means buy dollar. In this simple trading simulation, we will switch all of our cash between one currency and the other.

So we have a history of exchange rates and a simulation of trading that takes trading signals. How do we decide what trading signals to generate? This, of course is the million (billion/trillion?) dollar question. If only we knew!

For simplicity, I'm going to use an old trading strategy that is a kind of momentum trading. It uses two moving averages, one with a small length and one with a large length, and when one crosses the other, a trading signal is generated. Most trading strategies use some analysis of the past history of the instrument to generate trading signals. I've selected lengths of 3 and 9 for these moving averages. You may well want to experiment.

A warning. Financial time series have been shown (by myself and others) to be Chaotic. This means that they are unpredictable long term. What this means in practice is that, even if you find a trading system that works over a period of time, it is very unlikely to continue to work as time goes by. I don't advise you to actually trade with these ideas. In fact, the example I've given loses money. But no doubt some combination of moving averages could be found that didn't.

The DARL Code

Our simulation consists of the exchange rate history, DARL ruleset code, the REST interface on the DARL site which will perform the simulation, and some C# code to load the data, orchestrate the simulation and report the results.

All the examples so far of DARL code have had only a single ruleset in the code. In this example, there are two rulesets. The first is called "tradingrules":

ruleset tradingrules
{
  input numeric price;
  input numeric price_1;
  input numeric price_2;
  input numeric price_3;
  input numeric price_4;
  input numeric price_5;
  input numeric price_6;
  input numeric price_7;
  input numeric price_8;

  output numeric average3;
  output numeric average9;
 
  output categorical trade {long,short};
 
  if anything then average3 will be sum(price, price_1, price_2) / 3;
  if anything then average9 will be sum
    (price, price_1, price_2, price_3, price_4, price_5, price_6, price_7, price_8) / 9;
  if average3 is >= average9 then trade will be long;
  if average3 is < average9 then trade will be short;
}

This takes in 9 inputs, the price and 8 delayed versions thereof. It creates two moving averages and a single trading signal.

By now, how this works should be obvious. price_1 is the price delayed by one sample time - in this case one day, and price_8 is delayed by 8 days.

I'll describe how we get the delayed values later.

The trading simulation looks like this:

ruleset tradingsim
{
  input categorical trade {long,short};
  input categorical oldtrade {long,short};
  input numeric price;
  input numeric balance;
 
  constant spread 0.001;
  constant transaction_fee 10.00;
 
  output categorical transact {true,false};
  output numeric newbalance;
  output numeric sterling;
 
  if not trade is categoryof(oldtrade)  or trade is present and oldtrade is absent 
                                        then transact will be true;
  otherwise if anything then transact will be false;
 
  if transact is true and trade is long then newbalance will be balance / (price + spread) - 
                                                                        transaction_fee;
  if transact is true and trade is short then newbalance will be balance * (price - spread) - 
                                                                        transaction_fee * price;
  otherwise if anything then newbalance will be balance;
 
  if trade is short then sterling will be newbalance / price;
  otherwise if anything then sterling will be newbalance;
}

The inputs are the trading signal "trade" from the tradingrules ruleset, oldtrade which is the last trading signal, so trade delayed by one cycle, the current price and the current balance.

The outputs are transact, which responds to a change in trading signal, newbalance, which is the value of the pot after the transaction, and sterling, which is the sterling value of the pot.

So, transact is true whenever trade is not the same as oldtrade, or in the special case of the first trade after the moving averages fill up, when trade is known, but oldtrade is unknown.

Whenever transact fires, we switch the funds from one currency to the other, incurring charges as we go. The charges are set by the constants. So newbalance and balance are in alternately dollars and pounds sterling.

Finally, in order to keep track of things, sterling shows you the value of the pot in pounds sterling by converting newbalance whenever it contains dollars.

The last bit of DARL creates the delays and wires everything together. So DARL uses a circuit diagram/schematic paradigm where rulesets are like electronic components, mapinputs and mapoutputs are like edge connectors, and wires and delays wire everything up.

mapinput price;
mapinput balance;
mapoutput sterling;
mapoutput trade;
mapoutput newbalance;

wire price tradingrules.price;
wire price tradingsim.price;
wire balance tradingsim.balance;
wire tradingrules.trade tradingsim.trade;
wire tradingsim.sterling sterling;
wire tradingrules.trade trade;
wire tradingsim.newbalance newbalance;
delay trade tradingsim.oldtrade {0};
delay price tradingrules.price_1 {0};
delay price tradingrules.price_2 {1};
delay price tradingrules.price_3 {2};
delay price tradingrules.price_4 {3};
delay price tradingrules.price_5 {4};
delay price tradingrules.price_6 {5};
delay price tradingrules.price_7 {6};
delay price tradingrules.price_8 {7};
delay newbalance tradingsim.balance {0};

Delays behave like wires, but they delay the signal. The signal offset values in the curly brackets are perhaps confusing. {0} offset means the result of a processing cycle fed back, so the previous value. A delay of {1} means the previous value delayed a further cycle, and so on.

The DASL Runtime

The DASL runtime's job is to take a set of data values arranged in time and sequence them through the inference engine and through the rulesets.

Because DARL uses a fuzzy inference engine, any input or output can be in a known or unknown state and have degrees of certainty attached. If a data item is present in the current sequence of the data used, then it's marked as known. if not, unknown.

In a simulation, you often have data that runs out after a while. For instance, if you were simulating a factory using historical data right up to the present, but you would expect the simulation to continue into the future.

DASL is arranged so that, if you have an output with the same name as a data value in the historical data, it will use the historical value until it runs out, and then switch seamlessly to the simulated value.

This is what happens with the balance value. This is only set in the first set of data to set the initial value of the pot at £10,000, and thereafter the value is provided by the simulation.

Input Data

The time series data input to the simulation is required to be in a particular format.

public class DaslData
{
    /// <summary>
    /// The code
    /// </summary>
    public string code { get; set; }

    /// <summary>
    /// The time series history
    /// </summary>
    public DaslSet history { get; set; }
}

A DaslSet looks like this:

public class DaslSet
{
    /// <summary>
    /// Gets or sets the events.
    /// </summary>
    /// <value>
    /// The events.
    /// </value>
    [Required]
    [Display(Name = "The sequence of events",
             Description = "A sequence of time-tagged sets of values")]
    public List<DaslState> events { get; set; } = new List<DaslState>();

    /// <summary>
    /// Gets or sets the sample time.
    /// </summary>
    /// <value>
    /// The sample time.
    /// </value>
    [Required]
    [Display(Name = "The sample time",
             Description = "Will be used to set up the sample time of the simulation")]
    public TimeSpan sampleTime { get; set; }

    /// <summary>
    /// Gets or sets the description.
    /// </summary>
    /// <value>
    /// The description.
    /// </value>
    [Display(Name = "The description",
             Description = "Description of the contained sampled events")]
    public string description { get; set; }
}

The sampleTime determines the clock rate of the simulation. events contains the time series data as time tagged time slices.

DaslState looks like this:

public class DaslState
{
    /// <summary>
    /// Gets or sets the time stamp.
    /// </summary>
    /// <value>The time stamp.</value>
    [Required]
    [Display(Name = "The time stamp",
             Description = "The moment these values changed or became valid")]
    public DateTime timeStamp { get; set; }

    /// <summary>
    /// Gets or sets the values.
    /// </summary>
    /// <value>The values.</value>
    [Required]
    [Display(Name = "The values",
             Description = "A set of values that changed or became valid at the given time")]
    public List<DarlVar> values { get; set; }
}

Finally, DarlVar classes hold individual data values, annotated with a name, uncertainty, etc.

[Serializable]
public partial class DarlVar
{
    /// <summary>
    /// The type of data stored in the DarlVar
    /// </summary>
    public enum DataType
    {
        /// <summary>
        /// Numeric including fuzzy
        /// </summary>
        numeric,
        /// <summary>
        /// One or more categories with confidences
        /// </summary>
        categorical,
        /// <summary>
        /// Textual
        /// </summary>
        textual,

    }

    /// <summary>
    /// Gets or sets the name.
    /// </summary>
    /// <value>The name.</value>
    public string name { get; set; }

    /// <summary>
    /// This result is unknown if true.
    /// </summary>
    /// <value><c>true</c> if unknown; otherwise, <c>false</c>.</value>
    public bool unknown { get; set; } = false;
    /// <summary>
    /// The confidence placed in this result
    /// </summary>
    /// <value>The weight.</value>
    public double weight { get; set; } = 1.0;

    /// <summary>
    /// The array containing the up to 4 values representing the fuzzy number.
    /// </summary>
    /// <value>The values.</value>
    /// <remarks>Since all fuzzy numbers used by DARL are convex,
    /// i.e., their envelope doesn't have any in-folding
    /// sections, the user can specify numbers with a simple sequence of doubles.
    /// So 1 double represents a crisp or singleton value.
    /// 2 doubles represent an interval,
    /// 3 a triangular fuzzy set,
    /// 4 a trapezoidal fuzzy set.
    /// The values must be ordered in ascending value,
    /// but it is permissible for two or more to hold the same value.</remarks>
    public List<double> values { get; set; }

    /// <summary>
    /// list of categories, each indexed against a truth value.
    /// </summary>
    /// <value>The categories.</value>
    public Dictionary<string, double> categories { get; set; }

    public List<DateTime> times { get; set; }

    /// <summary>
    /// Indicates approximation has taken place in calculating the values.
    /// </summary>
    /// <value><c>true</c> if approximate; otherwise, <c>false</c>.</value>
    /// <remarks>Under some circumstances the coordinates of the fuzzy number
    /// in "values" may not exactly represent the "cuts" values.</remarks>
    public bool approximate { get; set; }

    /// <summary>
    /// Gets or sets the type of the data.
    /// </summary>
    /// <value>The type of the data.</value>
    public DataType dataType { get; set; }

    /// <summary>
    /// Gets or sets the sequence.
    /// </summary>
    /// <value>The sequence.</value>
    public List<List<string>> sequence { get; set; }

    /// <summary>
    /// Single central or most confident value, expressed as a string.
    /// </summary>
    /// <value>The value.</value>
    public string Value { get; set; } = string.Empty;

}

The C# Code

The purpose of the console app is to read the historical data, put it into the right format, send it to the REST service along with the DARL code, and to take the generated simulation data and output it.

The historical data is in CSV format, so we may as well output it the same way. For CSV handling, we'll use the CsvHelper NuGet package.

var initial_balance = "10000";
//get the data and ruleset from the exe
var ddata = new DaslData();
var csv = new CsvReader(new StreamReader
(Assembly.GetExecutingAssembly().GetManifestResourceStream("DaslTradingExample.GBP_USD.csv")));
var records = csv.GetRecords<TradingRecord>().ToList();
ddata.code = new StreamReader(Assembly.GetExecutingAssembly().GetManifestResourceStream
("DaslTradingExample.trading_simulation.darl")).ReadToEnd();
//convert the csv records to a DaslSet.
ddata.history = new DaslSet();
ddata.history.sampleTime = new TimeSpan(1, 0, 0, 0); // 1 day
ddata.history.events = new List<DaslState>();
foreach (var r in records)
{
    if (r == records.Last()) //records are in reverse order, 
                     //set the initial value of the balance and add the first price
        ddata.history.events.Add(new DaslState { timeStamp = DateTime.Parse(r.Date), 
        values = new List<DarlVar> { new DarlVar { name = "price", 
        Value = r.Price, dataType = DarlVar.DataType.numeric }, 
        new DarlVar { name = "balance", 
        dataType = DarlVar.DataType.numeric, Value = initial_balance } } });
    else // add the day's price
        ddata.history.events.Add(new DaslState { timeStamp = DateTime.Parse(r.Date), 
        values = new List<DarlVar> { new DarlVar { name = "price", 
        Value = r.Price, dataType = DarlVar.DataType.numeric } } });
}

The first part of the code reads the embedded historical data and ruleset.

We then create a DaslData class and populate the code and the history sections.

Note that the records in the historical data are in reverse order, so we add the initial balance as a data item to the last of them. Events can be presented in any order. The DASL runtime's temporal database will make sense of them.

//send the data off to the simulator
var valueString = JsonConvert.SerializeObject(ddata);
var client = new HttpClient();
var response = await client.PostAsync("https://darl.ai/api/Linter/DaslSimulate", 
               new StringContent(valueString, Encoding.UTF8, "application/json"));
var resp = await response.Content.ReadAsStringAsync();
var returnedData =  JsonConvert.DeserializeObject<DaslSet>(resp);

Next, we send of the DaslSet object to the free REST interface at darl.ai.

//now write out the results as a csv file
var outlist = new List<OutputRecord>();
foreach(var d in returnedData.events)
{
    var sterling = d.values.Where(a => a.name == "sterling").First();
    var ave3 = d.values.Where(a => a.name == "tradingrules.average3").First();
    var ave9 = d.values.Where(a => a.name == "tradingrules.average9").First();
    var price = d.values.Where(a => a.name == "price").First();
    var newbalance = d.values.Where(a => a.name == "newbalance").First();
    outlist.Add(new OutputRecord {
    price = price.values[0],
    date = d.timeStamp,
    sterling = sterling.unknown ? 0.0 : sterling.values[0],
    ave3 = ave3.unknown ? 0.0 : ave3.values[0],
    ave9 = ave9.unknown ? 0.0 : ave9.values[0],
    newbalance = newbalance.unknown ? 0.0 : newbalance.values[0],
    trade = d.values.Where(a => a.name == "trade").First().Value,
    transact = d.values.Where(a => a.name == "tradingsim.transact").First().Value
});
}
var csvout = new CsvWriter(new StreamWriter("results.csv"));
csvout.WriteRecords(outlist);

Finally, we write out all the simulated data to a CSV file.

date,price,trade,sterling,ave3,ave9,transact,newbalance
30/06/2016 00:00:00,1.3311,,10000,0,0,false,10000
01/07/2016 00:00:00,1.3262,,10000,0,0,false,10000
04/07/2016 00:00:00,1.329,,10000,1.32876666666667,0,false,10000
05/07/2016 00:00:00,1.3023,,10000,1.31916666666667,0,false,10000
06/07/2016 00:00:00,1.2931,,10000,1.30813333333333,0,false,10000
07/07/2016 00:00:00,1.291,,10000,1.29546666666667,0,false,10000
08/07/2016 00:00:00,1.2956,,10000,1.29323333333333,0,false,10000
11/07/2016 00:00:00,1.2994,,10000,1.29533333333333,0,false,10000
12/07/2016 00:00:00,1.3243,short,9982.44884089708,1.30643333333333,1.31022222222222,true,13219.757
13/07/2016 00:00:00,1.3147,long,10037.6985635023,1.3128,1.3084,true,10037.6985635023
14/07/2016 00:00:00,1.3342,long,10037.6985635023,1.3244,1.30928888888889,false,10037.6985635023
15/07/2016 00:00:00,1.3193,long,10037.6985635023,1.32273333333333,
                    1.30821111111111,false,10037.6985635023

Note that the output DaslSet contains all the data items in the simulation that are available, i.e., in a known state, for each step of the simulation. You can access ruleset values using the <ruleset>.<ioname> naming convention.

The sterling value of the simulated trading

This is the performance of the trading system.

Things You Can Experiment With

You can easily change the source data. Historical data is freely available from sites like Yahoo finance. The trading algorithm is based round the choice of two moving averages. If you want a different selection, like 4 and 17, you would need to add more delays and inputs and change the darl that calculated the averages.

Future Work

I created this example and split the darl into two rulesets in order to create an application I could use with Genetic Programming. This is a form of Machine learning that can optimise an element of a system. It is extremely expensive in processing power, so I shan't be providing a free version! However, fuzzy rules are uniquely suited to GP, see this now very old paper. The idea is that GP would modify the TradingRules ruleset, initially seeding a large number of randomly generated rules and improving them using simulated evolution.

Watch out for the next article.

History

  • 07/05/2018: Initial version

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

AndyEdmonds
United Kingdom United Kingdom
No Biography provided

You may also be interested in...

Pro
Pro

Comments and Discussions

 
QuestionMessage Closed Pin
20-Jul-18 21:43
member20-Jul-18 21:43 
NewsMessage Closed Pin
20-Jul-18 0:21
member20-Jul-18 0:21 
Questionupdate garmin nuvi Pin
20-Jul-18 0:08
member20-Jul-18 0:08 
NewsMessage Closed Pin
19-Jul-18 23:19
member19-Jul-18 23:19 
NewsMessage Closed Pin
19-Jul-18 23:18
member19-Jul-18 23:18 
QuestionMessage Closed Pin
19-Jul-18 21:12
member19-Jul-18 21:12 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Cookies | Terms of Use | Mobile
Web01-2016 | 2.8.180920.1 | Last Updated 5 Jul 2018
Article Copyright 2018 by AndyEdmonds
Everything else Copyright © CodeProject, 1999-2018
Layout: fixed | fluid