Simple Data Queue

An implementation of this data container is the SimpleDataQueue class; to access it in any code project, the FinAnalisys.dll must be referenced and the following using statement should precede the code:

1using FinAnalysis.Base;

 To enable custom listeners, the corresponding package should also be referenced:

1using FinAnalysis.Listeners;

Inner data presentation in the class is a two-side managed queue with elements (double numbers) positioned and accessible by indexes. The elements are generally ordered in such a way that the last added value has the minimal index 0 and the very early added element has a maximal index. The elements of the queue can be time-assigned (stored with their associated time stamps) and also accessible by time in this case.

The class implements multimode logic which is not always as simple as the class name suggests.

As a rule, each mode implies using of compatible methods to modify content so that an incompatible call causes an Exception. Some operations can force the instance of the class to change its current mode for flexibility reason.

To assure correct using, the class provides several mode-related properties that inform the careful user about instance current state:

A current instance mode is characterized by combination of the three features:

1. Regimen of data queue,
2. Time Assignment (whether elements are stored with time stamps or not),
3. A need to compute statistics.

The first feature, Regimen of the data queue, is the main characteristic which mostly affects class logic and usage. Possible regimens are specified by the members of the QueueType enumeration:

• ManualControl – the data queue realizes user-specific (by default expanding) data set and adopts it’s capacity to store as many elements as it is currently needed; the user is given with ability to add and remove data points at both sides of the queue and no data points are dropped until removed by a suitable method call;
• AutoDynamic – the data queue realizes sliding data set defined by a time period; adding of new data points forces out and drops the oldest ones if they become strongly out of the time period.  Actual queue capacity (the number of data points reserved) is adapted automatically and depends on data time stamps.

Note
This regimen directly defines Time Assignment to be applied and requires using of the compatible modification method (see detailed below).

• AutoStatic – the data queue realizes sliding data set of a fixed capacity (maximal number of data points) so that adding of a new element forces out and drops the oldest one if the added data point exceeds the capacity.
• SingleElement – the one-element version of the AutoStatic regimen which is default.

The actual data queue regimen is indicated by the class property Type

The table below shows how the Time Assignment feature corresponds to data queue regimens, where “+” means ability to operate with assigned or non-assigned elements:

A very important issue about Time Assignment (if switched on) is time check while adding of new elements which provides time-ordered elements in the data queue. The class instance detects and keeps the latest and the oldest time stamps of the elements and uses these values to apply the following rules:

• The time stamp of a newly added element must be equal or newer than the latest one in the queue, otherwise the new data point is ignored (not added).
• When adding an element to the end of the queue (what is possible only with the ManualControl regimen), it’s time stamp must be equal or older than the oldest one in the queue, otherwise adding is rejected.

Every successful adding modifies either the oldest or the latest time stamp of the queue.

Also, time check detects the DateTime.MinValue time stamp and considers it as meaningless. Depending on Initialized status, such a time stamp can lead either to Exception (if true) or to turning Time Assignment OFF (when false).

Class properties associated with the Time Assignment feature are (see the common properties above for details):

• NeedDateTime,
• LastDateTime,
• FirstDateTime.

Time Assignment is switched OFF by default.

Computing of statistics is applicable to any of the queue regimens independently of Time Assignment status. By default statistics are not provided until required via special input parameter while initialization. Statistics with the SingleElement queue regimen are practically useless but also supported for compatibility reason.

The class provides constructors that are generally focused on specifying of the queue regimen and statistical service status. 

With that, one of the queue regimens strongly predetermines Time Assignment status: as the table above shows, the AutoDynamic regimen implies only time-assigned elements.

All other regimens provide the user with ability to turn Time Assignment ON or OFF by using of a suitable method to add the very first data point to the (empty) data queue.

Manual Control of the data queue requires initialization either with the default constructor or its extended version with the input “statistic” flag:

SimpleDataQueue - initializes a new instance of the class that contains an empty data queue of initial capacity for 10 elements and no statistics provided.

SimpleDataQueue(Boolean) - acts like the default constructor but with statistic calculations (if the input flag is true).

Note

With the ManualControl regimen, the class adopts data queue capacity automatically what is resource consuming operation. For performance reason, it is a good practice to possibly estimate the maximal number of stored elements and, before adding of data points, define the required capacity with just one line of code like this: C# Copy 1manualInstance.Capacity = 1000; An attempt to assign a value less than one or if Initialized=true causes an Exception.

Compatible methods to add data points to the manually controlled queue are PutLast(…) and PutFirst(…) in two versions – either with input time stamps or without.

Using of a time-assigned method for the very first adding to the empty queue turns Time Assignment ON and requires meaningful time stamps for all further data additions when working with a given class instance. Similarly, if the very first adding call was without a time stamp or with the meaningless one, then any attempt to add a time-assigned data point will be rejected with an Exception thrown.

When using of time-assigned methods the user should remember about time check (see about Time Assignment feature above) and take care of valid and meaningful time stamps.

The PutLast(…) methods store an input data point at the beginning of the queue so that it is indexed with zero right after addition:

PutLast(Double)

PutLast(Double, DateTime)

The PutLast(…) methods store an input data point at the opposite side of the queue so that it is indexed at the Count – 1 position in the queue (see also about the Count property above):

PutFirst(Double)

PutFirst(Double, DateTime)

The two methods are to remove data points:

RemoveLast- removes the element at zero position and shifts the others;

RemoveFirst- removes the element at the Count-1 position.

All the methods return true if operation is successful.

The AutoDynamic queue regimen is established by constructors with an input time period:

SimpleDataQueue(TimeSpan) - initializes a new instance of the class which contains an empty data queue to keep elements for the period time interval including interval bounds; the queue reserves positions for 64 elements and computing of statistics is not provided;

SimpleDataQueue(TimeSpan, Boolean) - acts like the previous constructor but with calculations of statistics (if the input flag is true).

Note

Like with the ManualControl regimen, the class adopts data queue capacity automatically, so for performance reason it is a good practice to possibly estimate the maximal number of stored elements and, before adding of data points, define the required capacity with an assignment: C# Copy 1dynamiclInstance.Capacity = maximalNumber; the maximalNumber value must be more than zero to avoid an Exception.

Another (but not preferable) way to set the AutoDynamic regimen up is to assign a (positive) time interval of TimeSpan type to the Period property with the code line like the following:

1anInstance.Period = desiredPeriod;

The AutoDynamic regimen automatically applies Time Assignment and the only compatible method to add data points is

Put(Double, DateTime)- stores the input number at zero position in the queue and updates the newest and the oldest queue time stamps; the oldest data points strongly out of the time period are dropped. The returned true value indicates successful modification.

Note
An attempt to add data with the meaningless DateTime.MinValue time stamp causes an Exception.

To specify the AutoStatic regimen of the data queue, the constructors with an input capacity value should be used:

SimpleDataQueue(Int32) - initializes a new instance of the class which contains an empty data queue to store no more than  capacity  elements and no statistics provided;

SimpleDataQueue(Int32, Boolean) - does the same but sets computing of statistics up.

Using of any of the two constructors with the input capacity of one results in the SingleElement queue regimen, as well as assigning the Capacity property with one before adding of data.

Adding data points with the AutoStatic and SingleElement regimens requires using of the following methods:

Put(Double)- stores the input number at zero position in the queue and drops the very early added data point if the number of elements becomes more than the queue capacity;  returns true if successes;

public bool Put(double data, DateTime time) – does the same but stores also the associated time stamp.

If the first (after initialization) addition of data is done with time-assigned method then Time Assignment is turned ON; this means that the next additions must be also done with meaningful time stamps and time check is applied while adding (see about Time Assignment feature for details). Otherwise, if the first data point is stored without or with the DateTime.MinValue time stamp, Time Assignment is turned OFF and an attempt to add a time-assigned data point causes an Exception.

The class provides several ways to access the data stored; access is allowed if class instance signals Initialized = true, otherwise an Exception is thrown. A good practice is to examine the Ready flag before access to the content (see detailed above about the Initialized and Ready properties).

Whatever queue regimen is selected, the class provides access to all the stored elements by indexes with syntax like used to get an element of an array:

1double element = anInstance[index];

With an automatic queue regimen, AutoDynamic or AutoStatic (SingleElement), the zero index always corresponds to the last (newest) data point. This is not true for the ManualControl queue regimen since data can be added at both sides of the queue.

Apparently, maximal index must not exceed the Count- 1 value (see about the Count property), addressing to a non-existent element (i.e. with the index out of [0, Count-1] range) returns the double.NaN value.

If Time Assignment is applied (NeedDateTime is true) and the current queue regimen is not SingleElement, the class instance runs a conformity mechanism which transforms element positions to time stamps and vice versa so that it becomes possible to access elements by time stamps of DateTime type.

The line:

1double element = anInstance[someTime];

returns the historically latest element with the smaller (older) or equal time stamp.

If the addressing time stamp is strongly out of elements’ time interval (what can be examined with the LastDateTime and FirstDateTime properties) the returned value is double.NaN.

Methods that use index-time transformations to convert indexes and time stamps are:

GetTimeByIndex(Int32)- returns the time stamp associated with the element at the specified position (index) or DateTime.MinValue if addresses to non-existent element;

GetIndexByTime(DateTime)- returns the index of the latest element whose time stamp is equal or older than the specified time; of no such element found then -1 is returned.

Note
An Exception is thrown while attempt to get index or element by time when NeedDateTime=false or with the SingleElement queue regimen.

In addition to the indexers, the class provides convenient named properties to access the data:

To operate with the stored data set as a whole, the class provides two methods:

ToArray- copies the elements of the qeue into a new array and returns the array of originally ordered double numbers;

GetEnumerator- returns System.Collections.IEnumerator object to iterate through the collection of elements.

Note
For performance reason, it is a good practice to get the returned value into a local variable of suitable type and then operate with the elements of the local copy.

If specified while initialization, the class instance computes a number of descriptive statistics and provides them via class properties (the default value of each property is 0.0 which is returned if statistical computations are forbidden):

Refer to the Distribution Listener section for detailed specifications of the statistics mentioned.

The class obtains additional flexibility with the aid of external event handlers (listeners) that can catch and process added and removed values right after content modification is completed.

Listeners can be implemented either as a special class or via methods-delegates. 

Technique of using a class-listener includes the two stages:

• Defining a custom class that implements IDataQueueListenerT interface. The interface implies custom implementation  of the OnPop(…) and OnPush(…) methods to process respectively removed and added  data points;
• Connecting up the listener instance to the class instance with the method SetQueueListener(IDataQueueListenerDouble)

The DistributionListener class is an example of such a listener realized (see the Distribution Listener section for details).  

Delegation approach requires only declaring of delegates and registering them with the methods:

SetOnPopDelegate(SimpleDataQueueOnPopDelegate) – registers the method named onPop to process removed elements;

SetOnPushDelegate(SimpleDataQueueOnPushDelegate) – registers the method named onPush to process added elements.

Both delegate methods must be declared with the following signature:

1public void MethodName(double data, DateTime time)

Every time queue content is modified the methods-delegates will be automatically called to procees added and removed values.

The code snippet below demonstrates delegation technique.

 1//declare delegates
 2public void OnDelete(double data, DateTime time)
 3{
 4    // ...
 5}
 6public void OnAdd(double data, DateTime time)
 7{
 8    // ...
 9}
10
11//init SimpleDataQueue instance:
12SimpleDataQueue gueue = new SimpleDataQueue();
13//register delegates:
14gueue.SetOnPopDelegate(OnDelete);
15gueue.SetOnPushDelegate(OnAdd);

In the examples part of this section, there is a simple code demonstrating how to use custom listeners in class and delegate forms.

The example of SimpleDataQueue class usage:

 1using System;
 2using System.Collections.Generic;
 3using System.Linq;
 4using System.Text;
 5using FinAnalysis.Base;
 6
 7namespace SimpleDataQueueSample
 8{
 9    class SimpleDataQueueSample
10    {
11        const int SimpleDataQueueCapacity = 12;
12        const int Steps = 0xFF;
13
14        static void Main(string[] args)
15        {
16            Random r = new Random();
17
18
19            // AutoStatic Mode
20            SimpleDataQueue simpleDataQueue = new SimpleDataQueue(SimpleDataQueueCapacity, true);
21
22            for (int i = 0; i < Steps; ++i)
23            {
24                simpleDataQueue.Put(r.NextDouble(), DateTime.Now);
25            }
26
27            Console.WriteLine("Arithmetic Mean = {0:0.00}", simpleDataQueue.ArithmeticMean);
28            Console.WriteLine("Coefficient Of Variation = {0:0.00}", simpleDataQueue.CoefficientOfVariation);
29            Console.WriteLine("Expected Value = {0:0.00}", simpleDataQueue.ExpectedValue);
30            Console.WriteLine("First Raw Moment Mean = {0:0.00}", simpleDataQueue.FirstRawMoment);
31            Console.WriteLine("Quadratic Mean = {0:0.00}", simpleDataQueue.QuadraticMean);
32            Console.WriteLine("Second Central Moment  = {0:0.00}", simpleDataQueue.SecondCentralMoment);
33            Console.WriteLine("Second Raw Moment = {0:0.00}", simpleDataQueue.SecondRawMoment);
34            Console.WriteLine("Standard Deviation = {0:0.00}", simpleDataQueue.StandardDeviation);
35            Console.WriteLine("Standard Deviation Population = {0:0.00}", simpleDataQueue.StandardDeviationPopulation);
36            Console.WriteLine("Standard Deviation Sample = {0:0.00}", simpleDataQueue.StandardDeviationSample);
37            Console.WriteLine("Sum = {0:0.00}", simpleDataQueue.Sum);
38            Console.WriteLine("Sum Of Absolute Values = {0:0.00}", simpleDataQueue.SumOfAbsoluteValues);
39            Console.WriteLine("Sum Of Squares = {0:0.00}", simpleDataQueue.SumOfSquares);
40            Console.WriteLine("Variance = {0:0.00}", simpleDataQueue.Variance);
41            Console.WriteLine("Variance Population = {0:0.00}", simpleDataQueue.VariancePopulation);
42            Console.WriteLine("Variance Sample = {0:0.00}", simpleDataQueue.VarianceSample);
43
44
45            for (int i = 0; i < SimpleDataQueueCapacity; ++i)
46            {
47                Console.WriteLine("Time Index #{0} = {1}", i, simpleDataQueue.GetTimeByIndex(i));
48            }
49
50
51
52            // Manual Mode
53            SimpleDataQueue simpleQueue = new SimpleDataQueue(true);
54
55            for (int i = 0; i < Steps; ++i)
56            {
57                simpleQueue.PutFirst(r.NextDouble(), DateTime.Now);
58                simpleQueue.PutLast(r.NextDouble(), DateTime.Now);
59            }
60
61            for (int i = 0; i < Steps / 2; ++i)
62            {
63                simpleQueue.RemoveFirst();
64                simpleQueue.RemoveLast();
65            }
66
67
68            Console.WriteLine("Arithmetic Mean = {0:0.00}", simpleQueue.ArithmeticMean);
69            Console.WriteLine("Coefficient Of Variation = {0:0.00}", simpleQueue.CoefficientOfVariation);
70            Console.WriteLine("Expected Value = {0:0.00}", simpleQueue.ExpectedValue);
71            Console.WriteLine("First Raw Moment Mean = {0:0.00}", simpleQueue.FirstRawMoment);
72            Console.WriteLine("Quadratic Mean = {0:0.00}", simpleQueue.QuadraticMean);
73            Console.WriteLine("Second Central Moment  = {0:0.00}", simpleQueue.SecondCentralMoment);
74            Console.WriteLine("Second Raw Moment = {0:0.00}", simpleQueue.SecondRawMoment);
75            Console.WriteLine("Standard Deviation = {0:0.00}", simpleQueue.StandardDeviation);
76            Console.WriteLine("Standard Deviation Population = {0:0.00}", simpleQueue.StandardDeviationPopulation);
77            Console.WriteLine("Standard Deviation Sample = {0:0.00}", simpleQueue.StandardDeviationSample);
78            Console.WriteLine("Sum = {0:0.00}", simpleQueue.Sum);
79            Console.WriteLine("Sum Of Absolute Values = {0:0.00}", simpleQueue.SumOfAbsoluteValues);
80            Console.WriteLine("Sum Of Squares = {0:0.00}", simpleQueue.SumOfSquares);
81            Console.WriteLine("Variance = {0:0.00}", simpleQueue.Variance);
82            Console.WriteLine("Variance Population = {0:0.00}", simpleQueue.VariancePopulation);
83            Console.WriteLine("Variance Sample = {0:0.00}", simpleQueue.VarianceSample);
84        }
85    }
86}