Carlson-Sabelli L, Sabelli HC, Messer, J,  Patel M, Sugerman, A, . Kauffman, L and K. Walthall.  Process method: Part I. An empirical measure of novelty differentiates creative organization from static order and chaos.  Proc. International Systems Society, Kwanak Press, 1997, pp 1072- 1090.

 

                                         Process method:

               I. An empirical measure of novelty differentiates

              creative organization from static order and chaos

                 L. Carlson-Sabelli, H. Sabelli, J. Messer, M. Patel,

                   A. Sugerman and L. Kauffman and K. Walthall.

                      Chicago Center for Creative Development,

              Rush University and University of Illinois at Chicago.

            2400 Lake View Avenue, Chicago, Illinois 60614, U.S.A.

 

                                               Abstract

A series of four articles in this volume formulates anew process theory, a general theory of systems. "Process Method" illustrates the development of a method to study creative processes through the systematic comparison of empirical data (biological, economic, physical) with computer-generated series (random, periodic, chaotic, and biotic). "How is the universe, that it creates a human heart?" portrays evolution as a creative process determined by the asymmetry of action and the complementarity of opposite information. "Process equation" shows how a simple equation that abstracts just these two features of processes can generate series resembling physiological patterns, and numerical constants. "The Union of Opposites" brings these ideas to bear as a personal and social philosophy.

 

Here we present a systematic method to study creative processes empirically, through the mathematical analysis of time series. It involves ordered consideration from simple to complex components of variation in frameworks of 0, 1, 2 3, ... many dimensions, constructed by the calculation of differences between consecutive members of the series, and the embedding of N consecutive members generating vectors that may then be compared for similarity (recurrence). Novel techniques include: (1) Measurement of entropy with a range of bins allows to measure separately its two components, symmetry and diversity. (2) Calculating the entropy of differences to measure the entropy of velocity, acceleration, and higher order components of variation. (3) Use of recurrence time graphs to identify transient patterns (complexes) in natural processes as distinct from attractors (stable, lower dimensional, higher recurrence rate). (4) Quantification of recurrence rate to provide an empirical measure of novelty, differentiating creative organization (life) from static periodic and chaotic order. (5) Embedding plots to detect periodicity and chaoticity, and measure continuity and maximal entropy.

 

Key words: complementary opposition, complexity, coronary illness, depression, electrocardiography, entropy, novelty, process theory, symmetry.

 

This article presents work in progress, aimed at developing methods to study creative processes. The relevance and consequence of process thinking has become widely recognized. Notwithstanding, many efforts to study processes are hindered by adherence to a logic and a methodology based on opposite assumptions that force upon us unnecessary simplifications in data collection, and lead to models that are static, idealized, deterministic or otherwise incompatible with creative change. Equilibrium models, mutually exclusive categories, and linear scales prevent the detection of coexisting opposites. This article describes practical methods that can be implemented using a personal computer to study any type of process without undue simplifications.

Figure 1. Time series analysis distinguishes empirically random, periodic, chaotic and biological organization, and differentiates normal subjects from cardiac and psychotic subjects. In these and following figures, "random" means rectangular random, "periodic" refers to sinusoidal waves, "sine" from sin(10/t) at integer values of t from 0 to 1999, "chaotic" and "biotic" from process equation [Kauffman and Sabelli, 1997], "cardiac" is the time series of cardiac beat intervals (three 3500 data samples per subject) for 9 normal subjects.  Time graphs measure time as order in the sequence (e.g, 1st, 2nd, etc) rather than as duration.  The entropy of the time series, and of the differences between consecutive members of the time series and of recurrences is plotted against the logarithm of the number of bins used in the calculation.  The table below summarizes the results presented in the figure.

 

Measure- ment	Random	Periodic	Chaotic	Biotic	Cardiac (normal)	Cardiac (coronary)	Cardiac (psychotic)
Entropy of time series   Diversity     Symmetry	Maximal     yes	Large or small     yes or not	Large     yes	Large     No	Large     No	As normal     More asym-metric	As normal   As normal
Entropy of differences   Maximal   Symmetry	Maximal   yes	Large or small   yes	Large or small   yes	Large      No	Small     No	As normal     No	Smaller than normal   Same as normal
Structure:   Recurrences rate   Consecutive recurrences   Entropy of recurrence lines	Low     Lowest     Lowest	Periodic-ally high   Periodic-ally high   Periodic-ally high	Low   Chaotic-ally higher   Low	Lower than random   Medium       Low	Lower than random   Medium       Medium	Increased       Increased       Increased	Increased     Increased       Increased

 

 

 

The process method involves ordered consideration from simple to complex components of variation in frameworks of 0, 1, 2 3, ... many dimensions. Physics constructs its theory as well as its methods of measurement upon the definition of dimensions. All processes may be expected to have simple low-dimensional components, at least unidirectional time, two values of information, a tridimensional spatial structure. To describe the organization of biological and psychological processes, it is necessary to define further dimensions. Given their complexity, one may expect that a large number of dimensions may be necessary to describe biological and psychological processes. Two powerful methods are available to investigate multiple dimensions when the only data available is a unidimensional time series: the calculation of differences between consecutive members of the series, and the embedding of N consecutive members generating vectors that may then be compared for similarity (recurrence).

 

We develop the process method via its application to the study of variations in heart rate. Here we use thirty 24 hour electrocardiographic recordings obtained from human subjects [Carlson-Sabelli et al, 1994, 1995; Sabelli et al 1994, 1995 a and b]. This choice is based on both practical and theoretical considerations. Longitudinal recordings are obtain for clinical purposes, and allow to measure accurately the intervals between R waves (RRI). Excessive regularity predicts death within 24 hours [Ewing, 1991]; clinical significance validates, and may serve to evaluate, the methodology. Theoretically, heart rate is a function of energy consumption and of emotions. Accelerations and decelerations thus provide a portrait of action (energy consumption) and (neuropsychological) information --action and information are two fundamental dimensions of processes. Our heart is our personal clock. Other time series of physical, biological, and economic processes (from Sprott and Rowlands [1995] or in the public domain) are being studied. These data are compared with computer-generated numerical series. In addition to random, periodic, and chaotic series, we include biotic patterns derived from the process equation [Kauffman and Sabelli, This Volume], which we take as a model for natural processes, and the decimal sequence of pi, as a naturally occurring, apparently random but actually determined, numerical series. Here we shall let the results speak for themselves. In Part II we shall discuss the theoretical foundations of the process approach.

 

                                                          Process Methods

Figure 1 illustrates how we combine a number of available techniques, and modify them to focus on change and creativity. Time graphs of cardiac beat intervals reveal significant differences among normal, depressed and psychotic patients, but do not define their characteristics, or distinguish between random and chaotic data. Table 1 summarizes how other techniques distinguish different patterns, and diagnostic categories.

 

Measuring entropy with a range of bins to quantify separately its two components, symmetry and diversity (figure 2): Entropy (H) is measured in units of information as defined by Shannon's equation H = - S P_i log₂ (P_i). The data is divided in bins representing ranges of values; P_i is the relative frequency within a certain range of values. Entropy is highest when the data is uniformly distributed across the bins, as it occurs with random data. The measure of entropy increases with the number of bins as a function of the diversity of the data. A series made up of two alternating values (period 2) has an entropy of 1 regardless of the number of bins, whereas the entropy of rectangular random series is log₂ of the number of bins, the maximum possible value for entropy. The slope of the entropy-number of bins regression curve provides a scale of diversity (0 to 1). The slope of the entropy-bin regression line for many natural processes, such as cardiac beats, does not differ significantly from that of random numbers. Their entropy is lower because the time series are asymmetric. The value for entropy measured with two bins indicates the degree of asymmetry of the sample: it is always 1 (=log₂2) for symmetric distributions (random, periodic, or chaotic), and it is less than symmetric for biological data and biotic patterns. This is to be expected, as multiplicative processes generate asymmetric log normal distributions [Aitchison and Brown, 1957]. In the case of cardiac data, the series is likely to be asymmetric because of the unequal action of the opposing parasympathetic and sympathetic nerves. More generally, asymmetry is a fundamental feature of natural process (see Sabelli et al [This Volume]). The distributions of cardiac beat intervals are significantly (p < 0.05) more asymmetric in patients with coronary artery disease (2 bin entropy 0.56) than in normal (0.80) or psychotic (0.71) subjects.

 

The entropy of intervals measures its range of rate. Shuffling the data to create a randomly-ordered time series with the same statistical distribution as the original time series does not change their entropy, indicating that entropy does not measure temporal order. To measure organization we quantify the entropy of differences and recurrences.

 

Figure 2. Bin-variation method to measure entropy, disclosing the contributions of symmetry and diversity of action.

 

Measuring the entropy of differences between consecutive members of the time series to quantify acceleration (figure 1):  Rapid changes in rate are induced by the action of the sympathetic and parasympathetic nerves, that convey information from brain to heart. The entropy of differences is much lower for cardiac data than for random series, indicating a degree of continuity. It is lower in recordings obtained from depressed or psychotic patients than in normals; this is due to a decrease in diversity, rather than on symmetry. In the case of random, periodic and chaotic series, the entropy of differences increases monotonically with the number of bins as observed in the original time series. In contrast, the entropy of the differences between consecutive cardiac beat intervals alternatively increases and decreases with the number of bins; this phenomenon is particularly evident in psychotic patients (figure 1). These observations indicate the usefulness of measuring differences to detect subtle changes in pattern. As cardiac beat intervals are modulated by the accelerating and decelerating influence of sympathetic and parasympathetic nerves, we interpret these oscillations as an illustration of how two-dimensional frameworks portray oppositions underlying natural processes.

 

Difference method to measure higher order components of complex processes:  Intervals measure velocity; interval differences measure acceleration and deceleration. Differences of differences, and higher order differentials may be used to study more complex patterns of organization. Regardless of the number of differences calculated, the distribution of the differences between consecutive members of a rectangular random series is also rectangular random. For many other distributions, the distribution of differences departs from that of the original series. The entropy of cardiac beats decreases with the first difference, and a plateau is reached from the fifth difference on, in normal subjects, whereas cardiac and psychiatric illness led to different patterns (figure 3). No change in entropy as a function of differences were noted with rectangular or Gaussian random distributions, Poisson's distribution, sine waves, pink noise, many chaotic systems (Lorenz, Hénon, Lozi, Ikeda, the binary shiftmap x_n+1 = 1.999.. x_n, Mackey's 3,200-dimension iterated map), the Weierstrass function, or for Brownian movement, light intensity variation from a white dwarf star, Madison's mean daily temperature, human electroencephalogram, human speech, World Price Index, and the Standard and Poor's Composite Index of 500 stocks. In contrast, other series that could not be distinguished from random data by measuring the entropy of the original series, show distinct patterns of variation of entropy as a function of successive differences.

The entropy of the difference between consecutive intervals is 1 for zigzag series. Entropy decreases with the number of differences for some chaotic maps (logistic map, the Rossler's chaotic attractor, and Sprott's chaotic systems), the devil staircase, the Van der Pol equation, and also in recordings of Pacific ocean currents, for the price of gold, silver, crude oil and corn, and for the exchange value for 7 different currencies. Entropy increases with the number of differences for Cantor's dust, and for the four literary texts tested.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Figure 4. Recurrence time graphs and embedding plots differentiate complexes from simpler periodic, chaotic and random series. 3500 data points. Radius 0.1%. Recurrence time graphs constructed with 480 embeddings. Chaos from logistic equation.  

 

Recurrence complexes as portraits of pattern: Measuring differences allows one to detect relatively large changes, i.e a few dimensions of complexity. One may reasonably expect that processes are influenced by a large number of factors, that may not be readily detected in this manner. The recurrence method of provides an useful tool. Recurrences are measured with the method developed by Webber and Zbilut [1994]. Starting with each datum, vectors on N consecutive data are constructed, and compared to each other; N is the number of embeddings. When the Euclidean distance between two vectors is smaller than a preset radius (0.1%), a recurrence is counted. When recurrences are plotted as a function of time, one can detect patterns which are similar in biological data and in biotic patterns, while markedly different from periodic, chaotic and random data (figure 4). These similarities and differences can also be demonstrated using wavelet transforms and many other methods. Cardiac data never are stationary. Instead, cardiac beat intervals are organized in "complexes" separated by interruptions of recurrence. These complexes correspond to behaviors and emotions [Sabelli et al, This Volume]. They are patterns imposed upon simpler cardiac activity by the more complex brain activity. Complexes are transient, high dimensional, and have low recurrence rate, in contrast to attractors that are stable, lower dimensional, and have a higher recurrence rate. Attractors are low dimensional patterns intrinsic to the process, rather than imposed by more complex processes. Generalizing these ideas, we conceive of complexes as the most common form of patterning observed in nature, whereas attractors are useful models only for stable processes, which are rare.

 

Embedding plots as a measurement of pattern: Entropy measures information in bits; organization may be quantified by recurrence measurements. The percent of recurrences is a function of periodicity and chaoticity in the data. This can be detected only by varying the number of embeddings (embedding plots, figure 1, three right columns, and figure 4). Periodic series have high recurrence only when the number of embeddings coincide with the period. Chaotic series have high recurrence at low embeddings and then recurrence varies chaotically. Cardiac data show a specific form: an initial decrease in recurrence values is followed by an increase, with the inflection point at 6-8 embeddings; we interpret the initial higher values as representing the similarity between consecutive heart beats. The recurrence method provides several other measures of pattern. A most important one, just beginning to be explored, is the existence of specific forms that constitute a sort of alphabet that can be observed in a wide variety of processes.  In the case of cardiac data, some of these forms appear to be associated with certain emotions and behaviors [Sabelli et al, This Volume]. The percentage of consecutive recurrences can be calculated; Webber and Zbilut call it determinism, but we do not follow this usage, because patterned processes need not be determined (see below). Consecutive recurrences can be divided in bins according to the length of the lines (2, 3, 4,... m recurrences) and the entropy of these lines of recurrences can be calculated (recurrence entropy). Patterned data, periodic, biological, or literary, have a larger proportion of consecutive recurrences and a higher recurrence entropy than random data. Correspondingly, shuffling these data reduces their recurrence entropy, and the percentage of consecutive recurrences. Random data has 0 net recurrence entropy, whereas periodic and chaotic series as well as natural processes and biotic patterns from the process equation, music-like pink noise, have higher recurrence entropy than their random shuffled copies, indicating pattern.

 

Recurrence as an empirical measurement of novelty (figure 5): The number of recurrences increases with the number of embeddings for random series and for cardiac data. We thus calculate the recurrence rate of a time series by subtracting the percent of recurrences calculated after shuffling the data from the percent of recurrences in the original time series. Random data has 0 recurrence rate. Periodic and chaotic [logistic, Lorenz, Henon, Ikeda] series have more recurrences than their random shuffled copies, indicating order. Time series of natural processes, biotic patterns from the process equation, music-like pink noise, certain chaotic series (Rossler, Kauffman), the Sarkovskii's series, and the decimal sequence of pi, have less recurrences than their shuffled copies. Cardiac data, biotic time series, and the decimal sequence of pi, have a lower percentage of recurrences than random data up to extraordinarily high embeddings (figure 6). As a recurrence is a repetition of pattern, a lower than random recurrence rate indicates novelty.

 

Multiple measurements of complexity: A number of methods are available to measure the complexity of processes. Measuring the capacity and the correlation dimensions using the programs developed by Sprott and Rowlands [1995], the dimensionality of cardiac data varies between 4 and 6. This is the same order of magnitude of dimensionality as suggested by consideration of changes in entropy as a function of the number of differences (figure 3), as well as of the inflection point in the embedding plots (figures 1, 4) at which the number of recurrences attains a minimum. The recurrence method provides several putative measures of complexity. We measure the median embedding dimension, ie. the embedding 50% of recurrences are consecutive. It is approximately 50 for normal hearts, and much lower for coronary or psychotic subjects. There is a third measure of dimensionality that could be calculated by considering at what embedding the sample has 50 % recurrences or attains 50 % of maximal entropy. The results indicate a still higher dimensionality for cardiac data, in fact higher than for random data. We are exploring the possibility that each of these various dimensions capture one aspect of the processes under consideration.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Embedding-variation method -maximal entropy: As the number of dimensions of biological data may be expected to be much higher than physical processes and low dimensional attractors, we explore a wide range of embeddings. The mean distance between recurrences is lower for cardiac data than for random series at low embeddings, and higher at high embeddings. Extremely high embeddings and large radii are needed to demonstrate that random data have larger entropy than other data (figure 6). In comparison to random data, cardiac data (and biotic patterns generated with the process equation) have relatively high entropy at low embeddings and relatively low entropy at high embeddings. The interpretation of such data is not obvious. One is first tempted to question the validity and reliability of using large number of embeddings --Webber and Zbilut specifically warn against such usage of their method. Yet the fact that even at very high embedddings cardiac data has much less recurrences than random indicates that high embeddings do not abolish significant differences by turning everything into a recurrence. Further, high embeddings reveal the expected high entropy of random data, and the surprising lower entropy of pi's decimal sequence, neither of which is apparent at lower embeddings. While keeping in mind these reservations in the interpretation of the data, we also speculate that the higher-than- random recurrence entropy of cardiac data at low embeddings, and their lower-than-random entropy at higher embeddings may portray the ability of biological processes to produce entropy faster than their environment, as proposed by Swanson, yet to keep their internal entropy below that of their surroundings as proposed by Schrödinger and Prigogine. This is discussed by Sabelli et al [This Volume]. 

 

In summary, we have illustrated the practical usage of process methods, hoping to whet the readers appetite to develop some of their own. In Part II we shall discuss process thinking as a foundation for this and other methods, and focus on the study of coexisting opposites.

 

Acknowledgements:  This work was supported by the Society for the Advancement of Clinical Philosophy. The authors are deeply indebted to Drs. C. Webber and J. Zbilut for the use of their programs.

 

Aitchison, J. and Brown, J. (1957) The Lognormal Distribution Cambridge: Univ. Press.

Carlson-Sabelli, L., Sabelli, H.C., Zbilut, J., Patel, M., Messer, J., Walthall, K., Tom, C., Fink, P., Sugerman, A. and Zdanovics, O. (1994). How the heart informs about the brain. A process analysis of the electrocardiogram. In R. Trappl (Ed.), Cybernetics and Systems`94. (pp. 1031-1038). Singapore: World Scientific.

Carlson-Sabelli, L., Sabelli, H.C., Patel, M., Messer, J.,  Zbilut, J., Sugerman, A., Walthall K., Tom, C. and Zdanovics, O. (1995). Electropsychocardiography. Illustrating the application of process methods to comprehensive patient evaluation. Complexity and Chaos in Nursing,  2, 16-24.

Ewing, D.J. (1991) Heart rate variability: an important new risk factor in patients following myocardial infarction. Clin Cardiol 14:683-685.

Sabelli, H.C., Carlson-Sabelli, L., Patel M., Levy, A. (1995a). Anger, fear, depression and crime. Physiological and psychological studies using the process method. In R. Robertson and A. Combs (Eds.), Chaos Theory in Psychology and the Life Sciences. (pp. 65-88). Mahwah, New Jersey: Erlbaum.