ENTROPY DEFINED: It is defined as the average amount of information provided by each event that occurred. Generally, we look at event of occurring or not occurring, thus making the event occurrence in a form of discrete variable. The occurrence of event may be expressed as the probability: Pi: (p1, p2, ..., pn) for a set of event Xi; (x1, x2, ..., xn), i.e. for event 1 the probability is p1x1, etc.

Since we classified the event occurrence into "yes occurring" and "not occurring", the data is discrete. The probability function used for a set of observation is called the probability mass function (PMF) or P(X)---compared to cumulative distribution function (CDF) in continuous events observation.

Entropy is formally given as:

H(X) = E[-lnP(X))]

... where E = expected; ln = natural log base e and P(X) = PMF of the set.

As defined above, entropy is the average information given by the event, it implies that the event occurrence must be categorical and random. if the event always occur, thus a constant state, such condition does not produce information, i.e. the sun always come up in the morning and sets in the evening. However, if the event's occurrence and non-occurrence happens in unpredictable manner, i.e. random, then when the event occurs, it provides us with information, i.e. information about its occurring. Information obtained from such PMF has the following properties:

I(p) > 0 or = 0. The information must not be negative. As explained above, if the sun always rise and set at predictable time, not much information could taken from that.

I(1) = 0. Events that always occurs (i.e. sun rising every morning) does not give information.

I(p1, p2) = I(p1) + I(p2). information obtained from independent events in the same set is additive.

ENTROPY APPLIED: In the stock market, assume that we accept the fact that the stock price movement is random or at least not easy to predict, i.e. we do not know certain if it will rise or fall, and indeed it does not always rise, thus, it is not I(1). Since it is not I(1), the event of price rise (which does not always happens) does provide us with information. How much information? As much as H(X). Does stock price change in the stock market? Generally, Yes but it is also possible that on some days the price does not change. Thus the condition that I(p) > 0 is satisfied when the price changes and I(p) = 0 is also satisfied when the price does not change. However, this alternation of change and no change must happens, i.e. be dynamic otherwise the condition breaks down to I(1) = no information.

In a stock portfolio comprising of 2 stocks: s1 and s2. Each stock has its own price. The movement of each price is independent of one another, and each is considered a random variable. If neither s1 nor s2 is static, i.e. their price changes, up or down. We can determine the probability of each and that the probability of both is additive because they are the same kind and are in the same market. If each daily closing price is considered the event. If we track the event series over a period of 5, or 10 days, we can determine the PMF or P(X) for each stock. According to condition 3 above, their probabilities are additive: p(s1) + p(s2).

REFERENCES: See Shannon's seminal article on entropy.

Entropy is also a measure of uncertainty in in Information Theory. It is defined as the minus expected log probability for discrete processes, and as minus expected log density for continuous random variables. The log is usually base two.

To get the basic idea, see http://en.wikipedia.org/wiki/Entropy_(information_theory). For a deep understanding of the information theory, I suggest to read book Cover, T. M., Thomas, J. A. (2006), Elements of information theory, but it is not about economics.

The metric entropy is capable of detecting nonlinear dependence within the returns series, and is also capable of detecting nonlinear “affinity” between the returns and their predictions obtained from various models thereby serving as a measure of out-of-sample goodness-of-fit or model adequacy.

ENTROPY DEFINED: It is defined as the average amount of information provided by each event that occurred. Generally, we look at event of occurring or not occurring, thus making the event occurrence in a form of discrete variable. The occurrence of event may be expressed as the probability: Pi: (p1, p2, ..., pn) for a set of event Xi; (x1, x2, ..., xn), i.e. for event 1 the probability is p1x1, etc.

Since we classified the event occurrence into "yes occurring" and "not occurring", the data is discrete. The probability function used for a set of observation is called the probability mass function (PMF) or P(X)---compared to cumulative distribution function (CDF) in continuous events observation.

Entropy is formally given as:

H(X) = E[-lnP(X))]

... where E = expected; ln = natural log base e and P(X) = PMF of the set.

As defined above, entropy is the average information given by the event, it implies that the event occurrence must be categorical and random. if the event always occur, thus a constant state, such condition does not produce information, i.e. the sun always come up in the morning and sets in the evening. However, if the event's occurrence and non-occurrence happens in unpredictable manner, i.e. random, then when the event occurs, it provides us with information, i.e. information about its occurring. Information obtained from such PMF has the following properties:

I(p) > 0 or = 0. The information must not be negative. As explained above, if the sun always rise and set at predictable time, not much information could taken from that.

I(1) = 0. Events that always occurs (i.e. sun rising every morning) does not give information.

I(p1, p2) = I(p1) + I(p2). information obtained from independent events in the same set is additive.

ENTROPY APPLIED: In the stock market, assume that we accept the fact that the stock price movement is random or at least not easy to predict, i.e. we do not know certain if it will rise or fall, and indeed it does not always rise, thus, it is not I(1). Since it is not I(1), the event of price rise (which does not always happens) does provide us with information. How much information? As much as H(X). Does stock price change in the stock market? Generally, Yes but it is also possible that on some days the price does not change. Thus the condition that I(p) > 0 is satisfied when the price changes and I(p) = 0 is also satisfied when the price does not change. However, this alternation of change and no change must happens, i.e. be dynamic otherwise the condition breaks down to I(1) = no information.

In a stock portfolio comprising of 2 stocks: s1 and s2. Each stock has its own price. The movement of each price is independent of one another, and each is considered a random variable. If neither s1 nor s2 is static, i.e. their price changes, up or down. We can determine the probability of each and that the probability of both is additive because they are the same kind and are in the same market. If each daily closing price is considered the event. If we track the event series over a period of 5, or 10 days, we can determine the PMF or P(X) for each stock. According to condition 3 above, their probabilities are additive: p(s1) + p(s2).

REFERENCES: See Shannon's seminal article on entropy.

Thank you soo much prof. Paul Louangrath

@ Paul Louangrath  

ENTROPY APPLIED: In the stock market, assume that we accept the fact that the stock price movement is random or at least not easy to predict,

The above assumption is unsubstantiated. There is no need to make an assumption if you have the fact. Do you have the fact?

That is having found that the stock price movements are non-random what would be entropy applied? Entropy is defined in the context of a probability model. What makes you believe that such a model in applicable to stock markets? What would you do with entropy calculated your way?

Are you shure you are taking into account ALL information existent in the system?

What about your calculations when I show you that you miss A LOT OF hidden information in the system? 

Stock market is a complex system, and complex systems tend to be multidimensional, and difficult to model. In specific circumstances, they may exhibit low-dimensional behaviour. Barabasi and Buchanan said that if the system is forced to undergo a phase transition then power laws emerge - nature's unmistakable sign that chaos is departing in favor of order. I can show that stock markets undergo frequent phase transitions (marked by UPOs) making it essentially prerogative to study that order hidden in the self-organized structures of deterministic chaos, yielding ordered relationships and showing their ubiquity in full ripe. Your entropy model lacks the urgency of understanding why something disastrous happens. A small or a large perturbation in complex systems like stock markets can trip a major collapse, unexpectedly, dramatically, and resoundingly.  SOC, power laws and randomness seem to be a common property of both natural and human-made catastrophies. I insist on playing down the role of randomness in stock markets for phase transitions have names and fat fingers behind who prefer that the public dies without any knowledge of the same. That part unknowledgable is known as randomness. I do not ignore randomness completely, yet it is most evident on microscopic levels.   Let us not be fooled by randomness itself. 

Entropy is the measure of disorder. 

Look at the chart of the WIG20 index based on M30 candles. Note that it begins early January and covers roughly a period of two months. It ends on last Friday close.

The trigger down (causet) emerged in January and it is still active. I singled out the major causet and followed its trajectory until now. What you have got here is a fractal expansion, squeezing, folding and stretching. The Feigenbaum constant 4.669 and Feigenbaum alpha 2.5029 are shown there and also 8 UPOs.

The 144-dimensional embedding was used to retrieve the elements of the inner built memory. I am shure you have never seen an evidence of the two Feigenbaum constants in stock markets. Also note that they do not refer to one class of maps. But I will not delve into the problems of pure maths. 

So you judge for yourself how much order or disorder is there, I hope it helps to understand what entropy is in chaotic systems and whether we really need to calculate it. Those UPOs are the determinants of the system which is bound to return to them in future with certainty. The order of returns and the timing are unknown. 

Another example is ready from NZDUSD H4 (foreign exchange market). 

You want OIL? Or silver? Bonds? Dow Jones Industrial Average? S&P 500?IBM? Coffee?

Yeah, it is time for coffee. 

My compliments Paul ! The problem now is related to which type of coffee we (people) are invited to! The question you pose is the key, letting everybody of us  to quantify or qualify the extent of order or disorder in the economic system represented via WIG20 index based on M30 candles or NZDUSD H4 (foreign exchange market) or, or or...

I remember that a few weeks ago a colleague posed in the Q&A session an intriguing question on becoming rich based on mathematical , financial and economic knowledge.

Let me escape then from the "logistic of money" that are behind Feigenbaum's constants and let me consider the "logic concepts  behind money" that are now having effect on the economic system : too many in the financial sector  are trying to take the Coffee for free and therefore the economic system is seriously unbalanced! 

We were discussing about these themes a few weeks ago......and  the practice of huge Quantitative easings  , that may finance also the amount of OTC Derivatives outstanding,  is part of it!

Kind regards 

Alberto

Dear Alberto, 

There is a slight opposition of mine to offering good recipies for becoming rich on account of being knowledgable. Taleb has been trying to distance himself even from his own trading past postulating restraint and disdain for greedy fat fingers. The only logic about money is money itself. Let us think about it that it is always just an option. I have not come across that question you refer to. I bet (what a dirty word) that becoming rich is not a primary issue for mathematicians (exit polls). I do not want Feigenbaum to be thought of something from the casino-like environment only. But ask anyone who has ever stained his hands with trading. He would inevitably enquire how much can be made on such things now, or on Monday or on any given Monday, Tuesday, Wednesday, Thursday and Friday in the future. This is not the central part of entropy.but whoever raises the question about entropy, I become curious about what it is needed for.

Regards

Paul    

Dear Paul,

    I fully appreciate your position. The present discussion is related to scientific concepts and this is the purpose. However it is natural to make some comments on the present and complex economic  situation.

Personally I am keen to look to economic phenomena from the value side but  I do consider also the financial one. If there is an unbalancement between economomic and financial side, we should assume that the entropic level of the system increases.

I try to explain better the concept :  in the only paper I have listed here,  in my RG profile,  with the main scope  to introduce our Centro Studi , Applied Research Organisation, a Colleague of mine, Egidio Cascini , tried to emphasize the role of R&D within a Company ( e.g. within our Parent Group) to increase the Hentalpy and minimize the system Entropy , in an analogy tending to give a strategic parameter in the valuation of this type of investments.

Actually, I have to tell you that even if we do not make this type of scientific considerations "only" for money , at the end of the game, the resulting " applied "effect , in decisional and operational terms, becomes  also measurable in value terms.

There is no "sin" in becoming richer then others, if  this comes out from a scientific understanding  of the economic situation. The lesson of J.M. Keynes does not end  in the only development of economic theories but  might be extended  also to the fact that via that knowledge, he was understanding and investing, in a wise and succesful way, or " better the peers"!

Regards

Alberto

Dear Alberto,

I keep wonderful memories of Buccino where we rented a rustic 19th century marvel house hidden among very old olive trees and spent there some time with family some years ago. The place gave us unmeasurable value for time.  It should never have ended. What a coincidence now. Actually I believe less in coincidence more in fate. Our virtual meeting is a token of decreasing entropy, absolutely so. My mind has returned to the Unstable Periodic Orbit of memories of excellence. I should ask what was the probability of Alberto from Buccino responding to my post...    

Regards

Paul      

Dear Sultan Khan

Please be not confused by the term "entropy" appearing in such differing contexts. The interpretation of this concept in physics is usually quite different from its interpretation in the stock market example. In the statistical description of thermodynamics, however, the information entropy and the physical entropy can be shown to be intimately related.

You should note, that entropy is only defined for (large) ensembles, not for single particles or single events. Paul's definition above is correct, but the interpretation needs to be carefully read: it is the AVERAGE information of a single event. Averages only make sense, when there are a lot of events with a defined or measurable probability. I therefore prefer the equivalent notation of Shannon's entropy:

H(X) = - SUM {p(xi)log(p(xi))}

The sum runs over all events xi, and p(xi) is their respective probability of occurence. The base of the log can be freely chosen, usually in information theory one choses the base 2 and the unit of entropy (and that of information content) is called "bits".

Also, your understanding of physical entropy is not quite correct: entropy does not measure "disorder", since that is primarily a anthropocentric term. Physical entropy, in the statistical interpretation,  measures the number of possible microstates of a system when the macroscopic variables (temperature, pressure, etc) are given. When heat is added to a gas, for example, there are more possible configurations of the system for the same macroscopic state and thus the entropy increases. If you know what your are saying, you can use the term "disorder" here, but it has very little to do with the disorder you find on your desk.

I am not an expert on stock markets, but from what I understand entropy is used here to measure the uncertainty in the development of a stock portfolio. An investor would try to minimize this uncertainty (and at the same time maximize the returns). The entropy of the distribution of the stock option prices can serve as a measure of this uncertainty. Maybe others can help better with the interpretation of entropy in stock markets, but here is a fairly comprehensive summary: 

Applications of Entropy in Finance: A Review

Rongxi Zhou, Ru Cai and Guanqun Tong

http://www.mdpi.com/1099-4300/15/11/4909

Kind regards, Stefan

See also:

http://iopscience.iop.org/1742-6596/394/1/012033

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.27.1423

http://arxiv.org/pdf/1106.4957.pdf

The entropy of the distribution of the stock option prices can serve as a measure of this uncertainty. 

Actually the key point about stock markets is their volatility, not uncertainty. It has been argued that volatility is quite wrongly understood by theorists who never invested a penny in any market - volatile or non volatile. Contrary to common myth a volatile market is not more risky than a non-volatile one. One should avoid some risk metrics. Conventional metrics adjusted for large deviations, don't work. Let us not confuse absence of volatility with absence of risk. Conventional metrics using volatility as an indicator of stability fool us, because the evolution into catastrophic fractal expansions is marked by a lowering of volatility - and a greater risk of big jumps.This has fooled Greenspan and Bernanke and Yellen - as well as the entire banking system and a lot of intermediaries, like currency brokers (Alpari UK went bust on January 15 in the matter of minutes, some other brokers across the globe suffered severe losses).  Are we looking for entropy or for second-order effects? One needs to have "skin in the game" in the event of harm caused by reliance on his information or opinion. Something to lose from it, given that others rely on those opinions. The problem is that society is currently granting academizing academics, talkers and political politicians a free option without taking personal risks when they expose others. Stiglitz and the Orszag brothers looked at Fannie Mae and assessed that "on the basis of historical experience, the risk to the government from a potential dafault on GSE debt is effectively zero". What did they measure? Entropy? Uncertainty? Sigma?    

As a measure of uncertainty and risk, standard-deviation is very popular. It has long been recognized that it is not fully satisfactory due to the fact that it is severely affected by extreme values. 

As a measure of disorder, distance from equilibrium, entropy might present some advantages. But why are we so glued to the concept of equilibrium?  The Tsallis and the Shannon entropies: the results from both entropies have shown nonlinear dynamics in the volatility of stock market indexes. Ha! Then focus on those nonlinearities at last if you want to understand why and how social and economic evolution takes place by surprises, discontinuities and jumps in quite a nasty fashion.

We specially address the concept of entropy as an alternative to the standard deviation since it can captures the uncertainty and disorder in a time series without imposing any constraints on the theoretical probability distribution. The problem is that we do not need to know the number behind the rainfall in order to drive a car in rain. We already know that the area is rainy in general and that stock markets are non-linear systems.   Actually now comes the time for applying the theory of chaos because we deal with open systems which are far from equilibrium.  

A weak hand is someone who doesn't know his fragility and is killed by a false snse of security. Then comes the moment when many such weak hands rush to the door in stampede. A volatile market doesn't let people go such a long time without a cleanup, preventing big market collapses. Fluctuate, float but do not sink. When your entropy is finally a number it does not mean that the stock market is "less" non-linear or "less" chaotic at times and "very" non-linear or "very" chaotic at other times. It will show some intermittancy. But let us not be fooled by systems stuck in a dangerous impasse. Some perturbation can unlock them and set them free. Absence of perturbations equals guaranteed death. Presence of perturbations and fractal expansions, squeezing and folding means life.    

Dear Sulthan,

Check my latest paper about this topic, published in dec 2014: https://www.researchgate.net/publication/270221149_Entropy-based_financial_asset_pricing

Bests,

David

Article Entropy-Based Financial Asset Pricing

The paper, in the link below, tries to quantify a qualitative claim that " low entropy regions lead to improved prediction of stock markets". My regards.

http://www.phys.sinica.edu.tw/~socioecono/econophysics2010/pdfs/ZapartPaper.pdf

Entropy: A concept that is not a physical quantity

https://www.researchgate.net/publication/230554936_Entropy_A_concept_that_is_not_a_physical_quantity