Everyone who goes through the BSchool /CFA rut knows about mean variance optimization . Some of them might know that it does not work in the practical world. So, what if noble prizes have been awarded!! It doesn’t work in practice…Like all models, however, it does give a framework to think about and nothing more than that….So, What’s the alternative ? If you are given a couple of assets and are asked to create an asset mgmt plan, how would you go about it ? Pick up any random Bschool student and ask this question and I am certain that only a few of them would be able to answer coherently..Why ? Finance education is stuck in gaussian world!! anyways, the point of this post is to summarize this book which suggests alternatives. I came across a reference to this book in a paper and was curious to know more about it, almost a year ago. Was intending to use Bayesian estimates if ever I had to solve a portfolio allocation framework. This weekend , I had decided to take a look at this book out for sheer curiosity. What I was found in this book was a fascinating critique of classic MV solution and more than that, a statistically solid approach to the MV optimization problem.

Never bothered to think about this aspect until today, when the choice of the format made a ton of the difference to the output!

Via C# Corner :

First off, why use PNG instead of GIF? Probably the most important reason is that GIF supports a maximum of 256 colors. The second reason is Unisys has a lockdown on the rights to LZW compression which the GIF format can, and usually does use. PNG overcomes the color depth hurdle by providing up to 48Bpp (bits per pixel). Just as importantly, the PNG format is patent-free and available for use by anyone.

 Numbers do not lie, but they have the propensity to tell the truth with intent to deceive

-- E. T. Bell

It has been 2 months since I have coded a single line of C# . Today I have to use it again for some reason and I seem to have forgotten everything about it. My short term memory is killing me !.. Every once in a while I face this problem. A month ago, I forgot ruby syntax. It took me almost a day to get back to speed. A few months ago, same thing happened with PostgreSQL syntax. 3 months back with VBA Syntax!!

It has been more than one and half months since I have read any book. Was involving in developing some stuff @work, so was completely immersed in it. Decided to read this book on a beautiful Saturday.

Came across this book in an NY times article and was looking forward to read the book, as it was relating to an aspect which is close to my heart, hacking.Hacking in geek’s jargon is ripping apart software, packages, libraries, things and understanding stuff at level 0. In some sense,I guess there are two types of people, first kind who do not want to get their hands dirty a lot and would rather assemble, manage a set of people to accomplish some task, second kind are those who want to hack things and build things from scratch. By the kind of education I have undergone, I should be doing the former but I have found joy in the latter activity. In that sense, I could relate to the author was a Chicago Phd , academic, whom one would expect to write papers, do research and be a professor. Instead he chooses to become a mechanic. Why ? This book answers that question and much more. Author has organized the book in such a way that it traces author’s experiences from being a young apprentice in a repair shop , to being a PhD from Chicago, to working in a cubicle, to finally coming round the circle to repair shop again.

Via Jayanth:

A real hero never says anything without thinking and never thinks after saying

We are drowning in information and starving for knowledge.

–Rutherford D. Roger

Via Jayanth:

The relationship between two members of a family should be like fish and water,

but its should not be like fish and fisherman.

Via ZenHabits :

Have a disconnect time each day. It’s like setting office hours if you’re a professor — you set the times that work best for you, and you can even let people know about these times. Let’s say you are disconnected from 8-10 a.m. each day, or 4-5 p.m., or even anytime after 2 p.m. Tell people your policy, so they know you won’t be available for email or IM. And use this time to create.

“Human beings don’t like to trade against human nature, which wants to buy stocks
after they go up , not down”

_
- Nunzio Tartaglia (Pioneer of pairs trading)_

Even though this technique is Stat arb 101, the advantage of this method is designed more to exploit the investor’s over-reaction to the events.

The race is NOT always to the swift, but to those who keep on RUNNING

- Nike Caption !

I started blogging a few years back, a big thanks to my friend TP who motivated me…. Today Typepad had a message that I have blogged my 1000_th_ post. Did blogging help me? Its a resounding yes because I have understood things better once I had written them down, most relevant in this case are book summaries.

To Be of Use
- Marge Piercy

The people I love the best
jump into work head first
without dallying in the shallows
and swim off with sure strokes almost out of sight.
They seem to become natives of that element,
the black sleek heads of seals
bouncing like half-submerged balls

I love people
who harness themselves, an ox to a heavy cart,
who pull like water buffalo, with massive patience,
who strain in the mud and the muck to move things forward,
who do what has to be done, again and again

It has been almost 2 years since I have read any fiction book. Had to go to a nearby mall to get my weekly groceries, when I saw this book. Did not have any great inclination to pick up such a book but did an impulsive purchase .On Sunday evening, felt like taking a break and read this book in a few hrs time. Well, the book is a work of fiction where the protagonist meets his dead mother and has conversations with her , just when he decides to commit suicide. This book is a gentle reminder to readers that one has to take time out and be with their mothers,because not doing it consciously might result in a day , when they realize that its too late and they can never go back in time and make their mothers happy.

Nice points about “working out” via Zen Habits

Exercise:

• Do it 4 times per week for at least 30 minutes. Seriously, doesn’t matter. Basketball, jogging, weights, push ups, etc. Unless you have a specific goal just make sure to get it done!
• The two day rule! You can’t miss 2 days of working out in a row. You can skip one day. But not the next!
• Don’t let the “Exercise Monster” build up. When I don’t do laundry for a while it becomes what I like to call the ‘Laundry Monster’. It builds up in my mind and I avoid doing it at all costs. Until finally, I have to. And it’s never as bad as I think it’ll be! Don’t let it happen to you. Follow the two day rule!
• Don’t focus on exercising at a certain time. Just focus on making time to exercise!
• Make it fun! Listen to music; join a nice gym; etc. Bonus: If you love working out with music as much as I do, only listen to your favorite music while you workout. I’ve actually exercised just to listen to my music.
• Focus on how you want to feel. We always feel energized after we workout. Yet, we usually skip exercise when we’re feeling tired! Remember: Exercise will give you energy. Want more energy? Exercise more!

Recently I came across a nice illustration which captures the work of a data analyst. Each of the tasks mentioned below requires a specific set of skillsets and if one can work on learning at least 30-40% of skills from each of the disciplines mentioned, then data analysis becomes really interesting. One can walk through the entire task list and subsequently hope to understand things and probably build some useful models.

**
“First we make our habits, then our habits make us.”
**
- Charles C. Noble

Via NYTimes :

But as I traveled the land, the data did not fit the framework. The children of the lower castes were hoisting themselves up one diploma and training program at a time. The women were becoming breadwinners through microcredit and decentralized manufacturing. The young people were finding in their cellphones a first zone of individual identity. The couples were ending marriages no matter what “society” thinks, then finding love again. The vegetarians were embracing meat and meat-eaters were turning vegetarian, defining themselves by taste and faith, not caste.

It’s Saturday morning and it’s pouring in Mumbai. A perfect setting for a cup of hot tea and a book. Simple pleasures of life:) are sometimes all that one needs, I guess.

Was planning to read this book for quite sometime as it was touted as a book similar to “Against the Gods” and “Lady Tasting Tea”, books which I thoroughly enjoyed. After recovering from a brief illness this week, I thought I should spend my time peacefully with a book and managed to read this book in about 2 sittings. Let me summarize this book:

Here is a TED talk by Dr.Arthur Benjamin ( Mathemagic ted talk person) who says, that stats & probability should be elevated to the status that is given to calculus

I was familiar with this technique, had used it in a couple of places. This week I had a chance to apply this technique to a problem at hand. However, this time I did not want to rush in . Had in my archives an old book called “User’s guide to Principal Components”. Took some time out to know a bit more about the technique. As always, there is always some new learning by reading old stuff.

Parsimony is desirable but not always obtainable

- Edward Jackson (PCA Practitioner)

I guess it’s true sometimes, about of our lives too !.

With easier access to raw computing power, one might think that all it takes is to analyze data and let the data throw hypotheses.

Is theorizing over in this data intensive world ? Has the world become so complicated that theorizing is useless? May be not.

I read this short note(Tom Swift and His Electric Factor Analysis Machine) couple of years back but I always reread it frequently so that I dont make the mistake of living only in an empiricist world. Some amount of apriori assumptions/ hypotheses are required. In the financial world, traders provide this knowledge. It is easier sit at a desk and crunch data and build models. However it might be a complete useless exercise if you do not marry apriori assumptions with your empirical work.

Via NYTimes :

Robert Gentleman & Ross Ihaka (people behind R)

“R is a real demonstration of the power of collaboration, and I don’t think you could construct something like this any other way,” Mr. Ihaka said. “We could have chosen to be commercial, and we would have sold five copies of the software.”

“The great beauty of R is that you can modify it to do all sorts of things,” said Hal Varian, chief economist at Google. “And you have a lot of prepackaged stuff that’s already available, so you’re standing on the shoulders of giants.”

Often times most of the data / variables that we come across appear random and we start taking decisions based on gut. If gut works , its great. However before bringing in the gut part , it might be sensible to stretch a bit and look at data carefully and examine the details.Some time, just by changing the our view, we might see patterns. well PCA is all about that. I don’t have to go that far to make this point. For example, if one looks at the current graph of sin(x) for some values of x, it looks like this:

In God we trust; all others bring data.

—Dr. W. Edwards Deming

Paul Graham on Google :

Their hypothesis seems to have been that, in the initial stages at least, all you need is good hackers: if you hire all the smartest people and put them to work on a problem where their success can be measured, you win. All the other stuff—which includes all the stuff that business schools think business consists of—you can figure out along the way.

**A Random sample of my thoughts
**

1. If Nifty falls / Nifty rises , does the implied vol move in the same direction of the underlying or does it move the opposite direction. This becomes a very important question in the context of choosing a local volatility idea or chucking such an idea ? Local vol models assume that underlying and smiles move in different directions!!
2. As there is a short sale restriction on stocks, what exactly is a tradable underlying for Nifty option, for valuation purposes ? Yes you do hedge a long call option with whatever is a cheap, a future/ETF. But then is it sensible to treat a long call option as option on futures and put option as an option on index, the reason being that that they are the respective hedges for the options available
3. Does vol of nifty determine the atm imp vol level for nifty options? If so, how does it move ?
4. Is there a curvature for vol across strikes ? If so, does the curvature remain quasi-stable?
5. What is the functional/deterministic/stochastic relation between the comovement of nifty futures and nifty options in a 20,100 seconds(9:55AM - 3:30PM) trading day ?
6. What is the average trade volume per moneyness in a trading day ? This is easy…got to crunch a few numbers to get an idea
7. World-over the realized vol is explained by atm variance largely(80%)? Is it the case in Indian markets ? Are skew and convexity merely jazz in Indian markets which one need not worry too much ?
8. If you look at an option trade that happens at time t1, what would be the corresponding future price that you would consider for valuation? the instantaneous futures price, average of future prices in the interval (t1-delta, t1)? Can you choose delta with some rationale, meaning, some metric which justifies choosing last 5 min / last 10 min / whatever that one thinks is meaningful ?
9. Parameters for Vol of Vol for Nifty - Are they stable and more importantly what is their impact on Imp vols
10. Is imp vol in Indian markets a completely random animal ? Is Indian market completely a punter’s market , meaning every one is a momentum player ?and analyzing numbers beyond a point is a useless activity ? ( I hope not )
11. Are circuit breakers, fancy as they might sound, properly designed? Shouldn’t financial markets be there for the price discovery to happen? How can the third busiest exchange in the world trade for 30 seconds (Monday,after election verdict) and let the order flow go to other stock exchanges ?
12. Are Indian investors sensible ? Nobody wants to buy when nifty was 2600 level and now when it has reached 4300 , every one wants to buy? Shouldn’t it be the other way round ?
13. Who makes money in the entire game ? Are the only people who make money are Custodians, Fund account management fees, brokerage houses, financial advisors? Is “extracting decent alpha consistently”, a fantasy and there is survivorship bias amongst the successful fund managers that we see around ?
14. How do people trade reliance options or for that matter any of the American options on the Indian market? Are there good pricing and hedging tools for these options available with traders ?
15. Gap up and Gap down!!! , the biggest problem in the Indian markets – How do you sleep well when you know that you have got to deal with such an issue the next day morning ?

…………………………..May be I will find clues to the above stuff, someday!

Handling large amounts of data in R is tricky as R typically loads the entire dataset in to RAM. While this means that computations are going to be very fast, it also means that dataset that can be used for analysis is dependent on your RAM. One solution which I stumbled on was Filehash which seemed to offer me a solution than the usual way that I was going about. I was getting the entire stuff in postgreSQL and then doing computations using R.

Jay Walker @ TED shows evidence of the english mania. This year, China is going to be the country with the largest number of english speaking people. Mass chanting of english sentences is something which I have never heard of, till date!
But in China…

As we all know, every kind of work involves a definite amount of tedious, unpleasant,low-mid IQ work.I don’t subscribe to these labels though.

In the life of a data analyst, the data preparation and data cleansing is something which has to be done before he/she can even think of doing any sane analysis. So, how does one approach data prep and data cleansing? Is it something to be looked down upon ? Well, I was under that impression many years back. My view changed completely when I took the effort to understand the complexity of data prep and data cleansing phases. Also, I began to develop this feeling that working on these phases gives a tremendous amount of insight that no modeling on the cleansed data would give. Here is a fact which is interesting : Renaissance hedge fund employs around 25 top notch PhDs to work on data treatment such as missing and capping treatment, outlier effects etc. No wonder this amount of care at the initial stages translates to tons of insights in to the arb opportunities in the market. Many years working on databases, I have kind of developed great love towards it. One of the things which is a pre-requisite to successful modeling is data prep phase. Working on data, which to this day, many people have an aversion is a fascinating place to start the analysis. However I am amateur in terms of the possibilities that lie in DBMS world. I have to understand and implement lots of things in this space, at least in relation to fin world. KDB+, Hadoob(TP keeps reminding of this fascinating world that I have not explored at all). Well, I guess it is a life long process.

Via TP:

Caters to a nice gap and it is touted as a service which democratizes hedgefunds - http://www.covestor.com/how/

Will this make money ?

**“What can be done with fewer is done in vain with more”
**
_- William of Ockham

_

This school kid is amazing and as good as George Sampson(2008). He is definitely one of my fav , to win 2009.

Watch him @ BGT-2009:Aidan Davis

Continuing from my last post where I had left the juicy part of the book for this post, here I go :

The last topic contains ideas which I find fascinating. Markets are complex adaptive systems and one can understand it better if you have systems thinking. Peter Senge’s Book “The Fifth discipline ” had a great impact on my way of thinking and each essay in the last part of the book is a suggestion towards developing such a kind of thinking.

Via FE:

_With a rising number of market participants closely tracking India Vix, the volatility index that indicates markets’ expectation of volatility in the near term , the National Stock Exchange (NSE) would like to start trading in the index. An NSE official said the exchange would soon approach the Securities & Exchange Board of India (Sebi), the market regulator, for its approval.

_Introduction of Vix Futures and Vix Options would start an interesting phase in Indian markets.

This book contains about 30 essays , meaning 30 ideas which can be read, ruminated and possibly applied in real life situations wherever they are applicable. Though the book is about finance, it draws considerable amount of research evidence from Biology, Math, Sociology,etc . At a 10,000 ft view , actually each discipline is nothing but an effort to understand the world using a set of constructs.

I would term Finance as a soft science rather than hard science where the discipline needs to be understood from various eyes . For a number cruncher , it becomes all the more important to realize that numbers,statistics is after all ONLY one part of the story in markets. One must consider market as a complex adaptive animal where numbers is ONLY one part of understanding. The more the mental models one has, the better one is equipped to understand what’s going on.

Today I had to leave work early to go to Bandra to pick up some stuff from NSE. I had to be at Bandra by 6PM and there was no way I could reach Bandra with in a short time other than taking up a fast train from Churchgate. I took a fast train and as the train moved , there were 2 youngsters , early 20’s I guess, who were standing next to me, infact literally falling on me as the compartment was jam packed. I could not help but overhear their conversation. They were recounting some of their campus experiences a year ago. It was clear that they had graduated from one of the engineering colleges and hearing their lingo, it was easy to guess the institute. In any case, the thing which was caught my attention was that they were discussing /debating/ ruminating about some delta hedged position on an index option!!.They were both working for some brokerage firm in Mumbai and were barely an year in to their job. I somehow have this feeling, that when options are discussed by youngsters in a jam packed train compartment, it is a sign of positive things to come, as far as financial instruments evolution in India is concerned.

It is amazing what you can accomplish if you do not care who gets the credit

- Harry S Truman

It is almost 2 months since I read a book. Finally after the bhag daud in Mumbai , managed to find time to peacefully sit and read a book. Thanks to Sanjiv Sir’s awesome book collection, managed to get my hands on “SWAY”. The first time I came across a mention about this book was when a trader recounted his experience about a Variance swap trade and related his behavior to that explained in the book “SWAY”.

Many people around the world, after the subprime crisis more so , equate correlation with charlatanism. May be , but should that stop anyone from understanding data, not really. Knowing whatever metric about the unknown is much better than just having an intuitive feeling of things. Sometimes having a wrong map while exploring is OK, atleast it gives one confidence to venture and try things out.

Taylor effect is one such aspect about correlation. If you look at a series of returns and consider various metrics, like abs(ret), (ret^2), abs(ret)^2.5 and so on, one can think of this question :

“To reach a port, we must sail - sail, not tie at anchor - sail, not drift. "
-Roosevelt

NSE in its new circular has dropped about 25% of the stocks from Futures and Options Segment.

Well before the number surprises you, the actual numbers are not that terrifying considering that the whole equity F&O market itself is small as compared to developed countries.

There were only 252 stocks that were allowed to be in F&O segment at the beginning of the year. 68 stocks have been dropped in this year citing liquidity reasons, stock manipulation etc. Hope these 184 scripts remain and provide liquidity to the market.

Bid ask spread movement is an interesting metric to track in a day. It has been observed in developed markets that it follows a U shaped pattern
More details in this paper :Limit orders and Bid-Ask Spread

In Indian markets case, came across a 3 pager in Economics and Political weekly that says that the bid ask spreads do follow the same U pattern.
Link : Are you getting a good deal?

“The best thing about the future is that it comes only one day at a time” - Lincoln

Academic literature on GARCH and flavors of GARCH runs in to tons of papers by various researchers. The idea behind GARCH and all its flavors is that there is a conditional variance equation governing the evolution of usual returns, the residuals of which were otherwise assumed homoscedastic. There are a lot of variations of this conditional variance equation which gives rise to various flavors of GARCH.

Some of the markets have seen EGARCH performing better than other flavors. In Indian markets though , some of the literature suggests that GARCH(1,1) outperforms ARCH and other flavors. Some researchers have also tried to improve GARCH(1,1) forecasts by incorporating Implied vol of the options as one of the predictors!!.However the data used in all these models is before the great crash in Indian markets where vix has seen crazy levels. Markets in 2008-2009 has seen Indian Market capitalization shrink from ~1T USD to 0.6T USD . A quick look at the Nifty log returns gives a good indication of the crazy vol levels that Indian markets is seeing. BTW, India VIX as of today is above 50. So, it was natural for me to think about Asymmetric GARCH in these markets and there might be a case for AGARCH models

Via NYT :

“If a foreign-born engineer doesn’t come to Google, there is a very good chance that individual will return to India to compete against us.”