Skip to content

Speaking of time series

This weekend I wrote a simple AR(1) process in Matlab.  For kicks just now I ported it to Python.  Here’s an example of an ipython session in which I created 10 simulations, each with 10,000 observations, using a coefficient of 0.25, mean constant of 8, and standard deviation of 0.4.


In [1]: import ar1

In [2]: beta=0.25

In [3]: c=8

In [4]: sigma=0.4

In [5]: sims=ar1.simulate(10, 10000, beta, c, sigma)

In [6]: for sim in sims:
print str(ar1.fit(sim))
...:
(0.2456241979316662, 8.039670345021344, 0.4005802839081577)
(0.26269173583406669, 7.8658157445985104, 0.40067700748671264)
(0.25080291694490975, 7.9829412941212041, 0.40008786155521492)
(0.23849164273828619, 8.1268425384433005, 0.40244757878872384)
(0.23731989072323761, 8.1306157454693579, 0.39673446453761163)
(0.24088332077603036, 8.0971291647248087, 0.40266273196208063)
(0.24092356282570837, 8.0980025783096252, 0.40075384950504139)
(0.24059702316383563, 8.0928273681935199, 0.39797438141261765)
(0.25950709107966674, 7.8905209140742265, 0.40305518484472025)
(0.26037304559450791, 7.893634672469922, 0.3986525853167327)
Advertisements

Note to self #75931

I’ve been thinking about financial data and time series analysis a bit in the past few weeks and want to save this for future reference.

Zynga’s Words with Friends dictionary considered harmful

If you play Zynga’s online scrabble game, Words with Friends, just be aware that their dictionary is flawed. I just got 65 points for “quate,” which so far as I can tell is not a word. It appears to be in their dictionary because of improper parsing of typesetting that hyphenates “inadequate” when there’s a line break. See, for instance, the snippets of text with which Lexic.us illustrates the uses of “quate.”  Then there’s the Urban Dictionary entries, one of which appears to have been created by a Words with Friends player who, like me, was surprised to have it accept “quate.”  They were even luckier than me — 136 points points, over my mere 65.  I don’t know whether Zynga is using Lexic.us’s dictionary or rolling their own, but they clearly have a quality control problem when it comes to parsing text with words that are hyphenated by the typesetter.

Reacting to reviews, or what doesn’t kill me only makes me stronger

On the occasion of having my first paper rejected by a journal — with two reviews, one strongly positive, one strongly negative — a kind colleague send me these inspiring quotes from statistician Bradley Efron:

Earlier I had been editor of JASA, and this reminded me of a rule I had. When a paper made people angry, you should look at it more closely. Anger-arousing papers divide into a bimodal class. There are the worst papers you ever saw, that was a big class, and a few good ones.

I’ve had many papers turned down. I usually work really hard on revisions. I try hard to rewrite and take referees seriously, but I’m never discouraged by referees not liking something because sometimes it’s because you may have a new idea.

Structural power, intrinsic power, and Halo

I noticed that Aaron Clauset, one of the professors in my department at grad school, is going to be researching behavior in a competitive environment using data from massive multiplayer online games.  That got me thinking again.  My first paper is about using return probability on random walks on graphs as a computationally efficient measure for identifying powerful actors in some network (where power is defined as the ability of an actor to exclude its neighbors from some valuable interaction).  I called the measure the Pólya power index (PPI); it’s quite similar to other measures — the graph-theoretical power index (GPI), Bonacich’s beta centrality, and Neal’s alter-based centrality, for instance — but neither my nor the other measures are very well known.  In the literature of exchange networks, in which both the GPI and beta centrality were born, there is a strong assumption that an actor’s position in a network determines their power.  (See David Willer’s Network Exchange Theory (1999)  for a thorough overview of the field.)

After I finished my first paper, I wondered where these power-related measures might be relevant.  Where in the real world, that is; because until then all of the exchange network literature and all of my own work focused on relatively small synthetic networks.  The networks have interesting properties, to be sure, but eventually one would like to show results that are tied to the way people interact.  And not just people.  Wouldn’t it be cool, I thought, if relatively narrow experiments by sociologists resulted in network measures that generalize to other kinds of networks, like protein-protein interaction networks?  After talking to some grad students in my department working on analysis of biological networks, my hunch was that it’s unlikely, at least for biological networks.  So I kept looking.  I didn’t have much luck, but in the end I concluded that the only kind of network where these power-related measure might apply is that of a competitive game, where players are trying to exclude other players from … something, which in the MMOG domain can include simple virtual life or death.  Even then, I thought, the assumption that position determines power seems ill suited to an MMOG.  After all, some players are more nimble with their fingers, some have more free time to spend learning the intricacies of a particular MMOG, and so on.  Ultimately, I think what can be salvaged from the exchange network literature, at least with respect to behavior in competitive environments, are questions about to what extent position determines power.  This becomes more complicated in the context of MMOG networks, because they grow and because the connections amongst actors are dynamic, so I look forward to reading the results of Prof. Clauset’s research.

Ripping and Burning DVDs with Mac OS X

The simplest way I’ve found doesn’t require installing any additional third-party software at all. Just use Disk Utility.

A number of people told us how much they enjoyed the poems and songs at our wedding in October. We’d like to share them with you here, including the opening remarks by Whit and the amazing toast by Cheng-Hsi’s father.
Whit’s opening remarks.
Welcome friends and family to this glorious day here in Boulder, Colorado. We all want to acknowledge the parents of the bride, Sobee and James, and those of the groom, Bob and Ginny. And of course, this could not occur without the entire cast of family and friends.
This is a great day. Let the celebration begin.
The great German philosopher Goethe said “Love is an ideal thing, marriage is a real thing”. Today we gather here to witness the marriage vows of Nicholas Andrew Dronen and Cheng-Hsi Ho. This marriage, my friends, is the real thing. It is no accident.
A very smart individual, Dr. Albert Einstein, commented: “Graviation can not be held responsible for people falling in love”. This marriage is far more profound than the passive forces on this earth. Nick and Cheng-Hsi – their relationship – is far more stunning than chance.
Nick met Ms. Cheng-Hsi Ho relatively recently. They had a very comfortable friendship. One evening my wife Mary and I had a dinner with the now bride and groom. We were at our home, and could recognize something very special between these two. Nick, at that time, did not. Yes, he thought Cheng-Hsi was a great woman and a fine friend.
But give Nicholas credit. He dug deep (at times, over the years, we weren’t sure he would or could), and he recognized that Cheng-Hsi was a really special woman. Someone he could love, cherish, and most importantly, share his life with. This is Nick’s part on this specialday. He recognized Cheng-Hsi Ho. He saw deeper than just the everyday activities, the meals they were sharing. If this would not have happened – recognition – we would not be gathered here today.
Cheng-Hsi as well had her part to play to bring us all together today. Recognition of Nick of course was essential to walk the path of marriage. But let’s look closely at Ms Cheng-Hsi Ho. Cheng means receiving something old and making it better. Hsi means happiness. I have studied Chinese philosphy and the traditional medicine of China for my entire adult life. However, I don’t have command of the language, so please, give me some leeway here in my interpretations. With Cheng, we are talking about a process of transformation … of taking the routine daily activities of life, sometimes not all that glamorous, and making them into glorious events. Making better old routines, old patterns of behaviour.
Cheng– receiving, transforming. Hsi– Happiness. Who could ask for anything more?
And that is where we, the community of friends and family, come in. We have the job of recognizing the marriage of Nick Dronen and Cheng-Hsi Ho. It is our job, as their friends and family, to witness the commitments and the vows that we are about to hear. And we need to embrace them, in support of our dear friends. Because, God knows, there will be times they may forget how happy they are in this moment Let’s make sure we remind them, and reflect this back to them, to support them, to nourish their marriage.
Interestingly, Nicholas, from the Greek language, comes from two root words – Victory and People. We are the community – The People – and this sure feels like Victory to me.
May Nicholas and Cheng-Hsi succeed beyond our and their wildest dreams. The great Chinese philosopher, Lao Tze, says: “Nature does not hurry, yet everything is accomplished.” We have the entire Universe supporting us. There is nothing to worry about.
Friends, let us continue with the Celebration. I call forth my wife, Mary Saunders, to read several poems.
Poetry read by Mary Saunders.
I cursed the rain for
pounding upon my roof and driving
away sleep.
I cursed the wind
for ravaging my garden.
Then you entered; and I gave thanks
to the rain because you must put off
your wet dress; and I gave thanks to
the wind
that he came and blew out my lamp.
— Chang Wu-Chien
O Marvel! a garden amidst the flames.
My heart has become capable of every form:
it is a pasture for gazelles and a convent for Christian monks,
and a temple for idols and the pilgrim’s Ka’ba,
and the tables of the T
orah and the book of the Quran.
I follow the religion of Love: whatever way Love’s camels take,
that is my religion and my faith.
— Ibn Arabi
The TrueLove
There is a faith in loving fiercely
the one who is rightfully yours,
especially if you have
waited years and especially
if part of you never believed
you could deserve this
loved and beckoning hand
held out to you this way.
I am thinking of faith now
and the testament of loneliness
and what we feel we are
worthy of in this world.
Years ago in the Hebrides
I remember an old man
who walked every morning
on the grey stones
to the shore of baying seals,
who would press his hat
to his chest in the blustering
salt wind and say his prayer
to the turbulent Jesus
hidden in the water,
and I think of the story
of the storm and everyone
waking and seeing
the distant
yet familiar figure
far across the water
calling to them,
and how we are all
preparing for that
abrupt waking,
and that calling,
and that moment
we have to say yes,
except it will not come so grandly,
so Biblically,
but more subtly
and intimately in the face
of the one you know
you have to love,
so that when
we finally step out of the boat
toward them, we find
everything holds
us, and everything confirms
our courage, and if you wanted
to drown you could,
but you don’t
because finally
after all this struggle
and all these years, you don’t want to any more, you’ve
simply had enough
of drowning, and you want to live and you
want to love and you will
walk across any territory
and any darkness,
however fluid and however
dangerous, to take the
one hand you know
belongs in yours.
— David Whyte
Poety read by Jeff Pontillo
You know, my Friends, how bravely in my House
For a new Marriage I did make Carouse;
Divorced old barren Reason from my Bed
And took the Daughter of the Vine to Spouse.
For “Is” and “Is-NOT” though with Rule and Line,
And “UP-AND-DOWN” by Logic I define,
Of all that one should care to fathom, I
Was never deep in anything but — Wine.
— Rubaiyat by Omar Khayyam
The song performed by Wendy Lewis (vocals, guitar), Kitty Jenson (vocals), and Ted Dronen (guitar) was “True Love Will Find You in The End,” composed by the gifted songwriter Daniel Johnston:
True love will find you in the end
You’ll find out just who was your friend
Don’t be sad, I know you will,
But don’t give up until
True love finds you in the end.

This is a promise with a catch
Only if you’re looking will it find you
‘Cause true love is searching too
But how can it recognize you
Unless you step out into the light?
But don’t give up until
True love finds you in the end.

Here’s a version performed by Beck (Amazon, Nick’s web server). I encourage you to buy the MP3 from your favorite download site.
Finally, the magnificent toast by Cheng-Hsi’s father, James Ho.

Ladies and gentlemen, I am Cheng-Hsi’s father. Please look at me carefully. Today is a big day. It is also an historical moment. We are so glad to attend this wedding ceremony. Congratulations! Nick, you are so lucky to marry my daughter, Cheng-Hsi. She is very kind, diligent, and humble. Also she is as beautiful as an angel. You are now building a home, a sweet home, and a family. I love her so deep and so much. She is my heart. Please love one another ever and forever. Well, if i were President Obama, I would speak fluent English. If you do not understand what I say, please pretend to understand. Ginny! Bob! I really love my daughter. Please love her as your own. Nick! I trust you and believe you are a good husband. And certainly Cheng-Hsi is a good wife. I promise you, Nick! You have something to do, whatever that is, to contribute to your family, society, nation, and all mankind. That is my wish. God bless you! Thank you.