Tormented by an Urn

Teaching, learning, love and fear; the aesthetic sensibility of math

The basic Polya urn works like this: the urn is filled with balls of two colours, say red and black, mixed together in unknown proportion. Draw a ball from the urn and note the colour, then return the ball to the urn and also drop a new ball of the same colour into the urn. Then do the same thing again, and again. 

Sounds simple enough? That’s how they get you. The damn thing’s behaviour is fiendishly complicated. Students — and teachers — have been tortured ever since. Whole books have been written about these urn models, many books, and thousands and thousands of papers. The extended forms of such models are connected to hundreds of disparate ideas in statistics, most of them even worse; for example, these urns have a connection with the Dirichlet distribution, which in turn yields the Dirichlet process, an object so cursed that should you wish to alarm a Bayesian statistician you need only say the words “Dirichlet process prior”.¹ One of the most beautiful aspects of statistics is how revealing it is of the complexity in the ordinary. So many things sound simple; most are not. 

A lovely introduction to Polya Urns and the resulting processes can be found online via Penn Engineering Professor Santosh S. Venkatesh, whose short and witty lectures are accessible on YouTube here . He offers a slightly broader version of the scheme, where one adds some unknown number of balls, A, to the urn, on each try, rather than just one. Listening to his smooth ultra-posh British accent and his illustrative story about royalists and seditionists, you might be forgiven for thinking you are listening to philosophy — and in fact, in a deeper sense, you are — (just wait, I will explain this). But first, if you are not familiar with these urns, I do highly recommend at least the first two 5 minute mini-lectures. I think they will work for any reader, including beginners, although it’s always a little hard to know once you’re ten years out from the last time you took any stats class. I myself have never taught these urns. 

Still I think I know good teaching (to a point), and I really like his lectures, with his moments of advice. One of the best such little moments is when he says to the students, “Now before we launch into this, it is always wise when dealing with problems to think about the setting — doodle, sketch, draw. Try to get a right understanding of the circumstances around a problem.” Too right. In the next video in the series, which introduces the population growth tree needed to reason intelligently about the Polya Urn, he asks us to use techniques such as “introspection”. Later on in the videos he describes mathematics as a way to “add precision to these verbal bones”. 

As an aside, I will say the real moment in this video where the veteran of statistical language can know shit is going to get real is when he introduces the notation R subscript 1 for the event of drawing R — royalist in his example, red ball in ours — in the first round of draws. The notation R sub k indicates drawing R on the kth round. That notation itself is tortured, and yet the experienced scholar knows via context that it can only be the best and safest option. The newbie finds it cumbersome; the veteran finds it spooky. Subscripts and superscripts are the mark of the devil in mathematics. 

By the third lecture, Venkatesh has lead us to Bayes. He starts by saying something absolutely remarkable: “Notice, we have put in a fair amount of work here, just to understand what the problem is saying. And we should always do this before we launch directly into an attack on the problem. I know that long tradition and custom through your early school years has brought in this instinct to just dive into a problem. But any worthwhile problem needs introspection, needs thought, needs time to understand what is going on. And this will well repay you.” 

This is deep, deep in the tradition of Polya himself, who wrote not just a book about how to solve mathematical problems in general but also a separate, still wonderfully fresh reflection on teaching — though it’s perhaps a bit depressing that it should remain so wonderfully fresh, which does not speak well of us or the system as a whole.

Polya was in fact outrageously, outstandingly prolific and polymathic. He published more than 58 scientific papers across probability, combinatorics, analysis and mathematical physics. From 1930-1932 alone, Polya published papers in German, French and English; his native language was Hungarian.  He might have coined the term “the central limit theorem.” And last but not least, in his capacity as advisor and teacher, he gave the world such students as Alice Roth, Albert Pflugler, and the incomparable John Von Neumann, who I guess truly birthed himself, and that’s another story, but still, here’s something fascinating:

Polya described himself as “afraid” of the student Von Neumann, who needed only to hear of an open problem before he might come up to his teacher at the end of the lecture having fully worked all of it out. I hope this essay will in part explain the sensibility that lies beneath that fear, that thought, that taste. 

***

And I hope the description of the urn did not bore you. I hope you learned something from these lectures, if you clicked through. It’s almost charming to have to get this information from an engineering department — but that’s exactly how they think of us (economists). Subfields of technical disciplines have an adorable, almost alarming insularity. Ideas which stretch across them do the larger body good. 

Polya introduced the urn in 1931. It’s now almost one hundred years later. Why do we still need it? Why do we still teach it and insist upon its centrality in disparate departments and subfields across the world? Yes such urns are useful but I think that isn’t why. The real reason we teach them is because the kind of people who become statisticians are prone to falling in love with them. 

Love is a complicated word to use here. The concept of love is abused in academia where it is often an excuse to exploit someone. But there is no getting around the fact that a kind of love is present in our work. The underlying reason these urn models are useful is because they contain a deep spark of some true connective meaning; a genie in a bottle, literal sense. What statistical minds are drawn to is the genie itself, not its use, nor even its power — these demonstrate that it is a truly magical genie, but that demonstration primarily convinces the layman. Those who can lay eyes on the bottle can see that it is powerful at once. I dare to suggest that statistically-minded people feel that the Polya urn is more fundamental and revelatory and useful than even its current uses, which are many, suggest. I think the time of the Polya urn has not yet come. But I promise you that doesn’t even matter. After 93 years of tender affiliation, these urns are a core element of the statistical canon. They form part of the deep grammar of statistical language, and those who want to perform foundational or conceptual statistics absolutely must have access to them. 

Let me try to explain why this is so. We still start once more considering the model of the urn. The first thing to realise is the model is dynamic. Things happen sequentially, and what happens the next time I dip my hand in is deeply connected with what happened before. The world inside the urn is always changing. I cannot observe it but that I adulterate it, that in fact I change it forever. This is unlike sampling with replacement, a scheme designed to remain in stasis no matter what I do or when or why. In some sense the simple Polya urn is the perfect opposite of sampling without replacement: sampling with super-replacement, a sample augmentation. So unlike sampling without replacement, where things must spiral inwards, getting muffled, dark and damp, the Polya urn spirals outwards in a way that can cascade. The core is that these changes must accelerate or amplify. Small tremors in first branches cause wild shaking in the leaves.

These dynamics do turn out to be useful. In real life the world is changing often and there are many things that are contagious. (I wonder if in fact LLMs which eat their own output will end up in dynamics that are like the Polya urn.) But an idea being useful does not guarantee success. As evidence, take the fact that the vast majority of econometricians, and almost the totality of PhD economists, will have no idea what Polya Urns are. These urns are not taught, nor are any of their related distributions, PhD econometrics contains modally no mention of the beta distribution, the Dirichlet distribution, the Gamma function, the Hoppe urn, the Chinese Restaurant Process, nor even De Finetti’s theorem, nor the concept of exchangeability.² Why? Because, whatever they say or whatever it is in theory, econometrics is in practice founded on a totally distinct sensibility, the roots of which lie in linear algebra. Econometrics is more foundationally about filtration and projection than it is even about sampling at all. (Economists know more about the Frisch-Waugh-Lovell theorem of residualisation in OLS linear regression than they know about the iid assumption.) 

It is never enough to be useful. Many things are useful, and with many applications. Things also have to be captivating if they want to enter the canon — specifically capitivating and indeed seductive to the kind of mind that wants to do statistics. Teaching is hard work. Research is hard work. It is poorly rewarded and remunerated in most cases. One does not do this work unless one is a little bit in love with something. The thing I’m driving at here is that the collected knowledge of statistics is in part driven by an aesthetic judgment or particular sensibility; in short, a taste. 

The Polya urn represents that taste so deeply as to almost be emblematic of it— it is deceptively, disturbingly simple for something shockingly thorny and profound. Hence, it is parsimonious. It is beautiful and utilitarian, streamlined and dense. You would not know it from the problem statement — the urn descriptions I and Santosh offered you — but that simple problem alone with nothing extra forces engagement with two of the deepest elements in statistics: recursion and conditionality. It’s almost mind-bendingly beautiful how neatly those two concepts are tucked in there. Yet the problem is relentlessly discrete and obstructionist. It seems to intentionally torment you. The complexity it offers has a shadow of the mystic. There is a feeling of inducement and seduction. This problem stares also into you. 

There is something profound in the statistically inclined person that seeks knowledge over uncertainty via repeated interactions with a demon. (The true foundations of Bayesian thinking are due in fact to Laplace; an example of Stigler’s law.) There is a desire for mastery and yet an equal, queer desire to be puzzled and bested and perplexed. Were you and I to study it for another hundred years we would never reach the end of the Polya Urn. The process is beyond us, like the universe itself, that’s part of why we love it even though it torments us. People who have deep statistical taste love to be confounded and obstructed, even shocked by a problem; to almost physically wrestle with what we cannot know. There is a yearning to plunge into uncertainty, to reach down and down again into the urn. 

***

This tendency puts us in company with poets. Indeed if you think of urns in culture you can only think of greeks, or at least I think that’s the association for most of us in the western intellectual tradition. You don’t see urns very often anymore. Other than in funerals, I guess. 

The actually poetically inclined must think of Keats now, stoked to madness, having a little mental breakdown in the museum (and elsewhere), he too so tormented by an urn. But it also overlaps with literature and legend, particularly Faust, the original tortured scholar. Goethe’s retelling has this decrepit academic seduce an innocent and brilliant young women to her ruin. Ouch. If you want to hear something even worse than that, according to Wikipedia, prince among the sources — if you will do your duty and click through to their footnotes — “In the Historia Brittonum, Faustus is the offspring of an incestuous marriage between king Vortigern and Vortigern's own daughter.” Another ungodly recursion. Yet degeneracy is not the province of the arts, or else, at least, not only. In waxing lyrical about the wonders of the sciences, it pays to consider our own little Byrons, in our history, in departments to this day. 

I claim that the sensibility of the Polya Urn is not only mathematical but poetic. I claim further that there are profound symmetries of this kind at every level between the arts and the sciences. The human aspects of these endeavours permeate the structures that we have built, will build and can build to house them. Our experiences are what shape our findings, via that fickle intermediary, our thoughts. Thinking about anything deeply is in some sense inescapably emotional and creative. 

This is not a new observation. I know literary figures and artists and writers have been making it — most spectacularly, Benjamin Labatut, who recently tackled Von Neumann and saved me undertaking the task (ie he got there before me, bah). But mathematicians and scientists have been making it as well:

“Many who have never had an opportunity of knowing any more about mathematics confound it with arithmetic, and consider it an arid science. In reality, however, it is a science which requires a great amount of imagination, and one of the leading mathematicians of our century states the case quite correctly when he says that it is impossible to be a mathematician without being a poet in soul. Only, of course, in order to comprehend the accuracy of this definition, one must renounce the ancient prejudice that a poet must invent something which does not exist, that imagination and invention are identical. It seems to me that the poet has only to perceive that which others do not perceive, to look deeper than others look. And the mathematician must do the same thing.”

That was the great 19^th century Russian mathematician Sofya Kovalevskaya, who went around generalising theorems of Cauchy, you know, for fun (“like it’s hard?”).

I would not try to claim, as I hope no-one does, that there’s complete and perfect overlap between the knowledge in the arts and sciences — there isn’t. Neither system is complete. Nothing in the arts repeats the Polya urn, just as nothing in the sciences gives us Nabokov or Shelley.  But the two things are completely interlocking. They form one shared endeavour. I feel, in a way I cannot yet fully articulate, that they need each other badly. You often hear this claim made flippantly as if it’s true without explanation or evidence — but why should it be true at all? I think that I don’t understand it. And they don’t need to be both present inside the same person… but I think it would be better if they can be. I don’t know if I can explain it — in fact I know I can’t. Yet I beg your permission to try.³ 

First, yes, I know, one has to tread carefully here. I struggle all the time on this blog when I write about art and literature. I know at times you hear the hollow ring of a person speaking boldly about things they only dimly understand. I apologise (but I won’t stop, I guess?). Scientists who don’t read or take in much art often sound ridiculous when they (we) try to converse about it — Conversely still there is probably nothing more cringe than humanities people or writers invoking scientific concepts they do not understand as inappropriate metaphors for emotional, spiritual, interpersonal and artistic ideas. (Quantum mechanics and Heisenberg’s uncertainty principle are probably ground zero for that. Almost every mention of Schrodinger’s cat in modern popular culture gives me an immediate shot of vicarious embarrassment.) And yet you almost cannot fault them. They are grasping for a union that they can only sense. Scientists, quants and mathematicians are doing ourselves also an equally embarrassing disservice by remaining ignorant of the artistic, and the world outside our models. As if only science matters? That’s how you get Faust. These are not superstitions, they are trying to teach you something quite important and profound, that is virtually impossible to grapple with in other ways but sex and love and therapy, religious devotion, communal chanting, music, literature, spiritual thought and art. 

There is no war between the arts and sciences. Much much more unites us than divides us; to be passionate about either of these things is to be a huge nerd and nerd is a polarity, and still, after all this time, not a very common one. In fact nerd is a gender — male and female nerds have much more in common with each other aesthetically, spiritually, behaviourally and psychologically than they do with culturally normative men and women. I do not believe that tech CEOs represent nerddom. I simply do not buy it. The academy is still the last bastion of nerds, I’m afraid. Yes, there is something to defend here. I don’t want to get into a culture war territory, but at times it does seem as if there is something under attack, though perhaps only by time itself, by erosion, the depreciation of knowledge. But it’s all knowledge. It’s not one type. The arts may be derided as “degenerate” — er, I mean “woke” — no! I mean of course “subjective” — but as soon as the sciences offer up something culturally unpalatable, one finds our own techniques and methods exposed and derided as well. Most critiques of form are really cloaked critiques of content. You can find that in yourself, if you look hard. 

I do believe there is virtue in the sciences, and something to defend in the profundity and rigor to be had in mathematics. It should be clear from the above however that I do not believe the defence of science lies in claims to objectivity. Which frameworks go into mathematics, or statistics? What questions do scientists ask? What defines the borders between stats and biostats, econometrics, CS, ML, EECS, AI? The choice is based on taste, at least in part. That’s good; we do it that way and we’re good to do it. Do you believe Mathematics is objective? Is it free from any kind of aesthetics? Is it liberated from the deep experience of being in some sense human? Let’s ask a mathematician, here’s Djalil Chafaï on this topic: “Personally, I believe that we should not only view physics as an application of mathematics. For me, mathematics is an autonomous self-consistent aesthetic abstraction of physics, which goes beyond the physical world, and which is intimately based on the human reasoning.” Mathematics, like all the sciences, is creative, collaborative, substantially and inextricably informal, and it has much in common with a folk art. 

If you want to collaborate and contribute to a body of knowledge, even “scientific” or “objective” knowledge (vomit), you have to know first what is in it. You have to know what came before. There are extraordinary, and extraordinarily rare exceptions to this, the obvious one being Ramunajan, and Von Neumann probably also in some slightly strange sense counts. What’s fantastic about these examples is not just that they developed new ideas but that they managed to independently develop a taste. Most people cannot do this and there’s nothing wrong in that. It’s kind of beautiful to depend on each other and our ancestors to know even what we might like at all. Even though most scholars at one time or another independently discover a (small) established result — the distraut 3rd year graduate student special — and nobody really gets credit correctly ( once again, that’s Stigler’s law) — in a certain way there’s no such thing as credit anyway. Discoveries are for the most part actually made collaboratively. People held up as “geniuses” in the public imagination usually are in fact figureheads around which a large collective movement coalesces —in art, in politics, and in science.

It sounds paradoxical, but if you want to do something new, you need to know approximately everything. I’m sorry. It isn’t really possible. That’s why doing something new isn’t possible. But still overall in some deep sense I think that’s how it it’s done. For there to be knowledge, any knowledge, there has to be teaching and learning.

***

I am trying hard to say all this without sounding at least subtly like a dinosaur who insists on the primacy of an old and static canon, who insists that children and students should have to sit through hours of boring torment where they are told that their tastes and interests are worthless and forced to choke down the tastes and interests of the dominant power structure instead. 

We don’t want that, that would be and indeed is disastrous, and to a first approximation that is still what happens, so I understand why there is fear of any sentiment that seems to prop up that world. A desire to liberate everyone from the oppressive experience of being a student, of being told anything, is natural to have sometimes. Being at times tempted by anarchy is part of being human. At the same time we mostly can not have anarchy, however much we want it, and children and students are so because they need us to teach them something, and until they get enough of their own knowledge, they need us to also decide what to teach. We should teach them the best things. I am not saying we should give them a lecture against their likes and interests, but until you teach them enough things, they can’t even know what their likes and interests even are. This stuff is their inheritance, but only if we give it to them.

I cannot accept calls to replace calculus with personal finance class, or for that matter calls to replace Keats with Taylor Swift as our number one tormented poet. The value of Keats and calculus is not that they are old but that they are deep and they are proven. They form part of the material that builds and shapes the world. I am all for reassessment and reinvention of the canon but it has to be done in fact and not in theory. We should rescue women and scholars of colour (in the West) from wrongful obscurity. We should put Emily Dickinson and DuBois and Hafiz next to Shakespeare. We should teach the discoveries of Rosalind Franklin, Emmy Noether and Hedy Lamar. In some cases we should retire canon texts and ideas which are harmful and not useful. That is all important. It does not mean pretending we believe idiotic things, like that there is equivalent importance between the invention of the limit and the mortgage; there is not. One of those things lies far underneath the other. One of these things is in all of our bones. 

I also oppose the contemporary motion to remove maths from core curricula. First of all good fucking luck teaching personal finance to someone without the underlying structure to understand compounding. Second of all, the mere idea of removing abstract mathematics from high school is just cope. You might as well admit you think that children cannot learn. It would be just as stupid not to teach them English, or whatever language they would need to speak. The truth is not that they can’t learn mathematics but rather that we can’t teach them — that is our problem.

Polya was all over this. “The teacher should regard himself as a salesman: he wants to sell some mathematics to the youngsters.” And then: “It is your duty as a teacher, as a salesman of knowledge, to convince the student that mathematics is interesting, that the point just under discussion is interesting, that the problem he is supposed to do deserves his effort.” (He goes on to provide several practical tips for how to do this — and by this account I can tell he was an excellent salesman.)

In the same sense that (almost) everyone can speak some native language, almost everyone can do some math. Our colossal failures to create and sustain a functional education system are the reason for these stupid moves, but it is maddening and classic and positively Shakespearean to note that the people who are going to pay for the sins of the elders are the children. They came into a world that’s entirely governed by math and science and code and art and literature. They need to understand it. To understand our history as a species properly you need practical knowledge of both humanties and sciences. Imagine how politically ignorant you would be if you did not know what defines propaganda, or what fear-mongering is. My thought is that there is a comparably debilitating ignorance to be had in both the artistic and scientific realms, and that far too many learned people have it.  

***

I took this photograph of an urn created by the Singaporean master ceramacist Iskandar Jalil. Jalil was trained in mathematics and chemistry. I took this photo at the National Gallery of Art in Singapore after my job talk at INSEAD in early 2017.

1. I actually remember this connection, from my own graduate studies, only very vaguely and most likely from talking with scholars in the EECS department at MIT; I later on found a dense and highly advanced little post about the connection here, on the blog Eventually Almost Everywhere. The cadence of that post suggests it’s a slightly nonobvious link, and I would struggle to articulate it more succinctly than the post does, but I still back my recollection nonetheless.

2. All of these underlie enormous swathes of modern academic statistics, for example, here are Tamara Broderick’s notes on Nonparametric Bayesian inference . Yet economists by and large know nothing about them. The only concepts related to Polya Urns that we learn in economics are martingales and the multinomial logit respectively, and the vast majority of even PhD economists do not know shit about these either, because they are considered advanced and specialised subfield knowledge.

3. This phrasing is a reference to Polya, who uses this construction in his writing about learning.