WAR, Pythagoras, Poisson and Skellam

Getting into some issues only makes you wish that you hadn’t, when you realize how messed up they are, at a fundamental level.

Here’s a great example involving statistical analysis, as applied to win/loss (“WL”) records of sports teams, the base concept of which is that it’s possible to estimate what a team’s WL record “should” have been, based on the number of goals/runs/points that it scored, and allowed, over a defined number of games (typically, a full season or more). This blog post by Bill James partially motivates my thoughts here.

Just where and when this basic idea originated I’m not 100 percent sure, but it appears to have been James, three to four decades ago, under the name “Pythagorean Expectation” (PE). Bill James, if you don’t know, is the originator, and/or popularizer, of a number of statistical methods or approaches applied to baseball data, which launched the so-called “SABR-metric” baseball analysis movement (SABR = Society for American Baseball Research). He is basically that movement’s founder.

In the linked post above, James uses the recent American League MVP votes for Jose Altuve and Aaron Judge, to make some great points regarding the merit of WAR (Wins Above Replacement), arguably the most popular of the many SABR-metric variables. The legitimacy of WAR is an involved topic on which much virtual ink has been spilled, but is not my focus here; in brief, it tries to estimate the contribution each player makes to his team’s WL record. In the article, James takes pointed exception to how WAR is used (by some, who argue based upon it, that the two players were basically about equally valuable in 2017). In the actual MVP vote, Altuve won by a landslide, and James agrees with the voters’ judgement (pun intended): WAR is flawed in evaluating true player worth in this context. Note that numerous problems have been identified with WAR, but James is bringing a new and serious one, and from a position of authority.

One of James’ main arguments involves inappropriate use of the PE, specifically that the “expected” number of wins by a team is quite irrelevant–it’s the *actual* number that matters when assessing any given player’s contribution to it. For the 2017 season, the PE estimates that Judge’s team, the New York Yankers, “should” have gone 101-61, instead of their actual 91-71, and thus in turn, every Yanker player is getting some additional proportion of those ten extra, imaginary wins, added to his seasonal WAR estimate. For Altuve’s team, the Houston Astros, that’s not an issue because their actual and PE WL records were identical (both 101-61). The WAR-mongers, and most self identified SABR-metricians for that matter, automatically then conclude that a team like this year’s Yanks were “unlucky”: they should have won 101 games, but doggone lady luck was against ’em in distributing their runs scored (and allowed) across their 162 games…such that they only won 91 instead. Other league teams balance the overall ledger by being luck beneficiaries–if not outright pretenders. There are major problems with this whole mode of thought, some of which James rips in his essay, correctly IMO.

But one additional major problem here is that James started the PE craze to begin with, and neither he, nor anybody else who have subsequently either modified or used it, seems to understand the problems inherent in that metric. James instead addresses issues in the application of the PE as input to the metric (WAR) that he takes issue with, not the legitimacy of the PE itself. Well, there are in fact several issues with the PE, ones that collectively illustrate important issues in statistical philosophy and practice. If you’re going to criticize, start at the root, not the branches.

The issue is one of statistical methodology, and the name of the metric is itself a big clue–it was chosen because the PE formula is similar to the Pythagorean theorem of geometry: A^2 + B^2 = C^2, where A, B and C are the three sides of a right triangle. The original (James) PE equation was: W = S^2 / (S^2 + A^2), where W = winning percentage, S = total runs scored and A = total runs allowed, summed over all the teams in a league, over one or more seasons. That is, it supposedly mimicked the ratio of squared lengths between one side, and the hypotenuse, of a right triangle. Just how James came to this structural form, and parameter values, I don’t know and likely very few besides James himself do; presumably the details are in one of his annual Baseball Abstracts from 1977 to 1988, since he doesn’t discuss the issue that I can see, in either of his “Historical Baseball Abstract” books. Perhaps he thought that runs scored and allowed were fully independent of each other, orthogonal, like the two sides of a right triangle. I don’t know.

It seems to me very likely that James derived his equation via fitting various curves to some empirical data set, although it is possible he was operating from some (unknown) theoretical basis. Others who followed him, and supposedly “improved” the metric’s accuracy definitely fitted curves to data, since all parameters (exponents) were lowered to values (e.g. 1.81) for which no theoretical basis is even possible to conceive of: show me the theoretical basis for anything that scales up/down according to the ratio of a sum of parts, and one component thereof, by the power of 1.81. The current PE incarnation (claimed as the definitive word on the matter by some) has the exponents themselves as variables, dependent on the so-called “run environment”, the total number of runs scored and allowed, per game. Thus, the exponents for any given season are estimated by R^0.285, where R is the average number of runs scored per game (both teams) over all games of a season.

Even assuming that James did in fact try to base his PE on theory somehow, he didn’t do it right, and that’s a big problem, because there is in fact a very definite theoretical basis for exactly this type of problem…but one never followed, and apparently never even recognized, by SABR-metricians. At least I’ve seen no discussion of it anywhere, and I’ve read my share of baseball analytics essays. Instead, it’s an example of the curve-fitting mentality that is utterly ubiquitous among them. (I have seen some theoretically driven analytics in baseball, but mostly as applied to ball velocity and trajectory off the bat, as predicted from e.g., bat and ball elasticity, temperature, launch angle, and etc, and also the analysis of bat breakage, a big problem a few years back. And these were by Alan Nathan, an actual physicist).

Much of science, especially non-experimental science, involves estimating relationships from empirical data. And there’s good reason for that–most natural systems are complex, and often, one simply does not know, quantitatively and apriori, the fundamental operating relationships upon which to build a theory, much less how those interact with each other in complex ways at the time and space scales of interest. Therefore one tries instead to estimate those relationships by fitting models to empirical data–often some type of regression model, but not necessarily. It goes without saying that since the system is complex, you can only hope to detect some part of the full signal from the noise, often just one component of it. It’s an inverse, or inferential, approach to understanding a system, as opposed to forward modeling driven by theory; these are the two opposing approaches to understanding a system.

On those (rare) occasions when you do have a system amenable to theoretical analysis…well you dang well better do so. Geneticists know this: they don’t ignore binomial/multinomial models, in favor of curve fitting, to estimate likely nuclear transmission genetic processes in diploid population genetics and inheritance. That would be entirely stupid, given that we know for sure that diploid chromosomes conform to a binomial process during meiosis the vast majority of the time. We understand the underlying driving process–it’s simple and ubiquitous.

The binomial must be about the simplest possible stochastic model…but the Poisson isn’t too far behind. The Poisson predicts the expected distribution of the occurrence of discrete events in a set of sample units, given knowledge of the average occurrence rate determined over the full set thereof. It is in fact exactly the appropriate model for predicting the per-game distribution of runs/goals scored (and allowed), in sports such as baseball, hockey, golf, soccer, lacrosse, etc. (i.e. sports in which scoring is integer-valued and all scoring events are positive and of equal value).

To start with, the Poisson model can test a wider variety of hypotheses. The PE can only predict a team’s WL record, whereas the Poisson can test whether or not a team’s actual runs scored (and allowed) distribution, follows expectation. To the extent that they do follow is corresponding evidence of true randomness generating the variance in scores across games. This in turn means that the run scoring (or allowing) process is stationary, i.e., it is governed by an unchanging set of drivers. Conversely, if the observed distributions differ significantly from expectation, that’s corresponding evidence that those drivers are not stationary, meaning that teams’ inherent ability to score (and/or allow) runs is dynamic–they change over time (i.e. between games). That’s an important piece of knowledge in and of itself.

But the primary question of interest here involves the WL record and its relationship to runs scored and allowed. If a team’s runs scored and allowed both closely follow Poisson expectations–then prediction of the WL record follows from theory. Specifically, the distribution of differences in two Poisson distributions follows the Skellam distribution, described by the British statistician J.G. Skellam in the 1950s, as part of his extensive work on point processes. That is, the Skellam directly predicts the WL record whenever the Poisson assumptions are satisfied. However, even if a team’s run distribution deviates significantly from Poisson expectation, it is still possible to accurately estimate the expected WL record, by simply resampling–drawing randomly several thousand times from the observed distributions–allowing computers to do what they’re really good at. [Note that in low scoring sports like hockey and baseball, many ties will be predicted, and sports differ greatly in how they break ties at the end of regulation play. The National Hockey League and Major League Baseball vary greatly in this respect, especially now that NHL ties can be decided by shoot-out, which is a completely different process than regulation play. In either case, it’s necessary to identify games that are tied at the end of regulation.]

If instead you take an empirical data set and fit some equation to those data–any equation, no matter how good the fit–you run the risk of committing a very big error indeed, one of the biggest you can in fact make. Specifically, if the data do in fact deviate from Poisson expectation, i.e. non-stationary processes are operating, you will mistake your data-fitted model for the true expectation–the baseline reference point from which to assess random variation. Show me a bigger error that you can make then that one–it will affect every conclusion you subsequently come to. So, if you want to assess how “lucky” a team was with its WL record, relative to runs scored and allowed, don’t do that. And don’t get me started on use of the term “luck” in SABR-metrics, when what they really mean is chance, or stochastic, variation. The conflation of such terms in sports that very clearly involve heavy doses of both skill and chance, is a fairly flagrant violation of the whole point of language. James is quite right in pointing this out.

I was originally hoping to get into some data analysis to demonstrate the above points but that will have to wait–the underlying statistical concepts needed to be discussed first and that’s all I have time for right now. Rest assured that it’s not hard to analyze the relevant data in R (but it can be a time-consuming pain to obtain and properly format it).

I would also like to remind everyone to try to lay off high fastballs, keep your stick on the ice, and stay tuned to this channel for further fascinating discussions of all kinds.  Remember that Tuesdays are dollar dog night, but also that we discontinued 10 cent beer night 40 years ago, given the results.

Advertisements

Everything is broken

Broken bottles, broken plates
Broken switches, broken gates
Broken dishes, broken parts
Streets are filled with broken hearts
Broken words never meant to be spoken
Everything is broken

Broken cutters, broken saws
Broken buckles, broken laws
Broken bodies, broken bones
Broken voices on broken phones

Broken hands on broken ploughs
Broken treaties, broken vows
Broken pipes, broken tools
People bending broken rules
Hound dog howling, bullfrog croaking
Everything is broken

Bob Dylan, 1989

What was it, roughly, that we were thinking there, if anything? Part two.

So, there was a high school class reunion a few months back, and it’s World Series time again, so now seems a good time for an overdue, second episode of our series of the above title. In episode one, I explored an incident involving sub-optimal decision making in high school so I think I’ll just continue on that theme here.

I saw a number of old classmates, and teammates, at the reunion. I think class reunions are great. They can cause one to reflect on really important topics, such as the passage of time, or the nature of life’s changes. Or to even deeper things. Explosives for example. Just what “loud” really entails. The nature of stupidity.

It seems that I had become aware that personal fireworks were legal in the next county, and had thus traveled the 40 miles to obtain a few dozen “M-80” fireworks, ostensibly for use during the Fourth of July. It also seems that sometime later, my friend Steve and I found ourselves parked in front of our friend Doug Brown’s house after dark, with said bag of M-80s and a lighter. Now, an M-80, we’d been told, contained the equivalent gunpowder of a quarter stick of dynamite, which I thought was pretty impressive but did no actual testing of. If one of these things goes off on, say, someone’s front porch, it would not typically go unnoticed, and that concept did seem, to us, worthy of some testing at that particular time.

It additionally seems that I was the driver and Doug’s house was off to our left. The plan, which I think we put a solid 30 seconds of thought into, was that we would launch one of these onto Doug’s porch–about maybe 75 ft away–while seated in the vehicle, so as to effect a prompt getaway. We came up with a fair and efficient division of labor in which Steve would light the fuse and hand the thing to me–I would then fire it toward the porch and immediately hit the gas, making ourselves rapidly scarce. It was a great plan as far as I was concerned: all I had to do was throw and floor it, whereas Steve had the equivalent of six to eight sticks of dynamite in a bag on his lap, with an open flame in his hand. This struck me as equitable, given that I was providing the vehicle and the right arm.

So…what’s the baseball connection here, you may wonder. Well, I played shortstop in high school, whereas Steve didn’t, and so it was logical that I should do whatever throwing was involved. Shortstop is a fun position, because you get to sprint to chase down ground balls, and then watch the first baseman sprint to chase down the throw you just sailed some distance beyond him. Now, 70-80 feet is a lot shorter than a typical throw from shortstop to first base…but an M-80 is also a lot lighter than a baseball. So I knew I should put some mustard on it to insure getting it at least somewhere near the porch. Being quite experienced at firing balls into the adjacent woods from deep short, I wasn’t too worried about it. If the M-80 banged off the front of the house first or whatever, no big deal, I mean assuming nobody opened the front door at the wrong instant.

Now may be a good time to remind ourselves of the importance of taking all potentially relevant variables into consideration–apriori even–in events like these. And do we think enough about the tangible value of trial runs? Probably some room for improvement there too.

Anyway, Steve successfully got said firework lit without blowing us up, and the ensuing exchange to me was also flawless. With right arm extended and a good five seconds or so to work with, I eyed Doug’s porch and applied my best Nolan Ryan fastball to the explosive. Now, I think it’s fair to say that (1) the average person is just not that aware of exactly where one’s car door meets one’s car roof, (2) that I qualify as quite average in that context, and (3) that that specific location took on above average significance, in that particular situation. In short, when my hand was just about to send said explosive device on it’s planned trajectory, said hand was inadvertently applied, with considerable force, to said vehicular location, and separated from said device, thereby placing the latter on a trajectory not nearly as likely to achieve the original objective. This in turn would necessitate a rapid adjustment in plan and action, not to mention vocalization.

This is more or less a science blog, and I ask you, are many topics more fascinating, really, than the physics of acoustics under confinement? Maybe heredity–I find that interesting too. Also, involuntary reflexes, impromptu vocalizations: good stuff. How about hand-eye coordination under duress? Personal safety and survival? Blood?
Bodily dismemberment? All topics worthy of consideration when you get down to it. Let’s explore some of these for just a moment.

Acoustic physics, let’s take. As we know, Newton’s First Law of Loud, states “Any acoustically active device, placed under spatial confinement, will manifest even more of its acoustic characteristics, in fact quite a lot more than you’d think just from theory alone”. Take spatial relationships: just how much room for rapid bodily movement is there, really, in the front passenger seat of a typical car? How can humans maximize movement efficiency in response to active, explosive devices experiencing random trajectories?

Now back to our story. To cut to the chase, upon hand-car impact, our active device–the one under current discussion–experienced a rather sudden change of x coordinate velocity–one markedly away from Doug’s front porch, opposite that really, which is to say in the general direction of one Steve. More specifically, toward Steve’s male-specific, hereditarily significant anatomy. And there it landed, for a brief moment. Although entirely stunned, and with my hand feeling possibly broken, I was still able to collect myself, breathe a sigh of relief and comment on just how fortunate we were, really, that said M-80 had not taken an alternate trajectory and landed instead, in or near the bag of 30 or so other M-80s, within the confines of our vehicle, in which we too were present, due to our plan, in the street in front of our friend Doug Brown’s house. Steve also reflected for a bit on this fortunate state and concurred that such an outcome would have been potentially problematic on several counts, not the least of which was just how autopsies and identifications based on scattered body parts are conducted.

The preceding is not in fact what transpired at that moment.

Rather, Steve executed what I think to this day is the most rapid series of body movements I’ve ever seen from a human being, with the possible exception of the time I scrambled up and over a rock to find my neck about three feet directly in front of the head of a large rattlesnake. Conscious thought was not part of the process. M-80s had fuse times of roughly six to seven seconds, going strictly from memory. I’d guestimate that at this point, about four of those remained. As I recall it, there were, in order (1) an involuntary yell, (2) a ceiling-constrained jump upward, and (3) a failed attempt to flick the thing, by a backhand motion, away from where it resided. This process took maybe two seconds, maximum, and led to another entirely frantic attempt–panicked would work–which succeeded in flinging the thing down towards Steve’s feet. This, very fortunately, was not where the bag of other M-80s had been placed, and additionally, it’s one thing to have your feet blown off but quite another to have your evolutionary lineage ended.

Down there our device detonated, with a flash, maybe 1 to 1.5 seconds later.

What M-80 detonations lack in duration and beauty of light display they make up for in sheer decibels; they aren’t fireworks so much as small bombs. This was the most unbelievably loud thing I’d ever heard, and that includes seeing Ted Nugent in the old concrete and steel Toledo Sports Arena (also with Steve). It was concussive. Steve told me he basically could not hear for several days. The car was immediately filled with an acrid cloud of sulfurous smoke. My hand felt very possibly broken. I could neither hear nor see, and my first thought was “We gotta get out of here right NOW, before Doug comes out and sees this”. Or even worse, his dad, with a possible call to the police. But even in the best of circumstances, it’s not easy to go straight from Nolan Ryan to Mario Andretti, quickly. I could not see without sticking my head out the window, which I did until the breeze created cleared out the cab. I’m not sure that Steve knew exactly what had happened or even where he was, but didn’t have time to investigate. I was pretty sure he was alive and that would have to be good enough for the moment.

I think the evening’s festivities were concluded with this event, although I wouldn’t necessarily place money on that either. If Doug is reading, I’d like to formally apologize for the rubber patch laid in front of his house and any subsequent effect on property values that may have resulted.

Thanks for reading and please stay tuned for the next episode, in which we’ll explore how surprisingly inconvenient cul-de-sacs can be in certain circumstances, and/or other fascinating topics.

Our Lady of Tectonic Process

The following excerpt is a description of what is now the Los Angeles CA area. Note the title of the work, at bottom. The Portola/Crespi expedition is considered the first land-based, Caucasian exploration of what is now the State of California. So…just who were the people these Native Americans referred to, where indeed did they come from, and what route did they take to get there? And how long did they stay? And were the fair- and red-haired children Crespi mentions as having observed, their children?

July 28 [1769]
We set out at six o’clock in the morning from this grand plain and watering place of Santiago, following the same northwestward course of these last days’ marches, keeping on over this same plain, skirting the range on our right (to the north)…The scouts returned last night and said they came upon a full-flowing river a league and a half away…Its course comes out of the mountain range that must lie about two or three leagues away from us, from northeast to southwest…This river bed is very much lined with trees: white cottonwoods, willows, sycamores and other kinds we don’t recognize. By what we’ve seen from the sands along its banks, this river must plainly carry very large floods, and we had some trouble crossing it even now, in the depth of the dry season and dog days. There will be no crossing it in the rainy season…

We made camp close to the river here, and we have felt three strong earthquakes within less than an hour today at noon. The first and most violent must have lasted the length of a Creed, the other two less than a Hail Mary; a great shaking of the ground however was felt during all three. This is a most beautiful spot–with a great amount of soil and water, and this beautiful river going as it does, through the midst of the wide and far-ranging plain here–for founding a mission…

The heathens of this village here, who have been spending the whole day with us, brought and showed us nine cutlasses without hafts, along with four or five eyeless matting needles and a thick spike about half a yard long, all of which, they gave us to understand, had been given to them upcountry, toward the north, by some people there like ourselves, and we also understood there to be Fathers like ourselves. Whether this means they have a connection with New Mexico or the Apaches we cannot tell, or, whether some nation may have intruded upcountry toward the northward whither they were pointing.

Cover Image of Henry Wagner’s book Juan Rodriguez Cabrillo,
California Historical Society

July 29
We set out at two o’clock in the afternoon from here at the famous, large, pleasant and full-flowing River of the Sweetest Name of Jesus of the Earthquakes, crossing its bed with difficulty because of its swiftness…

July 31
On going about two leagues we came across another stream with some running water, which must carry very large floods in season because of the great deal of sand it has on its banks. We came upon such a vast number of rose bushes that a large hundredweight could have been made up with the flowers that we saw open and blooming. From horseback I plucked more than four dozen of them that came into my hands. The grapevines are countless in number, some of them large with very large clusters. We twice came upon woods so dense that it was necessary for the soldiers to clear a way to get through the various sorts of trees, willows, grapevines, cumin, holythistles, and many other kinds of tall weeds, such that it is a vastly pleasant site to see. There are vast numbers of antelopes on this plain…tracks of very large animals are seen…they say that in the mountain range running along on the north, there are a great many bears.

August 2
Our Captain and the scouts reported that about half a league or more from this spot…to the west, they came upon volcanoes of pitch coming out of the ground like springs of water. It boils up molten (and there must have been about forty of these springs, and perhaps many more, they said), and the water runs off one way and the pitch another. They reported…seeing very large swamplands of it, enough they said to have caulked many ships with…we christened them The Volcanoes of Pitch of Porciuncula. We all felt four quakes at dawn today; since we began hearing them at the Sweet Name of Jesus river, there have been fourteen, very persistent and strong though not long-lasting, and we attribute these continual earthquakes to the pitch volcanoes here.

August 6
They told us that upcountry–pointing northeastward–there were people like us–pointing to the soldiers–with guns, swords, and horses–pointing to our mounts–and there were three Fathers like ourselves (pointing to our habits); that two or three of themselves had been there; that it was reached in thirteen days’ travel from sunrise to sunset, and there was sea close by, and many large animals, which from their commentary and gestures, we thought must have been buffaloes; and that a great many people from there had come on horseback to their country, and had returned. Whether this is New Mexico or not, who can say?

Brown, A.K., ed. 2001. A Description of Distant Roads, Original Journals of the First Expedition into California, 1769-1770, by Juan Crespi

On a mission on Independence Day

Independence Day, anniversary no. 241 is upon us, a holiday surely on the short list of important ones in the United States. There’s no end to the philosophical and political ideas that can be, and have been, expounded upon. But as for me, I’m thinking about connections inspired by the research and personal reading I’ve been doing lately. Bear with me grizzlies, while I attempt to connect genetic research on rare California plants to American history.

I started graduate school in genetics, not ecology, and that field has always been the most fascinating to me. I’m in fact currently working on a topic that bridges the two fields, referred to as genetic biocontrol, the aim of which is to use genetic methods to reduce fitness, and hence population sizes, of harmful or otherwise unwanted species. So, I was sad to learn in a recent special issue devoted to the work of Dr. Leslie Gottlieb of UC Davis, on polyploid genetics, that he had passed away five years ago. The major focal taxon of Gottlieb’s work was the genus Clarkia, and I had some good conversations with him when I was doing restoration and propagation work on a very rare Clarkia (C. lingulata) endemic to the Merced River canyon, the main river flowing into, and through, Yosemite Valley. This is a famous species in plant genetics, cited in textbooks as an example of instantaneous speciation (see here and here), derived from it’s progenitor, Clarkia biloba. Clarkia lingulata is found in only two populations near the junction of the South Fork Merced with its main stem, a few miles west of the western boundary of Yosemite National Park.

Said River of Mercy is not merciful this year though, or any time at high water, should you happen to be in it. It is uncontrolled, and at high water rages through a steep, boulder filled death sieve as it leaves Yosemite Valley and heads for the San Joaquin. I still remember well the day that a woman, with her three kids, fell asleep at the wheel after driving all night, and drove off the road and into the river at daybreak, just above this location, drowning them all. When I was kayaking, the Merced was the only river that ever scared me off the river, although part of that was due to being solo and exhausted, which is a full-on recipe for disaster. There is also some very interesting history regarding Clarkia lingulata‘s location, one involving the cause of the American discovery of Yosemite Valley in 1851, but I won’t go into that here.

In one of our discussions, Les discussed some aspects of another rare Clarkia species he was working with, Clarkia franciscana. The species is so named because it is found only in the immediate area of the San Francisco Presidio–and this makes for a segue from genetics to history, involving the SF Presidio, Spain, Mexico and the United States, Upper (or New) California, and July of 1776.

The history of what is now the state of California has, I think, to rank as one of the most interesting of any in the world, and especially so from 1846 to 1850. Just a week before the momentous event in Philadelphia, a small group of Spanish Franciscans, with a small military escort, arrived from the Carmel River area (just south of present Monterey, CA), to extend the Spanish Upper California dominion northward by establishing their third presidio (military base and/or fort, the first two being at San Diego and Monterey in 1769/1770), and with it another mission of course. These establishments were significant because both were to carry the name of the order’s founder, Saint Francis of Assisi.

On June 27 1776, this group came upon a small creek draining the peninsular hills east (toward San Francisco Bay), which they named Dolores, and decided that this would serve as the future mission site. On June 29 they established an altar and consecrated the site, and then began looking for a strategic location for the presidio and fort, which they located on a high bluff commanding the narrow entrance to the bay (the “Golden Gate”), just to the water side of what is now the southern anchorage of the Golden Gate Bridge.

A depiction of Mission Dolores circa 1893 by Edward Borein

At this point, books worth of material could be inserted discussing the Spanish (and after 1822, Mexican) discovery, settlement and management of Upper California, and with jaw fairly agape at both the process and the end result (circa 1848), for much of it. And we, with the benefit of 20/20 hindsight, wouldn’t be the first to do so either–the Russians, the British, and especially the Americans, of that time, did so also. It’s documented in various writings, which are for me at least, entirely fascinating. Some of it seems to defy logic. If anyone could write an authoritative book titled “Imperial expansion: how not to do it”, it would have to be the Spanish and/or Mexicans.

To say that things went downhill for the Spanish from 1776 to 1848 would be the understatement of the century. Twenty-one missions were established in Upper California during the “mission period” from 1769 to the American conquest in 1846, but for various reasons including Russian presence starting around 1805, only two north of Mission Dolores (Missions San Rafael and Solano), and both of those quite late in the game. The events in Philadelphia five days after consecration of the Mission San Francisco de Assis (= Dolores) site, caused the Spanish to pull back on several intended plans in northwestern New Spain, now the western United States, including expansion north of San Francisco Bay and another plan for a series of missions in the interior lands east and south of it.

Mission Dolores circa 1842 by Henry Miller

One argument has it that in so doing the Spanish were hoping to marshall their energies to re-claim parts of Spanish Florida lost initially to the French, and hence to the British after 1763, thinking that the Revolutionary War might offer their best chance to do so. But this plan, along with apparently about everything the Spanish and Mexicans did from 1800-1850, backfired. Not only did they not increase Spanish Florida, they quickly lost what they already had. This was followed later (1822) by the entire loss of New Spain, i.e. Mexican independence. The fracturing of the once vast Spanish empire then continued as the Mexicans in turn quickly lost Texas to independence, whose annexation by the United States about a decade later thus led to the Mexican-American war. In amazingly short order therein, they then managed to not only not reclaim Texan territory, but instead to astoundingly quickly lose both California and New Mexico. This was followed in less than a year by surrender, and with it the additional loss of what are now roughly the states of Utah, Nevada and Arizona. The total area is a large and very valuable chunk of real estate, by any standard.

I’m no historian, though I read my fair share, but compared to the serious difficulties and drain of two wars with the British spanning 40 years, and the even more extended and often ferocious wars with various Native American tribes, this sudden acquisition of a vast, important territory without much of a fight stands out. James Marshall discovered gold in the tail race of John Sutter’s sawmill east of what is now Sacramento just about two weeks before the Treaty of Guadalupe-Hidalgo officially ceded all this territory to the United States. That this should happen to a people who had historically been obsessed and sometimes deluded by various fantastic stories regarding that material, kind of says it all. Marshall and Sutter tried to keep the discovery secret for a couple of months but that was a lost cause, and they likely knew it–it was a virtual certainty that the gold veins and placer deposits that ran up and down half the Sierra Nevada, and elsewhere, would be found in almost no time by those about to pour unobstructed into what was now the United States. And so they poured. And the rest as they say…

I will knock–just like before

I’m the latest apparition
Cutting slices in the night
I come through without permission
Moving in and out of human sight

I’m the tapping on your shoulder
I’m the raven in the storm
I’ll take shelter in your rafters
I’ll be the shiver when you’re warm

I’m the gold in California
I’m the wealth in Mexico
Like the vultures in the valley–
I will wait for you to go

I’m the gypsy in your pocket
I’m the horseman in your dreams
I’m the reason dogs are barking
I’m the hand that stops the scream

I’m the baby’s cry that isn’t
I am the distant relative
I’m the scratching in the ceiling
I am advice you shouldn’t give

I’m the ghost of a traveling salesman
My foot will be there in your door
Though I can walk through walls and windows
I will knock–just like before

Raven in the Storm, John Gorka

The Shawnee

He [Tecumseh] came of one of the most energetic and warlike of the Indian tribes. The Shawnees have always been a restless people, more adventurous than any other Indians. They belong to that family of Indian nations known as the Algonquin…The history of the Shawnees, even after the settlement of America, is wrapped in obscurity. They moved about so incessantly, and were so often divided in their migrations, that we are unable to track the various divisions. Some are of the opinion that the Eries, who are said to have been destroyed by the Iroquois in very early times, were none other than the Shawnees before their wanderings began. Certain it is that when we first hear of them in early documents, they seem to be divided, wandering, and of uncertain habitation. We hear of a war which was being waged against them by the Iroquois at the time of Captain John Smith’s arrival in America in 1607. They were at that time located to the west of the Susquehanna, and on its banks. De Laet mentions them as on the Delaware in 1632. They are also said to have been located at the South, and to have come from near Lake Erie. We can only reconcile these conflicting accounts by supposing them to have already divided into several bands, some of which were in motion, for other authorities place their seat, in the latter part of the seventeenth century, in the basin of the Cumberland River in Kentucky. Later they are found on the Wabash, where Tecumseh long afterward made a new settlement, and in 1708 they are spoken of as removing from the Mississippi to South Carolina. The Swanee or Suwanee River, in Florida, derives its name from a party of Shawnees who had come from north of the Ohio.

Continue reading

Threaded between tragedies

August 24…just as we were commencing the ascent of the mountain, several Indians made their appearance, about fifty yards from the trail. The leader and chief was an old man, with a deeply-furrowed face. I rode towards him, holding out my hand in token of friendship. He motioned me not to advance further, but to pass on and leave him, as he desired to have no communication with us. I insisted upon the reason of this unfriendly demonstration; assuring him, as well as I could by signs, that we desired to be at peace, and to do them no harm. His response was, if I understood it, that we, the whites, had slaughtered his men, taken his women and children into captivity, and driven him out of his country. I endeavored to assure him that we were not of those who had done him and his tribe these wrongs, and held out my hand a second time, and moved to approach him. With great energy of gesticulation, and the strongest signs of excited aversion and dread, he again motioned us not to come nearer to him, but to pass on and leave him. The other Indians, some six or eight in number, took no part in the dialogue, but were standing in a line, several yards from their chief, with their bows and arrows in their hands. Finding that it would be useless, perhaps dangerous, to press our friendship further, we continued our march. I have but little doubt, that these Indians are the remnant of some tribe that has been wantonly destroyed in some of the bloody Indian slaughters which have occurred in California.

Continue reading

Rough and Ready

In the fall of 1849, the “Rough and Ready Company” of emigrants, under Captain Townsend, composed of some dozen men, from Shellsburg, Wisconsin, arrived by the Truckee route at a point on Deer Creek near the mouth of Slate Creek; they mined successfully there, a few weeks in the bed of the creek; one of their number went out to kill some game, deer and grizzly being plentiful, and in quenching his thirst at the clear stream of the ravine below Randolph Flat, discovered a piece of gold on the naked bed-rock. Consequent prospecting by the company satisfied them that the new found diggings were rich, and removing their camp, they prepared winter quarters by building two log cabins on the point of the hill east from and overlooking the present town of Rough and Ready. Two of their number struck out through the woods “on a bee line” for Sacramento, to procure provisions, and thus made the first wagon tracks on what afterward became the Telegraph road. From the name of this company, the settlement and town afterward derived its designation…

Continue reading

Behind the face of need

A man conceived a moment’s answers to the dream
Staying the flowers, daily sensing all the themes
As a foundation left to create the spiral aim
All movement regained and regarded both the same
All complete in the sight of seeds of life with you

Changed only for a sight, the sound, the space, agreed
Between the picture of time, behind the face of need
Coming quickly to terms of all expression laid
Emotion revealed is the ocean maid
All complete in the sight of seeds of life with you

 

close-to-the-edge-inner

 

 

 

 

 

Sad preacher nailed upon the color-door of time
Insane teacher be there, reminded of the rhyme
There’ll be no mutant enemy we shall certify
Political ends, as sad remains, will die
Reach out as forward tastes begin to enter you

I listened hard but could not see
Life tempo change out- and inside me
The preacher trained in all to lose his name
The teacher travels, asking to be shown the same
In the end we’ll agree, we’ll accept, we’ll immortalize
The truth of the man maturing in his eyes
All complete in the sight of seeds of life with you

And you and I climb, crossing the shapes of the morning
And you and I reach over the sun for the river
And you and I climb clearer towards the movement
And you and I crawl over valleys of endless seas

And You And I, Jon Anderson, Yes

close-to-the-edge-cover

When it happens to you

Well she was old enough, to know better
And she was strong enough, to be true
And she was hard enough, to know whether
He was smart enough, to know what to do

And you can’t resist it
When it happens to you
No you can’t resist it
When it happens to you

And you can tell your stories
And you can swear it’s true
But you can save your lies
For some other fool

And you can’t resist it
When it happens to you
No you can’t resist it
When it happens to you

You can’t resist it, Lyle Lovett (with Leo Kottke)

And if that doesn’t do it for you, this should: