Brunosan

Why is the mirror of the Google Glass inverted?

2013-05-06T00:00:00-04:00

These past days I have had the chance to play for the first time with the Google Glass from Silica Labs. This post is not about the experience, you can find about that online. What drove my interest is the hardware, in particular the eyepiece. If Google Glass is to become popularized, the eyepiece needs to be as small and invisible as possible. I´m pretty sure they have invested a lot of time on that. Upon close inspection I think they have decided to ship a first option, but they continue to work on that. Perhaps as an “innovation slack” to keep ahead of copycats, or because the main audience for now are developers to build apps for the consumer market later on. And developers don´t mind the geeky aspect that much.

If you look closely you´ll see that the eyepiece is a nanoprojector inside the bulk piece. The little image is sent sideways to a 50% mirror that directs half the light to the eye. This is done to also allow 50% of the light from the outside reach the eye, thus making the image superimposed over the real world.

But if you think about it, the mirror on the Google Glass is inclined in the opposite direction. Why?

On the first option (as the first diagram above) the image gets 50% of the light. With the opposite direction (as the Google Glass), you only get 25% of the light: the beam goes twice over the half-mirror, once through and once reflected. The second option also needs a mirror at the end of the eyepiece to reflect back the light. On the image below I´m taking a picture of the outside of the mirror, along the axis of the eyepiece.

Why the second option, with a double pass? I think the reason is augmentation. The mirror at the end is not flat, but slightly spherical (or parabolic). A concave mirror creates an augmented virtual image if the object is before the focus. On the side image you can see the piece is nearly flat but no quite so. Thus the image you see on the outside (convex mirror) is smaller. The radius is much bigger than the distance to the eye. The virtual image if the screen is therefore then bigger than it would be with a flat mirror.

This means that the overall light path on the eyepiece of the Google Glass looks like this:

After playing with the Glasses for a weekend and letting anyone use them, they all love it, but agree that it looks too… geeky, too bulky. I believe there must be better options, smaller and less visible when you wear them. Perhaps:

A single non-flat semi-transparent mirror instead of the flat one. This what we use on a special type of telescopes. They are called asymmetrical or off-axis. They come with their own set of problems (like chromatic aberrations), but in such a small light path I would still argue to try it, with some combination of software to correct these problems. Still, that option would probably introduce strange field aberrations on the transmitted background image.
Direct projection from the side of the glass into the lens. Problem here would be to place inside the lens and yet outside the normal field of vision, so it´s only available on the corner, not in the middle of your normal view.

Ideas for Apps for the Google Glass

After using them for a weekend, talking about them with colleagues at Silica Labs, and wear them at various situations, I got a few ideas for apps:

Climb app. Save videos and pictures of your climbs. At any climbing place, make a picture of the route to help you flash tricky points, locate best places for hooks and save videos as you make your way up. If the route is long, the belayer can talk and see the climber and videochat to give support without releasing the rope.
Talking queues. When you meet someone and start talking, start flashing cards of his/her interest, pictures, notes. Content could come from the CV if it´s a work interview, OkCupid profile if it´s a first date, live speech assistance from someone who is listening and watching what you see, …
Buffer record. Continuously save video or frames every few seconds, and overwrite them in a continuous loop. This can be important in case of accident if you are on the road, you are a police officer, or you think you saw something worth saving when it already started.
Glass Notifier. An app that just pushes notifications to your Glass. I´m thinking of any app that currently flashes notification on your phone. Just taking them and pushing them as a card to the glass. (Agenda, Whatsapp, Facebook, Instagram, Google Now, Email)

[Idea] Apple patent to protect your phone screen

2013-04-08T00:00:00-04:00

How to avoid breaking your glass screen when you drop your phone? Phones fall down, and sometimes nothing happens and sometime the screen shatters. Can it be avoided? Making the glass thicker is a solution but increases the weight. I got an idea to fix that.

Today I was attending a talk by Mark P Mills on Patent Law at the Notre Dame Law School. He mentioned that Apple had just patented a way to prevent shattered iPhone screens, but he didn´t go to the details, just that he thought it was difficult.

Apple indeed has a patent to protect the hard disk when your laptop falls. They do so by detecting a free fall via accelerometers. When the laptop detects it has been dropped it will park the read/write heads of the hard disk, thus avoiding those heads scratching the surface of the disk when it hits the ground.

But how do they protect the screen? The idea would then be to detect a free fall, and quickly orient the phone in the air so that it touches ground on the back, or on the side, or whatever position is deemed most likely to help survive the phone with minimal damage.

When I wrote this post I didn´t look at the patent, but I thought about it and this my idea: Drop the battery.

Rotating the phone mid air requires a burst of energy. You could get there using puffs of air, but that would take space for the hardware. You could also use gyroscopes (like satellites use) but they are not that quick. You need to move a significant portion of the phone mass to rotate the thing quickly…. the battery.

The idea is then to insert 4 special screws on each corner of the battery. When the accelerometers detect that the phone is falling, it would calculate the momentum of inertia (how exactly is the phone is rotating) and carefully release 2 of the screws to create a counter momentum of inertia that compensates that rotation. As the battery unwinds the rotation is compensated. If the rotation needed is less than the full 90˚ turn, the other 2 screws will also be released, letting the battery separate at the exact moment the phone is properly oriented ( thus reducing also the weight on impact).

To gain more momentum of inertia of the rotation of the battery, one could increase the angular speed using screws with small springs. Here is a quick sketch of the whole thing:

No idea is that is what Apple thought of, but may be there is a chance Apple would finally make replaceable batteries in the end. :)

Update: For reference, as I upload this, I find the Apple patent online. I still thing my solution is better and simpler :)

How a tweet led to a rocket factory

2013-03-14T00:00:00-04:00

This is the little story of how Foursquare and Twitter led to a VIP visit to Space X, the rocket factory, back in 2010.

I was at a conference in Irvine, CA. This is when I worked as a fellow of the Space Studies Board of the National Academies. The sessions were great but I had no plans the evening of the first day. After I was done with work I went to the bar for a quick bite. It occurred to me that may be they had a Foursquare deal, so I used my mobile to check in. Indeed they had a free appetizer if you ordered a beer. Cool.

I also noticed someone else had checked-in there 10 minutes before. Out of curiosity I clicked her profile and she had a linked Twitter account. She was a science journalist that was also participating on the conference. Few minutes before she had tweeted about the same appetizer I had ordered. She had done a great presentation earlier that day and I was dying to know more about her work. I tweeted her and waited for her phone to ping to make sure she would tie that random connection when I introduced myself:

@christienic funny, we were actually sitting next to each other at the bar, and I also got the offer. Btw, good points on the panel.
— Bruno Sánchez-A Nuño (@brunosan) November 8, 2010

We started chatting about Twitter and Foursquare which were fairly unknown services back then. Really nice conversation. Then we talked about the conference, our common background interest in space and what not. I must have had a poker face when she asked If I knew Space X. Turns out she is a good friend of Elon Musk, and she was going to visit the rocket factory the next day. She then offered me to come along.

Elon was somewhere else taking care of some “Tesla” thing that day, but his personal assistant Mary Beth and the astronaut Ken Bowersox gave us an awesome tour around the rocket factory. I had many technical questions, and Jeff was very keen to answer all of them. Had I been american I would have quit my job at NAS to work there (ITAR regulations prohibit us foreigners to do so, or even see some types of hardware. My previous work at Department of Defense calibrating a rocket camera was borderline, so I knew well about it. For my work at NRL and visits at the NASA JPL and the Pentagon I was kind of used to that. Very annoying). Still with the ITAR restriction Ken and Mary Beth were very accommodating to make sure we stayed on ITAR limits but make it a great experience.

And that´s how I got a personalized visit to Space X, by the hand of an astronaut. You can see all the pictures here on Flickr. Surely it was not only a tweet, but a combination of really awesome people like Christie, Mary Beth and Ken, my professional interest on space and being so proactive. But still, like in this story where Charlie Loyd landed a job at Mapbox it was the flapping of the butterfly wings.

Thank you Christie, Mary Beth and Ken. You are awesome.

Young Global Leader 2013

2013-03-12T00:00:00-04:00

Today the World Economic Forum (WEF) announces this year awards of “Young Global Leaders” (YGL). I am one of them.

According to the website:

“Young Global Leaders represent the future of leadership, coming from all regions of the world and representing business, government, civil society, arts & culture, academia and media, as well as social entrepreneurs.

Nominated under 40, these young leaders are proposed through a qualified nomination process and assessed according to rigorous selection criteria accepting only the very best leaders who have already demonstrated their commitment to serving society at large."

Back in May 2012 I was asked by a work colleague who was interested in our work if I minded to be proposed to be YGL. I didn’t know what that was, but we then talked about my life, and the motivation behind my perhaps peculiar professional path. Plain honest conversation.

I then read a bit about the YGL, and I was highly impressed. Nominations go through a internal and external evaluation and screening process. Some of the previous awardees include Sergey Brin (Google), Jimmy Wales (Wikipedia), Marissa Mayer (Yahoo), Mark Zuckerberg (Facebook), and countless other successful CEOs, ministers, athletes, … I found a wonderful Youtube Channel with short videos to “Meet the YGL”. Very motivational and impressive.

And so I forgot about my nomination.

To my surprise, just few weeks ago, I received an email with this Subject:

“You Have Been Honored as a Young Global Leader 2013”

And so I started receiving information about the group, activities, a fellow “mentor”, events, … Turns out I am the only spaniard this year.

Since then I have spent some time back on that YouTube channel, reading information provided, talking with fellow YGLs, interacting with the YGL headquarters in Geneva and hunting tweets and blog posts of other YGLs.

I don´t know yet what really is the YGL, but I am certain this is an amazing opportunity and an honor. I am truly humbled and thankful for this nomination. Reading the roster of fellow YGLs, I can only take this as an acknowledgement of a trajectory that is still very short. It has not been easy, and the passion that drives me rejoices with this opportunity: a clear motivation to help bring Science and Technology closer to Society.

Being a YGL, I am told, is a transformational experience. Certainly, having the chance to get to know such a Forum of doers will be motivating.

For now, I´m planning to squeeze as much time as possible to take the most of this opportunity WEF is offering. Let´s do it.

“I don´t know where is the limit, but I know where the limit is not” @josefajram

Update: Yahoo Spain has run an interview, in spanish. At some point it was on the front page of Yahoo. Crazy day.

Walking the talk in climate change

2013-02-13T00:00:00-05:00

“It is not about the Science, it is not about mitigation, it is not about projections… it´s about all of that, on top of the current adaptation gap”.

This is what I was thinking as I left the NAS/NASA workshop Walking the talk: Climate Science in Service to Resilient Federal Properties.

This workshop, incidentally, was canceled few hours short of its original date, October 31st, due to Sandy. On the second attempt, it was again almost cancelled, due to exceptional wind and rain conditions in DC. That pretty much summarizes my take-away from the meeting.

NASA, as a forerunner, shared their progress securing their federal facilities against the changing climate. They talked about the process from the science of data and models, to the implementation of adaptation measures such as physical storm surge barriers at their coastal rocket facilities.

I found of special relevance the importance they gave to avoid a purely science driven approach. It´s just not only about the science pushing data to decision makers. Indeed, NASA partnered with architects, contractors, journalists and other stakeholders with the goal of being responsive to all actors involved on this process.

Still, the disconnect exists. Even on a room filled with federal employees whose boss in 2009 established a presidential Climate change Adaptation Task Force (along with the mandate to “manage the effects of climate change in short and long term”), as well as concrete 2011 Implementing Instructions for Federal Agency Climate Change Adaptation Planning. One of the first question from another federal employee was about the uncertainty of climate science. To him, inaction was wiser until we know to what we should adapt to. And so we debated and polarized even more the issue.

Where is this disconnect coming from? Part of this, I believe, is that climate change is linked to mitigation and future consequences. The future overshadows the present. For example, sea level rise is present on every discussion on climate change, while storm surges are often not so. Yet these already have costly consequences. Perhaps we´ll see this loop closing as people connect with the unfortunate, reality of increasing extreme events like storm surges, droughts, blizzards… Here at home, not overseas on the TV. As we connect with our current climate vulnerability: There is already an adaptation gap that needs to be addressed. Building resiliency, protecting lives, saving ecosystems, safeguarding livelihoods, protecting investments and infrastructures, identifying new opportunities and challenges. This, in turn, will unleash an inclusive role of all stakeholders, including the underplayed private sector.

Of course, on top of this current vulnerability, challenges will increase, specially for the most vulnerable.

Perhaps americans, as constituency, will connect with the climate and thus push it up on the political agenda. Perhaps then political leadership will catalyze. For that, a second term president certainly helps.

Treating climate change as a mitigation issue with incremental impacts in the future is ill-informed. Manhattan probably has the greenest least polluting modern buildings, and yet it will continue to flood until we take action on building resilience to climate.

Upgrading the World Data Bank

2013-01-17T00:00:00-05:00

The Open Data Catalog at the World Bank is a fantastic resource. I´ve used it many, many times. Specially while developing the GAIN Index. I´m sure I´ve spent hours on that domain. It´s astonishing the amount of information they´ve gathered on a consistent way.

Our Gain Index indeed uses, and references, roughly 80% of it´s components directly back to the Catalog. Proudly, but oddly, a number of people have told me that they use our page instead of the Bank´s to get a glimpse of the data for a particular country. As humbling as that is, it´s odd that our page gets visits to see the components, not the aggregation we do for our purposes. It´s like buying a book for the Bibliography.

Last Thursday the data team posted a call for input. They are upgrading the Catalog to be “more technologically capable, but also more user friendly, readable and understandable”. On their summary of the proposition they highlight: Search, Federation, user friendliness, better metadata, and scalable.

These are my six points of feedback: fast, flat, visual, better feedback, support more granularity of data, and open the source.

SPEED

The Catalog must be fast. And I can propose a concrete benchmark:
Faster than using the traditional sets of Excel files many people use. That´s the inertia from part of their target users. Why would I prefer to fire up the website every time If I can download the whole thing to my disk? This is specially important if you don´t have a fast connection, like it happens in many parts of the world.

Avoid full page reloads. A substantial percentage of the page will be the same, and it will not have the content I am looking for ( navigation links, legal/institutional notices, …). Having light overheads of headers and technologies like AJAX could help with this.
Transfer to the client browser as much as possible. Browsers can do amazing things, not just painting server states. Either using things like node.js or via other options, it could speed things up significantly. Also, the less the servers do, the more the IT and security guys at the Bank will like it. Back when I worked inside the Department of Defense, Javascript was golden, and PHP was evil.
Asynchronous calls. Avoid linearity. This is related to avoiding full page reloads if I, e.g., want to add a country to a plot, or I´ve requested more info on a particular data point.

All these points are actually related to make the Catalog feel less like a website and more like an app.

As flat as possible

This is also related with speed. Minimize the clicks-to-target. The distance from the front page to any piece of information should less than, say, three clicks, or two. Same for the most common cases like i.e. showing the world´s GDP compared to x and y country.

Low friction for basic users

If journalists and general citizens are target groups, I believe having a simple visual landing page is very important. Something they might be used to. Something like a Google Search style. Faceted searches are good for experts but intimidating if you don´t know the answer to all options presented. And they occupy a lot of space on the screen.

I also love maps, interactive maps. Personally, and this is the way I´m leading the upgrade of GAIN, a simple interactive map like GAIN and then adding search fields like Google (or Wolfram Alpha). Procrastinating folks can navigate the map slipping towards deeper levels of detail. The search box, small but prominent, gives you access to everything. Coupling it with some predictive/popular suggestion could be very powerful to allow one click links to queries like:

GDP of the World, and Spain and Sub-Saharian countries.
Top 10 countries by Population.
What measures and years aer available for Nuaru since 2000?

Feedback

Over the course of these 2 years working with the Catalog I´ve found around around 50 cases of small bugs, missing explanations or unexpected behaviours. The feedback channel, an email, is very opaque to the rest of the users. Certainly the answer has been always positive and every email I´ve sent was replied. Even when I tweeted about it, like when I found out that apparently 99% of the nurses in Netherlands had been fired:

Thanks for the Netherlands Nurses find @brunosan - I'll take a look! The source data from the WHO looks the same, so we can ask them too :-)
— Tariq Khokhar (@tkb) August 1, 2012

…

@brunosan Bizarre. That or maybe they only divided the midwives and not midwives + nurses. Will pass it on - thanks again!
— Tariq Khokhar (@tkb) August 1, 2012

It would be great If you could flag data, or measures, and leave feedback, or requests. Other users could reply, or vote, raising prominence to the issue. I can see how this could also pressure the original sources of the data which on many causes are also the root of the issue, like the example above with the nurses data.

Fixing identified issues with the data is specially important if you intend to use the data directly via APIs. Otherwise, you need extra steps to fix those issues.

Towards organic resolution

This is a big one I´m also struggling with. Most of the available data is at the national level. There are many reasons for that, from feasibility and gathering constrains, to pragmatism to the scale most users wanted. Nowadays it´s increasingly easy to get data, but also we demand more granularity. For some Indicators national scale makes sense, like fiscal policies, but for many other the higher the resolution the better. For example access to energy at the national level is important to prioritize in the global context, but knowing on which region to focus is equally relevant. This is specially the case for big heterogeneous countries, like India, China, Russia, USA …

What I’m proposing, and is the direction I´m planning to develop at GAIN, is an organic resolution protocol:

There is a background database with all available measures, at all available resolutions, as a network of nodes with plenty relations among them based on geography, economy, culture, correlations, …
If the user request one of them, it´s instantly served.
Sometimes one might need aggregates of available data, like for example all countries from LAC, or spanish speaking, or cities inside this watershed. The interface then requests that job and it´s served aggregated as the median or envelope values (this can probably be also done at the client level via Javascript).
Sometimes the data is simply not available, and then some working assumptions could be suggested like based on geographical neighbours, similar countries in other correlated measures, temporal extrapolations … In these cases probably more computation is needed, specially to provide degree of uncertanty.

Open the Source

Just like the White House releases much of the web tools they developed, the World Banks could do that. Even more, more than releasing, engaging in a conversation with the community e.g. using github: aggregating forks, issues, pull requests…

The data portal is a fantastic tool for both researchers and casual users. It´s already well ahead of the curve. But there is much more it could be done, no only to improve the portal and consumption of data, but also to spread best uses and help others catch up on their online resources.

Back in April 2010, the former President of the World Bank, Robert Zoellick, said:

"It's important to make the data and knowledge of the World Bank
available to everyone. Statistics tell the story of people in developing
and emerging countries and can play an important part in helping to
overcome poverty."

or as an american told me:

"In God we trust... but everyone else bring data"

2nd Marathon

2012-12-02T00:00:00-05:00

Last week I ran the NCR Trail Marathon in Baltimore. This is only 4 weeks after my first marathon.

I didn’t actually train for it, as it was not on my plans four days before the race. I crossed the start line breaking plenty of recommendations, like getting some time off, new shoes, new clothes, no music for the first time, first trail race, no setting goals or paces, …

Somehow I managed to pull quite an enjoyable race and beating my best marathon time.

A friend of mine was running this race with someone else, but that person cancelled. A thought I could just join my friend. The race takes place north of Baltimore, so he had to take the train the day before, then the tram, get a hotel, go to race, and then come back. I decided to go with him, start the race to help him pace. I had run my first marathon only 4 weeks prior, so I didn´t want to push my body for too much. Deep inside I also wanted to try, as I felt pretty well recovered from the first race.

This is a trail marathon but, thankfully most of it goes using an old train line. This means that it goes through a forest on an unpaved road but is mostly flat. The course is back and forth, with roughly a 75m elevation change in the turn around:

One of the most important rules I´ve read about marathons is never to try something new on the race day. Well:

First time travelling for a race, with all the hotel, eating changes, and not seen the race track before.
New shoes.
Forgot my Gu energy gels at home, so I planned on just using dates as fuel, and a couple of Gu I could get.
New thermal clothes, since it was going to be quite cold.
First time running without music. (Not allowed, but then most people had music… too late!)
I´ve never paced someone else or run with anyone. …

This is also a small race in the forest with 400 participants, not a 30.000+ in DC, so most of the time you will be alone, and probably among better runners.

The race

This is the School where we gathered before the race:

When I crossed the start line, my goal was to keep my pace not faster than a 4:30 marathon, and run along my friend. Right on mile 3 or so, we decided to split ways. I was feeling great so I increased the pace a little bit, but always keeping it slow. Reading my bpm after the race, I managed to keep it well below my anaerobic limit of 150, and I was running pretty smooth in the 130-140 range. It was quite cold so sweating was not a problem, and the new tighter clothes kept me warm without chaffing risk. I was running on a small pack of 4 people, without minding my pace much, any race strategy or anything. It is surprising how your mind just drifts off. I had no music but watching the forest around, feeling my own body and keeping the pace of the group was actually good. I didn´t miss the music at all.

Since I only had 3 Gu energy gels instead of the 6 I used in the previous marathon, I decided to keep the dates I had brought for breakfast and eat 2 every 30 minutes or so. I rationed my Gu to eat one each way on the half point of the back and forth track, and also one the turn around. I have to say that it feel much better on the stomach and I had no lack of energy compared to using twice the gels like last time.

Getting to mile 13 was quite a good feeling. I only had to go back. It is a bit of an irony that you ran all these miles only to find a sign that says, “turn around”. That’s the image at the top of this post.

Despite having only run once since the last marathon 4 weeks prior I was in pretty good shape. I had a small pain on my right hip, which I think might be for running too slow. My hypothesis is that I was running slower than I usually do, so I started to bounce up and down instead of down and forward. That little extra bounce down was being absorbed by the knees and hip, and hence the pain. And of course, the lack of training.

On the turn around I made a small pause to make the picture and also drink water. When I went back, after a mile or so, I decided to do the rest of the race, now slightly downhill, enjoying the forest and pushing a little bit more. According to Strava that has all my runs since 2009, on that second half marathon I made my best ever 400m (4:07min/km) but also best ½ mile, 5km, mile, 10 km, 15 km and 10 miles. I think I overtook 50 runners or so in the second half.

The famous mile 20, where many people hit the wall went by and I was managing the struggle. The pain in the hip was worse but I wanted to finish without walking. Unfortunately mile 24 had a horrendous slope that I just couldn’t do, so I walked it. A fellow runner cheered me to push, as the finish line was just a mile away. It was hard, but I went back to running, and when I saw the end I managed to squeeze a sprint to cross with 180 bpm, a full smile and the surreal fact I had done a faster marathon.

Results data

The results for all runners are already available. This time there is no location data, so no mapping for these runners. What I could do is create the same visualizations as last time, and compare the statistical results with the last race.

This race is 8 times smaller, and my hypothesis is that you would find mostly good local runners yielding better times on average, runners from far away trying to qualify to other races, like Boston, also faster, and also another set with slower local runners trying the distance.

When I was nearing the turn around I found the runners on the lead. Turns out that the second one, who was already struggling when I saw him, was running his first marathon, and he did not use the water stops, or had any gel. He was crazy fast, and I later knew he hit hard the wall around mile 18 and was taken to the hospital. Not a joke!

Note: For the comparison of this race (NCR) with the Marine Corp Marathon (MCM) I had to normalize the histograms to the total number of race participants. Otherwise the NCR histogram is too small to be seen. So when you see 0.04 on y axis it meas 4% of the finishers.

Runners by Age

66% of participants were male. Most women were actually grouped on the 35-40 bin. Compared to the MCM, the 25-30 women were mostly missing, but on the other side, there were more women 35-40. Numbers for women are much more concentrated than men, whose biggest group was 45-50 years old. Overall there seems to be less younger participants.

Runners by time

In terms of time, this race (which is 7 times smaller) deviates significantly from the nice Gaussian shape of the MCM. Smaller races don´t yield scalable results. The 4 hours milestome is much more apparent in both men and women, with another peak for women around 4:30. After that time, there is consistently better times for this smaller race than the more popular MCM. Big races attract slower runners. On the other side fast runners are proportionally much more prominent, specifically faster men.

Combining both dimensions one gets again the scatterplot of Hours by runner Age and Gender. As I was mentioning in the beginning, the smaller NCR race seems to attract mostly faster runners, regardless of Age. The group of faster men in the outer rim is still apparent but vague.

From this plot one can also see the bigger more famous MCM race is able to attract many more (overall) slower runners, specially on the 15-35 age range.

Most of the results lie within the envelope of the bigger race, which thus indicates that a bigger, more popular race, also includes the type of participants of smaller races.

Also like last time, I try to estimate those who are not able to keep a constant pace, and e.g. hit the wall on the second half. This does not include those who have to leave the race, as only finishers are taken into account. This might be important, as I can guess that the cold temperatures made many walkers quit the race.

NCR runners on this smaller race seem to be able to keep their paces much better. The graph is remarkably vertical without much deviation on either side. The Wall is barely visible with an impact of less than 1 hour on the finish time.

Again women seem remarkably better at setting a pace and keeping it the whole race.

Some extra Resources

All the python code and data for the visualizations is available on this github repository.
I used this nifty Flickr Visualizer to check all race pictures quickly and find the only one among 400 or so where I am.

Does the Universe have a purpose?

2012-11-30T00:00:00-05:00

Neil deGrasse Tyson is, to my taste, one of the greatest outreach scientist of our time. Not long ago, the Templeton Fundation asked leading scientist and scholars “Does the Universe have a purpose?”. His answer is the one I like the most.

Anyone who expresses a more definitive response to the question is claiming access to knowledge not based on empirical foundations. This remarkably persistent way of thinking, common to most religions and some branches of philosophy, has failed badly in past efforts to understand, and thereby predict the operations of the universe and our place within it.

To assert that the universe has a purpose implies the universe has intent. And intent implies a desired outcome. But who would do the desiring? And what would a desired outcome be? That carbon-based life is inevitable? Or that sentient primates are life’s neurological pinnacle? Are answers to these questions even possible without expressing a profound bias of human sentiment? Of course humans were not around to ask these questions for 99.9999% of cosmic history. So if the purpose of the universe was to create humans then the cosmos was embarrassingly inefficient about it.

And if a further purpose of the universe was to create a fertile cradle for life, then our cosmic environment has got an odd way of showing it. Life on Earth, during more than 3.5 billion years of existence, has been persistently assaulted by natural sources of mayhem, death, and destruction. Ecological devastation exacted by volcanoes, climate change, earthquakes, tsunamis, storms, pestilence, and especially killer asteroids have left extinct 99.9% of all species that have ever lived here.

How about human life itself? If you are religious, you might declare that the purpose of life is to serve God. But if you’re one of the 100 billion bacteria living and working in a single centimeter of our lower intestine (rivaling, by the way, the total number of humans who have ever been born) you would give an entirely different answer. You might instead say that the purpose of human life is to provide you with a dark, but idyllic, anaerobic habitat of fecal matter.

So in the absence of human hubris, and after we filter out the delusional assessments it promotes within us, the universe looks more and more random. Whenever events that are purported to occur in our best interest are as numerous as other events that would just as soon kill us, then intent is hard, if not impossible, to assert. So while I cannot claim to know for sure whether or not the universe has a purpose, the case against it is strong, and visible to anyone who sees the universe as it is rather than as they wish it to be.

Running a marathon

2012-10-29T00:00:00-04:00

Last week I ran the 2012 Marine Corps Marathon. It was my first marathon, I did it as vegan, and it was a great experience. Finish time was 3h54m17s, much better than I expected. To get to that point I trained hard, I focused on my diet and my stride, I read papers on strategies, and I geeked with the data. I want to talk about all that.

I can proudly say that the marathon was a challenge, but not a suffering. I did the last mile with a smile on my face and a strong 15 seconds full sprint to the finish line. Two days later I barely had soreness in the legs, and none of the usual problems: my toe nails are intact, no blisters, injuries, chaffing or particular pain. I base this success in 4 things: Stride, Fitness, Nutrition, Motivation.

1. Stride

“Every year as many as eighty percent of runners suffer an injury that will keep them off the streets for a month”, quotes Christopher McDougall. It doesn’t make much sense to me. Running should be natural, why do we get so many injuries? These kind of thoughts lead me to consider the minimal shoes Vibram Fivefingers. I read a couple of articles, I followed a couple of threads and that was enough for me to switch to the forefoot strike

The key points to me were:

The running man hypothesis makes sense to me.
I’ve seen and stretched an Achilles tendon on a dead body (at the Faculty of Medicine). They really have some elasticity. I can see how part of the landing weight is absorbed by the elasticity and released when you push, minimizing the energy to run. Incidentally you can see on the image above a fellow runner with a prosthetic limb using the same principle with the arched metal.
Landing on the ball of the foot really uses the feet that otherwise are quite passive.
The stride changes your posture and makes you run more straight and stable above the waist, landing your feet right above the center of mass, so there is no vector pointing backwards. It’s just pushing back all the time.
The posture also make you land already with a slight bend on the knee, minimizing the impact on the knee and hips. This point was very important, as I would usually get pain there after long runs.
Quite literally the absorption of the impact is done by the muscles and tendons, not the bones and articulations.
It just feels smoother and lighter.

The only drawback is that, since we always use running shoes, we have never run like that, and muscles and tendons are not prepared. You need to go very slowly or you will injure yourself (like I did last year when I went cold turkey to the Vibram). At first it will feel awkward and it will be hard on your Soleus, Gastrocnemius, Aquiles, soles of the feet and thigh. It helps to walk barefoot at home. It soon pays back, I really believe this stride contributed to running longer, faster and with less pain or soreness afterwards.

I did not run barefoot like at least one people I saw, or with minimal shoes, like a few as well that I saw, but I would not discard it. As I run, I barely use the heel, so the extra cushioning isn’t really involved on my stride. However, having that option helps and on a few occasions I changed my stride to heel strike to release tension and change the cadence as I felt it.

As a side note. When I finished, after a few minutes, I started to feel my feet cramping. I removed the shoes and walked on my socks. It worked immediately, relaxing my feet. They just wanted to get free from the shoes and stretch out a little bit, just like my legs.

2. Fitness

Together with the stride, you need to have physical fitness to endure the 26 miles. For that you just need to bank as many miles as possible, without over training. I followed a training plan from Adidas MiCoach, but adjusting it whenever I didn’t feel good or I had any problems. I didn't´really pay much attention to the suggested change of pace, I just kept it on a comfortable pace of around 10km/h. Before this training the longest I did was a half marathon in March, before changing the stride, and that was my absolute maximum. I felt awful when I finished, much pain in the left hip and soreness for several days. In the 100 days I trained for this marathon, I went for 5 km/week the first week, to 70 km/week 4 weeks before tapering for the race.

From what I could learn, one of the keys of endurance training is learning about Glycogen dynamics. Glycogen is basically the main energy storage of the muscles. The other one, less efficient but more compact is lipids. Some is stored directly on the muscles for immediate use and most of the rest on the liver where, on demand, will be broken down to glucose and send with the blood elsewhere. So the more your muscles store, the better. Training helps muscles store more Glycogen. When we ran out of it, we start to use the other fat reserves. When we deplete glycogen, we hit the wall, typically around mile 20. That’s why is helpful to replenish glucose as you run. My choice was GU and Clif Shots every 45 minutes or so. With too many of either you get tired of their taste. Even so, my stomach was not too happy after so many of those. If think next time I´ll try peanut butter sandwich, falafel roll or something more… food resembling.

Another factor here, the metabolic rate, is very important. When there is enough oxygen the preferred cell respiration to get ATPs on the Slow Twitch fibers (the ones for endurance) is aerobic. You want that for low intensity over long periods of time. If you need more energy than the aerobic metabolism can provide, the body uses the anaerobic pathways, which help get that sprint (mostly on the Fast Twitch fibers), but also produce lactic acid at higher rates than the body can clean, which will make you cramp and even vomit. You want to keep the anaerobic rate down for as long as possible. There is a rough relation of heart rate (oxygen blood flow) and metabolic regime, for my age (31) is below ~150 bpm for aerobic exercise. You want to keep it below that range, or you will start to consume your precious glycogen anaerobically while your lactate acid builds up. I used a heart rate band for some runs, until I learn to recognize which levels of effort meant which heart rate, and then I was pretty good at keeping it around 150, which also meant around 10 km/h. See my heart rate progress on the marathon run:

It was actually quite quick to get there, just 250 m:

Unfortunately my phone battery died 2 miles before the end (…some call it progress…) so I don´t have the pace or bpm during the 15 seconds or so of the sprint. From previous times I think it would be around 25 km/h and closing in into 180 bpm or more when I stopped. I would have loved to see the recovery time back to the resting 60sh bpm or so (perhaps 10 minutes? no idea). From other runs I know I can go down from 170ish to 110 in one minute, but I presume the decay is proportional to the bpm, so from there to 60 perhaps another 5 minutes or more.

My guess is that a key to run faster marathons is to train your body to run faster keeping it on the aerobic side.

3. Nutrition

Since last Christmas I decided to try to be vegan. No meat (yes, chicken is also meat), eggs, milk, cheese, honey. Nothing from an animal. I did it mostly for environment reasons (although I find more and more), and switched back to vegetarian or even meat eater whenever it was too difficult (e.g. when I went to Rio de Janeiro, they eat meat on everything!). All in all I eat meat roughly once a month. I had gone to the doctor before becoming vegan of preference and 3 months later for blood and physical test. Everything was great, I felt better and lost weight. But my doctor did not approved it. When I told him I was planning to train for a marathon as a vegan, he called me stupid. Literally.

I find more and more reasons to reduce my meat intake as much as I can, but I wasn’t sure about running a marathon on plants. After reading some more, and learning about many ultramarathoners are indeed vegan, I decided to try, but monitoring my health. I am not an expert in nutrition so I keep it simple: tons of cereals, many legumes and veggies, some fruit and a bit of olive oil. This is my grocery shopping for a couple of days, 90% from the produce section:

In particular I enjoyed the extra intake of quinoa, beets, nuts… I plan to continue like this. Please read “In defense of food”, it´s a great book. I don’t think being vegan makes training for a marathon any harder, perhaps on the contrary. And it makes you be more conscious of what you eat, which is good. And Mint says no significant impact on my food budget.

4. Motivation

Probably the first lesson you learn when you train for a marathon is that commitment is a big part of it. The training will predate over your life. Forget about going out or drinking the night before a long run. The long run itself will make you wake up early on the weekends and will keep you busy at least half of that day alone. I was making 8 hours of actual running a week. You need to watch what you eat, not to get injured, what to wear, … You friends will tell you you are obsessed with running. Turns out its addictive! My life during the last month or so was pretty much running, sleeping and working.

You really need motivation specially on the first days. Running 40 minutes can get very long those early days. But as you train more and more, you start to enjoy and look forward that feeling of achievement and fitness at the end of the race. After a while, the running itself becomes the reward and, believe me, you will really enjoy running. You stop thinking about the time and the steps and you get lost in your thoughts, the music you hear, the nature around you.

Try to find various sources of motivations. What really worked for me those early days was the music I was playing via Pandora, and then later Spotify (btw this is my running playlist). And of course the mapping. I was mapping every race with RunKeeper, and knowing I was going to be able to monitor the track, the speed or the heart rate was a motivation on itself. I even made plans to make tracks of particular paths or shapes, like running the borders of the DC rhomboid. During the long runs I would post a live link so that friends or family could track it in real time. Knowing someone is following my run made me push harder. Getting those likes and comments in Facebook really helped to push through weak times.

Since November 2011 that I started using RunKeeper, I’ve logged 1,440 km, roughly 718Km for this marathon training. I’ve written before on how to make a map in TileMill with all your runs as a heatmap. I did it again, this time with the marathon itself in black, for contrast.

Together with the same process I used for the heatmap, this is the result. Pan and zoom at will [full screen here]:

The day of the Marathon

There should be nothing new the day of the race. Don´t try anything for the first time on that day, the slight deviation could risk the performance you have trained for 100 days or more. New shirt, new shoes, more coffee. Nope.

The day of the race I woke up at 4:30am to get up and have the usual pre long race breakfast: Granola, soy milk, banana, and a slice of bread with peanut butter. Get to the first metro around 5:15am and head to the Starting line, on the Pentagon. More than 30.000 people are expected, and there several steps to make, so you don´t want to miss the 8am start.

On the street a few Halloween walks of shame to cheer you up, and the metro is only has runners. Get off at the Pentagon station and head to the Parking lot. Go through the security lines (it´s the Pentagon after all), gently apply Body Glide or Vaseline on chaffing areas, visit one of the thousands of portajohns, drop your bag at the UPS trucks, get some water, … and wait. Everything was much faster than expected, and there you have an increasing number of runners, all ready, sitting down quiet on the parking lot, still before dawn. Most of them wearing plastic bags to avoid getting cold. Around 7:20am you head to you corral. People organize themselves in groups, corrals according to the expected finishing time. I chose one one the slow side of my expectation (4h30m), to help me push along if things go wrong, and to get that extra boost of overtaking people as you go stronger.

When the shot gun fires, it still takes a few minutes for your corral to start moving, and for the first 5 miles of so, you run on dense multipede of people snaking around the bends and turns of the course. Getting to an overpass or a downhill is quite a view of just people running, for miles. One of the common mistakes is to start too strong, so I tried to concentrate on the music and the feeling of any other long run. Slow but not walking on the uphills, easy but not letting go on the downhills. Some spots have a lot of people that cheer you up, and honestly carry you on with their support. Some spots have no spectators, so you just enjoy the beautiful autumn day, and look around fellow runners. Along with the music of Pandora, RunKeeper tells me the pace every 5 minutes. On my forearm I wrote my boundaries. Keep it between a 9:11 min/mile (for a 4 h marathon) and 9:55 for a (4h30m). I was always a bit faster than 9, so I was holding it most of the time. Every 45 minutes or so eat your Go. Keep running and remember you are running a marathon.

Until the half marathon mark, I was just running. Then I realized I was a bit slow, and had enough energy. But the stomach was acting up a bit. Too much Gu with caffeine? A technical stop for 60 seconds, and back to the race. The first part of the second half is where most of the people are watching and cheering, so I let myself be carried on with their cheers. Mile 20, the famous milestone for many runners, coincided with the bridge over the Potomac, the wind, no people to cheer you, seeing many fellow runners give up and walk, sit down or even lay down… I turned the music on just as Thievery Corporation started. 6 miles didn´t seem so short any more after 20. On that moment I though of my cousin who suffers a degenerative motor disorder and is on a wheel chair. I focused my thoughts on my luck to being able to get this far, and I pushed on faster. Nearing the mile 24 was when I realized it was almost over, and I saw the 4h pacer (a guy how runs from the 4h corral and keeps the pace, to help others) so I took my last Gu (probably unnecessary and not good for the stomach) and kept the pace, now more relaxed and even smiling. I knew I had energy to finish, so as soon as I thought I was less than 20 seconds on a full sprint, I gave it all and dashed to the finish line. Vamos!

The geek fun continues after the marathon. I got hold of all runners data and started playing around with it. This is the second part:

[Update: Added this SpeakerDeck with the slides I used for the GedoDC Meetup talk on Dec 2012]

How I got the data

Scrapping the data

The official website lists the results up to 100 per page, but it won´t allow to download it or get all the pages. Thankfully, just hacking the URL you can request “the first 50.000 results” for the first page. It takes a bit to load, it is indeed a huge table with all the data. This is public information, so I don’t see any problem taking the public data and mining it.

Converting the table into csv

To convert the HTML table into workable data, I tried several ways, and ended up just selecting all the text in the page via “Select All”, “Copy” and then paste it on an Excel Workbook. It takes a while, but it works. Just clean the extra lines and columns, and save it as a csv file. From the 34 Mb of the html or Excel, we go down to a 4 Mb csv file.

Cleaning and geocoding

I then used Google Refine to clean the data. In particular I did some Clustering to fix misspelled Location names, like “Alexxandria”. I also joined the “City” with “State” so that the geolocation would work better.

One important step for later is the Geocoding of the “City, State” column. As mentioned on this post, you can use Yahoo API to request the (latitude,longitude) for any location. To geocode your locations with Google Refine, you need to make a new column based on the Location column using the URL Fetch, and use the YAHOO geocode Yahoo API:

http://where.yahooapis.com/geocode?q=" + value.replace(/\s+/, " ") + "&appid=[appID]

The “replace” thing is to convert spaces into “%20” for the URL (like New York,NY into New%20York,NY). I minimized the milliseconds of throttling to 1 and still it took a good 15 minutes.

Then parse the extra column you just made to extract the (latitude,longitude) tuples from the XML:

forEach(value.split("<Result>"), v,v.partition("<latitude>")[2].partition("</latitude>")[0] +","+v.partition("<longitude>")[2].partition("</longitude>")[0])[1]

Then just export the data as csv for the next step, mining the data.

Mining the data with Python

Disclaimer: This is my very first Python code, so its is not very… pythonic

To import the data in Python I used:

import csv
runner=[]
f = open('MCM.csv', 'rt')
try:
    reader = csv.DictReader(f)
    for row in reader:
        runner.append(row)
finally:
    f.close()

With this I can get some insights on the Marathon:

There were 23517 finishers,
Runners came from 4274 different Locations.
42,5% where women, 57,5% men.
The most common runner was a male, 38 years old, from Washington DC, finishing in ~4h24m
The most common woman was ~20m slower than the most common man.

But let’s break it down more. You can see my whole code to mine the data and make the plots in python here.

Histogram of participation by gender

This histogram shows the distribution of Ages, according to gender (mouse over for tooltips):

Remember this is not a random sample of population. These are all marathon finishers (A small subset of the population… 1%??). At any age, there is more men than women, except from 25-30. Perhaps women are more physically active then. After that women decline while male participation increases. I wonder if having your first kid is reflected here. I can imagine it’s hard to train or run when you are pregnant or new mum. According to the statistics the mean age for first kid in the USA is 24, so it is plausible that effect shows here.

Men participation peaks at 40-45, after which the decline is similar for both genders, with always more men. Remarkable than 10% of the participants were above 53, and another 10% below 26. As I will show later, above 5 hours of finishing time there is not much difference in gender. Tell me another sport where this variety of ages can challenge themselves side to side. Running is a natural thing for humans.

Histogram of time by gender

Now, lets compare finishing times by gender, regardless of age. Men peak around ~4h30m, with another peak just below 4h (me!). Women peak more smoothly around ~4h50m.

Interestingly, the women distribution is close to a Gaussian (and you see similar values around the peak on both sides). Now, male runners faster than 5h seem to be able to squeeze better times than females of same age. The difference is highest just below 4h. I know this time is kind of a mental goal for many runners (me included). However, after 5 hours, both genders decay in the same manner, with women always slightly better then men.

What about time AND age, by gender?

Combining both histograms one can plot a Scatter diagram where the horizontal is Age and vertical is Finishing time. Different color for males and females.

Both genders show a similar distribution, quite broad both in Age and Time. That means neither variable is significative to the others. Youngsters can be slow, and seniors fast, regardless of gender. There is a slight convergence towards more time as age increases. There seems to be a slight slope of less than 1 hour every 20 years for the best runners. I think the race course closed after 7 hours, hence the flat upper boundary.

However, there is blue edge on the fast side, all across ages. Faster men tend to be faster than women of the same age, specially between 30 and 60 years old.

By the way, I am flabbergasted by the right side of the scatter, runners above 60 running on the same time band than people 40 years younger. Respect!

Pacing yourself

One can also plot the Excess time. This is the difference between twice the half-marathon time and the total time. If you keep a constant pace, this Excess time will be 0. If you hit the wall on the second half and need 1 hour more to finish, then your Excess time will be 1. I am plotting this against Age, to see how it depends on the age, and also with different colors for men and women:

As the figure shows, very few people are actually faster on the second half (means to the left of the vertical line above 0). Above 60 virtually everyone is slower on the second half.

The graph is pretty much a packed vertical shape, but with some convergence on the top. Older runners seem to be able to keep their pace better, specially women.

Again you can see a blue edge across all ages, mostly men that need more than one hour on the second half. I would say they “hit the wall” hard. In that sense, it seems men hit the wall more than women, specially young runners below 60. Combined with the previous graph, faster men are faster than faster women of the same age, but also more men end up hitting the wall.

Mapping the clusters of runners

For the next visualization, I wanted to map all runners on the US map. You can indeed upload the csv file with just the names of their cities to Google Fusion Tables (the file is too big for Google Docs), and Google will recognize the Location names and map then for you. Pretty awesome, quick, and easy.

There are several problems here. For once, I want to have some control to make the map more beautiful and customize the looks (and I dig open source). More importantly, any place with more than one runner will simply overlap everyone on the same place. We need to aggregate the data.

To aggregate the data I use again Python, and this little code:

#Get a list of cities
#Count uniques
from collections import Counter
Places=[r['Location'] for r in runner if 'Location' in r]
Places_counts = Counter(Places)
len(Places_counts)
Places_counts.most_common(10)
#Make a dictionary with Places, their location, and number of runners
Places_dic={}

from collections import Counter
#runner is the list with every runner a dictionary
Places=[r['Location'] for r in runner if 'Location' in r]
Places_counts = Counter(Places)
for r in runner:
 for P in Places_counts:

 if r['Location']==P:  #runner is from that place
    if P in Places_dic:
      #not the first time, add info to this location
      Places_dic[P]['runners']+=1
      Places_dic[P]['Time'].append(int(r['ChipTimeSeconds']))
      Places_dic[P]['Ages'].append(int(r['Age']))
      Places_dic[P]['Gender'].append(r['Sex'])
      if Places_dic[P]['lat']=='':
        Places_dic[P]['lat']=r['lat']
        print 'gotcha'
      if Places_dic[P]['lng']=='':
        Places_dic[P]['lng']=r['lat']
        print 'gotcha'
    else:
      #first time, add dictionary
      Places_dic[P]={}
      Places_dic[P]['runners']=1
      Places_dic[P]['Time']=[int(r['ChipTimeSeconds'])]
      Places_dic[P]['Ages']=[int(r['Age'])]
      Places_dic[P]['Gender']=[r['Sex']]
      Places_dic[P]['lat']=r['lat']
      Places_dic[P]['lng']=r['lng']
      print 'new place:',P

With a little bit of some more python magic, you can aggregate the times and ages to get the percentiles by gender, and save everything to a neat csv file with all the info you need. You can see the whole python code here.

Now you can go to TileMill, import the file on a new project and have the map styled, up and running in less than 10 minutes.

and this is the result, pan and zoom at will (Full Screen here)

Overtakers or overtaken?

When I was talking with another marathoner about the draft of this post he said it would be interesting to see a scatter plot of how many people you pass by gender, or age. This involves not only your pace, but also which corral you chose (your expectation). We only have data for Bib numbers at Gun Time, ChipTime, and partial ChipTime splits for 5K,10K,15K..40K. Can we get the data we want?

Ordering Bib numbers by “Gun Time” at any landmark (5K,10K, ..) gives you the order of arrival to that line mark, regardless of their position at the Start time. It´s just the time it took them to get to the landmark from the Gun Time.
Ordering Bib numbers by “Gun Time” minus Chip Time, gives you an estimate of the distance from the runner at the Gun Time and the Start Line. That is a rough estimate of the order at the start time.
Hence the difference of 1 and 2 for each land mark gives you the net change. Using the landmark you can even check for which Bib numbers changed in between.

Helpers in python to get there:

You can create a new key with to convert time in 1:34:10 into second with:

   def count_sec(lista):
     return np.sum(np.array([np.int(a) for a in lista.split(':')])*[60*60,60,1])

Get the rough order estimates with:

    for r in runner:
     r['Start_order_in seconds']=count_sec(r['ChipTime'])-count_sec(r['ClockTime'])
    Start=sorted(runners, key=lambda k: k['Start_order_seconds'])

Get the change in order for each Bib with:

    for r in runner:
    index_s=map(itemgetter('Bib'),Start).index(r['Bib'])
     index_e=map(itemgetter('Bib'),End).index(r['Bib'])
     r['Change']=index_s-index_e`

… but I leave the train here. There are surely many more things one can do. It has been an amazing experience, and many more will come. I enjoyed and learnt a great deal with the training. Running the marathon was a challenge but not a suffering and I really enjoyed doing it. Playing around with this data has also been very fun!

Now I don´t know If I want to try a trail marathon, an ultra, or a triathlon :)

Ideas? Comments? Corrections? Forks?

Thanks!!

Migration in progress

2012-09-14T00:00:00-04:00

I´m in the progress of moving my blog into Jekyll. I´m halfway done, so I´ve decided to publish the new version.

Sorry if things are broken. I´ve tried to respect the URL to the posts, so most inbound links should still work.

My old site still lives at Wordpress.

Feel free to send me your comments to brunosan@gmail.com. Thanks!