This entry documents the fourth of my projects for this summer's ITP 4-in-4. 4-in-4s are occasional events entailing the execution of one complete project each day for four consecutive days. They're a great opportunity to finally do a daydream project you've been talking about for ages or to play with a new technology or technique you'd be meaning to try out.
Foosball is the national pastime of ITP. An old, beaten-up, much beloved table is a fixture on the floor and during semesters a game is almost always in progress, from busy daytime hours when the clunk of the ball adds to the overall noise of the floor to late night procrastination sessions when students round up whoever's around to play intense games meant to block out any thought of a looming deadline or mountainous pile of work.
I'd played some foos before coming to ITP — most notably at UC Riverside with my cousins after a series of summer softball games — but, in my first year at ITP, I've become predictably obsessed: trying my best to learn from the masterful 2nd year players who, at the start of the year at least, routinely routed me.
In the course of this foosball education, I began to come up with certain theories. For example, I noticed that players were divided into those who moved both hands between the handles (advancing to the 5-man and the 3-man when on offense, for example) and those who kept their left hand anchored at the goalie, only advancing with their right. I instinctually favored the latter approach, feeling that scoring opportunities were created for the other player whenever hands moved: a left hand-anchored approach minimized movement and hand-off-handle scoring opportunities. Similarly, I started to see that a lot of the best players shared a certain style of passing and shooting from the outside players of the 5- and 3-man rows that was especially effective. Also, a few players (most notably Jeremiah Johnson) used their lightning quick reflexes to play an aggressive kind of defense where, instead of simply blocking an opponent's attempt to move the ball forward, they'd slap it back at full speed, as if hitting it from their own possession (an especially deadly move against the opponent's defense where it frequently results in a goal).
I started to wish for a formal vocabulary to describe these trends and ideas I was seeing and some structured way to test my hypotheses, a procedure that would allow me to transcribe and keep statistics on games in order to better understand them and discuss them. I realized that I needed a notation system like exists for chess or baseball, something analogous to the "1-6-3 double play" or " Bxd7+" (bishop to D7, check). And, on Scott Wayne Indiana's suggestion, I sought a system that could be represented visually (like a baseball scorecard) to enhance transcription.
So, for my final 4-in-4 project, I set out to design such a system.
Starting from Scott's suggestion about the baseball scorecard, I started by designing a simple graphical representation of the foosball table. I then labeled each handle with a "B" or "W" (for Black or White) representing each side (at ITP, at least, the actual players can be nearly any color, having been repaired, replaced, and becostumed so frequently) and a number corresponding to how many players were attached to that row. Hence: B5 would be black's 5-man row and W1, white's goalie. Then, for the multi-man rows, I added letters for each man so that the notation system could also capture which specific man hit the ball. For example: "W3a" would mean that white hit the ball with the man furthest from them on their 3-man row.
I also included the handles in the diagram because I knew that the position of the players' hands at the time of each contact with the ball was important data for exploring my theories about handle strategy. It also, when you have a series of events in sequence, turns out to give a really nice feeling of the momentum or drama of a moment of play. I didn't try to include the hand-on-handle positions in the written notation system as I wanted to make it easy and intuitively understandable. I'll present some real examples in a minute, but if I said: "W3a to W3c for the goal", you'd easily be able to picture a lateral pass between the two outside players on white's 3-man row followed by a scoring shot. All the extra cascade of notation that would be required to capture hand position would only water down that flavorful little bit of drama.
Now: an example of the notation in action! I printed out a handful of sheets of paper filled with a grid of scorecards and recruited Mike Cohen and Jeff Howard to play.
I told them that, after each goal, I'd interrupt them to transcribe the events that led from from the last coherent possession to the goal. In theory, it would be great to transcribe an entire game and to do statistical analysis on the resulting data, but that would require adding sensors or a camera tracking rig to the foosball table (not that it hasn't had both of those things applied to it before, the thing bristles with the leavings of old class projects) and spending inordinate amounts of time getting the technology working rather than focusing on the notation which was the point of this project. I would love it if someone constructed a system for machine-generating this notation automatically at some point, but it was beyond the scope of this one-day project. With that in mind, I figured incidents of scoring were the most interesting thing to transcribe and also imminently doable.
I designated Mike "Black" and Jeff "White" and had them kick things off. Before too long, Jeff scored the first goal. I had the players pause and reconstructed it thusly:
See the larger size on Flickr.
This diagram might appear dense at first glance; let's break it down. First, look at the notations on the handles. White has his hands on W3 (the 3-man) and W5 (the 5-man), a position I soon dubbed "classic offense". Black is on B1 (the goalie) and B2 (the 2-man): "classic defense". Their hands stay in those positions for all four events of this play. Now, look at the red 'x' that moves between the four diagrams: at first between two of white's players and then into the goal. That's a fancy bit of passing on Jeff's part before the shot that scored; he knocked the ball laterally from the middle 3-man (W3b) to the one closest to him (W3c) and then back to take the shot. So, in the final notation, this play reads as: "W3b W3c W3b W!" (where "W!" reads as "White goal").
You should be able to start to picture it in your head now: Jeff starts with control of the ball at the middle of his 3-man. Mike is back in defensive position. Jeff passes the ball to the side; Mike adjusts his defensemen to follow. Then Jeff passes it back to the middle, where he slaps it directly into the goal before Mike can recover. A classic bit of foosball strategy: using lateral passing between the 3-man to shake the defenders and create an opening for a goal.
The game continued looking much like a whipping administered by Jeff who got up to a 6-0 lead before Mike scored his first goal. I'll revisit some of those earlier goals in a minute, but first I wanted to show another, even simpler, goal: #5 of the match:
An incredibly straightforward goal: W3b W! with both players in "classic defense" and "classic offense" throughout. Jeff controlled the ball with the middle player of his 3-man handle and then shot it past Mike's goalie and 2-man and in.
The goal immediately before that was nearly as simple. At this point, you should be able to imagine the whole story just from the notation: W5c B1 W!:
Now, let's talk about some goals that relate to the theories that set me off on this project in the first place, for example #6: W5c W!:
At first glance, it looks nearly as simple as #5, just coming from W5 rather than W3. But look at the position of the handles in this exchange. Black starts off with his hands on B2 and B5. His goalie is untouched. White has control of the ball at W5. Black was caught in transition, right in the middle of trying to move his hands up towards "classic offense" when White got control of the ball at W5. The second position shows even more evidence of the chaos caused for Black by this reversed transition: he actually had only one hand on a handle at the time of W!: B5. His left hand was probably moving back towards B1, retreating in the face of White's sudden possession of the ball, but not yet having arrived to take charge of the goalie.
And look at White's hand position by contrast: he's on W5 and W1. Probably starting from classic defense a few seconds earlier, he advanced only his right hand, keeping his left anchored on the goalie, making for maximum stability and speed during the transition play, which let him pound the goal in from midfield in the midst of Black's chaotic transition.
Obviously, from the way I'm telling the story of this goal, I think it counts as evidence for my transition theory in favor of playing with a hand anchored at the goalie. And, in fact, the next goal in the match, Black's first, offers more evidence in support:
This time, Black is anchored with his left hand on the goalie when he shoots, only having advanced his right hand to B5b to take possession. White is on W5 and W2, one of the weakest positions on the field: back on defense but with no anchor on the goalie and no hand on it to prevent deflections or allow for quick reactions. And, what's worse, as we can see from the hand positions in the second frame of this goal, White is also in transition: both of his hands move between B5b and B!, meaning, while the ball was in travel, he was, in fact, touching no handles. Black's hands have both also moved, but this has no bearing on the result of the play as he does not touch the ball again. In fact, if White had been able to prevent the goal, Black might have been left quite vulnerable to the kind of quick reversal we saw in goal #6 due to being in transition himself.
This is a strong goal for Black and there's a reason it dramatically changed the momentum of the game. Up to this point, many of White's goals were scored in the position I've described as "classic offensive", i.e. with hands on W3 and W5. While this is an especially strong position for players (like Jeff) with lots of good fine shooting and passing skills (checkout White's goal at #11 for example) it is also especially vulnerable to transition scoring if the opponent can kick the ball up field: from classic offense, you inherently have to move both hands in order to retreat at all, creating lots of transition time for an anchored opponent to pass or shoot. After this first occasion, Mike seemed to get into a rhythm in exploiting this weakness in Jeff's style: advancing the ball aggressively out of a seemingly defensive position and then proceeding to score while Jeff was forced to transition both hands back (Goal #8 is another great example of this). Though Jeff would go on to win 10-5, Mike kept up goal-for-goal from this point, losing so badly only because of the nearly insurmountable lead Jeff had built up with those first series of goals.
In fact, the second half of the game is replete with bank shots from deep within Black territory (B2), shots from B2 while White is still in classic offense, and combinations of both of those effects.
You can see all of the goals for this game notated in this flickr set.
So, hopefully at this point I've demonstrated my notation system's virtues for recording and re-telling the story of a game as well as for making strategic and tactical arguments such as the one I've been rehearsing here about the importance of transition play and hand position.
Given all of that, I did discover a few downsides to the system in using it for the first time. For example, my graphic doesn't include the foosholes so, if a goal was scored directly on foos, or if a machine was to attempt to transcribe a full game, that's a lack that would need to be addressed.
Also, as you can see from this picture:
The scoresheets that I printed out did not include the lettering and numbering key I've included in the graphics presented here. I was trying to keep things simple and minimal. The main problem with that decision was that there were no visual cues for reading the orientation of the sheet: a couple of times I got started notating only to realized that I had black and white reversed due to the symmetry of the sheet. So, when I print out further sheets, I think I'll use the notated version for clarity's sake.
Another problem with the system is that it can occasionally be difficult to reconstruct complex interactions. Take, for example, this goal: W3c wall B1 W!
Clearly, the ball bounced off the wall after White's shot in a manner that surprised and evaded Black's goalie. But, you don't quite get as vivid of a picture of the logic at play as you do with better structured goals. It's my suspicion (or at least my hope) that this kind of ambiguity would be greatest with goals resulting from "slop" — the disorganized caroming of the ball around the table and off random players without the meaningful intervention of human intention — which, I think, you could argue is actually a virtue of a notation system, which must inherently reduce, simplify, and structure the thing it describes.
There was also a sense amongst the players that my notation didn't capture certain subtleties of particular goals, the angle of a shot that slipped in behind the goalie, for instance. This is definitely true, but again, I don't have a clear sense of how to capture that without making the notation system unwieldy. My intention is not to eliminate the need for human commentary, only to augment it and enhance its concision and clarity.
Topics for future research. Speaking of printing out further sheets, I plan to print out a handful of them and attach them to the ITP table so they'll be to hand for anyone who wants to sketch down a particularly interesting play or series of goals they see happening. I also plan to transcribe some additional games and to try to make a bit of a catalogue of signature plays and tactics that I've come across. And, of course, there's always the grand dream of machine transcription, which I intend to push on any unsuspecting first year looking for a topic for a camera tracking project or pcomp midterm.
This entry documents the second of my projects for this summer's ITP 4-in-4. 4-in-4s are occasional events entailing the execution of one complete project each day for four consecutive days. They're a great opportunity to finally do a daydream project you've been talking about for ages or to play with a new technology or technique you'd be meaning to try out.
For my second project, I wanted to carve something out of blue foam. I think blue foam carving is going to be a major ingredient in the work that I do at ITP this year and, while I know we'll be working with it in Materials and Building Strategies this coming semester, I wanted to get ahead start in familiarizing myself with it.
For a subject matter, I chose an Atari 2600 cartridge. To understand why checkout the 3D model of an Atari cartridge I built for last winter's 4-in-4. That post explains the backstory behind my interest in the cartridges and the eventual thing I want to build with them. In the meantime, suffice it to say that making to-scale models of Atari cartridges has become something of a 'hello world' for me in new fab techniques.
I started out by putting out a call on twitter for the correct dimensions of an Atari cartridge. Unsurprisingly, Don Miller wrote back almost immediately with the answer. He had one to hand and simply measured.
Once I had the dimensions, I measure them out and drew them onto my piece of blue foam with sharpie.
I then used a small handsaw to cut the basic rectangle loose from the large sheet of foam. The resulting rectangle was the right basic size as the cartridge but had an extremely rough surface texture and was lacking the rounded corners and the other few design details of a real cartridge.
With a metal file, I set out to smooth the rectangle's surface and to add the rounded corners on its side.
You can see from that picture that the far side of the cartridge got a little off square in the process. I also had a relatively hard time getting a nice smooth surface on the foam. As I'd file thing would sometimes get smooth, but when I applied to much pressure, the foam would seem to come apart a little in all directions, leaving a rough, ugly surface.
I was, however, able to shape a nice bevel that really reminded me of the real cartridges.
I then set out to carve out the slight cutaway on the front of the cartridge in which (in a real cartridge) a sticker with the game art sits.
Here you can see the texture of the carved away area before sanding.
At first I couldn't figure out how to sand down into a cutout like that since the files I'd been using for the rest of the process wouldn't reach. Hunting around, I found what became the perfect tool for blue foam work: a sandpaper-covered wedge of squishy foam:
Using this tool, I was able to get down into the sticker cutaway and sand it relatively smooth.
At this point I added the cutaways on the top and bottom of the cartridge (at the top for another sticker, at the bottom to expose the actual circuit board that interfaced with the console itself) and struggled to sand them.
Finally, I used the hand saw to carve a seam that runs all the way around the cartridge, which, in the original, is produced by it being constructed of two separate pieces of plastic that join imperfectly.
This first attempt at a cartridge was ok, but had some major flaws. The shape was slightly irregular and the surfaces were rough, the cutouts especially so.
I did some googling and read some online advice about working with blue foam (something I'd resisted doing before starting, wanting to get a clean experience of the material's behavior). I immediately discovered that I was on the right track with the sandpaper foam: lots of sanding was recommended for getting smooth surfaces. Further, sanding slowly, with low pressure, and in only one direction was recommended for preventing the foam's surface from breaking up like I experienced. And, finally, pre-starting cuts with a sharp knife was recommended as a way of keeping them from getting ragged.
So, keeping all of this advice in mind, I set out to make a second cartridge. After a few false starts getting the basic proportions correct when cutting the raw rectangle out of the block of foam, my second cartridge came out much better than my first.
Smoother bevel and finish all around:
Tighter cut for neater corners in the inlays (both cut with an exacto):
Comparing the two side-by-side you can really see the improvement in control and surface on the second piece:
Obviously I still have a long way to go in improving my blue foam carving, but today was a satisfying start.
Two thoughts to end with. First, out of all the tools I had available...
...the simple hand saw was best for cutting basic shapes (though if I'd gone so far as to use power tools for that, they might have proved far superior — and a lot of online resources recommend hot knives or wire for that stage in the process), the exacto for starting detailing, and the sanding block for finishing and shaping.
And, this may be obvious, but: blue foam gets everywhere! It's messy and clingy and, apparently, even not the greatest thing to breathe. Regardless, I plan to become much more intimately familiar with it in the coming months. So, my future is likely to look a lot like this:
This entry documents the first of my projects for this summer's ITP 4-in-4. 4-in-4s are occasional events entailing the execution of one complete project each day for four consecutive days. They're a great opportunity to finally do a daydream project you've been talking about for ages or to play with a new technology or technique you'd be meaning to try out.
Unsurprisingly, the ITP community has a lot of places it posts information online. There's the homeship, itp.nyu.edu, with all its offshoots, such as the pcomp site and the people directory. There's hundreds of student blogs, dozens of class sites, even our own wiki: ITPedia.
This is totally to be expected in a contemporary, technically literate, information-generating organization. The problem is: how to navigate it all? When you're looking for a specific piece of information — that tutorial about motors your classmate mentioned or a list of which alumni have previously worked with animatronics — it can become quite laborious to browse one-by-one through each of the many ITP-related sites. And a general Google search will tend to bury the ITP needle in a large stack of non-ITP hay.
Considering this, it occurred to me that what we needed was an ITP-specific search engine. Such a tool would let students explore and hunt across all the program's varied online resources from one familiar interface.
The problem is that, having done it before, I know from personal experience that building a search engine is no quick hack. However, I realized that, using the AJAX API for Google site search, I could easily put together a prototype of an ITP-specific search engine that would do at least part of the job — enough, hopefully, to discover if such a thing would be useful enough to consider building for real.
Hence, I would like to present: ITPoogle.com, the ITP-specific search engine. Enter a query and ITPoogle presents results in three columns: one from ITPedia, one from itp.nyu.edu (which includes all student blogs and class websites hosted on that domain), and a third from a growing list of class syllabi and other resources past and present on other domains.
I'll spend the rest of this post briefly discussing how I built ITPoogle since I think it's an interesting case study of using AJAX to explore interface and tool design quickly without having to setup any server components whatsoever.
I'd built a similar site in this style once before: rtumblr, another Google search mashup, this one designed to help Tumblr users find people who were reblogging them, a service Tumblr itself inexplicably fails to provide. I wrote up a full post about rtumblr at the time and if you read it and follow the links, you'll see that I'd built myself a few interesting tools for creating dynamic single-page sites in javascript: a library for handling templating, one for routing URLs, etc.
In the year and a half since I built rtumblr, I met the great Aaron Quint, and discovered that he'd also been thinking about dynamic, single-page javascript-driven sites. And AQ had taken the idea much further than I had, developing the full-featured and quite cool Sammy.js, a framework for solving exactly the common problems of these types of sites: routing, events, templating.
So, I decided I'd use Sammy.js as the basis for ITPoogle. I started the project by writing a simple HTML page with a search form, installing jQuery and Sammy.js on it, and getting Sammy configured to handle search URLs. The idea was to be able to provide permalinks for searches so that people can share results pages (like this: http://itpoogle.com/#/search/4-in-4) even though the site is pure javascript. Sammy's routing API makes stuff like that super easy; the basic outline of the code looks like this:
var app = $.sammy(function() {
this.get('#/search/:q', function() {
// do stuff here with params['q']
};
}
$(function() {
app.run();
});
And then I simply used jQuery to take over the submit event on my search form to change the URL, re-triggering Sammy's routing ("#q" here is the id of the input where the user enters their search query):
$("#search").submit(function(){
window.location.hash = "#/search/" + encodeURI($("#q").val());
return false;
});
Once I had the basic structure of the page in place, the next step was to start running the Google searches. I'd already written the start of a wrapper around the Google AJAX Search API when creating rtumblr, so I decided to extract that out into a library I called Giggle. Giggle is a relatively straightforward aid to using the Google Search API. It handles things like composing queries, iterating through pages of responses, and packaging the results into something convenient.
Initially, I built Giggle naively as a singleton object, but I quickly realized that doing so meant the danger of having results get intermingled when conducting multiple different searches simultaneously. So, I spent some time getting the scope right within Giggle so that I could create multiple instances of it, each of which could fire off their own searches without polluting the results coming from the others.
The advanced features of Javascript scope like this always trip me up when I come back to it after a break but it's not too complicated once you remember how it works.
In the end the Giggle API ends up looking like this:
var search = new Giggle();
search.q("ITP", {title: "ITP"}, function(results, locals){
console.log(locals.title);
console.log(results);
})
This weekend I served as a judge for Come Out and Play, an experimental urban games festival held in a different location around New York every year. During the course of the festival, I had an experience that I think might be worth describing for developers thinking about the native apps vs. web apps debate. It was an experience that confirmed everything that's lead me to want to learn to build iPhone apps rather than sticking with web apps for mobile development.
One of the games, called Pathfindr, was billed as "The Amazing Race" meets "Foursquare." They said in advance that it required an iPhone or Android phone to play. They'd built a web app that used javascript APIs to access the phone's location, then they used that to check if you were close enough to the series of checkpoints and questions that made up the game terrain. Your typical treasure hunt.
The experience of trying to play this game pointed out everything that is inferior about the web app experience vs. the native experience. And not as minor technical annoyances, but in a way that completely ruined the experience of the game.
(Note: none of what follows is meant as a criticism of Pathfindr's developers. I think they are diligent individuals who's only mistake was naively believing all of the hype around web apps as a solution for mobile development. In fact the game works so well as a case study because they seem to have done everything exactly how the current conventional wisdom would have them do it.)
The app consisted of a series of menus, styled to resemble native iPhone UI, through which you signed up for the site with an email address and password and chose which game iteration you wanted to join or resume (they'd run Pathfindr once before in Minneapolis as a tour of notable locations in the life of F. Scott Fitzgerald). Once you reached a game, the play took place on a satellite map where a set of markers appeared noting three checkpoints that you had to visit in turn and a series of questions you had to answer along the way about things in the local environment. In order to eliminate a checkpoint or answer one of the questions, you had to get within a certain distance of the marker on the map and then tap that marker. The map also displayed the blue iPhone-style "you are here" indicator. The game took us an hour and a half to complete and stretched from the Brooklyn Lyceum, where the festival was based, all the way to Grand Army Plaza at the entrance of Prospect Park.
Here is a rundown of the technical problems we encountered in the course of the game:
The web app was slow. Pages took a long time to load from the server with each new question or with new map tiles when you tried to move around the map. This was not caused by the rate of the iPhone 3G connection (other pages, such as Google, loaded perfectly rapidly), but because of the 20 or 30 people simultaneously accessing the site over-and-over. The problem was worst when all the teams were standing together and trying to register at the start of the game and was slightly mitigated over the course of 90 minutes as we spread out around the course.
The app was unresponsive. Dragging and zooming around the map was sludgy and painful. Redraws took forever.
The GPS location was jumpy and inaccurate. Far worse than I've ever seen it in the iPhone Maps application or any native iPhone app that uses location.
The app sucked battery life. In an hour and a half of playing it killed one phone completely dead from about a 30% charge and was close to killing a second one.
The app's map UI was non-standard and confusing. It used satellite-only tiles which made navigation difficult; accidentally tapping the blue "you are here" dot would zoom the map all the way out, which was extremely annoying; it had extra chrome within Mobile Safari's chrome leaving only tiniest number of pixels available for the actual map.
At one point, someone in our team accidentally closed the app's tab while trying to do a google search as part of the game. We had to suffer through lost state within the game that required four or five page loads to get back to where we were and hence about 5 minutes of sitting still with 5 people all hovering around a phone.
Overall, the sludginess of the web app experience meant that, in a game designed to take us out into the neighborhood, make us more attentive to the landmarks around us, and force us to collaborate with a group of strangers, we spent a large amount of the time staring at the phone's tiny screen, waiting for pages to load. The technology's flaws overcame the game's virtues and severely damaged the experience.
I'm sure mobile web app performance experts could explain a number of ways in which Pathfindr's developers could have improved their app to avoid some of the problems I mention here. However, one of the main talking points for mobile web apps is always that developers "already know how to make them". And talking to the one of the developers after the game, he touted the fact that they'd been able to build the first version of the game platform in only a few days' development time. If there are a bunch of new skills that an average web developer will have to learn to build an effective mobile web app, then that goes a long way to mitigating the advantages of their existing experience and it makes the argument for web apps seem a lot more like an argument for familiarity rather than ease of use.
Again, the Pathfindr team was professional and polished. Their app had obviously been the subject of a great deal of loving care. Give them the benefit of the doubt. The problems with the app didn't derive from their incompetence, but from current limits in mobile browser technology in this kind of situation.
Also, as someone who's just started learning iPhone development, there was nothing in the game that was beyond the level of the first four or five chapters of a good intro iPhone development book, for example Big Nerd Ranch's iPhone Programming Guide, which I'm currently in the process of working through.
In less than a weeks' time, a dedicated developer just getting started on the iPhone could have built a native app with much the same functionality as Pathfindr's web app that would not have suffered from any of the problems I describe above. And for an experienced iPhone dev, the process would have been more like the one or two days Pathfindr themselves took.
I can't speak for Android, but at least in the case of iPhone development, as a newbie iPhone dev and a highly experienced web dev, I think the idea that native development is so much more difficult than web development is largely FUD. While native development means learning Java or Objective-C and getting up to speed with someone else's large API, there's no server to maintain on an ongoing basis, there's only one language rather than the three or four that are typical for a web application (HTML, CSS, Javascript, and at least one server-side language), and there are no scaling issues to consider with the acquisition of multiple users. The "ease of use" factor with web apps exists solely for a developer community that already knows how to make web apps. For new developers or anyone willing to learn something new, I think the level of difficult argument is actually a lot more evenly balanced than it might appear from the outside.
What about device independence? In the case of Pathfindr's audience at Come Out and Play, at least, nearly every single participant used an iPhone. There may have been one or two Android players, but I didn't see them. They'd advertised the game as "smart phone-only" and said they'd tested it on Android, iPhone, and Palm Pre Plus. A few people with Blackberries came thinking they might qualify, but had to be turned away or added onto teams of players with iPhones. With the incredible battery usage my team saw, I can't imagine that an Android team would even have been able to finish the course (especially with what I've heard about the HTC 4G).
The result on the ground, then, was an application that was the worst of both worlds: device-limited without the benefits of device-specific design. It had all the performance disadvantages of a web app that I've outlined above with none of the imagined benefits of cross-device compatibility.
Advocates of web applications as the solution for mobile development need to do some observations of real world scenarios like this before they convince more developers that native apps are unnecessary or evil in some way. The new creative and surprising uses to which a big games festival like Come Out and Play puts mobile apps is a terrific proving ground for development platforms.
For my part, I want mobile apps to be weirder and more fun than their desktop ancestors and to explore all the new affordances offered by the position of phones and new devices in urban space and in our lives. And I'll take whatever development platform keeps the battery running, the GPS coordinates flowing, and the sensors sensing. Right now that means native apps.
A couple of months back, the guys at GitHub bought a stoplight. They ordered it on Ebay from a seller in the UK. Their plan was to hook it up to their build system.
The idea was that when they check code into their continuous integration system (see defunkt/CI Joe) the stoplight would turn yellow while the code was building, red if the build failed because of failing tests, and green if the build succeeded. It would give them a large unmissable indicator in their office for the status of their current code.
The stoplight at the start of the project:
One of the GitHub founders, PJ Hyett played around with using an Arduino to integrate with the continuous integration server. He wrote a ruby script that polled the server and then sent a serial message to an Arduino which turned on an appropriate green, yellow, or red LED.
This worked, but had a couple of downsides: first it required a laptop to be connected to the Arduino to talk to the CI server; second it only lit up little LEDs rather than the big beautiful stoplight. For good reason, as a beginner PJ balked at the idea of controlling the AC power needed by the stoplight. After all, AC can kill you.
That's where I came in.
Melissa Severini, GitHub's admin, got in touch with me to see if I was interested in taking on the project of hooking up the stoplight. I told her I definitely was and got to work putting together a proposal.
My goal was to improve PJ's take on the project by handling all of the networking on the Arduino itself and, of course, to control the actual stoplight rather than just LEDs.
On the AC side, having read Sparkfun's tutorial on controlling AC with relays, my plan was to connect one relay to each of the stoplight's lights and then to run all of those to a single Arduino which would itself connect to GitHub's CI Joe instance over their office wifi.
Initial proposal sketch:
As you can see from that photoshop mockup, I was planning on using the Asynclabs BlackWidow for the networking part of the project. The BlackWidow is a neat little Arduino variant which has wifi hardware built into it.
In the final planning stages, this was the component of the project that changed the most: I ended up ditching the wifi in favor of using Ladyada's ethernet shield with a Wiznet module to connect the Arduino directly to GitHub's router via an ethernet cable. There were two main reasons for this decision: first of all, I realized that trying to run a wifi antenna inside the solid steel box of the stoplight's housing was probably not the smartest idea, and second I found asynclabs' wifi code for the Arduino to be simply too complicated for me to wrap my head around in the few days I had to prepare before traveling out to SF. I was able to get the BlackWidow to connect to a wifi network, but never to actually reach out beyond it to real URLs on the internet.
So, my proposal submitted and accepted by GitHub, I ordered parts and scheduled flights.
The second week of May, my parts began to arrive and I soldered up the Sparkfun relay boards and tested them with my Arduino. I also talked with Tom Igoe, ITP professor and member of the core Arduino team, who gave me a bunch of great advice about using the relays safely and also explained how to power the Arduino itself off of the AC power as well as the stoplight (a subject I'll return to later).
Last friday, I flew out to SF with a bag full of electronics: 7 Arduinos, 3 USB cables, a soldering iron, a pair of wire cutters, 2 packages of rubber feet, three spools of threaded hookup wire, 3 relays, etc. etc. I was surprised my bag made it through security with nothing more than a TSA inspection notice slip. And on Saturday, the build process began.
The first thing I did on arrival was to open up the stoplight and take a look inside:
As you'd expect for a piece of urban infrastructure that needs to work reliably for a long period of time, the stoplight's design is extremely simple. The AC runs in from its external plug (the white cable in the picture above) to a bank of screw terminals through which it's distributed to all three of the lights:
From there a pair of wires travels out to each light fixture, one hooked up to each side of the AC:
Thanks to Andy Delcambre, I'd seen photos of this setup in advance and taken it into account when coming up with my plan. Andy was extremely helpful in the planning of this project and throughout its execution; I couldn't have done it without him.
After having checked out the wiring for myself, I spent the rest of my first day in the GitHub office installing the relays.
A relay is a physical switch that can be closed by running a small amount of electricity through a pair of electromagnets. The result is a device that lets you control large amounts of electricity (like 120V AC) via small amounts (like the 5V DC that comes off an Arduino's pins).
As I mentioned before, I'd soldered up three of the Sparkfun relay breakout boards before coming out to SF. These have three wires that get connected to the Arduino: ground, +5V, and a control line that goes to one of the digital pins. On the other side, it has two terminals for the AC connection. The idea is that you break one of the two AC wires going to your target device (in this case one of the stoplight lamps) and solder the relay into the break. Then when you pull the Arduino control pin high, the AC part of the circuit is closed and your devices turns on; when the control pin is low, the device is off.
Relay with control board soldered up to one of the stoplight lamps:
Once I had the first relay soldered up and plugged in, I wrote a basic blinking LED-style Arduino sketch that would simply toggle the relay control pin on and off every 500ms. After being sure that no one was touching any part of the stoplight, I plugged it in and the result was a blinking lamp:
Note the satisfying clicking sound. (Brief aside: next time you're standing at a street corner waiting for the "walk" signal, listen closely to the big box attached to one of the utility polls nearby and you'll hear a giant relay clicking away blinking the "don't walk" hand.)
Having proven that the relays could control the stoplights, I went ahead and attached relays to the other two lamps as well following the same procedure, testing each one in turn.
Now, I had control of all three of the lights. The next step was to connect to the internet. To do this, I attached the Ladyada ethernet shield with the wiznet module to my Arduino and plugged an ethernet cable from that into the GitHub office router.
I had experimented with Arduino and Ethernet before so I was pretty familiar with the hardware and software involved. The main new element this time was that instead of serving up webpages from the Arduino I needed to use the Arduino as a client and specifically I needed to capture the status of a particular URL since that's how CI Joe reports its status (a 200 response means a successful build, a 412 with the word "building" means the build is in progress, and a 412 with a git sha as the body means that the build is broken and the commit corresponding to that particular sha is to blame.)
After a bit of googling, I came across the Arduino String library which includes a "contains" method that will search a given string for some substring and return true if the substring is found. If I formulated the request correctly from the Ethernet library I could get the library to print out the response headers which include the status, which I could then pull out using this contains method.
Once I had that figured out, I pretty quickly put together a sketch that would poll a given URL and blink the red lamp if that URL was 400 and blink the green lamp if it was 200. I tested my code against a URL on a server I controlled so I didn't have to ask the GitHub guys to force a rebuild of their site every few minutes while I was working on things.
Before too long, I had the basic status switching working:
Now at this point, all the pieces of the project were theoretically in place; all that was left was putting it all together, pointing it at the real GitHub CI server, and rewiring it so the Arduino itself would also be powered from the AC coming into the stoplight from the wall.
Up to this point, I'd been powering the Arduino with a simple AC to USB power converter:
Before I'd left New York, Tom Igoe had suggested simply wiring a two-prong outlet in series with the stoplight's AC power cable and using that to power the Arduino so I didn't need a separate power supply.
So, on Monday this is what we did. For some reason, the stoplight's AC cord had been routed out of a hole in the top of its case:
I cut this cord just below the bank of screw terminals and ran it back into the stoplight through a small hole in its steel base (a hole I'd also run the ethernet cable through earlier).
Then with Andy's help (he had some experience with wiring AC and I was starting to be a bit feverish and under the weather by this point) I connected a two prong outlet as well as the leads that went to the screw terminals for all of the lamps to the wires out to the plug, screwing the whole mess of wires together with plastic screw caps:
Then, as you can see in that picture, I just plugged in the AC-to-USB power converter and ran a USB cable to the Arduino. Not the most elegant solution (and I could definitely have benefitted from a shorter USB cable that wouldn't have taken up so much space inside the cramped stoplight case), but it did the job and was pretty straightforward and danger-free.
Now, the whole project was powered by simply plugging the stoplight's trailing power cord into a wall outlet. I switched the Arduino over to point at GitHub's real CI server and updated my response parsing and boom, the lights were switching based on the real status of the build.
I also rewrote the code slightly to use the Arduino's timer library so that the current light would stay lit even while the Arduino was keeping track of the 10 seconds it was supposed to wait between polling the CI server.
Theoretically, the project should have been basically done at this point. But after I let it run for an hour or so, I noticed that the Arduino would start to get stuck, getting only an "incomplete headers" message from the Ethernet library on every response even when I could see that the server was doing the right thing when I hit it in the browser.
Thankfully, I had one more day before it was time for me to leave to head back home so I asked the ITP mailing list for help. Tom Igoe was nice enough to step in and save the day again by letting me know that there's apparently a memory leak or some other problem in the Ethernet library and that the best way to keep it reliable over long periods of operation is to connect the Ethernet shield's reset pin to one of the Arduino's digital outputs and then pull it low after each successful request so that it will reset.
The next day, I came back into the GitHub office and followed his advice. After I figured out that I needed to call Ethernet.begin() again after resetting the shield, it worked like a dream, making requests reliably for a few hours.
At this point I declared the project feature complete and set about shoving all the electronic guts back inside of the stoplight case. In the planning stages this was something I'd actually been quite worried about because I assumed that, being steel, the stoplight case would be conductive and hence prone to making shorts on my boards. On the second day of the project, however, I had the presence of mind to test the stoplight with my multimeter and, much to my pleasure, I discovered that the layer of paint coating the thing was not conductive at all. In fact, nothing inside of the stoplight besides the screw terminals themselves were the slightest bit conductive. This made the prospect of stuffing quite a bit of wire and PCB into the stoplight safely much more likely.
So, I covered the AC connections of the relays with hot glue as a safety measure for anyone who might accidentally end up in close proximity to them with the stoplight still plugged in, and basically just shoved all of the wires inside until the case would close. It was a tight fit, but it worked in the end with space to spare.
I closed the whole thing up and got the guys to run a build and, lo and behold, the stoplight turned green!
Here's how it looks from the other side of the office from the table around which most of the work at GitHub actually gets done:
I just want to say a big thanks to PJ Hyett, Chris Wanstrath, and Tom Preston-Warner, GitHub's founders, for bringing me out as well as GitHub's Kyle Neath who helped set up the CI Joe integration. This project would have been much harder and more dangerous without Tom Igoe's advice and Andy Delcambre's help at every stage of the build. And, of course, a big thank you to Melissa Severini for thinking of me for this project in the first place.
If you want to learn more, the code is, of course, on GitHub: atduskgreg/GitHub-Stoplight. And if you want some hardware bling for your own startup, don't hesitate to get in touch.
This post is the second in a series I'll be writing this week catching up on documenting projects I worked on this semester. The ITP semester goes by like a steam-powered locomotive and good documentation is frequently the first thing thrown under the wheels. This week, I intend to pull some of my projects out from under that rushing train by writing them up here.
Some of the most surprising projects at ITP arise out of collaboration. When you work with someone with a different background and a different set of interests than your own you frequently end up making work that neither of you would have conceived alone.
A great example of that for me this semester was the project Morgen Fleisig and I built for Living Art in response to an assignment about symmetry. Morgen is a practicing architect and somehow whenever we work together we end up exploring the relationship between music and built space. For example, he and I worked together earlier this semester to compose a set of rules for a group of people to perform a piece of music that would transform based on the space in which it was performed: Instructions for Indoor Music.
For the symmetry assignment, Morgen and I decided to explore ways of expressing a single symmetrical pattern in both musical and spatial form. We started with a scale from Slonimsky's Thesaurus of Scales and Melodic Patterns.
Slonimsky's Thesaurus is a monument from the era of Modernism in music when composers explored the formal and mathematical relationships inherent in the tempered scale and the intervalic relationships that constitute it. It achieved wider fame later on when John Coltrane professed its importance in developing his "sheets of sound" style (see the wikipedia entry on the Coltrane Changes for more info).
The Thesaurus proceeds systematically through every possible equal sub-division of the octave to define a series of scales and then explores a set of patterns and arpeggiations on those scales. Put another way, Slonimsky provides an exhaustive exploration of the mathematical patterns and symmetries within the tempered scale beyond the traditional major, minor, and other modes.
After some discussion, Morgen and I chose to work with the Whole Tone Progression:
This scale divides the octave into six equal parts. To match this scale, Morgen chose a Hexagonal Truncated Trapezohedron, a geometric shape constructed out of two layers of six pentagons anchored at the top and bottom to the sides of a hexagon:
A Hexagonal Truncated Trapezohedron is the spatial equivalent of the whole tone scale because it divides the full 360 degrees of the circle into six equal parts, just as the whole tone scale does the octave. Looked at from the point of view of the Whole Tone Progression, the Trapezohedron provides a number of affordances: you could map the six tones of the scale to each of the pentagons on either layer in a number of different configurations.
In addition to the scales themselves, Slonimsky provides a set of patterns and sequences to explore those scales. For each progression, the sequences start simple and get more complicated—moving from Ultrapolation of One Note through Infrapolation of One Note to Infra-Interpolation and eventually all the way to Infra-Inter-Ultrapolation.
We decided to use the simplest pattern, Ultrapolation of One Note:
Once we had chosen the pattern, we then started working on turning it into a structure and bringing the pattern to life with speakers and LEDs. I took on the electronics and Morgen took on the structure.
After some work and some bumping up against the timer limitations on the Arduino, I got the Arduino Tone library playing three speakers, and three LEDs to play the beginnings of the progression:
Simultaneously, Morgen was working on the structure. After some discussion, we decided that it should be a helmet. Helmets are an efficient way of creating internal spaces that can be inhabited by one person at a time without having to build a whole room.
They also, we discovered, have a a great 1950s-flying-saucer look to them:
Especially when worn on the head (modeled here by Todd) and covered with blinking LEDs:
We ended up arranging the speakers symmetrically around the helmet in pairs so that the musical sequence would move around your head without the stereo image changing. We also covered each surface of the helmet with vellum to diffuse the LED indicating the active panels.
Here are a couple of videos of people in the class wearing the helmet in its final form.
The real joy of the final piece for me was watching people wear it. It ended up turning them into 1950s robots from Planet Modernism in a way that embodied both the techno-seriousness and the ridiculousness of modernist music.
You can see more images of it on my flickr set about the project: Whole Tone Ultrapolation on a Hexagonal Truncated Trapezohedron.
This post is the first in a series I'll be writing this week catching up on documenting projects I worked on this semester. The ITP semester goes by like a steam-powered locomotive and good documentation is frequently the first thing thrown under the wheels. This week, I intend to pull some of my projects out from under that rushing train by writing them up here.
Snake Party is a game I designed with Meredith Hasson, Mike Kneupfel, Tianwei Liu, and Luis Volante for the "urban games" unit of Big Games. The assignment was to design a game to be played in the immediate blocks around NYU between Broadway and Washington Square Park.
Our group decided early on that we wanted to model our game after an old fashioned video game. Hence, we seized on the NYU Stern plaza, which is a large flat space with a built-in grid inscribed in it:
After experimenting with a number of different mechanics based on games ranging from Battleship to Tic-Tac-Toe, we settled on an adaptation of the 1970s game, Snake:
As Wikipedia describes, in Snake "the player controls a long, thin creature, resembling a snake, which roams around on a bordered plane, picking up food (or some other item), trying to avoid hitting its own tail or the "walls" that surround the playing area."
To recreate this mechanic in real life, we divided the players up into two teams, each connected together by a rope. Each snake had a head and a tail. The game grid was littered with balloons and party hats. The goal of each snake was to collect a hat for each of its players. Only the frontmost player in each snake could pick up a hat. When the snake ran into a balloon, the frontmost player moved to the back of the snake. Each snake moved one square forward, left, or right every five seconds, triggered by the blowing of a horn.
One interesting thing happened during the course of play that we didn't expect: one team's snake was able to trap the other team by moving perpendicularly in front of it. We hadn't anticipated this being possible and so we were forced to improvise a rule on the spot whereby the trapped snake simply stood still until the moving snake past by.
The game was relatively well-balanced, both snakes were close to collecting all of the hats when one of them won.
We received a lot of good feedback in the discussion after the play session. Best of all was the suggestion of breaking up the teams into smaller snakes so that more players would have a more active role at each moment. As the game went on the players began to cooperate more, the players not at the head of the snake helping to advise the head about where they should move. Smaller teams would encourage that communication and create more opportunities for interactions between the teams.
Thanks to Meredith Hasson for the photos in this post.
I wrote recently about the design and production process for an ongoing video I was making for the song Ambulance by TV On The Radio in collaboration with Calli Higgins. Well, I'm proud to announce that we finished the video this week and it's now online for your viewing pleasure (best enjoyed full screen):
Ambulance (unofficial video) from Greg Borenstein on Vimeo.
After completion, I calculated that between the two of us Calli and I put around 370 hours into making this video. I worked on it for twelve weeks and she worked on it for five.
One striking contrast in the animating process was how quickly the edit came together in comparison with the actual animating. Since shooting each second of video and applying the character's faces was so time consuming, the video was storyboarded relatively tightly to not waste any precious seconds of footage.
We couldn't be prouder of the final result and are currently trying to get in touch with the band to show it to them. If you have any leads for how we can do that, please get in touch with me.
If you want to learn more about the design and construction process for the set and characters, take a look at my earlier post: Ambulance Video Design Process and Animatic.
For the last five weeks or so, I've been working on stop motion video for the song Ambulance by TV On The Radio. I started it for the stop motion assignment for Methods of Motion, have continued it on through the After Effects assignment, and will be pressing on work with it through the rest of the semester for my final project.
The process has had four stages so far: building the environment, designing the characters, animating, and putting in the faces. I'll go through them one by one.
When I first heard the song the idea for a narrative popped into my head almost immediately. The imagery was inspired by the combination of the running water sounds that open and close the track and the "dark barbershop" quality of the harmonies. In my mind these combined to summon imagery that juxtaposed warm domesticity with violence and raw violent nature. The basic outline of the story would be: at a warm household on a riverside cliff, a woman waits. Her man returns home infuriated and attacks her. She escapes and he chases her along the cliffside until she falls into the water. He chases her through the river until she finally manages to fight back, kicking him into the water where he drowns as the lights of an ambulance flash in the distance.
I started with the part I was most comfortable with: building the set. Having already built a miniature mountain for a previous project, I knew how to proceed with the construction. I set about laying out the landscape with balls of rolled up newspaper.
Then I covered the paper in strips of plaster cloth
and painted it:
Finally, I added grass (from plastic modeling dust) and trees (out of sticks from Tompkins Square Park).
Since the scene takes place at night, I imagined it lit by a ghostly blue light that seemed to originate from the water. This took a little bit of experimenting with to get right. Finally I found that a combination of cut-up blue cellophane and vellum gave the light the right diffuse quality so you couldn't see
The other great advantage of this technique was that it's extremely easy to move the blue cellophane around between shooting each frame to create a compelling illusion that the water is actually moving.
The next thing I had to figure out was how to do the characters. I struggled with this for a couple of weeks. At first, I experimented with taking photographs of a couple of people in my class, making them into paper dolls and moving them through the environment. I recruited two actors, photographed them in a variety of poses and facial expressions and then had high quality prints made so I could cut them into paper dolls. Here's the print I made of David Phillips who I'd recruited to play the male character:
This approach had two problems. When I finished constructing the paper dolls, I rapidly realized that they were going to be extremely difficult to animate. I didn't have enough poses to really just swap them out and all of my attempts to cut them up and attach them to different armatures failed miserably.
Also, while these photographic prints did look somewhat compelling on the set, a number of people in the class gave me the feedback that they didn't quite match it. They were both too specific and too mundane.
So, after some more experimenting, I decided to go with a hybrid approach. All along in the process, I'd been using small posable wooden drawing dummies while constructing the set to get its scale correct:
These figures had a nice neutral quality that made them iconic and I liked the way that the moved. The main problem with them was that they were identical making it hard to distinguish the male and female characters. Also, lacking faces, they lacked a certain subtlety of expression.
To solve these problems, I decided to team up with Calli Higgins, a fellow student in the class. Calli had some pre-existing experience with After Effects and we decided to experiment with using it to superimpose faces onto the drawing dummies.
It took some brilliant finagling by Calli with multiplication layers and transparency, but here was our first test footage (it's a short clip so watching full screen will make it dramatically easier to see the faces):
Ambulance - test from Calli Higgins on Vimeo.
We used our own faces for each of the characters, taking pictures with the built-in iSight camera whenever we needed a new pose. Then Calli worked painstakingly through frame-by-frame to match the faces to the dummies' poses.
With the world and characters in place, we've begun pushing through animating the rest of the piece. I'm shooting the raw stop-motion footage and then handing off to Calli to apply the faces. This week, in order to show the whole story to the class and to let Calli prepare for what was coming, I edited together the existing minute or so of animated footage that I'd shot along with an animatic, a kind of moving storyboard made by dissolving between still shots.
Ambulance Video Animatic and partial assemblage from Greg Borenstein on Vimeo.
The animation is a slow and painstaking process, with an hour of work resulting in about two seconds of footage. From here on out it's just a slow slog where I need to shoot about 6-10 seconds of footage per day for the next three weeks to get the thing finished.
For more "behind the scenes" on the building of the set, you can take a look at my Ambulance video set on Flickr.
For Mechanisms and Things That Move this week, Dustyn asked us to draw a Free Body Diagram for some common household object. Since I've been spending nearly every waking minute beside my camera shooting a stop motion animation, I chose my tripod.
What's a Free Body Diagram? A Free Body Diagram portrays the forces acting on a system. To draw a Free Body Diagram you break your system down into a set of simple machines and use the laws of mechanics to analyze how the various parts of those machines balance each other. Drawing one lets you know how the variables in your system relate to each other: how the weight of one object might affect the necessary torque in a motor, or how the angle of a support might affect the kind type of fastener you use, etc.
To start off with, here's a drawing of my camera on my tripod.
I made a drawing because, obviously, my camera was busy applying torque to the tripod. Now, here's the Force Body Diagram:
(View the large size.) There are two parts of the system I'm analyzing here. First, the screw which holds the camera in place at an angle at the top of the tripod. Second, the three legs that support the weight of the camera and tripod system. I'll take the first part first.
The screw-system holds the camera in place at an angle by balancing the torque applied by the camera at the ball joint with the force from the screw which holds the ball joint in place. As you can see in the diagram, the torque is equal to the length of the shaft to the camera (in real life no more than a couple of inches) and the weight of the camera itself. The product of those two is the minimum amount of force the screw must be applying since the camera is currently aloft at quite the jaunty angle.
The second system, the tripod legs, has two things going on. First, the angle of each of the tripod legs serves to distribute a portion of the total weight of the camera and the above part of the tripod. Each leg's angle lies at a vertex of a triangle between the leg, the floor, and the extended vertical line of the tripod's shaft. The tripod's weight is applied along that vertical line adjacent to the angle and we want to know the portion distributed to the leg, which makes up the hypotenuse of the triangle. Hence, the part of the weight carried by each leg is equal to the total weight divided by the cosine of the angle.
Now, let's take this a bit further. Take a look at the three equations at the bottom of the diagram. We know that the sum of the portions for each leg added up to the total weight of the camera plus tripod system. If we do a little algebra, something interesting happens. The weight of the camera plus tripod actually cancels out of the equation. We're left with a relationship between the three leg-angles themselves directly. This makes sense. The angle of each leg only determines what portion of the weight that particular leg bears, not the total weight its capable of bearing. As long as each leg is doing its share of the work, then (all other things being equal) they should be able to hold up the weight above them.
Another interesting observation falls out of this bit of algebra. The cosine of an angle decreases as the angle rise from 0 to 90 degrees. So, of course, the inverse of that cosine increases along that same range of angles. In other words, the further out you pull each leg from the tripod, the less of the weight that leg supports.
This might be a counterintuitive result since when we spread the legs out the tripod becomes more stable. That is due to a factor illustrated in the second inset in my diagram: when we spread out the legs, we make it more likely that the center of mass of the tripod plus camera system will be within the plane defined by the tripod's legs.
Anyway, I'm sure this analysis is incomplete. One of the main experiences of making this kind of diagram is that the more you put into it, the more you think of to put into it. And part of the art is choosing when to stop.
