Michael Young [81nassau]

Nick Bilton wrote a post on the NYTimes Bits Blog last week about a visualization project we worked on together in the NYTimes R+D lab. Along with Noriaki Okada (our intern in R+D), we created an app that visualizes 24hours worth of web + mobile traffic to nytimes.com on a map. The visualization gives us a peek in to the access patterns of nytimes.com readers throughout the day.

The app was built using Processing and takes advantage of the MovieMaker library to create a time-lapse video. We took an entire day’s worth of access to nytimes.com (web and mobile sites) and trimmed it down so that it’s viewable in a minute and a half long video. We’ve set up a process that will create multiple versions of the time-lapse video for any day: a world view and a view of the US (zoomed-in).

Below are two sample videos, showing traffic to nytimes.com and mobile.nytimes.com the day Michael Jackson died. A few notes about the videos:

• yellow circles = traffic to nytimes.com, red circles = traffic to mobile.nytimes.com
• the size of each circle represents the number of hits coming from each location at a given time
• watch how the mobile traffic (the red circles) grows during the morning hours, as people wake up and commute in to work
• the large red circles (mobile traffic) around Toronto and Dallas are due to those cities being large mobile hubs. E.g. Waterloo, near Toronto, is home to RIM (makers of BlackBerry) and a lot of the BB mobile traffic is proxied through Waterloo before hitting nytimes.com
• watch the “pulse” when the Michael Jackson news broke, around 5:50pm EST
• there are plenty of other interesting observations, such as the mobile traffic patterns in areas like Africa, India, and parts of China.
• the videos below are pretty low-res…hoping to post higher res versions later

I’m hoping to post more about the project later (especially the data collection and aggregation part) and possibly even release some of the code (if I get a little free time).

U.S. Traffic to NYTimes on June 25, 2009

The New York Times site traffic, US, June 25, 2009 from Nick Bilton on Vimeo.

World Traffic to NYTimes on June 25, 2009

The New York Times site traffic, World View, June 25, 2009 from Nick Bilton on Vimeo.

A simple geocode tool that I wrote 3 years back just won the “Mashup of the Day Award” by the site mashupawards.com!

The tool is very basic – it allows you to look up the latitude and longitude of any address (well, any address that is supported by the Yahoo Geocode API). It came in handy when I needed to find some lat/lng pairs for another app I was building…and so I put it the tool out there for people to use. This was back in the day when Google had just released their mapping API, but didn’t have a public geocode API. Ahh…the good ole days.

I’m not sure how/why it won the award, but the good folks at outside.in just linked to it from their new API page…so I’m guessing that’s where the interest came from. I had forgotten about it!

I just threw together an app that pulls down the daily record reviews from Pitchfork and Metacritic and creates a twitter feed of all of the reviews over 7/10 (or 70/100 in the case of Metacritic). All of the reviews in the Twitter feed link off to tracks on hypem. I’m hoping it creates an easy way to find new music…just follow the feed on Twitter and it’ll typically give you 2-4 new albums in your feed each day with links off to some tracks from each album.

Here is a link to the Twitter feed:

www.twitter.com/new_music

Inspiration came from an old New York Magazine article that talked about people who only “Listen To Anything rated over 8.0 on Pitchforkmedia.com.” This could be the killer app for those folks.

Update:
Silicon Alley Insider says the new_music twitter feed is one of Ten News & Information Twitter Feeds You Should Follow

Nick Bilton and I were in London last month to take part in the Yahoo!/BBC Hack Day and ended up winning the award for “Overall Best Hack!”

The application/hack that we built is something that we are calling “shifd”, which is pronounced “shift”, not “shifty”…though Nick is kind of a shifty guy… Anyway, we wanted to build something that allows you to shift between your desktop, mobile, tv, car, etc experiences in a more seamless way…

Hack Day was a tremendous experience…I won’t reiterate everything here…we did a few interviews on our Hack Day experience with the great folks at BBC Backstage and Yahoo! Developers Network (part 1, part 2). Below is a video that explains our hack (we shot it right after we got back from Hack Day and hadn’t slept in three days, so be warned!) and some links to some of the tools that we used in our application.

In the News
Integrating RFID with Web Content on your Phone (makezine.com/blog)
Interview with Shifd Creators: Nick Bilton and Michael Young, Pt. I (developer.yahoo.net)
Interview with Shifd Creators: Nick Bilton and Michael Young, Pt. II (developer.yahoo.net)
Hackday: The overall winners point of view (backstage.bbc.co.uk)
The winning hacks from Hack Day London (developer.yahoo.net)
Shifd: A Clever Mobile App From The NY Times (techcrunch.com)
Computers Noticing Mobiles Via RFID (moconews.net)
Shifd allows seamless shift between PC and cellphone (venturebeat.com)
Geeks Bust Out Brollies as Rain Falls Indoors at Hack Day London (wired.com)

What We Used to Build It
RFID Reader (USB) and an array of RFID Tags (from phidgets.com)
Mobile Phone (Sony Ericsson)
Java (Phidget API/Library)
PHP
MySQL
Scriptaculous
Yahoo! UI Library
Yahoo! Local Search API (version 2)
Yahoo! Maps – Map Image API
SMS
BBC, Yahoo and The New York Times Feeds
Guinness Beer

Why RFID??
RFID may or may not be the proper technology to use here. Of course Bluetooth or NFC achieves the same thing that we were trying to achieve with RFID – detecting presence. We’ve gotten a lot of “Why the hell did you use RFID for this? You should have used Bluetooth!”comments. Sure, Bluetooth is much more prevalent and no one has an RFID reader hooked up to their laptop/desktop… but remember that this was all about a “hack” that we had to throw together in 24 hours. Here’s a few reasons why we ended up using RFID:

1. It was very easy to implement a quick “hack” that would detect “presence” for us using the RFID tags/readers that we took along with us to Hack Day. We only had 24 hours to build this thing and I didn’t want to mess around with a Bluetooth library that I’ve never used before. RFID was easy and that’s why we used it.
2. A lot of the developers at Hack Day that tried to use Bluetooth in their hacks had to resort to “wired” solutions because of all of the noise/interference in the room at Alexandra Palace. Everyone that got up on stage to present their Bluetooth hack said something along the lines of “well, we tried to use bluetooth but couldn’t get it working in this room…”. I’m glad we ended up using RFID…..

What’s Next?
We are working on putting out a version of shifd.com in the next few months. We’re going to launch the web and mobile pieces first and then will layer in the “presence” in a later release (using Bluetooth, etc).

If you’re interested in checking it out, please sign up on shifd.com or shoot me an email and we’ll let you know when it’s ready!

There was an article in the Financial Times last week about my group at the NYTimes.

‘Futurist in residence’ a sign of the NY Times

As he ticks off their qualifications – a maths whizz, a linguist, a newspaper design-er and a “business catalyst” – one imagines the early scenes of a Hollywood action film, in which superheroes with diverse talents are assembled for a big mission.

Classic – superhero reference.

The New York Times launched their new mobile site this week: mobile.nytimes.com. You can also access the site at m.nyt.com, which is nice when typing in the URL on your phone.

The site is rich with content — you can get pretty much all of the “text” content you can get on nytimes.com. The site looks great on phones with larger screen (Treos, Blackberrys, etc) but is text-heavy on smaller phones. This is the first launch of the site, and it will go through some changes in upcoming releases. Some UI changes need to be made to it, especially for smaller-screen phones (e.g. I have to scroll across the Log In and TimesSelect links, as well as the main ad, before I can access the first story link). Jakob Nielsen makes some valid comments about the new site’s UI on paidcontent.org. Look for some updates to the site in the coming months…

Here is a screenshot of the site’s main page, with only the first story shown:

I added a few more news categories to the AP News mashup. It now plots the following AP feeds on the Google map:

• National News
• Sports
• Business
• Technology
• Strange

You can also toggle the news categories on the map so that you can show/hide the various pins. Hopefully it’s a little more useful now…

The AP News Mashup I put together last weekend was mentioned in the Where in the World… Wall Street Journal article (sub. required) yesterday.

81Nassau.com/apnews
This map plots the locations of recent U.S. news stories from the Associated Press. Users can click on each location to view the first few sentences of the story or link to the full article.

Thanks to the author Jessica Vascellaro for the mention!

I just finished a simple app that plots the current AP National news stories on top of Google Maps.

http://81nassau.com/apnews/

The app is a mix of the AP’s National News RSS Feed + Yahoo! Geocoding API + Google Maps.

I have a process running on my server that grabs and parses the AP RSS feed. I parse the location from each story — AP puts a location at the beginning of each story, for example “NEW YORK (AP) — Millions of New Yorkers…“, which is typically city, state or just a city (e.g. “Washington” as in DC). I then convert the location to a latitude/longitude point using the Yahoo! Geocode API. Once that’s done for each of the stories, I write a simple XML file back out to the server — the XML file is basically the RSS file with additional fields for the latitude and longitude of each story. When you load the mashup site in your browser, I use some JavaScript AJAX calls to load, parse and display the stories from the new XML file on the Google Map.

There are typically ten stories in the AP feed and it’s usually updated every 30 minutes to an hour.

It’s a pretty simple app, but I figure it might be useful to some. It was more of a proof of concept than anything. Hopefully my mom can use it in her classroom to help teach the elementary kids about geography + current events…

I put an app together this weekend that geocodes an address from an input form and displays the location on a Google map. I’ve cleaned it up a little and put it online for anyone who’s interested…

http://81nassau.com/demos/geocode/

The app consists of a simple HTML form that allows you to input street, city and state. The lookup/submit button on the form initiates an XmlHttpRequest (ajax) to a simple PHP proxy script that I wrote. The PHP script takes the form input (street, city, state) and creates a Yahoo! Geocode REST request, and makes the request to the Yahoo! geocode service. The script then returns the Yahoo! XML data back to the browser. Some javascript is used to parse the XML from Yahoo! and put the lat, lng and zip on the form. A pin is also drawn on the map (Google).

Below is the code for the simple PHP proxy. Feel free to take/use the code as well as the html/javascript. Just do a view-source on the demo link above to see the html/javascript. Please note that there is minimal error checking in all of the code…add where needed.

Enjoy!


< ?php
 
// Simple proxy script for Yahoo! Geocode API
// Note: there is no error checking done on the input or return from Yahoo!
 
header('Content-type: text/xml');
 
$street = $_REQUEST["street"];
$city   = $_REQUEST["city"];
$state  = $_REQUEST["state"];
 
$url = 'http://api.local.yahoo.com/MapsService/V1/geocode?appid=YOUR_APP_ID_GOES_HERE' .
                '&street=' . urlencode($street) .
                '&city=' . urlencode($city) .
                '&state=' . urlencode($state);
 
$xml = get_content($url);
 
print $xml;
 
function get_content($url)
{
        $string = '';
        $handle = fopen($url, "r");
        if ($handle)
        {
                // php 5
                $string = stream_get_contents($handle);
 
                /* php4
                while (!feof($handle)) {
                    $string .= fread($handle, 8192);
                }
                */
              
                fclose($handle);
        }
        return $string;
}
 
?>