For backers only. If you're a backer of this project, please log in to read this post.
Open Source, Teamin' Up, The Stack, and Looking Forward
Shirts and Shipping
Upcoming.org: The Shirts
Final 24 Hours!
One Week Left!
In case you missed it, the Upcoming.org homepage was updated seconds after the Kickstarter project ended on Friday:
For those who didn't recognize it, or clicked through before the music started playing, the original music on the page for the duration of the Kickstarter project was a MIDI version of Elton John's "Don't Let the Sun Go Down on Me." Naturally, I had to change it once the project ended.
And don't worry, the new Upcoming won't autoplay music. (Probably.)
Did you know modern browsers don't support playing MIDI files? With all the advancements in HTML 5 Audio, I still had to convert the MIDI tracks to MP3 to get them to work, ballooning their filesize to over 6000%. So inefficient, but those are the sacrifices you make for that old-school sound.
Okay, the shirt designs are finalized and ready to print. All of you who backed the project for $75 or more get your choice of two designs:
1. The Retro Jersey. Printed on Champion raglan baseball jerseys with three-quarter sleeves, these are perfect reproductions of the first Upcoming.org t-shirt from 2005, minus the Yahoo logos. These are unisex sizes, available from small to 3XL. A sizing chart, with body length/width and sleeve length, is available here.
2. The Modern Minimalist. I commissioned a new hand-lettered version of the Upcoming logo from Clarke Harris, a promising young illustrator in the process of moving to Portland. These will be printed grey on black, on American Apparel's ridiculously soft tri-blend, the same fabric we use for all our XOXO shirts. They're available in men's sizes from small to 2XL and women's sizes from small to XL. American Apparel tends to run small, so please check your size.
Both of these shirts are only available to backers, putting you in an elite group of certified hardcore badasses. If you see anyone else wearing these shirts, you'll know you share something in common, and you've just found a new best friend. Maybe even marriage material.
By the end of this week, you'll receive a Kickstarter survey asking for your shirt pick, size, and shipping information. These shirts are being printed for each one of you, so please double-check your requested shirt size. If the size is wrong, I likely won't have any additional extras to replace it.
You'll be able to change your shipping information on Kickstarter at any point until I send the final confirmation, which will be later this month once the shirts have been printed.
IT'S ALL HAPPENING, PEOPLE
We made it to the last 24 hours, within spitting distance of $100,000 raised! (We went over the $100k briefly on Tuesday night, but dipped back down when a $5,000 backer pulled their pledge. They wanted to advertise their marketing analytics company to you, and I said it wasn't a great fit.)
The Kickoff Party
First things first—if you're in the Portland, Oregon area, please join us for drinks tomorrow as we count down the project's end, and celebrate the kickoff for Upcoming's relaunch. The Kickstarter project ends at 3pm PST, and we'll be on the large outdoor patio at Green Dragon for the countdown, arriving around 2:30pm. I expect we'll be there for several hours, so feel free to drop by after work.
As predicted, there's been a small flurry of media about the project over the last two days. The Verge gave Upcoming the feature treatment, a wonderful profile by Casey Newton about "the brilliant life, stupid death, and improbable return of Upcoming.org." It hit the coveted Top Story slot on The Verge this morning, and has stayed at the top of their homepage all day.
Moments ago, Engadget published their own feature by Nicole Lee, with some stats from former Upcoming PM Vince Maniago and thoughts from Laughing Squid's Scott Beale about how Facebook Events' semi-private state never made it a viable Upcoming replacement.
For the few of you who aren't sick of hearing my voice yet, I did an hour-long interview with Myke Hurley for the Cmd-Space podcast on 5by5.
Finally, I wrote a long, personal story on Medium about "selling out" online communities, and mending past mistakes. It was difficult to write, but I'm happy I did. It includes this sweet GIF I created from Wayne's World. Party on.
Restoring the Archives
I've been working with members of Archive Team to restore the original Upcoming data from its current state—a staggering 3.5 terabytes of compressed HTML, CSS, JSON, and images scraped from the site shortly before it was shut down.
For those who've never heard of them, Archive Team is a group of volunteers working to save websites and other digital material in danger of disappearing from the Web entirely. They were responsible for saving most of Geocities before it was shut down, later releasing it as a massive 640GB torrent, and many other prominent sites that have since been deleted.
They led an effort to back up Upcoming with a decentralized tool called Warrior, a virtual machine that anyone can install and run to help preserve dying sites. It takes tasks from a central server, retrieves a set of files from a web server, and sends them back for collection.
In three days, everyone running the Warrior application collected every event from Upcoming and Archive Team uploaded everything to the Internet Archive.
The data is preserved in a format called WARC, a format designed by the Internet Archive specifically for storing large-scale data from web crawls, supporting any number of binary types and all relevant metadata.
From these WARC files, and the right tools, a site can be reconstructed to its original state. There are a number of tools built upon the standard, like a one-click GUI for preserving and playing back archived sites, a Chrome plugin for creating WARCs from any page, and proxies for viewing their contents.
To reconstruct the archives, I'm using an incredibly useful tool called Warcat, built by Christopher Foo. Warcat expands and parses WARC metadata to reconstruct every saved URL as files on disk, sorted into directories based on the original archived URL. http://example.com/path/test.html will be saved on disk as test.html in a new subdirectory called "path" in a directory called "example.com".
Warcat overwrites the file every time, leaving a single copy of every file referenced across all of Upcoming.
The drawback is that this process takes time, which is why I started automating it now. Currently, it takes around 40 straight hours to process the 10 million records found in each 25GB piece of the Upcoming backup. Two archives have been processed so far, with 140 to go. At this rate, it would take seven straight months to extract the full 3.5TB archives from a single server.
Christopher's looking at ways of optimizing warcat's performance, and worst case, I could split up the tasks among multiple servers, though moving that much data isn't fun.
I knew this would take time, which is why I started automating the process now. While it's lower priority than working on the new Upcoming, I want the archival process to run in tandem while I do design and development on the new site.
So far, 211,090 legacy events have been extracted using Warcat. The oldest is event #200, a Goldfrapp show in Seattle from October 2003. The most recent is #9,995,523, an October 2012 event offering free haircuts for breast cancer survivors at a Tucson, Arizona JC Penney's. (What a difference a decade makes.)
If you're curious about what the process looks like, this is a constant stream on my desktop.
See you on the other side, everyone. And thanks again!
We're in the last week of this Kickstarter, and though backings have slowed, it's still far over 300% the initial goal and I'm deeply grateful to every one of you for proving that Upcoming should exist again.
My other project, XOXO, is gearing up to launch ticket sales and while that's taking a good chunk of my time, I've started preliminary design work on Upcoming—sketching out what I'm going to build, how it should behave, and what's most important for a first work-in-progress build.
Some people have asked about whether you'll need a Twitter or Facebook account to use the new Upcoming. Nope. My plan is to use the Twitter API for authentication and the social graph, but allow anyone to use Upcoming with a valid email address.
I don't plan to support Facebook, at least for now. Most event-related apps in the recent past have focused on Facebook for their identity and social graph, but I don't think this makes as much sense for a community like Upcoming. Partly because Facebook already has its own native events, but more importantly, because I don't think its model of bidirectional friendships makes as much sense on Upcoming as Twitter's one-directional following model.
Broadly generalizing, you follow people on Twitter because they're interesting to you, even if you don't know them in real-life. On Facebook, you follow people who you've met at some point in your life. My hunch is that Twitter's social graph will surface more interesting events on Upcoming than Facebook. Maybe I'm wrong. We'll see!
One thing I've realized while digging into this: there are two problems that Upcoming could solve beyond event discovery, and I'm looking closely at both.
- Where your friends are. Recently, my friend Andy was traveling to Chicago and asked if there was any way to see which of his Twitter followers live there. This is surprisingly difficult. There have been a number of mashups in the past, but as far as I can tell, all of them are dead.
- Where your friends will be. I was a big fan of Dopplr, another dearly-loved website that was shuttered after acquisition, and it solved one problem really well—letting you know when your friends were coming to town, or which of your friends are in town when you travel. Since your interest in events already captures those future plans, this could be something Upcoming could tackle.
I'll focus on the baseline features for event discovery at first, and make sure those work well, but I think these two problems are things Upcoming could help solve.
As for the historical data, I've done some tests with the WARCs captured by Archive Team but haven't started parsing the data yet. With Archive Team's assistance, I now have more direct access to the sizable file collection at the Internet Archive and I plan to start writing scripts to extract the data this weekend.
In the meantime, I'm using the Internet Archive's wonderful 404 Handler to redirect all requests to old Upcoming pages to the Wayback Machine. If the page was indexed by Wayback, you'll see a link to the archived page—if not, the link isn't displayed. You can see that in effect right here, on a 404 link to the very first event I ever posted to Upcoming, created on June 7, 2003.
This makes me happy.
In the chaos of the project's launch, I missed Gina Trapani, Matt Cutts, and Leo Laporte covering the story on This Week in Google, starting at the 1:36:10 mark.
I was also interviewed live on The Social Hour with Amber MacArthur and Sarah Lane, starting at the 21:30 mark, in case you feel like hearing me talk about the past and future of Upcoming for a solid 20 minutes.
I've done a couple more big interviews which should appear sometime next week, so that should be fun. (Amazingly, no sign of Valleywag.)
Seven insane people have backed for the site-wide sponsorship, and an astounding 12 people have taken the City Patron reward, adopting (so far) Los Angeles, San Francisco, Portland, Seattle, Austin, Boulder, Chicago, Twin Cities, Atlanta, New York, and London, England. Together, those sponsorships represent fully half of the funds pledged! Thank you.
As I mentioned before, I'm doing two runs of t-shirts—a set of reproduction baseball jerseys and a newer minimalist design. For the latter, I've reached out to a couple illustrators I love to rethink the logo.
I'll post the results as soon as I have them, and you'll be able to choose which shirt you like when the Kickstarter project ends.
Revisiting the Past
Last week, I appeared on a live late-night talk show here in Portland called Late Night Action, talking about the history of Upcoming, and how I found myself revisiting a project I'd done a decade before. One of the other guests, App Camp for Girl's Jean MacDonald, pointed out it was kind of like Groundhog Day.
I started thinking about that and that sounds about right. I'm repeating a part of my past, fixing mistakes I've made until I can make things right. Partly borne from my guilt and frustration about how Upcoming ended, but more importantly, because I miss it and want it back.
Over the last few years, I've had people ask me if I'd ever consider building Upcoming again and I always balked. Until the domain fell back in my lap, I had no intention of doing another events community because I felt like I'd done it before. I was really just hoping someone else would come along and solve that problem for me. But, dammit, nobody ever did.
So here we are. Now, thanks to you, I'll get to take another crack at it. With luck, I'll break the cycle and never have to do it again. :)