Day 868: A thousand steps forward...
TL;DR: We've shipped 1000ish keyboards to backers; we expect to ship the rest in Q4, despite some supply chain issues.
Hello from Oakland!
It's been about five weeks since we last wrote. To say that the past month has been eventful would be a bit of an understatement. As of this week, we've now shipped about 1000 keyboards to Kickstarter backers. (If we shipped you a keyboard from MP1, you should have received a shipping notification from us last week.) We're working with the factory to get the rest of your keyboards built and shipped as quickly as possible.
On that note: If you’ve moved since July, please update your address in BackerKit (or email us at firstname.lastname@example.org and we can update it for you)!
Shipping and fulfillment
To date, we've shipped out just over 1000 keyboards to backers. For a variety of reasons, we ended up not being able to fulfill keyboards in strict backer number order. This time around, the factory sent us 200 loud-click keyboards and 802 quiet-click keyboards. This was a little heavier on the 'loud' keyboards than we'd expected. At the same time, many of you backed us for a pair of keyboards. We decided that it didn't feel right for some of you to get two keyboards from the first mass production run, while far more of you got none. So, we decided that anybody who backed us for a pair would get a single keyboard from the first run. Nobody will get charged any extra shipping fees due to the split shipments.
If you backed us for two keyboards and chose to get both a loud and a quiet keyboard, we randomly selected which one you'd get from this run. (Fun fact: We had to run the random sorter about a dozen times before we got numbers of loud and quiet keyboards that matched the available inventory.)
As of now, we believe we've sent one keyboard to:
- Anybody who backed us before midnight US/Eastern on the first day
- Anybody who backed us for a two-pack on Kickstarter
- Anybody who backed us for a single loud-click keyboard on Kickstarter
- Anybody who volunteered to help us test out the 'PVT' (pilot run) keyboards
We're working hard to get the rest of your keyboards produced and shipped as quickly as we can.
In the last update, we told you that Jesse was on his way to China to train up a new Quality Control contractor, since the previous one told us that he wasn't going to be able to spare any time for us.
The new contractor has been working out great so far. His process is a whole lot more rigorous than the old vendor. Jesse spent a full day with him working through 5% of the first 1000 units one by one, building a new quality standard. After that, he worked directly with our factory's QC team to re-audit all 1000 keyboards fixing a number of issues, mostly related to the quality of the wooden enclosures. At the end of each day, we got a report about what he'd found and worked together to figure out where to draw the line on "marginal" items.
One thing we discovered during the first day of QC testing was that the RJ45 cable vendor had made every cable 1cm longer than we'd specified, which made the keyboard a bit harder to put together than it ought to be. In the end, we decided to accept these cables for the first 1000 keyboards, but to require that they be fixed for future production runs.
There were also, still, a couple of small infelicities in the laser engraving for the MP1 keyboards. Our team from the factory took Jesse to the new laser engraving supplier's workshop and we spent a couple hours moving individual key labels by a millimeter or so until things looked the way we wanted them to. Overall, we really like the new laser engraving supplier. The engineers working there seem to take great pride in their work and seem to really know what they're doing. Changes that took an hour at the last supplier took minutes for these folks to sort out.
Our wood supplier
At some point, one of us remarked to someone that the wooden enclosures for the Model 01 had turned out to be less heartache than we'd expected. Well, we spoke a bit too soon.
The wood supplier we've been working with for more than two years has been having a really hard time meeting their commitments to us. The defect rates we've been seeing for the wooden enclosures have been really, really high. The factory has rejected between 50 and 80 percent of all enclosures delivered by the wood factory.
(We wish it weren't so, but we need to leave some detail out of this story for business reasons.)
Some of the issues are related to the wood itself. We don't allow knots with wood filler on visible surfaces of the keyboard. Similarly, "weird looking" discolorations on visible surfaces are something we've been pretty clear we can't ship to you. This is the sort of thing that is a straightforward consequence of using natural materials; both we and the factory expected a certain reject rate. That hasn't stopped the wood CNC factory from shipping those over.
The bigger quality issues are related to how the wood factory has been doing the 'finish' work on the enclosures. The factory found a large number of enclosures where the nice curved edge of the palmrest had been sanded to an angle. Worse, a bunch of enclosures looked like someone had dremeled the cutout around the thumb keys, leaving it pitted and uneven.
Once we'd come up with the new internal quality standard, the factory's QC went through and pulled out keyboards with enclosures exhibiting these issues. A few that Jesse personally thought weren't very noticeable got left in; otherwise we wouldn't have been able to ship even the 1000 keyboards we did ship. So far, we've had one backer report of something that should not have passed QC making it into the wild. (We'll be shipping that backer a replacement enclosure.)
It was pretty clear that there was a disconnect between what the wood supplier thought was acceptable quality and what we were willing to accept, so Jesse, along with a team from the factory, paid them a visit.
We spent the better part of the day with the factory boss and their sales guy working through every class of defect we'd found, trying to determine the cause of the issue and an acceptable mitigation or resolution.
When we asked why the production enclosures didn't match the 'golden sample' we signed off on last year, the sales guy told us that they were produced using a completely different process, by a different team, on different machines.
We asked them to walk us through the production process. That's when we found out the first 'fascinating' detail. They haven't been CNC machining both sides of our enclosure. They've only been CNCing the bottoms. Then, they've been applying the fillet (rounded edge) on the outside edge of the keyboard on a router table. After that, the enclosures get sent down the street to their finishing workshop.
At the finishing workshop, we found out why some of the thumb key areas looked like they'd been dremeled by hand.
The factory had been dremeling them by hand.
After that, we learned why it seemed like the fillet on the top of the keyboards sometimes ended sharply, almost like the tops of the enclosures had been sanded down too aggressively.
The wood shop was smoothing the enclosures' top surfaces on a giant belt sander, before they were sent to the next room to be sealed with polyurethane.
After the production tour, we returned to the wood shop's business office to talk through what we could do to reduce the defect rate and get the schedule back on track.
We proposed that they could rework enclosures that had over-sanded fillets by simply increasing the fillets on those units, so long as the two sides of a keyboard match. (After all, with the over-aggressive sanding, we'd already been shipping fillets that didn't match the design we sent them.) This doesn't impact the sturdiness of the keyboard and, if anything, a more aggressive fillet will be a little more comfortable.
For the keyboards with discolored wood, we proposed that they try a dark 'walnut' stain and that we would be willing to buy some quantity of those pieces from them, though we couldn't ship them to any of our pre-order customers as a "surprise".
For enclosures that were simply scratched, we suggested that they just refinish them.
The only things we said we absolutely couldn't accept under any circumstance were enclosures with visible wood filler or over-aggressive dremeling.
When we left, the boss of the factory had promised to get the second 1000 enclosures to the factory before the National Day holiday started on October 1 and to get the next 1000 enclosures to the factory by the middle of October. They promised to start milling new enclosures for the third batch of 1000 on October 4.
The proposed schedule
Before leaving China, Jesse discussed delivery dates with our factory. They made us promise not to share those dates with you unless we included a disclaimer along the lines of “This is the best possible scenario. If a supplier does not deliver on time or some other problem comes up, we will not be able to meet these dates.” In literature, they call that “foreshadowing.”
If everything had gone to plan, the second 1000 keyboards would have been ready for us by October 18 and the third 1000 should have been ready… now.
What actually happened
Everything did not go to plan.
On October 4, about 500 sets of enclosures showed up at our factory. The factory's QC team audited them, rejecting about half of them out of hand. As far as they could tell, they hadn't been reworked at all. About a week later, this happened again.
So we thought, "Hey, we've got 400+ good enclosures. That's something."
The wood shop promised to rework the rest and deliver them to our factory, along with an engineer to work with our factory's QC team about a week later.
Once again, they showed up with only a couple of hundred enclosures, most of which didn't meet the quality standard they'd agreed to, in writing, with our factory. When, at the same time, they reviewed the "good" enclosures in our factory's warehouse, they found that about half of them had unshippable defects.
For those keeping score, this was now about when we'd been promised the /third/ batch of 1000 enclosures. The factory sent someone out to the wood CNC factory's workshop to check on those new enclosures. It turns out there weren't any.
When pressed, the wood CNC shop told our factory that they'd changed their mind and that, due to the high defect rate, they wouldn't be taking the contract for the remaining 3000 enclosures, including 1000 of the units that were now overdue.
They did say that while they were committed to delivering the second 1000 enclosures, they didn't have a date on which they thought they would deliver them, just yet.
Our factory spent some time working with the wood factory and got them to say they'd honor the promise they'd made earlier to deliver at least the third 1000 sets of enclosures. (This is the part where we can't talk about some business details we really want to talk about.)
What happened next will amaze you.
Ok, fine, it probably won't amaze you, because this wouldn't be a Keyboardio backer update without multiple, cascading catastrophes.
The next day, our factory told us that the wood shop's finishing and painting operation had had a fire and would be out of commission for weeks. To the best of our knowledge, nobody was hurt.
We don't think they made this up.
Finally, this past Thursday, the wood factory told us that the finishing and painting workshop was going to be back in operation by this coming Monday and that we could sleep easy and should expect our 1000 sets of pristine-quality enclosures by "the middle of November."
(As an aside, it sounds like there's an opportunity for us to buy a gorgeous looking historic bridge between two of the boroughs of New York City. If we did a Kickstarter, would anyone want to get in on it?)
Plans B, C, and D
About two weeks ago, when we first found out that the wood factory hadn't been making the enclosures they'd promised to, we started a search for backup suppliers, reaching out to more than 20 other suppliers mostly in the Pearl River Delta area (near our factory) that looked like they might be a good fit.
We got quotes from 10 wood factories ourselves. Our factory found a few more suppliers and the folks who have been helping us with project management found another couple.
At this point, we have at least five potential suppliers who will finish samples of the enclosure for us this coming week. At the same time, the factory is working through site visits at each supplier, to make sure they appear to be on the up and up.
Most of the suppliers are within a few dollars of the original supplier in one direction or the other. Most of them have quoted a lead time of around 20 days for the first 1000 sets.
Our current plan is to place orders for 1000 sets of enclosures with two or three of these new suppliers. We'll ask each one to deliver their first 500 sets as soon as possible.
If the original factory somehow manages to deliver 1000 high-quality enclosures, we'll ship 'em. If the new suppliers deliver before the original factory, we'll ship those.
Once we have 1000 sets of good quality enclosures, the factory can turn a 1000 unit mass production run in 7-10 days. After that, third party QC will take a few days, then air shipment to the US for distribution should take somewhere between 5 and 8 days, depending on a bunch of factors.
Once it got to our fulfillment partner, this first mass production run took about two weeks to ship out. That’s a good deal longer than we’d expected or planned for. We're working on a few ways to try to cut this down and get things shipped out much faster than happened for MP1.
As soon as the factory has another set of 1000 good quality enclosures, they'll do another mass production run again. We believe that third set of 1000 keyboards will cover all existing pre-orders, possibly leaving us with a few units in stock to sell for immediate fulfillment.
After that third batch of 1000 keyboards gets shipped out by the factory, the factory will keep working on the remainder of our order: MP4, the fourth mass production run. (These are keyboards that nobody's bought yet.) We've come to terms with the fact that we won't have new keyboards in stock to sell on Black Friday this year.
We're well aware that we're poster children for Murphy's Law and that making the following assertion has a decent chance of invalidating it: As of today, we still believe that everybody who has pre-ordered a keyboard will get it before December 25, 2017.
Extra keycap sets
As we've mentioned before, all keyboards are shipping with QWERTY keycaps. Any extra keycap sets we owe you will ship later under separate cover. The factory says that the tooling for the packaging for the extra keycap sets has been completed, but we haven't seen samples or photos of it yet.
The factory really doesn't want to produce the 'extra' keycap sets until keyboard production is properly in hand.
To help control shipping costs and shorten timelines, we're looking at shipping your keycaps to you directly from the factory. As we know more, we'll definitely tell you.
The Keyboardio community forums
If you haven't already checked out our community forums, head on over to https://community.keyboard.io. There's plenty of interesting discussion about code, mounting options, keyboard layouts, and all sorts of other things.
What to do if you're having trouble with your Model 01
Generally the reports we've been getting from the field are overwhelmingly positive, though, as with any physical product, a few backers have received units that weren't as perfect as they should have been. We've been working with the factory to make sure that issues found with MP1 keyboards won't be repeated during MP2.
If your keyboard isn't behaving as you'd expect, please drop us a line at email@example.com and we'll make it right.
Seeing us in person
We're taking part in the big Bay Area Mechanical Keyboard Meetup that's happening on November 11th in Palo Alto. There will be a lot of keyboards there, ours and others' and old rare ones, plus some talks and good times. The event is free, but requires advanced sign-up. Hope to see some of you there!