Where we're at
We promised to give you more details on our progress with the project. The last week has been quite busy for me (preparing a trip to China), so it has taken me some time to write it down. But here we are!
(In advance, sorry for the non-techies, feel free to skip to the end. But then, as Obama said, you should really learn to code ;) )
First things first, let me just do a quick recap of the different parts of the project to help you visualize where we are in the development process. The Lima is composed of :
- A hardware adapter, with a USB and an Ethernet port, which enables you to connect any USB hard drive to your Internet router. This adapter enables Lima to use your hard drives as the common and unique memory of your devices. It’s a small embedded computer. We produce it in China.
- A firmware: that’s the program we run on this little adapter, to give it its functionality. The firmware we use is based on a variant of OpenWRT, which is a distribution of Linux. The programs we put in this firmware enable the Lima adapter to be constantly connected to all your devices and to manage the storage of your files. It’s this firmware that maintains a database of all your files, and makes sure they are always available to your devices and backed up.
- Three desktop apps (PC + Mac + Linux): these apps are at the very heart of Lima. They change the way your computers manage their files, by introducing a new file system for your devices. They are the ones making Windows, Mac OS or Linux display all of your files in your computer, no matter its size. They will be used by your OS every time you open or press [Save] on a document.
- Two mobile apps (Android + iOS): they enable you to browse and stream your files from anywhere, and to get all your content always with you, in your pocket.
- A web app, which enables you to access your files directly from the web, in case you're not on your own computer.
Today, the product is still under development. As explained in the last update, we’ve decided to extend our development timeline to make sure we deliver a product stable enough for everyday use.
The part we’re working on the most is the component that serves as the “main engine” of our solution. We call it the “core”, because all our software programs share it: our firmware, our desktop apps and even our mobile apps. This core is essential to the functionality of Lima: it determines how your devices should display and organize files, and how they connect with each other. As long as we’re still working on it, there’s no point in shipping the Lima device to you. Even if our hardware adapters are ready for mass production, they still lack the firmware that transforms them into working Lima devices.
In the process of developing this core component, we’re facing a major challenge: transform our software deliverable from a « working » product – shipped to 1000 early adopters – to a stable and everyday-use product, shipped to 13,000 people. From a software standpoint, shipping to more people doesn’t make the product more difficult to program. But the quality requirements, which come with growing from a low-scale project to a large-scale project, do. Having 13,000 backers means that not everyone is a techie. It’s also already too big a feedback loop to let us iterate fast-enough on the very first versions of the project. So we need to anticipate problems before they appear, and come up with an A-level software right away. That’s what we’re doing right now.
Two factors are specifically important for us: file integrity and performance. As Lima manages all your files, we put a special care in making sure it stores them safely. But it’s also important that the application is very responsive and efficient, as it is used every time you save a file on your device.
The version we planned to ship in December was programmed in mono-threaded Python, for faster prototyping. For techies, we had decided to base our solution on an event-based asynchronous reactor, with PyPy’s JIT enabled for better performance. During our tests, we figured out the response time of file access IOs was going to be below our expectations with this technology. Everything indicated that if we continued to use it exclusively, our first version wasn’t going to be responsive enough for an acceptable use, even in beta. So we took several measures to correct this.
- 1. Change of technology. Our first decision was to change the technology behind the critical parts of our apps, as soon as possible. This has been a bold move, as it meant taking back months and months of work to design a more efficient solution. We decided to stop using Python for current development, and to recode the critical parts of our apps in multi-threaded C. The upside is that using C gives us a far better control on the performance of the applications. It makes Lima way faster. Using multi-threaded code also gives us the opportunity to use the multiple cores available in modern processors, to increase responsiveness even further. So we are in a clear path to achieve our goals in terms of performances. The downside is that we had to readapt in depth the architecture of our program. This process is costly in terms of delays, and is the main cause behind our decision to delay shipping.
- 2. Change of components. We also changed some external components used in our firmware. We were initially using the open source solutions provided with Linux and OpenWRT to manage NTFS and HFS+ file systems. But ntfs-3g proved to be too slow, and hfsplus didn’t handle the HFS+ journal well when writing. So we switched to proprietary components instead, which improves performances and response time.
Another challenge we faced lately is the compatibility with the Mac platform. Thanks to the backer dashboard, we know about 20% of you are using Macs. We realized that Mac OS Maverick broke some of the OS integrations of the Lima Mac app. This is something we need to fix before shipping.
From our performance tests, we also figured out additional work is required to make Lima more integrated with SpotLight and Time Machine, which index your files extensively. So we added these items to our planning.
Currently, we have four people in our software team. From left to right, here are: Denis, Gawen (our CTO), Remy and Avetis. An additional developer, Pierre, will join us full time in January.
Since September, the team has been mostly working on improving the Lima core performance. Here is the current status for each part of the app:
- Firmware: basic firmware mechanics (cross-compilation toolchain, bootloader, linux distribution customization, auto-update routines, rescue mode) are ready. We’re waiting for a stable version of the core to finish integration.
- Desktop apps: most OS integration components (file system drivers, icon overlays) are ready. Some still need fixing to get compatible with Mac OS Maverick. We’ve postponed the development of auxiliary interfaces (file sending window, configuration window) for a few months to focus on priority subjects. The team is working on the core and its installation on the various platforms for the time being.
- Mobile apps: this has also been shifted a bit in the planning. We’ve decided to develop the iOS app slightly before Android, so we can focus on one platform to shape the first version of the interface. Development of the iPhone app (interface only) is giving rapid results. Our internal pilot app is able to browse files and stream media, using a simulated version of our core. Android development starts in January.
- Web app: front end development has started. Our current iteration enables us to browse files online. We're now working on file management features. Backend development is planned to start in February.
- Hardware: as mentioned in last update, the device is now ready for production. We have begun provisioning the components, starting with the processors and the SDRAM, which both require larger provisioning delays. However, to compensate for the additional wait imposed by our new software timeline, we’ve decided to put in additional work on the production process so we can improve the external case design of the device. More info on that very soon.
Here’s how we plan to organize our work for the next steps:
As you can see, we don’t have a full visibility yet on the time needed to refactorize the core. This explains why, even if we know we’ll be able to ship in Spring, we can’t give an exact shipping date yet. Of course, we’ll let you know as soon as we know more. In parallel, we’re continuing the development of the desktop and mobile apps.
That’s it! This was a bit technical, but I think it was necessary to give you a relevant picture of the challenges we’re facing today.
I hope this gives everyone a better idea of where we are and what remains to be done. Lima is a very ambitious and complex project. It requires rethinking most of the file storage foodchain, from the hardware up to the file system, and up again to your desktop & mobile file management interfaces.
Lima will rock storage, but it won’t be just working with random files. Lima will be working with your data. And we take this responsibility very seriously. We must make sure the product we ship keeps your data entirely safe and improves your overall computer experience. That’s why we preferred anticipating problems and working a few more months to make sure our software fits to our (high-level) quality standards.
We’re glad to have you by our side to finish Lima’s development. We’re really eager to ship you the first Lima devices and to start getting your feedback. We’re all working days and nights for this. And we’ll make it.
Thank you for your support and enjoy your day!