Project image
)}
$1,859
pledged of $90,000pledged of $90,000 goal
17
backers
0seconds to go
Funding Unsuccessful
The project's funding goal was not reached on Sat, January 23 2016 8:00 PM UTC +00:00
Thomas HintzBy Thomas Hintz
First created
Thomas HintzBy Thomas Hintz
First created
$1,859
pledged of $90,000pledged of $90,000 goal
17
backers
0seconds to go
Funding Unsuccessful
The project's funding goal was not reached on Sat, January 23 2016 8:00 PM UTC +00:00

About


Update: Lighting talk and slides from Bay Area Lisp Meetup

(Correction: a byte port advances 8 bits, not 8 bytes.)

Slides

What is the 3L project?

The 3L project is building an operating system from the ground up to be secure and extensible.

Secure

With 3L you can download an untrusted program from the internet and run it safely.

Extensible

Anything in 3L can be modified while in use. This means software updates will be easy and seamless. And if you are a programmer, it means development and experimentation will be much easier.

And More

3L also brings many other innovations and improvements as well. One example is that 3L can save the entire system as you are using it. This means that if the power gets cut or the battery dies you can (instantly) reboot the system and it will be like you never lost power. All your apps are running and with the same content they had before.

Why?

It might seem like our current operating systems could be modified to support these features but they practically can't.

The operating systems we use today were initially developed when hardware was much more limiting and security was at most an afterthought. There is potential for operating systems to be much more than they are today but they are held back by outdated architectures. While an OS that we have today could potentially be retro-fitted with a better architecture it would break all the programs that run on it and everything would have to be rewritten. Since that is the case we might as well start clean and build something designed for the present and the future.

We desperately need a more secure operating system. As more and more systems are digitized and more money is involved in computer systems, security will become more and more paramount. The only way to have a truly secure operating system is to create one that is secure from the ground up and today's operating systems are anything but.

How?

3L is built on a simple security principle: instead of programs choosing what they have access to, the user chooses. When you run a program you get to choose if it has access to the network or your printer. It doesn't matter what the program tries to access it will only succeed if you allow it.

3L is also extensible from the ground up. Anything can be modified, added, or removed while the system is running. The 3L system is built on a very dynamic programming language (Lisp) that makes this easy and natural.

When?

This is a very large and long term project. This first round of funding is to just take the current working proof-of-concept and turn it in to an open source project and something that one can play around with and start developing applications and drivers for.

Who?

3L is a general purpose operating system; it is for everyone. It can be tailored to run on many different devices and for varied scenarios from mobile devices to hardened servers.

3L is also free. It does not cost anything but it will also be run as a fully open source project for anyone to contribute to and the 3L project will be a non-profit with the purpose of serving it's users, not making money off of them.

Current Status

3L is currently a proof-of-concept more than anything else. It will run by itself on "bare metal". It will also run simple programs and supports the development of drivers although it does not yet contain any major hardware support. (A driver is a piece of software that allows the system to interface with things like printers and web cameras.) This round of funding is to develop a more polished "alpha" version that can be more widely used. A more developed system with things like networking and graphical display support is still a long ways off.

Funding Details

This kickstarter is to get the ball rolling on creating the OS of the future. The money raised will support 12 months of further development on the OS by me, Thomas Hintz. It is hoped that by the end of that period more money can be raised to get more people involved in development.

The goal by the end of the 12 months is to have an alpha version available that people can run. It would at least: implement the basic security model, include a full Scheme runtime, allow for interactive development and introspection, and include a basic ACPI driver to load the kernel and save and load data from disk. If things go quickly it could also include more drivers, an editor, and more polish in general.

Demo

Technical Details

3L is really just a single Lisp program. There is only one address space and one runtime that everything else runs inside of. We can do this securely because the language runtime provides and enforces "first class environments". An environment is an object containing references to resources available in the system. It's like running a program but the program doesn't choose what libraries or hardware resources it has access to. Instead programs are run inside their own environment that provides bindings to things it is allowed to use. If you don't want a program to access the network then run it inside of an environment that has no network bindings. If the program calls a function that interacts with the network that function won't be defined in its environment and an exception will be triggered. This mechanism provides for a very robust, fine-grained, comprehensive, and simple security system.

At its core 3L is a R7RS Scheme runtime. It provides all relevant portions of the R7RS Scheme standard as well facilities to manage first class environments and hardware access. The runtime also provides facilities for hardware and memory access, debugging, introspection, first class environments, and first class source code and documentation manipulation.

Technical Summary

  • Interactive, iterative development of the entire OS

You can boot into a REPL (Read Evaluate Print Loop) and can inspect or rewrite any part of the OS as you are using it.

  • Dead simple and strong security

Whenever you run a program you tell it what it has access to, it doesn't get to choose. And every operation is type checked and bounds checked removing many common security vulnerabilities.

  • Command driven

Just like with emacs, even the GUI, desktop environment, or window manager can be controlled with commands and functions.

  • Introspective

Find a critical bug with an OS function while writing your app? Just put your cursor over the buggy OS function or enter it into the console and your editor will jump you to the source code for it. Fix the function and viola, every program that uses that function instantly gets the fixed version, no need to even recompile or reload anything.

  • Debuggable

Insert a breakpoint or trace any part of the OS while using it. If an error occurs you can catch it, fix it, and restart execution to just before the error occurred.

  • Documented

Documentation is provided for the entire system in a first class fashion. (Meaning you can read and change the documentation in your programs.) Along with a comprehensive document explorer.

  • Fully Source Mapped

At runtime the user can view and modify the source of any component from the REPL, an editor, or in a program itself.

  • Symmetric Multi-Processing

3L is designed from the ground up to utilize multiple processors.

  • 64-bit

3L only supports 64-bit architectures, starting with x86_64. 3L is for the future, not the past.

Why (in detail)?

Why rewrite the OS?

The structure and interface of OSes in use today are based mostly on the state of computer science 45 years ago and much more research has been done since then that improves dramatically upon our knowledge at that time. Hardware is also not as limiting of a factor now and is likely to continue improving. The concepts behind this OS is the result of decades of research and development. While implementations of OSes has improved considerably since the first Unix systems were created the fundamental design has changed little. This is not limited to Unix based systems either. All major OSes are built with the same underlying model of an OS kernel being separate from a language runtime and user programs. And they generally use the same tacked-on security measures like user privileges and access-control-lists.

The OSes of today aren't going away for a long, long time but that doesn't mean they are the end-all-and-be-all of the OS. Instead we need to look to the future and build a system based on our new knowledge and understanding.

Why not just improve the OSes we have now? That would be ideal but they are so fundamentally far away from where we need to go that it would have the same net result as starting from scratch, except take longer. That being said, not all is lost. It is still possible to port programs over but how easy that is will depend greatly on how high level they are and how well the language they are written in can deal with the module system and security model. Emulation layers can also be created but should be avoided since they will require an almost unrestricted access to all hardware and system libraries which would completely ruin the improved security infrastructure.

Dropping the filesystem

The filesystem is not the best way to store every type of data. 3L instead just provides a simple key-value storage mapping. A key is a label that maps to a set of blocks on a storage device. Traditional filesystems can be built on top of this mechanism as simple libraries. Security is provided via libraries and the first class environments. If you want to only allow a program access to a specific set of blocks you can create a function that will only read and write to those blocks and pass that function in to the program you want to utilize it. (A more flexible abstraction is provided in practice.) This means you have 100% control of the storage available. Dropping OS level filesystem control also makes it much easier for things like databases to optimize their read and write operations.

Performance

The safety and programmer facilities come a cost but mechanisms will be provided to over come much of these limitations. You can choose to block compile parts or all of the OS and programs and functions can be integrated and inlined along with many other optimizations enabled by block compilation. One major performance issue that current OSes deal with is interrupt handling which requires a lot of memory manipulation and processing power; 3L does not have to deal with that issue. With those performance enhancements 3L could likely achieve 80-90% the speed of current OSes. (Also, see note on 'unsafe operations' below for another performance improvement possibility.)

Unsafe Operations

3L does not architecturally prevent an extension that would allow programs to directly access the memory or run "native" code. It could even be extended to allow programs to run in their own address space. This extension won't be in the default installation but could be provided as a separate version of 3L. This is an option if you want the maximum performance.

Languages Besides Lisp

Even though the core of 3L is written in lisp that does not mean other languages won't be available. Transpilers, byte code compilers, and interpreters will be provided for many languages, including popular ones like Javascript, Python, and Ruby. In fact, every language can be support in a safe way and with all or most of the debugging features and introspection available to lisp programs. Even languages like C can be supported via memory access emulation (although those programs will necessarily run slowly).

PreScheme

Scheme itself needs runtime support, like a garbage collector, to manage things like anonymous functions. This means that before the full Scheme can be used something else has to set it up. 3L uses "PreScheme" which is a restricted subset of Scheme that provides low-level memory access. Aside from a few things that must be written in assembly the entire Scheme runtime and OS is written in at least a subset of Scheme. The advantage of this, aside from being easier to develop and more concise, is that a full Scheme system can be used for development, including a debugger and profiler. Of course some things, like drivers, must be emulated but much of the code, like standalone functions and libraries, can still be developed on a full Scheme runtime. This greatly speeds up development.

Current Status

Right now 3L is still very early in development. It currently supports much of the Scheme language but significant parts are still missing. Currently the syntax-rules facility is not finished although getting close and the runtime does already support a lower level explicit renaming macro facility. Much of the rest of R7RS Scheme can be done from the reference implementation in the R7RS standard once syntax-rules is available. A memory access facility is provided but still needs a lot of work.

Next stage of development:

  • R7RS features: syntax-rules, dynamic-wind, ports
  • More low-level error handling
  • Continue work on R7RS port interface for low-level memory access
  • Garbage Collector
  • Rewrite PreScheme terminal/REPL in 3L's Scheme
  • First Class Environments
  • Modules
  • Rewrite the assembly ACPI driver BMFS in PreScheme (currently provided by BareMetal OS), which is used to load the kernel from disk
  • Basic editor
  • key/value storage mechanism
  • Debugger
  • Profiler
  • Disk Drivers

Licensing

It is planned for 3L to be dual licensed under the 3-clause BSD license and GPL version 2. The goal is for all code to be owned by the organization developing 3L so that other licenses can be added in the future if desired. Currently some code 3L uses is licensed under BSD. Eventually the goal is to rewrite that code under 3L's management organization and license it accordingly or get a copywrite change on that code. PreScheme will also need to be likely rewritten to meets these goals, although it likely needs to be regardless, at some point.

Development Process

If funding is successful a non-profit organization will be created to facilitate development of 3L. Initially development will be mostly limited to a few core developers, although the source will still be available. When it makes sense the development will be opened to everyone to contribute and will be run in an open manner. It will remain a mostly closed development process to start with for practical reasons. The overhead of maintaining an open project early on will greatly slow down the process and would likely kill any chance of long term success. As soon as possible it will be opened up though. Of course anyone is free to provide feedback or fork the project if is not developing in the manner desired.

References

A Security Kernel Based on the Lambda Calculus

First Class Environments

Specification of a Lisp Operating System

Scheme48

Scheme48 Module System

VLISP verified PreScheme compiler

The E Programming Language

Risks and challenges

An operating system is complex and presents many challenges. It is difficult to predict how long one specific component might take to develop. It helps to have a detailed plan but even then one can't know for sure. In a project of this size it is inevitable that some features won't make it in the first release but some features that weren't planned for the first version will certainly make it in. Every effort will be made though to stick with the original as best as possible.

Most of the features planned for the first version are already partially implemented. The biggest challenge will be polishing them off so they are not rife with pitfalls and bugs. To help meet this challenge some new features may be cut to devote more time to polishing off more critical components. This will help keep quality high and provide a base for a healthy project long term.

Learn about accountability on Kickstarter

Questions about this project? Check out the FAQ

Support

  1. Select this reward

    Pledge $1 or more About $1.00

    You will be credited in a list of supporters that ships with the OS for all users to see!

    Less
    Estimated delivery
    4 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  2. Select this reward

    Pledge $10 or more About $10

    You will be credited in a list of bronze level supporters that ships with the OS for all to see!

    Less
    Estimated delivery
    0 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  3. Select this reward

    Pledge $25 or more About $25

    You will be credited in a list of silver level supporters that ships with the OS for all to see!

    Less
    Estimated delivery
    4 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  4. Select this reward

    Pledge $50 or more About $50

    You will be credited in a list of gold level supporters that ships with the OS for all to see!

    Less
    Estimated delivery
    1 backer
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  5. Select this reward

    Pledge $75 or more About $75

    You will be credited in a list of platinum level supporters that ships with the OS for all to see!

    Less
    Estimated delivery
    0 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  6. Select this reward

    Pledge $100 or more About $100

    You will also be credited in a list of super-platinum level supporters that ships with the OS for all to see!

    Less
    Estimated delivery
    4 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  7. Select this reward

    Pledge $250 or more About $250

    When the OS starts up it will display your name crediting you as being a major backer. Your name will appear by itself on a rotating basis with other supporters at this level and above.

    Less
    Estimated delivery
    1 backer
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  8. Select this reward

    Pledge $1,000 or more About $1,000

    A small program of your choice will be included in the default distribution.

    Less
    Estimated delivery
    1 backer
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.

Funding period

- (15 days)