Funded! This project was successfully funded on October 27.

Photo-main
Play
00:00
00:00

A Practical Handbook on Developing Dataflow-Based Programming Systems

 ~~~ Buy the Book at DataflowBook.com ~~~

CTO & Co-founder of Typesafe Inc. (http://typesafe.com/)
CTO & Co-founder of Typesafe Inc. (http://typesafe.com/)

Some Quotes from our Backers...

  • "When I saw your book I was like "FINALLY" the world has woken up to dataflow." - Stewart Mackenzie
  • "I like to pay some attention to new developments... but don't always have the time..., "straight to the chase" books like the one you are proposing to write are very valuable." - Samuel Rødal

Dataflow concepts are the heart of Reactive Programming, Flow-Based Programming (e.g. NoFlo), Unix pipes, Actors and message passing in general.

Dataflow-based systems are easy to design once you understand the large number of implementation details that could drastically change how the system operates. Understanding these vectors of change is important so you don't waste your time developing the wrong system.

Embedded dataflow-like languages are used in a wide range of applications. Video games, web pages, circuit simulation and music production are just a few of the domains that have been using dataflow for years. Every one of those has a specialized dataflow engine designed for the task at hand. This book will help you understand the whole dataflow universe before starting your own system.

By the end of the book you will understand...

  • All possible design choices with dataflow-based systems
  • How their effects interplay
  • How to develop your own dataflow-based system


Public domain source code of an example dataflow system will be available for download from the book's web site once completed.

Table of Contents

(Preliminary)

General Architecture: (Brief introduction to dataflow and its terminology)

  • Overview
  • Benifits of Dataflow
  • Dataflow in a Nutshell


The Nouns of Dataflow:
(Definitions of the common parts of dataflow-based systems and their possible design choices)

  • Nodes & Ports
  • Links
  • Data


The Verbs of Dataflow: (Covers the design choices for the runtime operation of dataflow-based systems and their affect on its operation)

  • Scheduling & Executing
  • -> Static Scheduling
  • -> Dynamic Scheduling
  • Transmitting
  • Time (Glitches & Synchronization)
  • -> Synchronous Dataflow
  • -> Asynchronous Dataflow 


Problematic Graphs: (Understand how implementation choices can change the operation of certain graphs -- programs)

  • Cycles
  • Joins
  • Splits


Specific Implementations: (Describes the key characteristics of common dataflow models with sample code for each section. Discusses how the design choices affects operation -- its pros and cons in comparison to other dataflow models)

  • Pure Dataflow Programming
  • Flow-Based Programming
  • Reactive Programming
  • Unix Pipes
  • Actors


Code samples in C# and other major languages:

  • Download code of a complete dataflow-based programming system that you can extend
  • The .Net Reactive Extensions will not be used
  • Instead, we will show you an example of a complete system built from first principles



Reward Levels

Supporter: $1

Contribute at least $1 if you would like to support us but don't need the book.

E-Book: $12 ($10 for early backers)

Receive access to download the source code and book (DRM free) in PDF and possibly other formats.

E-Book + Updates: $15

Receive access to download the book and source code *PLUS* get 2 years of updates and revisions from the book's web site.

Paperback: $45 ($40 for early backers)

We will mail you a professionally bound paperback (within the US). Also receive access to download the book and updates for 2 years.

Signed Paperback: $50

I will personally, hand sign your copy with any!?! short comment you want ;). It will then be packaged and mailed to you. Also receive access to download the book and updates for 2 years.

Sponsor: $100

If your company would like to sponsor our book, we will put the name, logo, link and short, twitter size description in the book and also on the book's web site for one year from date of publication. You'll also get to download the book and updates for 2 years.

Stretch Goals!!!

250 Facebook Shares

  • I will port the sample code to another programming language in addition to C#. After the funding is completed, all the backers can vote for their choice of language. Please use the button at the top of the page so that I can track the count.

Music: "Someday Soon" by Joe Marson, http://www.joemarson.com/ under Creative Commons licence CC BY-SA 3.0, http://creativecommons.org/licenses/by-sa/3.0/

Risks and challenges Learn about accountability on Kickstarter

Over the years I have built numerous dataflow implementations that I can draw knowledge from. I am continuously searching for more views on dataflow-like systems and started compiling my links in the subreddit reddit.com/r/DataFlowProgramming.
In preparation, I have been taking notes to ensure that I will have enough content for at least a few hundred pages. Printing and shipping costs have been taken into account.

FAQ

  • You should have a little programming under your belt but no PhD is required. Many of our backers have offered to port the sample code to their favorite language. As dataflow is a computational method, I will be writing about the concepts and techniques of dataflow so that it can be implemented in any programming language.

    Last updated:
  • From my experience, the term "Dataflow programming" is an overarching concept for both Flow-Based Programming and Reactive Programming.

    I use the term "Pure Dataflow Programming" in reference to the original idea put forth by Bert Sutherland's paper "The On-line Graphical Specification of Computer Procedures" and others. It is characterized by fine grain operations (small operations like add or subtract) and the assumption that dataflow concepts will be employed for all levels of an application.

    Flow-Based Programming takes the Dataflow concepts but says that the nodes (a.k.a. components) should be programmed using our current, sequential, control-flow programming languages. The Dataflow graph is a layer above and it controls the execution of the nodes.

    Reactive Programming is a newer term. I use it to mean Dataflow using the push model (nodes only execute when data is pushed to it). It is "reactive" to changing data.

    There is a Tower of Babel in respect to Dataflow terminology because we haven't yet defined a common meaning for all the terms. These are my definitions for the terms but everyone seems to define things slightly different.

    I will have a chapter in the book that explains all the terms and their common synonyms.

    As Dataflow concepts are just now starting to come to the attention of the masses, we don't have any consensus on the best, or most common, Dataflow implementation architecture. So the book will cover all the tiny variations of Dataflow architectures and compare them to each other for their pros and cons.

    If you're old enough to remember when OOP started to hit the front pages, you may remember the debate between prototype based OOP and class based OOP. For the most part, class based OOP is the most common model by far. Thus, books now only cover class based OOP. Unfortunately for me, Dataflow Programming has not reached that maturity level yet so I try to cover all variations and let the reader decide what is best.

    Last updated:
  • Big data will show just how much we need to adopt dataflow concepts to deal with the influx of information. No longer can we just run batched analyses of our data... there's just too much of it. We need to handle the data as it arrives.

    A core concept of dataflow is that the data is always in motion. Our ability to generate data has outpaced our ability to store data. With dataflow it is possible to process data as it arrives, analyze it and only store the parts of importance.

    Dataflow offers us so many benefits that to ignore them wouldn't be prudent. The promise of true component based programming, parallelism and reinstating Moore's Law. Yet we have become jaded due to all the hyped up, over-promising technologies that were designed to line the pockets of the gurus, not developers.

    To our benefit, dataflow concepts don't require a PhD to use properly and so everyone can make their own decision as to if it's just all smoke and mirrors or if it is real.

    Last updated:
  • Yes. Initially, I may move sections around when it makes sense. Once the form is stable I will let everyone read and comment on the drafts.

    Last updated:
  • Dataflow is a passion of mine. I've noticed a lot of confusion on the Internet over Dataflow and wanted to write a practical description for a long time. The money is a small compensation to do something I love.

    Last updated:
451
Backers
$8,129
pledged of $3,000 goal
0
seconds to go
Img_20130729_162447.medium

See full bio

  • Pledge $1 or more
    You selected

    3 backers

    SUPPORTER: If you believe in our cause but don't need the book... my deepest thank you.

    Estimated delivery:
  • Pledge $10 or more
    You selected

    50 backers All gone!

    E-BOOK (EARLY BACKERS SPECIAL): Same as the $12 reward but at a reduced price for the first 50 people.

    Estimated delivery:
  • Pledge $12 or more
    You selected

    47 backers

    E-BOOK: Download the book from our website when completed in PDF and possibly other formats.

    Estimated delivery:
  • Pledge $15 or more
    You selected

    298 backers

    E-BOOK + UPDATES: Receive the e-book *AND* 2 years of updates to the book and sample code.

    Estimated delivery:
  • Pledge $40 or more
    You selected

    36 backers Limited (14 left of 50)

    PAPERBACK (EARLY BACKERS SPECIAL): Same as the $45 reward but at a reduced price for the first 50 people.

    Estimated delivery:
    Ships within the US only
  • Pledge $45 or more
    You selected

    9 backers

    PAPERBACK: Receive both the paperback and the electronic versions of the book with updates for 2 years.

    Estimated delivery:
    Ships within the US only
  • Pledge $50 or more
    You selected

    2 backers

    SIGNED PAPERBACK: Your copy will be hand signed by the author with your special message and mailed to you. You also receive the electronic version with updates to the book and sample code for 2 years.

    Estimated delivery:
    Ships within the US only
  • Pledge $100 or more
    You selected

    6 backers

    SPONSORSHIP. Put your company's name, link and short description in a prominent position in the book and on our website for one year. You can also download the book with updates for 2 years.

    Estimated delivery:
Funding period

- (30 days)