About this project
~~~ Buy the Book at DataflowBook.com ~~~
Some Quotes from our Backers...
- "When I saw your book I was like "FINALLY" the world has woken up to dataflow." - Stewart Mackenzie
- "I like to pay some attention to new developments... but don't always have the time..., "straight to the chase" books like the one you are proposing to write are very valuable." - Samuel Rødal
Dataflow concepts are the heart of Reactive Programming, Flow-Based Programming (e.g. NoFlo), Unix pipes, Actors and message passing in general.
Dataflow-based systems are easy to design once you understand the large number of implementation details that could drastically change how the system operates. Understanding these vectors of change is important so you don't waste your time developing the wrong system.
Embedded dataflow-like languages are used in a wide range of applications. Video games, web pages, circuit simulation and music production are just a few of the domains that have been using dataflow for years. Every one of those has a specialized dataflow engine designed for the task at hand. This book will help you understand the whole dataflow universe before starting your own system.
By the end of the book you will understand...
- All possible design choices with dataflow-based systems
- How their effects interplay
- How to develop your own dataflow-based system
Public domain source code of an example dataflow system will be available for download from the book's web site once completed.
Table of Contents
General Architecture: (Brief introduction to dataflow and its terminology)
- Benifits of Dataflow
- Dataflow in a Nutshell
The Nouns of Dataflow: (Definitions of the common parts of dataflow-based systems and their possible design choices)
- Nodes & Ports
The Verbs of Dataflow: (Covers the design choices for the runtime operation of dataflow-based systems and their affect on its operation)
- Scheduling & Executing
- -> Static Scheduling
- -> Dynamic Scheduling
- Time (Glitches & Synchronization)
- -> Synchronous Dataflow
- -> Asynchronous Dataflow
Problematic Graphs: (Understand how implementation choices can change the operation of certain graphs -- programs)
Specific Implementations: (Describes the key characteristics of common dataflow models with sample code for each section. Discusses how the design choices affects operation -- its pros and cons in comparison to other dataflow models)
- Pure Dataflow Programming
- Flow-Based Programming
- Reactive Programming
- Unix Pipes
Code samples in C# and other major languages:
- Download code of a complete dataflow-based programming system that you can extend
- The .Net Reactive Extensions will not be used
- Instead, we will show you an example of a complete system built from first principles
Contribute at least $1 if you would like to support us but don't need the book.
E-Book: $12 ($10 for early backers)
Receive access to download the source code and book (DRM free) in PDF and possibly other formats.
E-Book + Updates: $15
Receive access to download the book and source code *PLUS* get 2 years of updates and revisions from the book's web site.
Paperback: $45 ($40 for early backers)
We will mail you a professionally bound paperback (within the US). Also receive access to download the book and updates for 2 years.
Signed Paperback: $50
I will personally, hand sign your copy with any!?! short comment you want ;). It will then be packaged and mailed to you. Also receive access to download the book and updates for 2 years.
If your company would like to sponsor our book, we will put the name, logo, link and short, twitter size description in the book and also on the book's web site for one year from date of publication. You'll also get to download the book and updates for 2 years.
250 Facebook Shares
- I will port the sample code to another programming language in addition to C#. After the funding is completed, all the backers can vote for their choice of language. Please use the button at the top of the page so that I can track the count.
Risks and challenges
Over the years I have built numerous dataflow implementations that I can draw knowledge from. I am continuously searching for more views on dataflow-like systems and started compiling my links in the subreddit reddit.com/r/DataFlowProgramming.
In preparation, I have been taking notes to ensure that I will have enough content for at least a few hundred pages. Printing and shipping costs have been taken into account.
You should have a little programming under your belt but no PhD is required. Many of our backers have offered to port the sample code to their favorite language. As dataflow is a computational method, I will be writing about the concepts and techniques of dataflow so that it can be implemented in any programming language.
What is the difference is between Flow-Based Programming, Reactive Programming and Dataflow Programming?
From my experience, the term "Dataflow programming" is an overarching concept for both Flow-Based Programming and Reactive Programming.
I use the term "Pure Dataflow Programming" in reference to the original idea put forth by Bert Sutherland's paper "The On-line Graphical Specification of Computer Procedures" and others. It is characterized by fine grain operations (small operations like add or subtract) and the assumption that dataflow concepts will be employed for all levels of an application.
Flow-Based Programming takes the Dataflow concepts but says that the nodes (a.k.a. components) should be programmed using our current, sequential, control-flow programming languages. The Dataflow graph is a layer above and it controls the execution of the nodes.
Reactive Programming is a newer term. I use it to mean Dataflow using the push model (nodes only execute when data is pushed to it). It is "reactive" to changing data.
There is a Tower of Babel in respect to Dataflow terminology because we haven't yet defined a common meaning for all the terms. These are my definitions for the terms but everyone seems to define things slightly different.
I will have a chapter in the book that explains all the terms and their common synonyms.
As Dataflow concepts are just now starting to come to the attention of the masses, we don't have any consensus on the best, or most common, Dataflow implementation architecture. So the book will cover all the tiny variations of Dataflow architectures and compare them to each other for their pros and cons.
If you're old enough to remember when OOP started to hit the front pages, you may remember the debate between prototype based OOP and class based OOP. For the most part, class based OOP is the most common model by far. Thus, books now only cover class based OOP. Unfortunately for me, Dataflow Programming has not reached that maturity level yet so I try to cover all variations and let the reader decide what is best.
Big data will show just how much we need to adopt dataflow concepts to deal with the influx of information. No longer can we just run batched analyses of our data... there's just too much of it. We need to handle the data as it arrives.
A core concept of dataflow is that the data is always in motion. Our ability to generate data has outpaced our ability to store data. With dataflow it is possible to process data as it arrives, analyze it and only store the parts of importance.
Dataflow offers us so many benefits that to ignore them wouldn't be prudent. The promise of true component based programming, parallelism and reinstating Moore's Law. Yet we have become jaded due to all the hyped up, over-promising technologies that were designed to line the pockets of the gurus, not developers.
To our benefit, dataflow concepts don't require a PhD to use properly and so everyone can make their own decision as to if it's just all smoke and mirrors or if it is real.
Yes. Initially, I may move sections around when it makes sense. Once the form is stable I will let everyone read and comment on the drafts.
Dataflow is a passion of mine. I've noticed a lot of confusion on the Internet over Dataflow and wanted to write a practical description for a long time. The money is a small compensation to do something I love.
Support this project
- (30 days)