A data flow diagram (DFD) is a graphical representation of the flow of data through an information system without any indication of time. DFDs are commonly used to provide an initial top-down analysis of a system, identifying the processes to be carried out and the interactions and data exchanges between them.
Michael Isard received a D.Phil in computer vision in 1998, and gradually started doing more systems research over 13 years at Microsoft Research Silicon Valley. While at Microsoft he participated in a variety of distributed systems research including leading the Dryad distributed dataflow project. Since 2015 he has worked at Google Brain, mostly on the TPU back-end for TensorFlow. Much of his systems research is related to the interaction between programming languages and systems, and how language features can make it easier to write high-performance distributed and concurrent programs.
![Flow Flow](/uploads/1/2/5/3/125362139/693614356.png)
Is a new programming language which provides high-level programming idioms that compile down to code that is intended to be as good as (or better than) one could do by hand. At the core of Rust is the concept of 'ownership', where all values, resources, allocations, handles, etc have a unique 'owner', with a statically known lifetime. This allows Rust to provide automatic memory management, providing safe code with relatively few (not zero) leaks, without requiring a tracing garbage collector. In exchange, there is some amount of additional cognitive overhead when resources have weird or unknown lifetimes.Timely dataflow is a model for data-parallel dataflow computations, introduced as the foundation for the high-throughput, low-latency distributed system. Naiad achieved top-tier performance by providing programmers with a well-engineered data plane and simple but effective control mechanisms, but otherwise staying out of the way.
At the same time, much of Naiad's engineering work went in to fighting against the language and runtime in order to extract the intended performance, similar to what several JVM based systems are currently struggling against.Having recently been given the opportunity, I re-implemented timely dataflow in Rust and found it to be a very liberating experience. Rust's concept of ownership dovetails very elegantly with the data-driven computation of most distributed dataflow systems. At the same time, by providing Rust clear information about data ownership the compiled code avoids the overheads (and pain) associated with a garbage collected runtime. The look and feel of user code is mostly the same as in managed languages; ownership issues are rarely raised to the data-driven, and so ownership-friendly, user code.What result is a system that presents a high-level dataflow programming interface, but whose performance resembles that of carefully implemented HPC codes.
![What is timely information What is timely information](http://4.bp.blogspot.com/-G0jiTEVRS4o/UxZD-DJ63ZI/AAAAAAAABkE/G8f7zifi7Tw/s1600/Screen+Shot+2014-03-04+at+4.21.22+PM.png)
I'll try and explain a few key moments that make this possible, and how systems implemented over other platforms should at least understand and explain what they are giving up.