README.md

# Distributed Dataflow Machine Controllers

**Status**

The project is ongoing. Circuit designs listed here are stable, but software across the board is undergoing a big shift: things are getting better. Currently aiming at March 1 *release* of dataflow-all-the-way-down.

## What it Is

The *Distributed Dataflow Machine Controllers* project (pending a decent name) is a collection of reconfigurable open hardware and software for machine-building, process development, and robotics projects. 

It uses a Distributed Dataflow Programming Paradigm: hardware and software objects are all nodes in a graph, executing computing and physical tasks, together!

[RuNDMC](https://gitlab.cba.mit.edu/jakeread/rndmc) serves a development environment for those graphs: and Integrated Development and Operation Environment (IDOE).

![atkapi](images/machine-with-atkapi.jpg)

This means that *adding and removing hardware to a system is easy, and possible.* It also means that *re-writing machine controllers,* to accept new inputs, or deliver new outputs (material, or feedback) is easy. Machine controllers are transparent: their operating principles are obvious to their users, and so intelligent intervention (and modification!) of controllers is possible. We can re-use algorithms and processes that we have developed across many machine instantiations, and we can build custom systems with less overhead. Tools are pliable again, and under our control. 

## Machines Running DDMCs

 - [Little Rascal](https://gitlab.cba.mit.edu/jakeread/littlerascal)
 - [Mother Mother: a Machine Generalist](https://gitlab.cba.mit.edu/jakeread/mothermother)
 - [Small Stress and Strain Machine (uSSM)](https://gitlab.cba.mit.edu/jakeread/ussm)
 - [*A Machine* for playing *Music for Pieces of Wood* by Steve Reich by Jake Read](https://gitlab.cba.mit.edu/jakeread/reich)
 - [MPVMachine](https://gitlab.cba.mit.edu/jakeread/mpvmachine)
 - [SmallGantries](https://gitlab.cba.mit.edu/jakeread/smallgantries)
 - [ClayStacker](https://gitlab.cba.mit.edu/jakeread/claystack)

# The Endpoints

![endpoints](images/endpoints.jpg)

When I say 'Endpoint' I mean the circuit boards (and computing) we expect to find at the edge of the network that we use to connect each of our modular controllers. I also mean: the modular controllers. These are the circuit boards that you seek. 

### Mother Boards

I have started using 'Motherboards' with 'Daughters' - Mother Boards (of which I have only one so far) are home to Computing and Network Hardware: they are the minimum viable package needed to start having a Modular Controller. They feature one 60-pin (with 30 cpu pins connected) 'mezzanine' / 'board-to-board' connector.

[XMEGA Motherboard](https://gitlab.cba.mit.edu/jakeread/motherboard-xmega)  

### Daughter Boards

Through the 60-pin mezzanine connector mentioned above, I connect the hardware-that-does-work. These are the gate-drivers, mosfets, sensing elements etc, that you need to actually do modular control.

[Router](https://gitlab.cba.mit.edu/jakeread/daughterboard-router)  
[Stepper](https://gitlab.cba.mit.edu/jakeread/daughterboard-stepper)  
!TODO migrate doc from BLDC, DC, Ultra, and Breadboard.  
!TODO make heating board (also current sensing big current board)  

# The Computing Contexts

Each of the computers participating in the control of these machines runs a *context* - contexts, while written in different languages and running on different platforms, share a set of messages they can use to describe their internals to one another - and they run the same *execution model* internally. This is the (simple) reconfiguration that allows us to build controllers whose execution can be understood (and modified) cohesively, despite their operation across a network, in physical space.

I call these 'DMCCs' - Distributed Machine Control Contexts. None are ready for use.

[Node.js DMCC](https://gitlab.cba.mit.edu/jakeread/dmcc-node)  
[Browser DMCC: the IDOE, aka RuNDMC](https://gitlab.cba.mit.edu/jakeread/rndmc)  
[XMEGA DMCC (forthcoming)]

# The Network

To make individual motors and sensors modular, endpoints use a port-forwarding, source-routed network. This is *not a bus* and can be connected in a complete graph. All connections are full-duplex and include hardware for hardware clock synchronization.

Packets typically originate on the network over a usb-to-serial bridge, so you can interface with networked endpoints using any piece of software you can successfully use to open a COM port with: JavaScript, Python etc. The [odb](https://gitlab.cba.mit.edu/jakeread/odb) includes one UBS-to-UART Bridge, for very simple interface. [XMEGA Motherboard](https://gitlab.cba.mit.edu/jakeread/motherboard-xmega) and XMEGA DMCC will offer a USB serial link.

The hardware (or 'PHY' layer) of the network is simple UART. That means that all the network is, is a few UART links between microcontrollers, and some C code that listens for incoming packets and forward messages according to the instructions in the packet. Critically, there are not *addresses* in the network, instead, packets themseleves carry routes: this is 'source routing' in network nomenclature. 

### The Packet

Because our UART hardwares typically operate on bytes, our packets are sets of bytes. Packets are structured like this:

![packet typ](/images/apa-networking-diagrams-packet.png)

### Port Forwarding and Route Shifting

Bytes between the Length Byte and the *Packet Header End Delimiter* define the route. As the packet moves through the network, it is forwarded on the ports in the list, in order. To keep track of the current position of the packet (which port is next in the list) we shift the list of ports along at each forwarding instance, such that the 2nd byte in the packet is always denoting the next port to forward on. We append to the end of the header the port which the packet arrived on, and that way we keep a return route in the packet. When the packet arrives on a port and the *Packet Pointer* is the next byte, the recipient knows the packet has been sent to them, and they handle it.

![packet typ](/images/apa-networking-diagrams-traversal.png)

# Usage

## Wiring

#### Powering Boards

The network cables don't carry any power, just four pairs of differential signals. So each board needs a power connection as well.

The boards should all share a ground, but can run on different voltages of input power. Most will take 24v on two M3 screw-mounts - I use eye terminals soldered to 18ga wire to deliver power. 

The router can also accept power from a USB device. If you're powering it over USB, *do not* also power it via 24v. 

Last thing, don't power your supply on before you go to screw power connections onto the boards. Wire them up, and then switch on.

#### Network Cables 

At a bare minimum, you're going to be hooking these things up to power, and to each other (network). 

 - network cables can be made two ways: only one is correct - *rj45 tabs should be on the same side of the ribbon cable* i.e. the cable is a 'straight through' type, not a crossover. this means that tx meets rx, etc. 
 - the transmit / receive ports are RS-485 Differential Driven, meaning there is no common gnd connection between boards besides the power bus. 

The connectors I use are called 'RJ45' Jacks and Plugs. These are standard for Ethernet, but also 'generally useful'. This is not ethernet, but these *are* RJ45. It's 8 wires, and in our case that's four differential pairs - two duplex lines on each side.

One cool thing about RJ45 is the modularity of the cables. We can use commodity crimping tools to make our own lengths:
 - one side cuts, one side strips. use both at the same time to get the right length of stripped wire
 - use the '8p' crimp, note the tab direction in the crimp
 - pinch! the plug has a plastic tab inside that should come down to meet the wire jacket 

![rj45 video](images/rj45-assembly.mp4)

```make sure those tabs are on the same side of the flat cable```

![rj45](images/rj45-tabs.jpg)

To power the boards, I put M3 'studs' on each PCB. These can handle lots of current, and have a good mechanical connection. To hookup, I solder or crimp eye-terminals onto ~ 18ga - 14ga wire, then screw them down with short M3 Socket Head Screws. 

Power can be bussed - i.e. we can screw two terminals onto the same stud to make the circuit a 'drop' on the line. 

## Network Interface

Once you understand the packet structure, any program you can imagine to write that has access to a USB or Serial Port, can issue and receieve packets. For us, this means [RNDMC](https://gitlab.cba.mit.edu/jakeread/rndmc) - a Reconfigurable Numeric Dataflow Machine Controller. For you, it might mean some other Node.js program (some exaple code for a simple version of which is available [here](atkterminal.js)), or Python, etc. 

## [Firmware -> XMEGAs](reproduction/firmware.md)

Once you've gotten your hands on / or built some ATK Hardware, you'll need to load some C Code (firmware!) onto each chip. The link above is a guide for this.

## [Circuit Reference](reproduction/circuits.md)

If you'd like to make your own hardware for the network, I recommend using the [ATKBreadBoardBoard](https://gitlab.cba.mit.edu/jakeread/atkbreadboardboard) whose documentation also includes a 'daughter' board starter - this will let you plug some new circuitry right onto the available XMEGA pinouts, and use known-to-work programming, voltage regulation, and network interface hardware. 

Along with this note, there's more detailed explanation in the link above.