# Squid Works: Distributed Dataflow Machine Controllers

![explainer](video/squidcut-july2019-explainer-1m30s-enc.mp4)

**Status 2019/11/13**

![works](2019-11-13_saturn-works.mp4)

**Status 2019/07/18**

The project is ongoing. Managers exist and run dataflow programs in Node, in the Brower, and in Embedded CPP. Each manager is now essentially complete. Network serialization algorithms are written, as is a scheme for program description via sequential messaging.

Largely, work is ongoing in application development, and in completing cuttlefish's heirarchichal tools like route building and system restore / save. Work is also ongoing in networking, and many serialization types remain unwritten, but will be filled in as applications call for them...

**Jogging for One Motor via direct USB**

![motors](video/squidworks-2019-07-18-motors.mp4)

That's great, and here's a screenshot and video of running this system through a 2nd link (browser -websocket-> node -usb-> embedded (router) -rj45uart-> embedded (motor))

![L4 Graph](images/2019-07-19-systemone.png)

![L4 Video](video/2019-07-18-systemone.mp4)

**Jogging 3DOF w/ 4 Motors**

![3dofjog](video/squidworks-2019-07-21-4axjog.mp4)

The program runs keystroke events from the browser (that pink box, a collector), down through a websocket to node.js (nautilus), where they are used to run three jogging hunks. Each jogging hunk runs an output / input loop, sending moves to each motor designating how many steps are to be taken during each time interval. Steppers run clocked interrupt intervals (in this case, at 20ms per clock) and execute as many steps in each interval, returning those increments back upstream once they're complete.

*L1*

![l1](images/2019-07-21-l1.png)

*L2*

![l1](images/2019-07-21-l1.png)

*L3*

![l1](images/2019-07-21-l3.png)

*L4*

![l1](images/2019-07-21-l4.png)

## What it Is

The **Squid Works** project is a bundle of modular hardware and software resources for the development of Distributed Dataflow Machine Controllers: a class of controllers that use dataflow programming across network links to orchestrate global control across heterogeneous computing.

This means that *adding and removing hardware to a system is easy, and possible.* It also means that *re-writing machine controllers,* to accept new inputs, or deliver new outputs (material, or feedback) is easy. Machine controllers are transparent: their operating principles are obvious to their users, and so intelligent intervention (and modification!) of controllers is possible. We can re-use algorithms and processes that we have developed across many machine instantiations, and we can build custom systems with less overhead. Digital tools are pliable again (soon), and under our control.

![atkapi](images/machine-with-atkapi.jpg)

### Contexts

Each of the computers participating in the control of these machines runs a *context* - contexts, while written in different languages and running on different platforms, share a set of messages they can use to describe their internals to one another - and they run the same *execution model* internally. This is the (simple) reconfiguration that allows us to build controllers whose execution can be understood (and modified) cohesively, despite their operation across a network, in physical space.

[Cuttlefish](https://gitlab.cba.mit.edu/squidworks/cuttlefish), for the browser, also serves visual representations of its own dataflow graphs, as well as dataflow across a network. This allows us to visualize, interact with, and build and edit distributed programs.

[Nautilus](https://gitlab.cba.mit.edu/squidworks/nautilus), for node.js, can run on servers and on local hardware. This is especially handy when we need to hook up to hardware resources like usb ports, etc. Its internals are directly scraped from cuttlefish.

[Ponyo](https://gitlab.cba.mit.edu/squidworks/ponyo), the smallest fish (and queen of the sea), runs dataflow graphs on embedded microcontrollers. At the moment this is limited to the ATSAMD51J19, a 120MHz Arm M4F. It connects to nautilus via usb links, and to other fishes via a very-fast-uart link that includes a synchronization line. Ponyo is very much under development.

### Circuits

At the end of the day, current moves through coils, voltages are read and delivered, etc. These machines aren't going to operate themselves (or, ) - so these are the hardware tools we use to make them dance.

![endpoints](images/endpoints.jpg)

[ATSAMD51 Motherboard](https://gitlab.cba.mit.edu/squidworks/motherboard-atsamd51) is a host to Ponyo, boasting a 40-pin (30 IO, GND, 5V and 3V3 lines) GPIO 'port'. This connects to daughter boards;

[Stepper Motor Daughterboard](https://gitlab.cba.mit.edu/squidworks/daughterboard-stepper) uses a TMC262 gate driver to drive two discrete H-Bridges, current chopping excited stepper motor coils. Spins those motors. Includes a home for an AS5047 encoder if I ever find time to close that loop.

[H-Bridge Daughterboard](https://gitlab.cba.mit.edu/squidworks/daughterboard-hbridge) uses an A4955 gate driver to drive one discrete H-Bridge, with a sense resistor for closed loop current control, or optional hardware current chopping. The board can also be used to drive one or two heavy dc loads, like heating elements. Includes an SPI breakout port for an encoder, or fancy thermometer, etc.

[Router Daughterboard](https://gitlab.cba.mit.edu/squidworks/daughterboard-router) adds four (4) very-fast-uart (codename VFP) links to the motherboard, establishing system hubs, etc. These can be chained / etc - there is no graph size limit.

[Power Distribution Boards](https://gitlab.cba.mit.edu/squidworks/pdbs) are handy circuits for bussing power (separate from network) around a machine.

[SPI Daughters](https://gitlab.cba.mit.edu/squidworks/spies) are small circuits used to locate sensors off board, i.e. the encoder for brushless servos and spindles.

**Planned Endpoints** include a DC motor driver and BLDC motor driver, both exist in revisions for earlier architectures. Also including an Ultrasonic driver. Many of these boards are just variations on the h-bridge, so there will be some attempt to anneal towards generic power control for i.e. heating, melting, solenoid switching, etc. Sensing is another topic.

## Machines Running DDMCs

 - [Little Rascal](https://gitlab.cba.mit.edu/jakeread/littlerascal)
 - [Mother Mother: a Machine Generalist](https://gitlab.cba.mit.edu/jakeread/mothermother)
 - [Small Stress and Strain Machine (uSSM)](https://gitlab.cba.mit.edu/jakeread/ussm)
 - [*A Machine* for playing *Music for Pieces of Wood* by Steve Reich by Jake Read](https://gitlab.cba.mit.edu/jakeread/reich)
 - [MPVMachine](https://gitlab.cba.mit.edu/jakeread/mpvmachine)
 - [SmallGantries](https://gitlab.cba.mit.edu/jakeread/smallgantries)
 - [ClayStacker](https://gitlab.cba.mit.edu/jakeread/claystack)

(Most of these machines are built with parametric designs for their constituent components, that project is here: [RCT Gantries](https://gitlab.cba.mit.edu/jakeread/rctgantries)).

# Usage

## Wiring

### Powering Distribution

The network cables don't carry any power, just four pairs of differential signals. So each board needs a power connection as well. I am becoming partial to XT30 connectors, and the power-top board [in here](https://gitlab.cba.mit.edu/squidworks/pdbs) has one of those included, but there are no rules.

I have a small set of power distribution boards:

[PDBs](https://gitlab.cba.mit.edu/squidworks/pdbs)

I hook these up end-to-end to make blocks of just-the-kind of power splitter, etc, that I want.

The boards should all share a ground, but can run on different voltages of input power. Most will take 24v on two M3 screw-mounts - I use eye terminals soldered to 18ga wire to deliver power.

The router can also accept power from a USB device. If you're powering it over USB, *do not* also power it via 24v.

Last thing, don't power your supply on before you go to screw power connections onto the boards. Wire them up, and then switch on.

### Network Cables

At a bare minimum, you're going to be hooking these things up to power, and to each other (network).

 - network cables can be made two ways: only one is correct - *rj45 tabs should be on the same side of the ribbon cable* i.e. the cable is a 'straight through' type, not a crossover. this means that tx meets rx, etc.
 - the transmit / receive ports are RS-485 Differential Driven, meaning there is no common gnd connection between boards besides the power bus.

The connectors I use are called 'RJ45' Jacks and Plugs. These are standard for Ethernet, but also 'generally useful'. This is not ethernet, but these *are* RJ45. It's 8 wires, and in our case that's four differential pairs - two duplex lines on each side.

One cool thing about RJ45 is the modularity of the cables. We can use commodity crimping tools to make our own lengths:
 - one side cuts, one side strips. use both at the same time to get the right length of stripped wire
 - use the '8p' crimp, note the tab direction in the crimp
 - pinch! the plug has a plastic tab inside that should come down to meet the wire jacket

![rj45 video](images/rj45-assembly.mp4)

```
make sure those tabs are on the same side of the flat cable
```

![rj45](images/rj45-tabs.jpg)

## Software

Very in-flux, pls check back soon.

## [Firmware -> XMEGAs](reproduction/firmware.md)

## [Circuit Reference](reproduction/circuits.md)