Commit e0b6162d authored by Jake Read's avatar Jake Read
Browse files

big docu overhaul

parent 61123bc6
# Squid Works: Distributed Dataflow Machine Controllers
# Squidworks: Distributed Dataflow Control
![explainer](video/squidcut-july2019-explainer-1m30s-enc.mp4)
**Status 2019/11/13**
![works](2019-11-13_saturn-works.mp4)
**Status 2019/07/18**
The project is ongoing. Managers exist and run dataflow programs in Node, in the Brower, and in Embedded CPP. Each manager is now essentially complete. Network serialization algorithms are written, as is a scheme for program description via sequential messaging.
Largely, work is ongoing in application development, and in completing cuttlefish's heirarchichal tools like route building and system restore / save. Work is also ongoing in networking, and many serialization types remain unwritten, but will be filled in as applications call for them...
**Jogging for One Motor via direct USB**
![motors](video/squidworks-2019-07-18-motors.mp4)
That's great, and here's a screenshot and video of running this system through a 2nd link (browser -websocket-> node -usb-> embedded (router) -rj45uart-> embedded (motor))
![L4 Graph](images/2019-07-19-systemone.png)
![L4 Video](video/2019-07-18-systemone.mp4)
**Jogging 3DOF w/ 4 Motors**
![3dofjog](video/squidworks-2019-07-21-4axjog.mp4)
The program runs keystroke events from the browser (that pink box, a collector), down through a websocket to node.js (nautilus), where they are used to run three jogging hunks. Each jogging hunk runs an output / input loop, sending moves to each motor designating how many steps are to be taken during each time interval. Steppers run clocked interrupt intervals (in this case, at 20ms per clock) and execute as many steps in each interval, returning those increments back upstream once they're complete.
*L1*
![l1](images/2019-07-21-l1.png)
*L2*
![l1](images/2019-07-21-l1.png)
*L3*
![l1](images/2019-07-21-l3.png)
*L4*
![l1](images/2019-07-21-l4.png)
## What it Is
The **Squidworks** project is a bundle of modular hardware and software that implements *Distributed Dataflow Machine Controllers*. These are a kind of controller whose constituent parts are located throughout a network, with no explicit central state machine. Since the coordination of networked computing resources is often opaque, squidworks proposes to implement virtual dataflow engines for modular code in each of these resources, to align code configuration and execution with that of the network itself.
The **Squid Works** project is a bundle of modular hardware and software resources for the development of Distributed Dataflow Machine Controllers: a class of controllers that use dataflow programming across network links to orchestrate global control across heterogeneous computing.
This means that *adding and removing hardware to a system is easy, and possible.* It also means that *re-writing machine controllers,* to accept new inputs, or deliver new outputs (material, or feedback) is easy. Machine controllers are transparent: their operating principles are obvious to their users, and so intelligent intervention (and modification!) of controllers is possible. We can re-use algorithms and processes that we have developed across many machine instantiations, and we can build custom systems with less overhead. Digital tools are pliable again (soon), and under our control.
![atkapi](images/machine-with-atkapi.jpg)
### Contexts
Each of the computers participating in the control of these machines runs a *context* - contexts, while written in different languages and running on different platforms, share a set of messages they can use to describe their internals to one another - and they run the same *execution model* internally. This is the (simple) reconfiguration that allows us to build controllers whose execution can be understood (and modified) cohesively, despite their operation across a network, in physical space.
[Cuttlefish](https://gitlab.cba.mit.edu/squidworks/cuttlefish), for the browser, also serves visual representations of its own dataflow graphs, as well as dataflow across a network. This allows us to visualize, interact with, and build and edit distributed programs.
[Nautilus](https://gitlab.cba.mit.edu/squidworks/nautilus), for node.js, can run on servers and on local hardware. This is especially handy when we need to hook up to hardware resources like usb ports, etc. Its internals are directly scraped from cuttlefish.
[Ponyo](https://gitlab.cba.mit.edu/squidworks/ponyo), the smallest fish (and queen of the sea), runs dataflow graphs on embedded microcontrollers. At the moment this is limited to the ATSAMD51J19, a 120MHz Arm M4F. It connects to nautilus via usb links, and to other fishes via a very-fast-uart link that includes a synchronization line. Ponyo is very much under development.
### Circuits
At the end of the day, current moves through coils, voltages are read and delivered, etc. These machines aren't going to operate themselves (or, ) - so these are the hardware tools we use to make them dance.
![endpoints](images/endpoints.jpg)
[ATSAMD51 Motherboard](https://gitlab.cba.mit.edu/squidworks/motherboard-atsamd51) is a host to Ponyo, boasting a 40-pin (30 IO, GND, 5V and 3V3 lines) GPIO 'port'. This connects to daughter boards;
[Stepper Motor Daughterboard](https://gitlab.cba.mit.edu/squidworks/daughterboard-stepper) uses a TMC262 gate driver to drive two discrete H-Bridges, current chopping excited stepper motor coils. Spins those motors. Includes a home for an AS5047 encoder if I ever find time to close that loop.
[H-Bridge Daughterboard](https://gitlab.cba.mit.edu/squidworks/daughterboard-hbridge) uses an A4955 gate driver to drive one discrete H-Bridge, with a sense resistor for closed loop current control, or optional hardware current chopping. The board can also be used to drive one or two heavy dc loads, like heating elements. Includes an SPI breakout port for an encoder, or fancy thermometer, etc.
[Router Daughterboard](https://gitlab.cba.mit.edu/squidworks/daughterboard-router) adds four (4) very-fast-uart (codename VFP) links to the motherboard, establishing system hubs, etc. These can be chained / etc - there is no graph size limit.
[Power Distribution Boards](https://gitlab.cba.mit.edu/squidworks/pdbs) are handy circuits for bussing power (separate from network) around a machine.
[SPI Daughters](https://gitlab.cba.mit.edu/squidworks/spies) are small circuits used to locate sensors off board, i.e. the encoder for brushless servos and spindles.
**Planned Endpoints** include a DC motor driver and BLDC motor driver, both exist in revisions for earlier architectures. Also including an Ultrasonic driver. Many of these boards are just variations on the h-bridge, so there will be some attempt to anneal towards generic power control for i.e. heating, melting, solenoid switching, etc. Sensing is another topic.
## Machines Running DDMCs
- [Little Rascal](https://gitlab.cba.mit.edu/jakeread/littlerascal)
- [Mother Mother: a Machine Generalist](https://gitlab.cba.mit.edu/jakeread/mothermother)
- [Small Stress and Strain Machine (uSSM)](https://gitlab.cba.mit.edu/jakeread/ussm)
- [*A Machine* for playing *Music for Pieces of Wood* by Steve Reich by Jake Read](https://gitlab.cba.mit.edu/jakeread/reich)
- [MPVMachine](https://gitlab.cba.mit.edu/jakeread/mpvmachine)
- [SmallGantries](https://gitlab.cba.mit.edu/jakeread/smallgantries)
- [ClayStacker](https://gitlab.cba.mit.edu/jakeread/claystack)
(Most of these machines are built with parametric designs for their constituent components, that project is here: [RCT Gantries](https://gitlab.cba.mit.edu/jakeread/rctgantries)).
# Usage
## Wiring
![explainer](video/squidcut-july2019-explainer-1m30s-enc.mp4)
### Powering Distribution
![works](video/2019-11-13_saturn-works.mp4)
The network cables don't carry any power, just four pairs of differential signals. So each board needs a power connection as well. I am becoming partial to XT30 connectors, and the power-top board [in here](https://gitlab.cba.mit.edu/squidworks/pdbs) has one of those included, but there are no rules.
Because our computing - and hardware - resources are modular, *adding and removing capability to a system should become easy.* It also means that *re-writing machine controllers,* to accept new inputs, or deliver new outputs (material, or feedback) should be simpler. Machine controllers are transparent: their operating principles are obvious to their users, and intelligent intervention (and modification!) of controllers should be possible. We can re-use algorithms and processes that we have developed across many machine instantiations in any particular case, meaning that we should be able to build unique systems with less overhead. Digital tools are pliable again, and under our control.
I have a small set of power distribution boards:
## How does it Work?
[PDBs](https://gitlab.cba.mit.edu/squidworks/pdbs)
### Virtual Dataflow Environments
I hook these up end-to-end to make blocks of just-the-kind of power splitter, etc, that I want.
Each of the processors (big or small!) participating in the control of these machines runs a *virtual dataflow environment.* These are a kind of operating system that simulates network operation between modular pieces of code. Codes are self-contained programs that interface to other codes via data inputs and outputs: we can think of these as virtual data ports.
The boards should all share a ground, but can run on different voltages of input power. Most will take 24v on two M3 screw-mounts - I use eye terminals soldered to 18ga wire to deliver power.
[Cuttlefish](https://gitlab.cba.mit.edu/squidworks/cuttlefish) is one such VDE, for the browser. It also serves visual representations of its own dataflow graphs, as well as dataflow across a network. This is the tool that allows us to see, interact with, build and edit distributed programs.
The router can also accept power from a USB device. If you're powering it over USB, *do not* also power it via 24v.
[Ponyo](https://gitlab.cba.mit.edu/squidworks/ponyo), the smallest fish (and queen of the sea), runs dataflow graphs on embedded microcontrollers. At the moment this is specific to the ATSAMD51J19, a 120MHz Arm M4F.
Last thing, don't power your supply on before you go to screw power connections onto the boards. Wire them up, and then switch on.
### Links, and the Common Serialization
### Network Cables
To nest virtual dataflow environments inside of networks (which are 'real' dataflow environments!) Squidworks develops a common serialization for known datatypes (i.e, in network byte-space, what does it mean to send a 64-bit float from port:0 in context:a to port:2 in context:b - etc), as well as common descriptors for graph configurations (based on the same serialization scheme).
At a bare minimum, you're going to be hooking these things up to power, and to each other (network).
Squidworks networks aren't picky about physical connections: these can be (and often are) USB links, RS-485 UARTs, and WebSockets, and could be (but are not yet) Bluetooth, LoRA, Laserbeams, Wiggled-Rope, Deep-Sea Cable, Interstellar Radio Link, etc - physical links. Anything that can ferry bytes from one context to another will do.
- network cables can be made two ways: only one is correct - *rj45 tabs should be on the same side of the ribbon cable* i.e. the cable is a 'straight through' type, not a crossover. this means that tx meets rx, etc.
- the transmit / receive ports are RS-485 Differential Driven, meaning there is no common gnd connection between boards besides the power bus.
## Circuits
The connectors I use are called 'RJ45' Jacks and Plugs. These are standard for Ethernet, but also 'generally useful'. This is not ethernet, but these *are* RJ45. It's 8 wires, and in our case that's four differential pairs - two duplex lines on each side.
At the end of the day, if we're still just talking about machine control, current moves through coils, voltages are read and delivered, etc. These machines aren't going to operate themselves (or, uh oh...) - so the squidworks project umbrella's real physical work on developing the hardware tools we can use to make them dance.
One cool thing about RJ45 is the modularity of the cables. We can use commodity crimping tools to make our own lengths:
- one side cuts, one side strips. use both at the same time to get the right length of stripped wire
- use the '8p' crimp, note the tab direction in the crimp
- pinch! the plug has a plastic tab inside that should come down to meet the wire jacket
| **The Router (1)** |
| --- |
| A message passing device, this thing hooks 6 of the ATSAMD51's SERCOM USARTS up to RS-485 Differential Driver and then hooks those up to RJ10 connectors (read: phone handset jacks). It runs [ponyo](https://gitlab.cba.mit.edu/squidworks/ponyo) and you can see the [schematic, board and documentation here](https://gitlab.cba.mit.edu/squidworks/routerboard-atsamd51). |
| ![router](images/2019-11-17_router.jpg) |
![rj45 video](images/rj45-assembly.mp4)
| **The Module Board** |
| --- |
| A do-what-you-will-with-it device, this thing breaks all of the ATSAMD51's pins out to fab-lab-friendly sized castellated pins, so that you can solder it to some-circuit-of-your-design. The thing is ready to run [ponyo](https://gitlab.cba.mit.edu/squidworks/ponyo), and invites you to, indeed, write some CPP and integrate some new devices onto the network that it will happily join over that RS-485 link. [Schematic, board and documentation here](https://gitlab.cba.mit.edu/squidworks/moduleboard-atsamd51). |
| ![module](images/2019-10-30_mw-module.jpg) |
```
make sure those tabs are on the same side of the flat cable
```
| **The Steppers (4)** |
| --- |
| A motor-turning device, this thing is one of the aforementioned module-boards, soldered to a heavy duty, motor-wrastling, no-amps-barred TMC262 stepper driver which *itself* slices and dices 24v of *power* with the help of four (4!) PN-Pair mosfets (that's two whole h-bridges baby) to drive (probably) NEMA17 stepper motors, to which these things will be attached when you get them. This also runs [ponyo](https://gitlab.cba.mit.edu/squidworks/ponyo) and you can see the [schematic, board and documentation here](https://gitlab.cba.mit.edu/squidworks/moduledaughter-stepper). |
| ![stepper](images/2019-11-18_stepper.jpg) |
![rj45](images/rj45-tabs.jpg)
| **Power Distribution Bits** |
| --- |
| These are tiny bus-bar type devices, that should make power routing a little bit easier. Included in the kit are (1) bypass capacitors for spare charge (these are actually important for the stepper motors to work properly), (2) TVS Diodes and Bleeders, (3) 5V Regulators (also necessary) and (4) routing blocks. These are all documented and explained in [this power distribution repo](https://gitlab.cba.mit.edu/squidworks/powerdistribution). |
| ![psu](images/2019-10-30_mw-psu.jpg) |
## Software
## What is the Status of the Squidworks Project?
Very in-flux, pls check back soon.
The project is relatively nascent, but ongoing, and we are excited to continue its development. Nothing is set in stone, but most components are annealing well. While the project proposes radical re-usability of code- and hardware resources, designing software to be appropriately general takes some careful thought, and as a result the software isn't yet appropriate for individuals who do not know how to, or do not want to learn how to program: you will still find that accomplishing most tasks requires your developing some custom 'hunks' (our modular code blocks). Stay tuned, and try it out now if you're feeling adventurous and curious.
## [Firmware -> XMEGAs](reproduction/firmware.md)
## How can I build a Squidworks Controller?
## [Circuit Reference](reproduction/circuits.md)
To start, you can download and run [cuttlefish](https://gitlab.cba.mit.edu/squidworks/cuttlefish), and then acquaint yourself with [ponyo](https://gitlab.cba.mit.edu/squidworks/ponyo). It's possible (though not tested) to build Ponyo for almost any embedded device (no part of 'ponyo' itself is tied to hardware directly), but if you're looking to start fast, we recommend [Adafruit's Feather M4](https://www.adafruit.com/product/3857), which is the board we developed it on. In fact, all of our circuits still think they are Feather M4s (we are still using their bootloader: thanks Adafruit!).
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment