early-mkxnotes.md 9.82 KB
Newer Older
Jake Read's avatar
Jake Read committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108

## What is MKXM?

A networking technology for very-fast asynchronous message passing.
A flexible, expanding set of hardware endpoints for said network.
An IDE & Interface for distributed computing and control.

## Why do we think we want this?

Hardware is complicated. Building automation systems is expensive, requires expertise, and prevents beginners designers and entrepreneurs from meaningfully participating in the manufacturing economy.

Ibid robotics.

## Why do we think this is the answer?

Modern datacenter (hardware and software) architectures are impressive models for the management of complex tasks. As robotic systems scale, we feel that a similar approach is warranted: computation and complexity can be added all modular-like, so that systems can grow and scale and change without complete re-engineering of requisite parts. In a distributed system, parts which comprise the whole are paired off piecewise: their individual complexities become tractable for beginners.

# Two Network Candidates

## Napoleon's Messenger / TinyNets

In [this implementation of a realtime network](https://gitlab.cba.mit.edu/jakeread/tinynets/tree/master), nodes use realtime, local information to make per-packet port forwarding decisions. This is a minimal state distance-vector routing protocol that allows a highly-connected network to adaptively multipath messages and find optimal paths without much overhead or any global state. As opposed to any existing multipath network technology (many are found in existing datacenter architectures), TinyNet does not require any node to have global information about network topology, and adapts to network loads in realtime. Best case re-routing protocol for existing multipath approaches incur ~200ms minimum re-routing interrupts to service.

Nodes are somewhat intelligent and have some state. Each has a unique address.

## APA

In an Asynchronous Packet Automata network, nodes have no address and are instead located by their network graph coordinates. Rather than containing a destination address, packets contain a list of ports (a route) and are forwarded along whichever port is next in the list.

APA is dirt simple (this is good)

## Transport

Both networks can be implemented with UART transport (likely candidate, bc speed, ubiquity) or with 'ATP' - asynchronous token passing, a kind of 'collaboratively clocked' serial line. Of interest in the development of the system is the development of a very-fast ATP protocol using FPGAs. While UART can run near 6MBPS with small amounts of processor overhead (a few clock cycles per byte), ATP requires some thinking on every single bit and is limited to 1/2 of the slowest participating node's ['ring speed'](https://pub.pages.cba.mit.edu/ring/). UART transport requires predetermined agreement between endpoints on the applied bitrate, which is a bummer.

## In Either Case

Network graphs can be reconstructed or discovered by sending 'flooding' packets, which return with graph (or address) information. Flooding is tricky as it requires some method for the identification of duplicate messages (which have travelled over different routes but originate and terminate at the same nodes).

# Events for Control

In either case (so far considered), endpoints have *inputs* and *outputs.* Systems are assembled by connecting outputs to inputs.

# Representation and Interface

The hardware networks presented use asynchronous event-based message passing. [Mods](mods.cba.mit.edu) uses the same, but passes messages between small javascript programs (rather than between embedded microcontrollers). The graph of these connections is displayed, and can be edited, by a human user of Mods (or by mods itself, what a trick!).

The MKXM project seeks to mesh these systems together: we want some software representation of hardware endpoints where if a user connects an 'output' of one to another's input, or to a 'mod' (a small javascript program), or from a 'mods' output to a software-representation-of-a-hardware-object, messages are then passed reliably as expected.

An example, using software mods (red) to send tuning and command values to a hardware node (black):

![whiteboard sketch](https://gitlab.cba.mit.edu/jakeread/mkxmods/raw/master/images/whiteboard-bldc.jpg)

This is largely complicated when we want to somehow remotely program hardware endpoints to send messages to eachother.

![whiteboard sketch](https://gitlab.cba.mit.edu/jakeread/mkxmods/raw/master/images/whiteboard-hardware-ambiguity.jpg)

The implementation of this system is the key design / engineering problem here, and may require some chooching of the network design.

## Passing Messages from Mods to a TinyNet System

Every hardware actor has an address, including the computer / mods node.

We have a hardware graph, discovered. Message to each hardware mod: "pls flood to your neighbours (1 hop) and report neighbours' address".

Hardwares 'serve' virtual graph objects during / after graph discovery. These graph objects display the hardware's advertized inputs and outputs.

When we connect a wire from one hardware output to another hardware input, mods (with global knowledge of the graph) sends a message to that hardware node: *please send this output event to this address.* This is a little bit non-ideal because we have to do something 'behind the scenes' (where mods sends a message to that node to update it's event-forwarding information). Some of the beauty of Mods, and something we want to carry forward, is that everything is exposed: we see all of the possible message routes, we see events fire, etc. The 'wires are cut' to begin with.

When we connect a software-mod output to a hardware input, we simply send that message to the address of the node we have plugged it in to. Only some software outputs are viable key:values.

When we connect a hardware output to a software-mod, we send a message to the hardware node: please send this output to my address.

All of these events are, well, event based. We can write script that has outputs firing on certain intervals, etc.

* critically, the 'mods' computer *does not* need to be the one sending 'system-config' type messages: i.e. pls send this message to this address. These are standard 'system' type messages and can originate anywhere. In this way we can have the self-editing graph where software can edit system: but physical transport happens god-knows-where, we just rely on Napoleon's Messenger.

## Passing Mesages from Mods to an APA System

Oddly, while TinyNet has more embedded state, it needs less embedded knowledge to successfully pass messages between nodes. Or, nodes need to find routes to get messages to where they want them to go. We can basically do the same thing above, but rather than telling embedded hardwares to send messages to a particular address, we tell them to send messages along a particular route (from their position). This is actually, maybe, quite simple. The only caveat is that, while we mentioned above that it would be easy for some hardware node to instruct another to send a message to some other address, with APA the instructing-mod would need to know enough about the graph to construct that route.

That said, we can almost consider APA a 'bare minimum transport-routing layer' and do some level-3 stuff wherever warranted - i.e. we can do a little bit of multipathing (although without packet-local information) if we see that it's worth it to gather enough information about the graph to do so.

# SO!

## Concluding (from here) and next:

While I think the Napoleon's Messenger thing is pretty cool, and probably fits some niche for embedded computing (where we have really highly connected graphs and want to sling messages really far), APA is so dirt simple (again, this is good) and probably can be made to be quite reliable. Its simplicity makes it a good candidate for one-day-maybe implementation on FPGAs (oooooh, aaaaah) and fancy things like multipathing can be written on top of it (including the realtime-decisions thing, if we add unique endpoint identification on top [addresses]).

So next step is to write (because my head is still in embedded programming land) reliable APA-type uart handlers (I hereby choose UART because of its simplicity in implementation and ubiquity and relative speed) - a small software library I can include in whatever embedded project I want to bring it to.

Then is to think carefully about how to bridge between Mods and Hardware, most of which I think I have a handle on, save for:
 - Flood / Graph discovery
 - How to properly terminate an APA packet (needs a length byte, I think)
 - How to implement flows (pieces of packets) in APA
 - Overcoming the multiple-message-delivery from flood messages without any unique message IDs?
 - How to deliver really long messages, like the json packages Will uses [here](https://gitlab.cba.mit.edu/langfordw/jsonMachineInterface) being a good candidate for how hardware mods can enumerate as javascript mods
 - How to tell hardware mods to pass message to other hardware mods

So, software architecture problems abound.

# Implementation

![Notes Here](https://gitlab.cba.mit.edu/jakeread/mkxmods/raw/master/apaapi.md)

# An Aside on Napoleon's Messenger

This is a nod to work by Manuel DeLanda, in *One Thousand Years of Nonlinear History* and *War in the Age of Intelligent Machines*. DeLanda recounts the histories of the first *global* wars (largely during the Napoleonic Wars) where for the first time, Armies were so large and complicated that ancient / medieval tactics involving the concentration of control into a singular point (say, one general with 10,000 soldiers) failed entirely - because message-delay times from the lowest levels of the battle (a solder) to the top (a general) were so large that control was impossible. Rather, armies that adopted Geurilla Warfare (where autonomy and decision making is passed to lower and lower sub-units) were incredibly successful - as they were able to adapt quickly to changing conditions, etc.