diff --git a/README.md b/README.md index bca73d0ae0e80ff3f8c2eb1414680ebd4235cdd6..daec7d95fb1489217ce86e1e6d4418279a77787d 100644 --- a/README.md +++ b/README.md @@ -1,32 +1,44 @@ -## SPI +# Low Level Link Speed Tests -- actually-realized data rates are often smaller in practice than according to underlying specs, - - see even i.e. transfer of some tens-of-megabytes onto an SD card (copying a gcode file) - we observe only ~ 20mb/sec of advertized 80mb/sec or so -- otherwise, you have notes on what-the-point is in your `scratch` -- and consider linking to the ring-test page -- and discuss that at the embedded limit, we often run into interrupt-request-handling-time troubles before anything else... +For a more conversational write-up, see [the blog post.](https://ekswhyzee.com/2024/01/04/link-layer-speed-tests.html) -## TODO +TL:DR; we want to find out actually how-fast links are between embedded systems and high-level systems, mostly for machine building, but useful across the board. In quick tests, I found that most links under-perform their apparent rates by more than half. -- test the D51 for device-to-device uart -- likewise for RPI (five?) to device SPI link +### USB + + +> RP2040, 0.45Mbit/s + + +> Teensy 4.0, 20Mbit/s + +### WizNet Ethernet + + + +### SPI to PI + + --- -## Next - -- the SPI can work, I suspect - - probably needs that sweet sweet 250MHz-es - - can crank up to 16-bit words in the FIFO, for 2x less starvation - - can PIO do 32-bit words, perchance ? - - fire the interrupt on transmit buffer not-full, rather than half-full ? - - actually no interrupt exists for this - - can we do a proper echo test, then check for drops, etc, for max speed ? -- do the RP2040-to-RP2040 UART PIO test -- finish the thought, blog it up (1 day, max) -- take i.e. [notes](https://abyz.me.uk/rpi/pigpio/pigs.html), right ? - - incl. the maybe-likely-change to use PIO spi-slave, not this bs -- i.e. go-notes/ref/rp2040 ? - - https://cec-code-lab.aps.edu/robotics/resources/pico-c-api/index.html - - https://github.com/raspberrypi/pico-examples/tree/master - - https://www.circuitstate.com/tutorials/making-two-raspberry-pi-pico-boards-communicate-through-spi-using-c-cpp-sdk/ \ No newline at end of file +# TODO Here + +I am electing microcontrollers for the position of routers-et-cetera, and the RP2040 was a top candidate but has me spooked after these tests. I want to do some more testing: + +## Better Tests + +I want to use all of these links in the context of my link-agnostic transport layer, which currently uses shoddy ad-hoc links. I would like to have some generic wrapper code that would bundle questionable links (like these) and do i.e. error catching and test-suite-implementation in a more regular manner. It could presumably be built with a kind of simple HAL to put and to get packets. + +## Testing other Systems + +- test the D51 for device-to-device uart +- the D51 and RPi 5 for big-cpu-to-small-mcu SPI test redux +- consider the Teensy for the router, that 600MHz and advertised-to-be-small ISR seems juicy +- Teensy USB might be blazing fast to boot, Paul seems like the Real Deal (TM) and Teensy 4.0 page mentions 480Mbit/s USB... + +## Improving Existing Tests + +- the RP2040's PIO *aught* to be killer at this, so it is likely that I simply haven't really cracked it. for example, each has an 8-space wide FIFO, each of which is 32 bits long, this means one interrupt should be able to switch a 32-byte long packet (in SPI-mode) - UART (it looks like) can go to 16 bit words +- we also haven't really implemented interrupts on the 2040 properly, far as I can tell: we should be able to get one latched whenever a fifo is *NOT FULL* rather than being half-full, etc, which would help prevent some chunking (earlier warning) + \ No newline at end of file diff --git a/usb_serial/code/serial_sink/plot_stamps.py b/usb_serial/code/serial_sink/plot_stamps.py new file mode 100644 index 0000000000000000000000000000000000000000..1451ba027adc5c74381153922e50da2a13a9b92f --- /dev/null +++ b/usb_serial/code/serial_sink/plot_stamps.py @@ -0,0 +1,44 @@ +import pandas as pd +import matplotlib.pyplot as plt + +def plot_stamps(stamps, stamp_count, pck_len, misses): + # make df from stamps + df = pd.DataFrame({'timestamps': stamps}) + + # calculate deltas between stamps + df['deltas'] = df['timestamps'].diff() + + # clean NaN's + df = df.dropna() + + # wipe obviously-wrong deltas (i.e. the 1st, which goes 0-start-us) + df = df[df['deltas'] < 10000] + + # Plotting + fig, ax1 = plt.subplots(figsize=(11, 3)) + + # ax1.set_xlim([0, 100]) + + # Primary x-axis (time deltas) + df['deltas'].plot(kind='hist', bins=10, ax=ax1) + ax1.set_xlabel('Time-Stamp Deltas (us) and equivalent (MBits/s)') + ax1.set_ylabel(f'Frequency (of {stamp_count})') + + # get axis ticks to calculate equivalent bandwidths + x_ticks = ax1.get_xticks() + # x_ticks = [100, 200, 300, 400, 500, 600, 700, 800, 900] + # ax1.set_xticks(x_ticks) + bandwidths = [((pck_len * 8) * (1e6 / x)) / 1e6 for x in x_ticks] + ticks = [] + + for i in range(len(x_ticks)): + print(i, x_ticks[i], bandwidths[i]) + ticks.append(f"{x_ticks[i]:.0f} ({bandwidths[i]:.3f})") + + ax1.set_xticklabels(ticks) + + plt.title(f'Single-Source COBS Data Sink Deltas, pck_len={pck_len}, miss={misses}') + + plt.tight_layout() + + plt.show() \ No newline at end of file diff --git a/usb_serial/code/serial_sink/sink.py b/usb_serial/code/serial_sink/sink.py index 0506d945e86ef521051498128fb3fda9c275a3fb..e2dc9ba5f39f97de1c09e4f706b242a5801e29eb 100644 --- a/usb_serial/code/serial_sink/sink.py +++ b/usb_serial/code/serial_sink/sink.py @@ -1,13 +1,12 @@ from cobs_usb_serial import CobsUsbSerial +from plot_stamps import plot_stamps import struct import numpy as np -import pandas as pd -import matplotlib.pyplot as plt -ser = CobsUsbSerial("COM23") +ser = CobsUsbSerial("COM43") stamp_count = 1000 -pck_len = 250 +pck_len = 128 stamps = np.zeros(stamp_count) @@ -19,41 +18,4 @@ for i in range(stamp_count): print("stamps, ", stamps) -df = pd.DataFrame({'timestamps': stamps}) - -df['deltas'] = df['timestamps'].diff() - -# clean NaN's -df = df.dropna() - -# wipe obviously-wrong deltas (i.e. the 1st, which goes 0-start-us) -df = df[df['deltas'] < 100000] - -# Plotting -fig, ax1 = plt.subplots(figsize=(11, 5)) - -# Primary x-axis (time deltas) -df['deltas'].plot(kind='hist', bins=100, ax=ax1) -ax1.set_xlabel('Time-Stamp Deltas (us)') -ax1.set_ylabel(f'Frequency (of {stamp_count})') - -# Secondary x-axis (bandwidth) -ax2 = ax1.twiny() -ax2.set_xlabel('Estimated Bandwidth (Mbits/s)') - -# Set the limits of the secondary axis based on the primary axis -# new_tick_locations = np.linspace(df['deltas'].min(), df['deltas'].max(), num=len(ax1.get_xticks())) - -# Convert tick locations to bandwidth -# bandwidths = [(pck_len * 8) * (1e6 / x) for x in new_tick_locations] - -x_ticks = ax1.get_xticks() -bandwidth_ticks = [((pck_len * 8) * (1e6 / x)) / 1e6 for x in x_ticks] - -ax2.set_xlim(max(bandwidth_ticks), min(bandwidth_ticks)) - -plt.title(f'Single-Source COBS Data Sink Deltas, pck_len={pck_len}') - -plt.tight_layout() - -plt.show() +plot_stamps(stamps, stamp_count, pck_len, 0) \ No newline at end of file diff --git a/usb_serial/code/serial_source_teensy/.gitignore b/usb_serial/code/serial_source_teensy/.gitignore new file mode 100644 index 0000000000000000000000000000000000000000..89cc49cbd652508924b868ea609fa8f6b758ec56 --- /dev/null +++ b/usb_serial/code/serial_source_teensy/.gitignore @@ -0,0 +1,5 @@ +.pio +.vscode/.browse.c_cpp.db* +.vscode/c_cpp_properties.json +.vscode/launch.json +.vscode/ipch diff --git a/usb_serial/code/serial_source_teensy/include/README b/usb_serial/code/serial_source_teensy/include/README new file mode 100644 index 0000000000000000000000000000000000000000..194dcd43252dcbeb2044ee38510415041a0e7b47 --- /dev/null +++ b/usb_serial/code/serial_source_teensy/include/README @@ -0,0 +1,39 @@ + +This directory is intended for project header files. + +A header file is a file containing C declarations and macro definitions +to be shared between several project source files. You request the use of a +header file in your project source file (C, C++, etc) located in `src` folder +by including it, with the C preprocessing directive `#include'. + +```src/main.c + +#include "header.h" + +int main (void) +{ + ... +} +``` + +Including a header file produces the same results as copying the header file +into each source file that needs it. Such copying would be time-consuming +and error-prone. With a header file, the related declarations appear +in only one place. If they need to be changed, they can be changed in one +place, and programs that include the header file will automatically use the +new version when next recompiled. The header file eliminates the labor of +finding and changing all the copies as well as the risk that a failure to +find one copy will result in inconsistencies within a program. + +In C, the usual convention is to give header files names that end with `.h'. +It is most portable to use only letters, digits, dashes, and underscores in +header file names, and at most one dot. + +Read more about using header files in official GCC documentation: + +* Include Syntax +* Include Operation +* Once-Only Headers +* Computed Includes + +https://gcc.gnu.org/onlinedocs/cpp/Header-Files.html diff --git a/usb_serial/code/serial_source_teensy/lib/README b/usb_serial/code/serial_source_teensy/lib/README new file mode 100644 index 0000000000000000000000000000000000000000..6debab1e8b4c3faa0d06f4ff44bce343ce2cdcbf --- /dev/null +++ b/usb_serial/code/serial_source_teensy/lib/README @@ -0,0 +1,46 @@ + +This directory is intended for project specific (private) libraries. +PlatformIO will compile them to static libraries and link into executable file. + +The source code of each library should be placed in a an own separate directory +("lib/your_library_name/[here are source files]"). + +For example, see a structure of the following two libraries `Foo` and `Bar`: + +|--lib +| | +| |--Bar +| | |--docs +| | |--examples +| | |--src +| | |- Bar.c +| | |- Bar.h +| | |- library.json (optional, custom build options, etc) https://docs.platformio.org/page/librarymanager/config.html +| | +| |--Foo +| | |- Foo.c +| | |- Foo.h +| | +| |- README --> THIS FILE +| +|- platformio.ini +|--src + |- main.c + +and a contents of `src/main.c`: +``` +#include <Foo.h> +#include <Bar.h> + +int main (void) +{ + ... +} + +``` + +PlatformIO Library Dependency Finder will find automatically dependent +libraries scanning project source files. + +More information about PlatformIO Library Dependency Finder +- https://docs.platformio.org/page/librarymanager/ldf.html diff --git a/usb_serial/code/serial_source_teensy/platformio.ini b/usb_serial/code/serial_source_teensy/platformio.ini new file mode 100644 index 0000000000000000000000000000000000000000..4adb7a6978d26faf2631f9cc8c4a0de7fe838813 --- /dev/null +++ b/usb_serial/code/serial_source_teensy/platformio.ini @@ -0,0 +1,14 @@ +; PlatformIO Project Configuration File +; +; Build options: build flags, source filter +; Upload options: custom upload port, speed and extra flags +; Library options: dependencies, extra library storages +; Advanced options: extra scripting +; +; Please visit documentation for the other options and examples +; https://docs.platformio.org/page/projectconf.html + +[env:teensy40] +platform = teensy +board = teensy40 +framework = arduino diff --git a/usb_serial/code/serial_source_teensy/src/COBSUSBSerial.cpp b/usb_serial/code/serial_source_teensy/src/COBSUSBSerial.cpp new file mode 100644 index 0000000000000000000000000000000000000000..a601582ad1b5702114d0acde6b2eae2da22d3ab7 --- /dev/null +++ b/usb_serial/code/serial_source_teensy/src/COBSUSBSerial.cpp @@ -0,0 +1,96 @@ +// link ! +// TODO: pls use nanocobs, for encode- and decode-in-place, to save big (!) memory +// on new link layer... to fit into D11s... + +#include "COBSUSBSerial.h" +#include "cobs.h" + + +#if defined(ARDUINO_ARCH_RP2040) || defined(ARDUINO_ARCH_RP2040) +COBSUSBSerial::COBSUSBSerial(SerialUSB* _usbcdc){ + usbcdc = _usbcdc; +} +#elif defined(ARDUINO_TEENSY41) || defined(ARDUINO_TEENSY40) +COBSUSBSerial::COBSUSBSerial(usb_serial_class* _usbcdc){ + usbcdc = _usbcdc; +} +#else +COBSUSBSerial::COBSUSBSerial(Serial_* _usbcdc){ + usbcdc = _usbcdc; +} +#endif + +void COBSUSBSerial::begin(void){ + usbcdc->begin(9600); +} + +void COBSUSBSerial::loop(void){ + // check RX side: + // while data & not-full, + while(usbcdc->available() && rxBufferLen == 0){ + rxBuffer[rxBufferWp ++] = usbcdc->read(); + if(rxBuffer[rxBufferWp - 1] == 0){ + // decoding in place should always work: COBS doesn't revisit bytes + // encoding in place would be a different trick, and would require the use of + // nanocobs from this lad: https://github.com/charlesnicholson/nanocobs + size_t len = cobsDecode(rxBuffer, 255, rxBuffer); + // now we are with-packet, set length and reset write pointer + // len includes the trailing 0, rm that... + rxBufferLen = len - 1; + rxBufferWp = 0; + } + } + + // check tx side, + while(txBufferLen && usbcdc->availableForWrite()){ + // ship a byte, + usbcdc->write(txBuffer[txBufferRp ++]); + // if done, mark empty + if(txBufferRp >= txBufferLen){ + txBufferLen = 0; + txBufferRp = 0; + } + } +} + +size_t COBSUSBSerial::getPacket(uint8_t* dest){ + if(rxBufferLen > 0){ + memcpy(dest, rxBuffer, rxBufferLen); + size_t len = rxBufferLen; + rxBufferLen = 0; + return len; + } else { + return 0; + } +} + +boolean COBSUSBSerial::clearToRead(void){ + return (rxBufferLen > 0); +} + +void COBSUSBSerial::send(uint8_t* packet, size_t len){ + // we have a max: we need to stuff into 255, + // and we have a trailing zero and the first key + if(len > 253) len = 253; + // ship that, + size_t encodedLen = cobsEncode(packet, len, txBuffer); + // stuff 0 byte, + txBuffer[encodedLen] = 0; + txBufferLen = encodedLen + 1; + txBufferRp = 0; +} + +boolean COBSUSBSerial::clearToSend(void){ + // we're CTS if we have nothing in the outbuffer, + return (txBufferLen == 0); +} + +// we should do some... work with this, i.e. +// keepalives, to detect if other-end is open or not... +boolean COBSUSBSerial::isOpen(void){ + if(usbcdc){ + return true; + } else { + return false; + } +} \ No newline at end of file diff --git a/usb_serial/code/serial_source_teensy/src/COBSUSBSerial.h b/usb_serial/code/serial_source_teensy/src/COBSUSBSerial.h new file mode 100644 index 0000000000000000000000000000000000000000..9b5fac1c8e65184b5b5a4b26a9e0c923f8792d35 --- /dev/null +++ b/usb_serial/code/serial_source_teensy/src/COBSUSBSerial.h @@ -0,0 +1,41 @@ +// example cobs-encoded usb-serial link + +#include <Arduino.h> + +class COBSUSBSerial { + public: + #if defined(ARDUINO_ARCH_RP2040) || defined(ARDUINO_ARCH_RP2040) + COBSUSBSerial(SerialUSB* _usbcdc); + #elif defined(ARDUINO_TEENSY41) || defined(ARDUINO_TEENSY40) + COBSUSBSerial(usb_serial_class* _usbcdc); + #else + COBSUSBSerial(Serial_* _usbcdc); + #endif + void begin(void); + void loop(void); + // check & read, + boolean clearToRead(void); + size_t getPacket(uint8_t* dest); + // clear ahead? + boolean clearToSend(void); + // open at all? + boolean isOpen(void); + // transmit a packet of this length + void send(uint8_t* packet, size_t len); + private: + #if defined(ARDUINO_ARCH_RP2040) || defined(ARDUINO_ARCH_RP2040) + SerialUSB* usbcdc = nullptr; + #elif defined(ARDUINO_TEENSY41) || defined(ARDUINO_TEENSY40) + usb_serial_class* usbcdc = nullptr; + #else + Serial_* usbcdc = nullptr; + #endif + // buffer, write pointer, length, + uint8_t rxBuffer[255]; + uint8_t rxBufferWp = 0; + uint8_t rxBufferLen = 0; + // ibid, + uint8_t txBuffer[255]; + uint8_t txBufferRp = 0; + uint8_t txBufferLen = 0; +}; \ No newline at end of file diff --git a/usb_serial/code/serial_source_teensy/src/cobs.cpp b/usb_serial/code/serial_source_teensy/src/cobs.cpp new file mode 100644 index 0000000000000000000000000000000000000000..9f243ad4a2618360ac2acd8ee80a4f48d023513b --- /dev/null +++ b/usb_serial/code/serial_source_teensy/src/cobs.cpp @@ -0,0 +1,72 @@ +/* +utils/cobs.cpp + +Jake Read at the Center for Bits and Atoms +(c) Massachusetts Institute of Technology 2019 + +This work may be reproduced, modified, distributed, performed, and +displayed for any purpose, but must acknowledge the osap project. +Copyright is retained and must be preserved. The work is provided as is; +no warranty is provided, and users accept all liability. +*/ + +#include "cobs.h" +// str8 crib from +// https://en.wikipedia.org/wiki/Consistent_Overhead_Byte_Stuffing + +/** COBS encode data to buffer + @param data Pointer to input data to encode + @param length Number of bytes to encode + @param buffer Pointer to encoded output buffer + @return Encoded buffer length in bytes + @note doesn't write stop delimiter +*/ + +size_t cobsEncode(const void *data, size_t length, uint8_t *buffer){ + + uint8_t *encode = buffer; // Encoded byte pointer + uint8_t *codep = encode++; // Output code pointer + uint8_t code = 1; // Code value + + for (const uint8_t *byte = (const uint8_t *)data; length--; ++byte){ + if (*byte) // Byte not zero, write it + *encode++ = *byte, ++code; + + if (!*byte || code == 0xff){ // Input is zero or block completed, restart + *codep = code, code = 1, codep = encode; + if (!*byte || length) + ++encode; + } + } + *codep = code; // Write final code value + return encode - buffer; +} + +/** COBS decode data from buffer + @param buffer Pointer to encoded input bytes + @param length Number of bytes to decode + @param data Pointer to decoded output data + @return Number of bytes successfully decoded + @note Stops decoding if delimiter byte is found +*/ + +size_t cobsDecode(const uint8_t *buffer, size_t length, void *data){ + + const uint8_t *byte = buffer; // Encoded input byte pointer + uint8_t *decode = (uint8_t *)data; // Decoded output byte pointer + + for (uint8_t code = 0xff, block = 0; byte < buffer + length; --block){ + if (block) // Decode block byte + *decode++ = *byte++; + else + { + if (code != 0xff) // Encoded zero, write it + *decode++ = 0; + block = code = *byte++; // Next block length + if (code == 0x00) // Delimiter code found + break; + } + } + + return decode - (uint8_t *)data; +} diff --git a/usb_serial/code/serial_source_teensy/src/cobs.h b/usb_serial/code/serial_source_teensy/src/cobs.h new file mode 100644 index 0000000000000000000000000000000000000000..b47070ca26d021f113da680a6835df65712d4007 --- /dev/null +++ b/usb_serial/code/serial_source_teensy/src/cobs.h @@ -0,0 +1,24 @@ +/* +utils/cobs.h + +consistent overhead byte stuffing implementation + +Jake Read at the Center for Bits and Atoms +(c) Massachusetts Institute of Technology 2019 + +This work may be reproduced, modified, distributed, performed, and +displayed for any purpose, but must acknowledge the osap project. +Copyright is retained and must be preserved. The work is provided as is; +no warranty is provided, and users accept all liability. +*/ + +#ifndef UTIL_COBS_H_ +#define UTIL_COBS_H_ + +#include <Arduino.h> + +size_t cobsEncode(const void *data, size_t length, uint8_t *buffer); + +size_t cobsDecode(const uint8_t *buffer, size_t length, void *data); + +#endif diff --git a/usb_serial/code/serial_source_teensy/src/main.cpp b/usb_serial/code/serial_source_teensy/src/main.cpp new file mode 100644 index 0000000000000000000000000000000000000000..b16f30ed42ad3d5c3f8629577796fc4fc9fe34b0 --- /dev/null +++ b/usb_serial/code/serial_source_teensy/src/main.cpp @@ -0,0 +1,36 @@ +#include <Arduino.h> +#include "COBSUSBSerial.h" + +#define PIN_LED 13 + +COBSUSBSerial cobs(&Serial); + +void setup() { + cobs.begin(); + pinMode(PIN_LED, OUTPUT); + digitalWrite(PIN_LED, HIGH); +} + +union chunk_uint32 { + uint8_t bytes[4]; + uint32_t u; +}; + +chunk_uint32 chunk; + +uint32_t lastBlink = 0; + +void loop() { + // run comms + cobs.loop(); + // tx a stamp AFAP + if(cobs.clearToSend()){ + chunk.u = micros(); + cobs.send(chunk.bytes, 128); + } + // blink to see hangups + if(lastBlink + 100 < millis()){ + lastBlink = millis(); + digitalWrite(PIN_LED, !digitalRead(PIN_LED)); + } +} diff --git a/usb_serial/code/serial_source_teensy/test/README b/usb_serial/code/serial_source_teensy/test/README new file mode 100644 index 0000000000000000000000000000000000000000..9b1e87bc67c90e7f09a92a3e855444b085c655a6 --- /dev/null +++ b/usb_serial/code/serial_source_teensy/test/README @@ -0,0 +1,11 @@ + +This directory is intended for PlatformIO Test Runner and project tests. + +Unit Testing is a software testing method by which individual units of +source code, sets of one or more MCU program modules together with associated +control data, usage procedures, and operating procedures, are tested to +determine whether they are fit for use. Unit testing finds problems early +in the development cycle. + +More information about PlatformIO Unit Testing: +- https://docs.platformio.org/en/latest/advanced/unit-testing/index.html diff --git a/usb_serial/images/2024-01-04_teensy-usb-128.png b/usb_serial/images/2024-01-04_teensy-usb-128.png new file mode 100644 index 0000000000000000000000000000000000000000..c9a85df47dc70c682d3e1e47130022a400799a83 Binary files /dev/null and b/usb_serial/images/2024-01-04_teensy-usb-128.png differ diff --git a/usb_serial/images/2024-01-04_teensy-usb-32.png b/usb_serial/images/2024-01-04_teensy-usb-32.png new file mode 100644 index 0000000000000000000000000000000000000000..4121851f97f9bde28b5c65d6766a4c562a8b09ef Binary files /dev/null and b/usb_serial/images/2024-01-04_teensy-usb-32.png differ diff --git a/wiznet_ethernet/images/2023-12-28_ethernet-wiz-spi-01.jpg b/wiznet_ethernet/images/2023-12-28_ethernet-wiz-spi-01.jpg index 8b75f87edd305304bccfd09ed61d04369e5b24b3..9bd9244ea560cfb988d9386208839cac6363605c 100644 Binary files a/wiznet_ethernet/images/2023-12-28_ethernet-wiz-spi-01.jpg and b/wiznet_ethernet/images/2023-12-28_ethernet-wiz-spi-01.jpg differ diff --git a/wiznet_ethernet/images/2023-12-28_ethernet-wiz-spi-02.jpg b/wiznet_ethernet/images/2023-12-28_ethernet-wiz-spi-02.jpg index 855bd3a73ac8eb47f9821d43f205b6cbd8639469..2be0d3c54f3ce675a0c69ca300d8edad7100d3b7 100644 Binary files a/wiznet_ethernet/images/2023-12-28_ethernet-wiz-spi-02.jpg and b/wiznet_ethernet/images/2023-12-28_ethernet-wiz-spi-02.jpg differ