diff --git a/2023-12_usb-real-data-rates.md b/2023-12_usb-real-data-rates.md
index 2499f6ea0ebc15925b07b40c8622ca56e0e9b7e8..c9826ccf562c6fbec0cd87c664451880e284ff4e 100644
--- a/2023-12_usb-real-data-rates.md
+++ b/2023-12_usb-real-data-rates.md
@@ -4,25 +4,19 @@ I've been trying to sort out how to get enough data out of the NIST printer to g
To start, I spun up some max-rate-ingestion tests, on ubuntu and windows,
-### In Ubuntu (Lattitude 5490): 0.4 -> 0.5 Mbits/s
+
+
+
-
-
-
-
+So, we learn that there's no real change to the underlying data rate with increasing packet sizes, and we also learn that windows and ubuntu are not actually so different:
-### In Windows (XPS 15 2019): 0.6 -> 0.9 Mbits/s
+**Warning** the data in the table below are faulty (bad Mbits/s plotting and overclocked RP2040 reporting speedier deltas than were real), but they serve to show the relative performance:
-
-
-
-
+| <h3>Windows</h3> | <h3>Ubuntu</h3> |
+| --- | --- |
+|  |  |
-### Windows Subsystem for Linux (WSL)
-
-Another option - though I'm not sure how it will handle drivers etc - is to run this mf'er out of WSL - which I will try because I am having to install ubuntu anyways on a different machine to test on the real deal.
-
-That requires some [additional middleware](https://github.com/dorssel/usbipd-win) that looks stateful / PITA-ish, so I will just try on a separate machine; this is meant to be simple.
+Ubuntu packets are arriving a little more spaced out than in windows - but, also, the ubuntu machine (a Lattitude 5490) was under-spec'd relatively to the windows machine (XPS 15 2019).
### Surprisingly...
@@ -30,15 +24,19 @@ Windows out performs ubuntu here. Now, there is a big difference in machines as
We do also win some total bandwidth when we use bigger packet sizes, sort of as expected.
-But we have a clear cap near 1Mbit/sec in terms of real performance, which FWIW is 1/12th of the USB 2.0 Full-Speed-Device reported max, so there's clearly a lot missing here. Our data-printer spec asks for about 4Mbit/sec.
+But we have a clear cap near 0.5 Mbit/sec in terms of real performance, which FWIW is 1/24th of the USB 2.0 Full-Speed-Device reported max, so there's clearly a lot missing here. Our data-printer spec asks for about 4Mbit/sec, which is also worrying: we would need to have up to five of these devices to make that possible, and have them each working in parallel.
+
+### Other Externalities
This is also not accounting for i.e. multiple devices, flow control, flow going down (as well as up), etc. For that I will have to improve the system / etc.
+This also doesn't properly inspect whether / not there is significant performance dings due to i.e. cobs, which is [some looping python, anyways](https://github.com/cmcqueen/cobs-python/blob/main/src/cobs/cobs/_cobs_py.py) - so, maybe there is real evidence that we want to i.e. ethernet to the first thing, etc.
+
## 2023 12 20
So, we want to... consume and produce data, as fast as possible in either direction, and open multiple ports.
-I think I will get some devices w/ OLEDs on 'em, and then also do the due dilly of checking if I2C display writes are blocking or not?
+I think that the multiple ports thing is going to teach me what I want to know about asyncio, and is simple enough that I can try to replicate it with multiprocessing as well.
---
diff --git a/code/serial_multi_sink/cobs_usb_serial.py b/code/serial_multi_sink/cobs_usb_serial.py
new file mode 100644
index 0000000000000000000000000000000000000000..484d7454a445485cc3a26395fa728366d094a294
--- /dev/null
+++ b/code/serial_multi_sink/cobs_usb_serial.py
@@ -0,0 +1,17 @@
+from cobs import cobs
+import serial
+
+
+class CobsUsbSerial:
+ def __init__(self, port, baudrate=115200):
+ self.port = port
+ self.ser = serial.Serial(port, baudrate=baudrate, timeout=1)
+
+ def write(self, data: bytes):
+ data_enc = cobs.encode(data) + b"\x00"
+ self.ser.write(data_enc)
+
+ def read(self):
+ data_enc = self.ser.read_until(b"\x00")
+ data = cobs.decode(data_enc[:-1])
+ return data
diff --git a/code/serial_multi_sink/multi_sink.py b/code/serial_multi_sink/multi_sink.py
new file mode 100644
index 0000000000000000000000000000000000000000..0c666ab025cf51367e01484fea956fe7e8b7ef80
--- /dev/null
+++ b/code/serial_multi_sink/multi_sink.py
@@ -0,0 +1,88 @@
+from cobs_usb_serial import CobsUsbSerial
+import asyncio
+import struct
+import numpy as np
+import pandas as pd
+import matplotlib.pyplot as plt
+
+pck_len = 32
+
+async def read_ser(cobs_serial, rx_func):
+ while True:
+ data = cobs_serial.read()
+ if len(data) == pck_len:
+ stamp = struct.unpack('=I', data[:4])
+ rx_func(stamp)
+ await asyncio.sleep(0)
+
+def port_handler(stamp):
+ print(stamp)
+
+async def main():
+ port_one = CobsUsbSerial("COM23")
+
+ task_one = asyncio.create_task(read_ser(port_one, port_handler))
+
+ await asyncio.gather(task_one)
+
+asyncio.run(main())
+
+# from cobs_usb_serial import CobsUsbSerial
+# import struct
+# import numpy as np
+# import pandas as pd
+# import matplotlib.pyplot as plt
+
+# ser = CobsUsbSerial("COM23")
+
+# stamp_count = 1000
+# pck_len = 250
+
+# stamps = np.zeros(stamp_count)
+
+# for i in range(stamp_count):
+# bts = ser.read()
+# if len(bts) == pck_len:
+# stamp = struct.unpack('=I', bts[:4])
+# stamps[i] = stamp[0]
+
+# print("stamps, ", stamps)
+
+# df = pd.DataFrame({'timestamps': stamps})
+
+# df['deltas'] = df['timestamps'].diff()
+
+# # clean NaN's
+# df = df.dropna()
+
+# # wipe obviously-wrong deltas (i.e. the 1st, which goes 0-start-us)
+# df = df[df['deltas'] < 100000]
+
+# # Plotting
+# fig, ax1 = plt.subplots(figsize=(11, 5))
+
+# # Primary x-axis (time deltas)
+# df['deltas'].plot(kind='hist', bins=100, ax=ax1)
+# ax1.set_xlabel('Time-Stamp Deltas (us)')
+# ax1.set_ylabel(f'Frequency (of {stamp_count})')
+
+# # Secondary x-axis (bandwidth)
+# ax2 = ax1.twiny()
+# ax2.set_xlabel('Estimated Bandwidth (Mbits/s)')
+
+# # Set the limits of the secondary axis based on the primary axis
+# # new_tick_locations = np.linspace(df['deltas'].min(), df['deltas'].max(), num=len(ax1.get_xticks()))
+
+# # Convert tick locations to bandwidth
+# # bandwidths = [(pck_len * 8) * (1e6 / x) for x in new_tick_locations]
+
+# x_ticks = ax1.get_xticks()
+# bandwidth_ticks = [((pck_len * 8) * (1e6 / x)) / 1e6 for x in x_ticks]
+
+# ax2.set_xlim(max(bandwidth_ticks), min(bandwidth_ticks))
+
+# plt.title(f'Single-Source COBS Data Sink Deltas, pck_len={pck_len}')
+
+# plt.tight_layout()
+
+# plt.show()
diff --git a/code/serial_multi_sink/serial_list.py b/code/serial_multi_sink/serial_list.py
new file mode 100644
index 0000000000000000000000000000000000000000..5f25377227f77c79dc2db925a16f5a5a280f412b
--- /dev/null
+++ b/code/serial_multi_sink/serial_list.py
@@ -0,0 +1,20 @@
+import serial.tools.list_ports
+
+def list_serial_ports():
+ ports = serial.tools.list_ports.comports()
+ for port in ports:
+ print(f"Port: {port.device}")
+ print(f" - Description: {port.description}")
+ if port.serial_number:
+ print(f" - Serial Number: {port.serial_number}")
+ if port.manufacturer:
+ print(f" - Manufacturer: {port.manufacturer}")
+ if port.product:
+ print(f" - Product: {port.product}")
+ if port.vid is not None:
+ print(f" - VID: {port.vid:04X}")
+ if port.pid is not None:
+ print(f" - PID: {port.pid:04X}")
+ print()
+
+list_serial_ports()
diff --git a/code/serial_sink/sink.py b/code/serial_sink/sink.py
index ebd0aabb4c2dbeedb7ffefdffef813c7d431c101..0506d945e86ef521051498128fb3fda9c275a3fb 100644
--- a/code/serial_sink/sink.py
+++ b/code/serial_sink/sink.py
@@ -4,10 +4,10 @@ import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
-ser = CobsUsbSerial("/dev/ttyACM0")
+ser = CobsUsbSerial("COM23")
-stamp_count = 10000
-pck_len = 32
+stamp_count = 1000
+pck_len = 250
stamps = np.zeros(stamp_count)
@@ -29,10 +29,6 @@ df = df.dropna()
# wipe obviously-wrong deltas (i.e. the 1st, which goes 0-start-us)
df = df[df['deltas'] < 100000]
-# Bandwidth calculation (bytes/us to Mbits/s)
-# 1 byte/us = 8 Mbits/s
-df['bandwidth'] = (pck_len / df['deltas']) * 8 * 1e3 # Convert to Mbits/s
-
# Plotting
fig, ax1 = plt.subplots(figsize=(11, 5))
@@ -46,10 +42,15 @@ ax2 = ax1.twiny()
ax2.set_xlabel('Estimated Bandwidth (Mbits/s)')
# Set the limits of the secondary axis based on the primary axis
-new_tick_locations = np.linspace(df['deltas'].min(), df['deltas'].max(), num=len(ax1.get_xticks()))
+# new_tick_locations = np.linspace(df['deltas'].min(), df['deltas'].max(), num=len(ax1.get_xticks()))
# Convert tick locations to bandwidth
-# ax2.set_xlim([pck_len / x * 8 * 1e3 for x in new_tick_locations])
+# bandwidths = [(pck_len * 8) * (1e6 / x) for x in new_tick_locations]
+
+x_ticks = ax1.get_xticks()
+bandwidth_ticks = [((pck_len * 8) * (1e6 / x)) / 1e6 for x in x_ticks]
+
+ax2.set_xlim(max(bandwidth_ticks), min(bandwidth_ticks))
plt.title(f'Single-Source COBS Data Sink Deltas, pck_len={pck_len}')
diff --git a/code/serial_source/serial_source.ino b/code/serial_source/serial_source.ino
index cce655d133d2f97123210dc5a5d6ec4623829f2f..610940615df3a2decc89f341a90fe774b12ec599 100644
--- a/code/serial_source/serial_source.ino
+++ b/code/serial_source/serial_source.ino
@@ -40,7 +40,7 @@ void loop() {
// tx a stamp AFAP
if(cobs.clearToSend()){
chunk.u = micros();
- cobs.send(chunk.bytes, 32);
+ cobs.send(chunk.bytes, 250);
digitalWrite(PIN_LED_G, !digitalRead(PIN_LED_G));
}
// blink to see hangups
diff --git a/images/2023-12-20_ingest-histogram-single-source-pck-128-133mhz.png b/images/2023-12-20_ingest-histogram-single-source-pck-128-133mhz.png
new file mode 100644
index 0000000000000000000000000000000000000000..ff120001d89dbea771ceb237d5874a98b249b38d
Binary files /dev/null and b/images/2023-12-20_ingest-histogram-single-source-pck-128-133mhz.png differ
diff --git a/images/2023-12-20_ingest-histogram-single-source-pck-128.png b/images/2023-12-20_ingest-histogram-single-source-pck-128.png
new file mode 100644
index 0000000000000000000000000000000000000000..860c803ec09c9329fa83f7711849d08b5c951d42
Binary files /dev/null and b/images/2023-12-20_ingest-histogram-single-source-pck-128.png differ
diff --git a/images/2023-12-20_ingest-histogram-single-source-pck-250.png b/images/2023-12-20_ingest-histogram-single-source-pck-250.png
new file mode 100644
index 0000000000000000000000000000000000000000..76231c1d8be3be80dd4f96cadf41bb306086e3c8
Binary files /dev/null and b/images/2023-12-20_ingest-histogram-single-source-pck-250.png differ
diff --git a/images/2023-12-20_ingest-histogram-single-source-pck-32.png b/images/2023-12-20_ingest-histogram-single-source-pck-32.png
new file mode 100644
index 0000000000000000000000000000000000000000..5b8718a0c2f5bb750f6cdf88af64e701709920df
Binary files /dev/null and b/images/2023-12-20_ingest-histogram-single-source-pck-32.png differ