confounding ethernet results

2023-12_ethernet-real-data-rates.md

@@ -16,6 +16,10 @@ There is also some confusion about [max frame sizes](https://forum.arduino.cc/t/
 
 OK, this is looking alive-ish? I should see about getting a UDP ping with Python and then I can record the setup / etc. 
 
+### Ethernet Hookup Guide 
+
+- Laptop and Device are booth hooked up to a small switch 
+- Switch is *not* hooked to any other higher-level-internet-devices.
 - SET Ethernet Interface (in laptop OS) to use fixed IP,
   - i.e. I am using 192.168.1.178
 - SET the same to use Subnet Mask 255.255.255.0 
@@ -23,7 +27,50 @@ OK, this is looking alive-ish? I should see about getting a UDP ping with Python
 
 OK, now I can ping a message down and up, god bless... speed testy time. 
 
-### Polling Speed Test
+### Ping-Polling Speed Test
 
 AFAIK, UDP can source and sink packets at either end of the pipe - it's just a straight IP datagram. It seems like the normal pattern is to use it "transactionally" - i.e. send-req-get-res, and I will need to bootstrap delivery guarantees, etc. I suspect that I will ultimately want TCP here to get those tasty delivery guarantees etc.
 
+So, ping times look good: centered around 500us, but this initial packet exchange is mad small:
+
+![hello-eth](images/2023-12-28_eth-begin.png)
+
+Let's try it with some increasing size packets:
+
+![64](images/2023-12-28_eth-ping-64.png)
+![512](images/2023-12-28_eth-ping-256.png)
+![512](images/2023-12-28_eth-ping-512.png)
+![512](images/2023-12-28_eth-ping-768.png)
+
+This code seems to fail at 1024 bytes per packet, and **keep in mind that these are echo times now** - not the same as previous tests. This is a single packet down, a flip, and a packet back up. 
+
+So, I want to see, for 64 bytes, what the turn-around time is in embedded (I'll watch the CS line and flip a debug pin as well), and I suppose I should measure something in the python as well.
+
+Then I should see about flow control options, and how to just straight dump data upstream / downstream. 
+
+### Embedded Turnaround Time 
+
+OK, I instrumented this a little, looking at SPI lines and a debug pin on the scope:
+
+- the SPI looks to be running at 12.5MHz, that's nice and fast (but could probably be increased?)
+- transactions (packet-in-out) take ~ 400us each in embedded time, so most of our bottleneck is just there 
+- it looks to be non blocking: SPI is operational even outside of our calls to the thing 
+
+So if we see ~ 800us (first plot in the quad-tet above) average round-trip-time, and 400us of that is in the embedded hardware, we know that improving the embedded side would be worthwhile... and perhaps suggests also that there is not gobs of improvement to be had (i.e. going fast is turning out to be difficult). 
+
+The wiznet datasheet supposes that the SPI can get up to 80MHz, but it seems that [this is also limited in the Arduino library](https://forum.arduino.cc/t/how-to-increase-the-spi-communication-of-ethernet-h/926228).
+
+### How to Improve 
+
+On closer inspection, this thing looks like it's blocking, but it calls `yield()` internally. Basically I don't want to fuck with this too much, forreal. 
+
+I'll admit, actually, I'm a little stumped. I was expecting ethernet to be a magic bullet, but we are up against a very similar limit, and the troubles-at-be seem to be in these hidden layers. 
+
+Things to try would include... taking the Ethernet library offline (into-repo) and fiddling it up to that 80MHz and instrumenting it with some amount of non-blocking flow-control action (this is, actually, probably the move), but I could also i.e. bust out an RPI and see how fast I can un-frame a UART packet into python there... that might be, after all, the answer - or SPI. 
+
+So, for quicksies, and to settle this current debate, I should try not-pinging with this code, just straight up receiving hella UDP upstream...
+
+### Non-Pining Speed Tests 
+
+- setup Arduino to, after one packet rx (to get an IP to tx-back-to) just free-form wrips packets up north, and occasionally prints rates to the OLED
+- setup an async (?) version of the python inheritor, and collect them data 
\ No newline at end of file

code/ethernet/ethernet_sink/plot_stamps.py

+import pandas as pd 
+import matplotlib.pyplot as plt 
+
+def plot_stamps(stamps, stamp_count, pck_len):
+  # make df from stamps 
+  df = pd.DataFrame({'timestamps': stamps})
+
+  # calculate deltas between stamps 
+  df['deltas'] = df['timestamps'].diff() 
+
+  # clean NaN's 
+  df = df.dropna()
+
+  # wipe obviously-wrong deltas (i.e. the 1st, which goes 0-start-us) 
+  df = df[df['deltas'] < 10000]
+
+  # Plotting
+  fig, ax1 = plt.subplots(figsize=(11, 3))
+
+  ax1.set_xlim([50, 4000])
+
+  # Primary x-axis (time deltas)
+  df['deltas'].plot(kind='hist', bins=50, ax=ax1)
+  ax1.set_xlabel('Time-Stamp Deltas (us) and equivalent (MBits/s)')
+  ax1.set_ylabel(f'Frequency (of {stamp_count})')
+
+  # get axis ticks to calculate equivalent bandwidths 
+  x_ticks = ax1.get_xticks()
+  ax1.set_xticks(x_ticks)
+  bandwidths = [((pck_len * 8) * (1e6 / x)) / 1e6 for x in x_ticks]
+  ticks = [] 
+
+  for i in range(len(x_ticks)):
+    print(i, x_ticks[i], bandwidths[i]) 
+    ticks.append(f"{x_ticks[i]:.0f} ({bandwidths[i]:.3f})")
+
+  ax1.set_xticklabels(ticks)
+
+  plt.title(f'Single-Source COBS Data Sink Deltas, pck_len={pck_len}')
+
+  plt.tight_layout()
+
+  plt.show()
\ No newline at end of file

code/ethernet/ethernet_sink/sink.py

-import socket
+import socket, time 
+from plot_stamps import plot_stamps
+import numpy as np 
+import pandas as pd 
+
 
 # Arduino's network settings
 arduino_ip = '192.168.1.177'
 arduino_port = 8888
 
+
 # Create a UDP socket
 sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
 
+
+# test stats
+stamp_count = 100000
+stamps = np.zeros(stamp_count)
+
+
+# test data 
+pck_len = 64
+out_pck = bytearray(pck_len)
+
+
+for i in range(stamp_count):
   # Send a message
-message = b'Hello, Arduino!'
-sock.sendto(message, (arduino_ip, arduino_port))
-print(f'Transmitted...')
+  # message = b'Hello, Arduino!'
+  sock.sendto(out_pck, (arduino_ip, arduino_port))
+  # print(f'Transmitted...')
 
   # Receive response
   data, addr = sock.recvfrom(1024)
-print(f'Received message: {data}')
+  # print(f'Received message: {data}')
+
+  stamps[i] = time.perf_counter() * 1e6
+
+print('stamps', stamps)
+
+plot_stamps(stamps, stamp_count, pck_len) 
\ No newline at end of file

code/ethernet/ethernet_source/ethernet_source.ino

@@ -55,8 +55,8 @@ byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
 IPAddress ip(192, 168, 1, 177);
 unsigned int localPort = 8888;
 
-char packetBuffer[UDP_TX_PACKET_MAX_SIZE];  // is 24, we should test sizes, should go up to nearly 1500 per frame ! 
-char replyBuffer[] = "ack!";
+char packetBuffer[2048];   // what's the max, actually ? 
+char replyBuffer[2048] = "ack!";
 
 EthernetUDP EUDP;
 
@@ -103,9 +103,16 @@ uint32_t rxCount = 0;
 void loop() {
   size_t len = EUDP.parsePacket();
   if(len){
+    digitalWrite(DEBUG_PIN, HIGH);
+    // get the pck, 
     rxCount ++;
-    display_print("rx " + String(len));
+    // display_print("rx " + String(len));
     EUDP.read(packetBuffer, UDP_TX_PACKET_MAX_SIZE);
+    // reply to the pck, 
+    EUDP.beginPacket(EUDP.remoteIP(), EUDP.remotePort());
+    EUDP.write(replyBuffer, len);
+    EUDP.endPacket();
+    digitalWrite(DEBUG_PIN, LOW);
   }
   
   // report status
@@ -115,19 +122,16 @@ void loop() {
   if(lastUpdate + updateInterval < millis()){
     digitalWrite(LED_BUILTIN, !digitalRead(LED_BUILTIN));
     lastUpdate = millis();
-    // write again, 
-    digitalWrite(DEBUG_PIN, HIGH);
     // display_print("STAMP: " + String(lastUpdate));
     if (Ethernet.hardwareStatus() == EthernetNoHardware) {
       display_print("Ethernet shield was not found.");
     } else if (Ethernet.linkStatus() == LinkOFF) {
       display_print("Link OFF");
     } else if (Ethernet.linkStatus() == LinkON){
-      display_print("ON: " + String(rxCount) + " " + (Ethernet.localIP().toString()));
+      display_print("ON: " + (Ethernet.localIP().toString()) + "\n" + String(rxCount));
     } else if (Ethernet.linkStatus() == Unknown){
       display_print("Link UNKNOWN");
     }
-    digitalWrite(DEBUG_PIN, LOW);
   }
 
   // maintain is required to renew DHCP leases, but we are static-ip'ing this mf'er I think