Wifi-Ethernet version

[arkypita]

As requested by the user @mkeyno i want to add the ability to connect via WiFi-Ethernet connection.
I have developed a version with tcp-ip connection, but i am not able to test it since I don't have an ethernet adapter for my arduino board.

Could someone test it and give some feedback?

Download link: https://github.com/arkypita/LaserGRBL/releases/tag/v2.6.10

[arkypita]

@mkeyno please give some feedback

[mkeyno]

dear @arkypita thank you very much for your concern and sorry for being late , I ran some test to check if there is stable websocket link , however the session log and my esp log not report any websocket link , also I've tried to read your code for any websocket connection but nothing find similar (please correct me if there is )
I might summarized couple of presumption I used for wifi bridge
1)In ESP side I used async websocket server which is handshake & html upgarde happend at 192.xxx.xxx.xx/ws , so at first step your websocket client should send the request on that links and the establish the websocket links
2) ESP rx, tx pin directly connected to Arduino uno tx,rx pin
3) In ESP loop function if there is serial data from Arduino , it will be send to established websocket link

#include <ESPAsyncWebServer.h>
     AsyncWebServer server(80);
     AsyncWebSocket WS("/ws");

....
void loop ( void )
 {
 SerialEvent();
 if(NewLine && (millis()-serialTS)>50){  WS.text(inputString); inputString = ""; serialTS=millis();NewLine=false;}    
if(INFO) {if( (millis()-Last_query)>210 ) {Serial.print('?'); Last_query=millis();}} // send ? char to GRBL
}

4. if there is the data from wifi bridge it will print it to the serial

following is the session log, please let me know if need more data or test
`

6/15/2017 10:10:53 PM SetStatus Machine status [Connecting]
6/15/2017 10:10:53 PM OpenCom System.InvalidOperationException: Port already opened
6/15/2017 10:10:53 PM OpenCom at LaserGRBL.ComWrapper.Ethernet.Open()
6/15/2017 10:10:53 PM OpenCom at LaserGRBL.GrblCore.OpenCom()
6/15/2017 10:10:53 PM SetStatus Machine status [Disconnected]
6/15/2017 10:10:55 PM SetStatus Machine status [Connecting]
6/15/2017 10:10:55 PM OpenCom System.InvalidOperationException: Port already opened
6/15/2017 10:10:55 PM OpenCom at LaserGRBL.ComWrapper.Ethernet.Open()
6/15/2017 10:10:55 PM OpenCom at LaserGRBL.GrblCore.OpenCom()
6/15/2017 10:10:55 PM SetStatus Machine status [Disconnected]
6/15/2017 10:11:00 PM SetStatus Machine status [Connecting]
6/15/2017 10:11:00 PM OpenCom Open 192.168.4.1:80
6/15/2017 10:11:00 PM SendImmediate Send Immediate Command [24]
6/15/2017 10:11:34 PM SetStatus Machine status [Disconnected]
`

[arkypita]

Hi @mkeyno thank for the reply

Websocket? HTML? So you implement a sort of web server in ESP? Why? To me it seems a strange thing to use http and html to transfer a serial protocol over tcp-ip!

ok ok... i have read https://github.com/LaserWeb/LaserWeb3-ESP8266
so, this is the way LaserWeb talk with ESP8266... ok.

For me the simplest way is to implement a TcpListener on ESP8266 and redirect both read & write to rx-tx pin. And what I have implemented in LaserGRBL is simply a TcpClient that connect to a port and send-receive data. (this is the most easy and intuitive way to do that, i really don't understand why they decide to use websocket).

However I can implement a sort of "LaserWeb ESP8266" compatible wrapper. Now I have virtualized the part of the LaserGRBL that manage communication, so I can easly add a new wrapper.

Luckly i must have a ESP8266 module somewere, so when I have time I can program it with the code provided in from LaserWeb and test it.

you must wait some weeks, i am quite busy now!

[mkeyno]

sure dear @arkypita , I might add for reliable link we need live protocol such as websocket otherwise tcp link will be chock upon the stream data, websockt easy to install on web platform or .net platform that's why most people choose the websocket

[arkypita]

Hi @mkeyno
I successfully programmed my ESP8266 as explained in https://github.com/LaserWeb/LaserWeb3-ESP8266 (download library, board, replace WifiManager.h, arduino sketch and successful programming)

I use this schema for programming:

The right button, when pressed, connects the RESET pin to the ground and when it is released, connects the RESET pin to the VCC through a pull-up resistor. The left button, when pressed, connect GPIO0 to the ground. Using this two buttons you can do all the tasks you need with the ESP8266:

Working on normal mode - Both buttons are released.
Resetting the ESP8266 - Press the reset button and release.
Start in flash mode - Press both buttons, release the reset button and then release the GPIO0 button.

Like in the .gif image i could connect to the Emblaser2 network, contact the ESP on default IP adress, configure my network and connect ESP as part of my LAN and now I can ping its address (192.168.1.170) assigned by my router DHCP.

I also have connected ESP to arduino as in this schema, arduino has Grbl v1.1 loaded on it.

Both LaserWeb3 and LaserWeb4 cannot connect to ESP, I am stuck on this problem.

LW4 console say;

 Connecting Machine @ ESP8266,192.168.1.170
 Server error: unexpected server response (404)
 Machine disconnected
 Machine disconnected

If I open a browser on 192.168.1.170 I have a 404 page with Not found: /

Can you help me? Having a WiFi bridge working with laserweb is the only way to be sure that the code I am writing will be 100% compatible.

[arkypita]

Solved: I must use port 81

Now I have other kinds of problem, but I think I can solve them

[arkypita]

@mkeyno I have got it!

[arkypita]

@mkeyno
What serial speed do you use between ESP and arduino?
I am using 115200 but I am experiencing a lot of communication error, like when you have electrical noise caused by stepper motor (but i am not using stepper!)

[mkeyno]

well done @arkypita can you direct me to your websocket methods in your program? have you use reconnect websocket client to make the link robust?
the ESP websocket server in the laserweb is old and use the sync server whereas the async server is much faster and reliable , personally I used the 250K for serial with no error or losing data, however I'm not sure GRBL can handle such speed or even it will enhanced the engraving feature , I'm design new PCB with ESP as links for Arduino GRBL ,let me check your new update with my ESP module and report back here
thanks for your concern

[arkypita]

Beta version: https://github.com/arkypita/LaserGRBL/releases/download/v2.6.12/setup.exe
Source code: https://github.com/arkypita/LaserGRBL/tree/LaserWebESP8266

All the code regarding ESP is in this file: https://github.com/arkypita/LaserGRBL/blob/LaserWebESP8266/LaserGRBL/ComWrapper/LaserWebESP8266.cs

I have used the websocket-sharp implementation, because microsoft implement websocket only in .NET framework 4, and I like to use framework 3.5 because framework 4 is not Windows XP compatible.

[arkypita]

Now I have tested using 9600baud betweeen ESP and Arduino, but still I have the same problem of 3 message ago.

In this moment I cannot figure out:

• if I lose character overfilling up some buffer (but I don't think) and where (between websocket client-server or between ESP and arduino?)
• if it is an electrical problem (noise, ...)
• if I have level problem (ESP work at 3.3v arduino ad 5v, maybe not to much voltage to read data correctly?)

Maybe you can help me testing with your hardware that is more stable, so we can understand if it is HW or LaserGRBL SW problem.

[arkypita]

@mkeyno
please use v2.6.13 for testing
Websocket URL must be in the format "ws://ip:port/path" (ie. ws://192.168.1.120:80/ws)

[mkeyno]

@arkypita the v2.6.13 is not has wifi connection , however I see you truly use async websocket which is so stable than sync version , as previous discussion the client side must has re connect routing to server side , is the websocket c# has such feature?
also I've try to implement the streaming method that GRBL repo suggested , because ESP need to know how much line successfully transmitted
maybe its good though that your software ask the ESP current status to keep transmission stable, either of ESP or your software should the check the transmission status(software flow control)

TTL is not issue caz ESP can handle 5 vcc as soon as the sinking and sourcing current not exceeded than 10, 12 mA, I use simple divider for ESP Rx pin but simple resistor also work fine

[arkypita]

Can you also share your ESP code with me?
I think that the code I use from LW repo is not so good. I have found some of bug in it.

[arkypita]

@arkypita the v2.6.13 is not has wifi connection

@mkeyno go to menu "grbl->settings" you will found a config box to select LaserWebESP8266 protocol (websocket)

[arkypita]

I also implemented a sort of "grbl emulator via websocket" (emulate both ESP Websocket Server and Arduino response). Emulator is available from "? -> grbl emulator" and listening as ws://127.0.0.1:81/

Of course with the emulator all work fine :-)

[mkeyno]

dear @arkypita please check the html file in data folder , I've design that webpage so user can change the GRBL setting as well as wifi bridge , my plan was to upload the file with file reader javascrip and then send the codes to ESP according the GRBL advance stream method, in ESP print the receive data to Arduino serial

else if(cammand=="RAW") {if(!STARTING) {Serial.print(value); CODE="CNC@"; serialTS=millis(); }} 

and send the GRBL respond on websocket link

SerialEvent();
 if(NewLine && (millis()-serialTS)>50){  WS.text(MainWSclinet,CODE+inputString); inputString = ""; serialTS=millis();NewLine=false;}    

[arkypita]

@mkeyno thanks for your share. I will need a moment to understand the code of your wifi-bridge, because it is very complex (there are so many features, not just basic streaming).

Did you ever try version 2.6.13? Communication via WebSocket should be ok now.

ESP8266 protocol must be enabled in settings, than could be configured as a simple URL.

If I use it with my ESP8266, programmed with LaserWeb3-ESP8266 code, I can connect successful and I also can stream some GCode, but the streaming process is slow, it's hiccups, and grbl responses give errors (like in prev pictures) highlighting some data loss in the whole trip.

I am very interested in knowing if LaserGRBL v2.6.13 connects to your ESP8266 or if it does not connect, and whether the streaming behavior is right or it has the same issues that I have.

LaserGRBL implement the suggested character counting streaming protocol (like stream.py) since its very first version.

From the LaserGRBL side I use character counting with both USBSerial and WebSocket protocol, since its implementation is outside and independent of the chosen communication channel.

[mkeyno]

@arkypita I connect to ESP with ws://192.168.4.1/ws address and websocket link is stable , however My ESP not connected to GRBL so it can not respond and hence I can't send stream of gcode data , can you create test version so I could stream cnc data to ESP without any restriction and check stream link reliability,

[arkypita]

@arkypita I connect to ESP with ws://192.168.4.1/ws address and websocket link is stable

Ok, good!

can you create test version so I could stream cnc data to ESP without any restriction and check stream link reliability,

I have 2 different restriction preventing the ability of stream a file when arduino/grbl not connected

1. I need "Idle" state sent from grbl. If no idle received LaserGRBL stay in "connecting" and prevent sending any command.
2. When sending commands LaserGRBL need to receive "ok" to decrement pending command and freeing character counting. If it don't receive "ok" LaserGRBL stop sending command after 128 character

It is easy for me to remove both test. It's ok or you prefer I remove only the first check, and you send back "ok" from ESP?

[arkypita]

I found all the reasons of my problems!

It is because I was sending "\r\n" as line termination, since grbl standard require "\n" only.
Seem that with direct serial is more tolerant, but with ESP there are problems. I am correcting my code, and finally we have a ESP8266 ready release!

[mkeyno]

well done @arkypita , have you check my lib ? what's your idea about control flow mechanism in ESP

[arkypita]

have you check my lib ?

Yes, and no. I have tried to load your code, but I can't find a way to connect ESP to my network, so I continue development using the code provided for LaserWeb

what's your idea about control flow mechanism in ESP

Grbl has only 128 wide character command buffer. Because a classic arc movement (G2 X0.000 Y0.000 I0.000 J0.000) is about 30 character wide you can't enqueue more then 4-5 movements command.

This could cause some bottleneck when a series of a lot of small movements are enqueued. When the execution time of the movements is less than the serial transmission time some slowness can happens. Buffers can help (averaging chars-long exec-fast command with chars-short exec-slow command) but not so much if the buffer is so small.

Doing some maths, the transmission of each command character require 10bits (8 bit for the ascii code + 1 start bit + 1 stop bit) so 30 character require 300 bit. With a baudrate of 115'200 this mean a bandwith of 384 different commands of 30 character, per each second. (about 2-3 millisecond per each command transfer).

This speed is high enough for almost any vector engraving needs, especially with low power lasers, but as you know it could be a problem in high speed raster images engraving with medium power laser. #38

All of these are limitations due to the serial channel speed, and to a protocol - the gcode - not intended for the transmission of raster images.

[arkypita]

When adding an ESP8266 all these limitations continue to be valid, since the use of the serial is not completely replaced by WiFi and the grbl buffer size still remain the same.

However what may happen is that disturbances or slowdowns of the WiFi can hiccup the communication with ESP8266. In those cases it might sound smart to take advantage of the ESP memory to have a larger buffer on the side of the engraver.

[arkypita]

A good code for ESP should async receive any character from the streamer program and puts in its own "big buffer". Obviously the streamer must know the "big buffer" size and use this limit instead of 128 character limit. to transfer command from PC to ESP as much as possible, as fast as possible.

Another async job in the ESP must implement the "128 character counting" mechanism, enqueuing commands to grbl (without removing from its knowledge) and check for grbl responses (ok or error) to remove that command from its queue and "free" the count of 128, and passing back responses to streamer (that implement the same character counting, but using the "big buffer" size).

This is very easy to do in high level programming language, where you have managed objects like lists, queue, strings, but it is not so easy in pure arduino c/c++

Since WiFi speed is usually higher than serial speed, in my opinion all this complexity is not really an advantage. So I suggest that the ESP only do wifi-serial bridge without any buffering or "intelligence"

[arkypita]

I have also created a TelnetToSerial implementation for ESP8266 to rid off the needs of WebSocket, that is really unnecessary to gain the goal, so now LaserGRBL can easily connect via WebSocket or via Telnet to its own ESP.

https://github.com/arkypita/ESP8266-SerialTelnet