Printers stopped for few sec, console message included

pantherius

Hello,

I'm using Repetier Server on two Raspberry PI 4, server A runs 4 machines flawless, server B has some issues with 3 machines (all Ender 3)

Here is the console error in the moment when all 3 machines stopped: https://ibb.co/4mwPSK0 Could you help how to solve the issue?

Thank you so much!

pantherius

Forgot to mention, all machines has the same Marlin 2.1.1

Repetier

Log does not really show much. Connection is still active as it seems, just no data is received or being send. Is that server using MQTT? If so you should update as soon as 1.4.3 appears where we fix a possible issue causing locking.

What that happens you should check if you can reload website and if all data appears or not. If it does not appear and UI seems to hang it might be a dead lock. If it works normally and if unplugging usb and reconnecting helps it is more likely a driver hanging.

Does this happen frequently? Any advaced features enabled?

pantherius

Thank you for your answer!

All printers stopped, then continue around 20-30sec later normally. Certainly, it is still not good enough as they leave faults on the print where stopped.

The printers' icons switch to orange bars from the green connected one. When all complete and I unplug and plug in the cables, all switch back to green connected status.

I'm starting around 6-8hours jobs and it is happens generally around 1-2 times in this interval.

What does it mean "MQTT"? I'm using Repetier-Server Monitor and rarely the browser based local server page.
Is there a chance it is caused by a bad and/or long cable?

This is the current config for all 3 machines (certainly the Device port is different): https://ibb.co/gwD5d7r The input buffer was 63byte before, now I tried with 127byte, but the problem is the same.
I don't think any advanced feature is enabled.
Thanks!

pantherius

Just realized I'm using 1.1.2, let me try updating to 1.4.2
Still weird my other server doesn't has this issue, so I'm not sure it will help.

Will be back soon.

Repetier

Is your timeout maybe 30s? With marlin 2 it supports normally busy messages, so you can reduce timeout to 3s which reduces the pauses. Especially if only one printer stopped for 30s it is just a communication error that needed to timeout for detection. I already saw you have checksum errors from time to time so this might also happen when returning "ok" and when we miss an "ok" you get a timeout.

pantherius

Thank you, will try that too.

pantherius

Sorry for the delay, I have to test the things to see how does it works.
Reduced the timeout to 3sec as you suggested and it seems to be the period while the machines stopping is better, sometimes I even don't notice it was a break. Also the prints better, I can't see any issues from the stops.
However, I still have the communication error, after each print I have to unplug and plug in the cables to all printers (green online sign switch to orange connected)
Furthermore since I updated the server to 1.4.2, my Repetier-Server Monitor no longer shows the progress of the prints, it is always displaying 0% and the bar also doesn't go up. I'm not sure it is related with the communication error or something else. This is a new issue that come with the 1.4.2 version so probably there is no relation.
One other thing, when I send a print job from Repetier-Server Monitor, all jobs placed in the Print Queue on the browser based Repetier screen of the specific printer. Sometimes there are hundreds of jobs listed there. It is not a big deal, I can delete them, just I thought it is worth to mention.

Could you help with the communication error and the new progress bar issue, please?
Just uploaded one of the printer's server log here: https://filetransfer.io/data-package/LNX3KZtf#link Thank you!

pantherius

Just sent few commands via the console to the printer and I got this (might help):

Send:16:38:45.560: N27 M900

Mesg:16:38:49.564: Warning: Communication timeout - resetting communication buffer.

Mesg:16:38:49.564: Connection status: Buffered:39, Manual Commands: 0, Job Commands: 0

Mesg:16:38:49.564: Buffer used:39 Enforced free byte:0 lines stored:3

Send:16:38:52.699: N28 M900

Mesg:16:38:56.702: Warning: Communication timeout - resetting communication buffer.

Mesg:16:38:56.702: Connection status: Buffered:12, Manual Commands: 0, Job Commands: 0

Mesg:16:38:56.702: Buffer used:12 Enforced free byte:0 lines stored:1

Mesg:16:38:59.703: Reconnecting usb port to fix serial driver problems ...

Mesg:16:38:59.776: Connection closed by os.

Mesg:16:39:00.900: Dtr: true Rts: true

Mesg:16:39:00.902: Connection continued

Recv:16:39:00.908: echo:Unknown command: ""

Send:16:39:13.572: N29 M900

Recv:16:39:13.578: echo:Advance K=0.00

Mesg:16:40:21.681: Connection closed by os.

Mesg:16:40:25.358: Dtr: true Rts: true

Mesg:16:40:25.362: Connection started

Mesg:16:40:25.362: Printer reset requested false

Mesg:16:40:25.362: Dtr: false Rts: false

Mesg:16:40:25.384: Dtr: true Rts: true

Recv:16:40:25.390: Response while unconnected:echo:Unknown command: ""

Recv:16:40:25.390: echo:Unknown command: ""

Recv:16:40:25.390: Connection verified by:ok

Seems to be the green online status turn to orange when communication timeout happens. Does it related with the cable or something else?

Repetier

Looks like you should be lucky the print worked at all (due to silent reconnect). I saw e.g.

2022-11-01 15:14:17: error: Reading serial conection failed: End of file. Closing connection.

2022-11-01 15:14:17: Connection closed during print ... trying reconnect for 10 seconds to continue ...

2022-11-01 15:14:17: Port closed for MAX

2022-11-01 15:14:17: Connection closed: MAX

2022-11-01 15:14:17: error: Reading serial conection failed: End of file. Closing connection.

2022-11-01 15:14:17: Connection closed during print ... trying reconnect for 10 seconds to continue ...

2022-11-01 15:14:17: Port closed for KALI

2022-11-01 15:14:17: Connection closed: KALI

2022-11-01 15:14:17: error: Reading serial conection failed: End of file. Closing connection.

2022-11-01 15:14:17: Connection closed during print ... trying reconnect for 10 seconds to continue ...

2022-11-01 15:14:17: Port closed for HOPE

2022-11-01 15:14:17: Connection closed: HOPE

2022-11-01 15:14:17: Connection continued: MAX

2022-11-01 15:14:17: Connection continued: KALI

2022-11-01 15:14:17: Connection continued: HOPE

When multiple connections get closed by linux the same time it is normally either a power issue or EMI from printer. So you have a hardware issue making linux disconnect usb and depedning on how fast it can reconnect you see it better or less.

You should install 1.4.3 coming in the next minutes. There we add syslog messages from usb around the disconnect time so you can easily see what linux says about the disconnects.

You can also check the bolt icon menu. There you see if linux detected undervoltage since bootup. Especially after such a disconnect it might allow distinguishing power issue from EMI problems on usb from printers.

pantherius

Thank you!

Thunder icon shows everything works fine: https://ibb.co/37vFL7S Just updating to 1.4.3 and will come back with the log later. Thanks a lot!

Repetier

Yes, thats good power so far. So guess you will now see a message like
Sep 21 14:49:34 Repetier-Server kernel: [27513.884540] usb usb1-port1: disabled by hub (EMI?), re-enabling...

Telling that USB hub disconnected device.

amayaatp

Glad to be a member of this fourm, the first post to tell everyone that I am here now.

pantherius

<a rel="nofollow" target="_blank">Repetier</a> said:<blockquote class="Quote">

I didn't see any similar notice. Now I'm using the latest version, still there is some kind of communication error.
Could you check the logs, please?
Find it here: https://files.fm/u/dxxxh366v Thank you!

By the way, all other issues fixed with the new release only the green status switch to orange (communication error) problem persist.

Repetier

I think this is one of the times it happened:

2022-11-07 09:06:33: error: Reading serial conection failed: no data on read.End of file

2022-11-07 09:06:33: error: Reading conection failed: End of file. Closing connection.

2022-11-07 09:06:33: Port closed for MAX

2022-11-07 09:06:33: Connection closed: MAX

2022-11-07 09:06:33: MAX: Your OS syslog contains these messages close to this disconnect that might help to understand:

2022-11-07 09:06:33: MAX: syslog: [1086348.598693] ch341-uart ttyUSB2: usb_serial_generic_read_bulk_callback - urb stopped: -32

2022-11-07 09:06:33: MAX: syslog: [1086348.599215] ch341-uart ttyUSB2: usb_serial_generic_read_bulk_callback - urb stopped: -32

2022-11-07 09:06:35: Connection started: MAX

2022-11-07 09:06:35: Internal reset printer MAX

Linux closed port due to driver stopping with broken pipe error. If you google
"ch341-uart usb_serial_generic_read_bulk_callback urb stopped 32"
you find lots of users with same problem and several issues. Some seem to get this mor eoften if 5v line is taped on usb cable. Others say it's overload on usb if you have a webcam (probably with high resolution/FPS) making driver bail out. In the end something did go wrong in printer<->pi communication and driver disconnected. What helps to prevent this is something you must test, there seems to be no one reason.

What you can do and probably already have done is select the automatic contntinue of print after reconnect. This can not prevent driver failure but can continue in most cases so print is not ruined.

pantherius

Thank you so much for the help. I'll try to investigate it further. At least it is now seems to working without failed prints.

Have a great day! Thanks again for your professional support!

pantherius

Changed the power and data cables, moved the Raspberry Pi 4 to another location, also added dwc_otg.speed=1 (based on Google search) that helps a bit, the issue happens less frequently, but doesn't solved the issue. Still looking for the solution.

Repetier

google

ch341 uart usb_serial_generic_read_bulk_callback urb stopped "-32"

to get more ideas on it. I got 990 hits on it.

pantherius

Yes, thank you, I'm working on it. Just thought it would be useful to share my experiences with anyone who has the same issue in the future.

Repetier

Sure. If you find a solution I'd also be happy to know it. Not sure why I often see it with ch341 chips, maybe just because they have own driver. My only printer with that chip worked without issues so far also I bought it due to reports with same error. So I guess it might depend on some extra conditions in hardware. Maybe some cable placement or gorund contacts. I remember one post where a user meant the housing had contact to usb ground causing it. Not sure how this would affect it as I'm more software than electronic expert.