Locking up and sometimes crashing 1.3

Hi,
I have been using 0.93 for ever as I had some reliability issues with the versions just after that. I just upgraded to 1.3, and also did a complete reinstall (latest image) on a fresh sd card (due to crashes). Also tried a different pi (pi 4) was using a 3b.
Anyway to cut a long story short, sometimes the pi locks up causing the printers to reset and sometimes the printers just stop (2 prusas mk3s). The latest time I got the following from the trouible shooting and console:

error: Reading serial conection failed: End of file. Closing connection.
[05.07.2022 18:58:29] : error: Reading serial conection failed: End of file. Closing connection.

Recv:18:58:26.093: LCD status changed (3)
Mesg:18:58:28.979: Connection closed by os during print ... trying reconnect for 10 seconds to continue ...

So another wasted print. 

It is connected via a powered usb hub and each printer has a usb camera.

Everything was working fine until I upgraded.

PLease help.

I have ordered 2 4 port powered usb hubs for delivery tomorrow to try to separate the printers or the printers and cameras.

Other than that no idea what to do.

Cheers

Richard

Comments

  • > sometimes the pi locks up
    What do you understand from this? For us a lock up is that nothing works any more. No response from API so gui is non functional if it even loads. This is normally dead lock but only a full backtrace as described here
    https://www.repetier-server.com/knowledgebase/debugging-crashes-hangs-on-linux/
    can reveal what is causing it and we can then eventually fix it (if reason is clear) or see that it is fixed for upcoming 1.4.0 already (coming soon).

    This means the pi is still accessible so you can login over ssh. If pi crashed completely so linux is not running as well it is unlikely a server issue.

    > error: Reading serial conection failed: End of file. Closing connection.

    This is quite clear. Linux closed the serial normally when usb is deactivated. Linux does it when it detect EMF or bigger undervoltage. So you should have a look at our gui if you see in the flash menu a hint for undervoltage. If that is the case from experience this often causes random issues on pi including crashes and usb disconnects.

    > It is connected via a powered usb hub 
    You mean there is no designated power source for pi just backpower from hub (which as far as I know should not exist from standard but most hubs have)? As you read everywhere power is the critical part on the pi. You should use a 5.1V power supply.

    You might ask what changed - well linux and drivers and pi firmware are newer with newer image. Maybe it is just using more power due to more cpu usage or drivers/linux is more sensitive when they disable things. Hard to say. In general we have no known printing issues in 1.3 except hardware related problems.

    With 1.4.0 we also have a new image that will be based on current Raspbian OS bullseye while 1.3 is based on buster. So upgrading image for next update might also help.
  • Hi,

    Thanks for the response as always. By connected via a usb hub I meant the printers and cameras. I have now connected the two prusas to the two USB 2 ports directly on the pi, and left the cameras on the hub and have so far ran a couple of prints ok (more running now). When the two new hubs arrive I will try them (as I need to connect my 3rd printer again). There are no under voltage on the flash menu.
    I will let you know how it all goes.

    Thanks

    Richard


  • Hi,
    So managed a couple of 7 hour prints or more on each. Pi has been up for 21 hours.
    Then a few minutes ago it disconnected both printers, pi and repetier still running. Printers connected direct to usb 2 ports on the pi 4.
    Same error as before. I have pasted the section from the syslog below from when it stopped - 16:49. A copied and pasted in two parts so may have some overlap or a few missing records but they are all the same.

    Jul  6 16:48:35 RepetierServer dbus-daemon[416]: [system] Activating via systemd: service name='org.freedesktop.timedate1' unit='dbus-org.freedesktop.timedate1.service' requested by ':1.2329' (uid=0 pid=22742 comm="timedatectl ")
    Jul  6 16:48:35 RepetierServer systemd[1]: Starting Time & Date Service...
    Jul  6 16:48:35 RepetierServer dbus-daemon[416]: [system] Successfully activated service 'org.freedesktop.timedate1'
    Jul  6 16:48:35 RepetierServer systemd[1]: Started Time & Date Service.
    Jul  6 16:49:05 RepetierServer kernel: [75117.770790] usb 1-1.2.1: clear tt 1 (90a3) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.770923] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.771622] usb 1-1.2.1: clear tt 1 (90a3) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.771940] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.772628] usb 1-1.2.1: clear tt 1 (90a3) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.773000] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.773759] usb 1-1.2.1: clear tt 1 (90a3) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.774033] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.774886] usb 1-1.2.1: clear tt 1 (90a3) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.775702] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.776031] usb 1-1.2.1: clear tt 1 (90a3) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.776225] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.780188] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.780230] usb 1-1.2.1: clear tt 1 (90a3) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.780770] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.780934] usb 1-1.2.1: clear tt 1 (90a3) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.781669] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.782032] usb 1-1.2.1: clear tt 1 (90a3) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.782753] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.783034] usb 1-1.2.1: clear tt 1 (90a3) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.783837] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.784161] usb 1-1.2.1: clear tt 1 (90a3) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.785695] usb 1-1.2.1: clear tt 1 (90a3) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.785737] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.786201] usb 1-1.2.1: clear tt 1 (90a3) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.786442] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.787316] usb 1-1.2.1: clear tt 1 (90a3) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.787519] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.788630] usb 1-1.2: USB disconnect, device number 3
    Jul  6 16:49:05 RepetierServer kernel: [75117.788652] usb 1-1.2.1: USB disconnect, device number 5
    Jul  6 16:49:05 RepetierServer kernel: [75117.788671] usb 1-1.2.1.4: USB disconnect, device number 9
    Jul  6 16:49:05 RepetierServer kernel: [75117.789015] usb 1-1.2.1: clear tt 1 (90a3) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.789076] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.790869] uvcvideo: Failed to resubmit video URB (-19).
    Jul  6 16:49:05 RepetierServer kernel: [75117.791615] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75117.795473] uvcvideo: Failed to resubmit video URB (-19).
    Jul  6 16:49:05 RepetierServer kernel: [75117.795964] usb 1-1.2.2: USB disconnect, device number 8
    Jul  6 16:49:05 RepetierServer kernel: [75117.798921] uvcvideo: Failed to resubmit video URB (-19).
    Jul  6 16:49:05 RepetierServer kernel: [75117.802923] uvcvideo: Failed to resubmit video URB (-19).
    Jul  6 16:49:05 RepetierServer kernel: [75117.806954] uvcvideo: Failed to resubmit video URB (-19).
    Jul  6 16:49:05 RepetierServer thd[433]: Error reading device '/dev/input/event0'


    Jul  6 16:49:05 RepetierServer kernel: [75117.798921] uvcvideo: Failed to resubmit video URB (-19).
    Jul  6 16:49:05 RepetierServer kernel: [75117.802923] uvcvideo: Failed to resubmit video URB (-19).
    Jul  6 16:49:05 RepetierServer kernel: [75117.806954] uvcvideo: Failed to resubmit video URB (-19).
    Jul  6 16:49:05 RepetierServer thd[433]: Error reading device '/dev/input/event0'
    Jul  6 16:49:05 RepetierServer systemd[1]: systemd-timedated.service: Succeeded.
    Jul  6 16:49:05 RepetierServer kernel: [75118.264858] usb 1-1.2: new high-speed USB device number 19 using xhci_hcd
    Jul  6 16:49:05 RepetierServer kernel: [75118.275812] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75118.276233] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75118.276738] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75118.277239] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75118.277732] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75118.278228] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75118.278794] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75118.279254] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75118.279768] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75118.280271] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75118.280795] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75118.281306] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75118.281813] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75118.282315] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75118.282842] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75118.283354] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:05 RepetierServer kernel: [75118.365134] usb 1-1.2: device descriptor read/64, error -71
    Jul  6 16:49:06 RepetierServer kernel: [75118.585096] usb 1-1.2: device descriptor read/64, error -71
    Jul  6 16:49:06 RepetierServer kernel: [75118.785329] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:06 RepetierServer kernel: [75118.785829] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:06 RepetierServer kernel: [75118.786344] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:06 RepetierServer kernel: [75118.786852] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:06 RepetierServer kernel: [75118.787365] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:06 RepetierServer kernel: [75118.787923] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:06 RepetierServer kernel: [75118.788399] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:06 RepetierServer kernel: [75118.788932] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:06 RepetierServer kernel: [75118.789444] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:06 RepetierServer kernel: [75118.789958] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:06 RepetierServer kernel: [75118.790470] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:06 RepetierServer kernel: [75118.790980] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:06 RepetierServer kernel: [75118.791490] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:06 RepetierServer kernel: [75118.792048] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:06 RepetierServer kernel: [75118.792604] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:06 RepetierServer kernel: [75118.793075] usb 1-1.3: clear tt 1 (9083) error -71
    Jul  6 16:49:06 RepetierServer kernel: [75118.804941] usb 1-1.2: new high-speed USB device number 20 using xhci_hcd
    Jul  6 16:49:06 RepetierServer kernel: [75118.905060] usb 1-1.2: device descriptor read/64, error -71
    Jul  6 16:49:06 RepetierServer kernel: [75119.125082] usb 1-1.2: device descriptor read/64, error -71



  • Error 71 is protocol error, so linux is not happy with received data. Likely a communication error triggering this. No idea what the driver means with clear tt 1.

    Are you using webcams with mjpg or yuyv encoding? If you have more then one webcam with yuyv apart from mjpg_streamer being very busy with encoding the usb bus will be 100% occupied. At least I see errors from webcam as well at same time so there might be a connection. Can you also provide
    lsusb -t
    output so it is easier to see which usb devices syslog is reporting for.

    There is an option to reconnect usb on disconnection - just written for such cases where normally after disconnect linux reconnects usb quite quickly. Make sure you have it enabled. Could at least make printers continue print so they are not lost. Also of course reason should still be found out, but it is quite surely not server related. Print stop is just a result of linux disconnecting the usb. Next releas eis nearly finished maybe it works better as it also uses newer kernel with more fixes. On the other side if it is overload or communication disturbances it will not change. Anything changed like cables or position of cables, new printer/webcam already connected compared to old solution.
  • Hi

    I have just moved the location of the pi, changed the usb hub (new hub) that the cameras are connected too.
    Running some prints now. If it happens again I will send the lsusb output. Fast reconnect is on but once the usb's go they are gone until a reboot. Even disconnecting the usb cable and reinserting does nothing.

    I will keep trying

    Cheers

    Richard
  • BTW the only difference to the old solution is the upgrade of repetier and changed pi (changed pi because it was happening on the original). Oh and latest Prusa firmware
  • > they are gone until a reboot. Even disconnecting the usb cable and reinserting does nothing.
    That is hard. Normally crashed usb drivers restart on cable reconnect. We even have a function to disconnect usb to force driver to reconnect (USB reconnect on timeout) - normally it should be never. If it is never you might try to enable it but if really only reboot helps it should not help here:-(
  • I am going to be away until middle of next week so may not get back with anything this week.
  • Just upgraded to 1.4 new image. Only issue so far is that repetier server monitor thinks I don't have a license, yet the server is licensed and working
  • I just got the issue my self first time with some reinstalls. What helped was while monitor was running restarting the server. That made it init correctly. Actually as far as I could debug it seems like server is not correctly changing state variables when activating a license. On restart it is set correctly in my test. So will now check server code for next update to prevent this.
  • Thanks. I will restart when I can
  • Thanks for the suggestion.
Sign In or Register to comment.