ssh_dispatch_run_fatal on Rpi4 building www/firefox

[bob prohaska]

While attempting to compile www/firefox on an RPi4 running -current
the make process stopped with

Bad packet length 3554809687.
ssh_dispatch_run_fatal: Connection to 192.168.1.11 port 22: Connection corrupted

on the controlling terminal.

After updating to 14.0-CURRENT FreeBSD 14.0-CURRENT #15 main-8cca7b7f2:
Tue Apr 13 17:46:39 PDT 2021

the error promptly recurred if make was restarted without cleaning.

As an experiment, I've restarted the make process via a serial connection
and it didn't immediately reproduce the error. In all cases the make
command is

make -DBATCH MAKE_JOBS_UNSAFE=yes MAKE_JOBS_NUMBER=4 DISABLE_VULNERABILITIES=yes > make.log

and is run in the background. The same make command successfully compiled
www/chromium via ssh, so I don't think the culprit's the command line.

I rather doubt this is a www/firefox problem, but include freebsd-ports
in hopes somebody might recognize the error.

Make generates a torrent of warnings using the serial connection, such as:

#if IN_HEADER(__GTK_ITEM_FACTORY_H__)
^
./gtkalias.h:10:19: note: expanded from macro 'IN_HEADER'
#define IN_HEADER defined
^
./gtkalias.h:5366:5: warning: macro expansion producing 'defined' has undefined behavior [-Wexpansion-to-defined]
#if IN_HEADER(__GTK_LABEL_H__)
^
./gtkalias.h:10:19: note: expanded from macro 'IN_HEADER'
#define IN_HEADER defined
^
but seems to keep running via the serial connection. Top reports that
cc is in state TTYOUT, with WCPU no higher than a few percent, running
only one thread.

Thanks for reading, and any ideas.

bob prohaska

[Valery Seys]

Post by bob prohaska
While attempting to compile www/firefox on an RPi4 running -current
the make process stopped with
Bad packet length 3554809687.
ssh_dispatch_run_fatal: Connection to 192.168.1.11 port 22: Connection corrupted
on the controlling terminal.
Tue Apr 13 17:46:39 PDT 2021
the error promptly recurred if make was restarted without cleaning.
As an experiment, I've restarted the make process via a serial connection
and it didn't immediately reproduce the error. In all cases the make
command is
make -DBATCH MAKE_JOBS_UNSAFE=yes MAKE_JOBS_NUMBER=4 DISABLE_VULNERABILITIES=yes > make.log
and is run in the background. The same make command successfully compiled
www/chromium via ssh, so I don't think the culprit's the command line.
I rather doubt this is a www/firefox problem, but include freebsd-ports
in hopes somebody might recognize the error.
#if IN_HEADER(__GTK_ITEM_FACTORY_H__)
^
./gtkalias.h:10:19: note: expanded from macro 'IN_HEADER'
#define IN_HEADER defined
^
./gtkalias.h:5366:5: warning: macro expansion producing 'defined' has undefined behavior [-Wexpansion-to-defined]
#if IN_HEADER(__GTK_LABEL_H__)
^
./gtkalias.h:10:19: note: expanded from macro 'IN_HEADER'
#define IN_HEADER defined
^
but seems to keep running via the serial connection. Top reports that
cc is in state TTYOUT, with WCPU no higher than a few percent, running
only one thread.
Thanks for reading, and any ideas.
bob prohaska
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-arm

Hi Bob,

so you compile rpi'side, perhaps could you start sshd with '-E <logfile>' in
order to get more infos.
On the RPI, as root:
1) echo 'sshd_flags="-E /var/log/sshd_debug.log"' >> /etc/rc.conf
2) touch /var/log/sshd_debug.log
3) service sshd restart

(if you already have sshd flags in rc.conf, plz edit instead of the echoing as
above ...)

Then restart your ssh session, and see what happens in the log,

Ps : firefox need a huge amount of memory to compile, according to the number of
compiling instances (eg cpu threads) you have, it could easily reaches >4G, so
your trouble could come from this (sshd cannot be swapped ...), in this case,
try to lowered the number of threads make will use, through one of these options:
DISABLE_MAKE_JOBS: Set to disable the multiple jobs feature. User settable.
MAKE_JOBS_NUMBER: Override the number of make jobs to be used. User settable.
MAKE_JOBS_NUMBER_LIMIT: Set a limit for maximum number of make jobs allowed to
be used.
Get your number of cpu thread:
sysctl -n kern.smp.cpus


hope it helps,

v/

[bob prohaska]

Post by Valery Seys
so you compile rpi'side, perhaps could you start sshd with '-E <logfile>' in
order to get more infos.
1) echo 'sshd_flags="-E /var/log/sshd_debug.log"' >> /etc/rc.conf
2) touch /var/log/sshd_debug.log
3) service sshd restart
(if you already have sshd flags in rc.conf, plz edit instead of the echoing
as above ...)
Then restart your ssh session, and see what happens in the log,

Seems I can't reproduce the problem, despite re-starting the make
of www/firefox after cleaning. I think the system got past the
problem point when building via the serial console connection.
Makes me wonder if it was a dependency, rather than firefox.

Post by Valery Seys
Ps : firefox need a huge amount of memory to compile, according to the

Seemingly not a problem, but this is on an 8GB Pi4. From past experiments
firefox is not obviously worse (!) than www/chromium.

Thanks for writing!

bob prohaska

[tech-lists]

Hi,

Post by bob prohaska
While attempting to compile www/firefox on an RPi4 running -current
the make process stopped with
Bad packet length 3554809687.
ssh_dispatch_run_fatal: Connection to 192.168.1.11 port 22: Connection

Is your compiling done on the microsd?

If it is, then that's your problem. The microsd is too slow. It causes
bottlenecks and timeouts all over the place.

Fix would be to mount /usr/src and /usr/obj (and ccache =
/var/cache/ccache) on usb3-connected hardware.

[bob prohaska]

Post by tech-lists
Hi,

Post by bob prohaska
While attempting to compile www/firefox on an RPi4 running -current
the make process stopped with
Bad packet length 3554809687.
ssh_dispatch_run_fatal: Connection to 192.168.1.11 port 22: Connection

Is your compiling done on the microsd?

No, mechanical SATA drive in a USB3 case.

The immediate oddity was that make, running in the background,
was stopping when the ssh connection dropped. Normally, disconnecting
the controlling terminal on a background job doesn't kill the job.

Post by tech-lists
If it is, then that's your problem. The microsd is too slow. It causes
bottlenecks and timeouts all over the place.

Agreed, but this wasn't a timeout. Maybe another misleading error message?

After several restarts of make in www/firefox which all stopped with
the same error I tried using the serial console. Make ran without
complaint and made considerable progress.

When I learned of the -E option to ssh I stopped the make session,
set up error logging, ran make clean in www/firefox and started over.

Make subsequently ran to completion and produced a runnable www/firefox.
This tempts me to think whatevever triggered the error was among the
dependencies, which is difficult to understand and much harder to repeat.

Thanks for replying!

bob prohaska