Strange u-boot behavior

Post by bob prohaska
For some time now I've been booting an Rpi3B+ using a microSD
card configured with
U-Boot 2019.10 (Mar 17 2020 - 22:01:14 -0700)

I'm unclear. Do you have the whole msdos file system
content on the microsd card and only the FreeBSD UFS
kernel+world on the USB drive? No msdos file system
on the USB drive?

Some of the confusion may become clearer about why I
might care in my later notes.

Another question: why still U-Boot 2019.10 instead of
updating to match, say, what is on modern RELEASE or
snapshot builds for the version of FreeBSD that you
are using on the RPi3B+ ? You seem to be using a
generally not-used combination by sticking to an older
U-Boot but updating other things.

What RPi3B+ firmware version? This can be found via:

# strings start.elf | grep "VC_BUILD_ID_"

Depending on what is reported: this might have
questions about the vintage used.

Post by bob prohaska
by stopping it at the u-boot prompt and issuing
usb reset
followed by
run bootcmd_usb0
after which the usb hard disk boots and all is well.
Lately, usb reset has taken to reporting
resetting USB...
Request Sense returned 02 04 01
5 USB Device(s) found
scanning usb for storage devices... 0 Storage Device(s) found
However, usb tree reports
1 Hub (480 Mb/s, 0mA)
| U-Boot Root Hub
|
+-2 Hub (480 Mb/s, 2mA)
|
+-3 Vendor specific (12 Mb/s, 90mA)
| FTDI FT232R USB UART AM00FGTR
|
+-4 Vendor specific (480 Mb/s, 2mA)
|
+-5 Mass Storage (480 Mb/s, 0mA)
ASMT ASM105x 12345678D558
The problem isn't wholly new; the usb reset command sometimes had
to be repeated once or twice before the disk was found. Now it seems
a persistent failure.
Once FreeBSD has booted, the disk is seen and accessed as usual.

What alternatives have you tried?

*) Have you tried starting a boot sequence once with
program_usb_boot_timeout=1 in config.txt , per notes in:

https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/bootflow.md

This programs a One Time Programmable bit to cause
extra some time to be allowed. The
program_usb_boot_timeout=1 does not need to be kept
in place. After another reboot (to a RaspiOS
in order to have the command available):

vcgencmd otp_dump | grep 66

should have bit 24 set in what it reports.

This might only be useful for option (B) below. I've
never found wording specifying the range of modes
that this adds time to. But it might apply to all
or most modes.

@) Have you tried any mixes of bootcode_delay=???,
boot_delay=???, boot_delay_ms=??? in config.txt ?
These are documented in:

https://www.raspberrypi.org/documentation/configuration/config-txt/boot.md

bootcode_delay can add time before any start*.elf
is read from whatever media --but it is not clear if
the config.txt containing the bootcode_delay=???
and start*.elf can be from different media.

boot_delay and boot_delay_ms control waiting some
amount of time in start*.elf before loading the
next stage (U-Boot for FreeBSD's normal sequence).

A) Using a microsd card with only a modern bootcode.bin
should be able to have the rest of the material on the
USB device, including U-Boot. This way tends to have
more fixes/improvements than depending on the internal
support for direct USB booting: It supposedly works for
more USB devices.

I do this on a RPi2B v1.1 that does not have support
(B) below. I have done this in temporary contexts
on a RPi3B where (B) below seemed to have a
problem for some reason. Dealing with booting from
(some?) powered hubs can be a type of context where
this proves useful.

See:

https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/README.md

and "Special bootcode.bin-only boot mode" in:

https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/msd.md

If the msdos file system that contains bootcode.bin also
contains an empty file named "timeout" it will wait
longer for a USB device to be ready. See "Known issues"
in:

https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/msd.md

B) "The Raspberry Pi 3B+ supports USB mass storage boot out
of the box". So, unless some of those improvements
metioned in (A) turn out to be necessary, no microsd card
should be needed, presuming that all the required
material (including U-Boot) is on the USB drive.

I do this on a RPi3B and RPi2B v1.2 (after enabling them:
not "out of the box") and all the RPi4B's. (Some RPi4B's
required eeprom updates to enable this.)

This does not give any control over getting extra time
for the USB device to be ready: it would first have to
have already read something from the device. But the
details of this might be better than the details of
some specific microsd card based RPi3B+ firmware boots.
I.e., it might be worth a try.

C) I will note that it is possible to have the FreeBSD
kernel also on the microsd card in a UFS partition and
to have "world" on the USB drive. This leads to only
FreeBSD's kernel and later using the USB drive. But
keeping the copy of the kernel and such on the
separate media is messier and atypical.

(Note: I do this sort if thing for the ROCK64 where
the dd'd U-Boot does not support USB for the FreeBSD
loader to put to use. I defer USB first-use to the
kernel.)

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

bob prohaska

2021-06-06 16:00:40 UTC

Post by bob prohaska
For some time now I've been booting an Rpi3B+ using a microSD
card configured with
U-Boot 2019.10 (Mar 17 2020 - 22:01:14 -0700)

I'm unclear. Do you have the whole msdos file system
content on the microsd card and only the FreeBSD UFS
kernel+world on the USB drive? No msdos file system
on the USB drive?

It's a dual-boot system, with a complete FreeBSD-current install
on both USB and microSD storage.

Post by Mark Millard via freebsd-arm
Some of the confusion may become clearer about why I
might care in my later notes.
Another question: why still U-Boot 2019.10 instead of
updating to match, say, what is on modern RELEASE or
snapshot builds for the version of FreeBSD that you
are using on the RPi3B+ ? You seem to be using a
generally not-used combination by sticking to an older
U-Boot but updating other things.

Purely inertia. U-boot doesn't update via the normal make install
cycle, requiring a careful reading of notes for manual upgrade.
It was usable two months ago, though not without hiccups.

Post by Mark Millard via freebsd-arm
# strings start.elf | grep "VC_BUILD_ID_"

# strings start.elf | grep "VC_BUILD_ID_"
VC_BUILD_ID_USER: dc4
VC_BUILD_ID_TIME: 15:31:38
VC_BUILD_ID_BRANCH: master
VC_BUILD_ID_TIME: Jun 7 2018
VC_BUILD_ID_HOSTNAME: dc4-XPS13-9333
VC_BUILD_ID_PLATFORM: raspberrypi_linux
VC_BUILD_ID_VERSION: 4800f08a139d6ca1c5ecbee345ea6682e2160881 (clean)

That's old, but it used to work with this USB train.

Post by Mark Millard via freebsd-arm
Depending on what is reported: this might have
questions about the vintage used.

What alternatives have you tried?
*) Have you tried starting a boot sequence once with
https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/bootflow.md

Just tried it, usb scan still fails to find the mass storage device while
usb tree sees it.

Post by Mark Millard via freebsd-arm
This programs a One Time Programmable bit to cause
extra some time to be allowed. The
program_usb_boot_timeout=1 does not need to be kept
in place. After another reboot (to a RaspiOS
vcgencmd otp_dump | grep 66
should have bit 24 set in what it reports.
This might only be useful for option (B) below. I've
never found wording specifying the range of modes
that this adds time to. But it might apply to all
or most modes.

The "boot from USB" OTP was set during initial experiments
some time ago.

Post by Mark Millard via freebsd-arm
@) Have you tried any mixes of bootcode_delay=???,
boot_delay=???, boot_delay_ms=??? in config.txt ?
https://www.raspberrypi.org/documentation/configuration/config-txt/boot.md
bootcode_delay can add time before any start*.elf
is read from whatever media --but it is not clear if
the config.txt containing the bootcode_delay=???
and start*.elf can be from different media.
boot_delay and boot_delay_ms control waiting some
amount of time in start*.elf before loading the
next stage (U-Boot for FreeBSD's normal sequence).

I'm seeing lots of
MESS:00:00:01.300591:0: hdmi: HDMI:EDID error reading EDID block 0 attempt...
errors from u-boot. Up to now they didn't seem to matter.

Post by Mark Millard via freebsd-arm
A) Using a microsd card with only a modern bootcode.bin
should be able to have the rest of the material on the
USB device, including U-Boot. This way tends to have
more fixes/improvements than depending on the internal
support for direct USB booting: It supposedly works for
more USB devices.

I'm using that method on a Pi2, but haven't tried it on the
Pi3B since it's a dual-boot.

Post by Mark Millard via freebsd-arm
I do this on a RPi2B v1.1 that does not have support
(B) below. I have done this in temporary contexts
on a RPi3B where (B) below seemed to have a
problem for some reason. Dealing with booting from
(some?) powered hubs can be a type of context where
this proves useful.

Indeed, if the disk is moved to the hub it's overlooked by
usb tree in addition to being missed by usb scan.

Post by Mark Millard via freebsd-arm
https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/README.md
https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/msd.md
If the msdos file system that contains bootcode.bin also
contains an empty file named "timeout" it will wait
longer for a USB device to be ready. See "Known issues"
https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/msd.md
B) "The Raspberry Pi 3B+ supports USB mass storage boot out
of the box". So, unless some of those improvements
metioned in (A) turn out to be necessary, no microsd card
should be needed, presuming that all the required
material (including U-Boot) is on the USB drive.
not "out of the box") and all the RPi4B's. (Some RPi4B's
required eeprom updates to enable this.)
This does not give any control over getting extra time
for the USB device to be ready: it would first have to
have already read something from the device. But the
details of this might be better than the details of
some specific microsd card based RPi3B+ firmware boots.
I.e., it might be worth a try.
C) I will note that it is possible to have the FreeBSD
kernel also on the microsd card in a UFS partition and
to have "world" on the USB drive. This leads to only
FreeBSD's kernel and later using the USB drive. But
keeping the copy of the kernel and such on the
separate media is messier and atypical.
(Note: I do this sort if thing for the ROCK64 where
the dd'd U-Boot does not support USB for the FreeBSD
loader to put to use. I defer USB first-use to the
kernel.)

It looks to me like upgrading u-boot on the microSD should
be the first priority. There remains a lingering suspicion
that the USB-SATA adapter (a Startech) might be part of the
problem, but why it might stop working now is unclear.

Thanks for writing!

bob prohaska

Mark Millard via freebsd-arm

2021-06-06 17:34:26 UTC

Post by bob prohaska
For some time now I've been booting an Rpi3B+ using a microSD
card configured with
U-Boot 2019.10 (Mar 17 2020 - 22:01:14 -0700)

I'm unclear. Do you have the whole msdos file system
content on the microsd card and only the FreeBSD UFS
kernel+world on the USB drive? No msdos file system
on the USB drive?

It's a dual-boot system, with a complete FreeBSD-current install
on both USB and microSD storage.

How do you control which device provides kernel+world
if both have a kernel_world?

Purely inertia. U-boot doesn't update via the normal make install
cycle, requiring a careful reading of notes for manual upgrade.
It was usable two months ago, though not without hiccups.

I will note that U-Boot 2021.04 has problems with USB
booting for armv7. I've had to stick with 2020.10
U-Boot for such devices that I want to have USB based
booting, with an RPi2B v1.1 and a OPi+2E. Ports has
progressed to U-Boot 2021.04 .

Post by Mark Millard via freebsd-arm
# strings start.elf | grep "VC_BUILD_ID_"

# strings start.elf | grep "VC_BUILD_ID_"
VC_BUILD_ID_USER: dc4
VC_BUILD_ID_TIME: 15:31:38
VC_BUILD_ID_BRANCH: master
VC_BUILD_ID_TIME: Jun 7 2018
VC_BUILD_ID_HOSTNAME: dc4-XPS13-9333
VC_BUILD_ID_PLATFORM: raspberrypi_linux
VC_BUILD_ID_VERSION: 4800f08a139d6ca1c5ecbee345ea6682e2160881 (clean)
That's old, but it used to work with this USB train.

bootcode.bin has improvements since back then, as do
other things in the RPi firmware.

I suggest trying the same vintage that is on 13.0-RELEASE's
media. Later possibly newer from snapshot media if it has
more recent. In other words: a bias to vintages more other
folks are also using, where there is a wider knowledge
context for how things are going.

For reference, from a RPi4B (and a ROCK64), I am using:

# strings start4.elf | grep "VC_BUILD_ID_"
VC_BUILD_ID_USER: dom
VC_BUILD_ID_TIME: 12:10:40
VC_BUILD_ID_VARIANT: start
VC_BUILD_ID_TIME: Feb 25 2021
VC_BUILD_ID_BRANCH: bcm2711_2
VC_BUILD_ID_HOSTNAME: buildbot
VC_BUILD_ID_PLATFORM: raspberrypi_linux
VC_BUILD_ID_VERSION: 564e5f9b852b23a330b1764bcf0b2d022a20afd0 (clean)

Post by Mark Millard via freebsd-arm
Depending on what is reported: this might have
questions about the vintage used.

What alternatives have you tried?
*) Have you tried starting a boot sequence once with
https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/bootflow.md

Just tried it, usb scan still fails to find the mass storage device while
usb tree sees it.

The "boot from USB" OTP was set during initial experiments
some time ago.

RPi3B+'s have this set before they are shipped out.

I'm seeing lots of
MESS:00:00:01.300591:0: hdmi: HDMI:EDID error reading EDID block 0 attempt...
errors from u-boot. Up to now they didn't seem to matter.

Do you have a HDMI screen attached?

Whatever, bootcode_delay allows extra time for slow
HDMI devices.

I'm using that method on a Pi2, but haven't tried it on the
Pi3B since it's a dual-boot.

Which gets back to the question: how do you control which
device's kernel+world is booted?

Indeed, if the disk is moved to the hub it's overlooked by
usb tree in addition to being missed by usb scan.

I'd include RPi4B firmware updates to match 13.0
with that.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

bob prohaska

2021-06-07 18:40:16 UTC

Post by bob prohaska
It's a dual-boot system, with a complete FreeBSD-current install
on both USB and microSD storage.

How do you control which device provides kernel+world
if both have a kernel_world?

If the machine is powered up and not touched, it boots from the
microSD card. If boot is interrupted at the u-boot prompt it's
(was) possible to issue a
usb reset
command, at which point the external USB hard drive was discovered.
At that point issuing a
run bootcmd_usb0
command would boot kernel and mount world from the USB device.

Post by Mark Millard via freebsd-arm
I suggest trying the same vintage that is on 13.0-RELEASE's

I've written the 13.0-release image to a microSD card and tried
that. It reports a version of
U-Boot 2020.10 (Apr 09 2021 - 03:55:54 +0000)

Stopped at the u-boot prompt and given the
usb reset command, zero storage devices are found. However,
usb tree still reports

USB device tree:
1 Hub (480 Mb/s, 0mA)
| U-Boot Root Hub
|
+-2 Hub (480 Mb/s, 2mA)
|
+-3 Vendor specific (12 Mb/s, 90mA)
| FTDI FT232R USB UART AM00FGTR
|
+-4 Vendor specific (480 Mb/s, 2mA)
|
+-5 Mass Storage (480 Mb/s, 0mA)
ASMT ASM105x 12345678D558

The mismatch between what usb reset and usb tree report strikes me
as extremely puzzling.

As an experiment, I tried booting with no microSD card installed at
all. This got confusing; u-boot still came up, apparently from the
USB hard drive. The USB disk is still not found by usb scan, but it
is recognized by usb tree.

As a final test I tried connecting a monitor that had been used with
this Pi previously. The monitor's presence made no difference, the
display looked normal.

A hands-off boot to single user is successful from the microSD card.
It's possible to see and access the USB hard drive.

At this point I'm beginning to suspect some obscure mischief with the
USB-SATA adapter, based only on earlier intermittent problems detecting
the USB hard drive. In those cases a few repeats of usb scan found
the disk and allowed booting from it.

Thanks for following this goose chase!

bob prohaska

Mark Millard via freebsd-arm

2021-06-07 20:09:52 UTC

Post by bob prohaska
It's a dual-boot system, with a complete FreeBSD-current install
on both USB and microSD storage.

How do you control which device provides kernel+world
if both have a kernel_world?

I see I force to to tell me what you had already reported.
Sorry for that.

As for your manual usb commands . . .

Looking around on the web I see reports of the:

Request Sense returned 02 04 01

(and the matching Device NOT ready) mean that the
problem will occur and that repeating usb start
or usb reset again until it does not report that
leads to things working.

But I've also seen other, more complete information
indicating that there is a environment setting
(showing an example value):

usb_ready_retry=5

to set up before the usb restart (or usb start)
command. It deals with the issue more explicitly
for slow devices.

Another one is (units: msec):

usb_pgood_delay=10000

There are also device that have problems with
large transfers and require extra protocol to
deal with transfer problems before they will
work again, U-Boot not doing that.

usb_max_blk=20

sets a old historical value that generally
just works for such devices form what I read.

I see no indication that other usb commands are
worthwhile until one has avoided that "Request
Sense returned 02 04 01" message for usb reset
(a.k.a. usb start).

The reports of this sort of thing are not limited
to RPi's and go back to at least 2014.

If I understand correctly, usb_ready_retry and
usb_pgood_delay and usb_max_blk are part of
normal U-Boot builds these days. But I'm not
certain of that.

I will note:

QUOTE
Common USB Commands:
- usb start:
- usb reset: (re)starts the USB. All USB devices will be
initialized and a device tree is build for them
- usb tree: shows all USB devices in a tree like display
. . .

Storage USB Commands:
- usb scan: scans the USB for storage devices.The USB must be
running for this command (usb start)
. . .
END QUOTE

Note that the wording indicates that having the tree is not
the same as having classified the storage devices: storage
classification is an seprate step.

"usb reset" seems to automatically do a scan. But, still,
the tree listing things that "usb storage" or "usb part"
would not is apparently normal util after a successful
"usb scan" has occured (possibly automatically executed).

Post by Mark Millard via freebsd-arm
I suggest trying the same vintage that is on 13.0-RELEASE's

See my earlier notes about tree vs. storage and part information.

Post by bob prohaska
As an experiment, I tried booting with no microSD card installed at
all. This got confusing; u-boot still came up, apparently from the
USB hard drive.

The RPi internal code is handling the USB drive
initially and loading the RPifirmware from the
USB drive. That firmware is in turn loading
U-Boot in this case.

It is when U-Boot tries to take over and set up
the USB drive for its own use that things are
having the problem.

Post by bob prohaska
The USB disk is still not found by usb scan, but it
is recognized by usb tree.

See the earlier notes about tree vs. storage and
part information.

Post by bob prohaska
As a final test I tried connecting a monitor that had been used with
this Pi previously. The monitor's presence made no difference, the
display looked normal.
A hands-off boot to single user is successful from the microSD card.
It's possible to see and access the USB hard drive.

This sequence by-passes U-Boot having to have usb
storage operational. The FreeBSD kernel is better
behaved for handling the properties of your devices
involved.

Post by bob prohaska
At this point I'm beginning to suspect some obscure mischief with the
USB-SATA adapter, based only on earlier intermittent problems detecting
the USB hard drive. In those cases a few repeats of usb scan found
the disk and allowed booting from it.

But did the problem cases show:

Request Sense returned 02 04 01

or:

Device NOT ready

? Or were such details different?

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

bob prohaska

2021-06-07 23:52:48 UTC

Post by bob prohaska
Request Sense returned 02 04 01
(and the matching Device NOT ready) mean that the
problem will occur and that repeating usb start
or usb reset again until it does not report that
leads to things working.
But I've also seen other, more complete information
indicating that there is a environment setting
usb_ready_retry=5
to set up before the usb restart (or usb start)
command. It deals with the issue more explicitly
for slow devices.
usb_pgood_delay=10000

Presto! using editenv usb_pgood_delay prompted for input, typing 10000
and hitting return set the value and the disk was found.

It looks like the setting can only be saved to microSD. With
no card saveenv reports
Saving Environment to FAT... Card did not respond to voltage select!
Failed (1)

Post by bob prohaska
There are also device that have problems with
large transfers and require extra protocol to
deal with transfer problems before they will
work again, U-Boot not doing that.
usb_max_blk=20
sets a old historical value that generally
just works for such devices form what I read.
I see no indication that other usb commands are
worthwhile until one has avoided that "Request
Sense returned 02 04 01" message for usb reset
(a.k.a. usb start).
The reports of this sort of thing are not limited
to RPi's and go back to at least 2014.
If I understand correctly, usb_ready_retry and
usb_pgood_delay and usb_max_blk are part of
normal U-Boot builds these days. But I'm not
certain of that.
QUOTE
- usb reset: (re)starts the USB. All USB devices will be
initialized and a device tree is build for them
- usb tree: shows all USB devices in a tree like display
. . .
- usb scan: scans the USB for storage devices.The USB must be
running for this command (usb start)
. . .
END QUOTE
Note that the wording indicates that having the tree is not
the same as having classified the storage devices: storage
classification is an seprate step.

I didn't appreciate how independent u-boot's usb commands are.
Info and tree could see the disk but dev, storage and reset didn't.

Post by bob prohaska
As an experiment, I tried booting with no microSD card installed at
all. This got confusing; u-boot still came up, apparently from the
USB hard drive.

The RPi internal code is handling the USB drive
initially and loading the RPifirmware from the
USB drive. That firmware is in turn loading
U-Boot in this case.
It is when U-Boot tries to take over and set up
the USB drive for its own use that things are
having the problem.

[regarding earlier, intermittent failures]

Post by bob prohaska
Request Sense returned 02 04 01
Device NOT ready
? Or were such details different?

Not sure. When I saw that the disk wasn't detected, I waited a
few seconds and re-ran usb reset. Within a few tries the disk
was found.
It's unclear to me what changed, probably something I did, but
setting u-boot environment variable usb_pgood_delay=10000 seems
to have put matters right again.

Thank you!

bob prohaska

Mark Millard via freebsd-arm

2021-06-08 02:06:14 UTC

Presto! using editenv usb_pgood_delay prompted for input, typing 10000
and hitting return set the value and the disk was found.

Cool.

Post by bob prohaska
It looks like the setting can only be saved to microSD. With
no card saveenv reports
Saving Environment to FAT... Card did not respond to voltage select!
Failed (1)

I didn't appreciate how independent u-boot's usb commands are.
Info and tree could see the disk but dev, storage and reset didn't.

Looking in some actual U-Boots for other reasons, I
found no "usb scan" command. usb start (the first time)
and usb reset both did a scan automatically and the
help description indicated that scanning explicitly.

The behavior that you got, however, suggests that
internally the scan is still separate (and such
can be observed via some types of failures having
tree information left behind but not storage or
partition information).

Post by bob prohaska
As an experiment, I tried booting with no microSD card installed at
all. This got confusing; u-boot still came up, apparently from the
USB hard drive.

The RPi internal code is handling the USB drive
initially and loading the RPifirmware from the
USB drive. That firmware is in turn loading
U-Boot in this case.
It is when U-Boot tries to take over and set up
the USB drive for its own use that things are
having the problem.

[regarding earlier, intermittent failures]

Post by bob prohaska
Request Sense returned 02 04 01
Device NOT ready
? Or were such details different?

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

Mark Millard via freebsd-arm

2021-06-06 18:24:39 UTC