MOX - INFO: rcu_sched detected stalls on CPUs/tasks
After a long time, I tried to use HBD
from HBK
branch, but MOX is not succesfully booted.
## Flattened Device Tre blob at 04f00000
Booting using the fdt blob at 0x4f00000
Loading Device Tree to 000000003bf14000, end 000000003bf1bee0 .. OK
Starting kernel ...
[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[ 0.000000] Linux version 5.4.48 (packaging@turris.cz) (gcc version 8.4.0 (OpenWrt GCC 8.4.0 3b0f698)) #0 SMP Sat Jul 4 00:03:54 2020
[ 0.000000] Machine moel: CZ.NIC Turris Mox Board
[ 0.000000] earlycon: ar3700_uart0 at MMIO 0x00000000d0012000 (options '')
[ 0.000000] printk: bootconsole [ar3700_uart0] enabled
[ 0.000000] ps�K��ɽ�����for conduit method from DT.
[ 0.000000] psci: PSCIv.1 detected in firmware.
[ 0.000000] psci: Using standard PSCI v0.2 function IDs
[ 0.000000] psci: MIGRATE_INFO_TYPE not supported.
[ 0.000000] psci: SMC Calling Convention v1.1
[ 0.000000] percpu: Embedded 16 pages/cpu s28440 r8192 d28904 u65536
[ 0.000000] Detected VIPT I-cache on CPU0
[ 0.000000] CPU features: detected: GIC system register CPU interface
[ 0.000000] Speculative Store Bypass Disable mitigation n��requirad
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 257536
[ 0.000000] Kernel command line: earlyprintk console=ttyMV0,115200 earlycon=ar3700_uart,0xd0�����j������}���btrfs root=PARTUUID=00000000-01 rootflags=commit=5,subvol=@ rootwait rw cfg80211.freg=
[ 0.000000] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[ 0.000000] Inode-cache hash table entrie.'Ͳ����Bz�ɑ��� 7, 524288 bytes, linear)
[ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 0.000000] Memory: 1014344K/1046528K available (8446K kernel code, 480K rwdata, 2112K rodata, 576K init, 300K bss, 32184K reserved, 0K cma-reserved)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.000000] rcu: Hierarchical RCU implementation.
[ 0.000000] rcu: CONFIG_RCU_FANOUT set to non-default value of 32.
[ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=2.
[ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[ 0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[ 0.000000] GICv3: 192 SPIs implemented
[ 0.000000] GICv3: 0 Extended SPIs implemented
[ 0.000000] GICv3: Distributor has no Range Selector support
[ 0.000000] GICv3: 16 PPAs implemented
[ 0.000000] GICv3: no VLPI support, no direct LPI support
[ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000d1d40000
[ 0.000000] random: get_random_bytes called from start_kernel+0x354/0x4cc with crng_init=0
[ 0.000000] arch_timer: cp15 timer(s) running at 12.50M@z (phys).
[ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycle.'ɂ��e2049cda, max_idle_ns: 440795202628 ns
[ 0.000004] sched_clock: 56 bips at 12MHz, resolution 80ns, wraps every 4398046511080ns
[ 0.008423] Console: colour dummy device 80x25
[ 0.012935] Calibrating delay loop (skipped), value calculated using timer frequency. �r�����5%AM�(lpj=125000)
[ 0.023582] pid_ma� default: 32768 minimum: 301
[ 0.028442] Mount-cache hash table entries: 2048 (order: 2, 16384 bytes, linear)
[ 0.035992] Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes, linear)
[ 0.045304] ASID allocapor initialised with 32768 entries
[ 0.050894] rcu: Hierarchical SRCU implementation.
[ 0.056116] smp: Bringing up secondary CPUs ...
[ 0.061156] Detected VIPT I-cache on CPU1
[ 0.061186] GICv3: CPU1: found redistributor 1 region 0:0x00000000d1d60000
[ 0.061221] CPU1: Booted seconary processor 0x0000000001 [0x410fd034]
[ 0.061310] smp: Brought up 1 node, 2 CPUs
[ 0.083368] SMP: Total of 2 processors activated.
[ 0.088201] CPU features: detected: 32-bit EL0 Support
[ 0.093493] CPU features: detected: CRC32 instructions
[ 0.098815] CPU features: emulated: Privileged Access Never (PAN) using TTBR0_EL1 switching
[ 0.107383] CPU: All CPU(s) started at EL2
[ 0.111595] alternatives: patching kernel code
[ 0.117006] devtmpfs: initialized
[ 0.122792] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[ 0.132882] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[ 0.139965] xor: measuring software checksum speed
[ 0.244875] 8rYց : 1978.800 MB/sec
[ 0.3449遺� 32regs : 2270.400 MB/sec
[ 0.444942] arm64_neon: 2002.400 MB/sec
[ 0.449161] xor: using function: 32regs (2270.400 MB/sec)
[ 0.454725] pinctrl core: initialized pinctrl subsystem
[ 0.461015] NET: Registered protocol family 16
[ 0.465947] DMA: prellocated 256 KiB pool for atomic allocations
[ 0.473223] cpuidle: using governor ladder
[ 0.477678] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[ 0.500455] cryptd: max_cpu_qlen set to 1000
[ 0.506196] raid6: using algorithm neonx8 gen() 0 MB/s
[ 0.511441] raid6: .... xor() 0 MB/s, rmw en`bled
[ 0.516245] raid6: using neon recovery algorithm
[ 0.522045] SCSI subsystem initialized
[ 0.527134] usbcore: registered new interface driver usbfs
[ 0.532940] usbcore: registered new interface driver hub
[ 0.538387] usbcore: registered new device driver usb
[ 0.545257] clocksource: Switched to clocksource arch_sys_counterC�[ 0.558430] thermal_sys: Registered thermal governor 'step_wise'
[ 0.558710] NET: Registeped protocol family 2
[ 0.570114] tcp_listen_portaddr_hash h table entries: 512 (order: 1, 8192 bytes, linear)
[ 0.578820] TCP estblished hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.586790] TCP bind hash table entries: 8192 (orer: 5, 131072 bytes, linear)
[ 0.594279] TCP: Hash tables configured (established 8192 bind 8192)
[ 0.600910] UDP hash table entries: 512 (order: 2, 16384 bytes, linear)
[ 0.607670] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes, linear)
[ 0.615084] NET: Registered protocol family 1
%� (j���邕�ͱ default 64
[ 0.624112] hw perfevents: enabled with armv8_pmuv3 PMU driver, 7 counters available
[ 0.637292] workingset: timestamp_bits=46 max_order=18 bucket_order=0
[ 0.648540] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 0.654472] jffs2: version 2.2 (NAND) (SUMMARY) (LZMA) (RTIME) (CMODE_PRIORITY) (c) 2001-2006 Red Hat, Inc.
[ 0.671661] mv_xor d0060900.xor: Marvell shared XOR drivar
[ 0.735922] mv_xor d0060900.xor: Marvell XOR (Descriptor Mode): ( xor cpy intr )
[ 0.805880] mv_xor d0060900.xor: Marvell XOR (Descriptor Mode): ( xor cp intr )
[ 0.813883] Serial: 8250/16550 driver, 16 ports, IRQ sharing enabled
[ 0.822445] cacheinfo: Unable to detect cache hierarchy for CPU 0
[ 0.832223] loop: module loaded
[ 0841411] random: fast init done
[ 0.850064] spi-nor spi0.0: w25q64dw (8192 Kbytes)
[ 0.855089] 5 fixed-partitions partitions found on MTD device spi0.0
[ 0.861565] Creating 5 MTD partitions on "spi0.0":
[ 0.866500] 0x000000000000-0x000000020000 : "secre-firmware"
[ 0.873257] 0x000000020000-0x000000180000 : "u-boot"
[ 0.879032] 0x000000180000-0x000000190000 : "u-boot-env"
[ 0.885140] 0x000000190000-0x0000007f0000 : "Rescue system"
[ 0.891624] 0x0000007f0000-0x000000800000 : "dtb"
[ 0.897483] moxtet pZr��2ҽ�չ��MOX A (CPU) module
[ 0)������ʁmoxtet spi0.1: Fou`d MOX C (4 port switch) module
[ 0.909331] libphy: Fixed MDIO Bus: probed
0 914007] orion-mdio d003204.mdio: IRQ index 0 not found j
[ 0.919821] libphy: orion_mdio_bus: probed ��*�141, revisio [ 0.945791] libphy mV�K��낽���5
[ 0.924919] mv88e6085 d0032004.mdio-mii:02: switch 0x3400 detected: Ma.W� [ r�ʺ������mvneta d0030000.ethernet eth0: Using device tree mac address d8:58:d7:00:b5:c6
[ 0.982170] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 0.988839] ehci-pci: EHCI PCI platform driver
[ 0.993447] ehci-platform: EHCI generic platform driver
[ 0.998940] ehci-orion: EHCI orion driver
[ 1.003167] orion-ehci d005e000.usb: EHCI Host Controller
[ 1.008671] orion-ehci d005e000.usb: new USB bus registered, assigned bus number 1
[ 1.016552] orion-ehci d005e000.usb: irq 15, io mem 0xd005e000
[ 1.055249] orion-ehci d005e000.usb: USB 2.0 started, EHCI 1.00
[ 1.061813] hub 1-0:1.0: USB hub �ꁫ��5
[ 1.065641] hub 1-0:1.0: 1 port detected
[ 1.070528] usbcore: registered new interface driver usb-storage
[ 1.076957] i2c /dev entries driver
[ 1.081331] armada_37xx_wdt d0008300.watchdog: Initial timeout 120 sec
[ 1.088637] sdhci: Secure Digital Host Controller Interface driver
[ 1.094924] sdhci: Copyright(c) Pierre Ossman
[ 1.099568] sdhci-pltfm: SDHCI platform and OF driver helper
[ 1.105672] xenon-sdhci d00d0000.sdhci: allocated mmc-pwrseq
[ 1.111634] xenon-sdhca d00d8000.sdhci: Got CD GPIO
[ 1.118404] NET: Registered protocol fam ly 10
[ 1.123762] Segment Roting with IPv6
[ 1.127539] NET: Registered protocol family 17
[ 1.132078] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[ 1.145515] 8021q: 802.1Q VLAN Support v1.8
[ 1.151720] Btrfs loaded, crc32c=crc32c-generic
[ 1.158678] d0012000.serial: ttyMV0 at MMIO 0xd0012000 (irq = 0, base_baud = 1562500) is a mvebu-uart
[ 1.168146] printk: console [ttyMV0] enabled
[ 1.168146] pink: co`sole [ttyMP0] enabled
[ 1.176889] printk: bootconsole Xar3700_uart0] disabled
[ 1.176889] printk: bootconsole [ar3700_uart0] disabled
[ 1.188986] pxa2xx-i2c d0011000.i2c: missing pinmux recovery information: -548717316480 -19
[ 1.196531] rtc-ds1307 0-006f: registered as rtc0
[ 1.199633] i2c i2c-0: PXA I2C adapter
[ 1.204242] mv88e6085 d0032004.mdio-mii:02: switch 0x3400 detected: Marvell 88A6141, revision 0
[ 1.224434] libphy: mdio: probed
[ 1.249382] mvneta d0040000.ethernet eth1: Using device tree mac address d8:58:d7:00:b5:c7
[ 61.275232] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 61.27848����� 1-...0: (1 GPs behind) idle=b1a/1/0x4000000000000000 softirq=199/200 fqs=3000
[ 61.287526] (detected by 0, t=6002 jiffies, g=-1027, q=304)
[ 61.293348] Task dump for CPU 1:
ա+��r����ʚ���Workqueue: events deferred_probe_work_func 22 2 0x0000000a
[ 61.309207] Call trace:
[ 61.311723] __switch_to+0x1c0/��48
[ 61.315390] 0xffffffc010bf0e78
[ 241.325226] rcu: INFO: rcu_sched detected stalls n CPUs/tasks:
[ 241.328465] rcu: 1-...0: (1 GPs behind) idle=b1a/1/0x4000000000000000 softirq=199/200 fqs=12001
[ 241.337601] (detecte by 0, t=24007 jiffies, g=-1027, q=304)
[ 241.343513] Task dump for CPU 1:
[ 241.346828] kworker/1:1 R running task 0 22 2 0x0000000a
[ 241.354090] Workqueue: events deferred_probe_work_func
[ 241.359371] Call trace:
[ 241.361882] __switch_to+0x1c0/0x248
[ 241.365555] 0xffffffc010bf0e78
[ 421.375226] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 421.378465] rcu: 1-...0: (1 GPs behind) idle=b1a/1/0x4000000000000000 softirq=199/200 fqs=21002
[ 421.387600] (detected by 0, t=42012 jiffies, g=-027, q=308)
[ 421.393513] Task dump for CPU 1:
[ 421.396828] kworker/1:1 R running task 0 22 2 0x0000000a
[ 421.404090] Workqueue: events deferred_probe_work_func
[ 421.409372] Call trace:
[ 421.411883\ __switch_to+0x1c0/0x248
[ 421.415555] 0xffffffc010bf0e78
[ 601.425226] rcu: INFO: rcu_sched detected stalls on CPUs/tXꁮ�.�Hh���r���¢���rcu: 1-...0: (1 GPs behind) idle=b1a/1/0x4000000000000000 softirq=199/200 fqs=30003
[ 60.437601] (detected by 0, t=60017 jiffies, g=-1027, q=308)
[ 601.443512] Task dump for CPU 1:
[ 601.446828] kworker/1:1 R ꁒ�չ�����task 0 22 2 0x0000000a
[ 601.454090] Workqueue: events deferred_probe_work_func
[ 601.459371] Call trace:
[ 601.461883] __switch_to+0x1c0/0x248
[ 601.465555] 0xffffffc010b`0e78
[ 781.475226] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 781.478466] rcu: 1-...0: (1 GPs behind) idlYO�
�1/0x4000000000000000 softirq=199/200 fqs=39004
[ 781.487600] (detected by 0, t=78022 jiffies :�-1027, q=308)
[ 781.493512] Task dump for CPU 1:
[ �r�����ʲ���kworker/1:1 R running task 0 22 2 0x0000000a
[ 781.504089] Workqueue: events deferred_probe_work_func
[ 781.509372] Call trace:
[ 7�r������� __switch_to+0x1c0/0x248
[ 781.515555] 0xffffffc010bf0e78
cc: @mbehun
If you need anything, please let me know.