2024-05-02 08:21:05 <ncopa> clandmeter: I have pushed new sophgo kernels based on 6.1.89. I disabled HIGHMEM. there is a boot option for hiding RVV support that we could try. Are you available to help with a reboot today?
2024-05-02 08:21:28 <clandmeter> until 12
2024-05-02 08:21:39 <ncopa> ok!
2024-05-02 08:22:25 <clandmeter> why is highmem disabled?
2024-05-02 08:22:52 <ncopa> it was causing the boot failure last time we tried
2024-05-02 08:23:17 <ncopa> https://github.com/sophgo/linux-riscv/issues/117#issuecomment-2088444437
2024-05-02 08:23:41 <clandmeter> we dont need it?
2024-05-02 08:23:45 <ncopa> this is a 64bit machine, and to my understanding it can address 39 bits(?)
2024-05-02 08:23:57 <clandmeter> i remember i played with it
2024-05-02 08:24:01 <ncopa> so it should be able to address 256G mem at least
2024-05-02 08:24:08 <clandmeter> as it hides some option iiuc
2024-05-02 08:24:26 <clandmeter> if we dont need it and config is sane, im ok
2024-05-02 08:24:37 <ncopa> in theory we shouldnt need highmem for 64bit arches in theory
2024-05-02 08:25:00 <ncopa> the other option was to re-apply 3 commits that was previously reverted
2024-05-02 08:25:20 <ncopa> which I did first. but then I thought its easier to stay close to upsteram
2024-05-02 08:25:42 <ncopa> so I disabled HIGHMEM instead, so I don't need re-apply any patches
2024-05-02 08:25:54 <ncopa> then I merged in v6.1.89. latest 6.1 kernel
2024-05-02 08:26:30 <ncopa> how do I add a boot option to kernel cmdline?
2024-05-02 08:26:53 <ncopa> in /etc/extlinux/extlinux.conf?
2024-05-02 08:28:03 <mps> ncopa: 'append ...'
2024-05-02 08:28:27 <clandmeter> yes extlinux
2024-05-02 08:28:30 <ncopa> ok
2024-05-02 08:28:32 <clandmeter> linuxboot reads it
2024-05-02 08:28:41 <ncopa> I'm adding hide_v0p7_ext
2024-05-02 08:28:45 <mps> ah sorry, misread question
2024-05-02 08:28:56 <clandmeter> ok
2024-05-02 08:29:13 <ncopa> ok to reboot now?
2024-05-02 08:29:18 <clandmeter> which one?
2024-05-02 08:29:29 <ncopa> nld-bld-2
2024-05-02 08:29:30 <clandmeter> #2 has consile
2024-05-02 08:29:39 <clandmeter> ok go ahead ill follow
2024-05-02 08:29:56 <ncopa> rebooting
2024-05-02 08:30:27 <ncopa> there it stopped to respond to ping
2024-05-02 08:30:40 <ncopa> oh. im a bit stupid. I should have backed up old /boot
2024-05-02 08:32:11 <clandmeter> its not looking good
2024-05-02 08:32:16 <ncopa> aw
2024-05-02 08:32:16 <clandmeter> no screen after fw
2024-05-02 08:33:26 <ncopa> :-/
2024-05-02 08:33:52 <clandmeter> bummer, i need to debug from uart
2024-05-02 08:34:01 <clandmeter> need to get it out of the server room
2024-05-02 08:34:08 <ncopa> im sorry
2024-05-02 08:44:08 <clandmeter> bummer, my windows pc does not have a driver for the uart.
2024-05-02 08:48:52 <clandmeter> kernel crashes
2024-05-02 08:49:49 <clandmeter> https://tpaste.us/kXRZ
2024-05-02 08:49:54 <clandmeter> ncopa: ^
2024-05-02 08:50:10 <clandmeter> crashes very early
2024-05-02 08:50:51 <clandmeter> problem is i do no have a recover usb
2024-05-02 09:04:30 <ncopa> drats
2024-05-02 09:04:42 <clandmeter> im writing an old image to usb
2024-05-02 09:04:45 <clandmeter> see if that will boot
2024-05-02 09:05:00 <clandmeter> we should put a recovery image somewhere
2024-05-02 09:05:21 <ncopa> we also need some way to revert to older kernel
2024-05-02 09:05:24 <clandmeter> i think i had one which was booting from memmory
2024-05-02 09:05:39 <clandmeter> do you have a copy of the old kernel pkg?
2024-05-02 09:06:31 <ncopa> i think it may be on that machine
2024-05-02 09:07:00 <ncopa> on /var/lib/lxc/ncopa-edge-riscv64/rootfs/home/ncopa/packages/testing
2024-05-02 09:07:27 <clandmeter> usb is ready, but truck arrives now so i need to assist 
2024-05-02 09:07:38 <ncopa> apk add -u --repository  /var/lib/lxc/ncopa-edge-riscv64/rootfs/home/ncopa/packages/testing 'linux-sophgo<6.1.80' --allow-untrusted
2024-05-02 09:17:14 <mps> few days ago I thought to make script which will copy and set current working kernel to 'rescue kernel' and add option for it in bootloader (because we can't have installed different versions of kernel pkg)
2024-05-02 09:17:43 <mps> but didn't had time yet to start on this
2024-05-02 09:18:39 <mps> problem will be dtb dir
2024-05-02 09:19:02 <clandmeter> ok this image is not working
2024-05-02 09:40:02 <clandmeter> online again
2024-05-02 09:43:40 <clandmeter> ncopa: there is a .72 kernel in root
2024-05-02 09:54:30 <ncopa> i cannot ping it?
2024-05-02 09:55:23 <clandmeter> no
2024-05-02 09:55:29 <clandmeter> its booting from usb from my disk
2024-05-02 09:55:30 <clandmeter> desk
2024-05-02 09:57:37 <ncopa> ok
2024-05-02 09:57:56 <ncopa> the .72 kernel in root is probably ok
2024-05-02 09:58:45 <clandmeter> rebooting now
2024-05-02 09:59:25 <clandmeter> looks better
2024-05-02 10:01:53 <clandmeter> online in a minute or so
2024-05-02 10:02:23 <ncopa> so disabling HIGMEM was no success
2024-05-02 10:02:31 <clandmeter> so i have a rv iso on disk
2024-05-02 10:02:35 <clandmeter> this can be copied to usb
2024-05-02 10:02:41 <clandmeter> should allow for recovery
2024-05-02 10:02:53 <clandmeter> it would be nice to copy it to sd
2024-05-02 10:03:00 <clandmeter> and have it as second boot order
2024-05-02 10:03:21 <clandmeter> but the docs dont say how to manage the boot order
2024-05-02 10:03:30 <clandmeter> u-root/linuxboot
2024-05-02 10:11:37 <ncopa> thank you for your help clandmeter, and sorry for the fuzz
2024-05-02 10:12:06 <ncopa> im gonna build another kernel, with HIGHMEM enabled but the 3 commits re-applied
2024-05-02 10:12:44 <ncopa> but I will not reboot anything today
2024-05-02 14:41:19 <iggy> is there a better place to house these servers while they're being brought up fully that doesn't require so much manual work/proximity?
2024-05-02 14:43:04 <ikke> clandmeter wanted to setup an APU or something like that to provide OOB access
2024-05-02 14:43:38 <ikke> clandmeter: can't you use the starfive v2 board for that? :P
2024-05-02 14:44:53 <iggy> a serial ring between all of them
2024-05-02 14:44:56 <iggy> 
ACTION runs

2024-05-02 15:23:16 <mps> ikke: VF1 would be enough, why waste VF2
2024-05-03 08:11:09 <clandmeter> i think i didnt get one :)
2024-05-03 08:11:17 <clandmeter> i have an old one never touched
2024-05-03 08:11:20 <clandmeter> still in the can
2024-05-03 08:13:46 <mps> clandmeter: are you kidding?
2024-05-03 08:14:07 <clandmeter> nope
2024-05-03 08:14:13 <clandmeter> no time to play
2024-05-03 08:14:35 <mps> you didn't received VF1?
2024-05-03 08:14:45 <clandmeter> im not sure what i have
2024-05-03 08:14:53 <clandmeter> i only have 1 of those small boards
2024-05-03 08:15:04 <clandmeter> the first one we received
2024-05-03 08:15:11 <mps> omg, you organized this for rest of us
2024-05-03 08:15:15 <clandmeter> the later i didnt sign up for as i didnt use the firstr one
2024-05-03 08:15:34 <clandmeter> ofc, i arrange so you guys can work on it :)
2024-05-03 08:15:42 <mps> aha, then you have VF1
2024-05-03 08:15:49 <clandmeter> but i did work on the pioneer
2024-05-03 08:17:08 <mps> I can prepare mmc image for VF1 for you, ofc if have time to test it and make it as 'out-of-band' control device for Pioneer
2024-05-03 08:17:32 <clandmeter> its not very useful imho
2024-05-03 08:17:33 <mps> I bet ikke willl make interface
2024-05-03 08:17:57 <clandmeter> the bootloader is only showing over vga
2024-05-03 08:18:05 <mps> oh
2024-05-03 08:18:06 <clandmeter> and you need to reset it when it hangs
2024-05-03 08:18:27 <clandmeter> im not sure how to change the bootloader to serial
2024-05-03 08:18:35 <mps> I thought you sent me boot logs from serial console
2024-05-03 08:18:56 <clandmeter> ringbuffer does some part over serial and vga
2024-05-03 08:20:20 <mps> I saw in docs pioneer could use serial console for boot control
2024-05-03 08:20:34 <mps> ofc, I didn't tested
2024-05-03 09:21:58 <clandmeter> mps: which docs?
2024-05-03 09:25:12 <mps> clandmeter: one of these you send me urls
2024-05-03 09:27:43 <mps> https://milkv.fyi/en/bootloaders.html mentions serial console in extlinux.conf
2024-05-04 16:49:44 <vkrishn> clandmeter: re: ^need to get it out of the server room ... , there is a way to lessen this problem, https://wiki.insteps.net/Mobile/As-booting-device-for-linux (if the device is network enabled)
2024-05-04 16:50:45 <vkrishn> pls let me know if , it does require installing twrp first, any old device from up from 2013 should have atleast one lun device
2024-05-04 16:51:25 <vkrishn> i can try enabling wifi remotely
2024-05-04 16:52:09 <vkrishn> some devices i have seen has upto 4 lun devices, in theory linux allows upto 512
2024-05-04 16:53:28 <vkrishn> twrp maynot be needed, if root can be achieved otherwise
2024-05-06 09:36:31 <ncopa> seem like we need to either re-enable highmem or reduce the physical memory to 126G or below. https://github.com/sophgo/linux-riscv/issues/117#issuecomment-2090225289
2024-05-06 09:37:03 <ncopa> I rebuilt the kernel with HIGHMEM re-enabled, and with the 3 patches re-applied
2024-05-06 11:00:18 <clandmeter> ncopa: what is wrong with the .72 kernel?
2024-05-06 11:00:25 <clandmeter> except the vector stuff
2024-05-06 11:02:31 <ncopa> the vector stuff and nvme hickups. And I'd like to be able to keep kernels updated, so we get the latest security fixes as well
2024-05-06 11:04:04 <clandmeter> it will be challenging with the stability it currently offers. there is no stable branch or similar. 
2024-05-06 11:04:14 <clandmeter> every new kernel is trial and error
2024-05-06 11:06:11 <ncopa> do you know if the .72 kernel has the hide_v0p7_ext boot option?
2024-05-06 11:07:27 <clandmeter> not sure
2024-05-06 11:07:38 <clandmeter> i guess you can see from dmesg
2024-05-06 11:08:07 <ncopa> seems like 172.16.30.3 is still down?
2024-05-06 11:08:13 <ncopa> does not answer ping
2024-05-06 11:08:26 <clandmeter> yup
2024-05-06 11:08:28 <clandmeter> seems so
2024-05-06 11:08:30 <clandmeter> or its nvme
2024-05-06 11:08:40 <clandmeter> i was just trying to connect
2024-05-06 11:14:20 <clandmeter> rebooted
2024-05-06 11:14:26 <ncopa> and its back
2024-05-06 11:16:03 <clandmeter> [    0.000000] Hide vector 0.7 extension
2024-05-08 08:11:43 <ncopa> clandmeter: I'd like to fix the kernel package we have. I think it should work now. Do you mind if I give it another try?
2024-05-09 06:46:52 <clandmeter> ncopa: im not really available upcoming days
2024-05-09 06:46:54 <clandmeter> maybe tomorrow
2024-05-09 06:49:56 <ncopa> ok
2024-05-09 06:49:58 <ncopa> thanks
2024-05-09 06:50:09 <ncopa> we can try next week as well
2024-05-09 07:05:39 <clandmeter> im not available next week
2024-05-09 07:06:43 <clandmeter> i was thinking of connecting both boxes via usb to each other
2024-05-09 07:07:04 <clandmeter> you could attache to the serial port via the other machine
2024-05-10 08:45:15 <clandmeter> ncopa: you want to try new kernel?
2024-05-10 08:45:43 <clandmeter> i think we are missing some driver for serial access
2024-05-10 08:47:53 <clandmeter> I linked #1 to #2 but it does not create a serial device
2024-05-10 08:51:39 <clandmeter> think we are missing CONFIG_USB_SERIAL
2024-05-10 09:03:34 <ncopa> ok.. let me rebuild it
2024-05-10 09:05:07 <ncopa> which usb serial drivers should we enable? https://tpaste.us/5XbN
2024-05-10 09:05:10 <clandmeter> if this kernel works just try it now
2024-05-10 09:05:18 <clandmeter> and you can install the rebuild one later
2024-05-10 09:05:23 <clandmeter> i guess that will just work
2024-05-10 09:05:34 <ncopa> i hope so.
2024-05-10 09:05:34 <ncopa> ok
2024-05-10 09:05:58 <clandmeter> # CONFIG_USB_SERIAL is not set
2024-05-10 09:06:06 <ncopa> lest install and reboot. and leets start with nld-bld-2
2024-05-10 09:06:09 <clandmeter> why you have m?
2024-05-10 09:06:27 <ncopa> kernel modules. those are the ones we have in linux-lts x86_&4
2024-05-10 09:06:38 <clandmeter> ah ok
2024-05-10 09:06:51 <clandmeter> the serial driver is working
2024-05-10 09:07:00 <clandmeter> just not showing up the device
2024-05-10 09:07:06 <clandmeter> i guess of the missing usb_serial
2024-05-10 09:07:23 <clandmeter> go ahead with #2
2024-05-10 09:07:31 <clandmeter> ill watch the console
2024-05-10 09:08:09 <ncopa> (40/61) Upgrading linux-sophgo (6.1.72_git20240119-r5 -> 6.1.90-r0)
2024-05-10 09:08:11 <ncopa> rebooting now
2024-05-10 09:09:11 <mps> serial should be '=y', so output of boot msg can be seen early on console 
2024-05-10 09:09:30 <ncopa> btw, i think there is something in the firewall setup that may need be fixed. some missing FORWARD rule. but its not critical
2024-05-10 09:10:39 <ncopa> clandmeter: it now reponds to ping
2024-05-10 09:11:27 <clandmeter> ncopa: look at the screenshot
2024-05-10 09:11:37 <clandmeter> same issue as last time i think
2024-05-10 09:11:50 <ncopa> drats
2024-05-10 09:12:00 <clandmeter> network is up
2024-05-10 09:12:07 <clandmeter> but memory issues
2024-05-10 09:12:17 <ncopa> does not accept ssh connections
2024-05-10 09:12:23 <ncopa> ok. we have to revert
2024-05-10 09:12:43 <ncopa> I'll post the image to github issue
2024-05-10 09:13:10 <clandmeter> ok let me install the kenrel from /root
2024-05-10 09:14:17 <ncopa> thank you
2024-05-10 09:15:17 <clandmeter> rebooting
2024-05-10 09:18:02 <ncopa> and it is back
2024-05-10 09:18:17 <clandmeter> should we disable vector stuff on #1 also?
2024-05-10 09:19:33 <ncopa> yeah. good idea
2024-05-10 09:19:44 <ncopa> bummmer. we should have saved the dmesg into a file
2024-05-10 09:19:55 <ncopa> eg dmesg > file.txt
2024-05-10 09:20:16 <clandmeter> go ahead upgrade and ill do it
2024-05-10 09:20:23 <clandmeter> still have some time
2024-05-10 09:20:30 <clandmeter> let me reboot #1 with hide_v0p7_ext
2024-05-10 09:21:19 <ncopa> i used your screenshot :) 
2024-05-10 09:21:38 <ncopa> https://github.com/sophgo/linux-riscv/issues/117#issuecomment-2104261895
2024-05-10 09:23:36 <ncopa> clandmeter: please go ahead and reboot nld-bld-1
2024-05-10 09:23:39 <clandmeter> rebooting #1
2024-05-10 09:33:53 <clandmeter> #1 rebooted, but it shows some ATA issues 
2024-05-10 09:34:25 <clandmeter> looks different compared to the nvme issues we had
2024-05-10 09:35:14 <clandmeter> its an older drive, not sure its mb or drive 
2024-05-10 09:36:28 <ncopa> [   53.046849] I/O error, dev sda, sector 2272 op 0x0:(READ) flags 0x80700 phys_seg 2 prio class 2
2024-05-10 09:39:29 <clandmeter> it looks drive related
2024-05-10 09:39:43 <ncopa> so the disk is unreliable?
2024-05-10 09:39:50 <clandmeter> it only has distfiles 
2024-05-10 09:39:54 <ncopa> yeah
2024-05-10 09:40:01 <ncopa> i noticed. I think its fine
2024-05-10 09:44:34 <ncopa> maybe we should test the 6.6 kernel?
2024-05-22 21:43:48 <iggy> should the 3.20 release post mention the Milk-v Pioneer hardware donation?
2024-05-22 21:47:13 <zcrayfish> Someone should post some hints on the Requirements page on the wiki too.
2024-05-22 22:03:19 <mps> iggy: I think it is mentioned on mastodon post
2024-05-22 22:04:22 <mps> iggy: https://fosstodon.org/@alpinelinux/112484819175489604
2024-05-23 16:26:29 <clandmeter> riscv reached out to me yesterday, asking when they would show up on the download page :)
2024-05-23 16:28:21 <ikke> heh
2024-05-23 16:29:18 <ikke> We don't have an image yet due to lack of kernel in main
2024-05-23 16:39:40 <mps> but it could be added in linux-lts
2024-05-23 16:39:58 <mps> 6.6.x should work for some boards and for qemu
2024-05-23 16:53:25 <clandmeter> yes i highlighted we do not have iso's just minirootfs image
2024-05-23 16:53:52 <clandmeter> not sure iso makes sense, was hinting on disk image support
2024-05-23 17:35:46 <mps> I don't see big differences between iso and disk image
2024-05-23 17:36:47 <ikke> iso is usually read-only and meant to be provided as cdrom
2024-05-23 17:37:25 <ikke> ISO 9660 is read-only
2024-05-23 17:40:51 <mps> yes, but from usage side (especially nowadays) I don't see iso as important
2024-05-23 17:43:18 <ikke> yes, that's what clandmeter was mentioning