2022-09-01 00:15:12 <clandmeter> ikke: i saw some email but i'm unenviable this week due to meetings. lets discuss next week please.
2022-09-01 00:15:25 <clandmeter> unavailable even :)
2022-09-01 05:55:24 <ikke> Sure, works better for me as well
2022-09-06 14:22:02 <ikke> clandmeter: when do you have time to discuss migrating the equinix infra?
2022-09-06 14:22:12 <clandmeter> hi
2022-09-06 14:22:43 <clandmeter> good question
2022-09-06 14:23:24 <clandmeter> i was just checking gitlab if there was an related issue
2022-09-06 14:23:48 <ikke> I don't think we created anything yet
2022-09-06 14:33:55 <clandmeter> ikke: maybe we should do a video call or something to catch up?
2022-09-06 14:34:02 <clandmeter> and write something on paper/gitlab
2022-09-06 14:38:02 <clandmeter> i need to do some work tonight in the garden, but maybe afterwards?
2022-09-06 14:39:31 <ikke> sure
2022-09-06 15:04:11 <clandmeter> lets say around 2100, ill send you an invite
2022-09-06 15:04:56 <ikke> 👍
2022-09-06 19:29:55 <ikke> nl.alpinelinux.org is an alias for nl3.alpinelinux.org.
2022-09-06 19:29:57 <ikke> nl3.alpinelinux.org has address 147.75.32.21
2022-09-06 19:35:41 <clandmeter> ikke: https://gitlab.alpinelinux.org/alpine/infra/infra/-/issues/10769
2022-09-06 19:39:59 <clandmeter> ncopa: if you have time, please go over this ticket and let us discuss these changes.
2022-09-06 19:40:07 <clandmeter> maybe have a small talk this week or next.
2022-09-06 19:40:18 <clandmeter> dont want to shoot ourselves in the foot :)
2022-09-08 13:32:01 <mps> is the riscv64 builder stuck on building lab
2022-09-08 13:32:36 <psykose> probably
2022-09-08 13:32:39 <psykose> algitbot: kick master
2022-09-08 13:32:46 <psykose> algitbot: retry master
2022-09-08 13:58:05 <ikke> mps: rv builder is constant stuck
2022-09-08 14:00:15 <psykose> algitbot: retry master
2022-09-08 14:04:53 <mps> ikke: hmm, yes
2022-09-08 14:05:19 <mps> we are in 'limbo' with riscv64 till we got real build machine
2022-09-08 14:06:28 <ikke> and until rust is fixed
2022-09-08 14:09:00 <mps> eh
2022-09-08 18:15:04 <mps> is there option to 'invoke' building main repo while builder working on community (or testing)
2022-09-08 18:17:40 <ikke> It will start with main
2022-09-08 18:18:08 <mps> when? when finish with community?
2022-09-08 18:19:22 <ikke> every time the build is restart
2022-09-08 18:21:06 <mps> algitbot: retry master
2022-09-08 18:21:13 <mps> this?
2022-09-08 18:21:28 <ikke> well, it's still building at the moment
2022-09-08 18:21:36 <ikke> community/gtk4.0 4.8.0-r0
2022-09-08 18:21:44 <mps> yes, when finish with it?
2022-09-08 18:22:31 <mps> I'm impatient to test new boot for qemu riscv64, sorry ;)
2022-09-08 18:23:45 <psykose> algitbot: kick master
2022-09-08 18:24:08 <mps> uh, too radically :)
2022-09-08 18:24:13 <psykose> nice networking :)
2022-09-08 18:24:17 <psykose> algitbot: retry master
2022-09-08 18:24:18 <psykose> there
2022-09-08 18:24:33 <ikke> it's now building main
2022-09-08 18:24:39 <ikke> main/busybox 1.35.0-r25
2022-09-08 18:24:42 <mps> i see
2022-09-12 19:38:57 <ikke> Disk space usage of builders: https://zabbix.alpinelinux.org/zabbix.php?action=dashboard.view&dashboardid=9
2022-09-12 19:42:15 <psykose> isnt that cpu load
2022-09-12 19:42:23 <psykose> ah
2022-09-12 19:42:25 <psykose> new tab !!
2022-09-12 19:42:57 <psykose> also 3.16 ppc is gone
2022-09-12 19:43:28 <psykose> also the s390x is mislabeled as ppc
2022-09-12 19:43:37 <psykose> and arm as x86
2022-09-12 19:44:21 <ikke> Fixed
2022-09-12 19:45:10 <ikke> building / hanging on openjdk12
2022-09-12 19:46:02 <ikke> or 13 :P
2022-09-12 19:46:50 <psykose> aha
2022-09-12 19:46:53 <psykose> yeah it hangs sometimes
2022-09-12 19:46:56 <psykose> not sure why
2022-09-12 19:52:21 <ikke> for arm, we still have space left in the VG
2022-09-12 19:54:54 <psykose> aye
2022-09-12 19:54:57 <psykose> big machine
2022-09-12 19:55:24 <ikke> s390x has a relative small disk
2022-09-13 18:01:54 <ikke> lxc build is hammering that server for some reason
2022-09-13 18:02:26 <ikke> Load average: 572.11 594.75 446.35
2022-09-13 20:46:44 <ikke> (that trigger alerts if the amount of processes is >1500, fyi)
2022-09-14 01:59:27 <psykose> that seems like a small number
2022-09-14 02:05:25 <psykose> ah, processes
2022-09-14 02:38:43 <psykose> also ppc ci is broken
2022-09-14 04:28:54 <ikke> psykose: how so?
2022-09-14 04:32:23 <ikke> ah, network issues
2022-09-14 04:34:04 <ikke> fixed
2022-09-14 06:21:41 <PureTryOut> turbo's Anitya integration has marked `community/s2geometry` out of date because 0.10.0 is deemed lower than 0.10.0-2022.02.23.00. However the latter is a pre-release of the former. This is a bug/problem in Anitya I suppose, but can the package be marked as up-to-date again somehow?
2022-09-14 07:54:47 <clandmeter> PureTryOut: not from out side as it would mark it again next time i believe
2022-09-14 07:55:20 <PureTryOut> darn, I was afraid of that
2022-09-14 15:32:36 <psykose> the prerelease flagging doesn't work, so beta versions get flagged as the new ones
2022-09-15 07:40:35 <clandmeter> ikke: ping
2022-09-15 07:43:03 <clandmeter> nvm
2022-09-15 07:45:33 <clandmeter> usa5 is awol again
2022-09-15 07:47:35 <ikke> Time to move the builder 
2022-09-15 07:47:55 <clandmeter> that is what i was doing
2022-09-15 07:47:59 <clandmeter> but i could not connect
2022-09-15 08:26:36 <ncopa> clandmeter: i had a look at https://gitlab.alpinelinux.org/alpine/infra/infra/-/issues/10769 and it looks good to me.
2022-09-15 08:29:11 <clandmeter> ikke: looks like dhcp is working on rv64 container :)
2022-09-15 08:29:36 <clandmeter> ncopa: i think the general idea is ok, its more like if we start changing dns records things could break
2022-09-15 08:30:04 <clandmeter> so we wanted to go over the things so we dont miss anything 
2022-09-15 08:30:34 <clandmeter> like when our rsync server showed empty root ones and that kind of blew up a few mirrors :)
2022-09-15 10:16:14 <ikke> clandmeter: nice
2022-09-17 20:54:48 <ikke> hmm
2022-09-18 19:51:11 <ayakael> Are their any issues with `armv7` pipelines right now? My build of `dotnet7` with clang is much much slower than `aarch64`
2022-09-18 19:51:16 <ayakael> there*
2022-09-18 19:51:27 <ikke> Not aware of anything specific
2022-09-18 19:51:36 <ikke> They all run on the same host though
2022-09-18 19:52:42 <ayakael> Copy that, quite likely a problem with upstream then. Testing out rc1 builds of a new packages, so maybe building on `armv7` is somewhat broken.
2022-09-18 20:33:49 <ikke> Gitlab has been upgraded to 15.2
2022-09-19 11:19:51 <clandmeter> ikke: looks like we can fix argv issues with qemu user and setting binfmt P flag
2022-09-19 11:30:05 <clandmeter> ikke: im switching rv64 builder to lxc
2022-09-19 11:30:08 <clandmeter> expect some issues 
2022-09-19 11:31:25 <clandmeter> ncopa: ^
2022-09-19 11:38:49 <ikke> 👍
2022-09-19 11:55:29 <ncopa> great!
2022-09-19 12:03:18 <clandmeter> ikke: ah you already spun the ci hosts
2022-09-19 12:03:29 <clandmeter> ikke: did you bump into issues?
2022-09-19 12:05:06 <ikke> I did fix mdev 
2022-09-19 12:05:21 <ikke> But no other issues 
2022-09-19 12:08:40 <mps> clandmeter: what was rv64 builder till now
2022-09-19 12:10:37 <clandmeter> ikke: did you upgrade the host and updated kernel?
2022-09-19 12:10:52 <clandmeter> mps: docker
2022-09-19 12:11:05 <mps> aha
2022-09-19 12:24:02 <ikke> I believe I did 
2022-09-19 12:48:06 <clandmeter> ok
2022-09-19 12:49:18 <clandmeter> ikke, ncopa: looks like the builder does not create a log file per pkg build. any idea what could cause that?
2022-09-19 12:51:00 <ikke> The - l flag to buildrepo? 
2022-09-19 12:51:23 <ncopa> there is a /etc/buildrepo/config.lua
2022-09-19 12:52:19 <clandmeter> i did not change much most is default setting
2022-09-19 12:52:27 <clandmeter> the config.lua, what it suppose to hold?
2022-09-19 12:52:41 <ncopa> logurlprefix = "https://build.alpinelinux.org/buildlogs"
2022-09-19 12:52:41 <ncopa> logtarget = "distfiles.alpinelinux.org:/var/cache/distfiles/buildlogs"
2022-09-19 12:52:41 <ncopa> mqttbroker = "msg.alpinelinux.org"
2022-09-19 12:52:54 <ncopa> that is what is on build-edge-x86_64
2022-09-19 12:53:25 <clandmeter> right but thats remote stuff 
2022-09-19 12:53:28 <clandmeter> i mean on local disk
2022-09-19 12:53:30 <ncopa> the /etc/conf.d/mqtt-exec.aports-build should have this: buildrepo="buildrepo -p -l /var/cache/distfiles/buildlogs/$(hostname)"
2022-09-19 12:53:42 <ncopa> which is the -l option ikke mentioned
2022-09-19 12:54:31 <clandmeter> hmm
2022-09-19 12:54:37 <clandmeter> thats a manual config option?
2022-09-19 12:55:30 <clandmeter> i am also missing the buildrepo plugins
2022-09-19 13:03:47 <clandmeter> i dont see this buildrepo -p -l settting on the arm builders
2022-09-19 13:06:38 <ikke> Not in /etc/conf.d/mqtt-exec.aports-build?
2022-09-19 13:08:34 <clandmeter> nopre
2022-09-19 13:08:36 <clandmeter> nope
2022-09-19 13:10:21 <clandmeter> hmm
2022-09-19 13:10:24 <clandmeter> some do
2022-09-19 13:14:38 <clandmeter> ok i guess i was looking in the wrong place :)
2022-09-19 13:14:53 <clandmeter> ikke: where do the plugins come from?
2022-09-19 13:15:59 <ikke> Somewhere /usr/share 
2022-09-19 13:16:15 <clandmeter> ah its copied over?
2022-09-19 13:19:42 <ikke> No
2022-09-19 13:19:59 <ikke> Comes from aports-build package 
2022-09-19 13:21:06 <clandmeter> i think it is copied to plugins.d
2022-09-19 13:21:15 <clandmeter> from usr/share
2022-09-19 13:21:30 <clandmeter> not sure why its copied instead of a symlink?
2022-09-19 13:28:14 <ayakael> What's the difference between the `armv7` build environment of pipeslines and builders? Are they both 32-bit userspace virtualized on `aarch64` cores?
2022-09-19 13:28:53 <clandmeter> its not virtualized
2022-09-19 13:29:13 <clandmeter> our cpu's can handle 32bit personality 
2022-09-19 13:29:31 <clandmeter> just like x86_64 can do 32bit
2022-09-19 13:30:44 <clandmeter> the only real difference is, builders are lxc containers and ci is docker containers
2022-09-19 13:33:56 <ayakael> Copy that. I'm dealing with a really strange bug with the new aport for dotnet7, armv7 is significantly slower (like 30 min vs 6 hours slower) in one of the build steps vs aarch64 and I don't have access to 32-bit arm stuff to debug what's up. I guess it wouldn't be infra given 32-bit personality being fully handled.
2022-09-19 13:34:16 <ikke> Ci does run in aarch64 qemu vms 
2022-09-19 13:36:08 <ayakael> So virtualized aarch64 on x86 cores?
2022-09-19 13:36:36 <clandmeter> no
2022-09-19 13:36:42 <clandmeter> right i forgot about that
2022-09-19 13:36:50 <ayakael> While builders are true aarch64 with 32-bit personnality
2022-09-19 13:38:17 <clandmeter> arm ci is qemu on arm hw
2022-09-19 13:39:06 <ayakael> Oh okay, so virtualized while builders are containerized
2022-09-19 13:39:18 <clandmeter> yes
2022-09-19 13:39:46 <clandmeter> its using kvm iirc
2022-09-19 13:41:12 <clandmeter> i think we recently moved ci to another server
2022-09-19 13:41:29 <clandmeter> ikke: so the ci server is still running in kvm?
2022-09-19 13:41:34 <clandmeter> qemu i mean
2022-09-19 13:42:07 <ayakael> When did that move occur?
2022-09-19 13:42:31 <clandmeter> not so long ago 
2022-09-19 13:42:35 <clandmeter> 1-2 months i guess
2022-09-19 13:42:52 <clandmeter> im not sure how its setup now, as ikke didnt add my key :)
2022-09-19 13:43:05 <clandmeter> ah it is now :)
2022-09-19 13:43:34 <clandmeter> hmm
2022-09-19 13:43:53 <clandmeter> not sure its actually moved already
2022-09-19 13:44:43 <ayakael> Do you know of any containerized armv7 providers?
2022-09-19 13:45:34 <ikke> clandmeter: it should 
2022-09-19 13:45:58 <clandmeter> ikke: i dont see anything running on usa10?
2022-09-19 13:46:04 <clandmeter> or is that the wrong server?
2022-09-19 13:46:35 <ikke> Yes
2022-09-19 13:46:42 <ikke> Oh you mean for arm 
2022-09-19 13:46:54 <clandmeter> uhm yes?
2022-09-19 13:46:57 <clandmeter> what else?
2022-09-19 13:47:42 <clandmeter> ayakael: not really
2022-09-19 13:47:50 <ikke> I stopped the vm there to see if it was the cause of the reboots 
2022-09-19 13:48:06 <ikke> usa10 is really unstable 
2022-09-19 13:48:08 <clandmeter> ah right
2022-09-19 13:48:19 <clandmeter> we should report it
2022-09-19 13:48:47 <ikke> I did
2022-09-19 13:48:59 <clandmeter> you did?
2022-09-19 13:49:02 <ikke> Yes
2022-09-19 13:49:08 <clandmeter> i was in cc?
2022-09-19 13:50:15 <ikke> No, I started the ticket through the interface 
2022-09-19 13:50:22 <ikke> I forwarded the thread now 
2022-09-19 13:50:30 <ikke> I did mention it to you 
2022-09-19 13:51:18 <clandmeter> im sure you did, just not enough memory available :)
2022-09-19 13:51:33 <clandmeter> but we got this one from arm folks right?
2022-09-19 13:52:02 <clandmeter> maybe we should report it directly to our contact
2022-09-19 13:52:51 <clandmeter> ayakael: i dont think you will find armv7 providers
2022-09-19 13:53:07 <clandmeter> but any altra system will support 32bit personality 
2022-09-19 13:53:38 <ikke> clandmeter: yeah, we should 
2022-09-19 13:53:55 <ikke> clandmeter: do you think it's worthwhile to try to reinstall it? 
2022-09-19 13:54:14 <clandmeter> no not really
2022-09-19 13:54:30 <clandmeter> i mean its not windows :)
2022-09-19 13:54:51 <clandmeter> you could try another kernel
2022-09-19 13:55:15 <clandmeter> but if its that unstable, i suspect a hw issue
2022-09-19 13:55:27 <clandmeter> or somebody is rebooting it remotly for fun
2022-09-19 13:57:38 <ikke> I tried different kernels 
2022-09-19 14:01:58 <clandmeter> ok im writing to her
2022-09-19 15:14:47 <ikke> let's see if we can get a reaction
2022-09-19 18:04:55 <ikke> clandmeter: did you see the issue regarding naming standard?
2022-09-19 18:39:27 <clandmeter> yes i saw it
2022-09-19 18:39:43 <clandmeter> and i see that golang hangs again on riscv64
2022-09-19 18:40:05 <ikke> Can you see what it's doing?
2022-09-19 18:40:09 <ikke> with strace or something like that/
2022-09-19 18:40:26 <clandmeter> thats a bit difficult with qemu user
2022-09-19 18:40:31 <clandmeter> or at least different
2022-09-19 18:40:32 <ikke> right
2022-09-19 18:40:59 <clandmeter> nmeum did it before
2022-09-19 18:42:17 <nmeum> you can have qemu spawn a specific process with a gdb stup using the -g option
2022-09-19 18:42:37 <nmeum> and then you can connect to qemu's gdb stub using gdb-multiarch and `target remote :<port>`
2022-09-19 18:42:52 <clandmeter> nmeum: but what if the process is already running?
2022-09-19 18:43:13 <nmeum> then there is not much you can do I think
2022-09-19 18:59:08 <clandmeter> ikke: do you remember if its mostly the same golang pkgs?
2022-09-19 18:59:42 <ikke> The only one I remeber is traefik, but that's because it hung recently
2022-09-19 19:00:00 <ikke> and stern today
2022-09-19 19:00:10 <clandmeter> thats the one now 
2022-09-19 19:00:18 <clandmeter> go test
2022-09-19 19:00:18 <ikke> so 2 times in a row
2022-09-19 19:00:34 <ikke> https://build.alpinelinux.org/buildlogs/build-edge-riscv64/testing/stern/stern-1.21.0-r3.log
2022-09-19 19:00:38 <ikke> that was last time
2022-09-19 19:00:54 <ikke> still building there
2022-09-19 19:30:39 <clandmeter> algitbot: retry master
2022-09-19 19:39:41 <clandmeter> nmeum: any idea how we could enable that -g option when using binfmt_misc or similar?
2022-09-19 19:42:35 <nmeum> hm, no not really. since it opens a socket at a fixed port it is probably also not feasible to enable it by default
2022-09-19 19:42:57 <nmeum> I think the -g option is more suitable for bugs that you can deterministically reproduce
2022-09-19 19:43:47 <clandmeter> nod
2022-09-19 19:50:48 <clandmeter> looks like we can use QEMU_STRACE
2022-09-19 19:51:06 <clandmeter> but that will blow up the logs 
2022-09-19 22:43:14 <Hello71> write a wrapper program?
2022-09-21 07:14:19 <clandmeter> ikke: i think i forgot one thing on the rv64 builder :)
2022-09-21 07:15:53 <ikke> What? 
2022-09-21 07:17:17 <clandmeter> it had tests disabled
2022-09-21 07:17:46 <clandmeter> and now it looks like they fail due to qemu user
2022-09-21 07:17:59 <clandmeter> algitbot: retry master
2022-09-21 07:18:47 <ikke> You mean you wanted to try to have tests enabled on the new builder? 
2022-09-21 07:19:04 <clandmeter> no i didnt disable them, as i forgot about it.
2022-09-21 07:19:10 <clandmeter> and now tests are failing
2022-09-21 07:19:21 <ikke> Ah 
2022-09-21 07:19:24 <clandmeter> and this latest py test error looks like a qemu user thingy
2022-09-21 07:19:41 <clandmeter> complaining about  parent pid or whatever
2022-09-21 07:23:07 <clandmeter> i disabled the tests again
2022-09-21 07:33:47 <ikke> So the new builder is officially active? 
2022-09-21 07:50:41 <psykose> is it fully ported now?
2022-09-21 07:50:49 <psykose> can try rust building again for fun..
2022-09-21 07:51:42 <psykose> also i forgot, is it new host hw + latest qemu + lxc?
2022-09-21 09:02:40 <clandmeter> yes its active
2022-09-21 09:02:47 <clandmeter> its running 3.16 qemu
2022-09-21 09:03:52 <clandmeter> qemu-riscv64 version 7.0.0
2022-09-21 09:06:09 <clandmeter> Hello71: not sure how a wrapper would work with binfmt host
2022-09-21 09:06:58 <clandmeter> i guess the invocation of qemu-riscv64 would open a port, but what if its invoked multiple times and using the same port? 
2022-09-21 09:45:39 <clandmeter> uptime
2022-09-21 09:45:40 <clandmeter> 09:45:01 up 13 days, 13:43,  0 users,  load average: 23.30, 17.23, 15.87
2022-09-21 09:45:44 <psykose> the kick master hook doesn't seem connected to riscv64 anymore either (or it hanged again, haha)
2022-09-21 09:46:04 <clandmeter> no i did not apply the patch
2022-09-21 09:46:18 <clandmeter> i am thinking of just adding it upstream
2022-09-21 09:46:31 <clandmeter> have something like kick hostname
2022-09-21 09:47:25 <clandmeter> psykose: when did you try to kick it?
2022-09-21 09:47:30 <psykose> 5 seconds ago
2022-09-21 09:47:35 <psykose> algitbot: kick master
2022-09-21 09:47:46 <clandmeter> its busy, so no need to kick
2022-09-21 09:47:50 <clandmeter> look at uptime
2022-09-21 09:48:03 <psykose> there is
2022-09-21 09:48:50 <clandmeter> is what?
2022-09-21 09:50:58 <psykose> there are more interesting things to be doing (like kicking and restarting it to try build rust again on new setup) than waiting 95 hours (literally) for it to finish the 507-deep package queue in testing
2022-09-21 09:51:12 <psykose> anyway, not implemented anymore, understood
2022-09-21 09:53:31 <clandmeter> im sure it will be reimplemented, just dont know when it will be. 
2022-09-21 09:53:40 <clandmeter> i think its hanging again now
2022-09-21 09:53:51 <psykose> what makes you think that
2022-09-21 09:54:13 <psykose> the current thing has been building for 4-5 minutes
2022-09-21 09:55:02 <clandmeter> ps shows nothing is using cpu
2022-09-21 09:55:11 <clandmeter> htop i mean
2022-09-21 09:55:34 <clandmeter> and it was doing golang things
2022-09-21 10:09:35 <clandmeter> i killed madonctl
2022-09-21 10:39:23 <ikke> clandmeter: how do we call the replacement for nld3 for applications?
2022-09-21 10:40:13 <clandmeter> the one on linode?
2022-09-21 10:40:16 <ikke> yes
2022-09-21 10:40:26 <clandmeter> dunno
2022-09-21 10:40:34 <clandmeter> you want to use some new naming scheme?
2022-09-21 10:40:48 <ikke> Does not have to be a completely new scheme
2022-09-21 10:40:58 <ikke> But perhaps better defined / adjusted
2022-09-21 10:42:13 <clandmeter> you have an idea?
2022-09-21 10:45:52 <ikke> Do we want to keep everything connected with dmvpn? Things like monitoring is a bit easier if so
2022-09-21 10:46:38 <ikke> I think my main struggle is a bit what we define as a site
2022-09-21 10:47:09 <clandmeter> what do you think about adding a wrapper around go
2022-09-21 10:47:12 <ikke> but maybe we can decouple the dmvpn definition of a site and what we call a site
2022-09-21 10:47:27 <ikke> clandmeter: for the rv64 builder?
2022-09-21 10:47:32 <clandmeter> nod
2022-09-21 10:47:48 <ikke> and it would setup gdb I suppose?
2022-09-21 10:48:02 <clandmeter> that or just use the strace option
2022-09-21 10:48:06 <ikke> right
2022-09-21 10:48:31 <clandmeter> QEMU_STRACE
2022-09-21 10:48:32 <clandmeter> arguments don’t have decoders and will show up as numbers.
2022-09-21 10:48:32 <clandmeter> Print system calls and arguments similar to the ‘strace’ program (NOTE: the actual ‘strace’ program will not work because the user space emulator hasn’t implemented ptrace). At the moment this is incomplete. All system calls that don’t have a specific argument format are printed with information for six arguments. Many flag-style
2022-09-21 10:48:54 <ikke> hmm, ok
2022-09-21 10:48:57 <clandmeter> so this will blow up the golang build logs considerably 
2022-09-21 10:49:24 <ikke> hmm
2022-09-21 10:49:29 <ikke> Because it prints to stdout?
2022-09-21 10:49:32 <clandmeter> i think the problem with gdb is that there is no output on stdout
2022-09-21 10:49:58 <ikke> would be nice if it could log to a specific file
2022-09-21 10:51:40 <ikke> clandmeter: what do you think of decoupling our sites from dmvpn sites
2022-09-21 10:51:50 <ikke> where the dmvpn site just equates to the host it's connected to
2022-09-21 10:51:59 <clandmeter> that is fine by me
2022-09-21 10:52:16 <clandmeter> but what will be the names? what defines a hostname?
2022-09-21 10:53:34 <ikke> Is some kind of location indication usefull?
2022-09-21 10:53:52 <ikke> Do we want to distinguish where it's hosted?
2022-09-21 10:54:07 <clandmeter> maybe where and whats hosted
2022-09-21 10:54:19 <ikke> Yes, the type of host is usefull
2022-09-21 10:54:19 <clandmeter> i mean those 2 things are most logical 
2022-09-21 10:54:29 <clandmeter> but what is a type :)
2022-09-21 10:54:32 <clandmeter> docker/lxc?
2022-09-21 10:54:40 <clandmeter> app or something else
2022-09-21 10:54:58 <ikke> I was thinking: builder, application, development 
2022-09-21 10:55:00 <ikke> something like that?
2022-09-21 10:55:13 <ikke> docker vs lxc I guess is an implementation detail
2022-09-21 10:55:14 <clandmeter> and how to distinct two samilar servers? 
2022-09-21 10:55:26 <ikke> sequence numbers
2022-09-21 10:55:46 <clandmeter> bld app and dev?
2022-09-21 10:55:53 <ikke> nod
2022-09-21 10:55:57 <clandmeter> 1.app.a.o?
2022-09-21 10:56:15 <ikke> I was thinking about subdomains
2022-09-21 10:57:14 <ikke> We did want locatin in there, right?
2022-09-21 10:57:40 <ikke> app-<n>.<site>.a.o?
2022-09-21 11:00:47 <clandmeter> ok fine by me
2022-09-21 11:01:12 <ikke> ok, now, how do we define the site
2022-09-21 11:01:28 <clandmeter> site is region?
2022-09-21 11:01:37 <ikke> How granular
2022-09-21 11:01:46 <clandmeter> just country?
2022-09-21 11:02:12 <ikke> ok, so app-1.deu.a.o, bld-2.nld.a.o, etc?
2022-09-21 11:02:33 <clandmeter> that is ok for me
2022-09-21 11:02:55 <ikke> Do we want to distinguish between hosting facilities somehow?
2022-09-21 11:03:38 <ikke> and, do we keep <type>-<n> unique accross all sites?
2022-09-21 11:03:52 <ikke> so we only have 1 app-1 for example
2022-09-21 11:09:00 <ikke> clandmeter: I was also thinking about using <24 subnets for non-lxc hosts if that's possible
2022-09-21 11:09:10 <clandmeter> i guess unique for that area?
2022-09-21 11:09:35 <clandmeter> looks like we cannot use gdb stub 
2022-09-21 11:10:07 <clandmeter> the process is locked until released withint gdb
2022-09-21 11:10:19 <ikke> ok, that doesn't help
2022-09-21 11:10:39 <clandmeter> i think we can try using strace and see what it shows
2022-09-21 11:11:01 <clandmeter> but if we redirect the output, the bujildlogs wont show any output from go
2022-09-21 11:11:25 <ikke> right
2022-09-21 11:12:29 <ikke> oh, ci as well I guess as type
2022-09-21 11:14:14 <clandmeter> what if you need to combine CI with something else?
2022-09-21 11:14:38 <clandmeter> ok with strace it outputs to stderr, so i guess I could redirect that
2022-09-21 11:15:19 <ikke> but I guess that would catch other errors as well
2022-09-21 11:19:03 <ikke> clandmeter: I don't know, depends I guess what we deem the main purpose of the host
2022-09-21 11:19:08 <ikke> clandmeter: And we have mirror hosts as well
2022-09-21 11:19:23 <ikke> is that mir-<n>.<site>.a.o?
2022-09-21 11:20:29 <clandmeter> we do not have separate mirrors? 
2022-09-21 11:20:51 <clandmeter> maybe for mixed env we c an use mix :D
2022-09-21 11:22:16 <ikke> What do we call the tier-1 mirror hosts?
2022-09-21 11:24:49 <ikke> right now it's <country>.t1.a.o
2022-09-21 11:25:39 <clandmeter> i think its ok like this?
2022-09-21 11:25:48 <clandmeter> do they need to follow exacly the std?
2022-09-21 11:26:11 <clandmeter> https://tpaste.us/4oJO
2022-09-21 11:26:12 <ikke> I like consistency, but we can still have cnames
2022-09-21 11:26:34 <clandmeter> all the t1's are the same :)
2022-09-21 11:26:41 <clandmeter> thats consistent 
2022-09-21 11:26:42 <clandmeter> :)
2022-09-21 11:27:04 <ikke> But incosistent with the rest :P
2022-09-21 11:27:22 <clandmeter> I would move that paste as go in /use/local/bin
2022-09-21 11:27:43 <clandmeter> and see if we can catch something specific
2022-09-21 11:27:52 <ikke> right
2022-09-21 11:27:55 <clandmeter> and maybe show it to upstream
2022-09-21 11:28:00 <ikke> let's do that
2022-09-21 11:28:23 <clandmeter> ikke, if  you want to rename them, thats fine by me.
2022-09-21 11:28:42 <clandmeter> i really do not have a preference
2022-09-21 11:29:04 <clandmeter> but its nice to call something like t1.a.o and get one of the countries
2022-09-21 11:29:14 <ikke> We can still keep that
2022-09-21 11:29:14 <clandmeter> whatever name it would be
2022-09-21 11:29:26 <ikke> But I'd cname that to the hostname
2022-09-21 11:29:36 <clandmeter> thats fine for me
2022-09-21 11:29:52 <clandmeter> but what would you namea them?
2022-09-21 11:29:59 <clandmeter> mirror instead of t1?
2022-09-21 11:33:57 <ikke> I don't mind t1 as an alias, I'm mainly talking about the hostnames 
2022-09-21 11:38:42 <ikke> clandmeter: https://gitlab.alpinelinux.org/alpine/infra/infra/-/issues/10770#note_264007
2022-09-21 12:36:42 <clandmeter> looks like abuild does not pickup /usr/local/bin/go
2022-09-21 12:38:19 <clandmeter> 20339 buildoze  0:06 {go} /usr/bin/qemu-riscv64 /usr/bin/go go build -trimpath -o build/gobuster
2022-09-21 12:39:04 <ikke> Maybe PATH is limited? 
2022-09-21 12:39:59 <clandmeter> if im regular buildozer user it works
2022-09-21 12:40:45 <ikke> Yes, because /etc/profile is sourced 
2022-09-21 12:40:51 <clandmeter> ah 
2022-09-21 12:40:54 <clandmeter> yes 
2022-09-21 12:41:34 <clandmeter> path is different for buildozer
2022-09-21 12:43:10 <clandmeter> it looks like profile is sourced
2022-09-21 12:43:26 <ikke> For mqtt-exec? 
2022-09-21 12:43:27 <clandmeter> but path is has sorted differently
2022-09-21 12:43:34 <clandmeter> -has
2022-09-21 12:43:49 <clandmeter> usr/local is later 
2022-09-21 12:44:05 <clandmeter> im not sure how env for mqtt-exec looks like
2022-09-21 12:47:04 <clandmeter> ncopa: is it normal for profile to set /usr/local last in path?
2022-09-21 13:02:42 <clandmeter> i guess it inherits the previous path and it does not have usr/local as first 
2022-09-21 13:09:06 <ikke> A change was made to append items to path instead of overwriting it
2022-09-21 13:09:22 <ikke> Mostly because of WSL2
2022-09-21 13:09:41 <ikke> Or WSL in general 
2022-09-21 15:47:39 <ikke> clandmeter: downside of using subdomains is that the hostname would just be <function>-<n>
2022-09-21 15:47:49 <ikke> meaning easy to confuse if you have multiple of the same ones
2022-09-21 15:48:02 <ikke> app-1.deu vs app-1.gbr
2022-09-21 20:06:48 <ikke> clandmeter: what about gbr-app-1?
2022-09-21 21:11:25 <ikke> netbox upgraded to 3.3.4
2022-09-21 22:56:19 <clandmeter> ikke: what about it?
2022-09-22 05:12:46 <ikke> clandmeter: seems like algitbot is no longer anouncing when the builders uploaded the packages?
2022-09-22 08:04:59 <mps> is there option to set automatic 'delete branch' option in MR
2022-09-22 08:16:50 <clandmeter> ikke: uhm
2022-09-22 08:16:57 <clandmeter> i did restart algitbot
2022-09-22 08:24:48 <clandmeter> its weird as it does announce errors which is coming from the same script
2022-09-22 08:30:15 <mps> uh, how is gitlab interface is cumbersome and not intuitive
2022-09-22 08:30:45 <mps> how to create merge request of this https://gitlab.alpinelinux.org/mps/aports/-/commit/bdf97df78502122ff7202cfa4c021ca8c6752470
2022-09-22 08:31:54 <ikke> Go to the branches overview page, there is a button there to create an MR for that branch 
2022-09-22 08:32:36 <clandmeter> ikke: do you know who puts msgs for topic /rsync on mqqt?
2022-09-22 08:32:46 <clandmeter> i guess the builders themselves
2022-09-22 08:34:07 <ikke> Yes
2022-09-22 08:34:49 <clandmeter> i dont see these msg's on mqtt
2022-09-22 08:35:02 <clandmeter> and i didnt touch the other builders
2022-09-22 08:36:24 <clandmeter> i see mps is spamming mqtt :)
2022-09-22 08:36:44 <mps> meh
2022-09-22 08:37:10 <mps> I don't know what I do (or better say why I do anything)
2022-09-22 08:37:58 <clandmeter> dont worry, it didnt smoke yet.
2022-09-22 08:38:49 <mps> I'm lost in these new-modern-shiny web interfaces
2022-09-22 08:41:25 <ikke> mp
2022-09-22 08:41:49 <ikke> mps: you can use glab on the cli as well 
2022-09-22 08:42:57 <mps> ikke: yes I know, but I hesitate to put my keys on dev lxc
2022-09-22 08:43:37 <ikke> You can use access tokens with limited access 
2022-09-22 08:44:42 <mps> yet another 'strange' thing to learn
2022-09-22 09:03:33 <clandmeter> ikke: very interesting issues here
2022-09-22 09:03:41 <clandmeter> issue*
2022-09-22 09:04:09 <clandmeter> i guess we updated to docker compose golang version on this docker host?
2022-09-22 09:04:47 <ikke> On deu1? 
2022-09-22 09:04:53 <clandmeter> nod
2022-09-22 09:04:57 <ikke> Yup 
2022-09-22 09:05:05 <clandmeter> this is the problem
2022-09-22 09:06:20 <clandmeter> it interpreters the env vars differently 
2022-09-22 09:06:45 <clandmeter> it removes the hashes from the topic
2022-09-22 09:06:59 <clandmeter> topics even
2022-09-22 09:06:59 <ikke> Oh right 
2022-09-22 09:07:06 <ikke> It requires quotes now 
2022-09-22 09:09:31 <clandmeter> ok i quoted them now
2022-09-22 09:09:43 <clandmeter> so it should get back to regular business i hope
2022-09-22 10:10:31 <ncopa> mps: https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html
2022-09-22 10:31:49 <mps> ncopa: thanks
2022-09-22 10:32:41 <mps> (but not sure I could put all these in head)
2022-09-22 10:58:26 <ikke> kunkku: does dmvpn also support smaller subnets?
2022-09-22 11:10:08 <ikke> smaller than /24 (for example /30)
2022-09-22 11:22:48 <ikke> I think it should be possible
2022-09-22 13:44:52 <ikke> blaboon: Would it be possible to get an ipv6 subnet for gbr-app-1?
2022-09-22 15:23:48 <blaboon> ikke: sure, do you want another /64 like the one it has now?
2022-09-22 15:24:14 <ikke> Oh, does it already have one?
2022-09-22 15:24:29 <ikke> The ip addresses overview does not mention it
2022-09-22 15:25:09 <blaboon> :/
2022-09-22 15:27:02 <ikke> In any case, just a single would be enough
2022-09-22 15:48:56 <blaboon> ok, i think i see what's going on. at some point i must have manually assigned a /64 to this linode but the association got lost somehow. i just cleaned that up and you can go ahead and assign yourself a new range in the manager by going to the networking section of the linode and clicking "Add an IP Address"
2022-09-22 15:50:18 <blaboon> there will be a section in the popup dialog for adding a new v6 range
2022-09-22 15:50:53 <blaboon> it should show up in the addresses section right away once added
2022-09-22 16:06:53 <ikke> blaboon: fyi, this is a new linode, created yesterday
2022-09-22 16:07:33 <ikke> blaboon: thanks, managed to add one
2022-09-22 16:07:53 <ikke> Wasn't aware that it is self-service now :-)
2022-09-22 16:13:06 <blaboon> ahh ok gotcha what i was seeing makes sense then. but yea, v6 ranges are self-service now
2022-09-22 18:39:35 <ikke> For some reason I cannot get ipv6 to work for the lxc containers
2022-09-22 18:39:56 <ikke> Container has an external ip address, it can ping it's default gateway, but not outsie
2022-09-22 18:39:59 <ikke> outside
2022-09-22 18:40:14 <ikke> If I ping from outside to the container, I see traffic comming in, but not reaching the container
2022-09-22 18:41:01 <ikke> Ok, thanks for rubber ducking
2022-09-22 18:41:25 <ikke> I already set ipv6 forwarding for most interfaces, but I needed to set it for all
2022-09-23 10:20:28 <ikke> blaboon: can it be that the prefix was removed from gbr2-dev1? Maybe I misunderstood what you mean, but it's no longer reachable via IPv6 on that subnet
2022-09-23 10:22:05 <ikke> And I cannot manually add one, because it says our limit is exceeded
2022-09-23 13:18:06 <blaboon> ikke: do you mean the prefix that you added yesterday?
2022-09-23 13:19:20 <ikke> blaboon: that one is working, but lost ipv6 connectivity to another linode 
2022-09-23 13:19:31 <ikke> You mentioned you removed one 
2022-09-23 13:23:04 <ikke> 2a01:7e00:e000:2fc::/64
2022-09-23 13:23:28 <blaboon> taking a look...
2022-09-23 13:31:29 <blaboon> which linode is inaccessible?
2022-09-23 13:32:22 <ikke> gbr2-dev1
2022-09-23 13:34:59 <blaboon> can you give it a try now?
2022-09-23 13:40:05 <ikke> Cannot ping yet
2022-09-23 13:40:16 <ikke> b
2022-09-23 13:40:46 <ikke> blaboon: if the prefix is present, I can continue fixing if it's an issue on our side later 
2022-09-23 13:55:30 <blaboon> i definitely see the prefix associated with that linode on our side. i think i may have messed things up a bit yesterday though as the prefix is now primarily associated with gbr-app-1 whereas previously it may have been primarily associated with a different linode
2022-09-23 14:05:58 <blaboon> that said, if you do want to add another prefix to gbr2-dev1 i can do that for you. i think the manager only lets you add one
2022-09-23 14:12:10 <ikke> In the end, we only need a single prefix per linode 
2022-09-23 14:12:49 <ikke> But the one for gbr2-dev1 is not reqchabke 
2022-09-23 14:16:44 <ikke> If it's easier to assign a new one, that's fine with me as well 
2022-09-23 14:20:23 <blaboon> i just manually added new prefixes to gbr2-dev1 and gbr3-dev1, which you should see in the manager
2022-09-23 14:23:34 <ikke> Alright, thanks 
2022-09-23 14:25:58 <blaboon> sorry about all the confusion
2022-09-23 15:22:05 <ikke> blaboon: new subnet is working for gbr2-dev1, thanks
2022-09-26 09:02:50 <mps> does this means meson trying to download something `meson.build:397:4: ERROR: Automatic wrap-based subproject downloading is disabled`
2022-09-26 09:33:42 <Newbyte> mps: yes
2022-09-26 09:35:01 <mps> Newbyte: thanks for confirmation
2022-09-26 09:35:18 <mps> now have to find where it tries that
2022-09-26 09:40:13 <psykose> probably at meson.build line 397
2022-09-26 09:40:32 <psykose> where it adds either a subproject or a dependency fallback
2022-09-26 09:40:58 <psykose> meaning either you need to clone a submodule, or add a missing dependency, or find the full tarball that has everything already, whichever applies
2022-09-26 09:53:33 <mps> psykose: thanks, makes sense
2022-09-27 10:01:06 <mps> following #asahi channel I'm surprised how much alpine is better than other famous and big distros
2022-09-27 10:01:52 <mps> and ofc simpler and smaller :)
2022-09-27 10:55:20 <ncopa> :)
2022-09-27 10:56:03 <Newbyte> mps: just curious, better in what sense?
2022-09-27 11:13:15 <mps> Newbyte: talking from experience using distros on apple silicon. first, works out of the box, all other distros have to add tweaks and what not to make it working
2022-09-27 11:13:54 <mps> for example alpine worked from november previous year while fedora is not yet finished
2022-09-27 11:14:35 <mps> debian is better but still needs some tweaks
2022-09-27 11:15:48 <mps> didn't tried nixos so not sure but on IRC I see they have some problems, though not big like other distros
2022-09-27 11:16:29 <mps> even official asahi distro which is Arch linux is not in good shape as alpine
2022-09-27 11:17:39 <mps> for alpine to work as on all other arm64 making linux-asahi kernel pkg was enough, though I added also special u-boot in repo
2022-09-27 11:18:48 <mps> s/arm64/arm64 machines/
2022-09-27 11:20:15 <mps> some people use M1/M2 with alpine as servers already
2022-09-27 12:10:48 <clandmeter> ikke, psykose i added a new script to rv64 builder
2022-09-27 12:11:07 <clandmeter> it will check the output of abuild, and if no update for 1 hour it will kill all its childeren
2022-09-27 15:14:16 <ikke> clandmeter: ok, nice
2022-09-27 19:51:19 <clandmeter> ikke: the new rv64 host oomed
2022-09-27 19:51:34 <ikke> hmm
2022-09-27 19:51:58 <ikke> Is that an issue?
2022-09-27 19:51:59 <clandmeter> or something happend not sure what
2022-09-27 19:52:36 <clandmeter> take a look at dmesg
2022-09-28 08:31:09 <ncopa> clandmeter: speaking of dmvpn, i wonder if we should consider switch to wireguard?
2022-09-28 08:32:06 <ncopa> i added pub ssh key of new nld box to nld5-dev1 for rsync of container
2022-09-28 08:32:50 <clandmeter> yes, that would be nice, but no routing?
2022-09-28 08:33:06 <clandmeter> like nhrp
2022-09-28 08:33:56 <ncopa> dmvpn uses bgp for routing
2022-09-28 08:35:01 <ncopa> i dunno. i just thought that wireguard is kind of "the future" and there are clients for macos etc as I understand
2022-09-28 08:35:06 <ncopa> dont know who they do routing
2022-09-28 08:35:36 <clandmeter> we have wireguard already
2022-09-28 08:35:41 <clandmeter> im using it as we speak
2022-09-28 08:35:46 <ncopa> ok
2022-09-28 08:36:02 <clandmeter> but i mean to replace dmpvn
2022-09-28 08:36:30 <ikke> ncopa: we would need to have a central hub where all traffic goes over, or setup a mesh ourselves 
2022-09-28 08:36:39 <clandmeter> i thought you were talking about replacing dmvpn
2022-09-28 08:36:49 <ncopa> i was
2022-09-28 08:36:56 <clandmeter> not sure if there is any project that has the same feature set
2022-09-28 08:37:07 <clandmeter> there are some commercial ones i believe
2022-09-28 08:37:09 <ikke> tailscale kinda 
2022-09-28 08:37:26 <ncopa> ikke: we'd need set up the mesh ourselves? like every node would need to be configured on every other node?
2022-09-28 08:37:28 <clandmeter> right,  thats the one i refer to
2022-09-28 08:37:38 <ikke> ncopa: yes
2022-09-28 08:37:49 <ikke> ncopa: wireguard is p2o
2022-09-28 08:37:51 <ikke> P2p
2022-09-28 08:38:07 <clandmeter> is it any different than ipsec?
2022-09-28 08:38:24 <clandmeter> its just a bunch of tunnels right?
2022-09-28 08:38:29 <ncopa> ipsec is only the transport
2022-09-28 08:38:29 <ikke> Yes
2022-09-28 08:38:33 <clandmeter> with bgp on top
2022-09-28 08:38:39 <ikke> That's wireguard as well
2022-09-28 08:39:04 <ncopa> ipsec is transport, gre is the tunnel, nhrp is the "arp" (who is where), bgp is routing
2022-09-28 08:39:17 <ncopa> wireguard does tunneling also right?
2022-09-28 08:39:24 <ikke> Wireguard is transport + tunnel afaik 
2022-09-28 08:39:35 <clandmeter> nod
2022-09-28 08:39:35 <ncopa> thats how i understand as well
2022-09-28 08:39:49 <ncopa> what whould be interesting was wireguard + nhrp + bgp
2022-09-28 08:39:50 <clandmeter> so you need something that can handle the routing
2022-09-28 08:40:03 <ikke> And key exchange 
2022-09-28 08:40:52 <ncopa> ipsec uses certificates and certificate authority, so now key exchange is needed
2022-09-28 08:41:24 <ikke> Wireguard just uses a key pair 
2022-09-28 08:41:44 <ncopa> so wireguard cannot do PKI, with certs?
2022-09-28 08:42:38 <ncopa> anyway, seems like that is a bigger project
2022-09-28 08:42:45 <ncopa> we have other things to deal with
2022-09-28 08:43:40 <ncopa> hum.. no subnet is allocated for nld
2022-09-28 08:44:40 <psykose> wireguard doesn't do a lot of that, yeah, you have to wrap it yourself. just keypairs
2022-09-28 08:44:49 <psykose> it's also quite hard to debug when it doesn't work
2022-09-28 08:44:52 <psykose> part of the design
2022-09-28 08:44:56 <psykose> aside from that it's fantastic at what it does
2022-09-28 08:45:03 <psykose> never even look at it and it always works once set up
2022-09-28 08:45:10 <ncopa> seems like it can do certificates and CA https://xeiaso.net/blog/site-to-site-wireguard-part-3-2019-04-11
2022-09-28 08:45:55 <clandmeter> ncopa: the naming of that new server is not yet finalized
2022-09-28 08:46:11 <clandmeter> ikke is working on defining a new naming scheme
2022-09-28 08:47:10 <psykose> that's not wireguard, it's the extra things on top
2022-09-28 08:47:27 <ncopa> ok
2022-09-28 08:47:37 <ncopa> where is the dmvpn sqlite db we currently use?
2022-09-28 08:47:40 <psykose> i.e. they set up dns to map hostnames, then they map to 10.x wireguard addresses, and the things it resolves to serves the cert
2022-09-28 08:47:53 <psykose> the ca thing is after wireguard not before
2022-09-28 08:48:09 <psykose> so you still need all the mesh stuff first to route anything, etc
2022-09-28 08:48:44 <psykose> it's possible to do whatever of course, it's just more setup, unlike what seems to be (here) "just install dmvpn"
2022-09-28 08:50:50 <clandmeter> ncopa: dmvpn.a.o
2022-09-28 08:53:09 <ikke> dmvpn1 
2022-09-28 08:53:21 <ikke> dmvpn2 does not have it 
2022-09-28 08:55:06 <mps> WG is just a tunnel (secure)
2022-09-28 08:55:26 <mps> it is in kernel so it is fast
2022-09-28 08:55:52 <mps> similar to ipsec tunnel
2022-09-28 08:56:20 <mps> for setting userspace is needed, like IKE in ipses
2022-09-28 08:57:07 <mps> routing have to be set in userspace, like for ipsec
2022-09-28 08:58:07 <mps> there are different solutions for making different network types, tailscale is one example
2022-09-28 08:59:47 <mps> one of nice WG feature is client roaming
2022-09-28 09:01:01 <mps> cryptokey routing
2022-09-28 09:10:44 <ncopa> NLD10       172.16.25.0/24
2022-09-28 09:10:44 <ncopa> NLD3        172.16.8.0/24
2022-09-28 09:10:44 <ncopa> NLD5        172.16.4.0/24
2022-09-28 09:10:44 <ncopa> NLD8        172.16.21.0/24
2022-09-28 09:10:44 <ncopa> NLD9        172.16.22.0/24
2022-09-28 09:10:54 <ncopa> those are the NLD subnets we have so far
2022-09-28 09:11:04 <ncopa> should I call it NLD11?
2022-09-28 09:11:43 <clandmeter> ikke: ^
2022-09-28 09:12:08 <clandmeter> ikke is doing the naming thingy, so its best if he approves it :)
2022-09-28 09:13:35 <ikke> For the new dev box? 
2022-09-28 09:14:30 <ncopa> yes
2022-09-28 09:14:59 <ncopa> im allocating a 172.16.x.0/24 subnet for it
2022-09-28 09:15:06 <ncopa> setting up dmvpn
2022-09-28 09:17:29 <ikke> I'd say NLD-DEV1 as site, similar to GBR3-APP1
2022-09-28 09:18:38 <ncopa> and I guess 172.16.26.0/24?
2022-09-28 09:18:56 <ncopa> NLD10 has 172.16.25.0/24
2022-09-28 09:20:43 <ikke> I can't check, but if dmvpn-ca does not have it assigned, it should be fine 
2022-09-28 09:20:48 <ncopa> ok
2022-09-28 09:20:59 <ncopa> its not assigned. 26 will be for NLD-DEV1
2022-09-28 09:23:25 <ikke> 👍
2022-09-28 09:44:17 <ncopa> hum... i need a password to be able to generate a cert for dmvpn
2022-09-28 09:44:33 <ncopa> clandmeter: do you have the password somewhere?
2022-09-28 09:44:40 <clandmeter> yup
2022-09-28 09:44:46 <clandmeter> you mean for dmvpn?
2022-09-28 09:44:50 <ncopa> yes
2022-09-28 09:45:04 <clandmeter> shall i paste it here?
2022-09-28 09:45:07 <clandmeter> :D
2022-09-28 09:45:09 <ncopa> ha!
2022-09-28 09:45:18 <ncopa> paste it on tpaste.... :D
2022-09-28 09:46:48 <psykose> i think you have an unclaimed $10,000,000 too, check your inbox
2022-09-28 09:47:11 <panekj> 
ACTION 👀

2022-09-28 09:47:21 <psykose> italic eyes
2022-09-28 10:01:25 <ncopa> bah... i think i messed up
2022-09-28 10:01:29 <ncopa> i did apk upgrade
2022-09-28 10:01:40 <ncopa> before setup-dmvpn
2022-09-28 10:01:46 <ncopa> which means it installed new kernel
2022-09-28 10:01:56 <ncopa> which means modprobe gre etc fails 
2022-09-28 10:02:01 <ncopa> i need to reboot it
2022-09-28 10:02:05 <clandmeter> np
2022-09-28 10:02:08 <clandmeter> just reboot it
2022-09-28 10:03:30 <ncopa> lets see if it comes up again
2022-09-28 10:04:11 <clandmeter> its something that bites me often
2022-09-28 10:04:21 <clandmeter> our kernel logic is.... not so smart
2022-09-28 10:04:37 <clandmeter> would be nice to keep the modules or the running kernel
2022-09-28 10:04:47 <clandmeter> cleanup afterwards
2022-09-28 10:05:06 <clandmeter> s/or/of
2022-09-28 10:06:09 <ncopa> seems like it does not yet come up
2022-09-28 10:06:22 <clandmeter> it usually takes some time
2022-09-28 10:06:32 <clandmeter> you are not used  to baremetal anymore?
2022-09-28 10:07:03 <ncopa> :)
2022-09-28 10:07:53 <psykose> to be fair with the exception of huge-memory metal almost everything comes up in under a minute xD
2022-09-28 10:08:08 <psykose> damned server memory training taking 20 years
2022-09-28 10:08:19 <ncopa> and dmvpn is up!
2022-09-28 10:08:25 <psykose> good job!
2022-09-28 10:08:25 <clandmeter> depends also on arch
2022-09-28 10:10:14 <ncopa> hum
2022-09-28 10:10:57 <ncopa> ok seems like /var/lib/lxc was not mounted
2022-09-28 10:11:30 <clandmeter> aha
2022-09-28 10:11:37 <clandmeter> strange
2022-09-28 10:11:40 <ncopa> lvm service needs to be enabled
2022-09-28 10:11:46 <clandmeter> i made that change yesterday
2022-09-28 10:11:48 <clandmeter> oh yes 
2022-09-28 10:11:49 <ncopa> im fixing it
2022-09-28 10:11:49 <clandmeter> haha
2022-09-28 10:12:00 <clandmeter> fstab shjould be ok
2022-09-28 10:31:32 <psykose> for some reason all the arm ci is stuck
2022-09-28 10:32:26 <ikke> vms crashed
2022-09-28 10:32:30 <ikke> I suppose due to OOM
2022-09-28 10:32:55 <ikke> The whole host was rebooted
2022-09-28 10:32:59 <psykose> hah
2022-09-28 10:33:06 <psykose> yeah, runners poof
2022-09-28 10:33:14 <psykose> apologies, i ran a thousand things through ci
2022-09-28 10:33:26 <psykose> guess it went past all the host limits even with 2x job per arch
2022-09-28 10:34:24 <psykose> that failing arm machine we could put ci on would be useful if it worked
2022-09-28 10:34:32 <ikke> yeah
2022-09-28 10:34:47 <psykose> not that we have a lack of metal really
2022-09-28 10:34:50 <psykose> 99% of the time is idle
2022-09-28 10:34:57 <psykose> the issue is just scheduling so many things in so many places
2022-09-28 10:40:34 <ikke> Is it possible to disable caching of NXDOMAIN on unbound?
2022-09-28 10:41:40 <psykose> https://github.com/NLnetLabs/unbound/issues/533 ?
2022-09-28 10:42:49 <psykose> so basically, cache-max-negative-ttl: 1 (or maybe 0 works?) should work since 1.14.0
2022-09-28 10:42:58 <psykose> 1 is 1 but low enough going by that
2022-09-28 10:43:53 <psykose> to entirely disable, doesn't seem to be..
2022-09-28 10:43:59 <ikke> installed: unbound-1.13.0-r3
2022-09-28 10:44:38 <psykose> even 3.13 is 1.13.2 haha, where is that
2022-09-28 10:44:52 <psykose> but i'd recommend upgrading to 1.14 at least and using 1 for what i think you're trying to solve
2022-09-28 10:45:17 <psykose> otherwise it seems to be overriden by the other ttl
2022-09-28 10:45:31 <ikke> intra-ns1.nld3.alpin.pw
2022-09-28 10:46:03 <psykose> sounds like upgrade time :)
2022-09-28 10:46:49 <ikke> We're moving everything off that host, and I'm upgrading it as much as possible when doing that
2022-09-28 10:48:28 <psykose> good luck ^^
2022-09-28 10:48:36 <psykose> do you need my help with anything
2022-09-28 10:53:06 <ikke> The most 'interesting' one will be the wiki 
2022-09-28 10:55:34 <psykose> which one
2022-09-28 10:55:40 <ikke> wiki.a.o
2022-09-28 10:56:14 <psykose> i guess that has a few updates to add indeed
2022-09-28 10:57:08 <psykose> could do https://gitlab.alpinelinux.org/alpine/infra/infra/-/issues/10761 at the same time
2022-09-28 11:02:38 <ikke> FYI, all arm ci runners are back online
2022-09-28 11:04:06 <psykose> noticed
2022-09-28 11:04:07 <psykose> thanks
2022-09-28 15:09:29 <eletrotupi> stew
2022-09-28 15:09:40 <eletrotupi> oops
2022-09-28 15:16:42 <psykose> yes please
2022-09-28 15:17:11 <eletrotupi> 
ACTION blushes

2022-09-29 08:05:47 <ncopa> morning! im gonna try use the qemu-riscv64-static from debian sid and see if that makes any diffference
2022-09-29 08:39:13 <clandmeter> morning
2022-09-29 08:39:21 <clandmeter> ncopa: how will you use that?
2022-09-29 09:33:38 <ncopa> replace the current alpine built qemu-riscv64
2022-09-29 09:33:54 <ncopa> its statically built so it works
2022-09-29 09:53:53 <psykose> perhaps we should also ship a qemu-user-static
2022-09-29 09:53:59 <psykose> same as qemu-static (system only?)
2022-09-29 09:54:16 <psykose> but i guess it's effort for not that much use like the regular static
2022-09-29 09:54:30 <clandmeter> ncopa: if you play with it, things will also change for the builder
2022-09-29 09:54:41 <clandmeter> just so you are aware of it
2022-09-29 09:55:26 <clandmeter> there is also a docker project to do just that, provide qemu static stuff
2022-09-29 09:55:54 <clandmeter> it will register binfmt for you
2022-09-29 09:56:01 <clandmeter> and use the bundled static bins
2022-09-29 09:56:15 <ncopa> i know it will also change the builder
2022-09-29 09:56:49 <clandmeter> ncopa: with the docker project, you can just try it local
2022-09-29 09:57:10 <clandmeter> https://github.com/multiarch/qemu-user-static
2022-09-29 10:16:42 <psykose> the riscv builder has been offline for a bit
2022-09-29 10:16:50 <psykose> stuck on that without responding to kick
2022-09-29 10:17:02 <ikke> psykose: ncopa is working on it
2022-09-29 10:17:06 <psykose> ah
2022-09-29 12:41:15 <psykose> ikke: could you look into why approving is broken on gitlab for assignees
2022-09-29 12:41:38 <ikke> I did briefly check if there was an option to enable it, but could not find anything
2022-09-29 12:41:56 <psykose> hm
2022-09-29 12:42:06 <psykose> how does it normally work? who has the "permissions"?
2022-09-29 12:42:14 <psykose> i noticed merely reporter role can approve anything now, maybe
2022-09-29 12:54:07 <ikke> "GitLab allows all users with Developer or greater permissions to approve merge requests."
2022-09-29 12:58:08 <psykose> is there no settings or anything
2022-09-29 12:58:33 <ikke> Cannot find anything
2022-09-29 12:58:59 <psykose> strange
2022-09-29 12:59:00 <psykose> :/
2022-09-29 13:14:05 <ikke> psykose: even being guest is enough to bring it back
2022-09-29 13:14:20 <psykose> ok. anything stopping all-maintainer guest addition?
2022-09-29 13:14:34 <psykose> i don't think it has any downside via permissions (you checked before?), just have to write it
2022-09-29 13:14:51 <psykose> should also not accidentally downgrade someone's permissions
2022-09-29 13:26:47 <ikke> I don't think so, except for someone programming it
2022-09-29 13:35:21 <psykose> yep
2022-09-29 19:03:48 <clandmeter> Gitlab sec rel 
2022-09-29 19:04:33 <ikke> on it
2022-09-29 19:28:46 <ikke> I'm trying to setup the http(s) proxy on gbr-app-1, but for some reason I cannot reach port 80/443
2022-09-29 19:29:16 <ikke> It appears the firewall is blocking it
2022-09-29 20:40:33 <ikke> Found it, needed to make sure the NAT policy was loaded before adp-router
2022-09-29 20:40:44 <ikke> adp-router blocks everyhting to the private lan
2022-09-30 10:37:04 <ikke> clandmeter: I've created proxy.gbr-app-1.alpinelinux.org as cname target for sites hosted on gbr-app-1.alpinelinux.org to support IPv6
2022-09-30 10:38:25 <clandmeter> nod
2022-09-30 10:38:47 <ikke> I've moved over zabbix already there