2022-11-02 08:47:12 <ncopa> the CI build image for 3.15-stable is broken. apk fails to downgrade
2022-11-02 08:47:48 <ncopa> i wonder if we should have an alpinelinux/alpine-build-base image for each branch
2022-11-02 09:03:13 <ncopa> psykose: did you have a reproducer for glib allocator issue?
2022-11-02 09:03:36 <psykose> libadwaita testsuite
2022-11-02 09:03:42 <ncopa> IIRC it was G_DEBUG=gc-friendly ... something
2022-11-02 09:04:00 <ncopa> thanks!
2022-11-02 09:20:54 <ikke> ncopa: yes. Was wondering if there was anything against adding a replaces= between those packages 
2022-11-02 09:21:33 <ikke> I'm planning adding support for dedicated images, but there are other priorities atm. 
2022-11-02 09:56:16 <ncopa> other idea is to: `apk upgrade -U -a || apk fix`
2022-11-02 09:56:53 <ikke> That could work as well 
2022-11-02 11:52:48 <ikke> ncopa: In fact, it already does that
2022-11-02 11:53:29 <ikke> the problem is that after the issue, apk is broken
2022-11-02 11:54:00 <ncopa> oh ok
2022-11-02 11:54:03 <psykose> broken in what sense
2022-11-02 11:55:36 <ikke> 139754367834952:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1919:
2022-11-02 11:55:41 <ikke> ERROR: libcrypto1.1-1.1.1s-r0: Permission denied
2022-11-02 11:56:05 <psykose> ah
2022-11-02 11:56:06 <psykose> right
2022-11-02 11:56:12 <psykose> cause the certs are empty on disk
2022-11-02 11:56:15 <psykose> xD
2022-11-02 11:56:43 <ikke> yeah, if I switch to http, then it works
2022-11-02 11:57:45 <ikke> It does not reproduce directly, you first need to install alpine-sdk and then downgrade
2022-11-02 11:58:43 <psykose> my recommendation would be to use http anyway as this is one of the cases where the tls does not really add anything
2022-11-02 11:58:54 <psykose> the packages are signed/verified out of the tls band regardless
2022-11-02 11:59:23 <psykose> sure, hacky workaround in one way, but in another it fixes the issue and is a marginal speed improvement (less round trips + tls overhead)
2022-11-02 12:00:24 <psykose>  the only thing tls adds in this case is hiding the specific package being downloaded from our own infrastructure
2022-11-02 12:00:57 <ikke> reproducer: (on alpine:edge) apk upgrade -Ua && apk add alpine-sdk && sed -i s/edge/v3.15/ /etc/apk/repositories && apk upgrade -Ua
2022-11-02 12:01:50 <ikke> psykose: and protection against some pre-verification vulnerability
2022-11-02 12:02:13 <psykose> yeah
2022-11-02 12:06:31 <ikke> peak of +10gbps comined outgoign traffic from our new t1 mirrors
2022-11-02 12:06:58 <psykose> :)
2022-11-02 12:07:06 <psykose> was there some moment ~yesterday where it was at capacity?
2022-11-02 12:07:39 <ikke> Each server should have 20gbps bandwidth
2022-11-02 12:07:43 <ikke> so I would not say so
2022-11-02 12:07:47 <psykose> ok, good to know
2022-11-02 12:08:55 <ikke> And for now, this is only used by other mirrors to sync from
2022-11-02 12:09:53 <ikke> Need to switch dl-cdn soon
2022-11-02 16:25:22 <ikke> nld3 is completely full now
2022-11-02 16:25:26 <ikke> (mirror)
2022-11-02 16:32:04 <ikke> oof, dl-cdn certificate expires oct-6, but the interface shows the renewal is oct 7th
2022-11-02 16:35:53 <ikke> oh, n/m
2022-11-02 16:35:59 <ikke> 6 october is the not-before
2022-11-02 16:36:26 <ikke> it's valid until 2022-01-04
2022-11-02 17:19:52 <minimal> ikke: 2023? ;-)
2022-11-02 17:25:06 <ikke> Yes 😊
2022-11-02 17:40:30 <minimal> this is the future, not the past! :-)
2022-11-02 17:44:35 <ikke> Ah, that would explain a lot 
2022-11-02 20:34:51 <ikke> ncopa: fyi, arm mentioned that our servers will be moved on the 14th and will probably be down for 4 days
2022-11-03 07:37:11 <ncopa> ikke: i saw the email
2022-11-03 11:54:44 <ikke> ncopa: I think we need to switch the dl-cdn backend today. The nld3 mirror (current backend) 100% used
2022-11-03 11:55:02 <ikke> I updated the fastly config in a new draft, not sure if you want to verify it?
2022-11-03 12:29:43 <ncopa> ok. how can i verify?
2022-11-03 12:58:29 <ikke> https://manage.fastly.com/configure/services/4ZVgv5JlGHsuH7lfiWfzy/diff/90,91
2022-11-03 13:01:20 <ncopa> LGTM
2022-11-03 13:01:56 <ikke> We can always switch back in case something breaks
2022-11-03 13:02:05 <ncopa> good
2022-11-03 13:02:10 <ikke> Done
2022-11-03 13:03:24 <ikke> 404 not found :(
2022-11-03 13:04:26 <ncopa> aw
2022-11-03 13:04:44 <ikke> trying to find out what's wrong
2022-11-03 13:06:42 <ncopa> seems like t1.alpinelinux.org/ redirects to t1.alpinelinux.org/alpine
2022-11-03 13:06:59 <ncopa> while https://dl-t1-1.alpinelinux.org/ does not
2022-11-03 13:07:15 <ncopa> maybe revert for now?
2022-11-03 13:07:32 <ikke> yes
2022-11-03 13:07:59 <ikke> Done
2022-11-03 13:12:15 <ncopa> there is a t1.alpinelinux.org in fastly for testing?
2022-11-03 13:12:33 <ncopa> "Used for host shielding (remove when host is removed)"
2022-11-03 13:14:28 <ikke> trying to recreate the same config in the testing domain
2022-11-03 13:28:20 <ikke> reproduced it now here: http://dl-cdn-test.alpinelinux.org/
2022-11-03 14:15:35 <ikke> Traefik somehow does not match the request to a service, but cannot find details on why
2022-11-03 14:23:10 <ikke> ah, d'oh
2022-11-03 14:25:14 <ncopa> what was it?
2022-11-03 14:26:15 <ikke> Need to verify it, but it uses cdn hostname, which traefik was not setup to listen on
2022-11-03 14:26:50 <ikke> You _can_ override the SNI hostname, but the http hostname will be the same
2022-11-03 14:31:27 <ikke> ncopa: http://dl-cdn-test.alpinelinux.org/alpine/
2022-11-03 14:31:44 <ikke> So the config was good, it was just the backend that was configured incorrectly
2022-11-03 14:31:45 <ncopa> nice
2022-11-03 14:32:15 <ikke> It just took a bit too long to get proper logging out ouf traefik
2022-11-03 14:32:24 <ikke> turns out you need to enable access logging, and set it to json format
2022-11-03 14:32:39 <ikke> common log format has too little details
2022-11-03 14:32:47 <ikke> too few*
2022-11-03 14:33:21 <ncopa> http://dl-cdn-test.alpinelinux.org/ redirects to alpine/ . is that on purpose?
2022-11-03 14:35:39 <ikke> the alpine-mirror-sync project has an explicit redirect 
2022-11-03 14:35:42 <ikke> so I supppose so
2022-11-03 14:36:24 <ikke> https://tpaste.us/jPL5
2022-11-03 14:36:57 <ikke> ncopa: trying to switch the config again
2022-11-03 14:38:32 <ikke> Seems to hold now
2022-11-03 14:43:39 <ncopa> very nice
2022-11-03 15:26:03 <ikke> Seems like we receive no traffic anymore on nld3-dev1
2022-11-04 20:46:44 <ikke> ncopa: s390x builder is full :(
2022-11-04 20:48:15 <ikke> ok, we can clean up distfiles
2022-11-05 18:01:59 <ikke> I gotta love rubber ducking
2022-11-05 18:02:26 <ikke> trying to fix a connectivity issue, tried all kinds of things
2022-11-05 18:02:41 <ikke> Start making a post explaining the situation
2022-11-05 18:02:48 <ikke> and I suddenly realized the issue
2022-11-05 18:04:09 <minimal> ikke: that's a common situation, the act of trying to explain a problem to someone else suddenly results in the problem cause becoming self-evident to yourself :-)
2022-11-05 18:04:29 <panekj> btw. did you know that pkgs.a.o is down?
2022-11-05 18:04:38 <panekj>  /s
2022-11-05 18:04:47 <ikke> panekj: only if you are limited to IPv4 :p
2022-11-05 18:04:51 <ikke> but yes, I was aware
2022-11-05 18:05:08 <ikke> And no, it's not the issue I was trying to solve
2022-11-05 18:05:12 <ikke> though, related
2022-11-05 18:05:26 <panekj> my ISP router turned off IPv6 and I couldn't be bothered to enable it since then
2022-11-05 18:05:51 <ikke> I can declare alpine being ipv6 only from now and call it a day :D
2022-11-05 18:05:57 <panekj> also casually relaying information from IRC to Discord
2022-11-05 18:06:18 <ikke> It's a network related issue out of our hands
2022-11-05 18:06:20 <panekj> ikke: noipv6 on birdapp would be happy
2022-11-05 18:34:47 <ikke> panekj: everything should work again
2022-11-05 18:35:28 <panekj> o7
2022-11-05 19:13:25 <ikke> kunkku: with awall, how can you add filter rules that only affect the forward chain?
2022-11-05 19:16:03 <ikke> Oh, I think I have it
2022-11-05 19:17:01 <ikke> https://gitlab.alpinelinux.org/alpine/awall/-/blob/master/awall/model.lua#L284
2022-11-05 19:17:23 <ikke> Need to set both zones, but neither should be "_fw"
2022-11-06 07:02:31 <ikke> huh
2022-11-06 07:04:26 <psykose> just ipv4 probably?
2022-11-06 07:04:38 <psykose> no, works for me
2022-11-06 07:04:40 <psykose> funny
2022-11-06 07:06:05 <ikke> yeah
2022-11-07 06:44:56 <ncopa> algitbot: retry master
2022-11-07 10:38:00 <mps> tpaste report bug when trying post there
2022-11-07 10:38:34 <mps> &#47;usr&#47;share&#47;lua&#47;5.1&#47;redis.lua:258: closed
2022-11-07 11:02:52 <ikke> that host is quite busy atm
2022-11-07 11:23:57 <mps> ncopa: do we want drbd-tools upgrade for 3.17? currently 9.22.0 (new release) doesn't build, but I reverted https://github.com/LINBIT/drbd-utils/commit/4a1b590090bf676cdfb10c198505b95b9eeb3120 and it builds
2022-11-07 11:24:35 <mps> I plan to report this upstream and see what they will answer (if they answer)
2022-11-07 11:30:54 <ikke> mps: tpaste is working again
2022-11-07 11:31:19 <mps> ikke: thank you for fast fix
2022-11-07 21:06:35 <ikke> psykose: have to look at pkgs.a.o tomorrow. For some reason, the update seems to hang
2022-11-07 21:06:50 <ikke> probably ipv6 related
2022-11-07 22:07:11 <psykose> may be
2022-11-07 22:07:13 <psykose> is ok
2022-11-07 22:33:54 <ptrc> new gitlab update, new more funny bugs: the "rebase" button is available everywhere now, clicking on it when i'm not permitted shows a big red "Something went wrong" :')
2022-11-08 08:23:51 <ncopa> seems like the x86 and x86_64 builders are gone?
2022-11-08 08:46:25 <ikke> Hmm
2022-11-08 09:29:37 <ikke> Back now :-) 
2022-11-08 09:29:57 <ikke> Apparently something gobbled up all memory 
2022-11-08 09:34:18 <ikke> We need to clean up that mirror, it should not be used anymore 
2022-11-08 11:16:53 <ikke> hmm
2022-11-08 11:49:06 <ikke> for aports-turbo, I see that wget spawns ssl_client, which then is hanging
2022-11-08 11:58:55 <ikke> It's not ipv6 this time
2022-11-08 11:58:58 <ikke> but not sure what's going on
2022-11-08 17:59:53 <ikke> I suspect it's some mtu issue, but I have no idea why that suddenly is the case
2022-11-08 18:01:18 <ikke> and only from docker containers
2022-11-08 18:43:22 <ikke> so strange, apk add hangs one time, the next time it succeeds :/
2022-11-08 18:47:56 <mps> hm something is strange with gitlab. I created MR for community/crystal and searching with option "merge request I've created" doesn't find it
2022-11-08 19:03:36 <ikke> mps: what option do you mean exactly?
2022-11-08 19:04:42 <mps> "Search gitlab" and then select "merge request I've created"
2022-11-08 19:06:10 <ikke> And you are in the aports project?>
2022-11-08 19:06:49 <mps> yes
2022-11-08 19:07:10 <mps> if I type 'crystal' it is found
2022-11-08 19:07:34 <ikke> https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/?author_username=mps does not return anything?
2022-11-08 19:08:00 <mps> yes, it does
2022-11-08 19:08:48 <ikke> so it does work?
2022-11-08 19:10:25 <mps> yes, with url you posted, but not from search option
2022-11-08 19:11:20 <mps> "Search Gitlab" I mean
2022-11-08 19:11:32 <mps> it worked for long
2022-11-08 19:12:37 <ikke> mps: That url is what I get when I use that option
2022-11-08 19:12:46 <ikke> (I just changed my name to yours)
2022-11-08 19:12:54 <psykose> do you mean that you literally typed
2022-11-08 19:12:55 <psykose> "merge request I've created"
2022-11-08 19:12:57 <psykose> in the search field
2022-11-08 19:13:30 <mps> psykose: no, select it from drop-down list
2022-11-08 19:13:46 <psykose> ah
2022-11-08 19:13:51 <psykose> that just returns the above url and works
2022-11-08 19:14:24 <mps> hmm, no one option from this list works
2022-11-08 19:15:55 <mps> could be related to upgraded firefox
2022-11-08 19:17:05 <ikke> psykose: I think pkgs.a.o is some mtu issue with dl-cdn
2022-11-08 19:17:12 <psykose> possible
2022-11-08 19:17:39 <ikke> with t1.a.o I think it works, but I have to hotpatch aports-turbo, because to check if a package exists, it uses socket.http, which apparently does not support https
2022-11-08 19:21:01 <psykose> you can add luasec and ssl.https and it should workj
2022-11-08 19:22:36 <ikke> require("socket.http").request({
2022-11-08 19:22:38 <ikke>             url = uri, method = "HEAD"})
2022-11-08 19:22:40 <ikke> that's what it uses now
2022-11-08 19:32:15 <psykose> apk add lua-sec, https = require 'ssl.https', https.request(samething)
2022-11-08 19:34:18 <ikke> right now it does not update edge because it already updated the repo versions in the db, so it's skipping htme
2022-11-08 19:34:20 <ikke> them*
2022-11-08 19:35:57 <ikke> just set the version empty in the db
2022-11-08 19:36:57 <ikke> psykose: when I ping dl-cdn from gbr2 with packet size between 1498 and 1500, I get no response back when setting -Mdo
2022-11-08 19:37:10 <ikke> larger, and I get an mtu error, lower and it works
2022-11-08 19:38:26 <ikke> It's at least updating now
2022-11-08 20:03:33 <ikke> psykose: strange, from deu1-dev1 I have no issue with mtu to dl-cdn :/
2022-11-08 20:03:40 <ikke> ping -Mdo -s 1472 199.232.58.133 works there
2022-11-08 20:04:47 <ikke> but on gbr2-dev1 hangs...
2022-11-08 20:05:21 <ikke> psykose: good news, pkgs.a.o has been updated
2022-11-08 20:12:50 <psykose> :)
2022-11-08 20:13:01 <psykose> yeah mtu stuff really makes no sense sometimes :/
2022-11-08 20:13:59 <ikke> I wonder if it is related to the network issues we had this weekend
2022-11-08 21:54:40 <ikke> ncopa: I've moved your dev container to nld-dev-1
2022-11-09 07:21:34 <ncopa> ikke: ok. thanks! (which dev container?)
2022-11-09 07:22:11 <ncopa> found it. ncopa-edge-x86_64
2022-11-09 07:46:27 <ikke> Indeed 
2022-11-09 07:46:53 <ikke> sadly nld5 will be decommissioned 
2022-11-09 11:16:37 <ncopa> Partition /srv/mirror 100% used 
2022-11-09 11:17:05 <ikke> yes, I'm aware
2022-11-09 11:17:11 <ikke> That's why we moved away
2022-11-09 11:19:12 <ikke> ncopa: We have a very peculiar issue with gbr2-dev1. It seems to be mtu related, but from one server to one server (afaik) only
2022-11-09 11:19:49 <ikke> so anything trying to download from dl-cdn on that server is having issues
2022-11-09 11:20:18 <ikke> It seems to be that packets between 1497 and 1500 bytes are dropped
2022-11-09 11:21:44 <mps> ikke: psykose: now gitlab works fine in selecting "merge request I've created". Interesting, I didn't upgrade firefox or anything.
2022-11-09 11:21:57 <mps> Heisenbugs
2022-11-09 11:23:18 <ikke> It's as if something is adding 4 bytes to the packets
2022-11-09 11:26:44 <minimal> ikke: vlan tag perhaps?
2022-11-09 11:27:47 <ikke> Not using vlans ourself, and I would not expect that to persist accross the internet
2022-11-09 11:28:12 <minimal> ikke: I was thinking of somewhere in the infra there
2022-11-09 11:28:29 <minimal> a vlan tag is 4 bytes, first thing that came to mind
2022-11-09 11:28:44 <ikke> yeah, things like mpls labels as well
2022-11-09 11:28:45 <ncopa> ikke: fastly/dl-cdn drop PMTU discovery packets, so if there is lower MTU in the path you will get a black hole
2022-11-09 11:28:57 <ikke> ncopa: yes, I recall you mentioning that
2022-11-09 11:29:07 <ncopa> we could set the MTU down, but I don't like that...
2022-11-09 11:29:14 <ikke> I tried it, does not help
2022-11-09 11:29:48 <ncopa> hum. ok
2022-11-09 11:30:10 <ikke> strange thing is, I can install packages without issue on the host itself
2022-11-09 11:30:26 <ikke> but ping -s 1472 (1500 bytes total) fails
2022-11-09 11:31:25 <ncopa> but not from containers?
2022-11-09 11:31:29 <ikke> correct
2022-11-09 11:31:40 <ikke> or docker build
2022-11-09 11:31:45 <ncopa> smells like a local issue then
2022-11-09 11:32:12 <ikke> yes, but have a hard time figuring out what could cause it
2022-11-09 11:34:04 <minimal> ikke: wireshark time? ;-)
2022-11-09 11:35:46 <ikke> minimal: already been trying
2022-11-09 12:20:00 <ikke> ok, interesting. I disabled ipv6 in docker (afaik, for the default bridge network) and docker containers can fetch from dl-cdn
2022-11-09 12:32:08 <ikke> It's still strange though
2022-11-09 12:55:22 <psykose> ruh roh
2022-11-09 12:56:06 <ikke> huh
2022-11-09 12:56:44 <ikke> Good that I just fixed ssl certificate monitoring :P
2022-11-09 13:01:25 <psykose> not pulling a manjaro :)
2022-11-09 16:41:40 <ikke> nginx could not be reloaded due to container having moved and the dns record no longer being able to be found 
2022-11-09 18:53:17 <ikke> (that's me)
2022-11-10 13:05:42 <ikke> ncopa: fyi, I want to move alpine-msg and alpine-www today
2022-11-10 13:06:40 <ncopa> ok
2022-11-10 13:57:59 <ikke> Someone care to review https://gitlab.alpinelinux.org/alpine/infra/linode-tf/-/merge_requests/55/?
2022-11-10 14:02:35 <psykose> looks correct
2022-11-10 14:07:05 <ikke> Ok, here it goes
2022-11-10 14:41:31 <ikke> next up alpine-msg (mqtt)
2022-11-10 16:38:12 <ikke> Would anything bad happen if I enable add ipv6 to mqtt?
2022-11-10 16:43:20 <psykose> not unless it's unroutable i'd guess
2022-11-10 16:43:47 <psykose> there can also be some firewall stuff that's broken on our end : )
2022-11-10 16:44:09 <psykose> ah, and the listening mqtt daemon has to actually listen on ipv6
2022-11-10 16:44:23 <ikke> Yeah, I did work on that
2022-11-10 16:44:50 <ikke> 2a01:7e00:e000:2fc::4 port 1883 works
2022-11-10 16:51:24 <psykose> sounds good then
2022-11-10 18:19:41 <ikke> updated the msg.a.o records to point to the new server
2022-11-10 18:20:16 <psykose> and it's broken of course
2022-11-10 18:20:39 <ikke> ofcourse
2022-11-10 18:22:02 <ikke> takes a bit for dns records to update
2022-11-10 18:22:14 <ikke> and I stopped the old container
2022-11-10 18:22:35 <psykose> mhm
2022-11-10 18:31:28 <ikke> forgot to update one IP I thought I already updated
2022-11-10 18:32:21 <ikke> msg.a.o points to the nginx proxy container, which then forwards it mosquitto
2022-11-10 18:32:33 <ikke> (reason is I believe because of the websockets)
2022-11-10 18:33:52 <ikke> I think all builders are present now, rihgt?
2022-11-10 18:40:11 <ikke> ok, good, the webhook also still works
2022-11-11 16:19:46 <ncopa> ok. next week we dont have any arm machines. I should really get an rc1 out today
2022-11-11 16:27:00 <minimal> ncopa: please don't forget to do a new mdev-conf package so that that my MR you merged gets into 3.17
2022-11-11 16:28:51 <ncopa> speaking for that
2022-11-11 16:29:17 <ncopa> im not sure the /dev/disk/by-<id>  is a good idea
2022-11-11 16:29:47 <ncopa> currently we only escape space to \x20
2022-11-11 16:30:01 <ncopa> so anything using any other special char will be broken
2022-11-11 16:30:08 <ncopa> or incompatible with udev
2022-11-11 16:30:27 <minimal> sigh
2022-11-11 16:30:38 <ncopa> people may adapt and adjust their stuff so it works with mdev
2022-11-11 16:30:45 <ncopa> and then when they switch to udev it will break
2022-11-11 16:30:52 <ncopa> or
2022-11-11 16:31:04 <ncopa> if we fix it in the future, their hacks will break
2022-11-11 16:31:10 <ncopa> the latter worries me most
2022-11-11 16:31:43 <minimal> I did point out that only \x20 was currently handled as it is the only escaped sequence I've observed so far
2022-11-11 16:31:43 <ncopa> i am not sure that letting people depend on a broken behavior is a good idea
2022-11-11 16:32:16 <ncopa> have you tested create labels with å æ ø or other special chars?
2022-11-11 16:32:25 <ncopa> "Natanael'd disk"
2022-11-11 16:32:30 <ncopa> "Natanael's disk"
2022-11-11 16:32:36 <ncopa> who is ' handled in the label?
2022-11-11 16:32:37 <ncopa> etc
2022-11-11 16:32:51 <minimal> nope
2022-11-11 16:33:06 <ncopa> how will udev handle it?
2022-11-11 16:33:31 <ncopa> when i did the persistent_storage script initially I checked carefully what udev did
2022-11-11 16:34:01 <minimal> this 11th hour decision is getting me fed up - something similar happens with the previous incarnation of the patch where it sat for a long time unlooked at and then was rejected
2022-11-11 16:34:19 <ncopa> i think it would be much simpler to patch cloud-init to support LABEL=....
2022-11-11 16:34:26 <ncopa> im not saying im rejecting it
2022-11-11 16:34:26 <minimal> from memory udev uses a function in libblk to deal with it
2022-11-11 16:35:05 <ncopa> what im saying is that i will likely regret that i pushed it in the future
2022-11-11 16:35:21 <ncopa> because once we push it, it will be difficult to revert it
2022-11-11 16:35:29 <minimal> cloud-init DOES suport LABEL, that's why this patch was written
2022-11-11 16:36:02 <ncopa> it only support label via /dev/disk/by-label symlinks, right?
2022-11-11 16:36:11 <ncopa> i checked the cloud-init code
2022-11-11 16:36:13 <ncopa> anyway
2022-11-11 16:36:37 <ncopa> i will push it and hope nobody depends on it
2022-11-11 16:36:43 <ncopa> because i believe ti will bite us
2022-11-11 16:36:49 <minimal> I'm really just close to giving up on Alpine in general - I don't mind criticism of patches, it's the last minute nature of things
2022-11-11 16:37:58 <ncopa> the last minute thing stresses me as well
2022-11-11 16:38:02 <ncopa> horribly
2022-11-11 16:38:31 <ncopa> but we we do have a windows *now* after builders are done and monday morning, when arm servers are shut down for 4 days
2022-11-11 16:38:57 <ncopa> i have spent all day trying to fix as many bugs as possible in alpine-conf
2022-11-11 16:39:00 <ncopa> and written tests
2022-11-11 16:45:33 <ncopa> if i dont ship a rc1 *now* it will have to wait another week
2022-11-11 16:49:02 <minimal> I'm not saying don't do a RC, I'm just showing my frustration over months at trying to get something in place
2022-11-11 16:51:02 <ncopa> i pushed mdev-conf 4.3
2022-11-11 16:51:08 <ncopa> so it is in place
2022-11-11 16:52:20 <ncopa> what i am afraid of, is contributors that show up and add features, and then disappears, and and i need spend time debug and fix those features
2022-11-11 16:52:35 <ncopa> thats why i tend to resist adding new features
2022-11-11 19:54:43 <psykose> i mean you should see their perspective
2022-11-11 19:55:15 <psykose> they show up and submit a patch, it gets no response for 3 months, then suddenly the release is in 1 week and there's a bunch of feedback and it's too late and it's too risky and and
2022-11-11 19:56:20 <psykose> if you don't want to include it, then don't, that's fine
2022-11-11 19:56:49 <psykose> but there's never a correct time to include it because it's been literal months of nothing, it's not like there is some unaddressed feedback
2022-11-11 20:12:13 <mps> life in free software world /o\
2022-11-11 20:13:32 <mps> and non free world is worse, ime
2022-11-13 22:30:07 <ayakael> During Gitlab pipelines, where is the ./logs folder that gets scooped up as artifacts? I'm needing additional binlogs to be scooped up to debug an issue that occurs a lot more in Gitlab CIs for one of my packages. I tried sending them to '$HOME/logs' but that doesn't seem to be it.
2022-11-13 23:16:14 <ayakael> Nevermind, TIL $APORTSDIR
2022-11-14 09:26:06 <mps> I've got two mails from something called naptive.com (cloud service) with 'Subject: Cloud-Native Developer Self-Service` and 'We have been scouting *open source projects* in order to provide core project contributors like you with our *Pro Edition* completely *free* ...'
2022-11-14 09:26:58 <mps> is this spam or someone of you also received such mails. thinking maybe they target open/free source developers
2022-11-14 09:28:26 <mps> looks like they 'lure' devs
2022-11-14 09:30:29 <mps> ncopa: also for drbd-utils, I've got answer for bug report, here is url https://lists.linbit.com/pipermail/drbd-dev/2022-November/006861.html
2022-11-14 09:31:46 <ikke> mps: have not received such e-mails
2022-11-14 09:33:07 <mps> ikke: aha, thanks. so they 'think' I'm easy victim :)
2022-11-14 09:33:33 <Newbyte> is the ARM CI gone?
2022-11-14 09:34:12 <mps> ncopa: if you think we should have upgraded drbd-tools tell me so I can prepare and merge it
2022-11-14 09:34:31 <mps> I mean, upgraded in 3.17 release
2022-11-14 09:40:25 <ikke> Newbyte: yes, the server is being moved the comming few days
2022-11-14 09:40:37 <ikke> same for the builders
2022-11-14 09:40:41 <Newbyte> ikke: thanks for the clarification
2022-11-14 09:44:50 <clandmeter> good morning 
2022-11-14 09:45:05 <ikke> o/
2022-11-14 09:45:13 <clandmeter> how are we with infra? 
2022-11-14 09:45:28 <clandmeter> nothing special except arm servers?
2022-11-14 09:45:44 <ikke> usa6 has some weird routing issues
2022-11-14 09:45:50 <ikke> at least, installing routes
2022-11-14 09:46:25 <ikke> We still need to move rv64 ci
2022-11-14 09:46:42 <ikke> verify no dns records are still pointing to the old equinix hw
2022-11-14 09:46:50 <clandmeter> nod
2022-11-14 09:47:04 <clandmeter> i still got some catching up to do @work
2022-11-14 09:47:09 <clandmeter> but im here again
2022-11-14 09:47:15 <ikke> clandmeter: for some reason, rv64 dhcp on nld-dev-1 stopped working again
2022-11-14 09:47:22 <clandmeter> hmm
2022-11-14 09:47:25 <clandmeter> strange
2022-11-14 09:47:32 <clandmeter> what changed?
2022-11-14 09:47:34 <clandmeter> kernel?
2022-11-14 09:47:39 <clandmeter> qemu?
2022-11-14 09:47:50 <clandmeter> binfmt settings?
2022-11-14 09:48:00 <ikke> I have not rebooted
2022-11-14 09:48:04 <ikke> I did add firewall rules
2022-11-14 09:48:16 <ikke> But other containers have no issues
2022-11-14 09:48:19 <clandmeter> have you tried it with fw disabled?
2022-11-14 09:48:28 <ikke> not yet
2022-11-14 09:48:54 <clandmeter> i changed the default binfmt settings
2022-11-14 09:49:01 <clandmeter> not sure if that was reverted somehow
2022-11-14 09:49:02 <ikke> yes, I recall
2022-11-14 09:49:45 <clandmeter> related to argv
2022-11-14 09:50:53 <ikke> oh, another thing: gbr2-dev1 for some reason started having MTU issues to dl-cdn
2022-11-14 09:52:13 <clandmeter> gbr2 is which provider?
2022-11-14 09:53:25 <ikke> linode
2022-11-14 10:20:37 <clandmeter> looks like binfmt is still correct
2022-11-14 10:20:46 <clandmeter> flags: POCF
2022-11-14 13:49:53 <psykose> ikke: looks like gitlab crashed as nothing really loads
2022-11-14 13:52:21 <ikke> m
2022-11-14 13:52:35 <clandmeter> i can reproduce 
2022-11-14 13:52:52 <ikke> It loads, but slow
2022-11-14 13:52:54 <clandmeter> or better said, i cant reproduce anything
2022-11-14 13:53:18 <clandmeter> mine is still loading 
2022-11-14 13:53:19 <ikke> clandmeter: can you look into it, or are you busy?
2022-11-14 13:53:22 <clandmeter> or whatever its doing?
2022-11-14 13:53:48 <ptrc> looks like linode is having some issues? i can't reach the host at all
2022-11-14 13:54:02 <clandmeter> ikke: what is the host again?
2022-11-14 13:54:10 <ikke> deu2-dev1.alpinelinux.org
2022-11-14 13:54:32 <ikke> nothing on status.linode.com yet
2022-11-14 13:55:55 <clandmeter> looks like puma is eating all resources
2022-11-14 13:56:41 <clandmeter> load average: 12.39, 12.41, 9.40
2022-11-14 13:57:09 <ptrc> responds to ping now, but gitlab nginx times out with HTTP 500
2022-11-14 13:58:05 <clandmeter> hmm
2022-11-14 13:58:16 <clandmeter> postgress is complaining
2022-11-14 13:59:31 <clandmeter> ok i think we are getting hit by something 
2022-11-14 13:59:41 <clandmeter> could be some kind of ddos 
2022-11-14 14:00:51 <clandmeter> FATAL:  remaining connection slots are reserved for non-replication superuser connections
2022-11-14 14:05:55 <ncopa> need help with investigating that?
2022-11-14 14:06:08 <clandmeter> i just restarted the containers
2022-11-14 14:06:12 <clandmeter> see if that makes a diff
2022-11-14 14:06:56 <clandmeter> nah seems not a working solution
2022-11-14 14:07:09 <clandmeter> ncopa: sure if you can help
2022-11-14 14:09:43 <ikke> clandmeter: if you enable prometheus metrics on traefic, we could get some stats about amount of requests
2022-11-14 14:09:59 <clandmeter> im not sure i can do that right now
2022-11-14 14:10:01 <ikke> ok
2022-11-14 14:10:22 <clandmeter> i guess nginx should mention it also
2022-11-14 14:10:35 <clandmeter> but access log is 0
2022-11-14 14:10:45 <clandmeter> i guess it logs t o stdout?
2022-11-14 14:11:29 <clandmeter> does it even use nginx? :)
2022-11-14 14:11:32 <ikke> s6
2022-11-14 14:11:40 <ikke> /srv/docker/gitlab/log/s6/nginx/current
2022-11-14 14:13:24 <clandmeter> ok it has some traffic...
2022-11-14 14:19:51 <clandmeter> looks like its okish now?
2022-11-14 14:20:13 <clandmeter> still getting those postgres msmgs
2022-11-15 10:48:58 <ncopa> seems like gitlab is down again?
2022-11-15 10:53:17 <ncopa> i dont seem to have access to the host
2022-11-15 10:54:33 <ikke> Port 2222
2022-11-15 10:58:15 <ncopa> [11656385.652414] unregister_netdevice: waiting for lo to become free. Usage count = 2
2022-11-15 11:04:02 <ikke> I've seen that before
2022-11-15 11:04:55 <ikke> ftr, I did try to enable dmvpn on that host before. There was some timeout, but I checed and gitlab at least still worked then
2022-11-15 11:07:05 <ikke> connection to server at "172.21.0.3", port 5432 failed: Operation timed out
2022-11-15 11:19:47 <ikke> restarting docker on the machine
2022-11-15 11:29:05 <ikke> rebooting the hosts
2022-11-15 11:29:07 <ikke> host*
2022-11-15 11:29:29 <ncopa> are you rebooting deu2-dev1?
2022-11-15 11:29:50 <ikke> yes,
2022-11-15 11:29:52 <ncopa> yup
2022-11-15 11:29:54 <ncopa> :)
2022-11-15 11:30:04 <ikke> containers were not starting anymore
2022-11-15 11:30:58 <ikke> sorry if you were looking at something
2022-11-15 11:31:19 <ncopa> no worries. i was trying to investigate what was making postgres connections
2022-11-15 11:31:32 <ncopa> and if we could increase the max number of connections
2022-11-15 11:31:56 <ikke> ok, host is booted
2022-11-15 11:34:11 <ikke> something is already listening on the ports that the services need
2022-11-15 11:35:09 <ncopa> do we dare to upgrade the host while at it? there is a newer version of docker
2022-11-15 11:37:12 <ikke> good question
2022-11-15 13:06:51 <PJ[m]> mps: I did
2022-11-15 13:07:17 <PJ[m]> I still consider it spam though
2022-11-15 13:09:12 <PJ[m]> got it on 2 different email addresses
2022-11-15 14:39:27 <mps> PJ[m]: aiee, they are 'fishing' open source devs imo
2022-11-15 19:01:50 <psykose> PJ[m]: i got it twice too
2022-11-15 19:01:57 <psykose> usually i just quick reply a 'fuck off' but was too lazy this time
2022-11-15 19:03:08 <PJ[m]> i got it 4 times
2022-11-15 19:03:24 <PJ[m]> Sep 15, Oct 13, Oct 17, Oct 24
2022-11-15 19:12:19 <psykose> epic
2022-11-15 19:12:45 <ikke> PJ[m]: What's the subject?
2022-11-15 19:14:00 <PJ[m]> 1) Kubernetes Self-Service... (full message at <https://matrix.org/_matrix/media/r0/download/matrix.org/nUENsqTyZIEZdbjxWHweHDmY>)
2022-11-15 19:19:09 <ikke> Nope, don't seem to have received it
2022-11-15 19:28:31 <ikke> ah, I just now received somethign
2022-11-15 20:47:57 <PJ[m]> <ikke> "ah, I just now received somethig..." <- _be careful what you wish for..._
2022-11-15 20:55:06 <mps> some of us have spam filters some don't, I think
2022-11-16 12:06:01 <mps> do we know when will be arm servers ready
2022-11-16 12:06:38 <ikke> They mentioned 4 days
2022-11-16 12:06:51 <ikke> so I would expect them to finish around tomorrow
2022-11-16 12:07:10 <mps> and our FS will be preserved?
2022-11-16 12:07:21 <ikke> yes, that's the goalk
2022-11-16 12:07:22 <ikke> goal*
2022-11-16 12:07:33 <ikke> It's just moving existing servers
2022-11-16 12:07:49 <mps> aha, ok. thanks for info
2022-11-16 12:08:20 <ikke> I did make a backup of all the config, but not the individual containers
2022-11-16 12:08:34 <mps> pufff
2022-11-16 12:08:58 <mps> I hope that we will not have to recreate everything
2022-11-16 12:09:14 <ikke> Me neither
2022-11-16 13:35:29 <clandmeter> i think we will
2022-11-16 13:35:41 <clandmeter> whenever those new servers are ready :)
2022-11-16 14:07:24 <ikke> But we can copy over most 
2022-11-16 15:32:23 <clandmeter> if they get back online yes :)
2022-11-16 16:13:32 <ikke> clandmeter: I suppose we can decommission / stop the mirror on nld3 now?
2022-11-16 20:31:45 <ikke> good news
2022-11-16 20:38:25 <ikke> I'd love to upgrade that machine, but it's already occupied building
2022-11-16 20:43:09 <mps> hehe, nice cristmas tree above
2022-11-16 21:32:02 <clandmeter> nice
2022-11-16 21:32:05 <clandmeter> just got confirmation 
2022-11-16 21:32:17 <clandmeter> ikke: i guess you saw the equinix email
2022-11-16 21:35:20 <ikke> Yes, the one from Jeremy Brooks
2022-11-17 00:35:48 <psykose> ikke: i just noticed the x86_64 container is actually some way newer hardware
2022-11-17 00:35:49 <psykose> wowee
2022-11-17 00:35:57 <psykose> a 7402p
2022-11-17 00:35:58 <psykose> it flies
2022-11-17 10:09:36 <ncopa> i have removed release candidate releases for v3.15 and v3.16 on the builders
2022-11-17 10:12:29 <ncopa> i have now also removed the release candidates for v3.1[0-6] 
2022-11-17 10:12:35 <ncopa> on dl-master
2022-11-17 10:40:51 <ikke> Ok, thanks 
2022-11-17 22:42:57 <ayakael> Gitlab `aarch64` pipelines are giving me failures due to lack of free space. Should I restart the pipeline or wait for ya'll's okay?
2022-11-17 22:47:16 <ptrc> https://ptrc.gay/ByBClggL
2022-11-17 22:47:17 <ptrc> :/
2022-11-19 09:00:19 <mps> ppc64le CI is 'out of work'
2022-11-19 09:10:47 <mps> or it is busy with !41532
2022-11-19 14:35:36 <mps> how we can check at build time in APKBUILD if we are on 32bit or 64bit arch
2022-11-19 14:36:30 <ikke> I don't think there is a single variable that tells 32 or 64 bit
2022-11-19 14:37:05 <ikke> https://tpaste.us/Z4Yo 
2022-11-19 14:37:26 <ikke> s/armv/armv7/
2022-11-19 14:38:50 <mps> ikke: yes, I know this
2022-11-19 14:39:06 <mps> had a hope we have 'trick' somewhere
2022-11-19 14:39:53 <mps> does 'case $CARCH in' supports 'not in' operator
2022-11-19 14:40:07 <ikke> no
2022-11-19 14:40:09 <mps> case $CARCH not in
2022-11-19 14:40:43 <ikke> That's the *) after you specified what you don't want
2022-11-19 14:40:55 <ikke> You can leave a case empty
2022-11-19 14:41:06 <mps> debian have this 'dpkg-architecture -qDEB_HOST_ARCH_BITS'
2022-11-19 14:41:28 <ikke> https://tpaste.us/lV9a 
2022-11-19 14:42:54 <mps> ikke: line 1: syntax error: unexpected word (expecting "in")
2022-11-19 14:43:57 <ikke> mps: `getconf LONG_BIT`
2022-11-19 14:44:37 <mps> aha, this works
2022-11-19 14:44:54 <mps> and I forgot for getconf
2022-11-19 14:45:04 <ikke> I had to look it up again
2022-11-19 14:45:13 <mps> will that work on builders and CIs
2022-11-19 14:45:23 <ikke> I tried it on a build container, and it worked there
2022-11-19 14:45:57 <mps> I tried in dev lxc
2022-11-19 14:46:04 <ikke> also works in docker
2022-11-19 14:46:23 <mps> (getconf -a is useful thing)
2022-11-19 14:46:46 <mps> good, lets try to fix samba MR
2022-11-19 14:47:06 <mps> it fails on 32bit arches
2022-11-19 14:47:14 <mps> ikke: thank you
2022-11-19 14:48:07 <ikke> np
2022-11-19 14:48:22 <mps> 
ACTION wonders who use samba on 32bit machines nowadays

2022-11-19 15:41:53 <Hello71> openwrt
2022-11-19 15:44:48 <mps> and freedombox
2022-11-19 15:45:32 <mps> anyway with ikkes idea MR is fixed
2022-11-20 14:16:50 <ikke> qontinuum: is your mirror unreachable atm? Our monitoring indicates so
2022-11-20 14:17:00 <ikke> And cannot reach it myself eithe
2022-11-20 21:15:29 <ikke> clandmeter: where did we want to move rv64 CI to?
2022-11-21 09:54:54 <clandmeter> ikke: i guess we can rm -rf the mirror
2022-11-21 09:55:06 <clandmeter> i guess traffic is gone
2022-11-21 09:55:08 <ikke> Yes, I think so too 
2022-11-21 09:55:25 <ikke> I moved the dl-t1-* records as well
2022-11-21 11:58:00 <ikke> Guess what happened :)
2022-11-21 13:25:03 <clandmeter> you deleted the mirror?
2022-11-21 13:50:12 <ikke> Yes
2022-11-22 15:38:50 <ncopa> do we need to do anything for v3.17 sho up on pkgs.a.o?
2022-11-22 15:43:11 <mps> do we need !41547 for 3.17
2022-11-22 15:44:07 <ncopa> mps: thats too late now
2022-11-22 15:45:49 <mps> also I think so
2022-11-22 16:09:03 <ncopa> how do I add v3.17 to https://pkgs.alpinelinux.org/packages?
2022-11-22 16:11:48 <panekj> update aports-turbo db wherever it is
2022-11-22 16:13:07 <panekj> i presume the config also has to be updated to include 3.17
2022-11-22 16:17:43 <ncopa> Also, how do I add 3.17 to this: https://security.alpinelinux.org/
2022-11-22 16:18:07 <ncopa> im gonna call it a day. they both can wait til tomorrow
2022-11-22 16:18:15 <panekj> https://secdb.alpinelinux.org/
2022-11-22 19:05:02 <ptrc> ikke: ncopa: it looks like 3.17 is still building from master
2022-11-22 19:07:29 <ikke> ptrc: apparently so
2022-11-22 19:07:43 <ikke> that's not good
2022-11-22 19:08:15 <ikke> at least x86_64
2022-11-22 19:09:35 <ikke> yes, only that one
2022-11-22 19:20:40 <ikke> Not sure what to do with the upgraded packages that already have been built
2022-11-22 19:20:50 <ikke> most are innocent, but there is now a decrepancy
2022-11-22 19:20:57 <ikke> descrepancy*
2022-11-22 19:21:21 <ptrc> maybe just push the same commits to 3.17-stable?
2022-11-22 19:21:22 <panekj> rebuild against 3.17 branch?
2022-11-22 19:22:24 <ptrc> oh, nevermind, there was some bigger changes in the past few minutes
2022-11-22 19:22:58 <ikke> I stopped it while it was building clang
2022-11-22 19:24:26 <ikke> So in theory it's just these commits: https://gitlab.alpinelinux.org/firefly-cpp/aports/-/compare/3.17-stable...4ab7841?from_project_id=1
2022-11-22 19:35:38 <ikke> luckily just in time before the big python3.11 push
2022-11-22 19:42:17 <ptrc> ikke: apparently now edge-x86_64 is building 3.17..?
2022-11-22 19:42:23 <ptrc> something's still wrong
2022-11-22 19:43:02 <ikke> yeah, so the wrong container was changed
2022-11-22 20:04:19 <mps> do we have cbindgen rust binary in alpine?
2022-11-22 20:09:05 <mps> ahm we have it, but not on aarch64
2022-11-22 20:10:19 <mps> oh, it is on aarch64 too
2022-11-22 20:12:27 <mps> hm, I'm searching wrong pkg, I need rust-bindgen
2022-11-22 20:22:43 <ikke> clandmeter: powered off nld3-dev1 🥹
2022-11-22 20:35:25 <mps> nvm, I created local rust-bindgen pkg
2022-11-23 06:20:47 <ikke> clandmeter: do you have time to look at why it does not update? Appears it time-outs on connecting to dl-cdn over https
2022-11-23 06:32:42 <ncopa> morning
2022-11-23 06:32:50 <ncopa> so i messed up the 3.17 release :-/
2022-11-23 06:33:55 <ncopa> ikke: did you fix the build-3-17-x86_64 machine?
2022-11-23 06:34:31 <ikke> I changed the branch on build-3-17-x86_64 and build-edge-x86_64
2022-11-23 06:34:58 <ncopa> what was pushed?
2022-11-23 06:35:19 <ikke> https://gitlab.alpinelinux.org/firefly-cpp/aports/-/compare/3.17-stable...4ab7841?from_project_id=1
2022-11-23 06:35:53 <ncopa> 101 commits
2022-11-23 06:36:00 <ikke> huh?
2022-11-23 06:36:10 <ikke> for me that lists just 5 commits
2022-11-23 06:36:34 <ncopa> did you delete any packages?
2022-11-23 06:36:36 <ncopa> 101
2022-11-23 06:36:36 <ncopa> $ git log --format=oneline v3.17.0..| wc -l
2022-11-23 06:36:46 <ikke> no, I did not delete any packages
2022-11-23 06:37:18 <ikke> The packages that were pushed were just patch releases
2022-11-23 06:37:28 <ikke> so I decided to backport those commits (3) to 3.17-stable
2022-11-23 06:40:29 <ncopa> build-3-17-x86_64 [~/aports]$ find ../packages/ -type f -newer ../packages/releases/x86_64/latest-releases.yaml 
2022-11-23 06:40:46 <ncopa> shows only the 3 packages you pushed. good
2022-11-23 06:40:57 <ncopa> phew! thank you for helping me with this
2022-11-23 06:41:11 <ikke> thanks ptrc for noticing so quickly
2022-11-23 06:41:31 <ncopa> ptrc: thank you!
2022-11-23 06:49:39 <mps> oh my, 3.17 is released ? I see it is
2022-11-23 06:49:50 <ikke> yes, yesterday
2022-11-23 06:50:57 <mps> heh, being dev and don't know, funny :)
2022-11-23 06:51:49 <mps> anyway, good news this morning
2022-11-23 07:04:40 <ncopa> i sent the announcement email to alpine-devel not to announcement mailing list
2022-11-23 07:04:58 <ncopa> so i just sent it to  the announcement mailing list now
2022-11-23 07:57:18 <ncopa> what are the auth DNS servers for alpin.pw?
2022-11-23 08:57:56 <clandmeter> ncopa: 8.3?
2022-11-23 08:58:01 <clandmeter> thats what i have in my config
2022-11-23 08:58:37 <clandmeter> that host is probably down
2022-11-23 08:58:43 <ncopa> same here. but 8.3 does not respond.
2022-11-23 08:59:02 <ikke> 6.3
2022-11-23 08:59:07 <clandmeter> i think ikke turned off something yesterday
2022-11-23 08:59:16 <ncopa> right, thats why I asked :)
2022-11-23 08:59:46 <ncopa> i have .8.33 and .9.33 in my config
2022-11-23 08:59:59 <clandmeter> i guess you will need to use 6.3
2022-11-23 09:00:05 <clandmeter> that is the new host
2022-11-23 09:00:08 <clandmeter> lxc has moved to this one
2022-11-23 09:00:11 <ikke> 6.33
2022-11-23 09:00:38 <clandmeter> ah right 3 and 33
2022-11-23 09:01:24 <ncopa> thanks! that worked
2022-11-23 09:02:35 <ikke> The ns was the last thing I kept up, but figured it was time to shut it down as well
2022-11-23 09:03:22 <ncopa> no problemo
2022-11-23 09:03:30 <ncopa> thank you for taking care of it!
2022-11-23 09:03:55 <ncopa> there was a question about ipv6 for dl-cdn
2022-11-23 09:04:03 <ncopa> https://docs.fastly.com/en/guides/ipv6-support
2022-11-23 09:05:40 <ikke> Yes, we point our dns records to the multistack endpoints 
2022-11-23 09:05:53 <ikke> dualstack*
2022-11-23 09:06:11 <ncopa> Connecting to dl-cdn.alpinelinux.org ([2a04:4e42:38::645]:443)
2022-11-23 09:06:51 <ikke> clandmeter: afaik we migrated everything now 
2022-11-23 09:07:24 <ikke> usa5, nld7 and usa7 are still running, but I think we can power them down as well
2022-11-23 10:30:47 <clandmeter> ikke: ok sounds good.
2022-11-23 10:31:02 <clandmeter> have not touched alpine for a few weeks...
2022-11-23 14:28:05 <ayakael> Hello everyone! I'm trying port the dotnet7 package to armhf. Previously, I had crosscompiled the bootstrap to that arch, which made builds possible in gitlab pipelines. For some reason, the same build on the armhf builder failed - the runtime would just crash. What chipset do the builders emulate so that I can emulate it on my side?
2022-11-23 14:28:14 <ayakael> to port*
2022-11-23 14:37:07 <ncopa> for armhf? that's armv6, only for rpi1 and rip zero as I understand
2022-11-23 14:37:27 <ncopa> does dotnet7 even work with armv6?
2022-11-23 14:38:12 <ncopa> I would think that its not worth the time to get it running on armhf
2022-11-23 14:44:48 <ayakael> Yeah, armv6 support was added in dotnet7 actually via mono. Dotnet allows its runtime to be backed by coreclr (the original) or mono. Unfortunately, crosscompilating on Alpine with mono is not working due to https://github.com/dotnet/runtime/issues/77663. Crosscompilation does work with the coreclr backed runtime, but not the mono backed one. Both
2022-11-23 14:44:48 <ayakael> are with the armv6-alpine-linux-musleabihf crosscompiler so I find it odd that the crosscompiler would work, but then fail on builders, for coreclr backed runtime. For mono backed runtime, it should build via crosscompilation given armv6 support via mono, but it fails.
2022-11-23 14:45:40 <ayakael> I'm at that point of letting it go, indeed it isn't worth it given arm sunsetting arm32 in the next few years.
2022-11-23 14:46:18 <ayakael> Buuuut I'm a completionist. It's a curse.
2022-11-23 14:46:24 <ikke> The builder is armv8l with armhf userspace 
2022-11-23 14:47:02 <ayakael> Right, that's my understanding. Is there a functionnal difference between armv7 and armhf on the builders then?
2022-11-23 14:48:55 <ayakael> Its also my understanding that code built for armv6 should work on armv7, but not the other way around.
2022-11-23 14:52:35 <ikke> No, only difference would be userspace afaik 
2022-11-23 14:54:14 <ayakael> Copy that, are y'all planning to sunset armhf anytime soon?
2022-11-23 14:54:43 <mps> ayakael: last time I tested armhf userspace code on armv7 it worked
2022-11-23 15:05:17 <ikke> ayakael: no concrete plans yet
2022-11-23 15:06:16 <ikke> ayakael: I suppose as long as we have a server with 32-bits support, we'll try to support it, but if we don't, armv7 would be affected as well 
2022-11-23 16:39:18 <ncopa> i have been thinking of sunset armhf. rpi1 is not worth supporting anymore. so its only rpi zero. I think even the latest gen of rpi zero runs armv7 IIRC
2022-11-23 16:39:43 <ncopa> I would like to drop armhf
2022-11-23 16:50:37 <mps> I agree
2022-11-23 16:57:06 <ayakael> Copy that, I won't pursue armhf support for dotnet7 then. Thanks!
2022-11-23 19:53:33 <ayakael> Latest gen of rpi zero (2nd gen) still is based on armv6, FWIW
2022-11-23 20:03:14 <psykose> ikke: aarch64 is stuck fyi :)
2022-11-23 20:36:11 <ikke> and she left again
2022-11-25 08:25:58 <ncopa> I think we need to wait with upgrading the infra to v3.17. dmvpn is broken https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/33759#note_276035
2022-11-25 08:35:34 <ikke> Ack 👍
2022-11-25 09:05:41 <ikke> Hmm, dl-cdn slow for anyone else? 
2022-11-25 11:17:28 <ncopa> dmvpn is fixed on 3.17 now
2022-11-25 11:19:21 <ncopa> i havent noticed any slow dl-cdn
2022-11-26 14:36:23 <mps> we don't have 3.17-stable branch yet?
2022-11-26 14:36:28 <ikke> we do
2022-11-26 14:36:45 <mps> hmm, I don't see it
2022-11-26 14:36:55 <ikke> https://gitlab.alpinelinux.org/alpine/aports/-/tree/3.17-stable
2022-11-26 14:37:05 <mps> oh yes, sorry
2022-11-26 14:37:15 <mps> but I can't push to it
2022-11-26 14:38:16 <ikke> you found it :)
2022-11-26 14:38:21 <mps> ikke: thanks
2022-11-26 14:39:19 <mps> without you I will be blind with gitlab
2022-11-26 14:41:15 <mps> I made local rust-bindgen aports, it is needed to build rust source drivers in kernel, is it ok to add it to testing?
2022-11-26 14:41:26 <ikke> Why wouldn't it be?
2022-11-26 14:41:41 <mps> asking because rust pkgs are new things for me
2022-11-26 14:42:23 <ikke> mps: take a look at a few other rust packages to see how they are packaged
2022-11-26 14:43:00 <mps> ok, will create it with hope that someone with experience will fix it in case that's needed
2022-11-26 14:43:49 <mps> btw, I run asahi linux on apple silicon with gpu driver and xorg
2022-11-26 14:44:30 <panekj> post APKBUILD
2022-11-26 14:44:47 <mps> nearly everything works, only missing color temperature setting
2022-11-26 14:45:17 <mps> panekj: https://tpaste.us/JrX7
2022-11-26 14:46:37 <panekj> apart from single quotes and no check/!check looks fine
2022-11-26 14:46:51 <panekj> also s/aarc64/aarch64
2022-11-26 14:47:38 <mps> ah thank you
2022-11-26 14:47:55 <panekj> and you can change source url
2022-11-26 14:48:02 <panekj> to something like  source="https://github.com/zellij-org/zellij/archive/v$pkgver/zellij-$pkgver.tar.gz"
2022-11-26 14:48:27 <mps> I wonder how I manage to build it with aarc64 arch :)
2022-11-26 14:50:14 <panekj> Did my rust config is finally present in latest abuild?
2022-11-26 14:50:38 <panekj> seems to be
2022-11-26 14:51:07 <panekj> for bindgen you can probably add export CARGO_PROFILE_RELEASE_OPT_LEVEL="3"
2022-11-26 14:51:32 <mps> panekj: what is this option
2022-11-26 14:51:39 <panekj> there isn't much sense for it be 's'
2022-11-26 14:51:56 <panekj> mps: it's -O but for rust
2022-11-26 14:52:13 <mps> hm, yes
2022-11-26 14:52:19 <panekj> check your /etc/abuild.conf since it should have few opts for rust in 3.17
2022-11-26 14:52:25 <panekj> 3.17+
2022-11-26 14:52:32 <mps> but where is "-Os", in rust default
2022-11-26 14:52:37 <mps> ?
2022-11-26 14:52:45 <panekj> in /etc/abuild.conf
2022-11-26 14:52:50 <mps> aha
2022-11-26 14:53:10 <ikke> panekj: it's part of 3.10.0_rc1, which is available on edge
2022-11-26 14:53:50 <mps> panekj: I agree that "Os" is not 'proper'
2022-11-26 14:54:14 <panekj> ikke: well, seems like that version is on 3.17 so good :)
2022-11-26 14:55:22 <panekj> is edge builder using latest abuild (and config)?
2022-11-26 14:55:44 <ikke> panekj: the builders always self-ugprade after all packages have been built
2022-11-26 14:55:55 <ikke> The config needs to be manually updated though
2022-11-26 14:56:09 <panekj> yes, that's what I'm mostly conscious about
2022-11-26 14:56:32 <ikke> But I recently updated the abuild.conf on all builders
2022-11-26 14:56:34 <panekj> i remember someone mentioning that builders have custom config
2022-11-26 14:56:53 <ikke> https://tpaste.us/8bp6 
2022-11-26 14:59:01 <panekj> thanks ikke 
2022-11-26 14:59:27 <panekj> then we can slowly get rid of CARGO_PROFILE_RELEASE_* in APKBUILDs
2022-11-27 19:14:25 <ikke> clandmeter: 3 days before our old infra is definitively gone
2022-11-27 19:47:45 <ikke> clandmeter: anything you need / want to keep on usa5?
2022-11-28 09:21:10 <clandmeter> hi ikke 
2022-11-28 09:21:29 <clandmeter> i dont think so, but let me take a look
2022-11-28 09:23:55 <ikke> I shut it down, so you'd need to boot it again 
2022-11-28 09:25:25 <clandmeter> oh ok
2022-11-28 09:25:29 <clandmeter> i think its ok
2022-11-28 18:13:14 <ayakael> Do ppc64le pipelines tend to go about the same speed as s390x pipelines? Trying to debug really slow build of dotnet7 on that platform.
2022-11-28 22:00:06 <ikke> ayakael: both tend to be a bit on the slower side for some reason
2022-11-28 22:10:02 <ikke> clandmeter: removed our servers (RIP)
2022-11-28 22:10:17 <ikke> decomissioned servers from equinix
2022-11-29 08:17:10 <ncopa> thank you!
2022-11-29 09:42:07 <ncopa> so we are all good with the equinix metal machines?
2022-11-29 09:59:18 <ikke> Yes, we should. We have migrated all services to the new servers, and the old server have been powered off already for a couple of days 
2022-11-29 11:22:06 <clandmeter> nice
2022-11-29 11:22:21 <clandmeter> now we have the next challenge it seems 
2022-11-29 11:28:36 <ikke> clandmeter: which specific challenge? :P
2022-11-29 12:15:19 <ncopa> import tax for for the arm servers to EU apparently
2022-11-29 12:32:04 <ikke> Right
2022-11-29 13:00:33 <clandmeter> yeah
2022-11-29 13:00:47 <clandmeter> sound fun :)
2022-11-29 13:00:54 <clandmeter> but i guess arm will handle it
2022-11-30 16:55:30 <ikke> interesting..