2025-09-01 17:59:08 <ikke> lotheac: I'm working on gitlab-runners on k8s again, but having some strange DNS issue: https://gitlab.alpinelinux.org/kdaudt/hello-world/-/jobs/1994531
2025-09-01 17:59:37 <ikke> It cannot reach the dns servers
2025-09-01 23:11:02 <lotheac> ikke: typo in the domain name you are trying to resolve
2025-09-01 23:11:13 <lotheac> alpinelnux is missing an i
2025-09-01 23:12:41 <lotheac> other than that... i still haven't gotten around to setting up dmvpn on my side to try to replicate any future issues. hopefully will have some time this week
2025-09-02 00:09:04 <lotheac> heh, it's actually *my* typo, not yours :)
2025-09-02 00:11:45 <lotheac> pushed fix to https://gitlab.alpinelinux.org/alpine/infra/k8s/ci-cplane-1/-/merge_requests/7
2025-09-02 05:44:30 <ikke> lotheac: d'oh, that'd explain it. Though after testing, it seems like our helper image is giving an error ("exit code 2"). But I think we can just leave it out. The reason we used a custom image is to make usre all our arches are supported, but for kubernetes, the arches we have are limited anyway
2025-09-02 05:44:42 <ikke> https://gitlab.alpinelinux.org/kdaudt/hello-world/-/jobs/1994956
2025-09-02 05:47:42 <lotheac> hmm, i see
2025-09-02 05:48:08 <lotheac> any hints if you read the container logs?
2025-09-02 05:51:40 <ikke> it already has been removed
2025-09-02 05:52:33 <lotheac> i guess we should set up a logs pipeline to store all the container logs somewhere at some point :)
2025-09-02 10:23:54 <ikke> lotheac: I have already Loki running, but I'd also like to test out victor-logs
2025-09-02 10:24:30 <lotheac> i'm using vector and victorialogs myself
2025-09-02 10:24:57 <ikke> Right, I meant victorialogs
2025-09-02 10:25:16 <lotheac> that's only the storage part, you still need some logfile parser/ingester
2025-09-02 10:25:51 <ikke> Yes, there are several options for that
2025-09-02 10:28:33 <lotheac> yeah, sure are. i've used fluentd, fluent-bit and most recently vector
2025-09-02 10:31:31 <ikke> How is querying victoria-logs (have not checked it extensively yet). With loki, I found that the something like generating a top-n list of a specific field proves difficult
2025-09-02 10:32:13 <lotheac> vlogs is nice especially if you're used to metricsql/promql
2025-09-02 10:33:00 <ikke> I haven't used metricsql/promql yet
2025-09-02 10:33:37 <lotheac> https://docs.victoriametrics.com/victorialogs/logsql/#stats-by-fields
2025-09-02 10:41:00 <lotheac> random example for some request logs in my sso thing: kubernetes.pod_namespace:"authentik" and log.status:* | stats by (_msg) count() cnt | sort by (cnt desc) | limit 10
2025-09-02 10:42:13 <lotheac> for Reasons _msg is the request path in these messages
2025-09-02 10:42:27 <lotheac> but there's other types of messages in these streams too which is why the insistence on "log.status" field existing
2025-09-02 10:43:09 <lotheac> it tells me i've had 181436 health check responses in the past 7d :)
2025-09-02 10:44:21 <ikke> Does it also work with a lot of unique values? I think loki had some hard limits on that
2025-09-02 10:45:17 <lotheac> i haven't really used this at any reasonably big scale, so i don't know... but extrapolating from my experience with victoriametrics at a huge scale i would assume "yes"
2025-09-02 10:45:50 <ikke> The thing I tried to do was to get the top-n amount of IPs from an access log stream
2025-09-02 10:46:31 <lotheac> that happens to be one of the examples in the docs https://docs.victoriametrics.com/victorialogs/logsql/#stats-by-ipv4-buckets
2025-09-02 10:46:49 <ikke> Nice, even by subnet
2025-09-02 11:14:44 <ikke> lotheac: "/bin/sh: syntax error: unexpected redirection "
2025-09-02 11:14:48 <ikke> That's the error from the container
2025-09-02 11:15:44 <lotheac> maybe the command it feeds to the container assumes a certain entrypoint in the helper image
2025-09-02 11:15:56 <ikke> possible
2025-09-02 11:16:15 <ikke> https://gitlab.alpinelinux.org/alpine/infra/docker/gitlab-runner-helper
2025-09-02 11:25:38 <lotheac> registry.gitlab.com/gitlab-org/ci-cd/gitlab-runner-ubi-images/gitlab-runner-helper-ocp:v18.3.0 uses: CMD [\"sh\"]. i guess it also be an issue with what the runner feeds into that shell
2025-09-02 11:28:57 <lotheac> wild guess: https://gitlab.com/gitlab-org/gitlab-runner/-/blob/main/executors/kubernetes/kubernetes.go#L994
2025-09-02 11:29:18 <lotheac> and busybox sh does not like that syntax
2025-09-02 11:29:55 <ikke> ack
2025-09-02 11:31:29 <ikke> <<< is a bashism that bb ash does not support
2025-09-02 11:34:35 <lotheac> yup. i think you could workaround by using CMD ["bash"] if necessary, but well
2025-09-02 11:35:37 <lotheac> if we wanna use custom images i guess we need this too https://docs.gitlab.com/runner/configuration/configuring_runner_operator/#matching-helper-container-and-build-container-user-id-and-group-id
2025-09-02 11:54:02 <ikke> Yeah, but not sure if we really do need a custom helper image for this
2025-09-02 11:56:02 <lotheac> yup, it matters very little if missing architectures are no issue
2025-09-02 15:11:21 <ikke> clandmeter: ^
2025-09-02 15:21:47 <ikke> clandmeter: tried the USB console, but no response
2025-09-02 15:31:25 <ikke> clandmeter: Thanks :-)
2025-09-02 15:40:23 <clandmeter> yw
2025-09-02 15:40:27 <clandmeter> chekc both p[s
2025-09-02 15:40:44 <ikke> Yeah, could log into both
2025-09-04 03:46:22 <vkrishn> looks like builds happening but upload failed, https://build.alpinelinux.org/buildlogs/build-edge-x86_64/community/gitea/gitea-1.24.1-r3.log
2025-09-04 03:46:39 <vkrishn> there are lots in mqtt msgs
2025-09-04 03:50:00 <vkrishn> hope its raising some alerts
2025-09-04 04:04:03 <mio> the builders are busy rebuilding packages in community/, they will be uploaded when the build queue is completed
2025-09-04 04:04:35 <mio> and yes, there are notifications in #alpine-commits
2025-09-04 04:28:20 <vkrishn> ok, thanks, was inferring from build/build-edge-x86_64 "failed" msgs
2025-09-04 04:29:34 <vkrishn> like these, https://tpaste.us/W5lN
2025-09-04 04:35:46 <vkrishn> which says "uploading packages to main" but following msgs are for community "1/74 7091/7165 community/*"
2025-09-04 05:41:09 <ikke> clandmeter: ^ seems to be a network issue, I can reach it through the usb console
2025-09-06 14:51:52 <ikke> restarted the server
2025-09-06 20:30:54 <achill> ERROR: Job failed: prepare environment: waiting for pod running: pulling image "registry.alpinelinux.org/alpine/infra/docker/alpine-gitlab-ci:latest" for container build....
2025-09-06 20:31:04 <achill> looks like the runner still need some interaction
2025-09-06 20:31:07 <achill> https://gitlab.alpinelinux.org/alpine/aports/-/jobs/2001416
2025-09-06 20:32:21 <ikke> No, forgot that I need to apply registry migrations now
2025-09-06 20:32:32 <ikke> Need to automate that
2025-09-06 20:33:33 <achill> ah
2025-09-07 19:52:23 <ikke> lotheac: https://gitlab.alpinelinux.org/alpine/aports/-/jobs/2002691
2025-09-07 19:52:28 <ikke> Job passed now :-)
2025-09-07 19:55:56 <ikke> We still had the runner namespace set to gitlab-ci (which I prefer), but I had to change the role to a cluster role, and then another role binding to that namespace
2025-09-07 19:55:58 <ikke> and now it works
2025-09-08 00:33:14 <lotheac> nice :)
2025-09-08 00:41:25 <lotheac> a bit of a bummer that the priv reduction didn't work, but oh well
2025-09-08 05:27:27 <ikke> lotheac: the image does have a 'gitlab-runner' user, uid=100
2025-09-08 05:27:40 <ikke> So perhaps we just need to set the uid to 100
2025-09-08 05:30:53 <lotheac> the helper image?
2025-09-08 05:31:42 <ikke> No, the runner itself
2025-09-08 05:32:17 <ikke> https://docs.gitlab.com/runner/configuration/configuring_runner_operator/#root-vs-non-root
2025-09-08 05:34:17 <lotheac> i guess we can just patch the podSpec.securityContext to set runAsUser/runAsGroup then
2025-09-08 05:34:42 <lotheac> instead of patching runAsNonRoot
2025-09-08 05:36:22 <ikke> yeah. The user has gid=65533(nogroup)
2025-09-08 05:38:53 <ikke> lotheac: another thing we need to do is to specify requests / limits for the pods that the runner creates, and possibly affinity / nodeselectors
2025-09-08 05:39:26 <ikke> https://docs.gitlab.com/runner/executors/kubernetes/#define-a-list-of-node-affinities
2025-09-08 05:39:28 <lotheac> i thought nodeselectors was already there in the runner.spec.podSpec
2025-09-08 05:40:07 <lotheac> or is that not actually ending up in the actual pods that execute the jobs?
2025-09-08 05:41:12 <lotheac> i struggle to remember the terminology and how the pods are created here, partially because i'm dealing with github action runners in kube at one of my clients and it's similar but different enough :)
2025-09-08 05:41:19 <ikke> I think that's for the runners themselves. I see the runner pods has those node selectors we specified. But the runners use a kubernetes executor that creates pods as well. I don't see anything in the runner config for that
2025-09-08 05:41:35 <lotheac> hmm i see
2025-09-08 05:41:35 <ikke> So you have the runners, the runners then use executors to start jobs
2025-09-08 05:41:52 <ikke> So 1: we deploy the runners somewhere, in our case kubernetes
2025-09-08 05:42:02 <ikke> then we tell the runners how to run jobs, which happens to be kubernetes as well
2025-09-08 05:42:07 <ikke> these jobs are separate pods
2025-09-08 05:42:14 <ikke> created directly by the runners
2025-09-08 05:42:42 <lotheac> right, so then a "runner" is really just something that receives the jobs and creates executors for them
2025-09-08 05:42:42 <ikke> That's also where the helper image comes into play
2025-09-08 05:43:20 <ikke> Yes, the runner is an agent that connects to gitlab and waits for jobs
2025-09-08 05:43:57 <lotheac> https://docs.gitlab.com/runner/executors/kubernetes/#configuration-settings i guess we can configure the kubernetes executor from config.toml to add the resource config and affinities
2025-09-08 05:44:43 <lotheac> runner.spec.config allows to provide config.toml from a configmap
2025-09-08 05:45:17 <ikke> yes, exactly
2025-09-08 05:45:46 <lotheac> ok, that should not be difficult then
2025-09-08 05:46:26 <lotheac> there's even a specific guide here https://docs.gitlab.com/runner/configuration/configuring_runner_operator/#customize-configtoml-with-a-configuration-template
2025-09-08 05:47:21 <lotheac> but... if the runner pods don't actually execute the jobs, i guess the runner pods themselves don't really even need the nodeSelector i configured for them
2025-09-08 05:47:40 <lotheac> i mean, they should be able to run on any arch (but just create pods with affinity to a certain arch etc)
2025-09-08 05:50:54 <ikke> Correct, though it's nice that the runners are associated with the nodes that actually execute the jobs
2025-09-08 05:53:54 <lotheac> well, associated only by happening to have the same architecture, in this case :)
2025-09-08 05:54:32 <lotheac> though i guess you could add podaffinities to the executors to make them prefer being scheduled on the same node as runner pods, but i don't know if there is any point to that
2025-09-08 05:54:54 <ikke> The same node selectors as the jobs 
2025-09-08 05:55:18 <lotheac> ah, did we actually have some other selector than the arch...
2025-09-08 05:55:33 <lotheac> but, yeah, sure, i see your point
2025-09-08 05:56:23 <ikke> alpinelinux.org/arch, which we control
2025-09-08 05:57:03 <lotheac> right
2025-09-08 05:58:49 <ikke> I have to go now
2025-09-08 05:59:17 <lotheac> later
2025-09-08 10:37:05 <ikke> lotheac: Switching from Role to ClusterRole for gitlab-runner-app-role: https://gitlab.alpinelinux.org/alpine/infra/k8s/ci-cplane-1/-/merge_requests/10
2025-09-08 10:38:48 <ikke> (I moved all the *Role* resources to a separate deployment, because it's a pain to manage them from a pipeline without giving the pipeline a lot of permissions.
2025-09-08 10:51:41 <ikke> meh, the operator does not like that :(
2025-09-08 10:52:15 <ikke> So I suppose that means the runner at least needs to be in the same namespace as the jobs
2025-09-08 10:53:16 <ikke> https://gitlab.com/gitlab-org/gl-openshift/gitlab-runner-operator/-/blob/master/controllers/runner_controller.go#L56
2025-09-08 10:57:08 <ikke> lotheac: btw, it works to use runAsUser: 100
2025-09-08 10:57:22 <ikke> 1 gitlab-r  0:00 gitlab-runner run ///
2025-09-08 13:12:41 <lotheac> i would assume that the upstream operator is written in such a fashion as to allow you to create Runner objects in any namespace, and thus require a ClusterRole
2025-09-08 13:13:22 <lotheac> generally speaking for kube operators, you can often make it more controlled by allowing only namespace-specific roles those permissions though
2025-09-08 13:13:42 <lotheac> i can't have a very deep look right now, i'll check again tomorrow
2025-09-08 13:17:02 <ikke> lotheac: I think the assumption is that the runner only creates jobs in the same namespace 
2025-09-08 13:17:42 <ikke> Right now, the runner is in the gitlab-runner-system namespace, while jobs are created in the gitlab-ci namespace 
2025-09-08 13:18:16 <lotheac> if so, then the runner would only need a Role in the target ns to create the executors
2025-09-08 13:18:18 <ikke> The code looks for a RoleBinding with a role reference
2025-09-08 13:22:11 <ikke> And only in the operator and runner namespace 
2025-09-08 13:23:35 <lotheac> there's no reason for the code to be looking for anything to check which privs it has, it should just try to do the thing it's trying to do
2025-09-08 13:24:41 <lotheac> i mean, theoretically, why would this thing need permissions to read role/clusterrole objects anyway ;)
2025-09-08 13:24:56 <lotheac> but yeah, might be that it's implemented in an... interesting way
2025-09-08 13:27:01 <ikke> Apparently it tries to manage them 
2025-09-08 13:27:27 <ikke> "reconcile"
2025-09-08 13:29:25 <mio> hello, sorry if interrupting a discussion, what's the status of the 3.22 riscv64 builder?
2025-09-08 13:30:21 <mio> it seems to be on the same aport for some time, just wondered if it was temporarily offline for a test
2025-09-08 14:35:36 <ikke> mio: possibly related to a reboot 
2025-09-08 14:37:58 <mio> ikke: okay, thanks for checking. was looking at the error log to confirm whether it was recurring
2025-09-08 14:38:40 <mio> (in this particular case)
2025-09-08 15:21:07 <ikke> nope, it was just stuck on go
2025-09-08 15:21:15 <ikke> "just:"
2025-09-08 15:34:59 <mio> okay. checked with mzh about the initial cmd/compile error and they helped report it upstream
2025-09-08 15:37:02 <mio> initially thought it had recurred, but seems it's only the first time? would be good to confirm if it it is consistent or just transient. there was a report upstream of a similar-looking issue
2025-09-08 15:38:26 <mio> https://github.com/golang/go/issues/62355 possible memory corruption
2025-09-08 15:39:10 <mio> (thanks to mzh for the info)
2025-09-08 15:41:41 <mio> not sure if it is the same
2025-09-08 15:49:39 <mio> looks to be okay now, thanks for restarting it
2025-09-08 18:48:06 <ikke> Thanks for notifying
2025-09-08 18:59:22 <mio> :)
2025-09-08 19:00:04 <mio> btw, someone opened a 3.22 tailscale upgrade MR, is tailscale still pending kernel patch?
2025-09-08 19:00:08 <mio> !89629
2025-09-08 19:00:20 <ikke> yes
2025-09-08 19:00:48 <ikke> The kernel patch still needs to be accepted upstream afaik
2025-09-08 19:02:11 <mio> okay, thanks. will leave a comment on the pending status
2025-09-08 19:03:15 <ikke> tailscale issue: https://github.com/tailscale/tailscale/issues/16966 
2025-09-08 19:13:06 <mio> thanks for the link, included it for reference
2025-09-09 00:14:04 <lotheac> ikke: ah, ok. "reconcile" is the verb that is used when a kube operator owns other objects, and continually watches them to keep them in the desired state
2025-09-09 00:15:21 <lotheac> on the github side it was possible to configure the operator so that it would only watch for AutoscalingRunnerSet objects in a single namespace (in which case it would not try to manage Roles). i'll check if the gitlab runner operator doesn't have something like that too
2025-09-09 00:18:40 <lotheac> and even if it doesn't... we could potentially just pre-create the Roles it tries to manage without actually giving it privs to do so
2025-09-09 00:22:26 <lotheac> the code looks like it does reconcileRBAC() unconditionally though https://gitlab.com/gitlab-org/gl-openshift/gitlab-runner-operator/-/blob/master/controllers/runner_controller.go
2025-09-09 00:27:15 <lotheac> clusterrole/gitlab-runner-manager-role is the thing with the most privileges; in theory we could remove the ClusterRoleBinding related to that and just have a RoleBinding in gitlab-runner-system and gitlab-ci ns'es if you wanted to reduce privs (but still allow the operator to manage the target ns roles)
2025-09-09 10:26:42 <ikke> clandmeter: It's unavailable again, no output on serial
2025-09-09 10:33:51 <ikke> Wondering why it is unstable again
2025-09-09 13:39:10 <achill> someone minds looking at !103?
2025-09-09 13:39:14 <achill> eee
2025-09-09 13:39:18 <achill> https://gitlab.alpinelinux.org/alpine/infra/aports-qa-bot/-/merge_requests/103
2025-09-09 14:58:00 <omni> getting Bad Gateway on pkgs.a.o, f ex https://pkgs.alpinelinux.org/contents?file=*xen*&path=&name=&branch=edge&repo=&arch=
2025-09-09 15:16:53 <ikke> omni: can you limit it on a single arch?
2025-09-09 15:17:02 <ikke> path queries are expensive and there is a lot of data
2025-09-09 15:19:58 <pj> you would probably drop a lot of load if the filters were stored in cookies
2025-09-09 15:28:08 <omni> ikke: ah, that was unintentional
2025-09-09 15:28:10 <omni> to not limit
2025-09-09 15:31:40 <ikke> pj: apk-browser does not cause a lot of continuous load
2025-09-09 15:36:48 <pj> "lot" as in big percentage of the total of apk-browser, not total of whatever is on the host
2025-09-09 15:38:00 <pj> every time I open pkgs.a.o I have to change arch because I get duplicate stuff and in content search it just dies
2025-09-09 15:39:40 <pj> or maybe just default to x86_64 like in package search :>
2025-09-09 15:44:43 <ikke> Feel free to make an MR, achill reviews those
2025-09-09 15:49:15 <ikke> (or an issue)
2025-09-09 15:50:21 <achill> the default is already x86_64 or pkgs.a.o/packages
2025-09-09 15:56:03 <ikke> But not for contents
2025-09-09 16:02:11 <pj> "like in package search"
2025-09-09 16:02:52 <pj> if it won't fly out of my head until I have some spare time
2025-09-09 16:04:19 <achill> oh right i should read full sentences
2025-09-09 16:16:51 <clandmeter> if we enable cookies on pkgs, we would need to have a nice and shiny popup to opt in :)
2025-09-09 16:18:25 <ikke> If they're purely functional, we don't
2025-09-09 16:18:27 <pj> for?
2025-09-09 16:43:02 <clandmeter> you sure? i read mixed msgs
2025-09-09 16:49:52 <pj> strictly necessary cookies are exception, anything else requires user consent before cookie usage
2025-09-09 16:50:22 <pj> strictly necessary cookies are those that are required for the proper website usage such as remembering preferences
2025-09-09 20:01:07 <omni> I almost feel like the consent pop-ups are designed to annoy users into thinking that GDPR is a bad thing, as well as trick them into enabling non-essential cookies
2025-09-09 20:01:25 <ikke> almost?
2025-09-09 20:01:42 <ikke> gdpr does not say anything about cookie popups
2025-09-09 20:02:18 <pj> cookie banners were not a requirement by EU
2025-09-09 20:02:31 <pj> but it's malicious compliance
2025-09-09 20:02:51 <omni> right
2025-09-09 20:03:17 <pj> consent pop-ups I believe are an actual requirement, but it's not required to be a pop-up
2025-09-09 20:03:27 <pj> it has to be visible, accessible, etc.
2025-09-09 20:03:51 <pj> but then again every popup is malciously crafted so that you won't opt-out
2025-09-09 20:03:53 <omni> but wrt cookies on pkgs, it would be nice if it also works for users blocking them
2025-09-09 20:04:19 <pj> I'm not sure how saving preference would work without cookies
2025-09-09 20:04:30 <pj> unless you want to use JS
2025-09-09 20:04:36 <pj> and localstorage or something
2025-09-09 20:05:47 <omni> that was not what I meant, just that functionality wouldn't be broken if you have cookies blocked
2025-09-09 20:11:30 <pj> it's just a dropdown list tho
2025-09-10 19:23:27 <ikke> lotheac: fyi, the x86_64 runner is actually already accepting jobs :-)
2025-09-11 00:35:28 <lotheac> oh, nice :)
2025-09-11 15:17:42 <ncopa> do we have stats/zabbix charts on what CI runners/architectures are busy, or how long time jobs needs to wait etc
2025-09-11 15:30:14 <ikke> ncopa: yes
2025-09-11 15:30:29 <ikke> https://zabbix.alpinelinux.org/zabbix.php?action=dashboard.view&dashboardid=2&from=now-12h&to=now&page=3
2025-09-11 15:30:38 <ikke> Not necessarily how long they have to wait
2025-09-11 15:31:29 <ikke> wow, 48 pending jobs atm
2025-09-11 15:44:28 <ncopa> thanks! that was the one i was looking for
2025-09-11 15:45:26 <ncopa> do we have pending jobs per arch?
2025-09-11 15:52:52 <ikke> Yes
2025-09-11 15:54:15 <ikke> https://pasteboard.co/I6Re46EwV1dd.jpg
2025-09-11 15:55:20 <ikke> https://zabbix.alpinelinux.org/history.php?action=batchgraph&all_items=1&itemids%5B40720%5D=40720&itemids%5B40741%5D=40741&itemids%5B40717%5D=40717&itemids%5B40726%5D=40726&itemids%5B39913%5D=39913&itemids%5B40738%5D=40738&itemids%5B40723%5D=40723&itemids%5B40735%5D=40735&itemids%5B40732%5D=40732&itemids%5B40729%5D=40729&graphtype=0
2025-09-12 10:30:42 <ikke> rebooting the arm builder for a new kernel
2025-09-12 16:11:13 <PureTryOut> Is it just me or is the x86_64 CI an insane amount slower than the other architectures?
2025-09-12 16:11:58 <ikke> Depends on the runner
2025-09-12 16:12:49 <ikke> We have some vms that have less then optimal IO
2025-09-12 16:28:25 <PureTryOut> Hmm ok. For a while now I've noticed x86_64 finished way later than other arches
2025-09-12 20:31:08 <achill> gitlab down
2025-09-12 20:34:59 <f_> gitlab up
2025-09-12 20:37:11 <ikke> planned
2025-09-14 17:18:29 <ikke> (on purpose)
2025-09-14 20:23:40 <pj-> ikke: clandmeter: could we have more ops in #alpine-offtopic (or is ncopa the only one with op powers)? additionally could you op me there so I can clean up it from matrix side?
2025-09-15 11:08:26 <omni> getting a lot of gitlab errors
2025-09-15 11:08:33 <omni> like "An error occurred while fetching the assigned milestone of the selected merge_request."
2025-09-15 11:20:57 <ikke> Oh
2025-09-15 11:27:06 <ikke> omni: can you check now?
2025-09-15 11:43:16 <omni> works
2025-09-15 11:44:10 <omni> but interesting that, when it didn't, the tab process began using local resources like crazy
2025-09-15 15:10:59 <f_> ikke: you around?
2025-09-15 15:13:13 <ikke> I'm now
2025-09-16 08:42:46 <ncopa> gitlab is struggling. There are 24 concurrent git pack-objects now
2025-09-16 08:43:14 <ncopa> which isnt too bad actually
2025-09-16 08:55:06 <ncopa> its up to 33-35 now
2025-09-16 08:55:29 <ncopa> I added 2G swap on zram
2025-09-16 08:59:41 <omni> do you also have swap on disk?
2025-09-16 09:01:13 <omni> in that case it might be an idea to turn on zswap instead of adding zram swap
2025-09-16 09:18:15 <ikke> I don't think adding swap helps a lot
2025-09-18 11:13:46 <omni> riscv64 builders look stuck, at least for stable releases
2025-09-18 15:51:12 <ncopa> i restarted build-3-2[12]-riscv64
2025-09-18 15:54:15 <achill> thanks
2025-09-18 15:55:19 <ncopa> mariadb on build-3-21-riscv64 seems to be caused by gcc segfaulting apparently. we need to keep an eye on that
2025-09-18 17:20:41 <Saijin_Naib[m]> Hey, I was wondering if I could get guest(?) permissions in GitLab so that I can review/merge MRs assigned to me for my own aports
2025-09-18 17:28:55 <ikke> Saijin_Naib[m]: You should already have it (I've added you yesterday or so)
2025-09-18 17:29:06 <Saijin_Naib[m]> Ah, thanks, ikke!
2025-09-18 17:30:20 <ikke> You should've received an email about that
2025-09-18 17:47:30 <Saijin_Naib[m]> Yeah, they are not delivering to my outlook, and I have no idea why
2025-09-18 17:47:49 <Saijin_Naib[m]> Do you know what email address pkgs.alpinelinux.org and gitlab.alpinelinux.org will be sending outbound with?
2025-09-18 17:47:54 <Saijin_Naib[m]> Maybe I can force them in the allow-list
2025-09-18 18:06:41 <ikke> outlook has been blocking our mails for quite some time
2025-09-18 18:07:01 <ikke> gitlab@gitlab.alpinelinux.org is the origin
2025-09-18 18:07:41 <Saijin_Naib[m]> Fantastic... crap
2025-09-19 20:53:27 <fossdd> uhh something weird is going on with the builders i think
2025-09-19 20:53:33 <fossdd> all failed without a package. hmmm
2025-09-20 05:28:53 <ikke> /bin/sh is dash
2025-09-20 05:38:10 <ikke> Something caused dash-binsh to be installed, which then replaced busybox-binsh
2025-09-20 07:55:54 <achill> ohhh interesting
2025-09-20 07:56:16 <achill> guess thats a +1 for rootbld
2025-09-20 10:49:39 <omni> fungw has dash-binsh as checkdepends and was upgraded right before
2025-09-20 10:50:13 <omni> but it didn't happen when fungw was introduced by the end of last year
2025-09-20 10:50:24 <omni> did it?
2025-09-20 11:24:18 <ikke> omni: perhaps difference in apkv3 behavior?
2025-09-20 11:43:31 <ikke> Somehow apk is preferring dash-binsh
2025-09-20 17:19:54 <Sertonix[m]> apk might not switch back automatically due to the world file being fulfilled. Maybe try `apk add busybox-binsh && apk del busybox-binsh`
2025-09-20 17:25:56 <ikke> Sertonix[m]: that's what I tried, but it reverts back to dash
2025-09-20 17:38:01 <Sertonix[m]> Then it very much looks like a solver bug. You could try to build apk with solver debug logs, run apk del --simulate busybox-binsh and then create an issue with the logs. Sometimes it's very difficult to track down the bug though.
2025-09-20 17:50:42 <ikke> I guess this is the issue: `prefer removal hint`
2025-09-20 17:50:58 <ikke> The fact that we delete it causes the solver to deprioritize it
2025-09-20 17:54:29 <ikke> So the solution would be to add '!dash-binsh` and remove that again
2025-09-20 17:54:35 <ikke> But it feels brittle
2025-09-20 17:58:48 <ikke> V
2025-09-20 17:58:50 <ikke> https://tpaste.us/N5Lk 
2025-09-20 18:00:48 <Sertonix[m]> Does by any chance apk upgrade --available work after apk del busybox-binsh?
2025-09-20 18:04:29 <ikke> Nope, does not select busybox-binsh
2025-09-20 18:05:00 <ikke> adding dash-binsh to /etc/apk/world and then apk del dash-binsh works
2025-09-20 18:09:06 <ikke> cely: is there a reason fungw explicitly depends on dash-binsh?
2025-09-21 04:28:02 <cely> Sorry about that, dash-binsh is in fungw checkdepends because tests have some issue with busybox sh, while check() does not fail, i can see this diff in the build log: https://tpaste.us/xDdJ
2025-09-21 04:34:57 <cely> I see there is an apk-tools issue now, so ping me again if it's decided that this is not an apk-tools issue, and i'll see what i can do about fungw tests
2025-09-21 04:37:10 <cely> Speaking of depends not getting cleaned after a build, i actually encountered something like that recently, while building alpine-base with apk-tools3 (i think the last time the builders built this was with apk-tools2)
2025-09-21 04:40:15 <cely> So, from what i remember, alpine-base pulled in busybox-suid, after the build completed, busybox-suid was removed, and i had dangling symlinks for the commands bbsuid installs, they did not get switched back to busybox
2025-09-21 04:41:16 <cely> Not sure if that was due to something specific in my build environment though
2025-09-21 04:48:05 <cely> So if no one else sees the alpine-base busybox-suid issue, then disregard what i just said
2025-09-21 08:54:34 <Sertonix[m]> These symlinks are handled by a install scripts and not apk-tools. In this case the .pre-deinstall script from busybox-extras should probably be copied to busybox-suid
2025-09-21 10:26:01 <ikke> cely: fabled committed a fix to prefer higher provider_priority, so the issue should be fixed once apk-tools is updated
2025-09-21 16:09:54 <cely> Ok :)
2025-09-21 19:27:36 <iggy> would it be a bad idea (aside from having to update a million places) to use `/bin/busybox sh` in places that actually do need ash or can't work with dash/bash/etc
2025-09-21 19:50:20 <Sertonix[m]> The typical way is /bin/ash. For the sake of consistency I would recommend it