2022-02-01 08:50:38 ikke: did you test that MR? 2022-02-01 08:51:09 The last one? 2022-02-01 08:51:14 nod 2022-02-01 08:51:20 I did 2022-02-01 08:52:11 you are comparing major and minor version numbers correct? 2022-02-01 08:53:08 i guess this works because the sorting in the json is specific 2022-02-01 08:53:37 I do just compare major and minor indeed 2022-02-01 08:53:58 so if major is smaller it will compare minor 2022-02-01 08:54:35 then of minor is bigger it would return true 2022-02-01 08:54:50 even if major was smaller 2022-02-01 08:55:08 If major is smaller, it returns true 2022-02-01 08:55:27 Then minor is ignored 2022-02-01 08:58:05 what if compare 3.11 with 2.12? 2022-02-01 09:05:23 I see 2022-02-01 09:07:23 but i guess the json is already sorted, so it should kind of work :) 2022-02-01 09:07:31 Only if they're equal, it should compare the minor version 2022-02-01 09:09:11 btw, if the format for branch is fixed, i guess you can just use lua.match 2022-02-01 09:09:28 err string match 2022-02-01 09:10:14 i can add an example on gitlab 2022-02-01 09:32:40 Sure 2022-02-02 20:30:35 git.a.o has been significantly slower the past week or so, any idea if it's under heavy load or? 2022-02-02 20:31:47 zv: not particularly.. 2022-02-02 20:31:53 load on the host is <1 2022-02-02 20:32:31 I've reastarted uwsgi, it is faster again 2022-02-02 20:36:13 thanks 2022-02-02 21:29:50 ikke: looks like it is now ok 2022-02-02 21:30:43 got alert about 10 mins ago for 'my' business machines 2022-02-02 21:30:57 right 2022-02-02 21:31:08 zabbix agrees 2022-02-02 21:31:44 i think libera's services went offline due to that? 2022-02-02 21:31:47 they quoted "unexpected networking fun at a hub provider" 2022-02-02 21:32:49 yeah, I was suspecting that as well 2022-02-02 21:32:54 I also saw a netsplic 2022-02-02 21:32:55 danieli: that at libera was few hour ago 2022-02-02 21:33:22 linode had about 25 mins network problem 2022-02-02 21:33:23 the notification was 10 minutes ago 2022-02-02 21:33:27 i got a global message about 10 minutes ago yeah 2022-02-02 21:33:31 i see 2022-02-02 21:33:52 it's irrelevant here but found it an interesting coincidence 2022-02-02 21:35:52 danieli: ah I see another libera warning, you are right 2022-02-02 21:37:35 though I noticed intermittent network problem with linode in Frankfurt for about hour or hour and a half ago 2022-02-02 21:46:07 my wg connection stopped working 2022-02-02 21:46:07 mps: Me too, but didn't realize why the reason was 2022-02-02 21:47:20 heh, old alert 2022-02-02 23:12:15 ikke: indeed. our networking team scheduled that emergency maintenance earlier this afternoon, although i'm not exactly sure what it entailed. i can try to gather details if you'd like 2022-02-03 05:05:51 blaboon: this was added later 2022-02-03 05:05:58 https://status.linode.com/incidents/gv3fmfncfjbn 2022-02-03 08:09:20 ikke: did you see there was another MR for branch sorting for mirrors? 2022-02-03 08:11:01 No 2022-02-03 08:13:04 If so, and it works good, let's use the other MR 2022-02-03 08:28:32 its a year old :) 2022-02-03 08:28:54 never noticed it :( 2022-02-03 11:34:20 clandmeter: thinking about upgrading gitlab tonight 2022-02-03 11:36:18 do it right now 2022-02-03 11:36:30 I only have limited time now 2022-02-03 11:36:32 :p 2022-02-03 11:36:53 So I'd rather do it when I have plenty of time and have time to fix potential issues 2022-02-03 12:47:58 Sounds like a good idea 2022-02-03 19:50:20 psykose: mold 1.0.3 is released 2022-02-03 19:50:25 it is 2022-02-03 19:50:36 but i am waiting for an actual release with all the changes to enable tests 2022-02-03 19:50:43 so i don't have to backport 5 patches 2022-02-03 19:51:00 ok 2022-02-03 19:51:33 upstream fixed all the tests on musl/alpine :) 2022-02-03 19:52:28 I didn't looked details 2022-02-04 05:38:04 https://about.gitlab.com/releases/2022/02/03/security-release-gitlab-14-7-1-released/ 2022-02-04 07:39:22 ikke: :) 2022-02-04 07:39:24 upgrade time ;-) 2022-02-04 07:40:01 Do you want to restart gitlab? I already staged it 2022-02-04 07:40:24 me? 2022-02-04 07:41:05 Yes, otherwise I'll do it later 2022-02-04 07:41:19 you alraedy fetched t he containers? 2022-02-04 07:41:43 Yes 2022-02-04 07:41:52 lets do it later, i just came in and in my state to do adhoc stuff 2022-02-04 07:42:01 Ok 2022-02-04 07:42:24 s/my/no 2022-02-04 07:42:24 clandmeter meant to say: lets do it later, i just came in and in no state to do adhoc stuff 2022-02-04 07:43:24 grrr i just came into office to see whats wrong with my new router, just to find out my vpn uses a different subnet... even routing is difficult... 2022-02-07 09:09:48 seems like this channel doesn't require registered nicks unlike all other Alpine channels, is that on purpose? 2022-02-07 10:13:11 PureTryOut: most probably because spam bots didnt join here yet so nobody put register flag on it 2022-02-07 10:14:31 and those usually aim at channels with more people 2022-02-07 10:18:02 I'd personally do it before they start joining 😉 2022-02-08 21:55:55 clandmeter: someone is asking about an outdated boot.ipxe: https://gitlab.alpinelinux.org/alpine/infra/alpine-netboot/-/issues/2 2022-02-09 07:38:23 morning 2022-02-09 07:38:48 we should update it to use releases.json 2022-02-12 20:14:44 clandmeter: psykose indicated wanting to help the infra team 2022-02-12 20:15:17 maybe i did 2022-02-12 21:01:12 incidentally gitlab looks down 2022-02-12 21:01:35 It was most definitely not me 2022-02-12 21:01:47 uh oh.. 2022-02-12 21:01:48 what did you do 2022-02-12 21:02:17 It should be back in a minute 2022-02-12 21:02:50 done 2022-02-12 21:02:58 changed the compose config on the prod server instead of test 2022-02-12 21:03:18 heh 2022-02-12 21:03:38 i love doing that one :) 2022-02-12 21:04:22 And I even have different prompt colors to distinguish them 2022-02-12 21:05:29 time to make the loud prompt a bit louder then 2022-02-12 21:06:05 I have a script to turn prod into test (because test is a backup restore from prod) 2022-02-12 21:06:18 I needed to update the script on prod so that it would come with the next backup restore 2022-02-12 21:07:07 mhm 2022-02-12 22:24:36 Sure help is always appreciated 2022-02-12 22:25:55 We could talk and see what you can help with 2022-02-12 22:26:18 let me know when you have time 2022-02-12 22:29:32 clandmeter: gitlab-test upgraded to 3.15 :-) 2022-02-12 22:30:16 tomorrow or whenever is fine 2022-02-12 22:30:19 Are you sure its test? :p 2022-02-12 22:30:33 clandmeter: no :P 2022-02-12 22:30:37 though i just want ikke to throw things at me and to learn something :p 2022-02-12 22:30:47 or now.. though i'm not exactly sober 2022-02-12 22:31:04 Well, most of our stack is either lxc or docker + composer 2022-02-12 22:31:06 it’s late here 2022-02-12 22:31:08 compose* 2022-02-12 22:31:10 here too 2022-02-12 22:31:14 I was drunk before 2022-02-12 22:31:24 never used the former, used the latter some amount 2022-02-12 22:31:43 psykose: you use lxc containers :P 2022-02-12 22:31:52 use but not manage :p 2022-02-12 22:32:36 Lxc not lxd 2022-02-12 22:32:49 some confuse the two 2022-02-12 22:33:07 i don't, although i did not check what the usability difference is 2022-02-12 22:33:09 We use dmvpn between most of our infra 2022-02-12 22:36:47 And we have netbox 2022-02-12 22:36:53 yes 2022-02-12 22:37:07 though you need vpn access to get to it 2022-02-12 22:37:14 either openvpn or wireguard 2022-02-12 22:37:27 lots of cool things 2022-02-12 22:37:35 or dmvpn 2022-02-12 22:37:40 right 2022-02-12 22:38:03 And we have zabbix, as you may have noticed in this channel :) 2022-02-12 22:38:17 yes 2022-02-12 22:38:21 We need to automate more things 2022-02-12 22:38:23 and noticed it is gated behind a login 2022-02-12 22:38:24 :p 2022-02-12 22:39:07 We have too much gear to manage manually 2022-02-12 22:39:30 but somehow managing by 2022-02-12 22:40:48 Because of you :) 2022-02-12 22:41:25 ikke definitely knows too much 2022-02-12 22:41:36 I appologize 2022-02-12 22:41:53 but yes, you can throw whatever arcane knowledge at me too 2022-02-12 22:42:07 i have not exactly worked with (any of the above), docker aside, but i have the time 2022-02-12 22:42:18 and i definitely have enough time to try and automate a thing or two 2022-02-12 22:42:27 Sounds good 2022-02-12 22:42:49 and perhaps it slightly lowers the bus factor :) 2022-02-12 22:42:50 https://gitlab.alpinelinux.org/alpine/infra/docker and https://gitlab.alpinelinux.org/alpine/infra/compose should contain most of the relevant code / confdig 2022-02-12 22:43:16 Right now, clandmeter and I manually deploy and config these projects on a server 2022-02-12 22:43:33 mhm 2022-02-12 22:43:37 So when there is some kind of update, we need to manually go to the server and update that 2022-02-12 22:43:49 i remember ariadne mentioning moving to k8s at some point, for this 2022-02-12 22:44:11 Most of us think that’s overkill 2022-02-12 22:44:19 i wouldn't disagree, though i've never touched it 2022-02-12 22:44:57 yeah, it would increase the complexity by ∞ 2022-02-12 22:45:06 just one infinity? :p 2022-02-12 22:46:19 The idea of having a platform where you can just deploy an application is nice 2022-02-12 22:47:20 But not all the complexity that comes with it 2022-02-12 22:48:41 We have mainly to hosting providers for servers 2022-02-12 22:48:51 equinix provides bare metal hardware 2022-02-12 22:48:54 and linode virtual machines 2022-02-12 22:49:11 mm 2022-02-12 22:49:22 iirc it was also ibm that provided the ppc machines 2022-02-12 22:49:26 yes 2022-02-12 22:49:32 I excluded the builders 2022-02-12 22:49:39 right 2022-02-12 22:52:18 nld3 and nld5 are two large lxc hosts 2022-02-12 22:52:26 nld5 is used for developer containers 2022-02-12 22:52:46 nld3 hosts applications, for example www.alpinelinux.org and some other things 2022-02-12 22:53:26 makes sense 2022-02-12 22:53:56 gbr2, deu1 are docker + compose hosts, both for applications 2022-02-12 22:54:07 deu2 is dedicated for gitlab 2022-02-12 22:54:24 We have this information in netbox 2022-02-12 22:56:38 We use gitlab ci to build our docker images and pus them to docker hub 2022-02-12 22:57:16 makes sense so far 2022-02-12 22:57:20 For most images, we have a weekly automatic rebuild scheduled 2022-02-12 22:58:28 i noticed :edge is not in that list 2022-02-12 22:58:42 yea 2022-02-12 22:58:46 it's built by docker itself 2022-02-12 22:59:09 ah right, the specific docker images 2022-02-12 22:59:17 We need to explicitly tag an edge snapshot, and then ncopa makes a PR against docker library 2022-02-12 22:59:22 mhm 2022-02-12 22:59:30 lots of effort 2022-02-12 22:59:43 it should really be automatic whenever something changes in the base dependency list 2022-02-12 22:59:47 yeah 2022-02-12 23:00:03 But docker does not make that easy 2022-02-12 23:00:08 yep 2022-02-12 23:00:18 i remember recently there was the self-hosting registry discussion 2022-02-12 23:00:22 yes 2022-02-12 23:00:37 funnily docker-registry just tagged a release after some years 2022-02-12 23:00:43 https://gitlab.alpinelinux.org/alpine/tsc/-/issues/34 2022-02-12 23:00:57 mhm 2022-02-12 23:01:59 One hybrid option would be us providing a base image, and docker then pulling from that image for alpine:edge 2022-02-12 23:02:26 but that would still mean it's not updated as fast 2022-02-12 23:03:20 it sounds faster than now 2022-02-12 23:03:46 I think you'd still need to do a PR against docker-library for a rebuild 2022-02-12 23:04:07 Not sure how the other projects do it 2022-02-12 23:04:33 not k8s for alpine but at $day job yes 2022-02-12 23:04:46 ah 2022-02-12 23:06:18 hmm: https://github.com/docker-library/official-images/issues/11705 2022-02-12 23:06:44 Apparently bad experiences with pulling from external 2022-02-12 23:08:17 i'm not a huge fan of the whole 'everything must be on docker hub' thing anyway 2022-02-12 23:08:42 i would say it's slowly gained less and less 'marketshare' so to speak as time has gone on, and at some point nobody will really care about these official docker images anyway 2022-02-12 23:09:45 but i would guess there will be at least a few issues with having an external-only image instead of maintaining two, so it will end up on the latter for us, like in that issue 2022-02-12 23:09:47 docker adding pull limits at least drove a lot of users using custom registries 2022-02-12 23:09:54 yep 2022-02-12 23:10:12 i mean we can just let tianon figure it out 2022-02-12 23:10:29 he (his company) is being paid a lot of money 2022-02-12 23:12:41 by docker? 2022-02-12 23:12:51 tianon has said privately that they will give us an exception 2022-02-12 23:12:57 to do the same 2022-02-12 23:13:16 if he is writing cheques he cannot cash it’s not our problem frankly 2022-02-12 23:13:24 maybe 2022-02-12 23:13:35 but i'm not a massive fan of the nepotism model 2022-02-12 23:14:41 i’m not either 2022-02-12 23:14:52 but breaking alpine:… is also a no go 2022-02-12 23:14:59 of course 2022-02-12 23:15:05 we do need a better solution that we control though 2022-02-12 23:22:34 the solution i am building can be driven by any CI system, we just happen to use kubernetes and tekton at work 2022-02-12 23:23:11 i think, e.g. it could be adapted to work with gitlab CI 2022-02-12 23:23:15 its modular 2022-02-12 23:24:48 for example, there is a build tool (which replaces abuild with something that is more declarative), a container image builder (which is really cool, you can build alpine containers declaratively instead of with that dockerfile nonsense), a repository management service (which accepts built APKs and puts them into a repository after doing acceptance tests) 2022-02-12 23:25:00 for alpine, we would just use the repository management service + Gitlab CI 2022-02-12 23:25:52 the best part is that the repository management service can just shove the repo in an S3 bucket or whatever 2022-02-12 23:26:40 We can use linode object storage (which is s3 compattible) 2022-02-12 23:30:48 yep 2022-02-12 23:31:14 and then we do not have this "our builders have to maintain the repos" issue 2022-02-12 23:31:27 anyway check back with me in a few weeks 2022-02-12 23:31:31 i should be able to demo it then 2022-02-12 23:31:45 though what i show will be very... tekton 2022-02-12 23:33:27 alternatively, we can wait for apk3 2022-02-12 23:33:37 which can manage containers 2022-02-12 23:33:42 isn't that a hell of an easteregg 2022-02-12 23:33:49 what if we just made an alpine 2 2022-02-12 23:33:58 /s 2022-02-13 17:20:44 deu2-dev1 successfully upgraded to alpine 3.15 2022-02-13 17:20:56 It's crazy how fast it is 2022-02-13 17:21:16 the upgrade time? 2022-02-13 17:21:19 within 5 minutes a full system upgrade 2022-02-13 17:21:21 yes 2022-02-13 17:21:22 hehe 2022-02-13 17:21:31 i jump between release branches on the dev containers sometimes 2022-02-13 17:21:35 ~15 seconds 2022-02-13 17:21:35 nod 2022-02-13 17:22:17 On Ubuntu, it takes more than 30 minutes or so 2022-02-13 17:22:32 do they even support downgrades like that 2022-02-13 17:22:40 Not sure about downgrades 2022-02-13 17:22:46 Most likely not 2022-02-13 17:22:47 mm 2022-02-13 18:47:48 upgrade of packages should be less than 1 min i guess 2022-02-13 18:48:02 its reviewing config changes that takes most of the time 2022-02-13 18:49:57 Yup 2022-02-13 18:50:12 Zap zap zap 2022-02-13 18:56:08 :D 2022-02-13 18:56:25 and then zap and think, shit thats not what i wanted. 2022-02-13 19:16:16 psykose: oh, this is the board we use to try to plan work: https://gitlab.alpinelinux.org/groups/alpine/infra/-/boards/1 2022-02-13 19:16:57 ah, yes, mips still needs to be gone 2022-02-13 19:17:00 nod 2022-02-13 19:17:52 that is a large amount of open issues 2022-02-13 19:17:53 neat 2022-02-13 19:17:59 yup 2022-02-13 19:18:41 hah, i see some aports-turbo ones 2022-02-13 19:18:52 i remember ptrc was trying to set up a local pkgs.a.o and could not figure out how to even run it 2022-02-13 19:18:57 sadly some of it spam from the mirror application e-mail address 2022-02-13 19:19:04 that's alright 2022-02-13 19:19:07 not that much spam 2022-02-13 19:19:13 Because I have been removing it :P 2022-02-13 19:19:18 :) 2022-02-13 19:19:19 :p 2022-02-13 19:19:44 I would like to move mirrors to MR's style requests 2022-02-13 19:19:56 psykose: speaking of which, we were looking at possibly replacing aports-turbo with the one created by postmarket 2022-02-13 19:19:56 not sure there is an issue for it 2022-02-13 19:20:03 oh yeah, pmos did have a fork of it 2022-02-13 19:20:19 It's written in python instead of lua 2022-02-13 19:20:21 mhm 2022-02-13 19:20:31 yes i have a copy here 2022-02-13 19:20:36 which i modified 2022-02-13 19:20:47 i think its kind of good to go 2022-02-13 19:20:55 with the exception of flagging 2022-02-13 19:21:15 Is that already a docker/compose project? 2022-02-13 19:21:25 iirc we had some things to rework for flagging anyway 2022-02-13 19:21:31 but i think it was mostly the autoflagging system 2022-02-13 19:21:39 since the current one does not track a lot of the packages 2022-02-13 19:21:59 the logic is not inside the app 2022-02-13 19:22:03 afaik that's mostly a matter of registring the the packages 2022-02-13 19:22:06 yes 2022-02-13 19:22:31 so i was thinking of adding that to the app 2022-02-13 19:22:40 a link to setup the mapping 2022-02-13 19:27:01 But that is something psykose could help with I suppose? 2022-02-13 19:27:21 sure 2022-02-13 19:28:14 Does it already use releases.json ? 2022-02-13 19:28:37 what's stopping the mips removal from that 2022-02-13 19:28:54 Someone actually doing the work :) 2022-02-13 19:29:00 what defines it 2022-02-13 19:29:16 I suppose ssh-ing into the master mirror, and then rm -rf :) 2022-02-13 19:29:19 sure 2022-02-13 19:29:22 you have my key :p 2022-02-13 19:29:31 :) 2022-02-13 19:29:59 i think we would like to start with things that dont need a lot of keys to lots of castles 2022-02-13 19:30:18 fine by me 2022-02-13 19:30:52 and there is enough to do without needing that (i think) 2022-02-13 19:32:50 https://gitlab.alpinelinux.org/alpine/infra/docker/apkbuild-lint-tools/-/issues/2 can be closed 2022-02-13 19:33:18 psykose: what would you like to do/help with? 2022-02-13 19:33:30 clandmeter: any objecting adding psykose to the infra group on gitlab? 2022-02-13 19:33:32 whatever you throw at me 2022-02-13 19:33:34 is there anything specific that is in your interest? 2022-02-13 19:34:30 the bigger truth is i don't 'know' how to do a lot of these things, so it's just learning by trying 2022-02-13 19:34:33 so, no specific interest 2022-02-13 19:34:34 something like coding, sysop stuff, documentation, or adminstration stuff 2022-02-13 19:34:37 just doing whatever is available :) 2022-02-13 19:35:29 ikke: go ahead 2022-02-13 19:36:20 Done 2022-02-13 19:41:52 ikke: did you recently kill any linode's? 2022-02-13 19:41:58 gitlab-test 2022-02-13 19:42:08 nod 2022-02-13 19:42:10 but also deployed it again 2022-02-13 19:42:14 nothing else i guess 2022-02-13 19:42:31 Not that I recall :) 2022-02-13 19:42:34 hm 2022-02-13 19:42:39 ok i will keep searching 2022-02-13 19:42:46 What did you loose? 2022-02-13 19:43:03 i played with apk browser 2022-02-13 19:43:09 but i dont see the code anymore 2022-02-13 19:43:16 hmm 2022-02-13 19:43:22 maybe i had it on some vm or similar 2022-02-13 19:43:33 but i dont remember anymore :) 2022-02-13 19:47:45 Only gitlab-test shows as recently deleted in the notifications 2022-02-13 19:59:58 clandmeter: to remove mips from pkg.a.o, I suppose we need to manually purge it from the DB? 2022-02-13 20:08:15 i guess so 2022-02-13 20:09:33 wasn't there a wwwdev for alpine-mksite 2022-02-13 20:09:36 or was it completely manual 2022-02-13 20:10:02 wwwtest 2022-02-13 20:10:07 right 2022-02-13 20:10:11 but that's tracking master 2022-02-13 20:10:25 There's dev as well 2022-02-13 20:11:01 but it crashed :) 2022-02-13 20:11:06 hehe 2022-02-13 20:11:25 was just wondering if there was something to e.g. see a visual diff of https://gitlab.alpinelinux.org/alpine/infra/alpine-mksite/-/merge_requests/37 2022-02-13 20:12:10 I have to specify a branch to track right now 2022-02-13 20:12:17 ah 2022-02-13 20:12:25 no automagic deployments just yet :p 2022-02-13 20:12:29 Was thinking of creating something that would automatically deploy for merge requests 2022-02-13 20:17:13 psykose: it points now to that branch: https://wwwdev.alpinelinux.org/ 2022-02-13 20:17:45 hm 2022-02-13 20:17:51 still has the old git.a links 2022-02-13 20:18:10 hmm 2022-02-13 20:20:06 ok, fixed 2022-02-13 20:23:45 ptrc: ^ the generated commit links are still wrong 2022-02-13 20:24:25 hm, i thought they were pulled from the atom feed 2022-02-13 20:25:51 it does look that way 2022-02-13 20:25:51 hm 2022-02-13 20:26:02 i don't see any references to git.a.o in generate_atom.lua either 2022-02-13 20:26:39 or generate_git-commits.lua 2022-02-13 20:27:39 maybe something is cached wrong :p 2022-02-13 20:28:35 let me build it from scratch 2022-02-13 20:29:03 ok, that looks better 2022-02-13 20:29:21 yup, looks correct 2022-02-13 20:29:48 But not sure if that's going to give issues on the live site 2022-02-13 20:30:06 probably needs to be done 'from scratch' like you just did 2022-02-13 20:30:14 ah, I think it would be updated after the next git git push to aports 2022-02-13 20:30:23 i will merge something in a minute 2022-02-13 20:30:28 if you want to just throw it in there 2022-02-13 20:30:28 :p 2022-02-13 20:31:02 If push comes to shove, we can always do a make clean && make 2022-02-13 20:31:20 just a small pile of hacks 2022-02-13 20:31:37 hacks upon hacks upon hacks 2022-02-13 20:31:50 upon more hacks 2022-02-13 20:34:03 That's how the world turns 2022-02-13 21:47:53 should probably merge these https://gitlab.alpinelinux.org/alpine/infra/build-server-status/-/merge_requests/2 https://gitlab.alpinelinux.org/alpine/infra/build-server-status/-/merge_requests/3 2022-02-13 21:47:57 although they are probably now out of date again 2022-02-13 21:47:58 hah 2022-02-13 21:48:32 and mr4 2022-02-14 10:21:26 notice: I am upgrading listserv.alpinelinux.org to 3.15, there will be a brief outage 2022-02-14 10:29:34 nevermind, it's already on 3.15 2022-02-14 10:29:36 that was easy 2022-02-14 10:41:18 I don't seem to be able to access that site atm, is it not reachable via http/https? 2022-02-14 10:50:41 it's https://lists.alpinelinux.org 2022-02-14 10:50:49 the name of the box is listserv 2022-02-14 10:58:44 ah ok makes sense, thanks for the explanation 2022-02-14 11:16:57 ddevault: you upgraded it last time :-) 2022-02-14 11:17:19 I'm upgrading the rest of sr.ht and it was in my notes to take care of it 2022-02-14 11:17:28 right 2022-02-14 18:28:51 https://twitter.com/_saagarjha/status/1492809406869737473 2022-02-14 18:28:52 huh 2022-02-14 18:29:21 hmm 2022-02-14 18:30:06 looks like it's marked "fix for android" in the css file 2022-02-14 18:30:07 https://gitlab.alpinelinux.org/alpine/infra/alpine-mksite/-/blob/master/_static/css/styles.css#L80 2022-02-14 18:30:19 Was committed by Stefan Wagner in 2016 2022-02-14 18:30:49 hm, the website doesn't seem to have any animations now 2022-02-14 18:31:19 ah, apart from the menu dropdown on mobile 2022-02-14 18:32:06 https://gitlab.alpinelinux.org/alpine/infra/alpine-mksite/-/commit/0eb492e32c7e65e16c9e50a85feae87a212b42d6 2022-02-14 18:32:45 btw, it looks like the "security" menu entry is cropped out 2022-02-14 18:32:49 both on desktop and actual android 2022-02-14 18:40:11 ur hired 2022-02-14 18:40:40 as in, "please fix this"? 2022-02-14 18:43:21 i think that bugfix block can be deleted entirely 2022-02-14 18:45:49 twitter is my favorite issues platform :) 2022-02-14 18:48:02 i removed the block, tested on android 10 (latest firefox and chrome), nothing really changed 2022-02-14 18:48:56 https://css-tricks.com/webkit-sibling-bug/ 2022-02-14 18:49:02 >This is fixed in both stable releases of the WebKit browser Safari (5.1) and Chrome (13) 2022-02-14 18:49:05 that's... old 2022-02-14 18:49:16 imo that's certainly safe to remove 2022-02-14 18:49:21 it doesn't seem like there's any animations on the page anyway 2022-02-14 18:49:58 menu dropdown 2022-02-14 18:51:08 https://gitlab.alpinelinux.org/alpine/infra/alpine-mksite/-/merge_requests/39 2022-02-14 18:51:22 ah 2022-02-14 18:59:25 well, looks fine to me 2022-02-14 18:59:41 i would merge it if i could and knew how to redeploy 2022-02-15 00:51:33 seems like a lot of stuff is down right now? (wiki, alpinelinux.org, cdn) 2022-02-15 00:53:17 yep 2022-02-15 00:53:19 everything is 2022-02-15 00:54:11 ah ok. hope it's not too bad to resolve... 2022-02-15 00:54:47 me too 2022-02-15 00:54:54 sadly i can't log in to anything yet 2022-02-15 00:55:09 maybe over the coming months i can improve it a bit 2022-02-15 00:55:32 this is not the first outage in my memory 2022-02-15 00:55:41 and of course it happens at 2AM cet 2022-02-15 00:55:41 :p 2022-02-15 00:56:00 i just have a polyphasic sleep schedule so i just happen to be awake 2022-02-15 00:56:10 yeah the timing sucks when most(?) folks maintaining it are in europe 2022-02-15 00:56:14 yup 2022-02-15 00:56:36 or rather a sleep schedule of >24h 2022-02-15 00:56:39 whatever the term is 2022-02-15 00:56:53 too many words.. 2022-02-15 00:57:50 it's only 17:00 where I am, so in the future if you need some help at the wrong time of the day/night for y'all, I could probably volunteer myself 2022-02-15 00:58:25 western US? 2022-02-15 00:58:29 yep 2022-02-15 00:58:31 mm 2022-02-15 00:58:42 there are a few infra members but most are inactive 2022-02-15 00:58:55 it's just clandmeter/ikke and me(just joined) that are hopefully doing anything 2022-02-15 00:59:00 and we are all in europe 2022-02-15 00:59:14 ah I see 2022-02-15 00:59:15 i guess it would actually be nice to add someone hopefully active in another TZ 2022-02-15 00:59:24 and it will take a while for me to get to learn anything 2022-02-15 01:00:12 did not realize so many core alpine people were in the EU 2022-02-15 01:00:23 yeah of course. just thought I'd offer, since I suspected that might be the case (all active maintainers are too close together in timezones :P) 2022-02-15 01:01:23 almost every alpine developer is EU 2022-02-15 01:01:36 craftyguy: yeah, imo it's an actual issue 2022-02-15 01:01:52 not that i have much standing to say that on 2022-02-15 01:02:07 but it's nice to have at least a few people outside the main TZ 2022-02-15 01:04:15 heh, understood 2022-02-15 01:23:44 i'm in AU and am getting more and more time to work on things 2022-02-15 01:24:31 seems like we could get pretty good global coverage 2022-02-15 01:33:59 Hi someone pinged me on twitter claiming alpinelinux.org is down? I am at a place where I don’t know if that is true or the internet connection is bad 2022-02-15 01:35:41 it is 2022-02-15 01:35:54 everything is 2022-02-15 01:35:59 and ikke is aleep 2022-02-15 01:36:01 asleep 2022-02-15 01:36:02 :) 2022-02-15 01:36:19 Anybody knows what is wrong? 2022-02-15 01:36:26 i have no idea 2022-02-15 01:36:29 even zabbix is down 2022-02-15 01:36:57 DNS works? 2022-02-15 01:37:50 seems so, at least dns give an ip 2022-02-15 01:38:18 the ping doesn't work, aside from dl-cdn 2022-02-15 01:38:31 i don't have access to anything or i would check more 2022-02-15 01:38:41 i guess we may as well go to sleep for now 2022-02-15 01:38:47 mirrors are still up 2022-02-15 01:42:16 the cdn mirror is down though 2022-02-15 01:42:38 dl-cdn.alpinelinux.org 2022-02-15 01:42:47 it is 2022-02-15 01:42:50 i just mean you can ping it 2022-02-15 01:43:00 ah, right 2022-02-15 01:43:04 unlike anything else 2022-02-15 01:43:50 ikke will have quite the morning when he wakes up :) 2022-02-15 01:47:47 let me see what I can do here and now 2022-02-15 01:53:34 https://mirrors.alpinelinux.org is up 2022-02-15 01:54:46 funnily, yes 2022-02-15 01:54:57 but the main site, and dl-cdn, are not 2022-02-15 01:55:09 gitlab is up at least 2022-02-15 01:55:21 https://build.alpinelinux.org/ is up too, there's just nothing listed 2022-02-15 01:55:36 so basically down 2022-02-15 01:55:55 yeah the service is "down" but the http server at the other end is "up" 2022-02-15 01:56:15 gitlab.alpinelinux.org is up 2022-02-15 01:56:31 I logged in to the Linode account, and are trying to figure out what is wrong 2022-02-15 01:56:37 apparently everything on Linode is fine 2022-02-15 01:56:43 I don't know what goes where... 2022-02-15 01:56:52 not sure if dl-cdn is linode 2022-02-15 01:57:03 i did not check what is hosted where yet 2022-02-15 01:57:15 dl-cdn is fastly, and seems up to me 2022-02-15 01:57:18 iirc some things were do, some metal 2022-02-15 01:57:23 but i've been hitting it a bunch today 2022-02-15 01:57:45 I suspect the Equinix Metal machine(s) are down 2022-02-15 01:58:14 I suspect Equinix Metal machines are the backend for dl-cdn 2022-02-15 01:58:20 most likely yes 2022-02-15 01:58:30 probably all on the same metal and it went down 2022-02-15 01:58:41 gitlab is a linode instance so makes sense it's up 2022-02-15 01:58:50 and build. depends on msg. wich is EQX too 2022-02-15 01:59:26 yup 2022-02-15 01:59:44 but not everything hosted on Equinix is down 2022-02-15 01:59:52 I can access my dev lxc container 2022-02-15 01:59:58 as is www. 2022-02-15 02:00:03 wierd 2022-02-15 02:00:16 yep, nld5-dev1 is up 2022-02-15 02:00:30 mine as well 2022-02-15 02:00:54 i guess it's really just whatever had the core services on it 2022-02-15 02:02:45 .119 is what i can see down, which is nld3-dev1? 2022-02-15 02:02:59 which has www, msg and mail? 2022-02-15 02:03:29 nld3-dev1.alpinelinux.org 2022-02-15 02:03:55 yeah 2022-02-15 02:03:55 I got an email from Equinix about nld3-dev1 2022-02-15 02:04:05 what did it say 2022-02-15 02:08:01 "Our network team had detected unusual bursts of network traffic..." 2022-02-15 02:08:11 mm 2022-02-15 02:08:17 I suspect we are attacked 2022-02-15 02:08:29 is it generic or specific to our instance 2022-02-15 02:08:35 I don't know 2022-02-15 02:08:52 so far it looks like it is targeted a specific instance 2022-02-15 02:09:40 at the best time too 2022-02-15 02:11:17 i bet some kid is just ddosing www 2022-02-15 02:15:07 Hopefully people here already know that most of the *.alpinelinux.org sites appear to be down. If not, I'm letting you know :) 2022-02-15 02:16:09 we do be knowing 2022-02-15 02:16:21 psykose, Thanks! 2022-02-15 02:29:26 hm 2022-02-15 02:29:27 seems like it is back? 2022-02-15 02:29:29 yes 2022-02-15 02:30:59 \o/ 2022-02-15 02:31:13 i'm still mildly wary 2022-02-15 02:31:21 apparently there was some DDOS 2022-02-15 02:31:25 ci is back up, but 2022-02-15 02:31:28 yeah, its not over really 2022-02-15 02:31:29 figured 2022-02-15 02:39:32 wtf 2022-02-15 02:39:41 why would somebody ddos alpine 2022-02-15 02:39:51 i /hope/ it was just a ddos 2022-02-15 02:40:31 probably some disgruntled person who hasn’t heard about debian-slim 2022-02-15 02:40:45 I have been in touch with Equinix Metal support 2022-02-15 02:40:56 working on it. 2022-02-15 02:41:22 ACTION read that as Equinix Mental support 2022-02-15 02:41:26 we'll know in a bit 2022-02-15 02:41:27 hah 2022-02-15 02:41:32 i wish i had some of that 2022-02-15 02:41:35 six of one half dozen of the other 2022-02-15 02:42:12 hurra 2022-02-15 02:52:53 ok, Equinix confirms that it has stopped 2022-02-15 02:52:57 for now 2022-02-15 02:53:02 ddos for sure? 2022-02-15 03:01:32 seems so yes 2022-02-15 03:29:44 wild 2022-02-15 03:29:55 o/ Ariadne 2022-02-15 03:35:27 ddos targeted at alpine specifically or the cdn in general? 2022-02-15 04:00:59 alpine 2022-02-15 04:01:22 somebody probably mad and doesn’t know about docker-slim 2022-02-15 04:01:29 er debian-slim 2022-02-15 04:29:02 so it was a ddos? 2022-02-15 04:45:21 Ouch 2022-02-15 07:40:52 morning 2022-02-15 07:41:05 so we had a hickup 2022-02-15 07:47:26 Ahuh 2022-02-15 07:55:44 what's the thing about debian-slim? 2022-02-15 08:01:17 Just a joke 2022-02-15 08:01:56 ah ok :) 2022-02-15 16:22:30 anoying ^ 2022-02-15 16:24:42 yes 2022-02-15 16:25:27 the ci for it does not really be working 2022-02-15 18:06:07 Ariadne: https://twitter.com/n4of7/status/1493646681396629510 2022-02-15 18:06:32 alpine's infrastructure needs are a bit more highend than home PCs 2022-02-15 18:06:33 oh, you already replied lol 2022-02-15 18:07:05 my reply was mostly for meme purposes 2022-02-15 18:07:47 i mean, it is kind of BS that we even have to ask, alpine is heavily used on all these clouds, usually ranking #1 or #2 as top base image 2022-02-15 18:08:04 it's just like that thing: 2022-02-15 18:08:06 these cloud providers should step up and give us the infrastructure needed to service their customers 2022-02-15 18:08:07 "pay the writers!" 2022-02-15 18:08:18 we should just take it all down for 24h 2022-02-15 18:08:20 linode does :D 2022-02-15 18:08:40 these clouds are making *billions of dollars* on alpine 2022-02-15 18:08:42 and retweet that tweet :) 2022-02-15 18:08:45 they can give us a couple boxes 2022-02-15 18:09:24 actually it's more like the xkcd comic 2022-02-15 18:09:37 the one that looks like jenga 2022-02-15 18:13:18 Ariadne: i'm curious though, what is all that bandwidth for? 2022-02-15 18:13:23 it is literally package downloads 2022-02-15 18:13:33 its our rsync mirror 2022-02-15 18:13:34 yes 2022-02-15 18:13:37 others sync it 2022-02-15 18:13:45 well, the main concern right now is rsync 2022-02-15 18:13:56 but in aggregate, its all to support package downloads 2022-02-15 18:14:00 i wonder if maybe you could sync to a few 2022-02-15 18:14:06 and have everyone else sync from the few 2022-02-15 18:14:14 we do that already :P 2022-02-15 18:14:19 oh wow 2022-02-15 18:14:21 we need more tier1 mirrors basically 2022-02-15 18:14:25 then alpine is really that popular 2022-02-15 18:14:34 we had more than 1 mirror 2022-02-15 18:14:39 but the problem is storage 2022-02-15 18:15:10 mind boggling just how much infra projects need these days 2022-02-15 18:15:40 more than 1 t1 mirrors 2022-02-15 18:15:42 anyway, these cloud providers should give us infrastructure to put mirrors directly in their networks 2022-02-15 18:15:49 but our repo is over 1TB now 2022-02-15 18:16:03 i wonder if p2p rsync-like thing would be possible 2022-02-15 18:16:09 using bittorrent 2022-02-15 18:16:29 the problem with that is that mirrors down stream only sync from rsync. 2022-02-15 18:16:30 of the cloud providers 2022-02-15 18:16:35 linode is the only one that does 2022-02-15 18:16:48 which is why i bought a dedicated server from them today :) 2022-02-15 18:18:20 but i wonder 2022-02-15 18:18:37 what if package managers existed, that downloaded packages over bittorrent 2022-02-15 18:18:46 does that exist already? 2022-02-15 18:18:56 they would be very slow 2022-02-15 18:19:07 torrents take a lot of time to find the other peers 2022-02-15 18:19:12 oh, yeah 2022-02-15 18:19:17 and then it has to do that per package 2022-02-15 18:20:17 still though. it's an interesting challenge that maybe a distro could overcome one day 2022-02-15 18:20:34 p2p package downloads. then you wouldn't really need to have lots of rsync mirrors for projects 2022-02-15 18:21:54 if you could solve that bottleneck (of the time taken to find other peers) 2022-02-15 18:22:22 ok well this theoretical infrastructure does not exist today, meanwhile our problems do exist today, and can be solved by AWS giving us servers 2022-02-15 18:22:53 jeff bezos launched into space, saying it was on the back of amazon and its suppliers, hi yes, we're a supplier 2022-02-15 18:22:56 :) 2022-02-15 18:23:41 i will give this challenge some thought 2022-02-15 18:23:47 p2p package downloads. until then, cherrio 2022-02-15 18:23:50 cheerio* 2022-02-15 19:02:40 is there a similar post on mastodon that I can share? 2022-02-15 19:09:17 on the topic of mirrors, is the mirrors yaml file in some repo somewhere? mirror-status refers to alpine-mirrors, but that's been gone from aports for over a year now 2022-02-15 19:13:39 also, nld5-dev1 doesn't sound like germany to me.. 2022-02-15 19:15:02 Ariadne: how big is a single arch mirror rn? lets say.. aarch64? 2022-02-15 19:15:31 the issue is not mirrors 2022-02-15 19:15:35 the issue is tier1 mirrors 2022-02-15 19:15:44 which need to be Nx10G 2022-02-15 19:15:53 ah 2022-02-15 19:16:10 we are down to only one atm 2022-02-15 19:16:18 ...yay 2022-02-15 19:16:21 which is the weak spot that failed last night 2022-02-15 19:16:47 the increased traffic tripped ddos alarms at equinix 2022-02-15 19:16:55 mhm 2022-02-15 19:17:26 there are plenty of non-tier1 mirrors, that is fine 2022-02-15 19:17:45 the tier1 mirrors are not public 2022-02-15 19:17:50 they feed the real mirrors 2022-02-15 19:18:08 yeah 2022-02-15 20:22:41 🙄 2022-02-15 20:25:48 it's still fucked in ci 2022-02-15 20:26:17 Not a lot I can do about it 2022-02-15 20:26:43 yep 2022-02-15 20:26:53 just one of those ppc days 2022-02-15 21:34:10 https://www.akamai.com/newsroom/press-release/akamai-to-acquire-linode 2022-02-15 21:34:12 wow 2022-02-15 21:36:17 that is quite the acquisition 2022-02-15 21:36:50 yes 2022-02-15 21:37:58 blaboon: I hope I can say congratulations 2022-02-15 21:38:40 post from linode https://www.linode.com/blog/linode/linode-and-akamai/ 2022-02-15 21:53:26 interesting 2022-02-15 22:01:03 Wow…. I guess that is good news? Akamai is good on storage as I understand? 2022-02-15 22:01:55 They should have plenty 2022-02-15 22:11:29 yea, i only just found out myself less than an hour ago. sounds like it's gonna be business as usual for us though, so i guess we'll see what happens :) 2022-02-15 22:12:09 should be interesting! 2022-02-15 22:16:30 blaboon: congrats! 2022-02-16 11:51:02 upgraded Zabbix to 6.0 😎 2022-02-17 20:15:20 "We received your email, but were unable to deliver it because it contains HTML." 2022-02-17 20:15:23 oh well... 2022-02-17 20:15:43 non-plaintext email? :p 2022-02-17 20:16:04 The fun part is, if it contains a plaintext version as well, it was still delivered 2022-02-17 20:16:21 time to throw in the customary 2022-02-17 20:16:31 alternative text/plain with "get an html reader" 2022-02-17 20:16:34 as everyone likes to do 2022-02-17 20:16:35 :p 2022-02-17 20:16:47 I think im gonna throw out the laptop 2022-02-17 20:16:51 psykose: this has been a controversial point before 2022-02-17 20:17:08 in which space 2022-02-17 20:17:11 here 2022-02-17 20:17:12 ah 2022-02-17 20:17:19 generally it's better to not use html 2022-02-17 20:17:22 personally i read anything 2022-02-17 20:17:29 and re-send when I get back from vacation where I have access to proper email client 2022-02-17 20:18:06 I get the argument of plain text and yes, that is the "proper" way to do it 2022-02-17 20:18:21 unfortunately the rest of the world disagrees, so its an uphill battle 2022-02-17 20:18:26 yes 2022-02-17 20:18:31 what client are you using? 2022-02-17 20:18:35 and a battle that is impossible to win 2022-02-17 20:18:40 Mac OS Mail 2022-02-17 20:18:54 i like rich text atleast, because emphasis is sometimes required 2022-02-17 20:18:54 :P 2022-02-17 20:19:29 I found a "Format" -> make plain text" menu entry 2022-02-17 20:19:34 that should work 2022-02-17 20:19:35 lets see if that works 2022-02-17 20:19:45 if not this laptop is going out the window 2022-02-17 20:21:10 did anyone get an email via the alpine-devel list from a ncopa at alpinelinux dot org? 2022-02-17 20:21:20 yes, twice 2022-02-17 20:21:36 great :) 2022-02-17 20:48:28 ;-) 2022-02-17 20:50:59 "Dotnet build requires at minimum 50GB of space to build." oof 2022-02-17 20:51:12 yep 2022-02-17 20:51:28 The arm* based CI VMs have 50G disks :P 2022-02-17 20:51:30 i guess ayakael was #1 on the disk usage? :p 2022-02-17 20:51:31 hah 2022-02-17 20:51:34 tell them that 2022-02-17 20:52:07 psykose: No, that's mostly about artifacts 2022-02-17 20:52:11 ah 2022-02-17 20:53:02 some people probably overestimated the value of the expression "space is cheap" 2022-02-17 20:53:29 Yes, certainly 2022-02-17 21:00:56 ncopa: that's slightly better than "We received your email, but were unable to deliver it because it doesn't contains HTML. Everyone uses HTML, everyone needs to see images/logos and animated GIFs in emails. Get with the programme!" ;-) 2022-02-17 21:02:48 psykose: We have one host for armv7, armhf and aarch64, both for builders and CI 2022-02-17 21:03:01 yep, that i know 2022-02-17 21:03:14 ok 2022-02-17 21:03:20 ~600G reserved for CI alone is possible if it's like.. 2T disk 2022-02-17 21:03:22 otherwise not really 2022-02-17 21:03:29 well 2022-02-17 21:03:31 they also hold packages 2022-02-17 21:03:36 so 4T, to be safe 2022-02-17 21:03:48 if they don't, then bad luck 2022-02-17 21:03:49 :) 2022-02-17 21:03:58 We have 1.75 in total :P 2022-02-17 21:04:00 yep 2022-02-17 21:04:06 how much is the package store 2022-02-17 21:04:37 CI itself has no persistent package store 2022-02-17 21:04:50 The builders each have a repository 2022-02-17 21:05:08 yeah, i just mean it's on the same disk 2022-02-17 21:05:24 (or i assumed so, at least, since the builders sync to the mirrors) 2022-02-17 21:05:34 https://tpaste.us/Bgr4 2022-02-17 21:05:50 hm 2022-02-17 21:05:51 the builders are on lv_root 2022-02-17 21:05:58 that is missing quite a lot from the 1.75 2022-02-17 21:06:17 https://tpaste.us/NMly 2022-02-17 21:06:27 I have a habbit of not assigning all space 2022-02-17 21:06:38 i guess it's technically possible to add ~150 to each of those and keep a 150 spare 2022-02-17 21:06:48 then it's.. 200 for 2 runners 2022-02-17 21:06:50 I'd start with 50G 2022-02-17 21:06:59 sure, as long as nobody starts two dotnets at once 2022-02-17 21:06:59 yeah 2022-02-17 21:07:03 go for it 2022-02-17 21:07:11 Then they just have to retry :P 2022-02-17 21:10:25 psykose: would it possible for you to setup wireguard? Then I could give you access to netbox 2022-02-17 21:10:48 you can throw a wg.conf thing at me if you wish, i'll figure it out 2022-02-17 21:11:04 it's just like 2/3 k-v things iirc 2022-02-17 21:11:08 nod 2022-02-17 21:11:16 i will have to do some tinkering to either netns stuff to it, or uhh 2022-02-17 21:11:20 i guess i do have a spare laptop now 2022-02-17 21:11:22 i can just full wg from it 2022-02-17 21:11:35 but yeah, i know the process, probably 2022-02-17 21:13:10 psykose: you'd have to generate a key-pair though 2022-02-17 21:13:24 now this part i definitely don't remember 2022-02-17 21:13:42 https://wiki.archlinux.org/title/WireGuard#Key_generation 2022-02-17 21:16:08 i guess i send you the pubkey 2022-02-17 21:16:17 https://tpaste.us/xn4l 2022-02-17 21:16:19 yes 2022-02-17 21:16:26 M9dQtlg29wSTHyGU1+Vi0urcwjU39kiBehnIM+IJJWo= 2022-02-17 21:17:03 ok, set it up on the wg host 2022-02-17 21:17:24 now it should just work with private in there 2022-02-17 21:17:43 yes 2022-02-17 21:26:15 i don't think i set this up correctly 2022-02-17 21:26:16 hah 2022-02-17 21:26:44 assuming wg-quick is supposed to work 2022-02-17 21:26:56 https://img.ayaya.dev/zhJwA60iCYV0.png 2022-02-17 21:27:07 https://gitlab.alpinelinux.org/alpine/infra/infra/-/wikis/Alpine-wireguard-VPN 2022-02-17 21:27:16 ah 2022-02-17 21:27:20 theoretically it should be possible to save some space on dotnet build if you don't want to rebuild 2022-02-17 21:27:44 well i did literally that essentially 2022-02-17 21:27:53 it's the wg-quick 2-step thing 2022-02-17 21:28:31 oh the address is empty 2022-02-17 21:28:51 you didn't give me one of those, unless i am supposed to guess :) 2022-02-17 21:29:23 heh 2022-02-17 21:29:39 If you'd had access to netbox, you would not have to guess :P 2022-02-17 21:29:50 172.16.252.8/24 2022-02-17 21:30:07 yay netbox 2022-02-17 21:30:31 We (try to) keep our infra documented in Netbox :) 2022-02-17 21:30:44 same 2022-02-17 21:31:40 where this netbox thing at 2022-02-17 21:32:01 I don't see you connected yet? 2022-02-17 21:32:05 hm 2022-02-17 21:32:17 `wg show` 2022-02-17 21:32:53 heh 2022-02-17 21:33:16 https://img.ayaya.dev/TeCxosES2QUB 2022-02-17 21:35:40 Can you try the ipv4 address, just to rule that out? 2022-02-17 21:36:49 assuming i can just replace the wg.a.o with it 2022-02-17 21:36:50 done 2022-02-17 21:37:34 still nothing :( 2022-02-17 21:37:45 For me it worked instantly 2022-02-17 21:37:52 maybe my network is magic 2022-02-17 21:37:58 or i missed something really obvious 2022-02-17 21:41:18 you need a working internet connection 2022-02-17 21:41:28 anyway, if you would get it running, you could add 172.17.8.3 to resolve *.alpin.pw 2022-02-17 21:41:40 then you would be able to reach netbox at netbox.alpin.pw 2022-02-17 21:44:32 clandmeter: hah! 2022-02-17 21:44:33 ikke: are you sure about that ip? :) 2022-02-17 21:44:42 no :P 2022-02-17 21:44:46 172.16.8.3 :) 2022-02-17 21:44:51 ;-) 2022-02-17 21:44:57 I'm so used to type 172.17.x.x here 2022-02-17 21:45:03 hi clandmeter, hows it goin? 2022-02-17 21:45:06 dont make it any harder 2022-02-17 21:45:27 hi, im fine, just tired 2022-02-17 21:45:33 what about yourself 2022-02-17 21:46:01 good, much better! 2022-02-17 21:46:59 traceroute to netbox.alpin.pw after adding a dnsmasq entry on my resolver looks correct 2022-02-17 21:47:19 connecting to it gives a refused though 2022-02-17 21:47:29 and wg show shows handshakes and data transferred now 2022-02-17 21:47:33 ah, we have contact 2022-02-17 21:48:10 psykose: what IP does it resolv to? 2022-02-17 21:48:21 netbox.alpin.pw? 2022-02-17 21:48:23 yes 2022-02-17 21:48:31 172.16.8.3 2022-02-17 21:49:03 172.16.8.3 is the dns server to resolve it, sorry if that was not clear 2022-02-17 21:49:15 otherwise you could just add 172.16.14.1 for netbox.alpin.pw 2022-02-17 21:49:21 :) 2022-02-17 21:49:31 tfw don't have wireguard set up 2022-02-17 21:49:55 Ariadne: I saw that we reserved an IP for you, but it was not added to the config 2022-02-17 21:49:59 ah 2022-02-17 21:50:04 lmao 2022-02-17 21:50:08 that makes more sense 2022-02-17 21:50:40 and suddenly things start to work :) 2022-02-17 21:53:18 for some reason adding 172.16.8.3 to resolv.conf does not resolve anything 2022-02-17 21:54:04 ikke: yeah i never got around to setting it up because my server died :D 2022-02-17 21:54:09 I have server=/alpin.pw/172.16.8.3 in my dnsmasq config 2022-02-17 21:54:15 Ariadne: ouch 2022-02-17 21:54:23 ah, sure, let me try that 2022-02-17 21:54:29 psykose: can you ping 172.16.8.3? 2022-02-17 21:54:38 yes 2022-02-17 21:54:52 ok, so that should be just a dns config 2022-02-17 21:54:57 but i was doing it wrong :) 2022-02-17 21:55:11 welcome to my life 2022-02-17 21:55:22 no mine! 2022-02-17 21:55:24 aaand it connects 2022-02-17 21:55:41 and i can log in 2022-02-17 21:55:42 nice 2022-02-17 21:55:42 \o/ 2022-02-17 21:55:53 this looks profoundly less scuffed than the zabbix interface, let me tell you 2022-02-17 21:56:00 heh 2022-02-17 21:56:01 :) 2022-02-17 21:56:13 netbox is great, tbh 2022-02-17 21:56:22 zabbix interface is.... 2022-02-17 21:56:24 no idea why the wg link took like 5 minutes to work in the first place 2022-02-17 21:56:28 but we should be fine now 2022-02-17 21:56:38 I've gotten used to the Zabbix interface 2022-02-17 21:56:47 I use it daily though 2022-02-17 21:56:56 ikke mentioned he upgraded zabbix, so i checked it out to find out not much has changed in the interface leve, 2022-02-17 21:57:06 nothing changed 2022-02-17 21:57:11 Yeah, not big interface changes 2022-02-17 21:57:13 no* 2022-02-17 21:57:20 There is a new ' 2022-02-17 21:57:23 "services" menu 2022-02-17 21:57:47 SLA in the services menu 2022-02-17 21:57:54 yes 2022-02-17 21:58:01 best to not pay attention to that one 2022-02-17 21:58:02 x.x 2022-02-17 21:58:05 ftr, I use https://zabbix.alpinelinux.org/zabbix.php?action=dashboard.view&dashboardid=1 as an overview page 2022-02-17 21:58:22 that looks like the default 2022-02-17 21:58:30 yes, it is 2022-02-17 21:58:31 omitting &id=1 gives you the same 2022-02-17 21:58:32 aye 2022-02-17 21:58:59 It remembers the last one you have open 2022-02-17 21:59:10 so if you'd open a different one, and remove the id, it would still load the same 2022-02-17 21:59:18 ah, and i see now this is only a 172.16.x wireguard so i don't need any netns hacks 2022-02-17 21:59:19 very good 2022-02-17 21:59:44 Yes, we limited it only to the internal networks 2022-02-17 21:59:45 so now that you are in i can tell you our secret 2022-02-17 21:59:53 oh no 2022-02-17 22:00:08 we document everything in netbox 2022-02-17 22:00:12 uhoh 2022-02-17 22:00:14 except the things we forget 2022-02-17 22:00:23 :p 2022-02-17 22:00:42 it's documented except when it's not 2022-02-17 22:00:42 :p 2022-02-17 22:00:48 exactly 2022-02-17 22:00:53 Please _do not_ look at this page: https://netbox.alpin.pw/dcim/devices/ :P 2022-02-17 22:01:03 its 100% perfect with a few corner cases 2022-02-17 22:01:23 :eyes: 2022-02-17 22:01:37 heh 2022-02-17 22:01:42 this login is new to me 2022-02-17 22:02:11 wow, it has eye candy now 2022-02-17 22:02:24 yeah, 3.0 got a nice new interface 2022-02-17 22:02:50 yeah and funky new left menu animations 2022-02-17 22:03:06 i ❤️ it 2022-02-17 22:03:30 and no more secrets! 2022-02-17 22:03:32 yaaaaaaaaaay 2022-02-17 22:04:39 bed time for me, enjoy netbox :) 2022-02-17 22:05:01 everything on one page! 2022-02-17 22:05:01 so nice 2022-02-17 22:05:05 clandmeter: sleep well! 2022-02-17 22:05:07 dream of cute things 2022-02-17 22:05:37 22:29 ..........@ikke| 172.16.252.8/24 2022-02-17 22:06:03 ikke: netmask should be /32 for endpoints 2022-02-17 22:06:17 mps: yeah, I noticed 2022-02-17 22:06:27 Incorrectly documented in netbox :P 2022-02-17 22:06:41 so psykose should set it in wg0.conf on local machine 2022-02-17 22:06:47 updated 2022-02-17 22:06:48 thanks 2022-02-17 22:07:21 psykose: np 2022-02-17 22:08:01 ikke: you added psykose to netbox? 2022-02-17 22:08:04 yes 2022-02-17 22:08:29 nice, I don't have to look there, thanks 2022-02-17 22:08:42 don't have to look where 2022-02-17 22:08:52 https://netbox.alpin.pw/ipam/prefixes/44/ip-addresses/ 2022-02-17 22:08:59 psykose: to set your data in netbox 2022-02-17 22:09:14 ikke already did 2022-02-17 22:11:00 I had business and technical meetings three last days (and today) so I'm practically exhausted of all these 2022-02-17 22:11:35 ah 2022-02-17 22:11:36 yes 2022-02-17 22:11:38 meetings are not fun 2022-02-17 22:12:23 aha 2022-02-17 22:22:33 that changelog sure be a lot of 'kdaudt deleted xyz' 2022-02-17 22:22:35 :^) 2022-02-17 22:22:46 removing old stuff :P 2022-02-17 22:23:27 funny how usa9 is marked as aarch64 only 2022-02-17 22:23:41 psykose: the host and vms are all aarch64 :) 2022-02-17 22:23:47 yep 2022-02-17 22:24:28 I made a script that outputs all lxc container info in netbox import csv 2022-02-17 22:54:00 ikke: i noticed the in-progress builds don't have their log published until they are done (or at least you can't click them on build.a.o), is there a reason for that 2022-02-17 23:41:10 yes 2022-02-17 23:55:53 yes? 2022-02-18 00:17:59 the log is not uploaded from the builder until the build completes 2022-02-18 00:34:06 i already knew that much 2022-02-18 00:34:15 i'm wondering why it's not streamed like on the ci runs 2022-02-18 00:35:04 to which there is an obvious answer of 'the ci runs are gitlab magic, and nobody set something up like that on the builders', i guess 2022-02-18 00:35:07 which would make sense :) 2022-02-18 00:35:11 just wanted to make sure 2022-02-18 00:35:39 (and then perhaps think of a way to do it) 2022-02-18 00:37:28 pretty sure build logs are copied using scp executed from lua 2022-02-18 00:38:32 at least that's what it says here: https://gitlab.alpinelinux.org/alpine/aports/-/blob/master/main/aports-build/report-build-errors.lua#L35 2022-02-18 00:38:53 makes sense 2022-02-18 05:06:22 psykose: we just don't have anything that can do that that fits in the current build infra 2022-02-18 05:06:28 right 2022-02-18 05:06:34 good morning 2022-02-18 05:06:46 you did not sleep enough :p 2022-02-18 05:07:30 What about you? 2022-02-18 05:07:39 have not gone to bed 2022-02-18 05:33:12 buildrepo has a plugin system, so perhaps we can leverage that 2022-02-18 08:38:01 ikke: the only way to find a latest release (iso or whatever) is to check yaml or json? 2022-02-18 08:53:52 I think so 2022-02-18 09:07:13 i wonder if symlinks are possible. 2022-02-18 09:07:31 not sure its would work as expected with cdn 2022-02-18 09:17:07 We'd need to purge them every time we update the symlinks 2022-02-18 18:50:15 how much RAM does rspamd use on the alpine mail server? 2022-02-18 18:51:53 ddevault: which specific number are you interested in? 2022-02-18 18:52:17 turns out I have a login on that box so I found it myself 2022-02-18 18:52:19 but thanks! 2022-02-18 18:52:24 ok 2022-02-18 22:29:43 is the riscv64 builder down? 2022-02-18 22:30:46 no, just hanging 2022-02-18 22:31:01 nodjs 2022-02-18 22:33:26 Ah ok a 2022-02-18 22:44:15 darn, is it *still* building nodejs? it's been a few days, i think 2022-02-18 22:49:10 It's probably going to take a lot longer unless someone fixes the issue 2022-02-18 22:49:18 or disables nodejs 2022-02-18 22:51:36 any reason there's no ipv6 access for pkgs.alpinelinux.org? 2022-02-18 22:53:47 needing to request an address range for that server 2022-02-18 22:55:24 right, because a bunch of other services are reachable :) 2022-02-19 20:29:24 martijnbraam: pkgs.a.o should now work over IPv6 2022-02-19 20:29:34 \o/ 2022-02-19 20:30:02 it does work :D 2022-02-19 20:51:45 webscale 2022-02-19 20:52:49 but only mongodb is webscale! 2022-02-19 20:53:13 You forgot devnulldb 2022-02-20 20:08:51 clandmeter: https://gitlab.alpinelinux.org/alpine/infra/infra/-/issues/10750 opinion on adding 50G to the arm* CI vms? 2022-02-20 20:10:59 do we have that kind of space? 2022-02-20 20:11:08 atm we do 2022-02-20 20:11:29 VG #PV #LV #SN Attr VSize VFree 2022-02-20 20:11:31 vg0 2 5 0 wz--n- <1.75t 644.25g 2022-02-20 20:12:08 then go ahead 2022-02-20 20:12:21 but wtf 2022-02-20 20:12:39 next project will need 1TB of workspace 2022-02-20 20:13:34 at some point we can just say no :) 2022-02-20 20:14:00 sigh, so we need to define another policy.. :) 2022-02-20 20:55:09 that's me 2022-02-20 23:33:55 o7 2022-02-21 05:29:21 rails: is that battleships? 2022-02-21 05:29:38 because if so, then you missed :P 2022-02-21 05:29:43 where 2022-02-21 05:30:04 23:33 o7 2022-02-21 05:30:08 heh 2022-02-21 05:30:19 c3 2022-02-21 05:30:20 :) 2022-02-21 05:53:41 curses! 2022-02-21 11:26:17 clandmeter: I've upgraded all the arm CI vms to 3.15 yesterday 2022-02-21 11:37:17 victory 2022-02-21 11:37:39 We still have to upgrade ppc64le and s390x 2022-02-21 11:37:44 they are quite behind 2022-02-21 12:24:04 i noticed the host is also behind? 2022-02-21 12:24:08 arm host 2022-02-21 16:49:56 Hey all! 2022-02-21 16:49:56 I mentioned this Azure credits program (https://cloudblogs.microsoft.com/opensource/2021/09/28/announcing-azure-credits-for-open-source-projects/) to Ariadne on Twitter yesterday, and I wanted to followup to see who the best person to coordinate with is? 2022-02-21 16:50:52 I work at Microsoft, but not on this. I just know that dealing with bureaucracy can sometimes be challenging, and I'd like to help where I can 2022-02-21 16:52:13 hi JamieMagee 2022-02-21 16:55:07 JamieMagee: you can talk to me or to ikke 2022-02-21 17:15:41 JamieMagee: ah, i was going to write a mail syncing you all up :) 2022-02-22 09:55:38 do we have any nntp client except neomutt 2022-02-22 09:58:39 does anyone even use nntp these days? 2022-02-22 09:59:40 there are a lot mailing lists archives accesible with nntp 2022-02-22 10:46:19 ikke: did you restarted arm host 2022-02-22 10:46:44 or containers 2022-02-22 10:54:07 Not recently 2022-02-22 11:10:52 hmm, I noticed that my tmux sessions are killed 2022-02-22 12:05:37 oops 2022-02-22 12:06:06 mps: could be that i killed mine but did it not take care about other sessions. sorry about that/ 2022-02-22 12:06:39 thought i was in container, was probably exec it on the host :| 2022-02-22 12:18:39 clandmeter: :) 2022-02-22 12:19:27 clandmeter: np, nothing important was in sessions 2022-02-22 19:19:44 ikke: https://tpaste.us/xngl 2022-02-22 19:20:47 I assume thats dl-master? 2022-02-22 19:20:55 nod 2022-02-22 19:24:15 so our mirror is already 1.5T 2022-02-22 19:24:45 Yes, jirutka requested it to be increased some time ago 2022-02-22 19:25:00 i mean disk usage 2022-02-22 19:25:29 Ah 2022-02-22 19:49:17 Hmm 2022-02-22 19:58:59 host is still up, can reach it oob 2022-02-22 20:01:58 but not via ip? 2022-02-22 20:02:05 correct 2022-02-22 20:02:20 router i guess 2022-02-22 20:02:23 And it cannot reach the internet either 2022-02-22 20:02:24 or switch 2022-02-22 20:02:27 I see eth0 is down 2022-02-22 20:02:37 eth1 is up, but does not seem to matter 2022-02-22 20:02:42 Nothing here yet: https://status.equinixmetal.com/ 2022-02-22 20:02:48 There was maintenance planned, but for tomorrow 2022-02-22 20:03:37 btw, if we do not want to grow our mirror, we need to kill 6 releases to introduce a new one 2022-02-22 20:04:37 yeah 2022-02-22 20:04:43 its groing rapidly 2022-02-22 20:04:48 3.0 is more than 10x smaller compared to 3.15 2022-02-22 20:05:30 i guess a year from now, we are on 2T 2022-02-22 20:11:15 if we keep only the last 10 releases (5 years) that would save us another 100G 2022-02-22 20:24:04 I wonder how long we will keep armhf 2022-02-22 20:24:20 I guess the pi zero is the only thing keeping it alive 2022-02-22 20:26:03 Hmm, eth0 was already down according to Zabbix 2022-02-22 21:17:28 ikke: rpi zero will be EOL at 2026 iirc 2022-02-22 21:17:37 That's not very soon :) 2022-02-22 21:17:48 I know :) 2022-02-22 21:18:59 personally I wouldn't keep armhf too long on alpine, but I ncopa thinks alpine should keep it 2022-02-22 21:19:59 maybe some kind of community port, like ncopa mentioned 2022-02-22 21:20:04 I mean, Ariadne 2022-02-22 21:20:26 i'm in a meeting 2022-02-23 10:54:30 clandmeter: apparently natanael received a notification about t1.small instances, which is the case for dmvpn1 2022-02-23 10:54:36 We need to migrate the instance 2022-02-23 10:54:54 ok we need to change that email address 2022-02-23 10:55:00 or the contact window 2022-02-23 10:55:03 Yeah 2022-02-23 10:55:10 i transferd it to ncopa few years ago 2022-02-23 10:55:24 but it seems his mailbox is flooded :) 2022-02-23 10:55:30 "basically we are slowly but surely evacuating some chassis of t1.small machines" 2022-02-23 10:55:54 what is the solution? 2022-02-23 10:56:40 I don't have the e-mail where they provided the instructions 2022-02-23 10:57:08 no rootkit on ncopa's pc? :) 2022-02-23 10:57:17 clandmeter: I'll send an e-mail to equinix support 2022-02-23 12:36:58 So we need to redeploy dmvpn1 2022-02-23 14:45:48 ikke: yes but on what? 2022-02-23 14:47:16 A new instance 2022-02-23 14:47:34 Create new instance, copy, destroy old instance 2022-02-23 14:53:32 isnt it a bit overkill? 2022-02-23 15:00:30 What else do you suggest? 2022-02-23 15:01:23 They mentioned there is no migration 2022-02-23 16:52:06 I appreciate if you can take over the equinox emails. I thought you also got a copy of the emails 2022-02-23 16:52:25 apparently they only send it to the primary contact, which is you 2022-02-23 16:55:07 I figured. sorry about that 2022-02-23 16:56:47 Do you know if there is anytyhing we need to take into account when moving the dmvpn hub? 2022-02-23 16:56:57 We should copy the sqlite db 2022-02-23 16:57:55 Anything else? 2022-02-23 18:08:28 updated netbox to 3.1.8 2022-02-23 21:46:22 psykose: one thing you might be able to help me with, and learn some docker-compose along with it, is to look at setting up the latest aports-qa-bot 2022-02-23 21:46:37 They made some changes (upon request), which need to be integrated 2022-02-23 21:46:59 There are now 2 components 2022-02-23 21:47:15 one is a doas-like proxy, and the other is the 'bot' 2022-02-23 21:48:08 this is the build for it currently right https://gitlab.alpinelinux.org/alpine/infra/docker/aports-qa-bot/-/blob/master/Dockerfile 2022-02-23 21:48:14 then the docker-compose is in the usual place 2022-02-23 21:48:57 correct 2022-02-23 21:49:00 https://gitlab.alpinelinux.org/alpine/infra/compose/aports-qa-bot 2022-02-23 21:49:10 https://gitlab.alpinelinux.org/Cogitri/aports-qa-bot/-/merge_requests/25 2022-02-23 21:49:12 That's the MR 2022-02-23 21:49:51 so now it's both aports-proxy-bot and the other thing 2022-02-23 21:49:53 and some magic 2022-02-23 21:50:13 Yes, the idea is that most of the operations will use a project token 2022-02-23 21:50:49 But they need to be able to match maintainers to users 2022-02-23 21:51:14 A consequence is that it will no longer be algitbot 2022-02-23 21:51:29 hm 2022-02-23 21:51:43 do we run a proxy bot per compose-'pod' or just one 2022-02-23 21:52:06 ah it's for qa only 2022-02-23 21:52:07 sure 2022-02-23 21:52:18 yeah, they can be in the same project 2022-02-23 21:52:19 so i need to make a dockerfile for https://gitlab.alpinelinux.org/Leo/aports-proxy-bot 2022-02-23 21:52:25 then update the dockerfile for the qa bot 2022-02-23 21:52:27 then the compose 2022-02-23 21:52:29 and figure it out 2022-02-23 21:52:38 any advice for actually testing if it works :p 2022-02-23 21:52:49 I think we can test it against gitlab-test 2022-02-23 21:53:27 what magic do i press for all that 2022-02-23 21:54:06 also can a tag be added to https://gitlab.alpinelinux.org/Cogitri/aports-qa-bot for the new merged changes 2022-02-23 21:54:19 You should ping Cogitri for that 2022-02-23 21:54:24 though I have not seen him a lot lately 2022-02-23 21:54:27 he busy 2022-02-23 21:54:29 he double busy 2022-02-23 21:54:42 yeah, writing a thesis 2022-02-23 21:54:45 yep 2022-02-23 21:54:47 gitrev it is 2022-02-23 21:54:55 alright 2022-02-23 21:55:07 i'll look at all that tomorrow and hopefully not break anything 2022-02-23 21:55:16 i have written like a thousand dockerfiles before 2022-02-23 21:55:19 for now it's time to sleep 2022-02-23 21:55:21 sure 2022-02-23 21:55:23 goodnight 2022-02-23 21:55:24 nite 2022-02-24 16:13:25 Hi! Is there any documentation where I can read up on the Alpine Linux build infrastructure? Like, is this physical machines? Do they have access to the internet? How is it prevented that building a package compromises the build infra and hence all further packages? 2022-02-24 17:58:38 js: so, right now, it's all terrible 2022-02-24 17:59:07 we are designing a new build infrastructure that will be much better, with restricted network access, and per-package clean build instances and so on 2022-02-24 17:59:49 the current stuff is basically scripts that got cobbled together over the years and it does everything wrong and so we're trying to fix this :) 2022-02-24 18:02:03 the good news is that $dayjob really would like to see improvements to this situation, so i've been given carte blanche to spend as much time as necessary to work on fixing it 2022-02-24 18:04:40 I guess it is restricted what packages are built on it? So that not some random contributor can get the build infra to execute their code? 2022-02-24 18:06:14 yes 2022-02-24 18:06:17 js: a developer would need to accept a merge request 2022-02-24 18:06:21 only that which lands in aports.git gets built 2022-02-24 18:06:26 but in terms of like 2022-02-24 18:06:27 SLSA 2022-02-24 18:06:33 we are at SLSA-0 2022-02-24 18:06:36 :D 2022-02-24 18:06:50 the *new* build system will be SLSA-2 2022-02-24 18:06:53 SLSA? 2022-02-24 18:07:07 js: The builders themselves are all hardware with lxc running on top 2022-02-24 18:07:07 a software supply chain certification thing 2022-02-24 18:07:16 I think the system to the build infra is pretty restricted, running on dedicated HW, etc? 2022-02-24 18:07:22 yes 2022-02-24 18:07:26 "security levels for software assurance" 2022-02-24 18:07:26 ikke: heh, race 🙂 thx! 2022-02-24 18:07:36 so I guess binary package security is pretty ok 2022-02-24 18:07:43 it is okay, could be better 2022-02-24 18:07:52 will be better soon 2022-02-24 18:08:04 nobody likes the current setup 2022-02-24 18:08:17 its just nobody has written something that is acceptable either 2022-02-24 18:08:34 now the question is how secure the postmarket OS packages are 😀 I'm mostly looking into this because pmbootstrap without telling me downloaded apk.static, download qemu via that, and then installed stuff from pmOS repos executing the post-install scripts - all as root on my host system that I just wanted to use to write an SD card. So I am now going down the rabbit hole of figuring out just how much compromised my system is now. 2022-02-24 18:09:07 so it seems at least the Alpine infra is somewhat sane 2022-02-24 18:10:03 js: pmbootstrap does all of that in chroots fwiw 2022-02-24 18:10:16 well, as root in a chroot. 2022-02-24 18:10:25 and we all know root in a chroot == fail 2022-02-24 18:10:28 their build infrastructure is basically 2022-02-24 18:10:30 the same 2022-02-24 18:10:44 Our builders do not use root 2022-02-24 18:10:57 i meant for their packages 2022-02-24 18:11:02 not pmbootstrap 2022-02-24 18:11:09 ok, so guess it's not that bad then and all that happened is that i now need to not just trust the Fedora build infra, but also the Alpine and pmOS build infra. Which seems to be worse than Fedora (it has network access), but hopefully not all that worse. 2022-02-24 18:11:50 the new build infra will have restricted network access (e.g. only in fetch/prepare phase) 2022-02-24 18:12:09 network access is needed with stuff like rust and go 2022-02-24 18:12:18 otherwise you have to deal with managing 1000s of artifacts 2022-02-24 18:12:21 to build anything 2022-02-24 18:12:48 which is what Fedora does 😉 and even pkgsrc 2022-02-24 18:15:05 yes 2022-02-24 20:37:40 https://git.alpinelinux.org/ appears to be down, if anyone hadnt noticed yet? 2022-02-24 20:38:25 oh, it might just be very slow at the moment (even https://downforeveryoneorjustme.com/git.alpinelinux.org?proto=https thought it was down, not just me) 2022-02-24 20:40:10 j`ey: it is quite fine for me 2022-02-24 20:40:39 it was slow for me before 2022-02-24 20:40:42 now it seems fine 2022-02-24 20:40:46 it gave me one 504 and now it works 2022-02-24 20:40:49 weird blip in the interwebs 2022-02-24 20:40:51 But our monitoring did not pick anything up 2022-02-25 12:47:46 clandmeter: ^ the cron job still tried to reload traefik instead of nginx 2022-02-25 16:56:03 so, i am working on a toolset that allows one to manage an apk-tools repository as a cloud storage bucket on any S3-compatible service (e.g. Linode) 2022-02-25 17:19:18 ikke: well i guess it starts https://img.ayaya.dev/tm5fYgjCYEBK.png , what credentials and things do i need to actually go test it somewhere 2022-02-27 07:12:21 clandmeter: the riscv64 builder host is unresponsive 2022-02-27 07:12:43 tried oob access, could login, but now that's stuck as well 2022-02-27 08:19:37 rebooted it 2022-02-27 15:08:10 psykose: I saw some errors with gitaly on our test instance 2022-02-27 15:08:15 wanted to investigate it before upgrading 2022-02-27 15:08:15 ah 2022-02-27 15:08:20 good luck :) 2022-02-27 15:08:30 also i think you had typoed the message cause it only said 14.7 2022-02-27 15:08:33 without the minor 2022-02-27 15:08:49 The minor is a detail :P 2022-02-27 15:09:19 You mean, without the patch 2022-02-27 15:13:23 yes 2022-02-27 15:13:34 they're all the small numbers to me :) 2022-02-27 15:14:33 In this case, we switch from 14.5 to 14.7, so I left out the patch number 2022-02-27 15:14:41 ah 2022-02-27 15:14:43 makes sense 2022-02-27 15:39:37 psykose: if you have some time, you could try to find out why abstract_type-0.0.7 is not installed for gitaly-ruby :P 2022-02-27 15:39:51 INFO[0008] Bundler::GemNotFound: Could not find abstract_type-0.0.7 in any of the sources supervisor.args="[bundle exec bin/ruby-cd /etc/gitlab/gitaly /home/git/gitaly-ruby/bin/gitaly-ruby 83 /tmp/gitaly-internal3052918779/ruby.0]" superv 2022-02-27 15:39:53 isor.name=gitaly-ruby.0 2022-02-27 15:40:06 how would i check that 2022-02-27 15:40:10 gitaly isn't even in aports is it 2022-02-27 15:40:39 and that gem isn't either 2022-02-27 15:40:41 https://gitlab.alpinelinux.org/alpine/infra/docker/gitlab/-/blob/14.5-stable/gitaly/build/build-gitaly-ruby 2022-02-27 15:40:44 ah 2022-02-27 15:42:59 where is this libsetup thing from 2022-02-27 15:43:18 https://gitlab.alpinelinux.org/alpine/infra/docker/gitlab/-/blob/14.5-stable/lib/libsetup 2022-02-27 15:45:17 what is make /tmp/src/.ruby-bundler supposed to be calling 2022-02-27 15:46:56 https://gitlab.alpinelinux.org/alpine/infra/docker/gitlab/-/blob/14.5-stable/gitaly/Dockerfile#L46 2022-02-27 16:20:56 not sure, it's in the .lock 2022-02-27 16:21:15 no reference in the main gemfile, not sure how that works 2022-02-27 16:23:21 all i can say is that the `bundle install` is skipped(?) if you patch out the .ruby-bundle target from build 2022-02-27 16:23:28 since there is nothing else what would call it 2022-02-27 16:29:57 ah right the make calls it 2022-02-27 16:41:06 no idea tbh 2022-02-27 16:41:40 it doesn't get printed during the bundle phase for 14.7.4 2022-02-27 17:01:01 OK, thanks! I'll see if I can find something 2022-02-27 17:04:03 i think just hard adding it to the Gemfile would fix it 2022-02-27 17:49:54 hm, nope, still doesn't show up 2022-02-27 17:55:48 Maybe something is preventing it from being installed 2022-02-27 18:07:03 that should be an error with it being top-level defined 2022-02-27 18:07:18 but also this whole build is quite hard to work with 2022-02-27 18:16:13 oh, i patched the wrong stage 2022-02-27 18:16:16 nvm 2022-02-27 18:20:02 https://img.ayaya.dev/Eh0B4Y84W5Wr 2022-02-27 18:20:15 that should work in build-gitaly-ruby 2022-02-27 18:20:25 for 14.7.4 2022-02-27 18:21:50 i was applying it to the build-gitaly stage by accident 2022-02-27 18:21:54 they should really be the same stage 2022-02-27 18:33:05 They are separate so they can be built in parallel 2022-02-27 18:33:53 it doesn't save much time 2022-02-27 18:34:16 And, so that we don't need to have go installed in a ruby image or visa versa 2022-02-27 18:36:09 they're not, it's a building stage 2022-02-27 18:36:19 there is nothing to gain by splitting the make 2022-02-27 18:36:44 it's just calling make twice with two different environents for an identical end result that is copied 2022-02-27 18:37:08 maybe if each of them took 5 hours the parallelisation would mean something 2022-02-27 18:37:39 and it requires the patch to remove the bundle install from the one that uses go only too 2022-02-27 18:37:41 The ruby part takes 20 minutes 2022-02-27 18:37:45 much more effort than just.. add deps, call make 2022-02-27 18:38:36 I optimized for development time when rebuilding 2022-02-27 18:44:44 In my head it makes sense to have dedicated images for building for different languages 2022-02-27 18:49:01 We first just had a single image with gitaly, gitlab-shell and gitlab in one 2022-02-27 18:49:20 so if anything failed, you had to redo the entire installation again 2022-02-27 18:53:03 This is how gitlab builds it btw: https://gitlab.com/gitlab-org/build/CNG/-/blob/master/gitaly/Dockerfile 2022-02-27 18:55:09 that's already 10x easier to understand because everything is in one file 2022-02-27 18:55:18 the only thing imported is cleanup-gems and the entrypoint 2022-02-27 18:55:29 i would rather have that one :p 2022-02-27 18:58:27 If only the ruby part was not terribly slow 2022-02-27 18:58:44 i found that doing it locally actually used more cores 2022-02-27 18:58:51 but even adding JOBS as well didn't fix it in the image 2022-02-27 18:58:57 Hmm 2022-02-27 18:59:11 only the grpc part uses cores in the image, but everything uses them locally 2022-02-27 18:59:30 i also had to rebuild the whole thing each time, because editing the scripts didn't invalidate the cache so i needed --no-cache 2022-02-27 19:01:23 maybe the slightly newer bundler handles the cores better 2022-02-27 19:02:55 I never have to use --no-cache 2022-02-27 21:12:39 psykose: I think I know why it's not installed 2022-02-27 21:13:01 rspec-parametrized -> unparser -> abstract_type 2022-02-27 21:13:29 and we specify 'without test' 2022-02-27 21:14:00 so.. why does it need it at runtime then/ 2022-02-27 21:14:47 i dunno 2022-02-27 21:14:52 it's not used by anything in gitaly either 2022-02-27 21:15:32 the patch should make sure it's there if you're a fan of such a scuffed fix :) 2022-02-27 21:22:17 It's bundler itself that is trying to find it 2022-02-27 21:22:55 So probably the without flags not being persisted 2022-02-27 21:24:13 yup 2022-02-27 21:24:14 bundle config set --global deployment false 2022-02-27 21:24:16 bundle config set --global without development test mysql aws kerberos 2022-02-27 21:24:21 in the gitaly container seems to fix it 2022-02-27 21:46:58 ah 2022-02-27 21:47:00 makes sense 2022-02-27 21:47:56 I now copied .bundle/config over to the final image 2022-02-27 21:50:48 So if this works, I think I'll do the upgrade tomorrow evening 2022-02-27 21:51:18 mhm 2022-02-27 21:52:24 ? 2022-02-27 21:54:02 sounds good to me :) 2022-02-27 21:54:17 unless you really want to do it right now 2022-02-27 21:55:10 Not really 2022-02-27 21:57:56 https://gitlab.alpinelinux.org/alpine/infra/docker/gitlab/-/jobs/647099 2022-02-27 21:58:56 It's too late now, I don't want to have to troubleshoot any issues :) 2022-02-27 21:59:50 but the thrill.. 2022-02-27 22:01:37 I have enough thrill already :P 2022-02-27 22:01:51 glad we got the builders back into shape 2022-02-27 22:01:54 yes 2022-02-27 22:02:36 still impressed we identified 3 separate issues and it didn't actually fix it 2022-02-27 22:02:41 yeah, me too 2022-02-27 22:03:01 unintentional yak shaving 2022-02-27 22:03:44 we didn't manage to fix the unitialized memory access I think 2022-02-27 22:03:54 the memset broke more than it fixed 2022-02-27 22:03:57 no, that part we didn't 2022-02-27 22:04:05 last little piece 2022-02-27 22:04:15 which is mind boggling 2022-02-27 22:04:17 yes 2022-02-27 23:49:15 ikke: seems i missed your msg, is it fixed now? 2022-02-27 23:49:32 after the reboot 2022-02-28 15:15:00 Yes 2022-02-28 19:40:11 Upgraded gitlab to 14.7 so that we mitigated the announced vulnerabiltiies 2022-02-28 19:47:58 clandmeter: we still need to look at migrating / redeploying dmvpn1 2022-02-28 22:27:20 ikke: i think it makes sense to move it to linode? 2022-02-28 22:28:49 ikke: nice work! :) 2022-02-28 22:28:55 as usual ofc