2021-09-01 07:22:05 clandmeter: re rv64: isn't aports-build a shell script? if it's stale, shouldn't there be a stale subprocess? 2021-09-01 07:22:31 also: what do you mean by stale? is the process just in sleep state and doesn't do anything? 2021-09-01 07:22:53 im not sure what happens, but it happens more often, could be related to the kill feature i added. 2021-09-01 07:23:37 but it doesnt really make sense as it worked before and the logic is to kill all childs of abuild 2021-09-01 07:23:59 im testing it as we speak 2021-09-01 07:25:06 what we really need is a general timeout feature in abuild 2021-09-01 08:36:23 hmm something is really weird on rv64 builder 2021-09-01 08:36:37 it kicks me out of the session 2021-09-01 08:36:45 wonder if the container gets restarted 2021-09-01 08:37:11 is the mqtt-exec process crashing / exiting? 2021-09-01 08:37:31 docker ps doesnt mention it 2021-09-01 08:37:36 or im reading it wrong 2021-09-01 08:37:54 i had a: watch pstree running 2021-09-01 08:38:04 and it got killed 2021-09-01 08:38:27 it would explain the stale pid file 2021-09-01 08:39:21 Up 39 minutes 2021-09-01 08:39:58 maybe we are bumping into that issue we had with mqtt-exec like before 2021-09-01 08:41:21 It crashing? 2021-09-01 08:44:02 yeah 2021-09-01 08:44:08 although ncopa fixed that iirc 2021-09-01 08:44:24 something with client naming or such 2021-09-01 08:47:29 ok current container started 8:46 minus 47 minutes, so lets asume 8:00 for simplicity. ill remove the lock file again and see what happens. 2021-09-01 08:56:00 ikke: the container exists 2021-09-01 08:56:10 so thats what happens 2021-09-01 08:56:45 exists or exits? 2021-09-01 08:57:06 :p 2021-09-01 08:57:11 it dies :) 2021-09-01 08:57:26 heh 2021-09-01 08:57:31 ERROR: 137 2021-09-01 08:57:45 137 looks like a signal? 2021-09-01 08:58:05 its when im attached 2021-09-01 08:58:24 137 is signal 9 (sigkill) 2021-09-01 08:58:42 Do you see OOM messages in dmesg? 2021-09-01 08:58:47 no 2021-09-01 08:58:51 alraedy checked 2021-09-01 08:58:59 i use signal 9 for my patch 2021-09-01 08:59:09 but that should only fire on the right cmd 2021-09-01 09:00:18 but i guess if you are in a docker exec session and pid1 dies, you get that msg, or is it related to the msg pid1 gets? 2021-09-01 09:02:09 maybe its related to smokeping 2021-09-01 09:03:42 smokeping? 2021-09-01 09:05:57 funny, in #zabbix they were also talking about smokeping :) 2021-09-01 09:20:11 nmeum: fyi regarding 3768f088d9e5d672b9ac303a8b21368a2ac367bd 2021-09-01 09:20:27 these are disabled mostly because packages will just hang 2021-09-01 09:20:55 if there is a build error i "mostly" include the error in the commit. 2021-09-01 09:21:11 but i should probably included the hang too :) 2021-09-01 09:21:32 but boostrapping takes so much time, you get sloppy 2021-09-01 09:22:13 ikke: im now building it manually 2021-09-01 09:22:57 clandmeter: ah, I just went through all packages in main/ that were disabled on riscv64, tested them on my hifive unmatched and enabled them if they worked fine. wasn't aware that smokeping causes a hang with qemu-user 2021-09-01 09:23:45 just add a comment next time (: 2021-09-01 09:23:46 im building it manually now on the builder, as the builder gets killed if it tries to to build it 2021-09-01 09:24:12 why does it get killed? does it run out of memory? 2021-09-01 09:24:20 not sure its related at all 2021-09-01 09:24:39 the container is running mqtt-exec 2021-09-01 09:24:43 its pid1 2021-09-01 09:25:08 something happend now 2021-09-01 09:25:27 can you get a coredumb for mqtt-exec or something? 2021-09-01 09:26:06 smokeping is broken 2021-09-01 09:26:11 https://tpaste.us/8EWg 2021-09-01 09:29:25 interesting, that does 100% not happen on the unmatched 2021-09-01 09:30:02 let's see if I can reproduce this in a qemu-user chroot 2021-09-01 09:30:03 these kind of errors are common on qemu-user, its not perfect.. 2021-09-01 09:30:26 I prefer to debug these kinds of errors though instead of just disabling stuff since this might uncover some underlying issue 2021-09-01 09:30:38 that's also how I found the breakage in libatomic on riscv64 2021-09-01 09:31:33 sure, but if you want to bootstrap an entire arch on qemu-user you have to make concession, do the debugging later. 2021-09-01 09:31:57 there are also a lot of textrel issues 2021-09-01 09:32:06 yeah ofc 2021-09-01 09:32:07 i have no idea how to solve those 2021-09-01 09:32:22 I think the textrel thing is a gcc toolchain issue 2021-09-01 09:32:52 we have not spend a lot of time on the toolchain, so yes thats possible. 2021-09-01 09:33:05 there is also a patch in gcc for libatomic 2021-09-01 09:33:18 two actually 2021-09-01 09:33:35 did you find out what the issue is you are having? 2021-09-01 09:33:45 with libatomic? 2021-09-01 09:33:49 yes 2021-09-01 09:34:21 yes, I did. the libatomic configure script does not correctly autodetect atomic compiler builtins on riscv64 due to our --enable-autolink-libatomic patch 2021-09-01 09:34:33 see https://gitlab.alpinelinux.org/alpine/aports/-/issues/12948 and https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/24570 2021-09-01 09:34:50 that should hopefully be fixed now 2021-09-01 09:35:11 the someping build also fails here in a qemu chroot 2021-09-01 09:36:50 looks like perl segfaults or something? can look into this l 2021-09-01 09:36:53 i can reproduce the failure on the same moment, just after netdns is installed 2021-09-01 09:36:53 *later 2021-09-01 09:37:31 for me it fails on "GEN touch" 2021-09-01 09:37:43 ok, do we disable it for now? 2021-09-01 09:38:00 https://tpaste.us/r6Y1 2021-09-01 09:38:16 yes, I don't have time to look into it right now 2021-09-01 09:38:38 pls add a comment though that the build fails in qemu or something along those lines :) 2021-09-01 09:38:58 the builder is already a week behind 2021-09-01 09:40:02 nmeum: is that from abuild output? 2021-09-01 09:40:19 yep 2021-09-01 09:40:26 CBUILD=riscv64 abuild rootbld 2021-09-01 09:40:29 so it fails in different places on diff hw 2021-09-01 09:40:37 or diff env 2021-09-01 09:40:53 might be due to parrallel build 2021-09-01 09:41:18 there is not much parralel in this phase 2021-09-01 09:41:47 anyways ill disable it 2021-09-01 09:41:58 yes 2021-09-01 09:42:00 see if that also solves the builder error 2021-09-01 09:42:11 if so we need to check why this pkg can kill a builder :) 2021-09-01 09:42:29 yeah, that shouldn't happen :D 2021-09-01 11:06:27 seems like rv64 is moving along now 2021-09-01 11:06:36 so we need to take a look at someping 2021-09-01 11:09:51 clandmeter: so it was purely smokeping that was causing the issue? 2021-09-01 11:10:09 thats my observation for now 2021-09-01 11:10:43 maybe smokeping will kill parent processes? 2021-09-01 11:11:22 why / how? 2021-09-01 11:14:02 i have no clue, just thinking out loud 2021-09-01 11:16:03 or maybe qemu-user is involved, but i wouldnt have any idea how. 2021-09-01 11:20:57 https://tpaste.us/6Wja 2021-09-01 11:21:05 thats the log when it gets terminated 2021-09-01 11:21:26 it just stops in the middle of a configure 2021-09-01 11:21:34 so something is terminating it 2021-09-01 13:41:25 puh that smokeping thirdparty/Makefile is straight from hell 2021-09-01 13:42:05 it downloads some minified perl script https://cpanmin.us/ and then uses that to download more dependencies etc. pp. 2021-09-01 13:42:09 fun fun fun 2021-09-01 13:44:06 queue "how to make package managers cry" presentation 2021-09-01 13:45:06 nmeum: hell fire smoke, it kind of makes sense ;-) 2021-09-01 15:49:46 would probably be best to simply package the dependencies it needs (we have most of them already anyhow) https://github.com/oetiker/SmokePing/blob/master/cpanfile and thereby avoid using this thirdparty makefile alltogether 2021-09-01 15:49:56 though that doesn't tell us why it crashes the builder 2021-09-01 18:37:57 good thing is, its still running. 2021-09-02 07:10:50 clandmeter: are you aware of docker-compose service profileS? 2021-09-02 07:12:57 nope 2021-09-02 07:13:15 Just encountered them the other day 2021-09-02 07:13:22 You can attach one or more profiles to services 2021-09-02 07:13:32 reminds me i would like to try the new compose 2021-09-02 07:13:37 docker compose? 2021-09-02 07:13:42 yes 2021-09-02 07:13:49 golang one 2021-09-02 07:14:07 Need to still try it as well 2021-09-02 07:15:22 so in principle i can use this to lets say disable some service? 2021-09-02 07:16:59 one usecase is run a different set of services for different scenarios 2021-09-02 07:17:12 my main usecase is to have 'task' services 2021-09-02 07:37:22 right 2021-09-02 07:37:45 well i guess there are more use cases, and this seems an interesting feature 2021-09-02 07:42:27 indeed 2021-09-02 13:57:32 got notified about 1password account for opensource teams. dunno if infra team is interested in that. https://twitter.com/tcely/status/1433397726687944707 2021-09-03 07:05:58 i can recommend self-hosting bitwarden or something similar, especially considering 1p is proprietary saas 2021-09-03 07:08:43 Yeah, I'm not really interested in 1p myself to be honest 2021-09-03 07:13:03 the same goes for lastpass 2021-09-03 07:13:29 and overall, the more a company pays to advertise their "free" product, the less i trust it 2021-09-03 07:13:54 danieli: well told 2021-09-03 07:18:15 also I don't trust companies who suddenly opensfrees their products, though there are exceptions 2021-09-03 07:18:58 opens/frees* 2021-09-03 07:48:23 +1 for bitwarden 2021-09-03 09:27:48 ikke: rv64 is not yet on pkgs? 2021-09-03 09:28:34 no 2021-09-04 07:05:22 ikke: seen email? 2021-09-04 07:05:49 Regarding gitlab 2021-09-04 07:06:04 clandmeter: yes, afaict, 13.12 was not affected 2021-09-04 07:06:33 Ah ok 2021-09-04 07:06:50 Didn't know which version we are running 2021-09-04 07:07:02 I checked it when I saw the mail 2021-09-04 07:07:24 What do we want to do regarding gl upgrade? 2021-09-04 07:07:59 I need to arrange some time 2021-09-04 07:08:06 Little hard recently 2021-09-04 07:08:47 Got an email yesterday regarding rapidswitch 2021-09-04 07:08:56 Maintenance 2021-09-04 07:09:05 Kind of weird 2021-09-04 07:09:14 I could try to upgrade to 14.x with the current monolithic image 2021-09-04 07:09:20 14.0.x 2021-09-04 07:10:23 That's up to you, you can also switch to the new one if you think it's ok 2021-09-04 07:10:38 Which part do I need to look at? 2021-09-04 07:11:57 I guess just if your okay with the direction / way the images are setup 2021-09-04 07:14:56 I'm ok if you are ok :) 2021-09-04 07:15:19 I think I added some comments last time 2021-09-04 07:16:27 Yes, to which I responded again 2021-09-04 07:19:21 Oklet me check 2021-09-04 07:23:34 I guess we agree on most places, let get this thing moving :) 2021-09-04 07:24:13 One open point for me to address 2021-09-04 07:24:15 the version 2021-09-04 07:29:03 So we could expect some network issues to gbr1 2021-09-04 08:09:02 version and network issues? 2021-09-04 08:10:26 > I see that the versions in gitlab and the root GITLAB_VERSION file are out of sync? 2021-09-04 08:15:38 I don't follow 2021-09-04 08:15:59 But in not behind my PC :) 2021-09-04 08:20:46 2 separate things 2021-09-04 08:20:55 version is about gitlab, a comment you left behind 2021-09-04 08:21:07 network issues is about gbr1, rapidswitch maintenance 2021-09-04 09:32:29 ah now i get it :) 2021-09-04 09:32:38 its two different issues you are refering to 2021-09-04 09:33:45 It took an hour for you to get it? :P 2021-09-04 10:23:47 Yes I'm slow 2021-09-04 10:23:59 Reg gbr 2021-09-04 10:24:10 That's a nice window... 2021-09-04 10:24:26 Glad I'm not a paying customer 2021-09-04 10:24:41 23.00 to 4.00 BST 6-10 sept 2021-09-04 10:54:00 jesus, that's quite a large window 2021-09-04 10:54:20 it'll just be entirely offline in that period? 2021-09-04 10:54:29 no 2021-09-04 15:27:00 kunkku: trying to setup dmvpn on our new ppc64le host. When I run setup-dmvpn , I choose the defaults, and then it says " 2021-09-04 15:27:03 "% Can't find BGP instance" 2021-09-04 15:27:07 when starting nhrpd 2021-09-04 15:28:38 any idea what might be going on? 2021-09-04 17:56:07 the riscv64 builder seems to have some issues again 2021-09-04 17:56:11 https://build.alpinelinux.org/buildlogs/build-edge-riscv64/main/gzip/gzip-1.11-r0.log 2021-09-04 17:56:14 > Bus error (core dumped) 2021-09-04 17:56:17 :/ 2021-09-04 17:58:14 iirc ikke mentioned its disk is full 2021-09-05 07:34:07 who have access to rv64 builder? could s/he look what is the problem with it? 2021-09-06 12:28:28 ikke: I haven't seen the BGP error before 2021-09-06 12:31:49 I can check it out if you give me access to the host 2021-09-06 12:36:09 might actuall be harmless if it works 2021-09-07 13:46:13 What's up with the aarch64 builder? It seems to have been stuck for ages 2021-09-07 13:48:14 yes, for 2 or 3 days 2021-09-07 14:47:18 I'm seeing HTTP 502 errors today when trying to view aports repos like this: https://git.alpinelinux.org/aports/tree/community/edk2?h=master 2021-09-07 17:16:26 the aarch64 edge builder has been getting stuck a lot the last few days and many updates aren't making it out for aarch64. someone said in -devel that there is no watchdog (or it's broken?) for stuck builds. basically I'm wondering if there's anything I can do to help out... 2021-09-07 17:19:23 yes, you can. read prayer to saint ikke ;) 2021-09-07 17:20:26 huh? 2021-09-08 04:25:01 hi, git.alpinelinux.org gives Nginx 502 Bad Gateway 2021-09-08 04:31:50 clandmeter: can you fix aarch64 and x86 builders please 2021-09-08 05:33:21 whats wrong with the builders? 2021-09-08 05:33:32 i can have a look at them in an hour or two 2021-09-08 05:35:54 aarch64 is stuck for 3 days in one pkg, forgot which 2021-09-08 05:36:24 and iirc x86 also 2021-09-08 05:39:35 I didn't checked if I have access to builders (I'm infra team member) 2021-09-08 05:39:50 maybe I have but I don't know 2021-09-08 05:48:59 ncopa: they are stuck :( 2021-09-08 05:57:42 who here knows about the Git ? 2021-09-08 06:33:28 i restarted build-edge-aarch64 2021-09-08 06:57:23 thanks 2021-09-08 08:25:40 Ariadne: all ok now? 2021-09-08 08:25:44 sorry i was a bit busy 2021-09-08 08:26:06 yes, i think all faults are cleared now 2021-09-08 08:26:08 thanks 2021-09-08 08:26:17 sorry for interrupting your day :) 2021-09-08 08:26:26 i restarted git.a.o 2021-09-08 08:26:38 it happens from time to time, not sure whats the cause. 2021-09-08 08:26:54 need to delve into it ones i feel bored 2021-09-08 13:51:04 ncopa: could you 'kick' aarch64 and x96 builders again 2021-09-08 13:57:03 we have a new arch? 2021-09-08 13:58:25 :) 2021-09-08 14:01:29 clandmeter: could you kick these two builders, I disabled libtorrent-rasterbar on them 2021-09-08 14:01:44 im on it 2021-09-08 14:01:50 im on build-edge-x86 2021-09-08 14:02:10 i was looking what was hanging. it appears to be test_priority on build-edge-x86 2021-09-08 14:03:29 I didn't looked why it hangs, simply disabled it on these two arches 2021-09-08 14:03:53 (yes, bad attitude but for now I don't have better idea) 2021-09-08 14:04:18 on build-edge-aarch64 it appears to be test_resume 2021-09-08 14:04:58 hmm, lets check on dev lxc 2021-09-08 14:05:11 i was not able to reproduce it on x86 2021-09-08 14:06:26 strace showed that it was stuck waiting for a futex 2021-09-08 14:30:48 hmm, it passed on dev lxc 2021-09-08 14:31:05 aarch64, I mean 2021-09-10 20:09:20 hmm, I see this on aarch64/armv7 builer machine '[11034414.626429] list_del corruption. prev->next should be ffff080386fb7c98, but was ffff080be8cce898' dmesg output 2021-09-11 11:28:57 hmm 2021-09-12 19:57:23 lol 2021-09-12 19:57:31 do we have a site that doesnt have space issues? 2021-09-12 20:02:48 well, do we? 2021-09-12 20:04:54 no clue 2021-09-12 20:04:59 and welcome back 2021-09-12 20:05:06 i cannot login into nld8 2021-09-12 20:05:15 thats your box 2021-09-12 20:05:31 ikke: i think its also missing from netbox 2021-09-12 20:05:43 thanks 2021-09-12 20:05:46 i cleaned up gitlab a bit 2021-09-12 20:05:57 but its again the artifacts thats the blame 2021-09-12 20:06:12 i thought we fixed that recently 2021-09-12 20:06:19 that auto cleanup thingy 2021-09-12 20:07:19 We did fix at least something 2021-09-12 20:07:49 seems not enough 2021-09-12 20:08:09 s390x also has space issues 2021-09-12 20:10:32 working on x86 now 2021-09-12 20:31:20 s390x has 10G $HOME/.cache 2021-09-12 20:31:27 so we want to clean that up? 2021-09-12 21:06:03 arm* builders are full and failing completely 2021-09-12 21:14:15 maxice8: they are not full, but fail for some reason 2021-09-12 21:14:44 89% full 2021-09-12 21:14:56 huh 2021-09-13 05:56:24 hi 2021-09-13 05:56:36 whats up with the ARM builders :s 2021-09-13 06:11:34 I think someone cleaned up a bit too extensive 2021-09-13 06:15:35 What's wrong? 2021-09-13 06:15:51 /var/cache/distfiles is almost completely empty 2021-09-13 06:16:02 it was missing /var/cache/distfiles/buildlogs 2021-09-13 06:16:20 I didn't touch it 2021-09-13 06:16:25 me neither 2021-09-13 06:16:38 Sounds not good 2021-09-13 06:17:25 hm, is it possible that I did it by removing it from lxc? (: 2021-09-13 06:17:47 I doubt, but ? 2021-09-13 06:19:18 mps: no, it's not mounted for you 2021-09-13 06:19:51 oooff, good 2021-09-13 06:21:18 clandmeter: happened saturday 10pm 2021-09-13 06:22:57 huh, around this time I cleaned /var/cache in my lxc 2021-09-13 06:23:26 10pm utc for the record 2021-09-13 06:23:49 can't remember exact time but at the evening 2021-09-13 06:25:50 what arch? 2021-09-13 06:26:34 neither your aarch64 nor armv7 container have it mounted 2021-09-13 06:28:11 aarch64 2021-09-13 06:28:39 so, who did it, good if didn't 2021-09-13 06:34:45 ikke: which host is it? 2021-09-13 06:35:14 usa9 2021-09-13 06:39:34 ikke: it is 2021-09-13 06:39:53 via common.conf 2021-09-13 06:39:59 clandmeter: ooh 2021-09-13 06:40:01 oof 2021-09-13 06:40:12 mps: its your faulth :p 2021-09-13 06:40:14 jk 2021-09-13 06:40:43 i guess we should move that option 2021-09-13 06:41:23 we do have 100GB free space 2021-09-13 06:41:26 not bad 2021-09-13 06:42:08 clandmeter: I cleaned around 100GB ;) 2021-09-13 06:42:26 :) 2021-09-13 07:01:08 glad to see builders are fixed 2021-09-13 17:26:00 58G gained on usa2 just with rdfind 2021-09-13 17:26:16 89G on nld8 yesterday 2021-09-13 18:57:04 nice 2021-09-13 18:57:16 hmm, looks like git.a.o is not acting well 2021-09-14 10:17:42 hmf 2021-09-14 10:19:20 What is taking all this space again? 2021-09-14 10:58:25 hmm 2021-09-14 11:16:02 for the record, it was .cache/go-build 2021-09-15 12:17:39 clandmeter: we still need to commission the new ppc64le builders. I started with it, but got stuck on the dmvpn setup 2021-09-15 12:18:10 i will be on holidays from today 2021-09-15 12:18:16 so im not much of help 2021-09-15 14:41:19 i wish i could help :( but the alpine dmvpn implementation is unknown to me 2021-09-15 14:41:22 i only used it on cisco routers 2021-09-15 14:42:30 https://tpaste.us/7Vw6 2021-09-15 14:46:50 is dmvpn tied with awall or is this standalone? 2021-09-15 14:47:05 if it's iptables i might be able to help, i know that pretty well 2021-09-15 14:47:30 can you paste the output of iptables-save and redact anything potentially sensitive? 2021-09-15 14:48:18 actually - it looks like it's trying to load a saved file and that's what makes it choke 2021-09-15 14:56:25 Sorry, this is the output of setup-dmvpn: https://tpaste.us/1EgZ 2021-09-15 14:56:36 That before was setup-firewall 2021-09-15 14:56:57 It does not setup any gre interfaces 2021-09-15 15:11:40 looks like all the usual stuff for routing although that stack is a little old fashioned by now 2021-09-15 16:06:03 ikke: is there any created in /e/n/i ? 2021-09-15 16:06:29 yes 2021-09-15 16:06:59 weird :D 2021-09-15 16:07:01 https://tpaste.us/plBE 2021-09-15 16:07:21 ifup gre1 does nothing 2021-09-15 16:08:59 hmm 2021-09-15 16:09:19 do you have real iproute2 installed? 2021-09-15 16:09:46 Installed: 2021-09-15 16:09:48 iproute2-5.10.0-r1 2021-09-15 16:10:41 this is alpine 3.13 ftr 2021-09-15 16:23:46 interesting. After a reboot, the interfaces have different names, and the hostname is different 2021-09-15 16:24:12 ugh, ubuntu... 2021-09-15 16:49:17 ;) 2021-09-15 19:27:21 is ifupdown-ng supposed to handle tunnel ifaces in 3.13? 2021-09-15 19:28:03 or is the tunnel pkg still needed? 2021-09-15 19:40:36 Not sure myself 2021-09-15 19:43:13 https://github.com/ifupdown-ng/ifupdown-ng/blob/main/doc/interfaces-tunnel.scd 2021-09-15 19:44:00 That does not exist in 0.11.2 2021-09-15 19:44:20 So I suppose tunnel is still needed 2021-09-15 19:48:36 does the script get executed when running ifup gre1 ? 2021-09-15 19:49:18 sadly, I don't have access to the host atm 2021-09-15 19:51:03 ok, let me know if you need help debugging the issue 2021-09-15 19:52:42 thanks, would be nice. I'll ping you when I have access again\ 2021-09-17 11:13:39 aarch64 builder is stuck with python, I think 2021-09-20 13:24:29 clandmeter: not sure if you noticed it, but I have the gitlab changes I've been working on running on gitlab-test 2021-09-20 13:46:52 I didn't, I'm climbing a mountain in north of Italy.:) 2021-09-20 13:51:44 And still replying on IRC :D 2021-09-20 13:52:43 Yes, having a ๐Ÿบ now 2021-09-20 13:53:06 Need vitamin to keep climbing 2021-09-20 13:53:20 Good ๐Ÿ‘ 2021-09-20 17:20:23 hahaha, have a nice trip clandmeter 2021-09-20 20:11:12 this could be interesting for us (we are using openvpn for infra access) https://github.com/OpenVPN/ovpn-dco 2021-09-20 20:11:25 openvpn kernel module 2021-09-21 04:39:46 Oh, I missed that the mips builder is back 2021-09-21 14:49:05 mps: i would prefer to switch to wg instead 2021-09-21 14:56:34 clandmeter: also I prefer it. we talked about this here about year or more ago but iirc someone was against, ikke or ncopa, can't remember 2021-09-21 14:57:33 or it was kunkku, sorry I don't keep such things in brain 2021-09-21 14:58:54 I'm certainly not against 2021-09-21 14:59:37 nice to hear 2021-09-21 14:59:44 and sorry 2021-09-21 15:09:58 Nobody was I think, and now the clients are getting better on all platforms. Just needs some love. 2021-09-21 15:11:23 We really need some extra infra support 2021-09-21 15:11:46 'extra'? what you mean? 2021-09-21 15:12:09 additional, more 2021-09-21 15:12:21 machine, time, people? 2021-09-21 15:12:29 people 2021-09-21 15:14:13 well, I'm ready to help but there where I have knowledge (don't like idea to touch critical area and make a mess) 2021-09-21 15:14:34 I think you can help with wg if you like 2021-09-21 15:14:52 I use wg for years 2021-09-21 15:15:11 started to test it with first patches 2021-09-21 15:16:21 if you can tell where we should start I will look 2021-09-21 15:17:49 You need a host with dmvpn routing 2021-09-21 15:18:19 my local? 2021-09-21 15:18:49 Can but got don't have dmvpn 2021-09-21 15:19:15 We have one master where we run ovpn 2021-09-21 15:19:28 Master for dmvpn 2021-09-21 15:19:49 You mean to bridge wg with dmvpn 2021-09-21 15:19:54 But that host is limited in access 2021-09-21 15:20:17 ikke: do we have an alternative to play with? 2021-09-21 15:22:41 We could also setup an additional micro metal instance on equinix 2021-09-21 15:24:23 I think there is no reason why ovpn/wg should run on a dmvpn hub 2021-09-21 15:26:36 haven't gotten a response about the ppc64le hosts 2021-09-21 15:27:38 kunkku: on a spoke I guess 2021-09-21 15:27:43 we have tailscale pkg which maybe could be of help for wg network 2021-09-21 15:28:15 if you run it on spoke, your cert has to include the prefixes for all remote endpoints 2021-09-21 15:31:21 kunkku: what are the alternatives? 2021-09-21 15:33:10 I was thinking setting up a normal dmvpn spoke and running it there 2021-09-21 15:33:43 it would have a prefix of 172.16.x.0/24 2021-09-21 15:33:57 clandmeter: I'm btw pretty far with gitbal 2021-09-21 15:34:26 if that is too small for all remote users, we need another prefix in addition 2021-09-21 15:34:44 nah, that should be more than enough 2021-09-21 15:34:51 ikke: is that something new? 2021-09-21 15:36:11 clandmeter: yes, haven't you heard? 2021-09-21 15:36:32 Nope I'm not into balls 2021-09-21 15:36:52 Maybe meatballs 2021-09-21 15:37:33 I'm recreating gitlab-test to test the migration one more time, and there are some smaller things I need to address 2021-09-21 15:37:49 I made a small migration script that updates the settings 2021-09-21 15:37:56 Nod 2021-09-21 15:40:20 I've added a test to the acceptance tests to see if we can clone a project 2021-09-21 15:40:32 (via the API) 2021-09-21 15:42:09 ikke: do we want to spin an equinix or linode for wg? 2021-09-21 15:43:10 equinix is bare metal? 2021-09-21 15:43:36 I think so 2021-09-21 15:43:44 Not sure about the small ones 2021-09-21 15:44:58 Would be strange to call your service bare metal 2021-09-21 15:55:21 I have no strong opinion about it 2021-09-21 15:59:20 Ok let's spin a simple linode 2021-09-21 15:59:38 And make it a spoke 2021-09-21 15:59:42 I can arrange that 2021-09-21 16:00:09 Can you or kunkku create a cert? 2021-09-21 16:00:15 I can 2021-09-21 16:00:24 Do we let mps arrange the rest? 2021-09-21 16:00:31 Yes 2021-09-21 16:00:36 ๐Ÿ‘ 2021-09-21 16:00:38 If he is ok with it 2021-09-21 16:00:58 He can ask us if needed about dmvpn 2021-09-21 16:01:02 I should create a netbox account for mps as well 2021-09-21 16:01:18 Good idea 2021-09-21 16:05:32 mps: do you already have ovpn access? 2021-09-21 16:10:02 just had to intervene on one linode which lost network to some area in england, so don't mention it again, it will crash :D 2021-09-21 16:10:38 ikke: yes, I have ovpn 2021-09-21 16:11:12 and cold coffee >. .< 2021-09-21 16:12:55 mps: on your x86_64 devbox, I've created a file called netbox-account.txt 2021-09-21 16:15:45 I see 2021-09-21 16:20:10 what is IP od netbox.alpin.pw 2021-09-21 16:20:32 172.16.14.1 2021-09-21 16:20:45 What ssh key should I use for you? 2021-09-21 16:20:50 one from gitlab/ 2021-09-21 16:20:52 ? 2021-09-21 16:21:07 yes 2021-09-21 16:21:30 do I have two for alpine (have to check) 2021-09-21 16:21:55 no, just 1 2021-09-21 16:22:26 ohm, I have another one for tpaste.us 2021-09-21 16:22:45 so, yes, this one from gitlab is ok 2021-09-21 16:38:26 I'm looking table there and I'm surprised how much items we have 2021-09-21 16:39:40 I'm scared 2021-09-21 16:40:02 ๐Ÿค 2021-09-21 16:42:53 mps: deu7-dev1.alpinelinux.org 2021-09-21 16:43:01 dmvpn has been setup, nothing else 2021-09-21 16:43:34 and all these you three ( ikke, kunkku and clandmeter) settled and manage 2021-09-21 16:44:37 let me first to 'come to self' :) 2021-09-21 16:50:56 note that there are also many 'virtual machines', ie lxc containers 2021-09-21 16:51:38 I noticed them 2021-09-21 16:51:58 https://netbox.alpin.pw/dcim/devices/35/ 2021-09-21 16:53:43 yes, already on it 2021-09-21 16:53:57 and looking 2021-09-21 16:54:44 Wireguard dmvpn bridge 2021-09-21 16:54:47 my advise for firewall would be installing awall-policies and then running setup-firewall 2021-09-21 16:55:57 can I login with ssh there 2021-09-21 16:56:03 you should 2021-09-21 16:56:05 root@ 2021-09-21 16:56:11 lets try 2021-09-21 16:59:40 yes, I'm there 2021-09-21 16:59:55 ok, good 2021-09-21 17:01:15 and I locked out self :( 2021-09-21 17:01:44 And I suppose me as well 2021-09-21 17:01:45 running setup-firewall 2021-09-21 17:01:49 hmm 2021-09-21 17:02:34 hmm, this shouldn't happen by running setup-firewall, or I don't know something 2021-09-21 17:02:47 you are right, it should not 2021-09-21 17:03:27 fixed 2021-09-21 17:03:34 awall activate adp-ssh-server 2021-09-21 17:03:40 ah 2021-09-21 17:03:41 awall enable adp-ssh-server 2021-09-21 17:05:02 ok, first thing is I have to look more deeply in awall 2021-09-21 17:06:26 mps: I also enabled dmvpn policy 2021-09-21 17:07:50 why there is gre interface 2021-09-21 17:07:56 that's part of dmvpn 2021-09-21 17:08:58 if you run ip route 2021-09-21 17:09:04 you see the routes to eache site 2021-09-21 17:09:24 ah, this is VM, not bare metal 2021-09-21 17:09:42 yes, linode provides vms 2021-09-21 17:10:34 aha, dmvpn uses gre, I see 2021-09-21 17:10:40 so 172.16.252.0/24 is the subnet for this 'site' 2021-09-21 17:14:04 so, we want to make this vm WG node where spokes connect and have routes to rest of infra? 2021-09-21 17:14:19 yes 2021-09-21 17:14:31 clients will get an ip in that subnet 2021-09-21 17:15:00 then they should be able to connect to other dmvpn sites 2021-09-21 17:15:41 at the same time this sounds complicated and simple ;) 2021-09-21 17:17:11 You should announce 172.16.0.0/16 as route to clients 2021-09-21 17:17:20 Not sure how that works with wg 2021-09-21 17:17:45 in routing table on 'node' 2021-09-21 17:18:16 so each client should make sure they add it? 2021-09-21 17:18:21 or can you push routes via wg? 2021-09-21 17:18:22 no 2021-09-21 17:18:45 client will have ip and netmask 2021-09-21 17:19:00 yes, but that only covers 172.16.252.0/24 2021-09-21 17:19:06 and list of allowed nets 2021-09-21 17:19:59 if we want such 'out-of-the-box' solution we should consider tailscale 2021-09-21 17:20:31 but I think we could solve this without tailscale (we will see) 2021-09-21 17:23:41 main problem is how to force client to use specific ip address for their local wg interface 2021-09-21 17:24:12 because this i mentioned tailscale 2021-09-21 17:26:51 no dhcp? 2021-09-21 17:29:07 hmm, why? 2021-09-21 17:31:07 our users are technical people and I hope for them is not complicated thing to set up wireguard client manually and generate keys 2021-09-21 18:26:00 kunkku: btw, setup-dmvpn worked on alpine 3.14 2021-09-21 18:26:39 It did show a message: "% Can't find BGP instance", but everything seems to work 2021-09-21 18:34:46 good 2021-09-21 18:54:45 ikke: I guess you setup dmvpn? 2021-09-21 18:54:50 clandmeter: yes 2021-09-21 18:55:21 clandmeter: I mentioned it to kunkku because I had issues on the new ppc64le host 2021-09-21 18:55:42 know I know that the BGP instance message is innocent 2021-09-21 18:55:42 Ok, maybe next time let mps do it, so we have one person more with what experience. 2021-09-21 18:56:03 Well, after you have the certificate it's just a matter of setup-dmvpn 2021-09-21 18:56:08 not much to it 2021-09-21 18:56:23 (apk add dmvpn) 2021-09-21 18:56:37 :) 2021-09-21 18:57:16 And because I was not sure whether it worked, I wanted to verify it 2021-09-21 18:57:33 mps: you can't share config with users like we do with ovpn? 2021-09-21 19:06:47 clandmeter: no 2021-09-21 19:11:12 Why not? 2021-09-21 19:11:54 PrivateKey should be left out? 2021-09-21 19:13:18 clandmeter: users must create they private/public keys combo 2021-09-21 19:13:48 ofc, we can create them for users but this would be on all security news 2021-09-21 19:14:44 wg is like ssh with respect to key management 2021-09-21 19:14:54 yes 2021-09-21 19:15:15 so not PKI based 2021-09-21 19:15:22 not 2021-09-21 19:16:02 wg is small, simple and secure :) 2021-09-21 19:16:03 so when your key is compromised... try to remember in which servers it was installed and start contacting admins... 2021-09-21 19:16:13 heh :D 2021-09-21 19:17:57 ikke: we need another ip range for wg net 2021-09-21 19:18:24 For what exactly? 2021-09-21 19:19:07 wg interfaces on users boxes 2021-09-21 19:19:40 or is it 172.16.252.0/24 2021-09-21 19:19:49 yes 2021-09-21 19:19:57 ah, ok 2021-09-21 19:20:26 does other boxes know where to route traffic for this net 2021-09-21 19:20:28 You probably need on IP on the host as a default gateway or something like that 2021-09-21 19:20:31 yes 2021-09-21 19:20:37 just like you can route to the other sites 2021-09-21 19:20:44 ok 2021-09-21 19:21:03 how we disable awall on this box 2021-09-21 19:22:17 i set 172.16.252.1 on 'hub' and 172.16.252.2 on my local box, ping works to hub but not on other boxes 2021-09-21 19:22:59 mps: easiest is just to change the policies of all change to ACCEPT 2021-09-21 19:23:04 chains* 2021-09-21 19:24:05 i see this with tcpdump '19:23:26.995519 IP 172.16.252.2.43137 > 172.16.23.110.60012: UDP, length 64' 2021-09-21 19:24:07 awall enable adp-ping ? 2021-09-21 19:24:21 but no answers 2021-09-21 19:24:49 kunkku: is already enabled 2021-09-21 19:24:50 kunkku: I can ping it from this new wg hub/router 2021-09-21 19:25:29 /proc/sys/net/ipv4/ip_forward ? 2021-09-21 19:25:33 is enabled 2021-09-21 19:25:42 I assume you mean /proc/sys/net/ipv4/conf/all/forwarding? 2021-09-21 19:26:07 yes 2021-09-21 19:26:10 ip_forward is enabled as well 2021-09-21 19:26:13 not last one 2021-09-21 19:26:18 lets see 2021-09-21 19:26:51 yes, also last one is enabled 2021-09-21 19:27:22 we access net over gre1? 2021-09-21 19:28:16 yes, dmvpn traffic goes there 2021-09-21 19:29:02 was the tcpdump from the dmvpn host? 2021-09-21 19:31:25 my bad, fixed 2021-09-21 19:31:48 mps: what was it? 2021-09-21 19:32:07 netmask wrong in wg allowip param 2021-09-21 19:32:12 ah 2021-09-21 19:32:42 PING 172.16.4.44 (172.16.4.44) 56(84) bytes of data 2021-09-21 19:32:48 over wg 2021-09-21 19:32:54 \o/ 2021-09-21 19:33:05 nice! 2021-09-21 19:36:56 Would be nice if some could help test https://gitlab-test.alpinelinux.org/. This restructured things in the back-end, so would be nice if we could verify everything is working. 2021-09-21 19:37:33 (e-mail notifications are disabled, ftr) 2021-09-21 19:37:50 and there are no runners pointing there, so testing CI would not work 2021-09-22 10:39:55 clandmeter: ikke: now 'we' know limitations of wg, do you think we should continue work on switch to it 2021-09-22 10:40:10 kunkku: also 2021-09-22 10:40:18 mps: can you summarize them? 2021-09-22 10:40:39 in short, yes 2021-09-22 10:41:53 no simple auto config but users have to set their 'side' mostly manually (we can provide recipe or short guide) 2021-09-22 10:43:03 on hub 'site' we have to manually add users keys and assing ip address for each user 2021-09-22 10:43:25 We could use netbox to keep track fo that 2021-09-22 10:43:27 of that* 2021-09-22 10:43:58 if we want that users use infra DNS we should give 'recipe' for them 2021-09-22 10:44:26 netbox have wg 'plugin'? 2021-09-22 10:44:56 and ikke have to setup awall on hub ;) 2021-09-22 10:46:06 these are drawbacks, but good part is with wg we could have fast and nice vpn 2021-09-22 10:47:26 if anyone wants to test I can post user side config 2021-09-22 10:49:41 mps: what part of awall still needs to be setup> 2021-09-22 10:50:10 I can ping 172.16.4.44 but can't ssh to it 2021-09-22 10:50:48 10:11:58.536925 IP 172.16.252.2.49220 > 172.16.4.44.22: Flags [S], seq 3455620662, win 64860, options [mss 1380,sackOK,TS val 3756136715 ecr 0,nop,wscale 7], length 0 2021-09-22 10:51:13 no answer to this, looked on hub with tcpdump 2021-09-22 10:53:43 If we have a document outlining how to set it up that's also fine I guess. 2021-09-22 10:55:33 clandmeter: are you still in Alps - alpinist :) 2021-09-22 10:56:29 'we' can write guide, sure 2021-09-22 10:59:29 I'm still in the mountains yes 2021-09-22 11:01:58 which part? Brenero, Bolzano. this parts are beautiful 2021-09-22 11:12:04 mps: looking at the awall config. Trying to figure out what the best way to describe this within the existing policies 2021-09-22 11:12:47 I'm in Tuscany 2021-09-22 11:12:47 I can just copy what clandmeter did for ovpn 2021-09-22 11:14:05 mps: I suppose you did not had to open anything to allow wg traffic? 2021-09-22 11:15:12 ikke: here is my client wg0.conf https://tpaste.us/REMW 2021-09-22 11:15:40 I set udp port 41414 for wg 2021-09-22 11:16:36 did you manually add that to iptables? 2021-09-22 11:16:38 ikke: ping works fine, don't know why ssh doesn't 2021-09-22 11:17:00 no, I didn't touched awall 2021-09-22 11:17:20 I see this in iptables: ACCEPT udp -- anywhere anywhere udp dpt:41414 2021-09-22 11:17:44 huh, maybe I did, sorry if I forgot 2021-09-22 11:18:41 oh yes, I did this 'iptables -A INPUT -t filter -p udp --dport 41414 -j ACCEPT' 2021-09-22 11:19:05 ok, np. Just need to add it to a policy 2021-09-22 11:19:17 that is needed for hub to accept wg tunnel 2021-09-22 11:19:58 understood 2021-09-22 11:20:33 my current topology is: 2021-09-22 11:21:08 mps: is wg still working now? 2021-09-22 11:21:15 check /etc/awall/wireguard.json 2021-09-22 11:21:42 my host <---> masquerading router/gateway <---> wg hub <---> 172.16.4.44 infra host 2021-09-22 11:22:01 yes, it works 2021-09-22 11:22:07 ok, can you test ssh? 2021-09-22 11:22:50 did it 2021-09-22 11:22:54 \o/ 2021-09-22 11:23:08 nice 2021-09-22 11:23:10 ikke: you are meister 2021-09-22 11:23:20 Well, I just copied what clandmeter did : 2021-09-22 11:23:22 :P 2021-09-22 11:24:57 now we have to play with wg setting routes to all infra hosts 2021-09-22 11:25:57 we can have 'one route to rule them all' or fine grained and different routes for different users 2021-09-22 11:27:25 for ovpn we just have one route 2021-09-22 11:27:30 172.16.0.0/16 2021-09-22 11:28:13 aha 2021-09-22 11:28:29 that's simpler to setup then 2021-09-22 11:29:24 ikke: your turn to make client setup for you 2021-09-22 11:30:29 will do that later 2021-09-22 11:31:01 ok, I hope you know this 'wg genkey | tee privatekey | wg pubkey > publickey' 2021-09-22 11:31:14 s/I hope/I'm sure/ 2021-09-22 11:31:33 I didn't, so thanks 2021-09-22 11:31:47 ah, ok 2021-09-22 12:36:50 https://tpaste.us/9Pn4 2021-09-22 12:37:14 this is starting point on writing guide 2021-09-22 12:52:48 mps, but this is only useful when using alpine? 2021-09-22 12:53:17 yes 2021-09-22 12:53:30 you are using something else? ;P 2021-09-22 12:53:37 im using mac 2021-09-22 12:53:41 sometimes windows 2021-09-22 12:53:46 sometimes gnome 2021-09-22 12:53:51 holly uhhh 2021-09-22 12:53:53 nm 2021-09-22 12:54:04 does mac already support wg? 2021-09-22 12:54:06 i rarely use alpine as client 2021-09-22 12:54:17 as desktop i mean 2021-09-22 12:54:18 someone else have to write guide for these other systems 2021-09-22 12:54:55 I can ask my son to try it on mac and if it works to add something about it 2021-09-22 12:55:29 mps: https://github.com/ifupdown-ng/ifupdown-ng/blob/main/executor-scripts/linux/wireguard 2021-09-22 12:55:35 there is a wireguard executor 2021-09-22 12:55:58 long ago I wrote short guide for my customer how to setup client on windows, have to look if I find it 2021-09-22 12:56:56 yes, I know that ifupdown-ng have it but didn't tried 2021-09-22 12:57:34 ikke: yes mac have wg for few years 2021-09-22 12:57:40 ah ok 2021-09-22 12:57:49 and android 2021-09-22 12:58:48 maybe ifupdown-ng author can help with this 'scenario' 2021-09-22 12:59:03 This would be nice as well: https://github.com/ifupdown-ng/ifupdown-ng/issues/42 2021-09-22 13:01:19 well, I still don't use ifupdown-ng, want to have simplest possible system 2021-09-22 13:03:57 if we use wg-quick routes can be set in wg0.conf file 2021-09-22 13:04:55 actually with wg-quick setting /etc/network/interfaces not needed, iirc 2021-09-22 13:08:28 i think all os types have wg 2021-09-22 13:08:36 but most are userspace 2021-09-22 13:08:46 yes 2021-09-22 13:08:54 except linux, freebsd, and windows in beta 2021-09-22 13:09:03 even alpine have wireguard-go pkg 2021-09-22 13:16:09 wg is not a dmvpn 2021-09-22 13:16:33 it's just a point-to-point connection 2021-09-22 13:20:03 of course, you could build something similar where wg replaces the ipsec+gre part 2021-09-22 13:21:33 For us, this is just to replace openvpn, not dmvpn 2021-09-22 13:35:43 I manage 6 servers around europe interconnected each with others (mesh) over wg, and few clients, works well for more than 3 years now 2021-09-22 13:39:50 When you add a server, would it mean you need to add connections to each other node in the mesh (and visa-versa)? 2021-09-22 13:43:28 yes, ofc 2021-09-22 13:44:10 every server have wg tunnel to all others 2021-09-22 13:44:25 That's what dmvpn makes easy 2021-09-22 13:44:26 tunnels 2021-09-22 13:45:37 well, yes, if I have 100 servers to interconnect my simple solution would be hard maintain 2021-09-22 13:54:06 another| on #wirefuard reminds me that networkmanager also has 'plugin' to handle wg 2021-09-22 13:54:31 and systemd-networkd 2021-09-22 13:54:58 maybe even connman, I forgot 2021-09-22 14:46:18 I updated 'guide' with wg-quick example https://tpaste.us/n5dv 2021-09-22 15:42:48 forgot to add, we can create qr-code for users with config and send them, but securely 2021-09-22 16:22:56 There are some tools that can manage configs, but I guess we don't need it? 2021-09-22 16:23:48 From what I read your cannot comment the config 2021-09-22 16:39:08 clandmeter: you mean put comments in config file? yes you can with '#' at beginning on line 2021-09-22 18:41:41 looking around I see this https://github.com/burghardt/easy-wg-quick 2021-09-22 18:41:55 shell script to create wg config 2021-09-22 19:15:02 what is the topic? replacing alpine's openvpn (or dmvpn...?) with wireguard? 2021-09-22 19:15:47 danieli: introducing wireguard, and eventually decomissioning ovpn 2021-09-22 19:16:01 dmvpn remains 2021-09-22 19:16:16 yeah, hub-and-spoke can't easily replace a mesh vpn 2021-09-22 19:16:34 danieli: dmvpn is also mesh iirc 2021-09-22 19:16:41 it is, dynamic multipoint vpn 2021-09-22 19:16:48 that's what i was aiming at 2021-09-22 19:16:54 yup 2021-09-22 19:16:59 not all traffic over the hub 2021-09-22 19:17:14 replacing openvpn is probably a decent idea, but QR codes and convenience wrapper scripts seems very overkill 2021-09-22 19:19:47 oh, I misunderstood, you mean wg would be hub-and-spoke? 2021-09-22 19:19:57 s/would be/is/ 2021-09-22 19:30:30 danieli: re QR code and scripts: agree, I just mentioned these if our infra 'chefs' want these 2021-09-22 19:30:59 wg is quite easy to setup for our developers, I think 2021-09-22 19:31:26 ikke: it's usually hub and spoke or point to point 2021-09-22 19:31:31 nod 2021-09-22 19:32:06 basically it is point-to-point 2021-09-22 19:32:52 but different and sometimes complex net can be created 2021-09-22 19:33:00 look at tailscale 2021-09-22 19:36:24 wg is basically a building block 2021-09-22 19:37:46 a wire (and guard) 2021-09-22 19:38:29 or guarded wire :) 2021-09-22 19:45:23 i'm not sure a more complex network is necessary though, i don't see what it would achieve here 2021-09-22 19:46:09 danieli: the goal is to remove ovpn from the hub where it lives now, and then we can replace it with wireguard while we change it 2021-09-22 19:46:28 it's pretty much 'just' the entrypoint so it shouldn't be too bad 2021-09-22 19:46:35 yes, exactly 2021-09-22 19:46:41 imo vanilla wireguard (server-client) is way, way easier than ovpn 2021-09-22 19:50:39 We just setup a server as a dmvpn site, and then connect clients to that site so that they can connect to dmvpn. That's all 2021-09-23 08:48:33 should we announce on #alpine-devel that we are ready for testing wg for those who want to test it 2021-09-23 08:51:38 I suppose we want to add a cname for it 2021-09-23 08:51:40 ? 2021-09-23 08:53:28 agree 2021-09-23 08:53:50 it would be cleaner than use IP 2021-09-23 17:07:27 mps: would we call the endpoint wg.a.o? 2021-09-23 17:07:39 The current is just ' 2021-09-23 17:07:44 'vpn' for openvpn 2021-09-23 17:08:19 however you like 2021-09-23 17:08:36 maybe clandmeter have something to say 2021-09-23 17:08:41 I'd like to ask for your opinion :P 2021-09-23 17:08:55 wg is quite fine for me 2021-09-23 17:09:45 and I preffer short names for hosts and domain, less to type 2021-09-23 17:13:27 mps: https://gitlab.alpinelinux.org/alpine/infra/linode-tf/-/merge_requests/10 2021-09-23 17:14:27 approved :) 2021-09-23 17:14:43 heh 2021-09-23 17:16:13 merged 2021-09-23 17:16:21 should be available soonโ„ข 2021-09-23 17:16:23 good 2021-09-23 17:16:45 did you added you in config 2021-09-23 17:16:51 Not yet 2021-09-23 17:17:02 Was first working on gitlab upgrade 2021-09-23 17:17:10 aha, ok 2021-09-23 17:18:42 I started with IP addresses 172.16.252.2/32, that is my current endpoint, but if you have something else in mind please change this 2021-09-23 17:18:55 hub IP is 172.16.252.1/32 2021-09-23 17:19:46 I don't think it matters that much 2021-09-23 17:20:23 well, we will have to keep IPs for devs somewhere 2021-09-23 17:20:46 netbox 2021-09-23 17:21:02 https://netbox.alpin.pw/ipam/prefixes/44/ip-addresses/ 2021-09-23 17:21:24 Do you want to add the wg0 interface to the host in netbox? 2021-09-23 17:22:47 is it needed 2021-09-23 17:22:59 Nothing is 'needed' 2021-09-23 17:23:04 :) 2021-09-23 17:23:06 but we can use it to hold the .1 address 2021-09-23 17:23:34 ok, this is on you, you manage netbox 2021-09-23 17:23:49 ok, np. Just wanted to give you the opportunity 2021-09-23 17:24:05 thank you 2021-09-23 17:25:05 Now it's registered: https://netbox.alpin.pw/ipam/prefixes/44/ip-addresses/ 2021-09-23 17:25:24 Now you could add your address to it if you want? 2021-09-23 17:25:28 Just as a reserved IP 2021-09-23 17:26:21 I assume with wg you need to create separate keys for each machine you want to connect? 2021-09-23 17:26:43 well, no in theory 2021-09-23 17:26:50 oh, ok 2021-09-23 17:27:03 but in practice we *must* do that 2021-09-23 17:27:12 for security reasons? 2021-09-23 17:27:38 actually users should create they key pairs and send us public key 2021-09-23 17:27:47 yes, that makes sense 2021-09-23 17:28:16 yes, would be bad that two different tunnels have same key 2021-09-23 17:30:41 https://netbox.alpin.pw/ipam/ip-addresses/223/ 2021-09-23 17:31:33 ah, thank you again 2021-09-23 17:33:57 This way we can keep track of IP addresses 2021-09-23 17:34:46 I didn't yet managed to understand all options/features of netbox 2021-09-23 17:35:08 if I ever will ;) 2021-09-23 17:36:04 wg.alpinelinux.org is alive in DNS 2021-09-23 17:36:12 ๐Ÿ‘ 2021-09-23 17:36:46 now we change 'guide' to it instead of IP 2021-09-23 17:37:01 we can* 2021-09-23 17:51:44 wg is ready. good excuse for one glass of red wine :) 2021-09-23 17:52:50 I just got white 2021-09-23 17:53:08 Reisling 2021-09-23 17:53:23 eh, enjoy anyway ;) 2021-09-23 17:54:10 I took Macedonian Kratishija 2021-09-23 17:54:14 Riesling ofc 2021-09-23 17:54:42 understand, we call it Rizling here 2021-09-23 17:55:52 Nice work on wg 2021-09-23 17:55:59 Thanks for helping out 2021-09-23 17:56:22 that's your idea 2021-09-23 17:56:48 You insisted 2021-09-23 17:56:56 :) 2021-09-23 17:57:52 should we 'announce' pubkey of hub (wg gateway) in 'guide' 2021-09-23 18:06:52 Maybe we could use https://gitlab.alpinelinux.org/alpine/infra/infra/-/wikis/home for some infra documentation? 2021-09-23 18:08:45 this sounds fine 2021-09-23 18:09:07 not sure how to use it, still. have to look 2021-09-23 18:10:10 mps: It's a git repository you can push to 2021-09-23 18:10:16 (I'm not so versed in these web gui systems) 2021-09-23 18:10:59 https://docs.gitlab.com/ee/user/project/wiki/ 2021-09-23 18:12:06 can I 'commit' first page there 2021-09-23 18:12:43 don't see option to make it 'Draft' 2021-09-23 18:13:40 There is no MR / draft feature 2021-09-23 18:13:45 you just push to a repo, and it's published 2021-09-23 18:14:03 I see 2021-09-23 18:29:03 Are you working on something for that wiki? 2021-09-23 18:29:21 yes, but from web ui 2021-09-23 18:29:32 Ok sure 2021-09-23 18:29:36 Then I'll wait 2021-09-23 18:30:09 should I put hub pubkey there? I think it is ok 2021-09-23 18:30:22 yeah, should not be an issue 2021-09-23 18:30:47 https://gitlab.alpinelinux.org/alpine/infra/infra/-/wikis/Alpine-wireguard-VPN 2021-09-23 18:31:23 first part 2021-09-23 18:32:06 having it there we all can add/change/fix it 2021-09-23 18:32:16 ๐Ÿ‘๐Ÿ‘ 2021-09-23 18:32:27 ok, next part 2021-09-23 18:38:16 ikke: you can reload it now 2021-09-23 18:40:18 We assign client ips on the 'server' side? 2021-09-23 18:40:47 Ah, I see 2021-09-23 18:40:50 address your_assigned_ip_address 2021-09-23 18:42:11 just beatified it little 2021-09-23 18:43:03 your_patch_to/wg0.conf -> your_path_to/wg0.conf 2021-09-23 18:43:13 ikke: on hub we strictly allow client to use one address AllowedIPs = 172.16.252.2/32, 172.16.0.0/16 2021-09-23 18:43:26 path, heh 2021-09-23 18:43:32 mps: ok, good 2021-09-23 18:43:48 but now you can edit and change it, I think 2021-09-23 18:45:17 yes, I can 2021-09-23 18:45:26 But I'm first working on the wiki homepage 2021-09-23 18:45:39 And also looking into setting it up myself 2021-09-23 18:46:43 so this is first wiki on gitlab.a.o, good excuse for yet another (uhm) glass 2021-09-23 19:09:04 mps: I wonder, should we add 172.16.0.0/16 to allowedips on the server side? 2021-09-23 19:09:27 I mean, it's there, but I wonder if it's correct 2021-09-23 19:10:40 mps: I don't expect traffic with 172.16.0.0/16 as source address, except the clients local wg address 2021-09-23 19:10:44 https://techoverflow.net/2021/07/09/what-does-wireguard-allowedips-actually-do/ 2021-09-23 19:22:58 ikke: it is some kind of 'auto' adding route in routing table 2021-09-23 19:23:37 ofc, we can add more of them in list for finer grained control 2021-09-23 19:24:12 What I understand is that it just checks what source address the traffic from the peer has 2021-09-23 19:24:21 and if it's not in that list, it rejects it 2021-09-23 19:25:08 first one '172.16.252.2/32' is to allow client to use only this 2021-09-23 19:25:33 yes, that is clear 2021-09-23 19:25:55 ah, understand now what you ask 2021-09-23 19:26:46 yes, we can use list with more fine grained network/netmask if we want 2021-09-23 19:27:05 What I mean is, I think we should only list the client IP, nothing more 2021-09-23 19:27:32 then we will have to use NAT, I think 2021-09-23 19:28:34 I don't see why 2021-09-23 19:28:40 could you close vim there, I can check right now 2021-09-23 19:28:42 ok 2021-09-23 19:28:52 done 2021-09-23 19:29:59 ah, yes it works 2021-09-23 19:30:13 look at conf file 2021-09-23 19:30:18 you mean that 2021-09-23 19:30:26 yes 2021-09-23 19:30:47 ok, let me test with ifupdown/ifup 2021-09-23 19:31:31 works 2021-09-23 19:32:21 and reverse ping works 2021-09-23 19:32:33 good that you looked to this 2021-09-23 19:33:00 And maybe we should add a comment to each peer to note for who it is 2021-09-23 19:33:03 I missed it, I wrote config from the head :) 2021-09-23 19:33:27 yes, I do this on my customer hub 2021-09-23 19:34:02 this is good idea 2021-09-23 19:34:37 look config now 2021-09-23 19:35:07 Yes, exactly 2021-09-23 19:35:33 or maybe above [Peer] look better 2021-09-23 19:36:27 i'm not sure, in [Peer], above or bottom of [Peer] section 2021-09-23 19:36:42 but wherever comments are it is ok 2021-09-23 19:36:46 indeed 2021-09-23 19:36:56 either is fine 2021-09-25 15:09:20 clandmeter: lol, they _again_ broke retrieving ssh keys from gitlab without being logged in 2021-09-25 15:09:23 https://gitlab.com/gitlab-org/gitlab/-/issues/340908 2021-09-25 16:19:02 Cz almost out of disk 2021-09-25 16:19:10 hmm 2021-09-25 16:19:45 Jirutka maybe will fix it 2021-09-25 16:19:53 145G left 2021-09-25 16:19:59 9% left 2021-09-25 16:20:09 clandmeter: btw, I'm ready to upgrade / migrate gitlab this weekend 2021-09-25 16:20:55 Ok, I'm onlmosthome 2021-09-25 16:21:07 I won't be doing it tonight 2021-09-25 16:21:10 so probably tomorrow 2021-09-26 12:53:31 ping 2021-09-26 13:15:33 pong 2021-09-26 14:40:07 Planning to upgrade gitlab tonight 2021-09-26 17:59:31 gitlab has been upgraded :). Small hick-up (wrong socket path) preventing pushes, but that has been fixed 2021-09-26 17:59:40 For the rest everything seems to run fine 2021-09-26 18:01:20 lol 2021-09-26 18:01:33 Not happy it did not receive any attention today? 2021-09-26 18:30:54 nice, so all works? 2021-09-26 18:31:03 I hope so :-) 2021-09-26 18:31:26 ๐Ÿคž 2021-09-26 18:36:00 clandmeter: fyi: I've switched over building the gitlab images to gitlab now 2021-09-26 18:36:05 building + publishing 2021-09-26 18:36:10 nod 2021-09-26 18:42:43 somebody stole my gitlab menu 2021-09-26 18:43:58 heh 2021-09-26 18:44:15 https://about.gitlab.com/releases/2021/06/22/gitlab-14-0-released/#streamlined-top-navigation-menu 2021-09-26 18:46:51 clandmeter: with dive I've noticed there are some things we can still remove from the gitlab image 2021-09-26 18:48:13 dive? 2021-09-26 18:48:42 tool to give insight in a docker image 2021-09-26 18:54:15 with every new release gitlab is more ugly, imo 2021-09-26 19:04:18 clandmeter: something to take into account for later: https://about.gitlab.com/releases/2021/09/22/gitlab-14-3-released/#legacy-database-configuration 2021-09-28 17:49:54 clandmeter: we need to clean up older build artifacts on gitlab 2021-09-28 18:05:40 clandmeter: https://tpaste.us/BgnR 2021-09-28 18:20:34 in the top 10 projects, there are 2411 builds we can clear the artifacts for 2021-09-28 18:20:40 (older than 6 months 2021-09-28 18:21:11 16k older then 1 month 2021-09-28 18:32:52 reference: https://docs.gitlab.com/ee/administration/job_artifacts.html 2021-09-28 19:14:16 100G available atm 2021-09-28 19:20:55 134G now 2021-09-29 06:39:15 ikke: nice 2021-09-29 06:39:21 i was looking at that previously 2021-09-29 06:39:34 so you changed a setting now? 2021-09-29 06:39:38 or its manually? 2021-09-29 07:14:08 clandmeter: this was done manually 2021-09-29 07:14:27 but part of it is/was a setting where it would not expire artifacts for 'latest' pipelines 2021-09-29 07:14:32 This is a setting you can enable per project 2021-09-29 07:14:55 There are still a few projects left that have quite a large artifact storage 2021-09-29 07:15:03 so we need to keep doing it manually? 2021-09-29 07:15:13 or by some cron script 2021-09-29 07:15:44 clandmeter: my hope is that now we enabled that setting, and we removed the artifacts that were not expiring, it would no longer be necessary 2021-09-29 10:24:13 zoneminder is failing to build on ppc64le 2021-09-29 10:24:33 [ 59%] Built target zmonvifproxy 2021-09-29 10:24:35 make: *** [Makefile:136: all] Error 2 2021-09-29 10:25:09 looks weird... is there an easy way to investigate further? 2021-09-29 10:25:21 kunkku: look further back 2021-09-29 10:25:26 ": error:" 2021-09-29 10:25:36 It complains about strerror not being found 2021-09-29 10:26:56 interesting that it happens on only 1 arch 2021-09-29 10:27:00 es 2021-09-29 10:27:03 yes* 2021-09-29 10:27:17 ok gotta check #include's 2021-09-29 10:27:29 I see the files including string.h, which contains char *strerror (int); 2021-09-29 10:28:16 Is _STRING_H already defined for some reason? 2021-09-29 10:33:11 how do we feel about packages running useradd / groupadd as part of "make install" and then running on our builders? do we have a policy on this? 2021-09-29 10:33:17 https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/25920 2021-09-29 10:34:39 Does that even work without root? 2021-09-29 10:35:01 exactly :-) 2021-09-29 10:35:24 So the build would fail if it tries to do that 2021-09-29 10:35:43 https://github.com/tpm2-software/tpm2-tss/blob/master/Makefile.am#L641 2021-09-29 10:36:33 it the docker builder in gitlab ci running as root? 2021-09-29 10:36:49 when building the package it self 2021-09-29 10:37:17 no, it does not run as root 2021-09-29 10:38:34 hmm but the pipeline is green and the maintainer states that it would not build without having the shadow package installed 2021-09-29 10:41:14 https://gitlab.alpinelinux.org/whooo/aports/-/jobs/500241#L1207 2021-09-29 10:41:25 WARNING Failed to create the tss user and group 2021-09-29 10:41:32 So it tries but it fails 2021-09-29 10:45:32 thats good 2021-09-29 12:42:07 ikke: I'm not listed as aports project member. https://gitlab.alpinelinux.org/alpine/aports/-/project_members could that be why some people thinks that I'm Michaล‚ Polaล„ski 2021-09-29 13:13:06 No idea. You are part of the 'Developers' group, so you are not listed as individual member 2021-09-29 13:13:17 But part is that you share initials 2021-09-29 13:27:52 hmm, probably people don't look carefully then :)