2025-01-02 12:15:38 ikke: mip 2025-01-02 12:16:20 oops, ikke: mips64 servers don't push mqtt msgs ? 2025-01-02 12:18:14 are there still mips64 servers? 2025-01-02 12:18:24 didnt the EOLed long time ago? 2025-01-02 12:18:46 they show in pkgs.a.o (drop down menu) 2025-01-02 12:22:05 yes that is because older releases had mips releases (https://gitlab.alpinelinux.org/alpine/infra/apkbrowser/-/issues/14) 2025-01-02 12:23:39 thanks 2025-01-02 15:21:56 how do I increase the timeout for a specific CI job? 2025-01-04 14:45:49 ddevault suggested emailing ~sircmpwn/sr.ht-support@lists.sr.ht with the source addresses of our arm builders, they may have been included in DDoS mitigations 2025-01-04 17:34:38 because confy upgrade being stuck on build-3-21-a* 2025-01-04 19:00:48 sorry, looks like i took dev.alpinelinux.org up to 100% on / -- doing some cleanup now 2025-01-04 19:03:29 back down to 73% 2025-01-04 20:49:53 fossd[m]: is your comment is related to our previous investigation we did here? (v6 connection issues to git.sr.ht?) 2025-01-04 20:50:41 if i just enable linode ip that connects when opening gitlab.a.o, it would be ok? 2025-01-04 20:52:08 found another domain that also connects, "ctrls.in" 2025-01-04 20:53:48 and [bc.]googleusercontent.com as usual 2025-01-04 21:11:37 clandmeter: how's the situation with alpine matrix server? 2025-01-04 21:12:09 im not sure, nu was working on it 2025-01-04 21:12:25 there were some msgs here regarding it recently 2025-01-04 21:16:12 nm, will disable them and see if it goes ok 2025-01-04 22:55:01 matrix server: coming at latest feb 15 2025-01-04 22:58:43 i need to load all the dendrite and postgres configs into my ram, and finish the automation 2025-01-04 22:59:05 i could use some help with the former 2025-01-05 01:41:04 do you need more ram? 2025-01-05 11:37:59 always 2025-01-05 17:18:57 its actually insane a messaging server needs that kind of hardware. i never hear these kind of issues on an ircd 2025-01-05 17:20:18 >into my ram< i meant my brain:> 2025-01-05 17:21:28 ping fossdd[m] about the arm builder ip block 2025-01-05 17:25:19 ftr my dendrite uses 350Mi atm 2025-01-05 17:32:36 dendrite is better than synapse 2025-01-05 17:32:40 but also they improved synapse 2025-01-05 17:33:03 also I would not recommend using dendrite 2025-01-05 17:33:38 they dropped ball on it since they are not running it themselves but synapse and all development went back into synapse 2025-01-05 17:37:53 have you got sources? 2025-01-05 17:44:29 "So the reason that Dendrite is stuck is that effort is going into Synapse instead; there’s not enough bandwidth to do both. The fact that third party homeservers don’t support migration seems orthogonal to whether Matrix 2.0 is actually usable right now - it sounds like you’re assuming you’d need to migrate from Synapse, which is not necessarily true." 2025-01-05 17:44:36 https://lobste.rs/s/obi5ar/matrix_2_0_is_here#c_2uzacs 2025-01-05 17:47:24 and "Synapse is also increasingly sprouting rust for the hot paths, fwiw. The effort which was going into Dendrite has ended up converging on improving Synapse for now." 2025-01-05 17:47:31 https://lobste.rs/s/obi5ar/matrix_2_0_is_here#c_3ivoi0 2025-01-05 17:49:18 I believe there is more but I'm not a fan of Matrix or arathorn so I'm going to stop at this :] but it can be all looked up on lobsters or HN comments 2025-01-05 19:37:15 ty for the pointers. now my question is what life support means. if it doesn't get the latest features but still has a set of reliable basic ones that make most things work well then i wouldnt yet jump the ship. 2025-01-05 19:40:16 that reminds me that i should check back on the db rot github issue that exploded the previous alpine homeserver :p 2025-01-05 19:44:42 afaik didn't get much traction 2025-01-06 08:24:12 testing something on pioneer 2 2025-01-06 08:33:15 btw, the 6.12 sophgo kernel boots in qemu 2025-01-06 16:11:29 i think we have a problem with build-3-18-armv7 2025-01-06 16:12:06 looks like last time it built anything was nov 2024 2025-01-06 16:14:32 ncopa: want me to look at it? 2025-01-06 16:14:40 would be nice, thanks 2025-01-06 16:14:51 maybe it is beacuse it uses ipv6 to msg.a.o? 2025-01-06 16:15:14 error: Unable to create '/home/buildozer/aports/.git/index.lock': File exists. 2025-01-06 16:15:19 git lock 2025-01-06 16:15:22 ok 2025-01-06 16:15:58 it's building now 2025-01-06 16:16:02 thank you! 2025-01-06 16:18:59 armv7 3.17 has the same issue 2025-01-06 16:19:30 armhf* 2025-01-06 16:24:04 ok 2025-01-06 16:26:13 looks like build-3-17-x86 as well? 2025-01-06 16:27:28 no index.lock file exists there 2025-01-06 16:28:30 https://build.alpinelinux.org/ show pulling git 2025-01-06 16:28:37 it doesnt matter 2025-01-06 16:28:40 its EOL anyway 2025-01-06 16:28:40 yes, the process is running 2025-01-06 16:50:10 did gitlab just break? 2025-01-06 16:55:29 It's working, although sometimes a bit sluggish for me 2025-01-06 17:01:09 The google AI crawler was making a lot of requests 2025-01-06 18:21:34 ACTION suggests returning http/402 for ai crawlers 2025-01-07 21:32:52 anywho, i have a mac runner i need to connect to the apk-tools project. can someone assist me with that process? i guess i need some sort of registration code 2025-01-07 21:35:30 Ariadne: yes 2025-01-07 21:35:45 Did you setup a shell runner? 2025-01-07 21:35:57 (or about to) 2025-01-07 21:36:01 yeah i can just install ssh keys for someone else if they want to take over 2025-01-07 21:36:11 or however we want to do this 2025-01-07 21:36:29 though the device is only accessible via tailscale 2025-01-07 21:36:46 It can connect outside? 2025-01-07 21:37:03 yeah, just inbound connections 2025-01-07 21:37:17 like to SSH into it, i would have to provision access to that specific device on my tailnet 2025-01-07 21:37:25 understood 2025-01-07 21:37:31 I can provide you a runner token 2025-01-07 21:37:39 works for me 2025-01-07 21:39:06 just DM me the token whenever you have it and i'll type it in :) 2025-01-07 21:39:47 The arch is aarch64? 2025-01-07 21:39:52 (for tags) 2025-01-07 21:40:16 yeah, though this is a macOS runner, so we probably want a special tag for that 2025-01-07 21:40:24 yeah 2025-01-07 21:42:44 (technically darwin calls it "arm64" instead of "aarch64", but i am not sure if that makes any impact here) 2025-01-07 21:42:51 not really 2025-01-07 21:43:54 I dm'ed you the token, I suppose you would setup a runner with a shell executor? 2025-01-07 21:44:07 what should i name this runner? 2025-01-07 21:44:47 I think you can ignore that, with the new registration flow, that information is alreayd provided in advance 2025-01-07 21:45:14 gitlab-runner registger --url https://gitlab.alpinelinux.org 2025-01-07 21:45:20 gitlab-runner registger --url https://gitlab.alpinelinux.org --token 2025-01-07 21:45:29 should be up 2025-01-07 21:46:04 yes 2025-01-07 21:46:05 though i am rebooting the host 2025-01-07 21:46:18 That's not an issue 2025-01-07 21:47:28 The next step is to add a job to run tests for macos 2025-01-07 21:47:49 I suppose it would make sense to make sure all the necessary dependencies are already present? 2025-01-07 21:48:05 yeah i'm going to work with aparcar to figure out next steps 2025-01-07 21:48:23 nod 2025-01-07 21:48:37 (what is the name of the runner/tags for it?) 2025-01-07 21:48:46 The runner is available in the alpine/apk-tools project. For forks, we would need to manually enable the runner for that fork 2025-01-07 21:48:56 macos-shell (tag) 2025-01-07 21:50:01 sounds good 2025-01-08 10:56:45 seems like git.a.o is not syncing 3.18-stable 3.19-stable? 2025-01-08 10:56:57 https://git.alpinelinux.org/aports/log/?h=3.18-stable 2025-01-08 11:20:35 It seems it is now? 2025-01-08 11:27:28 ok. maybe it was just slow 2025-01-08 14:09:39 gitlab is dying when trying to view history of APKBUILD 2025-01-08 15:47:05 ikke can you help me with a toot for the new stable releases? 2025-01-08 15:48:16 Yes, will take care of that in ~20 minutes 2025-01-08 16:11:48 ncopa: You didn't publish the release post yet? 2025-01-08 16:22:35 I think I did? https://alpinelinux.org/posts/Alpine-3.18.11-3.19.6-3.20.5-3.21.2-released.html 2025-01-08 16:47:48 yep, cloud image build picked that one up for its motd 2025-01-08 20:46:06 hugo 2025-01-08 20:46:09 whoops 2025-01-08 23:31:07 cgit cannot color display .initd codes? gitlab does it 2025-01-10 11:00:48 Hi, I've been directed to this channel, as I asked if a discourse forum may be an option for Alpine-linux? 2025-01-10 11:08:55 we tried a forum before, not sure alpine is the right env for it. 2025-01-10 11:11:38 but alot of distributions have discourse forums now and Alpine appears to be growing in popularity for general desktop use. Although I admit I'm still very new to it myself I have seen alot of comments on Mastodon about the lack of a support forum 2025-01-10 11:12:30 from my experience of forums most of my support on projects I use are from support forums. Alot of people dont use IRC these days. 2025-01-10 11:21:43 greylinux: are you willing to set it up and maintain it? and help moderate it? 2025-01-10 11:25:04 I would be more than happy to assist with moderation, but honestly although I would love to say I could set up and maintain it, my skills are not enough to even know where to begin. Is this not something that could be requested on say mastodon to see what the community might offer? 2025-01-10 11:26:08 maybe even a poll on mastodon to see the general consensus? 2025-01-10 11:29:22 consensus about what? 2025-01-10 11:29:53 whether the community would like/benefit from a forum 2025-01-10 11:30:20 Not sure how usefull that is 2025-01-10 11:30:35 Most people would agree it would nice to have a forum 2025-01-10 11:30:48 But how many are willing to contribute to providing support? 2025-01-10 11:31:12 thats the usual problem. lots of good opinions on what to do, but everyone expect us to do it 2025-01-10 11:31:26 and tbh, I am more than overloaded 2025-01-10 11:31:31 cannot keep up with everything 2025-01-10 11:31:47 right now I'm trygin to find out whey the 6.6.70 kernel does not build 2025-01-10 11:32:00 so I could do that, or I could work on setting up a forum of some sort 2025-01-10 11:32:25 traditionally we have preferd self hosted, on alpine machines 2025-01-10 11:32:40 but i'm kinda ok to use something hosted by someone else 2025-01-10 11:32:48 then there are things like reddit and stack overflow 2025-01-10 11:33:20 from my experience on forums the community help each other with problems, I'm not saying I think you should setup the forum and manage it, its quite an ask to do what you are doing, but maybe the community would provide? 2025-01-10 11:33:22 but I tend to agree that it would be useful with some sort of forum 2025-01-10 11:33:55 i need to go out now. and get some sun before it gets dark 2025-01-10 11:34:05 have a nice weekend everyone! 2025-01-10 11:34:09 I personnal think reddit is on its way out 2025-01-10 11:34:16 have a good day ncopa 2025-01-10 11:34:38 What I dislike about discourse is that the only official way to deploy it is to use their pre-built docker solution 2025-01-10 11:35:14 thats fair, I only have experience using Discourse but I have seen others. 2025-01-10 11:35:18 Which is akin to a binary distribution 2025-01-10 11:36:24 What ncopa mentioned is that we strongly prefer to deploy everything on alpine (dogfood our own distro) 2025-01-10 11:37:07 Thats fair I would be the same, but at least you may find moderators etc from the community ? 2025-01-10 11:38:34 If the request was put on Mastodon or where ever really, we can see if the community would be willing to step and help. 2025-01-10 11:59:39 I leave the channel and let you get back to enjoying your day. If you need to contact me about anything i'm on this handle on mastodon. 2025-01-10 13:57:08 ikke: crazy idea. what if we set up a small kubernetes cluster on linode (1 controller, 2 or 3 workers) and move gitlab to that? then maybe we could have multiple gitaly instances? Or at least separate out gitaly to a separate node 2025-01-10 15:21:29 I was wondering if there were any plans to get off gitlab at some point 2025-01-10 15:27:49 but to what? 2025-01-10 15:28:07 i really like forgejo, but it's kinda lightweight and not really suitable for alpine's purposes i think 2025-01-10 15:42:27 there are no plans to leave gitlab 2025-01-10 15:43:33 anyone suggesting this does not understand the scope of it and availability of manpower to make it happen 2025-01-10 16:39:56 yeah true that as well 2025-01-10 16:42:55 "but to what?" <- nothing currently, because nothing really supports stuff that gitlab.a.o is using 2025-01-10 16:42:55 although gitea/forgejo will have features that would make it possible in future 2025-01-10 16:43:21 yes 2025-01-10 16:44:05 given that fedora is going to improve forgejo, this might be sooner than later 2025-01-10 18:16:35 what would be the benefit? the cost of switching is high 2025-01-10 18:18:09 having a maintainer that is vested in open-source (gitlab is looking for a buyer) 2025-01-10 18:19:31 WARNING: this is alpha release quality code and should not be considered secure enough to deploy in production. 2025-01-10 18:19:38 https://code.forgejo.org/forgejo/runner#forgejo-runner 2025-01-10 18:34:07 clandmeter: ^' 2025-01-10 19:04:57 Looks like connectivity issue 2025-01-10 19:21:20 ncopa: I'm afraid it's not just a matter of running multiple gitaly instances 2025-01-10 19:23:50 It at least also involves deploying praefect 2025-01-11 08:07:33 does anything depend on che-bld-1's v6 address, or could it be simply changed behind the dns? (the move might introduce a new v6 prefix) 2025-01-11 08:10:34 Except for documentation, I don't believe so 2025-01-11 08:10:54 No services run their that things are actively connecting to 2025-01-11 08:11:01 there* 2025-01-11 08:11:56 che-bld-1.alpinelinux.org points to the current ipv6 address 2025-01-11 08:13:35 Oh, I see, there are potentially some developer containers that are directly connected to by their ipv6 address 2025-01-11 08:49:14 ill try to keep the prefix then 2025-01-11 08:51:41 would be nice if e.g. podman could generate and update dns entries for each container 2025-01-11 08:52:46 or something external that scrapes container runners 2025-01-11 08:54:58 These are lxc containers 2025-01-11 08:55:16 nu_: I don't think it's a big deal to change the prefix 2025-01-11 08:55:52 for ipv4, we use dnsmasq to automatically resolve container names (but for an internal dns resolver that requires vpn access) 2025-01-11 08:56:17 v4 is not an issue because it's hidden behind nat 2025-01-11 08:57:23 maybe ill start to nat your v6 to v6 2025-01-11 13:39:54 i think we can setup a comment part more than a forum. for example, each package have its own comment section 2025-01-11 19:03:12 qaqland: Not all issues are tied to a single package though 2025-01-11 19:03:39 And if it's about a specific package, an issue in gitlab would work as well 2025-01-12 09:08:26 for the record the v6 natting idea was a joke 2025-01-12 09:09:03 :) 2025-01-12 09:09:25 Just update the prefix, we'll handle the rest 2025-01-12 09:10:46 thanks, i was just planning a bit how big fight i should put up with my provider 2025-01-12 12:08:42 https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/78017#note_470981 2025-01-12 13:45:19 ikke: thanks 2025-01-14 11:28:19 ncopa, ikke, not a single nvme issue with the samsung nvme on pioneer2 2025-01-14 11:28:36 theyve been stable since it seems 2025-01-14 11:29:16 clandmeter: do you still see them on pioneer1? 2025-01-14 11:29:27 nope 2025-01-14 11:29:52 so it could be coincidence 2025-01-14 11:29:53 Does that have the original nvme or also a samsung nvme? 2025-01-14 11:29:55 ok 2025-01-14 11:30:17 only 2 has a samsung one 2025-01-14 11:30:25 right 2025-01-14 11:30:32 its also smaller 2025-01-14 11:30:37 its an older gen 2025-01-14 11:31:38 if its really more stable we could try another nvme 2025-01-14 12:08:57 sounds good 2025-01-14 12:09:33 what do we need? 1TB? 2025-01-14 12:12:35 probably 2025-01-14 13:03:57 we need one or two? 2025-01-14 13:23:30 if you want to upgrade the 500G one, then its 2 2025-01-14 13:47:45 i think I'll ask on fediverse if someone wants to donate one or two high quality nvme SSDs 2025-01-14 15:20:20 m.2 pls 2025-01-14 15:20:36 high quality could be u.2 2025-01-14 16:26:55 clandmeter: are you on fedivers? 2025-01-14 16:37:38 https://fosstodon.org/@ncopa/113827653828892884 2025-01-14 16:40:14 ncopa: maybe also specify where they are needed? (for import / shipping purposes) 2025-01-14 16:42:00 👍 2025-01-14 18:45:14 ikke: this is where to make changes for git.a.o ? https://gitlab.alpinelinux.org/alpine/infra/compose/cgit 2025-01-14 18:45:46 yes, but sadly there have been made quite some changes on the server that have not been reconciled in this repo 2025-01-14 18:46:02 yes, i was guessing it 2025-01-14 18:47:50 could u add this on top on unknowns, https://tpaste.us/xDav 2025-01-14 18:48:52 its for /usr/local/bin/highlight.sh 2025-01-14 18:50:48 AI's have not overtaken parts of it, right :) 2025-01-14 18:50:58 nope 2025-01-14 18:51:23 The highlighter was switched to a python implementation, but I switched it back due to issues 2025-01-14 18:53:06 python takes some more resources 2025-01-14 18:53:58 iirc, kernel.a.o uses older version of hightlighter and can handle large files 2025-01-14 18:54:25 someone complained about it, few days back 2025-01-14 18:54:44 The main reason was to support dark themese 2025-01-14 18:54:47 themes 2025-01-14 18:54:52 for the python implementation 2025-01-14 18:56:20 highlighter has themes too 2025-01-14 18:57:31 It's not about being able to switch themes statically, but that it can adopt to preferring dark themes 2025-01-14 18:58:07 '@media only all and (prefers-color-scheme: dark) {' 2025-01-14 18:58:13 did some changes, exec highlight --inline-css --force -f -I -O xhtml -S "$EXTENSION" 2>/dev/null 2025-01-14 18:58:36 For this you don't want inline-css 2025-01-14 18:58:41 ok 2025-01-14 18:58:59 You want css classes so that you can override the style in css 2025-01-14 18:59:18 view-source:https://git.alpinelinux.org/cgit.css 2025-01-14 19:00:03 That part's broken now due to highlight not using classes 2025-01-14 19:04:08 ok, would prefer highlight for better perf 2025-01-14 19:05:12 thanks, for enabling it in .initd 2025-01-14 19:08:59 will look at class bug, think i got it working couple of yrs back, don't recall now (v3.14 maybe) 2025-01-14 19:11:54 It would still require updating the stylesheet to match the classes that highlight would use 2025-01-14 19:12:06 I do agree about performance 2025-01-14 23:38:59 When I try to git clone from git.alpinelinux.org (not gitlab) I get a 500. Do other people have the same issue? 2025-01-15 00:43:18 git pull works, if in urgent need, clone from other sites and change origin->url 2025-01-15 08:04:02 sertonix[m]: do you have url you are trying to clone? 2025-01-15 08:04:26 ah got it 2025-01-15 08:11:36 heh 2025-01-15 08:11:49 ikke made so many local changes its hard to see what caused it 2025-01-15 08:19:23 Not me 😞 2025-01-15 10:02:22 oh, maybe i did it ones myself, but i dont remember. 2025-01-15 10:02:55 anyways, nginx does not show anything, so i assume its traefik thats generating the 500 2025-01-16 02:56:16 ncopa clandmeter would you guys consider setting up with https://opencollective.com/ ? It would perfectly suit the use case of clandmeter needing new hardware. You would just make the purchase personally and submit the invoice for the records in OC and get reimbursed from funds that supporters have contributed. linuxserver.io would be quick to jump 2025-01-16 02:56:16 on making financial contributions there. Can also submit invoices for infra costs like hosting, or just for time spent contributing. You would admin the distribution of funds for your project so it's not like the platform decides what you can use the funds for. They just handle record keeping mostly for tax purposes. 2025-01-16 09:05:37 iXNyNe: yes 2025-01-16 10:44:34 I got offer to become rsync maintainer.... https://github.com/RsyncProject/rsync/pull/705#pullrequestreview-2553701442 2025-01-16 10:53:45 thanks for the fast reaction btw 👍 2025-01-16 14:07:54 ncopa: you are becoming the most influencing oss personality of 2025 :) 2025-01-17 21:26:08 is the edge armv7 builder under maintenance? 2025-01-17 21:27:58 seems to be showing as inactive on build.a.o 2025-01-17 22:50:17 mio: no planned maintenance 2025-01-17 22:54:00 okay, thanks 2025-01-17 22:56:30 the other arms are showing up as idle from my end 2025-01-17 22:57:08 thought they would be going through the go rebuild as well 2025-01-17 23:20:24 i think kodi is failing repeatedly 2025-01-18 15:18:00 (edge) s390x might be running low on space, "go: failed to trim cache: write /home/buildozer/aports/testing/adguardhome/src/go-cache/trim.txt: no space left on device" for adguardhome rebuild 2025-01-19 11:57:42 so armv7 and x86_64 builders are stuck? 2025-01-19 14:31:45 sorry for the downtime. the uplink is trolling:/ 2025-01-19 14:35:05 connection was terminated 2 days before the new one became active, and then they forgot to configure prefix delegation 2025-01-19 16:25:09 nu_: thanks 2025-01-19 16:27:15 nu_: did you receive any ETA? 2025-01-19 19:26:23 do u use the prefix i gave you to route traffic inside? if you only do egress then the connectivity is there 2025-01-19 19:27:32 For the builders to function, only egress is required 2025-01-19 19:27:51 The builders do get an IP assigned from the prefix they receive (through corerad) 2025-01-19 19:28:28 great, then ill give u an address to access it and a ULA v6 range 2025-01-19 19:32:48 Does that include ipv4 egress (through nat)? 2025-01-19 19:33:53 Sadly github for example still only supports ipv4 2025-01-19 19:34:22 yes 2025-01-19 19:37:07 ok, cool 2025-01-19 19:51:20 omni: i don't see the x86_64 builders being stuck 2025-01-20 19:22:37 is aws-lc still building on edge aarch64? 2025-01-20 19:23:10 just wondered, seems to be there for some time 2025-01-20 20:09:37 checking 2025-01-20 20:20:12 The HTTP 500 error when cloning from git.a.o has been reported on gitlab aswell: #16859 2025-01-20 20:20:20 yes, I saw 2025-01-20 20:30:35 fatal: detected dubious ownership in repository at '/var/git/aports.git' 2025-01-20 20:39:30 ok, fixed 2025-01-20 21:01:59 ikke: thanks 2025-01-21 08:01:09 GitLab is giving: 2025-01-21 08:01:09 remote: Internal API unreachable 2025-01-21 08:01:12 when pushing 2025-01-21 08:02:15 ah, seems there's more stuff broken 2025-01-21 08:07:30 ikke: any idea what's going on? 2025-01-21 08:21:43 fabled: no, I'd have to check 2025-01-21 08:22:00 ikke: seems it got fixed just now 2025-01-21 08:22:12 corresponds with the above notifications from deu2-dev1 2025-01-21 08:22:38 I was looking into rate limiting 2025-01-21 08:22:49 Lots of scrapers nowadays 2025-01-21 09:05:40 ikke: broken again 2025-01-21 09:05:45 ah, seems https is broke too 2025-01-21 09:05:48 git push gives: 2025-01-21 09:05:49 remote: rpc error: code = Unavailable desc = The git server, Gitaly, is not available at this time. Please contact your administrator. 2025-01-21 09:08:43 web UI seems unreachable too 2025-01-21 10:31:20 gitlab server is overloaded 2025-01-21 10:32:07 htop is even not responding, kernel is crashing 2025-01-21 10:34:15 ikke: im going to stop gitlab for now 2025-01-21 10:34:29 if it even wants to still 2025-01-21 10:34:40 no resources to do anything 2025-01-21 10:37:38 Yeah, stopping gitlab usually helps in that case 2025-01-21 10:43:04 starting again 2025-01-21 10:43:18 Thanks 2025-01-21 10:43:23 it's coming up 2025-01-21 10:43:28 what was the issue? 2025-01-21 10:43:31 memory leak? 2025-01-21 10:44:30 seems to be back up 2025-01-21 10:46:51 Thank you much appreciated 2025-01-21 10:48:33 zacksiri: not sure if it's the case now, but I've noticed that if the load on gitaly / io goes beyond a certain point, it has a hard time recovering 2025-01-21 10:48:36 be gentle pls 2025-01-21 10:48:47 Lots of git processes never finishing 2025-01-21 10:48:50 yes 2025-01-21 10:48:54 thats what i also notice 2025-01-21 10:49:03 all these git processes are not finishing 2025-01-21 10:49:35 you are hosted on cloud 2025-01-21 10:49:39 or bare metal? 2025-01-21 10:50:29 cloud 2025-01-21 10:50:49 i see 2025-01-21 10:51:18 its a linode, but its having resource issues lately 2025-01-21 10:51:23 i see 2025-01-21 10:51:41 any way we can offload some of the workload? 2025-01-21 10:51:48 make it more risilient 2025-01-21 10:51:56 the project is getting close to its limits of the resources we have. 2025-01-21 10:52:13 i see 2025-01-21 10:52:46 forgive me for asking but why don't you guys move to something like github hosted 2025-01-21 10:52:46 the cost of getting popular without a $$$ behind it. 2025-01-21 10:52:55 or even gitlab 2025-01-21 10:52:57 hosted 2025-01-21 10:53:02 zacksiri: we moved away 2025-01-21 10:53:24 having your own gitlab is more flexible 2025-01-21 10:53:29 i see 2025-01-21 10:53:36 To be honest, I think it's these AI scrapers causing a lot of load 2025-01-21 10:53:38 and we are a distro, we need to run it on alpine :) 2025-01-21 10:53:55 clandmeter: absolutely 2025-01-21 10:54:39 we are working on finding solutions. 2025-01-21 10:54:47 but we only have so much time 2025-01-21 10:55:22 ikke: should we tune some timeout so the git processes would clear up faster 2025-01-21 11:07:03 disable public access to gitlab? make all public stuff go through git.a.o and ratelimit it 2025-01-21 11:11:49 or you could run https://github.com/Xe/x/tree/master/cmd/anubis 2025-01-21 12:06:12 ikke: could blocking by ASN be feasible? Or maybe a proof of work proxy in front of the gitlab instance? 2025-01-21 12:06:27 durrendal: I' 2025-01-21 12:06:33 I've always doing something by ASN 2025-01-21 12:06:41 not for gitlab yet, but for git.a.o 2025-01-21 12:06:51 but some handpicked ASNs, not yet on a larger scale 2025-01-21 12:07:07 nginx has a geo feature that makes it easy to map IP addresses to ASNs 2025-01-21 12:07:41 I also found a publically available TSV files with prefixes per ASN 2025-01-21 12:13:24 Great minds think alike, maybe just adding a couple of ASNs from repeated offenders would be worthwhile while a more automated solution is found. 2025-01-21 12:13:32 ahuh 2025-01-21 12:13:52 Gitlab is behind traefik though? Is the topology nignx -> traefik -> gitlab? 2025-01-21 12:14:04 traefik -> nginx -> gitlab 2025-01-21 12:15:09 We use traefik is dynamic docker router 2025-01-21 12:15:26 gitlab itself uses nginx by default 2025-01-21 12:17:07 Ah that makes sense 2025-01-21 12:17:11 https://plugins.traefik.io/plugins/6294734d108ecc83915d778c/max-mind-geo-ip-2 2025-01-21 12:17:34 Looks like there's a traefik plugin as well, but it sounds like that would affect more than just gitlab 2025-01-21 12:17:40 Might not be ideal. 2025-01-21 12:31:16 funny, google bot reports itself to be a nexus 5x 2025-01-21 12:39:48 they use such old phones? :) 2025-01-21 12:41:43 i think i still have one in my draw, maybe we could use it as revproxy to talk to theirs 2025-01-21 14:15:37 folks, don't throw away old mobiles, send it to me, if possible :) 2025-01-21 14:17:01 or to ncopa 2025-01-21 16:29:57 maybe dumb question, but is there robots.txt setup with all the "piss off AI robots" lines? I know OpenAI (at least) has their own robots.txt lines they look for 2025-01-21 16:30:10 Yes, I have added it 2025-01-21 16:31:00 I need to do it more durable, apparently it's gone now 2025-01-21 17:07:28 it's not respected by many 2025-01-21 17:56:03 iggy: suggest again having a block for known crawlers and for those, return 402 2025-01-21 17:56:59 like bytedance and such especially 2025-01-21 18:20:29 this seems nice, https://commoncrawl.org/overview, if on spree to - block`em all 2025-01-21 19:54:02 it looks like git.a.o isn't in sync with gitlab.a.o anymore. Around 27 hours out of date 2025-01-21 20:23:52 sertonix[m]: hmm, due to the change yesterday. 2025-01-21 20:38:50 Issue is that 2 containers use different UIDs for the same user 2025-01-21 20:45:28 sertonix[m]: thanks for reporting, fixed now 2025-01-21 20:45:57 \o/ 2025-01-21 20:48:04 hmm 2025-01-22 03:59:44 Is there something new that blocks connections to gitlab.a.o? i can ping it but i can't access it, tried both web and ssh, the connection just gets stuck and eventually times out 2025-01-22 04:00:50 i can open the web here 2025-01-22 04:15:39 Yeah, something is probably blocking my connection, while still allowing pings 2025-01-22 05:53:45 cely: can you try again? 2025-01-22 06:06:43 It's working now, thanks 2025-01-22 06:06:51 cely: sorry about that 2025-01-22 06:10:00 I checked gitlab(gitaly)'s code, it use git-cmd internally 2025-01-22 06:10:14 Facing large repositories with many files like aports, the speed become slow. 2025-01-22 06:14:24 No problem 2025-01-22 15:54:38 ... 2025-01-22 15:54:43 Make it stop 2025-01-22 16:57:18 FYI, I've applied a rate limit per IP on gitlab (for non-asset requests). It seems to currently mostly hit google bot (mostly delaying) 2025-01-22 16:58:10 Next step is to do it by ASN 2025-01-22 16:58:25 delaying request, excess: 7.378, by zone "ip", client: 66.249.66.5 2025-01-22 17:00:28