2025-11-01 02:05:31 <anjan> Hi, can someone review: https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/91543
2025-11-01 05:37:11 <craftyguy[m]> sertonix: for ruff on 64K page systems... any reason why we just don't build it with JEMALLOC_SYS_WITH_LG_PAGE=16 so the vendored jemalloc works?
2025-11-01 10:43:43 <f_> omni: are you around for a little bit? I'd like to take over meli maintainership, is that ok to you?
2025-11-01 10:44:29 <f_> the package is a little bit out of date and seeing that you're busy on so many other things at the same time I think that would be less work for you
2025-11-01 10:44:53 <f_> (and I also need a reason to maintain packages in alpine, lol)
2025-11-01 10:56:45 <achill> fwiw there is !91726
2025-11-01 10:57:18 <f_> oh, cool, but I still want to takeover if that's ok
2025-11-01 11:06:43 <omni> no
2025-11-01 11:09:48 <f_> omni: then I still want to help out
2025-11-01 11:10:06 <f_> if I can help out in any way
2025-11-01 11:23:02 <f_> so if there's anything I can do to help feel free to let me know
2025-11-01 11:23:40 <f_> (rereading my messages, I guess it might've seemed a bit harsh, I didn't mean it to be)
2025-11-01 11:34:59 <omni> the only thing that is out-of-date is the version number, functionally it is up-to-date with the latest release
2025-11-01 11:40:45 <f_> oh, right
2025-11-01 11:41:02 <f_> I think there should be a new release coming soon™ also
2025-11-01 12:12:24 <raspbeguy> Hello, can someone merge those MRs please? !87884 !90941 !91118 !91337 !91546
2025-11-01 12:37:52 <Sertonix[m]> craftyguy: All advantages of devendoring apply: patch without rebuilding everything and duplicating the patches, avoid wasting cpu-cycles while compiling, pobably reduce size, faster security upgrades
2025-11-01 12:40:55 <Sertonix[m]> So I would question if there is any good reason to use the vendored jemalloc
2025-11-01 12:45:43 <omni> and rather give it the same treatment as fd just got !92264
2025-11-01 16:05:45 <WhyNotHugo> I have a list of 130 packages that need to be built, and need to do a topological sort on the dependency graph. Do we have any tool that does this already?
2025-11-01 16:05:51 <WhyNotHugo> I'm using APKBUILD as source here, not built apk
2025-11-01 16:23:40 <craftyguy[m]> sertonix: because it's literally a 1 line change in the apkbuild to fix that bug if we keep using the vendored jemalloc :P
2025-11-01 16:24:40 <craftyguy[m]> by de-vendoring don't we risk introducing possible bugs where maybe the app is incompatible somehow with a newer version we package?
2025-11-01 16:27:16 <achill> WhyNotHugo: lua-aports has some scrips that generate a package list
2025-11-01 16:27:51 <achill> https://gitlab.alpinelinux.org/alpine/lua-aports/-/blob/master/bin/ap.lua?ref_type=heads#L63
2025-11-01 16:28:59 <WhyNotHugo> achill: thansk!
2025-11-01 16:37:11 <WhyNotHugo> lua-aports only works intra-repository, but can't find dependants in community for a package in main?
2025-11-01 16:54:34 <WhyNotHugo> Passing multiple `-d` flags to `ap` handles them separately, so won't find cross-repo dependencies. I take it that patching this should not break any usage scenario?
2025-11-01 17:09:04 <WhyNotHugo> Fixed it: https://gitlab.alpinelinux.org/alpine/lua-aports/-/merge_requests/19
2025-11-01 17:14:32 <WhyNotHugo> Here I was writing lots of horrible code when  `ap recursive-revdeps python3 | xargs ap builddirs`  does exactly what I needed!
2025-11-01 22:14:45 <Ariadne> Sertonix[m]: we can't ship fd with musl malloc-ng, it aggressively does multithread scatter-gather work, which basically kills malloc performance
2025-11-01 22:15:21 <Ariadne> 
ACTION does not care if we ship vendored jemalloc or unvendored jemalloc, but simply changing to musl malloc isn't going to yield acceptable performance

2025-11-01 22:16:56 <Sertonix[m]> It's already using unvendored jemalloc
2025-11-01 22:17:10 <Ariadne> oh sorry, i thought we were talking about fd, not ruff
2025-11-01 22:17:23 <Ariadne> regardless, switching to musl malloc is almost always a mistake with rust programs
2025-11-01 23:38:40 <tokyovigilante> hi, I've just updated the Nim package to 2.2.6, would appreciate a review - https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/76837
2025-11-03 00:39:56 <angeloverlain[m]> would be nice to have another set of eyes on https://gitlab.alpinelinux.org/alpine/mkinitfs/-/merge_requests/206. maybe ncopa 👀
2025-11-03 07:35:34 <prabu> thanks ikke for merging !90460
2025-11-03 14:55:09 <WhyNotHugo> angeloverlain[m]: can we potentially move regular LUKS into hooks too?
2025-11-03 19:39:31 <realroot[m]> is it already possible to use unl0kr? I guess that it's better wait for mkinitfs hooks MR though
2025-11-03 21:46:01 <WhyNotHugo> Sertonix[m]: have you considered inserting the LUKS password into the kernel keyring and then using cryptsetup with --volume-key-keyring ?
2025-11-03 21:56:29 <Sertonix[m]> I haven't but I also didn't have the time to work on a (in my opinion) good mkinitfs flexibility improvment. Big question would be if it can supersede all previous methoda. If it can that would be great. But without looking into the details I doubt one can implement a good retry functionality with it.
2025-11-03 21:59:26 <angeloverlain[m]> WhyNotHugo: yes. we can potentially move that and even LVM into hooks 
2025-11-03 21:59:56 <angeloverlain[m]> <realroot[m]> "is it already possible to use..." <- I don’t think it’s currently possible 
2025-11-03 22:00:59 <WhyNotHugo> Retrying is the tricky part, because it needs to re-exec the prompter.
2025-11-03 22:02:00 <Sertonix[m]> angeloverlain[m]: It is. You can use the (arguably not very good) CRYPTKEY=EXEC= or overwrite the init script which is a documented mkinitfs option.
2025-11-03 22:03:19 <Sertonix[m]> WhyNotHugo: The tricky part is that it depends on the promper whether or not it should be re-executed or signaled to a still running process (eg. unl0kr).
2025-11-03 22:07:23 <angeloverlain[m]> <Sertonix[m]> "It is. You can use the (arguably..." <- ohh nice. I forget about that. the nice thing with hooks is that you can have TPM automatically try to unlock the disk, and when that fails, you can then launch unl0kr to try decrypt it. not sure if that’s currently possible unless you write a script 
2025-11-03 22:08:22 <angeloverlain[m]> but yeah hooks might also allow the initramfs to be more modular (the file systems still want to be in nlplug-findfs tho for now).
2025-11-03 22:08:51 <angeloverlain[m]> the most exciting use case for me is actually having plymouth
2025-11-03 22:12:15 <Sertonix[m]> angeloverlain[m]: That's very much a use case for setting init= in /etc/mkinitfs/mkinitfs.conf instead.
2025-11-03 22:14:36 <WhyNotHugo> Sertonix[m]: you're suggesting different providers for initramfs-init?
2025-11-03 22:25:57 <angeloverlain[m]> <Sertonix[m]> "That's very much a use case..." <- can you expand on that ?
2025-11-03 22:28:04 <Sertonix[m]> WhyNotHugo: Not really. I think that power users are better off overwrite that file and for everything else mkinitfs!206 is a footgun.
2025-11-03 22:29:32 <Sertonix[m]> algibot: Not quite
2025-11-03 22:30:06 <angeloverlain[m]> Sertonix[m]: how does that work when you set init= ?
2025-11-03 22:30:30 <angeloverlain[m]> this is how arch’s plymouth hook looks like btw https://gitlab.archlinux.org/archlinux/packaging/packages/plymouth/-/blob/12df42855251af1d5f6e380968d3e2b51600c14d/plymouth.initcpio_hook
2025-11-03 22:30:48 <angeloverlain[m]> it’s just running a daemon while in the initramfs
2025-11-03 22:31:18 <WhyNotHugo> Sertonix[m]: mkinitfs!206 has 5 hooks for 2½ use cases. I'm not convinced by it.
2025-11-03 22:31:58 <angeloverlain[m]> WhyNotHugo: what do you mean ?
2025-11-03 22:33:03 <WhyNotHugo> angeloverlain[m]: is there a use case for the 'early' hook, or is it "just in case we need it"?
2025-11-03 22:33:16 <Sertonix[m]> <angeloverlain[m]> "can you expand on that ?" <- I think you can do `cp /usr/share/mkinitfs/initramfs-init /etc/mkinitfs/init && echo init=/etc/mkinitfs/init >> /etx/mkinitfs/mkinitfs.conf && $EDITOR /etc/mkinitfs/init` and edit the script to whatever you want
2025-11-03 22:36:09 <Sertonix[m]> I also feel like and good hook implementarion would make stull like clevis useless due to everything that clevis can do being better integrated when interacting with the components that clevis is using directly.
2025-11-03 22:38:40 <angeloverlain[m]> WhyNotHugo: the early hook might probably be used to support additional filesystems, but not sure it’s in scope right now (delegated to nlplug)
2025-11-03 22:41:11 <Sertonix[m]> WhyNotHugo: If you have any other ideas the best place would probably be https://gitlab.alpinelinux.org/alpine/aports/-/issues/17603 cause it involves making mkinitfs easier to use safely.
2025-11-03 22:41:17 <angeloverlain[m]> <Sertonix[m]> "I think you can do `cp /usr/..." <- the only issue with this approach is that of course, as updates go on, you will need to resync /usr init with your duplicated /init in /etc. 
2025-11-03 22:41:17 <angeloverlain[m]> and of course, I’m thinking about upstreaming everything here. If you make a copy of the init, only you will benefit from the changes, while when hooks are supported, users can create, install and use hooks that add several new functionality (additively)
2025-11-03 22:42:42 <angeloverlain[m]> Sertonix[m]: I think making mkinitfs less of a monolith is probably the best first step, but I know we will probably disagree on this
2025-11-03 22:42:49 <Sertonix[m]> As I said, power users don't benefit from upstreaming, for everything else it's a footgun.
2025-11-03 22:44:49 <angeloverlain[m]> Sertonix[m]: how is having TPM disk encryption, a nice boot menu and a TPM input on touchscreens for power users? I fail to understand the logic used here.
2025-11-03 22:44:49 <angeloverlain[m]> most general purpose distros have this (e.g. fedora, ubuntu, even pmOS). not sure why alpine should be stuck 2 decades ago 😅
2025-11-03 22:45:23 <angeloverlain[m]> I also think TPM attestation is used severely on server infra on boot
2025-11-03 22:46:12 <angeloverlain[m]> anyway, to cut this short, If you think mkinitfs!206 is not the right approach, I can stop working on it to save some time and effort
2025-11-03 22:46:22 <Sertonix[m]> angeloverlain[m]: Less monolithic sure but exposing API which we will ether need to break in the near future or be unable to improve mkinitfs further (due to design limitations) is not really a path forward in my opinion
2025-11-03 22:48:18 <angeloverlain[m]> Sertonix[m]: what are the design limitations? and why need we break the API « soon » ?
2025-11-03 22:50:01 <Sertonix[m]> angeloverlain[m]: That's the footgun. Interacting with a TPM in a way that is's actually useful is **very** difficult (and depending on the hardware and who you ask impossible). Tempting users with doing DIY crpto for their disk encryption setup will in 99% of the cases end in very weak to no security. That can't be a good fit for alpine
2025-11-03 22:51:42 <Sertonix[m]> (Implying that the way the hooks are implemented encourages (bad) DIY solutions more that a more stricter API which tries to make safe usage easier than unsafe one)
2025-11-03 22:54:25 <angeloverlain[m]> Sertonix[m]: no it’s not…? this is all you need to encrypt your disk with TPM: clevis luks bind -d /dev/tpm2 '{"pcr_ids":"1,7"}'. this ensures that your disk will automatically be decrypted on boot as long as secure boot is on and the UEFI settings haven’t changed. this has been a thing on most other OSes for atleast a decade too, so it’s not something new…
2025-11-03 22:54:25 <angeloverlain[m]> anyway, I gave atleast 2 other examples, and I’m very sure that configuring TPM is not the hardest thing one can do in alpine 
2025-11-03 22:56:51 <angeloverlain[m]> <Sertonix[m]> "(Implying that the way the hooks..." <- the alternative you gave is an even more bad DIY solution (imo), while with hooks available, one would need to (1) apk add clevis-tpm2-hook (or similar) (2) add clevis-tpm2 to mkinitfs features (3) bind their disk to clevis
2025-11-03 22:57:41 <angeloverlain[m]> anyway I think there’s a lot of bikeshedding here, so I’ll be pulling away my MR. thanks
2025-11-03 22:57:46 <angeloverlain[m]> im done
2025-11-03 23:00:11 <Sertonix[m]> Did you check if the hardware uses an easy to MITM external TPM chip? Did you inform the user about the fact that an untrusted booting operating system can look 100% the same as one that has been verified by secure boot? That's stuff I hope to tackle some day in #17603 but it's unfortunatly a minefield of footguns
2025-11-04 07:19:16 <fcolista> morning all
2025-11-04 08:49:07 <lassebq> gm
2025-11-04 16:01:39 <raspbeguy> Hello, can someone merge those MRs please? !87884 !90941 !91118 !91337 !91546
2025-11-04 17:55:14 <Ariadne> personally, i would rather replace mkinitfs than grow it further
2025-11-04 19:22:20 <Sertonix[m]> I was considering a rewrite. But yes, growing the current code may may not be a good way forward.
2025-11-04 20:30:49 <Ariadne> software in general has a lifecycle, and a lot of alpine components are basically reaching the end of their useful life
2025-11-04 20:31:13 <Ariadne> where alpine is today is very different from where alpine was when alpine-conf, mkinitfs and even abuild were created
2025-11-04 20:32:11 <Ariadne> this isn't to say that we haven't gotten meaningful use out of these packages, but rather that it is probably worth taking a step back and examining whether we should be replacing these components rather than growing them further
2025-11-04 21:40:39 <Sertonix[m]> lua-aports would be on my priority for being replaced or made redundant. I try to ignore the existance of alpine-conf but personal preference is the main reason for that.
2025-11-04 21:50:48 <Ariadne> i agree, we should build something better in go
2025-11-04 21:50:55 <Ariadne> especially buildrepo keeps me up at night
2025-11-04 21:57:30 <angeloverlain[m]> very interested to have this discussion. I think the folks at pmOS have a rewrite of mkinitfs in go (without the /init). maybe that can be considered ? not sure what to do about alpine-conf and abuild though
2025-11-04 21:57:31 <angeloverlain[m]> the only frustration I have is that alpine projects seem to be very slow-moving
2025-11-04 21:58:16 <Ariadne> alpine will always move slower than pmOS.  that is why it is important that both projects exist and maintain distinct identities.
2025-11-04 21:59:05 <Ariadne> the risk calculus is wildly different for both projects.
2025-11-04 22:00:39 <craftyguy[m]> yeah we use a differnet mkinitfs, you give it lists of stuff (modules, files, hooks, whatever) to compose an initramfs. we package the actual init script separately
2025-11-04 22:01:04 <Ariadne> at present a breaking change to pmOS is at most an inconvenience for most people using pmOS
2025-11-04 22:01:41 <craftyguy[m]> (I'm not advocating you use our mkinitfs, use whatever makes sense for Alpine :D)
2025-11-04 22:01:45 <Ariadne> so, if it happens, we pursue a fix as quickly as possible, and it's not a big deal
2025-11-04 22:03:42 <craftyguy[m]> our mkinitfs is basically just a fancy cpio/archive generator, that can resolve+include dependencies for given binaries and kernel modules, and write paths to given things to put them elsewhere in the archive
2025-11-04 22:04:48 <Ariadne> but alpine is a different story.  large swaths of the modern world run on alpine.  so we have to be very careful and thoughtful about how we evolve the system over time.
2025-11-04 22:04:51 <craftyguy[m]> you have to bring your own init though
2025-11-04 22:07:48 <Ariadne> i think we could switch to the pmOS mkinitfs upstream
2025-11-04 22:08:01 <Ariadne> obviously not right now, but in the next iteration
2025-11-04 22:08:48 <Ariadne> my point is simply that alpine moves slowly because it is necessary
2025-11-04 22:09:10 <WhyNotHugo> Sertonix[m]: what's the big issue with lua-aports? 
2025-11-04 22:09:39 <WhyNotHugo> What are general thoughts on running init from the initfs, so that mounting and early services all fall within the same architecture?
2025-11-04 22:09:47 <Ariadne> WhyNotHugo: iteration of the "aports database" is not stable
2025-11-04 22:10:32 <Ariadne> it is why if there is a build failure you can sometimes ask the builders to retry and it will go down a different chain of dependencies instead of the failed package
2025-11-04 22:11:18 <Ariadne> it also spawns a shell process for every APKBUILD it processes
2025-11-04 22:13:50 <WhyNotHugo> I noticed today that determining the build order was not quite right (for the py3.14 rebuild), and it would sometimes list a package before one of its dependencies.
2025-11-04 22:14:08 <invoked> what problems are being solved in alpine mkinitfs & other compnonents (beyond vague technical debt) that makes it worthwhile? that's unclear to me
2025-11-04 22:15:09 <WhyNotHugo> invoked: missing features include being able to use unl0ckr, having multiple partitions with the same LUKS passphrase
2025-11-04 22:15:30 <WhyNotHugo> TBF, we ought to have a distinction between mkinitfs and mkinitfs-init, the latter is currently replaceble.
2025-11-04 22:15:35 <Ariadne> kmscon for unicode terminal
2025-11-04 22:15:48 <invoked> alright
2025-11-04 22:16:25 <WhyNotHugo> kmscon? Just put a compositor in there with a LUKS prompter. Later re-use the compositor for greeters.
2025-11-04 22:16:47 <Ariadne> for a text-only session ?
2025-11-04 22:17:09 <craftyguy[m]> yeah the actual init implements support for runtime stuff on boot. mkinitfs is just a tool to make an archive you can use as the initramfs (which should include an init in it). at least that's how we treat them in pmOS, mkinitfs and the init it bundles are separate things
2025-11-04 22:22:04 <invoked> i also suppose stuff like dracut was ruled out for good reasons
2025-11-04 22:22:23 <Ariadne> dracut i believe depends on systemd these days and starts systemd in initramfs
2025-11-04 22:22:50 <invoked> mm, aye that would be a good reason
2025-11-04 22:23:01 <craftyguy[m]> dracut is also slooow
2025-11-04 22:24:10 <WhyNotHugo> dracut is quite far from "simple".
2025-11-04 22:24:15 <WhyNotHugo> It depends on systemd now?
2025-11-04 22:24:31 <Ariadne> at least on my fedora system, it starts systemd in the initramfs
2025-11-04 22:24:35 <Ariadne> soooo
2025-11-04 22:27:12 <WhyNotHugo> IMHO, starting an init in the initfs is not a bad idea. you want a few service up, so why not have the service supervisor?
2025-11-04 22:27:34 <Ariadne> i did not say that it was a bad idea.  i said it was not a good fit *for alpine*
2025-11-04 22:28:35 <Ariadne> because, like it or not, alpine itself is not going to switch from openrc to systemd
2025-11-04 22:29:11 <Ariadne> maybe there will be systemd derivatives of alpine in the future (like pmOS is experimenting with today), but these would be distinct derivatives and not alpine itself
2025-11-04 22:30:00 <WhyNotHugo> Totally on the same page
2025-11-04 22:32:21 <abby> dracut doesn't *require* on systemd
2025-11-04 22:32:41 <abby> it's the default on void, works fine without it
2025-11-04 22:32:57 <Sertonix[m]> Ariadne: Everything except for go please.
2025-11-04 22:33:14 <Ariadne> ok, great, but dracut would have to be modified to support alpine features like run-from-ram
2025-11-04 22:34:17 <Ariadne> my advice to anyone wanting to pursue dracut is -- show us an ISO that runs from ram and uses dracut as initramfs
2025-11-04 22:34:28 <abby> void has one?
2025-11-04 22:34:36 <Ariadne> not like alpine runs from ram
2025-11-04 22:35:05 <Ariadne> on alpine, we compose a tmpfs using apk and then pivot into it
2025-11-04 22:35:10 <abby> •If rd.live.ram=1 is used in this situation, the full, uncompressed root filesystem is copied during boot to /run/initramfs/rootfs.img in the /run tmpfs.
2025-11-04 22:35:26 <Ariadne> that isn't the same thing as alpine's run from ram
2025-11-04 22:35:34 <pj> go is in community, rust is in main :>
2025-11-04 22:35:54 <Sertonix[m]> WhyNotHugo: lua-aports tends to be out of sync, slow and badly documented. I am already working on something which will completely replace it for me at least with also scans everything for recursive dependencies.
2025-11-04 22:37:52 <Ariadne> alpine's initramfs builds a tmpfs based on either default configuration or user-supplied configuration in an "overlay file"
2025-11-04 22:38:07 <Ariadne> the overlay file is a tar which contains configuration data like /etc/apk/world
2025-11-04 22:38:38 <Sertonix[m]> Not really a fan of rust ether but I would take ot over go. But in my opinion the programming language should be chosen to fit the requirments but we haven't agreed on requirements of a potential mkinitfs replacement/rewrite yet.
2025-11-04 22:39:08 <Ariadne> the reason i suggested go for lua-aports is because there are already go libraries for dealing with aports
2025-11-04 22:41:34 <Ariadne> but i am not convinced you can actually iterate aports "fast"
2025-11-04 22:41:39 <Sertonix[m]> Ariadne: Regardless of what others do I will sooner or (more likely) later finish and publish my project which is currently shell+awk (and targets simple+fast)
2025-11-04 22:41:51 <Ariadne> we have some APKBUILDs which do nasty stuff
2025-11-04 22:42:34 <Ariadne> like to determine what outputs a given APKBUILD will generate, you have to evaluate it
2025-11-04 22:42:39 <Sertonix[m]> Do you consider 1.9s for all edge aports including building a dependency graph in awk and running a recursion check as fast enough :)
2025-11-04 22:43:02 <Ariadne> i think that is a significant improvement given the current situation, sure
2025-11-04 22:43:41 <Ariadne> but if we did not have to physically evaluate APKBUILDs to learn their outputs, it could be even faster
2025-11-04 22:44:11 <Ariadne> because you have stuff like [ "$CARCH" = "foo" ] && subpackages="$subpackages bar"
2025-11-04 22:47:23 <Sertonix[m]> You have do do that for each target, fair. I have considered parsing APKBUILDs differently but it would make stuff like community/libreoffice ether difficult to parse or make the APKBUILD more complex
2025-11-04 22:47:34 <Sertonix[m]> *to do
2025-11-04 22:48:33 <Ariadne> i think the solution is to use structured data for package metadata section, like melange
2025-11-04 22:48:54 <Ariadne> so you could have, say
2025-11-04 22:49:39 <Ariadne> subpackages: [{"bar": {"architectures": ["all", "!riscv64"]}]
2025-11-04 22:50:14 <Ariadne> the problem is that the metadata can change as a side effect of shell evaluation
2025-11-04 22:52:24 <Sertonix[m]> Regarding the "init system" in initramfs discussion above: I feel like most service manager complexity is created by needing to connect inconsistent interfaces. By requring hooks/services/whatever to adhere to a strict (and simple to implement) interface we can probably avoid the need for any full blown service managmentm
2025-11-04 22:52:46 <Ariadne> agreed
2025-11-04 22:54:59 <skarnet> I still don't get why anyone would need a service manager in initramfs. It's not supposed to be a permanent system. It's not supposed to be so complex that you cannot remember what daemons you have launched and cannot kill them before execing the real init.
2025-11-04 22:55:30 <skarnet> sure, there might be a lot of work to do in the initramfs, but isn't most of it short-lived, like, forking normal processes and waiting for them?
2025-11-04 22:57:56 <Sertonix[m]> Regarding APKBUILD metadata: So far my conclusion has always been "too complicated/breaking for too little advantages". I am sometimes simplifying APKBUILDs in a way which may make them easier to statically analyze. Trying to do that for all APKBUILDs would require abuild to grow a lot of (probably package specific) features and would therefor probably make it a lot less feasable for software independent of abuild to parse APBUILDs
2025-11-04 22:57:56 <Sertonix[m]> correctly.
2025-11-04 22:58:03 <Ariadne> Sertonix[m]: re: community/libreoffice, something like https://github.com/wolfi-dev/os/blob/main/glibc.yaml#L569 might be a solution
2025-11-04 22:59:53 <WhyNotHugo> skarnet: right now we run mdev in the initramfs and them mdevd/eudev after the root_pivot. You could run just one.
2025-11-04 23:00:00 <Sertonix[m]> skarnet: Maybe service manager is a bad comparison. It's more like ninja build for mounting filesystems
2025-11-04 23:00:17 <WhyNotHugo> It's a short-lived service which is later replaced by another long-lived service which does the same
2025-11-04 23:00:26 <skarnet> yeah and it's a good thing
2025-11-04 23:00:47 <Sertonix[m]> WhyNotHugo: I think mdev is used in a one shot style
2025-11-04 23:01:10 <Sertonix[m]> Bit I would need to check the source for that
2025-11-04 23:01:15 <skarnet> daemons don't run in the same context in an initramfs and in the final system, it's good that nothing leaks from the initramfs
2025-11-04 23:01:36 <skarnet> kill the extra stuff before execing into init and run the real daemon later, it's fine
2025-11-04 23:01:41 <skarnet> don't try to optimize on that
2025-11-04 23:01:54 <Ariadne> notably, the initramfs might have older dependencies that haven't received security fixes
2025-11-04 23:02:02 <skarnet> that too
2025-11-04 23:02:02 <WhyNotHugo> Sertonix[m]: yeah, mdev is only run until the root fs is found. My point is that starting mdevd at that point would simplify things
2025-11-04 23:02:12 <Sertonix[m]> To be more percise: suspend to RAM requires initramfs and others to be independant
2025-11-04 23:02:13 <Ariadne> so we don't want those processes sticking around for that reason amongst others
2025-11-04 23:02:33 <WhyNotHugo> Ariadne: shouldn't you update the initramfs if there have been security updates?
2025-11-04 23:02:44 <Ariadne> yes, but there is not a mechanism right now to enforce that
2025-11-04 23:04:31 <Ariadne> such a mechanism wouldn't be hard to build, mind
2025-11-04 23:05:09 <Ariadne> but right now, mkinitfs only runs to regenerate the initramfs if a kernel update happens
2025-11-04 23:05:34 <Ariadne> if you upgrade busybox, to patch a CVE for example, the initramfs won't be regenerated automatically
2025-11-04 23:05:39 <Sertonix[m]> Probably by converting the trigger to something that always runs
2025-11-04 23:06:01 <WhyNotHugo> Allowing other packages to trigger a rebuild is also a missing feature.
2025-11-04 23:06:12 <Ariadne> yes, *but* we only want to actually run if there is a reason to
2025-11-04 23:06:23 <Ariadne> otherwise, rebuilding mkinitfs every time is just slow for no reason :p
2025-11-04 23:06:30 <WhyNotHugo> pmOS's mkinitfs does support this, by having each package place a file with its version in a directory which triggers a rebuild.
2025-11-04 23:06:35 <WhyNotHugo> Not sure if elegant or hacky or both.
2025-11-04 23:06:59 <Ariadne> i would say hacky
2025-11-04 23:07:16 <Sertonix[m]> Ether hash or timestamp the input and config files may be enough
2025-11-04 23:07:54 <Ariadne> yeah the solution is to put a manifest file in the initramfs, like /.manifest or such
2025-11-04 23:08:07 <Ariadne> which lists the hashes of the files in the initramfs
2025-11-04 23:08:11 <Sertonix[m]> Would be an expensive way to check though
2025-11-04 23:08:51 <Ariadne> if you arrange for the manifest to be the first block of the cpio, it probably wouldn't be that bad
2025-11-04 23:09:09 <Ariadne> because we can stop parsing the initramfs cpio once we have the manifest
2025-11-04 23:10:15 <Sertonix[m]> I mean probing all files on the host fs including checking if any glob pattern evaluates differently now
2025-11-04 23:10:25 <Sheila> maybe it could store mtimes too, and trigger a fuller scan if mtimes change?
2025-11-04 23:10:26 <Ariadne> if we have a way of telling mkinitfs what files changed in the last apk transaction, it could be even faster :)
2025-11-04 23:10:57 <iggy> reading through all that... one thing I'll say for the go bindings... they are _not_ fast
2025-11-04 23:12:32 <Ariadne> they are not, because the bottleneck is having to evaluate the APKBUILD as a shell script
2025-11-04 23:12:42 <Ariadne> this is going to be a problem in any language
2025-11-04 23:13:19 <WhyNotHugo> At least in go you can trivially do this concurrently.
2025-11-04 23:13:33 <Sertonix[m]> Ariadne: I have been thing of potentially special commit hooks getting null terminated paths from stdin. But unfortunatly commit hooks don't have a file ending to reliably destinguish such a new type
2025-11-04 23:13:35 <Ariadne> and put the system under heavy PID pressure
2025-11-04 23:13:53 <WhyNotHugo> Just use a semaphor with max 12 goroutines
2025-11-04 23:14:33 <Ariadne> my experiences working with go engineers is that they just send it
2025-11-04 23:14:36 <Ariadne> :p
2025-11-04 23:14:55 <Ariadne> 100GB image in ram?  why not, google servers have TBs of RAM
2025-11-04 23:16:47 <Ariadne> anyway, lots of lessons we can learn from melange re: build recipe metadata
2025-11-04 23:17:51 <Ariadne> but i think that moving to structured metadata in aports is a tall ask, since there's tens of thousands of APKBUILDs that would have be converted
2025-11-04 23:20:07 <WhyNotHugo> Looking at https://github.com/chainguard-dev/melange, I can't say that I understand the examples in the README
2025-11-04 23:22:56 <Ariadne> what is not understandable about them?  you have a pipeline, the pipeline does stuff
2025-11-04 23:23:07 <Ariadne> at the end, you have build output, which becomes APK
2025-11-04 23:23:39 <craftyguy[m]> WhyNotHugo: ya what we do to trigger rebuilds is hacky. but our mkinitfs is fast. like really fast. often <1 second (longer time is usually from the deploy script that it calls to install things to slow boot storage)
2025-11-04 23:25:36 <WhyNotHugo> Ariadne: The script format uses "uses: autoconf/make-install". Is this a bespoke scripting language which calls `autoconfig && make install`?
2025-11-04 23:25:58 <Ariadne> not a scripting language per se
2025-11-04 23:26:00 <WhyNotHugo> The syntax of this is not at all obvious from the README
2025-11-04 23:26:25 <Ariadne> melange pipelines are an AST
2025-11-04 23:26:45 <Ariadne> `runs` refers to a block of shell script, `uses` refers to an external pipeline
2025-11-04 23:27:14 <Ariadne> my point, incidentally, has nothing to do with using melange, but rather melange's *structured metadata* would be useful for aports
2025-11-04 23:27:47 <Ariadne> but anyway, `uses: autoconf/make-install` means "use the autoconf/make-install pipeline"
2025-11-04 23:28:12 <WhyNotHugo> I think APKBUILD files are extremely simple to read. The first time I saw one it waas clear how it worked.
2025-11-04 23:28:24 <Ariadne> ok?
2025-11-04 23:28:30 <WhyNotHugo> I have no idea what an "external pipeline" is in this context.
2025-11-04 23:28:34 <pj> I think they are absolutely terrible to read
2025-11-04 23:28:53 <Ariadne> again, i am not proposing we switch aports to melange
2025-11-04 23:28:56 <pj> lots of internal functions that do "things"
2025-11-04 23:29:26 <skarnet> pj: any package building system is going to have lots of internal functions that do things
2025-11-04 23:29:33 <skarnet> this is not imputable to APKBUILD
2025-11-04 23:29:56 <skarnet> the functionality is inherently complex
2025-11-04 23:30:04 <WhyNotHugo> Ariadne: what kind of structured metadata format do you have in mind?
2025-11-04 23:30:14 <invoked> any change should be intuitive, not a step towards yocto-ification
2025-11-04 23:30:42 <Ariadne> that also has not been proposed
2025-11-04 23:30:46 <skarnet> invoked: what happened with yocto?
2025-11-04 23:31:17 <invoked> skarnet: nothing
2025-11-04 23:32:30 <Ariadne> WhyNotHugo: the metadata format from melange, where PACKAGE METADATA is static
2025-11-04 23:32:49 <Ariadne> this is NOT proposing using melange itself, melange pipelines, or anything other than the metadata part
2025-11-04 23:33:34 <Ariadne> the largest problem with APKBUILD is that to determine anything about an APKBUILD, you must evaluate it as a shell script
2025-11-04 23:33:42 <Ariadne> i am proposing solving THAT problem and THAT problem ALONE
2025-11-04 23:33:49 <ikke> Ariadne: you have been mentioning cue in the past, do you still think that would be a good format?
2025-11-04 23:34:10 <Ariadne> i think cue has run out of gas, sadly
2025-11-04 23:34:14 <Sertonix[m]> melange is essentially equivalent to symbolic execution of current APKBUILDs
2025-11-04 23:34:22 <ikke> Too sad
2025-11-04 23:34:23 <Ariadne> yes
2025-11-04 23:34:56 <pj> people like KDL now
2025-11-04 23:35:15 <ikke> https://kdl.dev/?
2025-11-04 23:35:52 <Ariadne> the point is that evaluating an APKBUILD (and having side effects from doing so, which means you must use a subshell to do this safely) to learn what architectures and subpackages it generates is the performance bottleneck
2025-11-04 23:36:20 <Ariadne> so the idea is you would have
2025-11-04 23:36:33 <pj> or whatever was the apple config format
2025-11-04 23:36:37 <Ariadne> package.name: whatever
2025-11-04 23:36:38 <Ariadne> [...]
2025-11-04 23:36:39 <Ariadne> ---
2025-11-04 23:36:47 <Ariadne> prepare() { default_prepare }
2025-11-04 23:36:49 <Ariadne> [...]
2025-11-04 23:36:53 <ikke> separate metadata from build instructions
2025-11-04 23:37:02 <invoked> pj: plist
2025-11-04 23:37:10 <ikke> But like you said, there are APKBUILDs which do some dynamic stuff
2025-11-04 23:37:17 <pj> invoked: no, that's just xml
2025-11-04 23:37:19 <Ariadne> yes, i am saying this is Bad Actually
2025-11-04 23:37:29 <WhyNotHugo> If you design a static metadata format, you need to contemplate all the weird cases which are currently scripted. Value can be overriden per-architecture, and any field can use as variables some other field.
2025-11-04 23:37:55 <Ariadne> thank you for explaining operating system design to me
2025-11-04 23:38:00 <WhyNotHugo> Anytime!
2025-11-04 23:38:01 <skarnet> I understand the safety+security problems with having to interpret APKBUILDS with a shell, but is *performance* really a concern?
2025-11-04 23:38:05 <pj> https://github.com/apple/pkl
2025-11-04 23:38:23 <skarnet> is it slow to the point it is impacting builds?
2025-11-04 23:38:37 <Ariadne> yes, it is
2025-11-04 23:38:50 <invoked> pj: oh right "pickle"
2025-11-04 23:38:57 <Sertonix[m]> skarnet: Not using a subshell is kind of UB and usinf subshells for each APLBUILD is slow
2025-11-04 23:39:25 <skarnet> yeah the fact that you have to use a subshell is probably terrible for performance
2025-11-04 23:39:28 <WhyNotHugo> My point is that you can't just pick up the existing fields and make them static, the format needs to contemplate *all* cases.
2025-11-04 23:39:38 <WhyNotHugo> skarnet: findign all pacakges which depend on python3-dev takes 3s on my laptop
2025-11-04 23:39:44 <Sertonix[m]> It's 12000 subshells as far as I remember
2025-11-04 23:40:28 <Ariadne> WhyNotHugo: my point is that (1) i already know this, and (2) it's not helpful to the discussion, but rather is a form of "stop energy"
2025-11-04 23:40:30 <Sertonix[m]> WhyNotHugo: only checking the main repo?
2025-11-04 23:40:48 <WhyNotHugo> Sertonix[m]: only checking main
2025-11-04 23:41:07 <skarnet> I thought abuild built a global dependency tree once and that's what it checked afterwards
2025-11-04 23:41:15 <skarnet> the initial reading has to be slow, yes
2025-11-04 23:41:22 <Ariadne> abuild has no understanding of dependencies
2025-11-04 23:41:28 <skarnet> hm
2025-11-04 23:41:31 <Ariadne> at least not in that way
2025-11-04 23:41:34 <ikke> lua-aports is doing that
2025-11-04 23:41:46 <Ariadne> (obviously it knows what dependencies need to be installed)
2025-11-04 23:42:43 <Sertonix[m]> *lua-aports can't analyze subpackage dependencies
2025-11-04 23:42:53 <WhyNotHugo> that explains a lot
2025-11-04 23:43:11 <Ariadne> Sertonix[m]: yes, which is why the build order is sometimes wrong
2025-11-04 23:43:15 <ikke> correct, because you would need to execute functions
2025-11-04 23:45:05 <WhyNotHugo> Can we generalise depends_dev and depends_doc to depends_$subpkg?
2025-11-04 23:45:29 <Ariadne> sure, but if we are going to do that, we might as well just do metadata right to begin with
2025-11-04 23:45:39 <Ariadne> it's not that much more of a lift
2025-11-04 23:45:46 <Sertonix[m]> WhyNotHugo: Yes but no. It needs eval and I try to remove it im abuild
2025-11-04 23:47:37 <Sertonix[m]> And some dependencies are only every known post-build which means that this issue will always percist in some way even with a different format
2025-11-04 23:47:49 <ikke> Sertonix[m]: any example of that?
2025-11-04 23:48:10 <WhyNotHugo> Are you talkign about soname tracing?
2025-11-04 23:48:18 <WhyNotHugo> We know the -dev dependency in that case, which is enough to determine build order.
2025-11-04 23:48:20 <ikke> Or do you mean like automatically detected so depetendencies?
2025-11-04 23:48:21 <Ariadne> Sertonix[m]: outside of so:[...] and py:[...] dependencies, which don't matter for determining the build order, what is there?
2025-11-04 23:49:24 <Sertonix[m]> If a package includes a .pc files it will have a pkgconf dependency added but pkgconf was likely never in makedepends. This example is hidden by the fact that abuild depends on pkgconf but I hope the ideas is clear.
2025-11-04 23:49:46 <Ariadne> you do not need pkgconf to generate a .pc file
2025-11-04 23:50:38 <Sertonix[m]> I have seen many .pc.in files in repos. Or do you mean to read the .pc files?
2025-11-04 23:51:04 <Ariadne> .pc.in files are templates for sed expressions to generate the real .pc file
2025-11-04 23:51:12 <Ariadne> pkgconf does not have a generator for .pc files
2025-11-04 23:51:40 <Sertonix[m]> I missed the "not" in your message O_O
2025-11-04 23:52:11 <Ariadne> my point is that abuild adding `pkgconf` dep here does not influence build order
2025-11-04 23:52:24 <Ariadne> (though i am not sure it should do that)
2025-11-04 23:53:02 <Sertonix[m]> Yes, but is's enough to avoid the issue for all relevant cases.
2025-11-04 23:53:58 <WhyNotHugo> Ariadne: personally I think we should add pkgocnf. I'm not a fan of build-base being an implicit dependency for everything, as many packages don't need those depends.
2025-11-04 23:54:35 <Ariadne> we inherited that particular wart from debian's build-essential, but i agree
2025-11-04 23:55:13 <Ariadne> (re: build-base being harmful)
2025-11-04 23:55:34 <Sertonix[m]> Reducing build-base is a bit difficult but generally I agree
2025-11-04 23:56:40 <Ariadne> one option could be to replace the metadata section with lua
2025-11-04 23:57:08 <Ariadne> that gives us an escape hatch for dynamic evaluation, while still avoiding the subshell problem
2025-11-04 23:58:02 <WhyNotHugo> How can I say no to lua? How would you store the lua metadata and shell script functions?
2025-11-04 23:58:16 <Ariadne> use a delimeter
2025-11-04 23:58:21 <Ariadne> package = "foo"
2025-11-04 23:58:25 <Sertonix[m]> Advantage of lua would be easy sandboxing. Disadvantage is lua version differences.
2025-11-04 23:58:27 <Ariadne> version = 123
2025-11-04 23:58:29 <Ariadne> ---
2025-11-04 23:58:33 <Ariadne> shell code here
2025-11-04 23:59:30 <WhyNotHugo> Ariadne: I like it. Variables resolved from the lua metadata are exported to the shell?
2025-11-04 23:59:35 <Ariadne> yes
2025-11-04 23:59:51 <Ariadne> this would also allow us to clean up a lot of really meh stuff we currently do
2025-11-05 00:00:11 <Ariadne> like stacks of POSIX shell substitutions to mutate a variable
2025-11-05 00:00:35 <WhyNotHugo> How else would one learn every single form of POSIX shell substitution if not by working on APKBUILDS!?
2025-11-05 00:00:38 <angeloverlain[m]> @craftyguy:postmarketos.org I'm assuming postmarketos' mkinitfs is a drop-in replacement for alpine's mkinitfs right (without the initramfs-init bit)
2025-11-05 00:01:04 <Ariadne> WhyNotHugo: tbh i would rather arrive at a future where there is very little shell scripting in APKBUILDs
2025-11-05 00:01:23 <Ariadne> we do not need to be doing build() { ./configure; make; make install } all the time
2025-11-05 00:01:24 <WhyNotHugo> angeloverlain[m]: not really, the pmOS one handles installation of the file too (into different locations dependngin on hardware). The one on Alpine simply writes to an output path.
2025-11-05 00:01:31 <Ariadne> we can just say "this is an autoconf project"
2025-11-05 00:01:53 <skarnet> uh, no
2025-11-05 00:02:04 <Ariadne> (that was the goal of melange's symbolic execution design, that you shouldn't ever have to use shell unless you are working on something exotic)
2025-11-05 00:02:26 <skarnet> ./configure is an interface, and not all projects that follow this interface are autoconf, tyvm
2025-11-05 00:02:27 <WhyNotHugo> Ariadne: I'd rather see `./configure; make; make install` in packages that do that, and the explicit commands in packages that do something else. The indirection hurts readability.
2025-11-05 00:02:33 <skarnet> if you make that assumption things will break
2025-11-05 00:02:40 <abby> we have "build styles" for this on void
2025-11-05 00:02:41 <ikke> ncopa mentioned he wants to avoid too many magic templates
2025-11-05 00:03:00 <abby> build_style=gnu-configure for autoconf, =configure for non-autoconf ./configure
2025-11-05 00:03:19 <Sertonix[m]> Not that the how to interface with build systems is independant of using something like lua
2025-11-05 00:04:05 <Ariadne> WhyNotHugo: the problem is that the indirection helps with keeping things *consistent* at scale
2025-11-05 00:04:10 <WhyNotHugo> OpenBSD ports also have this "build style" interface. It's really hard to properly grasp what's being executed, compared to a current APKBUILD.
2025-11-05 00:04:56 <Ariadne> for example, with cmake and meson, we want to occasionally change settings across every APKBUILD that uses these systems
2025-11-05 00:05:18 <angeloverlain[m]> @_oftc_WhyNotHugo:matrix.org it seems you can just do `--no-bootdeploy`, in which case I assume it will just keep the "build artifacts" in the "work directory"
2025-11-05 00:05:22 <Sertonix[m]> Do we really want to discuss build styles? I have a lot to say about that :)
2025-11-05 00:05:42 <Ariadne> if every APKBUILD spells everything out, then we have to update every APKBUILD directly
2025-11-05 00:05:57 <WhyNotHugo> angeloverlain[m]: I don't recall the details. It's not a drop-in replacement in the sense that you need to use different flags, but it can be used in place of the alpine mkinitfs with some minor tinkering.
2025-11-05 00:05:59 <Ariadne> this is about to bite us hard with the usr-merge
2025-11-05 00:06:18 <Ariadne> because basically ~none of our autoconf builds are consistent
2025-11-05 00:07:12 <WhyNotHugo> Doesn't something like abuild-meson help here? abuild-autoconf?
2025-11-05 00:07:14 <Sertonix[m]> To make it short: In my opinion APKBUILDs should express their intend and when using a known build system abuild can make sure that the expressed intend is actually done in a sane way
2025-11-05 00:07:42 <Ariadne> WhyNotHugo: it does, but this is still indirection
2025-11-05 00:07:59 <Ariadne> what is the difference between "abuild-meson" and "build_type=meson"
2025-11-05 00:08:23 <Sertonix[m]> abuild-meson only helps a bit. Mainly suffering from abuild (and environment variables set for abuild) not being able to express things proplerly
2025-11-05 00:08:31 <angeloverlain[m]> @_oftc_WhyNotHugo:matrix.org it seems it would work very easily, if we create a file /usr/share/deviceinfo/00-alpine with a few of these options set to just place the generated image in /boot. https://wiki.postmarketos.org/wiki/Deviceinfo_reference
2025-11-05 00:08:43 <invoked> Sertonix[m]: aye, if we're gonna change things it should make reviews less work
2025-11-05 00:09:37 <Ariadne> to be clear, i designed melange as an anti-abuild
2025-11-05 00:10:02 <Ariadne> the internal design documentation literally says: when in doubt, do the exact opposite of whatever abuild does
2025-11-05 00:10:43 <Ariadne> i recognize that is not realistic for alpine, but the current design of APKBUILD hurts us bad when it comes to enforcing consistency
2025-11-05 00:10:50 <Sertonix[m]> So if I make abuild better you will make melange worse :)
2025-11-05 00:11:13 <Ariadne> Sertonix[m]: i will not make melange anything, i work elsewhere for years now
2025-11-05 00:11:34 <ikke> Sertonix[m]: I guess that would count as mission accomplished
2025-11-05 00:13:05 <Ariadne> if abuild is better, then that is good for alpine
2025-11-05 00:13:17 <angeloverlain[m]> anyway, re: supervisor in initramfs, it seems arch supports running either systemd or a busybox init as /init (https://wiki.archlinux.org/title/Mkinitcpio)
2025-11-05 00:14:03 <angeloverlain[m]> re: melange. I also agree that it's considerably harder to parse than current abuild, but maybe it comes with time
2025-11-05 00:14:29 <Ariadne> again: i am not proposing alpine use melange
2025-11-05 00:14:43 <Ariadne> i am just saying we can learn lessons from melange
2025-11-05 00:15:02 <angeloverlain[m]> yeah, we can definitely learn some things there
2025-11-05 00:15:05 <ikke> How are we going to avoid shedding bikes?
2025-11-05 00:15:15 <WhyNotHugo> Anyone tried using Arch's mkinitcpio on Alpine? I guess the explicit usage of bash is really weird.
2025-11-05 00:15:21 <Ariadne> we can also learn lessons from void
2025-11-05 00:15:26 <Sertonix[m]> Ariadne: If you want me to look into any of the discussed things in more depths feel free to create an issue on gitlab. Otherwise some stuff may be handled with very low speed/priority on my side.
2025-11-05 00:15:43 <WhyNotHugo> Ariadne: I think chimera also has something like build styles.
2025-11-05 00:15:44 <angeloverlain[m]> Sertonix[m]: yeah, I would follow that issue too :)
2025-11-05 00:16:12 <Ariadne> WhyNotHugo: yes
2025-11-05 00:16:28 <WhyNotHugo> Of course you knew this :P
2025-11-05 00:16:44 <Ariadne> right now, however, i think our largest concern is the fact that someone can send an alpine dev a patch that exfiltrates their signing key
2025-11-05 00:16:44 <angeloverlain[m]> @_oftc_WhyNotHugo:matrix.org never tried it, but what I like about it is that everything is cleanly separated into "hooks" (https://wiki.archlinux.org/title/Mkinitcpio#Common_hooks)
2025-11-05 00:17:04 <Sertonix[m]> Hot take: Write meson files for everthing so that buildstyle=meson can be an implicit constant :)
2025-11-05 00:17:05 <Ariadne> without even running `abuild`
2025-11-05 00:17:27 <WhyNotHugo> angeloverlain[m]: yeah. and hooks have an explicit order specified in the config.
2025-11-05 00:17:51 <abby> <WhyNotHugo> Ariadne: I think chimera also has something like build styles. <- chimera's package build system started out as a rewrite of void's
2025-11-05 00:18:29 <angeloverlain[m]> @_oftc_WhyNotHugo:matrix.org yeah, which is why I was so excited for mkinitfs!206, but I think we probably want something completely new
2025-11-05 00:18:35 <WhyNotHugo> Sertonix[m]: New precondition for packaging in alpine: upstream must use meson. :)
2025-11-05 00:18:46 <Ariadne> i don't think that is realistic, sadly
2025-11-05 00:18:51 <Sertonix[m]> Ariadne: Protecting the signing key also needs remote signing which is an open MR on the apk-tools repository.
2025-11-05 00:19:53 <Ariadne> Sertonix[m]: my point is, if i am not very careful, and `git apply` something, and then use `buildrepo` or `ap`, i can wind up executing code that hurts me
2025-11-05 00:20:59 <Sertonix[m]> Yes. Also on my TODO list for my mentioned, yet to be finished project :)
2025-11-05 00:21:02 <Ariadne> be that signing key exfiltration, be that personal data exfiltration, whatever
2025-11-05 00:21:04 <Ariadne> :p
2025-11-05 00:21:24 <Ariadne> i would normally say "crypto wallet" here, but i personally think crypto is bullshit and thus don't have one
2025-11-05 00:22:19 <Ariadne> so i think moving APKBUILD metadata to lua is a good start
2025-11-05 00:22:27 <Sertonix[m]> Signing from a non-file source is all we need.
2025-11-05 00:22:52 <angeloverlain[m]> 
ACTION wonders why lua

2025-11-05 00:22:58 <Sertonix[m]> Your "start" is one of the most complicated things :p
2025-11-05 00:23:11 <Ariadne> because alpine already use lua
2025-11-05 00:23:21 <WhyNotHugo> Ariadne: home banking cookies?
2025-11-05 00:23:35 <angeloverlain[m]> well, I'd say that alpine also uses shell scripts :p
2025-11-05 00:24:08 <Ariadne> Sertonix[m]: the issue is that if there is a boobytrap in APKBUILD, it will run when aports is enumerated
2025-11-05 00:24:21 <skarnet> angeloverlain[m]: because Arch does something doesn't mean it's a good idea. Running a generic init system in an initramfs does not sound useful, and if initramfs need a supervisor that can run as pid 1 and *also* exec into something else to run the real system, I have a better fit than what Arch uses. :P
2025-11-05 00:24:43 <skarnet> but really, a shell script or whatever should be amply sufficient
2025-11-05 00:25:39 <Ariadne> in fact, i think we have just gotten really lucky that this hasn't already happened
2025-11-05 00:25:49 <Ariadne> it seems like a fairly obvious attack to me
2025-11-05 00:26:00 <WhyNotHugo> Ariadne: do you apply patches before reading them?
2025-11-05 00:26:02 <Sertonix[m]> Yes, to me as well
2025-11-05 00:26:25 <Ariadne> WhyNotHugo: yes, all the time, because that's the point of a git worktree
2025-11-05 00:26:32 <abby> mkinitcpio doesn't use busybox /bin/init as init, it's a custom script, and same for systemd-style, systemd is not pid1
2025-11-05 00:27:01 <WhyNotHugo> Considering the "enumeration implies execution" angle only further convinces me that we should ought to use lua for metadata
2025-11-05 00:27:32 <Ariadne> WhyNotHugo: i think lua is fine, we can restrict it
2025-11-05 00:27:36 <angeloverlain[m]> @_oftc_skarnet:matrix.org @_oftc_abby:matrix.org  I think what we would want is to have an /bin/init that can execute other different "programs", instead of having one big shell script.
2025-11-05 00:27:40 <abby> (fwiw i do work on mkinitcpio, mostly for void compat, and i don't understand the point of sysd-ini-initramfs)
2025-11-05 00:27:44 <ikke> atools-go / apkgquery can evaluate APKBUILDs without execution
2025-11-05 00:28:14 <ikke> (it obviously has to ignore subshells that exec)
2025-11-05 00:28:18 <Ariadne> ikke: can it do so reliably?  does it handle [ "$CARCH" = "foo" ] && subpackages="$subpackages bar" ?
2025-11-05 00:28:22 <ikke> yes
2025-11-05 00:28:26 <ikke> It should
2025-11-05 00:28:41 <ikke> But let met check to be really sure
2025-11-05 00:28:41 <Ariadne> how do you handle that without evaluating it?
2025-11-05 00:29:06 <skarnet> angeloverlain[m]: what do you mean by "execute other different programs"? because executing other different programs is something sh does pretty well, but I understand you have other requirements :)
2025-11-05 00:29:39 <Ariadne> ikke: a good test would be community/libreoffice
2025-11-05 00:29:45 <Sertonix[m]> Metadata values can be a trap themselfs so it can't really fulfil any thread model alone
2025-11-05 00:29:46 <ikke> Ariadne: ok
2025-11-05 00:30:26 <Ariadne> Sertonix[m]: metadata values can't execute rm -rf /
2025-11-05 00:31:14 <Ariadne> at least, not normally
2025-11-05 00:31:37 <angeloverlain[m]> @_oftc_skarnet:matrix.org alpine's initramfs-init is 1105 lines long, it would be nice to separate that into different "features" that can be toggled while building the image (and thus never make it into the initramfs). like completely disabling zfs. or (hypothetical example) in the future, add support for bcachefs, without having to make initramfs-init even more complex
2025-11-05 00:31:50 <Sertonix[m]> Pretty sure that metadata can be enough to execute rm -rf / on your next build
2025-11-05 00:32:07 <Ariadne> i prefer to drop bcachefs support :p
2025-11-05 00:32:29 <angeloverlain[m]> I also don't use it, was just an example
2025-11-05 00:32:52 <Sertonix[m]> ikke: Even better test is community/gdal
2025-11-05 00:33:04 <skarnet> angeloverlain[m]: ah, so, you mean modularity at build time. That's very different from modularity at boot time.
2025-11-05 00:33:08 <angeloverlain[m]> better examples would be TPM2-based disk decryption, disk decryption on touch screens (with unl0kr), kmscon, plymouth, etc...
2025-11-05 00:33:42 <angeloverlain[m]> @_oftc_skarnet:matrix.org yes exactly, sometimes it's hard for me to make my point clear :)
2025-11-05 00:34:37 <Ariadne> Sertonix[m]: community/gdal is a great example of why letting people commit crimes is bad
2025-11-05 00:35:14 <WhyNotHugo> Arch Linux initramfs generation tools
2025-11-05 00:35:20 <WhyNotHugo> getvar() { eval "printf '%s\n' \"\${$1:-$2}\"" }
2025-11-05 00:35:21 <Ariadne> okay maybe that isn't kind :p
2025-11-05 00:35:31 <WhyNotHugo> (ignore arch message, accidental paste)
2025-11-05 00:39:26 <Ariadne> anyway my point is that dynamic code execution to determine package outputs is Bad(tm)
2025-11-05 00:39:43 <Ariadne> and the ideal end state is one where we no longer do that
2025-11-05 00:40:08 <Ariadne> (or at least where this dynamic code execution is heavily constrained)
2025-11-05 00:52:56 <elibrokeit> 19:28 <Ariadne> ikke: can it do so reliably?  does it handle [ "$CARCH" = "foo" ] && subpackages="$subpackages bar" ?
2025-11-05 00:53:26 <elibrokeit> poor design imo :P subpackages_amd64="..." is the better way to avoid code execution here
2025-11-05 00:53:39 <Ariadne> you don't see me disagreeing...
2025-11-05 00:54:48 <elibrokeit> Gentoo just uses a micro-language, so you'd end up with subpackages=" amd64? ( .... ) universal-subpackages"
2025-11-05 01:00:26 <elibrokeit> (the mini language is actually "build options" but CARCH is a forced-on global build option provided by the builder itself to all packages)
2025-11-05 01:37:20 <lotheac> i'm a bit late to the discussion, but separating metadata from the build code makes a lot of sense to me so i would support such an endeavour
2025-11-05 01:37:51 <lotheac> perhaps even regardless of metadata language; it doesn't matter terribly much :p
2025-11-05 01:38:31 <elibrokeit> one file or two? :P
2025-11-05 01:38:39 <lotheac> doesn't matter
2025-11-05 01:39:23 <elibrokeit> I'd say it does matter, polyglot files are how you end up with GNU m4sh
2025-11-05 01:39:48 <lotheac> i even had this weird idle thought that with the delimiter-approach someone might propose shell for the build instructions and... shell for the metadata :D but that would really only help solve part of the problem
2025-11-05 01:40:13 <lotheac> elibrokeit: yeah fair enough, let's hope we don't go that far :p
2025-11-05 01:41:18 <lotheac> APKBUILD i feel like generally are not directly fed to shell as is anyway so i don't see a terribly big problem changing the format a little bit
2025-11-05 01:41:25 <lotheac> but, not my bikeshed to paint
2025-11-05 01:41:33 <elibrokeit> if it's just a security issue, you could try an absolute ban on external commands in global scope and evaluate metadata using a restricted parser 
2025-11-05 01:41:58 <lotheac> there's the performance aspect too
2025-11-05 01:42:31 <lotheac> and "absolute ban" would need some kind of automatic enforcement
2025-11-05 01:42:48 <elibrokeit> depending on how you define the limits, it could be as simple as a safe simple subset of shell that can be parsed from any language
2025-11-05 01:43:03 <lotheac> right, but at that point it doesn't really need to be shell
2025-11-05 01:43:35 <elibrokeit> e.g. Gentoo's make.conf syntax is shell compatible but parsed via python shlex
2025-11-05 01:44:43 <elibrokeit> and Gentoo ebuilds are shell, but the tool automstically enforces an absolute ban on external commands during metadata parsing :D
2025-11-05 01:45:02 <ikke> ftr, ncopa has always spoken against splitting things up in multiple files
2025-11-05 01:46:46 <elibrokeit> it could be as simple as using rbash / bash -r
2025-11-05 01:47:43 <elibrokeit> which isn't strictly an absolute security boundary but does get you most of the way there -- anything that can escape rbash could likely also be caught easily during eyeball review
2025-11-05 01:49:02 <Ariadne> 
ACTION is not in favor of switching from GCC to clang as default system C compiler

2025-11-05 01:49:53 <Ariadne> 
ACTION is also not in favor of dropping vector support from loongarch64 port

2025-11-05 08:38:58 <Sertonix[m]> Through the power of PATH= I already can tell that tr is the only external command (actively) used in aports and external commands are already banned in the global scope. The missing thing is to enforce such a rule.
2025-11-05 11:28:45 <ikke> only one is gdal, it executes tr
2025-11-05 11:48:46 <lindsay> Some subpackage will change pkgdesc and install_if in their sub functions, I think it's not enough to only eval top-level expressions.
2025-11-05 12:02:05 <WhyNotHugo> lindsay: some also change their depends in the sub functions, that's kinda what led to this discussion: it's quite non-trivial to extract this information.
2025-11-05 13:05:13 <Sertonix[m]> One has to be clear that determining the dependencies of the final packages is a non-goal due to being impossible. The only intresting thing is to determine which APKBUILDs need to be build before any given other APKBUILD.
2025-11-05 13:13:40 <WhyNotHugo> Right, we only want to know build-dependencies.
2025-11-05 13:24:34 <elibrokeit> determining the dependencies of the final packages would be possible with a spec change, if it's simply a matter of statically overriding in sub functions. depends_$subpkg="..." -- if you want to support dynamic conditional computation e.g. modifying metadata variables based on the ELF metadata of built files, that's hopeless regardless of whether sub functions get evaluated via 
2025-11-05 13:24:34 <elibrokeit> untrusted bash, so it's not worth discussing 
2025-11-05 13:30:48 <elibrokeit> if you don't mind it being dangerously untrusted shell execution, there's even another option -- archlinux makepkg has some very clever code to evaluate sub-functions via `declare` piped to grep to locate assignments, for extracting variables out of a function without running the side effects of the function
2025-11-05 13:35:38 <WhyNotHugo> Any feedback for !92200? Probably okay to merge.
2025-11-05 13:36:13 <WhyNotHugo> Ditto !92480
2025-11-05 14:15:18 <raspbeguy> Hello, can someone merge those MRs please? !87884 !90941 !91118 !91337 !91546
2025-11-05 15:36:32 <Biswa96[m]> shai has an unresolved review.