2021-11-18 13:45:12 <ncopa> hey! how are things going? I'm planning to tag alpine 3.15.0 Monday or Tuesday next week
2021-11-18 15:21:23 <tomalok> hiya ncopa!  thanks for the heads up!  i have the new alpine-cloud-images build script working - feature partiy with alpine-ec2-ami at the moment.  used it to do a full build of all image variants last weekend (including 3.15.0_rc1).  i think we'll soon be able to test vs mcrute 's new identity broker code, and in the meantime, i'm woring on an
2021-11-18 15:21:23 <tomalok> interim way to gather artifact references (per region AMI ids) into a format compatible with the existing lua script that builds https://alpinelinux.org/cloud
2021-11-18 15:23:07 <ikke> I noticed this MR btw the other day: https://gitlab.alpinelinux.org/alpine/infra/alpine-mksite/-/merge_requests/32
2021-11-18 15:25:04 <tomalok> not familiar with builds.sr.ht ...  will take a closer look this morning
2021-11-18 15:25:33 <ikke> That's run by Drew DeVault, which also provides us with lists.alpinelinux.org
2021-11-18 15:25:55 <ikke> A mailing-list orriented git forge
2021-11-18 15:26:35 <ncopa> err, is that a commercial service?
2021-11-18 15:28:19 <ncopa> other cloud related thing I have been thinking of lately, I have been thinking about write a minimalistic cloud-init implementation. Something that can read subset of cloud-init config to provision alpine machines
2021-11-18 15:29:04 <ncopa> i found the need when setting up a local cluster in libvirt with terraform
2021-11-18 15:30:02 <ncopa> i also noticed that lima-vm (linux on mac) uses some cloud-init bits and needs to do workarounds for alpine
2021-11-18 15:30:04 <tomalok> we have a basic minimal cloud-init in tiny-ec2-bootstrap -- plan to extend it to other clouds and rename to tiny-cloud-bootstra[
2021-11-18 15:30:24 <ncopa> does it user the python cloud-init implementation?
2021-11-18 15:30:37 <tomalok> no, it's very minimal shell
2021-11-18 15:30:41 <ncopa> ah, right
2021-11-18 15:31:10 <ncopa> and nice!
2021-11-18 15:31:39 <ncopa> the issue with shell and cloud init is that it is not very nice to parse yaml from shell
2021-11-18 15:31:41 <tomalok> expands the root partition to use all space in the root volume, does some network and volume plugin stuff, and executes a script if provided as user_data
2021-11-18 15:32:10 <ncopa> right, i think i might have had a quick look on that
2021-11-18 15:32:12 <tomalok> tiny expects you to provide the code
2021-11-18 15:32:57 <ncopa> how do you provide the user_data? via url or via seed.iso image?
2021-11-18 15:32:58 <tomalok> but iirc it looks for a shebang and we may be able to have an optional plugin for doing stuff as specified from input yaml
2021-11-18 15:33:18 <tomalok> user_data comes from the cloud provider...  metadata you set when launching an instance
2021-11-18 15:33:38 <ncopa> how does the instance pick it up?
2021-11-18 15:34:14 <ikke> tomalok: sr.ht is still under construction, and during that time it's 'encouraged' to pay for hosting projects there
2021-11-18 15:34:18 <tomalok> instance metadata data service, (IMDS) usually acessible from 169.256.169.254
2021-11-18 15:34:21 <ikke> https://man.sr.ht/billing-faq.md
2021-11-18 15:34:59 <ncopa> ok, i guess that is what the cloud-init aws provider does
2021-11-18 15:35:11 <tomalok> we pull info (including user_data) from IMDS using curl
2021-11-18 15:35:24 <tomalok> (iirc)
2021-11-18 15:35:27 <ncopa> ok. makes sense
2021-11-18 15:35:48 <ncopa> i was thinking of implement the mini cloud-init in either lua or golang
2021-11-18 15:36:05 <tomalok> the sr.ht stuff looks like it's in addition to, instead of replacing the original cloud page contents
2021-11-18 15:37:26 <psykose> no encouragement for builds.sr.ht; it's paid only already
2021-11-18 15:37:47 <psykose> because of the crypto miners that abused it
2021-11-18 15:39:57 <tomalok> shell has been sufficient for tiny thus far...  ;)  but something with more features may make sense to go with go
2021-11-18 15:40:17 <tomalok> ok, i need to stop typing in bed and get breakfast now :)
2021-11-18 15:43:30 <ncopa> good morning!
2021-11-18 15:44:12 <psykose> enjoy your breakfast :)
2021-11-18 15:46:54 <ikke> That MR from ddevault rendered: https://wwwdev.alpinelinux.org/cloud/
2021-11-18 16:02:17 <tomalok> so sr.ht ... is a cloud provider?  The MR doesn't really provide any information why this should be added.
2021-11-18 16:10:40 <ikke> https://man.sr.ht/builds.sr.ht/
2021-11-18 16:11:05 <ikke> https://man.sr.ht/builds.sr.ht/compatibility.md
2021-11-19 17:11:38 <ncopa> tomalok: I think you have a good point. Maybe add a comment to the MR? Would be good to have the answer to “why” in the commit message
2021-11-19 17:12:15 <ncopa> i also think that we may want think about where we draw the line
2021-11-20 05:41:59 <Ariadne> i don't think builds.sr.ht counts as a cloud service
2021-11-20 05:44:35 <psykose> it's definitely not meant for any hosting, so i don't think it makes much sense being placed in the context of the amazon images
2021-11-24 14:46:47 <ncopa> 3.15.0 is out. what do we need to update the aloud images?
2021-11-24 14:47:20 <ncopa> https://alpinelinux.org/cloud/ still shows 3.14.2
2021-11-24 14:48:57 <tomalok> i'll need to run the new builder (first time with the 3.15.0 virt iso), and when we have images, i have a script that translates the new output format to the one used by alpine-ec2-images' releases.yaml -- i'll update that, and then alpine-mksite can rebuild the cloud page
2021-11-24 14:50:10 <tomalok> the new builder should be much more friendly for running from a gitlab pipeline, but there's some more to do there on the credential broker part of things, i believe.  
2021-11-24 14:51:13 <tomalok> (using a gitlab auth token to get the cloud credentials)
2021-11-24 14:52:18 <tomalok> up early for a day job early meeting today, but i'll start the ball rolling and see how far we get.
2021-11-24 15:00:57 <tomalok> 2021-11-24T15:00:43Z - build - INFO - Latest Alpine version 3.15 and release 3.15.0
2021-11-24 15:01:05 <tomalok> found it! ;)
2021-11-24 15:01:32 <tomalok> also,
2021-11-24 15:01:34 <tomalok> 2021-11-24T15:00:51Z - build - WARNING - 3.11-x86_64-bios-tiny-aws SKIPPED, "3.11" end_of_life 2021-11-01
2021-11-24 15:09:15 <tomalok> 2021-11-24T15:06:26Z: ==> alpine.qemu.3.15-aarch64-uefi-tiny-aws: /tmp/script_2281.sh: line 54: /usr/sbin/mkfs.fat: not found
2021-11-24 15:09:36 <tomalok> dosfstools changed its install location?
2021-11-24 15:10:23 <minimal> tomalok: yes, it moved as otherwise it did not replace the Busybox version
2021-11-24 15:11:22 <minimal> there was an issue with (I think) setup-disk where it run mkfs.fat and the Busybox version was found 1st in PATH
2021-11-24 15:11:32 <tomalok> yeah, it wasnt' replacing it before, which was causing problems...  i shouldn't need the full path to it any more
2021-11-24 15:12:58 <tomalok> (of course the x86_64 variant doesn't need it, so it's still chugging along with the initial test build/import/publish)
2021-11-24 15:14:14 <tomalok> i'll validate the test images and kick off the real builds in 45-60m
2021-11-24 17:16:55 <tomalok> 16.5 minutes to build & import 10 aws images with --parallel 5, now publishing (copying to regions) with --parallel 10
2021-11-24 17:17:28 <ikke> tomalok: if all goes well, the cloud is automatically updated now once you push it to the repo
2021-11-24 17:18:42 <tomalok> 
ACTION crosses fingers

2021-11-24 17:27:19 <tomalok> 7.5 mins to publish -- list of artifacts is including an extra 'null' region for some reason, will filter that in the script that converts to the legacy releases.yaml
2021-11-24 17:43:41 <tomalok> ikke: alpine-ec2-ami has the new releases YAML merged now.
2021-11-24 17:45:14 <ikke> hmm, it's not updated :(
2021-11-24 17:45:47 <tomalok> is it supposed to kick off a gitlab pipeline somewhere?
2021-11-24 17:45:54 <ikke> No
2021-11-24 17:46:20 <ikke> we have gitlab webhooks connected to mqtt, and use mqtt-exec to trigger the rebuild
2021-11-24 17:47:22 <tomalok> iirc we tried this last time (or the time before last) and it didn't work then, either
2021-11-24 17:47:45 <ikke> yeah, I recall clandmeter mentioning he fixed something, but apparently not
2021-11-24 17:47:56 <ikke> probably mqtt permissions
2021-11-24 17:48:46 <ikke> tomalok: lua5.3: _scripts/generate_cloud.lua:37: attempt to index a nil value (field 'creation_date')
2021-11-24 17:49:38 <tomalok> lets make sure i added that field key correctly
2021-11-24 17:50:21 <tomalok>     creation_date: '2021-11-24T17:17:33.103711'
2021-11-24 17:50:42 <ikke> yeah, I see it in there
2021-11-24 17:51:41 <tomalok> okay, i see the problem
2021-11-24 17:52:00 <tomalok> i'm missing another level
2021-11-24 17:52:22 <ikke> ah
2021-11-24 17:52:42 <tomalok> i have another day job meeting in 8m...  it's a race...
2021-11-24 17:53:01 <ikke> heh, I know, right
2021-11-24 17:53:17 <ikke> Just let me know when you fixed it
2021-11-24 17:55:29 <tomalok> pushed, merging soon
2021-11-24 17:56:35 <tomalok> merg'd
2021-11-24 17:57:38 <ikke> Yes, it's most likely an mqtt permission issue
2021-11-24 17:57:51 <ikke> I can see the messages ariving globally, but not on the site builder
2021-11-24 17:58:09 <ikke> Anyway, the site is updated now
2021-11-24 17:58:12 <ikke> \o/
2021-11-24 17:58:49 <tomalok> ,o/   \o, \o, \o,  "what do we want?"
2021-11-24 18:00:06 <tomalok> okay, time for grooming day job backlog with the team (after not having done it for years)
2021-11-24 18:35:38 <ikke> tomalok: I think I fixed it now (mqtt-exec was not listening the relevant topic)
2021-11-24 19:04:32 <tomalok> got around to checking out https://alpinelinux.org/cloud -- it's reporting version for each section, not release
2021-11-24 19:05:06 <ikke> You mean only for the major releases
2021-11-24 19:05:37 <tomalok> x.y = version, x.y.z = release (in the terminology i usually work with)
2021-11-24 19:05:52 <tomalok> and EOL isn't right either
2021-11-24 19:06:23 <tomalok> EOL for 3.15.x should always be 2023-11-01
2021-11-24 19:06:42 <ikke> I see, yes
2021-11-24 19:07:37 <tomalok> not sure if this is datasource related...  will check
2021-11-24 19:09:09 <tomalok> AHA.  the configs that built this didn't have an EOL set for 3.15 yet, when that happens (usually for edge builds) the EOL gets set to tomorrow.
2021-11-24 19:09:50 <tomalok> that should be fixable without rebuilding (we can update permissions & tags on published images)
2021-11-24 19:10:09 <ikke> ok
2021-11-24 19:11:00 <tomalok> the version vs. release is an issue with the new --> old releases conversion, easily fixable too.
2021-11-24 19:11:58 <tomalok> though edge versions now have release values equivalent to YYYYMMDD
2021-11-24 19:12:07 <tomalok> (can work around that)
2021-11-24 19:27:54 <tomalok> okay just pushed a fixed releases YAML to alpine-ec2-ami
2021-11-24 19:31:50 <ikke> 2021-11-24 19:07:32 build.sh received topic: gitlab/push/alpine/cloud/alpine-ec2-ami
2021-11-24 19:31:52 <ikke> yay
2021-11-24 19:32:10 <ikke> automatically updated now :)
2021-11-24 19:37:24 <tomalok> woohoo!
2021-11-27 19:21:59 <tomalok> ikke: got our first gitlab issue for the lack of sudo in the 3.15.0 aws image...  longer term, i think we'll add release notes to the image MOTDs, but in the short term, perhaps we can put a note on the alpinelinux.org/cloud page?
2021-11-27 19:23:27 <ikke> tomalok: 3.15.0 should still have sudo
2021-11-27 19:23:55 <ikke> Hmm
2021-11-27 19:24:19 <tomalok> the cloud image used to specifically install it, but with the deprecation we swapped it with doas.  it can still be installed with doas apk add sudo, though
2021-11-27 19:36:42 <minimal> tomalok: perhaps the way is to (a) add sudo to the cloud image but also (b) start to "educate" people to use "doas" rather than "sudo" as with the move of sudo to community in Alpine 3.16 the support for sudo will then be affected
2021-11-27 19:38:42 <minimal> specifically "the community repository is supported until next stable release", so once Alpine 3.17 comes out then sudo on 3.16 will no longer be supported
2021-11-27 19:39:37 <tomalok> (nod) it wouldn't be too hard to publish a '-r1' for 3.15.0 that also included sudo...  it was, for a while, already in community but moved back to main at some point wasn't it?
2021-11-27 19:41:01 <ikke> no
2021-11-27 19:41:23 <ikke> I made an MR to move it, but it was held until 3.15 was branched
2021-11-27 19:42:14 <ikke> But, the idea *was* to move it in 3.15 already
2021-11-27 19:44:25 <minimal> tomalok: maybe setup /etc/motd in 3.15 to warning/advise people to use doas rather than sudo? Then they can't say they haven't told (unless they're on older version - could also add to 3.14 and 3.13 images)
2021-11-27 19:44:55 <ikke> https://gitlab.alpinelinux.org/alpine/cloud/alpine-ec2-ami/-/issues/128#note_194868
2021-11-27 19:45:54 <tomalok> the way forward then - add sudo back to 3.15.0, add a MOTD about the deprecation and forthcoming removal, and publish a 3.15.0-r1 image?
2021-11-27 19:46:34 <minimal> wasn't thinking of a motd link to the release notes but more of a "Please be aware that sudo support will be reduced from Alpine 3.16, please use doas instead" banner
2021-11-27 19:46:43 <tomalok> cloud image has historically had community enabled by default, fwiw
2021-11-27 19:47:02 <ikke> Same with docker
2021-11-27 19:47:57 <minimal> even with community enabled the important issue is more the shorter support cycle for sudo once its in community
2021-11-27 19:48:13 <ikke> correct
2021-11-27 19:51:14 <tomalok> what i'm thinking of for the cloud image motd is a link to release notes in general, but also specifically adding a "hey, doas is replacing sudo, which is going away soon"
2021-11-27 19:51:59 <tomalok> is there any sort of support for /etc/motd.d?
2021-11-27 19:54:02 <ikke> Don't think so
2021-11-27 19:54:29 <ikke> apparent ubuntu uses update-motd, which executes scripts in that directory
2021-11-27 19:55:23 <minimal> tomalok: /etc/motd is handled by the shell (i.e. Busybox ash). I quick "strings" in /bin/sh looking for "motd" only shows /etc/motd so I guess no subdir support
2021-11-27 19:55:59 <ikke> You can have something that generates /etc/motd based on whats in /etc/motd.d/*
2021-11-27 19:56:00 <tomalok> (nod) i can work multi/variant motd into the build configs
2021-11-27 19:56:42 <tomalok> so, no additional in-image scripting needed
2021-11-27 19:56:57 <ikke> ubuntu relies on pam
2021-11-27 19:57:38 <tomalok> will allows us to add motd segments for different things as necessary (i.e. if there's something particular about aarch64 or azure or whatnot)
2021-11-27 19:58:38 <tomalok> no pam with the tiny alpine images -- though i think i recall seeing it for cloud-init (which isn't quite ready with this builder yet)
2021-11-27 19:59:07 <ikke> Yeah, I wouldn't suggest adding pam
2021-11-27 19:59:22 <tomalok> not _just_ for getting a motd.d ;)
2021-11-27 19:59:30 <ikke> nod
2021-11-27 20:00:03 <minimal> tomalok: for cloud-init I don't think I added a dependancy but expect PAM-enabled sshd installed as otherwise you can't disable user console login but keep key-based SSH login enabled
2021-11-27 20:00:38 <tomalok> minimal (nod) yeah, that's what i recall seeing
2021-11-27 20:00:41 <minimal> s/user console login/lock user password/
2021-11-27 20:00:51 <ikke> oh yeah, that weird openssh behavior
2021-11-27 20:01:19 <minimal> Openssh has an IF/ELSE block that treats locked passwords differently depending on whether PAM is enabled or not
2021-11-27 20:01:50 <ikke> yeah
2021-11-27 20:03:34 <minimal> it doesn't make sense as (at least to me) it's clear that locking passwords should not also block keys, as there's a different distinct way to disable Linux user accounts
2021-11-27 20:05:00 <ikke> Even if you think that's the case, deciding whether to do that based on whether pam is enabled is even stranger
2021-11-27 20:05:44 <minimal> yeah I never dug into it further to work out the "logic" behind it
2021-11-27 20:06:11 <minimal> I did start to look at tinyssh as an alternative but then got bogged down trying to add support for that to cloud-init
2021-11-27 20:15:37 <tomalok> release notes pretty much always at "https://www.alpinelinux.org/posts/Alpine-<release>-released.html", it seems?  (haven't gone back through them all)
2021-11-27 20:15:50 <ikke> Yeah
2021-11-27 20:16:03 <tomalok> cool, that'll get its own motd section
2021-11-27 20:16:10 <ikke> I just copy the previous post and do s/<prev-release>/<next-release>
2021-11-27 20:17:27 <tomalok> goes back as far as 3.12.x, which is as far back as we make images (for the next 5 months, at least)
2021-11-27 20:18:45 <tomalok> is it worth mentioning both x.y.z and x.y.0 release notes?
2021-11-27 20:19:28 <ikke> The minor releases are mostly relevant for the iso and docker images
2021-11-27 20:20:32 <ikke> Though, it might also matter for the cloud images in that they can contain security fixes for the base system
2021-11-27 20:20:59 <tomalok> (nod) big changes in x.y.0, security changes in x.y.z
2021-11-27 20:21:04 <tomalok> generally
2021-11-27 20:21:37 <ikke> kernels also in x.y.z
2021-11-27 20:21:51 <tomalok> ah, yeah, that's true and applicable here
2021-11-27 20:22:55 <tomalok> adding x.y.[0..z] to motd might be a bit much though
2021-11-27 20:25:26 <ikke> tomalok: not sure if you are aware, but https://alpinelinux.org/releases.json contains a reference to the post
2021-11-27 20:25:29 <ikke> posts*
2021-11-27 20:29:16 <tomalok> (nod) i found that shortly after release, will work that into some improvements in determining latest version, release, and EOL too.
2021-11-27 20:29:39 <ikke> tomalok: Would it help for you to have an explicity latest-stable reference?
2021-11-27 20:29:46 <ikke> I have a usecase for that
2021-11-27 20:30:23 <tomalok> latest-stable is good for choosing which virt ISO to use for building all the images, and which ovmf packages to pull in for qemu
2021-11-27 20:30:52 <tomalok> but it's been also handy to be able to build/ref _rc's too
2021-11-27 20:32:06 <tomalok> right now i kinda scrape     repo_url = f"{CDN_URL}/{alpine_version}/{repo}/{arch}"
2021-11-27 20:32:39 <tomalok> was considering parsing APKINDEX instead, but it appears there are some better sources for some of these things ;)
2021-11-27 20:37:57 <tomalok> hm, seems like someone decided to break github.com today?
2021-11-27 20:39:54 <ikke> I guess every day someome decides to at least try it :P
2021-11-27 20:40:41 <ikke> githubstatus.com: "all systems operational" 🤔
2021-11-27 20:41:00 <PJ[m]> it's offline
2021-11-27 20:42:07 <tomalok> i get a pic of their mascot falling down into a Roadrunner/Wile E. Coyote-style canyon with "Whoops!"
2021-11-27 20:44:45 <ikke> yea, me too
2021-11-27 20:44:47 <ikke> 500
2021-11-27 20:48:51 <PJ[m]> they updated status page
2021-11-27 20:50:34 <minimal> Github down, end of the world is nigh!
2021-11-27 20:54:13 <ikke> laugs in gitlab
2021-11-27 21:17:47 <danieli> gitlab(.com) was down just the other week
2021-11-27 21:19:14 <minimal> they're not very committed to uptime? ;-)
2021-11-27 21:23:04 <ikke> MS disconnected minimal
2021-11-27 21:38:34 <tomalok> i think i have a flexible motd config solution in place, making some test images for 3.14, 3.15, and edge to see how well it works
2021-11-27 22:12:18 <tomalok> hm, forgot that edge builds wouldn't have any release notes...  ;P
2021-11-27 22:14:33 <tomalok> 3.15.0 motd:
2021-11-27 22:14:52 <ikke> nod :P
2021-11-27 22:14:54 <tomalok> Welcome to Alpine!
2021-11-27 22:14:54 <tomalok> The Alpine Wiki contains a large amount of how-to guides and general
2021-11-27 22:14:54 <tomalok> information about administrating Alpine systems.
2021-11-27 22:14:54 <tomalok> See <https://wiki.alpinelinux.org/>.
2021-11-27 22:14:54 <tomalok> Release Notes:
2021-11-27 22:14:56 <tomalok> NOTE: 'sudo' has been deprecated, please use 'doas' instead.
2021-11-27 22:14:56 <tomalok> * <https://alpinelinux.org/posts/alpine-3.15.0/released.html>
2021-11-27 22:15:33 <tomalok> 3.14.3 motd does not have the NOTE and has links for 3.14.0 and 3.14.3 release notes
2021-11-27 22:15:59 <tomalok> edge has a different NOTE that says "NOTE: 'sudo' is no longer installed by default, please use 'doas' instead."
2021-11-27 22:16:16 <ikke> right
2021-11-27 22:19:49 <tomalok> I left out the "setup-alpine" bit from motd because in theory, the cloud image should have taken care of most/all of that
2021-11-27 22:21:19 <ikke> yeah, makes sense
2021-11-27 22:21:22 <tomalok> ikke: if the motd text is looking good to you, i'll publish revised 3.15.0 images -- probably don't need to do this for 
2021-11-27 22:21:29 <tomalok> older versions
2021-11-27 22:21:48 <ikke> Looks good
2021-11-27 22:22:08 <tomalok> and, in theory, the cloud page should auto-update when i update alpine-ec2-ami releases/alpine.yaml
2021-11-27 22:22:23 <ikke> yes
2021-11-27 22:22:46 <tomalok> i'm glad i sorted out the --revise parameter for the build tool... :)
2021-11-27 22:42:02 <tomalok> mcrute: working on the credential broker?
2021-11-27 22:43:21 <tomalok> not getting a json.load()-able response when trying to get credential url
2021-11-27 23:09:28 <tomalok> seems to have passed
2021-11-28 00:12:05 <tomalok> alpine-ec2-ami releases/alpine.yaml updated