2024-01-17 16:47:24 <ncopa> tomalok: I wonder if we can merge https://gitlab.alpinelinux.org/alpine/cloud/tiny-cloud/-/merge_requests/101 for now
2024-01-17 16:48:35 <ncopa> we cannot use the set_network_interfaces for all clouds because all clouds does nto support imds meta-data/network-interfaces. I think in cloud-init it is only nocloud and openstack datasources.
2024-01-17 16:48:38 <ncopa> https://cloudinit.readthedocs.io/en/latest/reference/network-config-format-eni.html
2024-01-17 16:51:37 <ncopa> the MR does enables the set_epemeral_network for all clouds, but we may want change the logic for each cloud provider via `want_ephemeral_network` function
2024-01-17 16:56:09 <minimal> the "Network configuration sources" section of https://cloudinit.readthedocs.io/en/latest/reference/network-config.html lists which DataSources support the legacy ENI
2024-01-17 16:56:21 <minimal> it is more than NoCloud and OpenStack
2024-01-17 16:56:40 <minimal> i.e. ConfigDrive, OpenNebula
2024-01-17 16:57:38 <ncopa> ok. but tiny-cloud does not target any of thos I suppose
2024-01-17 16:57:49 <ncopa> in tiny-cloud context i'd say only nocloud matters
2024-01-17 16:58:11 <ncopa> I also added the resolv_conf to meta-data, because it is needed early: https://cloudinit.readthedocs.io/en/21.1/topics/modules.html#resolv-conf
2024-01-17 16:58:13 <minimal> well Openstack can mean either (a) via Openstack IMDS, or (b) via ConfigDrive
2024-01-17 17:00:07 <ncopa> my point is that the current set_network_interfaces and set_resolv_conf (in https://gitlab.alpinelinux.org/alpine/cloud/tiny-cloud/-/merge_requests/101) can not be used by other cloud providers than nocloud/alpine
2024-01-17 17:01:19 <ncopa> and I'd really like to fix this for the 3.19.1 release (today or tomorrow)
2024-01-17 17:02:54 <minimal> ok, so alpine-specific then? I wasn't aware that cloud-init NoCloud meta-data file could define resolv-conf settings
2024-01-17 17:03:19 <ncopa> i dont think it can tbh. but it says that it needs it early
2024-01-17 17:03:33 <ncopa> early as in before user-data is pulled in
2024-01-17 17:04:32 <ncopa> because when user-data executes it is expected to have a fully working network
2024-01-17 17:06:12 <minimal> right, because in cloud-init the network is configured in either cloud-init-local or cloud-init stages and user-data is handled in cloud-init-config and cloud-init-final stages (and ephemeral networking is done in cloud-init-local)
2024-01-17 17:07:32 <minimal> for NoCloud though why use ENI rather than v1/v2 network-config?
2024-01-17 17:08:22 <ncopa> tbh i dont remember why I added resolv-conf to meta-data
2024-01-17 17:08:47 <minimal> both v1 and v2 network support nameserver config
2024-01-17 17:08:58 <ncopa> I implemented ENI because it was dead simple to implement
2024-01-17 17:09:01 <minimal> eni embedded in meta-data is legacy and limited
2024-01-17 17:09:14 <ncopa> was a cheap way to get the basic working
2024-01-17 17:09:41 <minimal> the only time I considered using it was for NoCloud via network
2024-01-17 17:09:42 <ncopa> tiny-cloud would be limited as alpine system installer if there was no way to do networking
2024-01-17 17:10:05 <minimal> no way?
2024-01-17 17:10:17 <ncopa> i intend to implement v2 network-config
2024-01-17 17:10:21 <ncopa> but not for 3.19.2
2024-01-17 17:10:26 <ncopa> but not for 3.19.1
2024-01-17 17:10:31 <minimal> ok
2024-01-17 17:11:20 <ncopa> but ENI was implemented for a minimal viable product
2024-01-17 17:11:32 <ncopa> was cheap
2024-01-17 17:11:40 <ncopa> that was the only reason
2024-01-17 17:13:05 <minimal> ok and now you're seeing the limitations of that approach
2024-01-17 17:15:39 <ncopa> the alternative was no network config at all
2024-01-17 17:15:48 <ncopa> for 3.18 release
2024-01-17 17:16:00 <ncopa> yes, it has limitations
2024-01-17 20:28:21 <tomalok> will take a look later today - but i think it should be possible to do ephemeral network earlier for all clouds to be able to get at metadata/userdata earlier in the boot process
2024-01-17 20:30:43 <tomalok> ncopa ikke while i have your ears -- azure image needs to be "fixed vhd" instead of dynamic which means those images would be 1 GiB in size -- so i'm thinking of compressing all downloadable cloud images -- savings is significant for azure, of course, but aslo makes QCOW2 and dynamic VHD 50% the size of noncompressed.  those who download will need
2024-01-17 20:30:43 <tomalok> to decompress before importing into the cloud provider...   thoughts there?
2024-01-17 20:35:28 <ikke> I don't have a strong opinion either way. For an end-user it means one extra step. Download times are faster, but are probably somewhat negated by the time to uncompress. 
2024-01-17 20:35:57 <ikke> Storage wise, using less is ofcourse better
2024-01-17 20:36:33 <tomalok> that was the question, basically -- storage for all the images with each release adding up
2024-01-17 20:37:10 <ikke> At the moment, our t1 mirror servers have plenty of storage, but we also have to take our 3rd party mirrors into account
2024-01-17 20:37:34 <ikke> And our master mirror server as well btw
2024-01-17 20:38:01 <ikke> so in light of that, I would be in favor of compressing
2024-01-17 20:48:36 <tomalok> 👍 thx
2024-01-17 22:11:30 <ncopa> I’m also in favor of compressing. It is possible to decompress while downloading too
2024-01-17 22:14:56 <ncopa> Re ephemeral net. The MR does it already. Just switch the want_ephemeral to true. Can be set individually for each provider too
2024-01-17 22:15:42 <ncopa> i just haven’t tested it on other providers, which is why I let it be false by default 
2024-01-17 22:24:37 <minimal> tomalok: is the minimum disk image size 1G? Trying to find my Azure notes...
2024-01-17 22:25:28 <minimal> I know it has to be multiples of 1MB
2024-01-17 22:49:30 <minimal> https://learn.microsoft.com/en-us/azure/virtual-machines/linux/create-upload-generic
2024-01-17 22:49:32 <minimal> "All VHDs on Azure must have a virtual size aligned to 1 MB"
2024-01-17 22:50:01 <minimal> nothing there about 1GiB minimum size
2024-01-18 04:00:01 <tomalok> minimal: that's the disk size during build for all these images
2024-01-18 04:13:06 <tomalok> ncopa: just a quick question on the ephemeral network MR - but barring any overnight revelation waking me from my sleep i'll merge in the AM and tag/release a new version
2024-01-18 17:22:03 <tomalok> ncopa: merged, released tiny-cloud-3.0.6...  haven't updated aports though
2024-01-18 17:29:08 <ncopa> ouch.. i never got to add the -q option to udhcpcd
2024-01-18 17:29:18 <ncopa> udhcpc
2024-01-18 17:30:07 <ncopa> and apparently i never go to press  send on my comment
2024-01-18 17:31:01 <ncopa> got distracted by ipv6 issues
2024-01-18 17:31:14 <ncopa> we need to add -q to udhcpc
2024-01-18 17:31:34 <ncopa> https://tpaste.us/NBoy
2024-01-18 18:14:31 <tomalok> ncopa: another PR? i can take a look / do another release in a couple hours, lunch-ish.
2024-01-18 19:33:36 <ncopa> i will test it first
2024-01-18 19:33:38 <ncopa> im on it
2024-01-18 19:50:26 <ncopa> I took the liberty to merge it myself. A new release would be nice. You just tag it?
2024-01-18 19:51:16 <ncopa> apparently so. a simple unannotated tag.
2024-01-18 19:54:12 <ncopa> I can do that too so you can have your lunch break for lunching...
2024-01-18 19:59:51 <ncopa> all done! thanks!
2024-01-18 20:56:23 <tomalok> there was another step for making the release from the tag, that's done now for 3.0.7
2024-01-18 21:44:39 <ncopa> thanks!
2024-01-21 20:02:52 <tomalok> compressing cloud images raises new questions -- should the original or compressed image be what's signed?
2024-01-21 20:03:15 <ikke> Good question, but I suppose the thing you check is the thing you downloaded?
2024-01-21 20:03:25 <ikke> ie, verify it before you do anything with it
2024-01-21 20:03:47 <tomalok> (nod) makes the most sense
2024-01-21 20:03:59 <minimal> tomalok: whichever is being provided
2024-01-21 20:07:13 <minimal> I'm surprised the (uncompressed) disk image is 1GB in the first place, I assume that only a fraction of the filesystem space is actually being used
2024-01-21 20:07:50 <minimal> wouldn't it make sense to repack it smaller as part of the image creation?
2024-01-21 20:14:02 <tomalok> the disk volume used to build the images is 1 GiB
2024-01-21 20:14:53 <tomalok> 1 GiB was originally chosen because that's the smallest you can have on EC2
2024-01-21 20:16:32 <minimal> but doesn't tiny-cloud grow the root partition/fs on 1st boot?
2024-01-21 20:16:49 <minimal> so therefore there's no need for the image to be the same size as the smallest disk
2024-01-21 20:18:22 <tomalok> build disk size is, in theory customizable...  though for azure it's easy to miss setting the size you want, and it defaults to image size.  (had an issue "hey why am i only getting a 1 GiB disk?" recently)
2024-01-21 20:20:25 <minimal> I'm guessing any created images are only using 80-200Gb so just don't understand why 1GB images are being created, it seems "wasteful"
2024-01-21 20:20:42 <minimal> especially if they're not compressed
2024-01-21 20:21:03 <tomalok> s/Gb/MB/ perhaps you meant  ;)
2024-01-21 20:21:11 <minimal> yeah
2024-01-21 20:21:30 <tomalok> even the compressed qcow2 and dynamic VHDs benefit from compression
2024-01-21 20:22:02 <tomalok> well it's probably not compressed, just more effecient storage layout
2024-01-21 20:22:03 <minimal> well at least those are "sparse"
2024-01-21 20:22:09 <tomalok> ^^
2024-01-21 20:23:03 <minimal> I mean personally I have VM images of the order of 180MB and those are not the "slim" images
2024-01-21 20:26:05 <minimal> in my own scripts at the end I log "du -h" and "du-hb" output to record the actual vs "theoretical" disk usage for sparse disk images
2024-01-21 20:30:31 <tomalok> good idea, adding that
2024-01-21 20:38:13 <minimal> tomalok: so what's the typical actually used disk space in the cloud images?
2024-01-21 20:57:55 <tomalok> minimal figuring that out now -- cloudinit images seem to have a "du -sh" of  183M or so.
2024-01-21 20:58:52 <tomalok> first edge tiny build is happening now -- also uefi.  smallest i suspect will be bios + tiny
2024-01-21 20:59:45 <tomalok> tiny + uefi is about 117M
2024-01-21 21:14:57 <minimal> so approx 1/8 of the 1GB image size...
2024-01-22 22:04:19 <ncopa> do we really need make the disk images 1G?
2024-01-22 22:14:12 <tomalok> no.  i changed it yesterday -- we start with a base disk_size and then if a "dimension" needs more disk space we add some more.
2024-01-22 22:14:44 <tomalok> haven't tested the resulting images just yet, but they all build fine, at least
2024-01-26 19:19:56 <ncopa> tomalok: FYI, i have tagged 3.19.1 release
2024-01-26 21:07:24 <tomalok> ncopa: i guess i know what i'm doing this weekend!  3.19.1 images probably won't be compressed, but they'll at least be using smaller disk image sizes
2024-01-26 21:08:07 <tomalok> and azure image should be the right VHD format
2024-01-26 21:08:53 <ncopa> im also tagging 3.16.9, 3.17.7 and 3.18.6
2024-01-26 21:09:23 <tomalok> even better :)
2024-01-28 16:32:37 <tomalok> Validated Azure and OCI images actually work...  Fixed an issue with aarch64 and OCI/Azure using the wrong serial console device...  So, bumping those up to "beta".
2024-01-28 16:33:20 <tomalok> Reading the docs for how to import GCP custom images... might be a bit more work to get these sorted.
2024-01-28 16:34:33 <ikke> ack
2024-01-28 16:38:30 <tomalok> seems they want it named "disk.raw" and then tar.gz'd.
2024-01-28 16:47:27 <minimal> tomalok: their import tool supports VMDK, VHD etc
2024-01-28 16:48:33 <minimal> ah, still needs to be called disk.raw....strange
2024-01-28 16:50:02 <tomalok> minimal: thought the import tool didn't handle aarch64 images
2024-01-28 16:50:27 <tomalok> so it's "disk.raw" -- but doesn't have to be raw format?
2024-01-28 16:50:30 <minimal> yeah just noticed that comment in the docs........really sounds like a mess
2024-01-28 16:50:37 <minimal> yeah all very confusing
2024-01-28 16:51:13 <tomalok> and also the "--format=oldgnu"  sounds really portable.
2024-01-28 16:51:34 <minimal> like I said it sounds like a mess
2024-01-28 16:52:03 <minimal> they must have had an internal competition for someone to design the most convoluted mechanism lol
2024-01-28 16:53:51 <minimal> you'd almost think they didn't want people to import custom images lol
2024-01-28 16:54:23 <tomalok> "you will use what we give you, and like it."
2024-01-28 16:55:19 <minimal> more like "we are the hive mind, you will be assimilated and think like us"
2024-01-28 16:57:53 <tomalok> "The image file must be of a s <a>supported image</a>" which takes to you a page that doesn't mention anything about supported images
2024-01-28 17:00:28 <minimal> oh it's mentioned in the document in the locked filing cabinet in the disused lavatory with the sign on the door "Beware of the Leopard" ;-)
2024-01-28 17:14:22 <tomalok> let's see if it'll take a VHD
2024-01-28 17:14:59 <minimal> I found this: https://googlecloudplatform.github.io/compute-image-import/image-import.html
2024-01-28 17:16:02 <tomalok> yeah found that before too -- that's the non-manual import that doesn't handle OSs that aren't on their known list (or aarch64)
2024-01-28 17:16:26 <minimal> and from that, the comments here may be useful: https://github.com/GoogleCloudPlatform/compute-image-import/blob/main/cli_tools/import_precheck/precheck/check_disks_linux.go
2024-01-28 17:16:32 <tomalok> it didn't choke on the VHD being named "disk.raw"
2024-01-28 17:17:19 <minimal> thpugh those comments likely apply only to x86/x86_64 BIOS booting
2024-01-28 17:17:42 <minimal> which is why then aarch64 is treated differently
2024-01-28 17:18:06 <tomalok> this first manual import attempt is x86_64 uefi
2024-01-28 17:20:09 <minimal> their instructions for manual import are very confusing, i.e. "The boot disk that you import must have a functional MBR partition table or a hybrid configuration of a GPT partition table with a MBR bootloader"....obviously that won't apply to aarch64
2024-01-28 17:22:35 <tomalok> well, aarch64 will have GPT
2024-01-28 17:23:06 <minimal> yeah but that quote says "GPT partition label with a MBR bootloader", aarch64 won't have a MBR bootloader
2024-01-28 17:25:11 <tomalok> yay console output...  boo "Boot failed: not a bootable disk"
2024-01-28 17:25:31 <minimal> lol
2024-01-28 17:25:56 <minimal> "Game over. Next player please!"
2024-01-28 17:26:09 <tomalok> next format
2024-01-28 17:26:18 <tomalok> dynamic vhd doesn't work
2024-01-28 17:27:45 <minimal> BTW I've got some (WIP) stuff packaged locally that I must get around to adding to testing: some OVA tools and some tools relating to UEFI variables storage (for AWS and QEMU)
2024-01-28 17:36:13 <tomalok> minimal: qq - do you know offhand what qemu uses for aarch64 serial dev?  ttyS0 or ttyAMA0?
2024-01-28 17:36:44 <minimal> ttyAMA0
2024-01-28 17:37:25 <tomalok> cool, just like OCI & Azure -- probably resolves an issue for nocloud
2024-01-28 17:38:05 <tomalok> don't know about GCP yet, but AWS is the only weird one that still uses ttyS0 for aarch64
2024-01-28 17:38:06 <minimal> that's the standard Arm name for serial devices
2024-01-28 17:39:50 <minimal> I assume the name is defined in the pl011 driver
2024-01-28 17:40:04 <minimal> so is aarch64 on AWS not using pl011?
2024-01-28 17:40:41 <tomalok> getty is running fine on ttyS0, at any rate --was not the case for OCI or Azure
2024-01-28 17:44:22 <tomalok> so raw image doesn't work either...  "SeaBIOS" -- maybe they don't support x86_64 uefi at all?
2024-01-28 17:45:37 <minimal> for getty's are not not relying on the initramfs' init to take care of that?
2024-01-28 17:45:44 <minimal> s/not not/you not/
2024-01-28 17:46:07 <minimal> the init will add entries to /etc/inittab for all the "active" serial devices it sees
2024-01-28 17:47:09 <tomalok> it's always been set at image build time, wasn't aware of any initramfs magic
2024-01-28 17:47:34 <minimal> I'm assume they must support x86_64 UEFI by now as otherwise they couldn't handle recent Windows versions....although they may not support Windows as an imported custom image (for license reasons)
2024-01-28 17:47:49 <minimal> the initramfs "magic" went in a couple of Alpine releases ago
2024-01-28 17:48:39 <tomalok> in theory the image's /etc/inittab should have had the right getty in it, but i hadn't seen it.
2024-01-28 17:48:59 <minimal> https://gitlab.alpinelinux.org/alpine/mkinitfs/-/blob/master/initramfs-init.in?ref_type=heads#L107dev@rockoon:~$
2024-01-28 17:49:15 <minimal> and also the function after that on line 122
2024-01-28 17:49:29 <tomalok> will have to table this mystery -- need to focus on whether or not i can get anything to work on GCP
2024-01-28 17:50:17 <minimal> for GCP try x86_64 BIOS as the "baseline"
2024-01-28 17:50:34 <minimal> with Grub
2024-01-28 18:10:26 <tomalok> manual import x86_64 bios still not working -- i did find an option to use VHD disk, going to attempt that next
2024-01-28 18:17:24 <tomalok> nope, import failed
2024-01-28 18:17:36 <tomalok> 🤷
2024-01-28 18:18:08 <tomalok> GCP stays alpha for this set of releases -- will finish that off this afternoon, hopefully.
2024-01-28 18:20:55 <tomalok> detected bootable uefi, but then complained "could not detect operating system. Please re-import with the operating system specified. For more information, see https://cloud.google.com/compute/docs/import/importing-virtual-disks#bootable"
2024-01-28 18:23:04 <ikke> seems like there is no alpine variant
2024-01-28 18:23:07 <ikke> https://cloud.google.com/sdk/gcloud/reference/compute/images/import#--os
2024-01-28 18:26:28 <tomalok> which is why i was trying a manual import, but its bios ended up not finding a bootable disk
2024-01-28 19:04:31 <minimal> "--os" is listed as an *optional* flag, indeed at the top of the page the def is:
2024-01-28 19:04:50 <minimal> [--data-disk | --byol --os=OS]
2024-01-28 19:05:10 <minimal> which implies that --os is only relevant when you're importing a licensed OS
2024-01-28 19:07:10 <minimal> though yes there appears to be no way to import an unlicensed OS....doh!
2024-01-28 19:12:26 <minimal> tomalok: what about the step mentioned here: https://github.com/GoogleCloudPlatform/compute-archlinux-image-builder
2024-01-28 19:12:47 <minimal> i.e. "gcloud compute images create"
2024-01-28 19:13:33 <minimal> https://cloud.google.com/sdk/gcloud/reference/compute/images/create
2024-01-28 19:14:51 <minimal> that supports both ARM64 and X86_^$ for "--architecture"
2024-01-28 19:16:11 <minimal> and --guest-os-features to specify UEFI_COMPATIBLE if relevant
2024-01-28 20:22:55 <tomalok> minimal: looks like the CLI version of what i've been trying, but worth a shot at any rate
2024-01-28 20:27:57 <minimal> looks like Google have reinvented MUDs, "you are in a maze of twisty corridors, which way next?" lol
2024-01-28 20:28:17 <tomalok> i have been nearly eaten by a grue
2024-01-28 20:29:08 <minimal> which is why you should never a grue to anything.......ba boom tish! ;-)
2024-01-28 21:54:35 <tomalok> success launching a gcp aarch64 image!
2024-01-28 22:04:24 <tomalok> success with x86_64 bios
2024-01-28 22:04:38 <tomalok> thank you minimal
2024-01-28 22:18:51 <minimal> tomalok: I started to collect CLI commands for various Cloud Providers to upload images to buckets, create available AMIs/image from them etc but never got around to trying to figure it out for GCP so far
2024-01-30 04:41:19 <tomalok> ikke, ncopa - 3.19.1 (etc.) cloud images have been releaesd -- MR https://gitlab.alpinelinux.org/alpine/infra/alpine-mksite/-/merge_requests/74
2024-01-30 12:29:46 <ikke> tomalok: The images have been synced
2024-01-30 16:18:01 <tomalok> minimal: not sure this issue belongs in alpine-cloud-images -- if so, could you redirect them? https://gitlab.alpinelinux.org/alpine/cloud/alpine-cloud-images/-/issues/153
2024-01-30 16:18:37 <ikke> always fun, issues with these large screenshots
2024-01-30 16:19:13 <tomalok> they'll look smaller on larger screens
2024-01-30 16:22:23 <minimal> tomalok: perhaps I'll look at it later, at the moment I'm really not in the mood to do any Alpine-related activities (due to being extremely annoyed with Alpine "issues")
2024-01-31 19:17:16 <ncopa> tomalok: thanks!
2024-01-31 19:35:31 <tomalok> ncopa: np - sorry it took a while, but i figured it'd be good to make sure they actually worked on azure, oci, and gcp first ;)