2022-11-12 20:38:25 <tomalok> 3.17.0_rc1 cloud images now available for testing in us-west-2...
2022-11-12 20:38:43 <tomalok> * ami-0238b4958bf8fb28b	alpine-3.17.0_rc1-aarch64-uefi-cloudinit-r0
2022-11-12 20:38:52 <tomalok> ami-0d796ed4292263c79	alpine-3.17.0_rc1-aarch64-uefi-tiny-r0
2022-11-12 20:39:07 <tomalok> * ami-01bdd3da76cdde61b	alpine-3.17.0_rc1-x86_64-bios-cloudinit-r0
2022-11-12 20:39:15 <tomalok> * ami-0ad24203055443786	alpine-3.17.0_rc1-x86_64-bios-tiny-r0
2022-11-12 20:39:28 <tomalok> * ami-002ad29c5ea7ae544	alpine-3.17.0_rc1-x86_64-uefi-cloudinit-r0
2022-11-12 20:39:43 <tomalok> * ami-03b91a626b104d9de	alpine-3.17.0_rc1-x86_64-uefi-tiny-r0
2022-11-12 20:40:12 <tomalok> ...these are AWS, btw.
2022-11-12 20:40:29 <tomalok> ncopa ikke mcrute mps minimal
2022-11-12 20:42:17 <tomalok> i've successfully launched test instances for each of these and SSH'd in just fine -- although the 'cloudinit' images seem to want to try to talk to the IMDS before the network is up, delaying boot by about 2m (this is maybe something that should be fixed in the cloudinit APK)
2022-11-12 20:43:34 <tomalok> new for 3.17- we're rolling UEFI images for x86_64 (now that https://alpinelinux.org/clouds can show multiple flavors of x86_64 images)
2022-11-12 20:43:49 <tomalok> s/clouds/cloud/
2022-11-12 20:47:30 <tomalok> i don't know if i'm going to have it ready before 3.17 is officially released, but i am working on providing images for 'NoCloud' (fairly well tested at this point), and "beta" images for Azure, GCP, and OCI...  the difference between these and AWS images is that the images will be downloadable for folks to import them themselves - we won't be (at
2022-11-12 20:47:30 <tomalok> first, at least) importing and publishing to cloud provider regions -- FWIW that makes no sense for NoCloud anyway!  ;)
2022-11-12 20:52:53 <tomalok> anyways, issues can be reported via https://gitlab.alpinelinux.org/alpine/cloud/alpine-cloud-images/-/issues
2022-11-12 20:59:22 <minimal> tomalok: cloud-init typically should use DHCP to get to the IMDS before it fetches the "proper" network config from there
2022-11-12 21:00:00 <tomalok> right, but without the 'lo' interface up, it's not able to do that
2022-11-12 21:00:11 <minimal> tomalok: if you enable debugging for cloud-init then I could have a look at the resultant /var/log/cloud-init.log to see why the delay
2022-11-12 21:00:40 <minimal> lo? I'd expect it to use dhcp on eth0
2022-11-12 21:00:56 <tomalok>  * Cleaning /tmp directory ... [ ok ]
2022-11-12 21:00:56 <tomalok> 2022-11-12 20:19:39,624 - url_helper.py[WARNING]: Exception(s) [UrlError("HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /latest/api/token (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xffffa0f28f40>: Failed to establish a new connection: [Errno 101] Network unreachable'))"),
2022-11-12 21:00:56 <tomalok> Cloud-init v. 22.3.4 running 'init-local' at Sat, 12 Nov 2022 20:19:19 +0000. Up 9.77 seconds.
2022-11-12 21:00:56 <tomalok> UrlError("HTTPConnectionPool(host='fd00:ec2::254', port=80): Max retries exceeded with url: /latest/api/token (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xffffa0f29cf0>: Failed to establish a new connection: [Errno -2] Name does not resolve'))")] during request to http://[fd00:ec2::254]/latest/api/token, raising
2022-11-12 21:00:56 <tomalok> last exception
2022-11-12 21:00:57 <tomalok> 2022-11-12 20:19:39,631 - url_helper.py[WARNING]: Calling 'None' failed [0/120s]: request error [HTTPConnectionPool(host='fd00:ec2::254', port=80): Max retries exceeded with url: /latest/api/token (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xffffa0f29cf0>: Failed to establish a new connection: [Errno -2] Name does
2022-11-12 21:00:57 <tomalok> not resolve'))]
2022-11-12 21:00:59 <tomalok> [...]
2022-11-12 21:01:28 <tomalok>  * Starting busybox syslog ... [ ok ]
2022-11-12 21:01:28 <tomalok> 2022-11-12 20:21:43,487 - url_helper.py[ERROR]: Timed out, no response from urls: ['http://169.254.169.254/latest/api/token', 'http://[fd00:ec2::254]/latest/api/token']
2022-11-12 21:01:28 <tomalok>  [ ok ]
2022-11-12 21:01:28 <tomalok> 2022-11-12 20:21:43,487 - DataSourceEc2.py[WARNING]: IMDS's HTTP endpoint is probably disabled
2022-11-12 21:01:28 <tomalok>  * Setting hostname ... [ ok ]
2022-11-12 21:01:29 <tomalok>  *   eth0 ...udhcpc: started, v1.35.0
2022-11-12 21:01:29 <tomalok>  * Starting networking ... *   lo ... [ ok ]
2022-11-12 21:01:31 <tomalok> udhcpc: broadcasting discover
2022-11-12 21:01:31 <tomalok> udhcpc: broadcasting select for 172.31.56.96, server 172.31.48.1
2022-11-12 21:01:33 <tomalok> udhcpc: lease of 172.31.56.96 obtained from 172.31.48.1, lease time 3600
2022-11-12 21:01:33 <tomalok>  [ ok ]
2022-11-12 21:02:19 <minimal> ah, a thought, cloud-init in general assumes dhclient is being used for DHCP
2022-11-12 21:02:33 <minimal> perhaps try installing that and see if it makes a difference?
2022-11-12 21:03:05 <minimal> There's talk of supporting something else in future as ISC dhclient is now out of support
2022-11-12 21:03:08 <tomalok> maybe its openrc script needs a "needs" or "before"
2022-11-12 21:03:24 <tomalok> (for general network or generic dhcp)
2022-11-12 21:03:46 <minimal> tomalok, I mean do you have the Alpine dhclient package installed?
2022-11-12 21:04:07 <minimal> cloud-init explicitly runs dhclient binary for its own temporary dhcp
2022-11-12 21:04:26 <tomalok> nope, we try to not install things beyond what's in the base if possible
2022-11-12 21:04:51 <minimal> right, but as a said upstream cloud-init assumes dhclient is available for its own use
2022-11-12 21:04:55 <tomalok> lemme check if dhclient gets added for cloudinit
2022-11-12 21:05:21 <tomalok>   cloud-init          = true
2022-11-12 21:05:21 <tomalok> packages {
2022-11-12 21:05:21 <tomalok>   e2fsprogs-extra     = true  # for resize2fs
2022-11-12 21:05:21 <tomalok>   openssh-server-pam  = true
2022-11-12 21:05:21 <tomalok> }
2022-11-12 21:05:28 <tomalok> just those three for cloud-init, currently
2022-11-12 21:06:02 <minimal> I don't have it as an explicit dep of the cloud-init, if I added deps for everything that cloud-init could possibly use the dep list would be a fair bit larger
2022-11-12 21:06:39 <minimal> so for everything other than NoCloud images try adding dhclient package
2022-11-12 21:06:42 <tomalok> i mean it eventually unblocks and is SSH-able, just is delayed.
2022-11-12 21:07:10 <minimal> tomalok: but it doesn't get to the metadata server
2022-11-12 21:08:14 <tomalok> so it just disables IMDS if it can't reach it?  (eventually it did install the SSH keys properly)
2022-11-12 21:09:02 <minimal> tomalok: if your imkage, the /etc/cloud/cloud.cfg file will define a list of DataSources to try in sequence
2022-11-12 21:09:33 <minimal> so you'll have Ec2 in there (for AWS), and probably "None" which it will fallback to if Ec2 doesn't get to metadata server
2022-11-12 21:10:29 <tomalok> for AWS images, datasource_list: [ "Ec2"] 
2022-11-12 21:11:06 <minimal> right, so you only have that enabled, so then as metadata access didn't work then it doesn't have any working DataSource
2022-11-12 21:11:23 <tomalok> and falls back to Ec2 later?
2022-11-12 21:11:28 <minimal> no
2022-11-12 21:11:45 <tomalok> then how'd it get the ssh keys from IMDS so i could log in?
2022-11-12 21:12:16 <minimal> if you wanted a more generic image to fork for AWS and Azure you could define it as "['Ec2', 'Azure']" or "['Ec2', 'Azure', 'None']"
2022-11-12 21:12:42 <minimal> tomalok: I have no idea, as you sure they aren't baked into the image?
2022-11-12 21:12:49 <tomalok> 100% sure
2022-11-12 21:13:12 <tomalok> (seeing as how i created the keypair after building the images)
2022-11-12 21:14:34 <minimal> well if cloud-init somehow set it up then debug-enabled logs would show it creating/adding stuff to authorized_keys file
2022-11-12 21:15:08 <minimal> for whichever is your default user
2022-11-12 21:15:26 <tomalok> so i'll try adding dhclient and adding that to services (and not start udhcpc)
2022-11-12 21:15:46 <tomalok> but, fwiw, the cloudinit-final did this...
2022-11-12 21:15:59 <tomalok> Cloud-init v. 22.3.4 running 'modules:final' at Sat, 12 Nov 2022 20:21:56 +0000. Up 166.83 seconds.
2022-11-12 21:15:59 <tomalok> ci-info: +++++++++++++++++++++++++++++++Authorized keys from /home/alpine/.ssh/authorized_keys for user alpine++++++++++++++++++++++++++++++++
2022-11-12 21:15:59 <tomalok> ci-info: |   Keytype   |                                       Fingerprint (sha256)                                      | Options | Comment |
2022-11-12 21:15:59 <tomalok> ci-info: +-------------+-------------------------------------------------------------------------------------------------+---------+---------+
2022-11-12 21:15:59 <tomalok> ci-info: +-------------+-------------------------------------------------------------------------------------------------+---------+---------+
2022-11-12 21:16:01 <tomalok> ci-info: | ssh-ed25519 | f4:ba:b1:ca:43:c1:cc:3f:46:85:42:e6:b0:cf:0d:a4:81:2d:98:94:bd:f8:89:3d:39:d7:e1:e2:73:01:04:b0 |    -    | tomalok |
2022-11-12 21:16:01 <tomalok> ci-info: +-------------+-------------------------------------------------------------------------------------------------+---------+---------+
2022-11-12 21:16:40 <tomalok> so maybe it retried the datasources again after the network was up
2022-11-12 21:17:30 <tomalok> (a fair bit of info in the console log, without enabling debug)
2022-11-12 21:18:37 <minimal> tomalok: that log is showing you it created keys for that user, that is a completely different thing from putting a different user's public key into the alpine's user's authorized_key file which is what I thought you were referring to
2022-11-12 21:19:09 <minimal> creation of a user's own keypair is nothing to do with a metadata server
2022-11-12 21:19:51 <tomalok> the user is 'alpine' the key it's getting from AWS has a comment containing 'tomalok'
2022-11-12 21:19:52 <minimal> ah, sorry, misread that output
2022-11-12 21:20:01 <tomalok> ;)
2022-11-12 21:20:28 <tomalok> anyways, let's see if i can fix the situation for the cloudinit images
2022-11-12 21:20:45 <minimal> am still confused, that would be from user-data which I don't see how it got it if it couldn't get to the metadata server
2022-11-12 21:21:26 <minimal> anyway, add dhclient package for all non-NoCloud images and that should get the temporary DHCP settings working
2022-11-12 21:21:49 <ikke> Ah, those nocloud images would be interesting for us as well
2022-11-12 21:22:14 <ikke> We still have some sponsored hosting on AWS we need to start using. 
2022-11-12 21:22:44 <minimal> ikke: NoCloud is a lot simpler as it just uses an ISO/disk partition with a few YAML files in it for config
2022-11-12 21:24:12 <minimal> tomalok: basically if you can enable debugging (edit /etc/cloud.cfg.d/05-logging.cfg) and put cloud-init.log somewhere then I can have a look at it
2022-11-12 21:25:12 <minimal> tomalok: once the upstream cloud-init guys make a start on supporting a DHCP client other than dhclient (I recommended dhcpcd to them) then I intend to add udhcpc support
2022-11-12 21:27:01 <tomalok> https://gitlab.alpinelinux.org/alpine/aports/-/blob/master/community/cloud-init/README.Alpine#L244-247
2022-11-12 21:27:34 <tomalok> does ifupdown-ng autodetetct dhclient and automagically uses it instead of udhcpc?
2022-11-12 21:28:50 <ikke> It should, yes
2022-11-12 21:29:17 <minimal> tomalok: yes I wrote the README :-) it says other client do work (although perhaps not as fully), so not as fully refers to cloud-init itself not using other clients for its temporary DHCP need to get to metadata server
2022-11-12 21:29:50 <minimal> yes ifupdown-ng takes care of that (forget which file defines the various clients it tries and in which sequence)
2022-11-12 21:30:06 <tomalok> ikke: cool, so just adding that apk should be sufficient.  let's see how that does, then...  minimal: then the current udhcpc situation is hereby 'not fully' working ;) )
2022-11-12 21:30:58 <minimal> tomalok: the README as you saw did point out cloud-init assumes the use of dhclient ;-)
2022-11-12 21:31:02 <tomalok> if/when cloud-init upstream switches, we'll just install whatever dhcp client it switched to...  LMK when that happens
2022-11-12 21:31:34 <ikke> minimal: I recall it being the dhcp executor 
2022-11-12 21:31:42 <minimal> tomalok: won't be a little while, the latest release was due out the past week but now should make it this coming week
2022-11-12 21:31:56 <minimal> releae of cloud-init that is, 22.4
2022-11-12 21:33:58 <minimal> I've got some testing to do to see how mdev/mdevd works (so I can remove eudev dep)
2022-11-12 21:34:16 <minimal> and I have code written to remove shadow dep
2022-11-12 21:34:42 <minimal> though that won't make 22.4 release
2022-11-12 21:35:11 <minimal> ikke: sounds about right
2022-11-12 21:37:46 <tomalok> how compatible is mdev to mdevd -- use the same config file?
2022-11-12 21:38:11 <tomalok> (Tiny Cloud adds some mdev rules)
2022-11-12 21:39:11 <minimal> yes, in Edge (soon 3.17) the mdev-related config moved to the mdev-conf package for this reason as both mdev and mdevd depend on it
2022-11-12 21:40:31 <tomalok> as long as the format and location of the config file is the same, should be good
2022-11-12 21:41:26 <tomalok> is mdevd the default in edge/3.17 or an option (at this point)?
2022-11-12 21:41:42 <tomalok> (i should have looked when i had launched the 3.17.0_rc1 images)
2022-11-12 21:54:30 <tomalok> looks like simply adding `dhclient` package fixed it...  waiting for EC2 system log to get updated to see all the console output
2022-11-12 22:00:13 <tomalok> minimal: did the trick.  cloud-init takes 5s instead of 2m now
2022-11-12 22:03:35 <tomalok> Updated list of 3.17.0_rc1 AWS images (us-west-2)...
2022-11-12 22:03:38 <tomalok> * ami-0dedc734de2f89846 alpine-3.17.0_rc1-aarch64-uefi-cloudinit-r0
2022-11-12 22:03:38 <tomalok> * ami-037ced78f26516035 alpine-3.17.0_rc1-x86_64-bios-cloudinit-r0
2022-11-12 22:03:38 <tomalok> * ami-0ad24203055443786 alpine-3.17.0_rc1-x86_64-bios-tiny-r0
2022-11-12 22:03:38 <tomalok> * ami-0d796ed4292263c79 alpine-3.17.0_rc1-aarch64-uefi-tiny-r0
2022-11-12 22:03:38 <tomalok> * ami-06178578b1a74652d alpine-3.17.0_rc1-x86_64-uefi-cloudinit-r0
2022-11-12 22:03:39 <tomalok> * ami-03b91a626b104d9de alpine-3.17.0_rc1-x86_64-uefi-tiny-r0
2022-11-12 22:41:31 <minimal> tomalok: mdev is still the default for Alpine
2022-11-12 22:41:47 <minimal> you can use setup-devd to select any of the 3, mdev, mdevd, or eudev
2022-11-13 19:59:36 <tomalok> ncopa ikke - i'm working on adding checksums to cloud images -- is there a preference between sha256 or sha512?
2022-11-13 20:11:38 <ikke> tomalok: good question. For alpine ISOs we offer both
2022-11-13 20:25:53 <tomalok> ikke: since this is new i'd like to standardize and move forward with one or the other...  but wouldn't be the end of the world to offer both...
2022-11-13 23:09:47 <tomalok> ikke: example storage for cloud images and metadata -- https://dev.alpinelinux.org/~tomalok/alpine-cloud-images/edge/cloud/aws/ -- this is just for today's edge builds...  could, in theory, add one more subdir for ARCH (similar to the CDN's /alpine/VERSION/releases/ARCH/ -- might be an especially good thing if we build NoCloud images for archs
2022-11-13 23:09:47 <tomalok> beyond x86_64 and aarch64...
2022-11-14 00:43:26 <minimal> tomalok: any particular reason the AWS images are VHD files? rather than RAW or QCOW2 or something else
2022-11-14 01:05:17 <tomalok> minimal: that's the best format for importing them into AWS.
2022-11-14 01:05:48 <tomalok> RAW's definitely going to be too big.  QCOW isn't directly supported for import
2022-11-14 01:06:35 <tomalok> other clouds may support QCOW import (I think OCI is one) so images for those (and obviously NoCloud) would be QCOW format
2022-11-14 01:07:25 <tomalok> s/QCOW/QCOW2/
2022-11-14 01:18:21 <minimal> ok
2022-11-18 00:46:34 <minimal> tomalok: upgrade to cloud-init package submitted, !41502. This version works with mdev or mdev as well as eudev
2022-11-18 00:46:58 <minimal> I also updated the README.Alpine to clarify the dhclient situation
2022-11-18 00:48:03 <minimal> oops, mdev or mdevd
2022-11-18 01:09:20 <tomalok> minimal: thanks for the heads up -- i expect that this version will be available in edge and 3.17, but not backported to 3.16?
2022-11-18 01:10:15 <tomalok> (and earlier)
2022-11-18 01:12:05 <minimal> correct. The Alpine changes that make mdev/mdevd possible are only in Edge/v3.17
2022-11-18 01:12:27 <minimal> I don't tend to backport cloud-init releases
2022-11-18 01:14:57 <minimal> tomalok: realistically I'd expect you to stil use udev/eudev for AWS as cloud-init's network interface hotplug stuff in the AWS DataSource assumes udev is in use
2022-11-23 00:16:48 <tomalok> ncopa: i guess it's time to roll 3.17.0 images (and 3.16.2), eh?
2022-11-23 00:29:27 <tomalok> well, 3.16.3, actually
2022-11-23 06:20:46 <ncopa> Yes! 👍🏽 
2022-11-23 06:23:23 <Solarbaby> ncopa, congrats maybe?  
2022-11-23 06:28:03 <tomalok> ncopa, ikke: i've got all the various AWS cloud images built, imported, and published -- i'll have a alpine-mksite MR tomorrow to update the clouds page.
2022-11-23 06:28:23 <ikke> alright
2022-11-23 14:38:49 <tomalok> ikke, ncopa: https://gitlab.alpinelinux.org/alpine/infra/alpine-mksite/-/merge_requests/55
2022-11-23 15:12:32 <tomalok> ncopa: thx