2024-02-03 18:14:47 <ayakael> I seem to be unable to join alpine-{infra,devel,linux,offtopic} channels due to Need to be identified and verified to join this channel error
2024-02-03 18:15:09 <ayakael> Anyone else has this issue?
2024-02-03 18:15:42 <minimal> those channels' settings were changed several days ago due to spam
2024-02-03 18:15:59 <ikke> but ayakael seems to be identified
2024-02-03 18:16:24 <minimal> some IRC clients don't autojoin such channels
2024-02-03 18:16:41 <minimal> i.e. I haven't figured out how to do so with HexChat
2024-02-03 18:16:50 <ayakael> Hmm, I'm using thelounge. I'll try leaving the channel and coming back
2024-02-03 18:17:07 <ayakael> ikke: also, thanks for the prometheus trick :)
2024-02-03 18:17:36 <ikke> np
2024-02-03 18:19:00 <ayakael> Looks like that did it, thanks for the help :)
2024-02-09 23:03:38 <Ermine> minimal: my logs https://pastesite.org/view/raw/3187783f . I was concerned about messages 'dhcpd/dhclient not found', but it proceeds to udhcpcd anyway
2024-02-09 23:14:29 <minimal> Ermine: those messages are fine, you can ignore them, cloud-init will use whichever of dhclient/dhcpcd/udhcpc that it finds
2024-02-09 23:15:22 <minimal> however the errors a couple of lines lower down don't look good
2024-02-09 23:15:45 <minimal> is this VM running on GCE or VMware?
2024-02-09 23:17:57 <Ermine> Idk. But its datasource is Ec2
2024-02-09 23:18:19 <minimal> you don't know which hypervisor/Cloud provider it is running on
2024-02-09 23:18:21 <minimal> ?
2024-02-09 23:18:59 <minimal> also this doesn't look like cloud-init.log output, it looks like syslog output
2024-02-09 23:19:07 <minimal> can you provide the cloud-init.log?
2024-02-09 23:19:19 <Ermine> I know which is my provider (yandex cloud), but not their hypervisor
2024-02-09 23:20:31 <minimal> well in that scenario the hypervisor isn't relevant then
2024-02-09 23:21:04 <minimal> the eni renderering errors don't look good
2024-02-09 23:22:47 <Ermine> https://pastesite.org/view/raw/3d9cb03b
2024-02-09 23:31:56 <minimal> Ermine: have you "tailored" (i.e. modified) either /etc/cloud.cfg or added extra files in /etc/cloud.cfg.d/ for this VM image?
2024-02-09 23:32:38 <Ermine> I don't think so
2024-02-09 23:33:20 <minimal> so then I don't understand why it searched through a list of network renderers, the Alpine package only specifies "eni" to be used
2024-02-09 23:33:39 <minimal> is this a OS disk image you created yourself?
2024-02-09 23:35:38 <Ermine> Or... actually I did modify /etc/cloud/cloud.cfg
2024-02-09 23:36:36 <Ermine> But system_info.network.renderers is ['eni']
2024-02-09 23:40:08 <minimal> you packaged this disk image yourself?
2024-02-09 23:40:18 <Ermine> Yes
2024-02-09 23:40:29 <minimal> how did you create the disk image?
2024-02-09 23:41:46 <Ermine> Installed alpine on qcow2 disk, installed s6-rc and prepared a service set, then uploaded it to cloud provider storage
2024-02-09 23:42:08 <Ermine> I don't remember details, it was almost 2 years ago
2024-02-09 23:42:50 <minimal> basically I'm asking if you ever booted the disk image after installing/enabling cloud-init BEFORE you then uploaded it
2024-02-09 23:43:19 <minimal> so in otherwords did cloud-init run at any time on the disk image BEFORE it was uploaded
2024-02-09 23:43:20 <Ermine> No
2024-02-09 23:44:08 <minimal> ok, and you just did "apk add cloud-init" and setup-cloud-init?
2024-02-09 23:45:43 <Ermine> Yes
2024-02-09 23:46:53 <Ermine> And wrote my s6-rc services for cloud-init, which mimick apline's openrc services
2024-02-09 23:49:22 <Ermine> There was a bunch of issues when I started the vps
2024-02-09 23:49:59 <Ermine> which i've resolved in place (with your help)
2024-02-09 23:52:07 <minimal> can you pastebin the /etc/cloud/cloud.cfg file?
2024-02-09 23:55:04 <Ermine> minimal: https://pastesite.org/view/raw/f904276e
2024-02-09 23:57:37 <minimal> why is the default_user: section commented out?
2024-02-09 23:58:15 <minimal> also WHERE is the renderer line? it is missing from the "network:" section
2024-02-10 00:01:21 <Ermine> Oh, missed it. Correct version: https://pastesite.org/view/raw/9a5d8f93
2024-02-10 00:02:09 <Ermine> minimal: I guess to prevent cloud-init from creating alpine user? Anyway, it shouldn't affect networks
2024-02-10 00:03:01 <minimal> "Correct version"? you have multiple files? are any of those in /etc/cloud.cfg.d/ directory (as those are merged with /etc/cloud/cloud.cfg
2024-02-10 00:03:54 <minimal> "I  guess to prevent cloud-init from creating alpine user? Anyway, it shouldn't affect networks" - cloud.cfg is a YAML file - indentation is important - you've put comments at line char 1 - which then affects the parsing of further lines...
2024-02-10 00:04:25 <minimal> so "network:" is no longer treated as being "system_info.network"
2024-02-10 00:04:45 <minimal> and "paths:" is no longer treated as "system_info.paths"
2024-02-10 00:05:05 <minimal> also if you don't want the default user then control that via the user_data you provide...
2024-02-10 00:08:55 <minimal> so that would explain why the logs mention a full list of (default) renderers as it cloud-init cannot "see" the ENI specified in /etc/cloud/cloud.cfg due to the comments above it
2024-02-10 00:10:08 <Ermine> <minimal> "Correct version"? you have multiple files? are any of those in /etc/cloud.cfg.d/ directory (as those are merged with /etc/cloud/cloud.cfg --- no, this is the single file, but I failed to paste renderers line
2024-02-10 00:10:26 <minimal> from the Wikipedia YAML page: "Whitespace indentation is used for denoting structure"
2024-02-10 00:10:32 <Ermine> <minimal> "I  guess to prevent cloud-init from creating alpine user? Anyway, it shouldn't affect networks" - cloud.cfg is a YAML file - indentation is important - you've put comments at line char 1 - which then affects the parsing of further lines... --- oh wow...
2024-02-10 00:10:57 <minimal> that's a fundamental aspect of YAML
2024-02-10 00:11:09 <minimal> in the same way that python code also relies on indentation
2024-02-10 00:12:04 <Ermine> python ignores indentation of comments
2024-02-10 00:12:12 <Ermine> iirc
2024-02-10 00:12:22 <minimal> ok I just used python as another example
2024-02-10 00:12:53 <Ermine> So I now have zero respect towards yaml inventors
2024-02-10 00:13:04 <Ermine> Anyway, thank you!
2024-02-10 00:13:22 <minimal> YAML gets worse, lol.........like the Version header handling lol
2024-02-10 00:14:45 <minimal> and sexagesimal handling between YAML versions lol
2024-02-10 00:16:25 <minimal> which is why MAC addresses in cloud-init YAML configs have to be quoted as YAML 1.1 parser may think they're a sexagesimal number instead
2024-02-10 00:17:40 <minimal> so I don't know if the YAML comments is the old problem you're having but it certainly is a problem
2024-02-10 00:18:15 <minimal> if you don't want the default user then in your user-data have a "users:" section but DO NOT specify a "- default" entry and the default user will not be created
2024-02-10 00:22:35 <minimal> also the same comment non-indentation applies to "cloud_config_modules:" where you commented out "set_passwords" (why? if you don't specify any passwords then obviously there'd be none to set)
2024-02-10 00:23:26 <minimal> running "yamllint" against your cloud.cfg file gives:
2024-02-10 00:23:53 <minimal>   55:1      warning  comment not indented link content  (comments-indentation)
2024-02-10 00:24:12 <minimal>   90:1      warning  comment not indented link content  (comments-indentation)
2024-02-10 00:26:10 <minimal> Ermine: ^^^
2024-02-27 22:13:56 <ikke> tomalok: I've tried to deploy an azure vm from the https://dl-cdn.alpinelinux.org/alpine/v3.19/releases/cloud/azure_alpine-3.19.1-x86_64-uefi-cloudinit-r0.vhd image. It deploys succesfully, I see in the serial console it finds a key from imds, but trying to login with alpine and the private key is rejected, any idea?
2024-02-27 22:14:28 <ikke> I let azure generate the key, and told ssh to use that key
2024-02-27 22:41:09 <minimal> ikke: did the console show that cloud-init completed with no errors?
2024-02-28 05:00:21 <tomalok> ikke: i hadn't tried an azure-generated key, I'd always pasted my pubkey in on launch...  i know tiny-cloud was working, i'm pretty sure i logged in with cloud-init too (but not 100%)
2024-02-28 05:00:57 <ikke> Tiny cloud failed for me, gave a timeout 
2024-02-28 05:04:37 <tomalok> odd. virtual machine type?  used the newish image gallery instead of the old image import?  default disk storage or some additional?
2024-02-28 05:36:35 <ikke> vm: b2s, ssd storage, no additional disks
2024-02-28 05:37:15 <ikke> The iso is stored in a blob and then an image created out of it (not sure what the old image import is)
2024-02-28 05:37:26 <ikke> tomalok: the serial console did show that the vm was running
2024-02-28 06:35:12 <ikke> tomalok: it works with cloud-init and self generated keys
2024-02-28 16:24:43 <tomalok> ikke: so the question is what's different with azure-generated keys?
2024-02-28 17:42:00 <ncopa> I miss aws images with linux-lts kernel for a1.metal
2024-02-28 17:42:23 <ncopa> debian is not playing nice with me today
2024-02-28 17:58:00 <ncopa> reminds me that we should ask amazon to sponsor the storage for our AMIs
2024-02-28 17:58:36 <minimal> ikke: so you managed to login via the serial console? did /var/log/cloud-init.log give any idea of what happened regarding the SSH keys?
2024-02-28 17:59:17 <minimal> ncopa: I did suggest putting you in contact with someone regarding this shortly after the cloud-init summit...
2024-02-28 18:00:11 <ncopa> yeah, i have just not been able to follow up yet
2024-02-28 18:00:36 <ncopa> and i dont want to do that without tomalok's blessing either
2024-02-28 18:00:49 <ncopa> or the cloud team's blessing
2024-02-28 18:01:31 <minimal> I believe I also mentioned it to him at the time
2024-02-28 18:02:17 <ncopa> noah, right? I talked with him on IRC today
2024-02-28 18:02:49 <minimal> yes, he is both (a) involved with Debian's cloud image across various providers, and (b) works for AWS AFAIK
2024-02-28 18:03:08 <ncopa> aha...
2024-02-28 18:05:02 <ikke> minimal: I recreated the VM with self generated keys
2024-02-28 18:05:06 <ikke> so no way to look at the logs
2024-02-28 18:05:48 <ikke> It's uncertain that the Azure program will be extended btw, we have an extension until may, after which we will hear more
2024-02-28 18:06:02 <minimal> from https://github.com/canonical/cloud-init/blob/main/doc/summit/2023_summit_shared_notes.md
2024-02-28 18:06:10 <minimal> "[ACTION] Noah M can help Dermot get in touch with right people for open source engagement to start talks about open source sponsor accounts for hosting "official images"."
2024-02-28 18:07:20 <minimal> ikke: it's not clear to me how Azure passes Azure generated keys to cloud-init, I'm assuming via vendor-data
2024-02-28 18:11:39 <minimal> ikke: some of the Azure functionality may rely on WAAgent, I don't know if this is one of those things
2024-02-28 18:21:32 <minimal> ikke: looks like cloud-init should get the Azure-generated SSH key via a OvfEnv.xml file passed to it - so if you try again this should show up in the cloud-init logs
2024-02-28 18:26:37 <minimal> if that works then the cloud-init.log should contain a "Retrieved X fingerprints from OVF" entry
2024-02-28 18:30:01 <ikke> minimal: I did see a message like that
2024-02-28 18:30:14 <ikke> At least, in the serial console, I saw it found a key 
2024-02-28 18:30:16 <ikke> at boot
2024-02-28 18:31:25 <minimal> hmm, not sure if that particular message would go to the console rather than, or in addition, to the logfile
2024-02-28 18:32:31 <minimal> if it *DID NOT* appear in the logs then I'd suspect perhaps a DHCP client related issue (as Azure uses a non-standard DHCP option to indicate the Wireserver's IP address)
2024-02-28 18:33:22 <minimal> e.g. the cloud-init people raised a PR with dhcpcd recently regarding adding support for this Azure-specific DHCP option
2024-02-28 18:34:01 <ikke> I could reach the server, it just did not accept the key
2024-02-28 18:34:05 <minimal> I don't remember about how udhcpc and dhclient handle it
2024-02-28 18:36:34 <minimal> ikke: well after that log message there there should be a "Successfully created reported ready marker file while in the preprovisioning pool" log entry
2024-02-28 18:37:09 <ikke> minimal: I cannot login via serial console, so I cannot check any logs, just what is shown in the serial console during bootstrap
2024-02-28 18:38:47 <minimal> ikke: you'd need to specify in user-data that whichever account has an unlocked password
2024-02-28 18:39:11 <ikke> How? I can provide userdata, but not idea what to provide
2024-02-28 18:41:16 <minimal> in the user-data, when you create/modify a user then as one of the specifc user attribute you'd specify "lock_passwd: false" - that's standard cloud-init configuration
2024-02-28 18:41:43 <minimal> it defaults to "true" so any created accounts have locked passwords
2024-02-28 18:42:17 <minimal> https://cloudinit.readthedocs.io/en/latest/reference/modules.html#users-and-groups
2024-02-28 18:43:07 <ikke> userdata is just an empty field
2024-02-28 18:44:01 <ikke> https://i.imgur.com/HyYUI3i.png
2024-02-28 18:44:10 <minimal> ok, so then it's just doing the default behaviour which includes creating an "alpine" used with a locked password
2024-02-28 18:44:24 <minimal> so you can provide user-data to override that
2024-02-28 18:44:26 <ikke> https://imgur.com/tlLd2S8
2024-02-28 18:45:42 <ikke> minimal: would I need to provide every detail of a user if I override it, or just the parts that I want to override?
2024-02-28 18:46:07 <minimal> so this should do it
2024-02-28 18:46:12 <minimal> #cloud-config
2024-02-28 18:46:16 <minimal> users:
2024-02-28 18:46:20 <minimal>   - default
2024-02-28 18:46:30 <minimal>     lock_passed: false
2024-02-28 18:46:59 <ikke> And would that be custom data?
2024-02-28 18:47:01 <ikke> or userdata
2024-02-28 18:47:06 <minimal> so that just overrides the lock_passwd (typo above) value for the default ("alpine") user
2024-02-28 18:47:10 <minimal> user-data
2024-02-28 18:47:14 <ikke> ok
2024-02-28 18:48:08 <minimal> I'm not completely what Azure "custom data" is, I suspect it relates to WAAgent
2024-02-28 18:48:43 <minimal> ikke: indentation is important as user-data is YAML
2024-02-28 19:20:58 <ikke> It neither accepts the ssh key now, nor does it allow me to login via serial. It seems like I forgot to set the local user, it uses `azureuser` now, but does not allow me to login. I guess the iso is hardcoded for the alpine user
2024-02-28 19:22:14 <minimal> ikke: the default user created by cloud-init is specified in the /etc/cloud/cloud.cfg file
2024-02-28 19:22:21 <ikke> https://tpaste.us/j4Ze 
2024-02-28 19:22:35 <minimal> in "system_user:" -> "default_user:" -> "name:"
2024-02-28 19:22:44 <ikke> Invalid format at line 4 column 16
2024-02-28 19:23:17 <minimal> that was my typo which I mentioned earlier ;-)
2024-02-28 19:23:29 <minimal> "lock_passed" => "lock_passwd"
2024-02-28 19:26:06 <minimal> re: the user, unless tomalok changed the default user in /etc/cloud.cfg when creating the images it should be "alpine". Perhaps Azure is creating an additional "azureuser"? If so then replace "default" in the user-data YAML with "name: azureuser"
2024-02-28 19:26:45 <minimal> actually no, KEEP the "- default" and add a line after it with "- name: azureuser"
2024-02-28 19:26:53 <ikke> You can provide the username when creating the vm
2024-02-28 19:26:57 <minimal> otherwise the default user will not be created
2024-02-28 19:27:21 <ikke> The instructions for the iso says that a user called alpine is created
2024-02-28 19:28:03 <minimal> ikke: ok but I'm not sure how Azure then passed that to cloud-init, i.e. does it use that to overrride the default user or to create an additional user? I'd have to dig into the cloud-init Azure DataSource code to see how it's handled
2024-02-28 19:29:06 <minimal> The instructions say that as cloud.cfg specifies the default user, what is not clear is if Azure overrides that if you provide a username when creating a VM
2024-02-28 19:31:50 <minimal> basically behind the scenes cloud-init merges its default config plus any provided config (whether by Azure itself and/or by you) to create a resultant desired config and then it acts on that merged config
2024-02-28 19:32:32 <ikke> in your example, is default also supposed to be a key
2024-02-28 19:32:40 <ikke> now it's a list element
2024-02-28 19:33:21 <ikke> Or would it need to be:
2024-02-28 19:33:25 <ikke> #cloud-config
2024-02-28 19:33:27 <ikke> users:
2024-02-28 19:33:29 <ikke>   - default
2024-02-28 19:33:36 <ikke>   - loc_passwd: false
2024-02-28 19:34:46 <minimal> #cloud-config
2024-02-28 19:34:49 <minimal> users:
2024-02-28 19:34:51 <minimal>   - default
2024-02-28 19:34:57 <minimal>   - name: azureuser
2024-02-28 19:35:04 <minimal>     lock_passwd: false
2024-02-28 19:35:06 <tomalok> ah, i think i recall someone having issues with azure, and they needed to set the "default" user to "alpine" and that solved their problem
2024-02-28 19:35:18 <ikke> tomalok: yeah, it worked when I did that
2024-02-28 19:35:27 <ikke> but the last time I forgot to adjust it
2024-02-28 19:36:09 <minimal> tomalok: set the default user where? you mean specify an "alpine" user when creating the Azure VM?
2024-02-28 19:36:16 <tomalok> fwiw, i'm heading off into the desert for a week or so -- will try to check in (or maybe even will check in more often, we'll see)
2024-02-28 19:36:32 <tomalok> minimal: when creating the VM it's one of the things you set in the UI
2024-02-28 19:36:49 <minimal> and they assume "azureuser" if you don't provide it?
2024-02-28 19:36:59 <ikke> It's the default value
2024-02-28 19:37:10 <tomalok> i ran through it a couple times a month or so ago, but i seem to have overwritten the details in my brain cells
2024-02-28 19:37:44 <minimal> tomalok: ok so then perhaps when you create the Azure cloud-image you should can /etc/cloud.cfg's default user to "azureuser" to avoid confusion?
2024-02-28 19:37:52 <tomalok> should write walkthroughs for using the images on non-AWS -- took me a while to sort out Azure / GCP
2024-02-28 19:38:04 <minimal> s/cloud-image/cloud-init image/
2024-02-28 19:38:22 <tomalok> minimal: i could, but then there's confusion because by default all our default users is alpine  ;)
2024-02-28 19:38:48 <tomalok> iirc, that's true for non-cloud to
2024-02-28 19:38:55 <tomalok> s/to/too/
2024-02-28 19:39:05 <minimal> well if that's the way they behave regardless of distro then should Alpine behave the same as other distros on Azure?
2024-02-28 19:39:27 <minimal> s/should/shouldn't/
2024-02-28 19:40:10 <tomalok> technically the non-AWS clouds are still "beta" so we can consider and weigh this
2024-02-28 19:40:40 <minimal> e.g. if you create Debian VMS on AWS and Azure do they both have a default "debian" user or a cloud-specific user?
2024-02-28 19:41:07 <ikke> minimal: the coud-config was still not accepted
2024-02-28 19:41:49 <ikke> schema.py[WARNING]: Invalid cloud-config provided: Please run 'sudo cloud-init schema --system' to see the schema errors.
2024-02-28 19:41:56 <ikke> But I can now login (with the alpine user
2024-02-28 19:42:08 <tomalok> and ncopa -- we can build "metal" images for cloud too.  there's a config overlay including that in the current alpine-cloud-images -- was hoping to reduce the uefi/bios dimention into a single hybrid before adding a vm/metal dimension
2024-02-28 19:42:30 <tomalok> gotta run, later...
2024-02-28 19:42:36 <minimal> there's pros and cons of hybrid images
2024-02-28 19:42:50 <ikke> minimal: https://tpaste.us/D7pe 
2024-02-28 19:43:34 <minimal> e.g. assuming Grub, any time that Grub is updated then "grub-install" needs to be run *twice* to update both BIOS and UEFI code
2024-02-28 19:43:57 <minimal> ikke: right, so the default user is "alpine"
2024-02-28 19:44:25 <minimal> and also, by default, lock_passwd is false so password-less logins work
2024-02-28 19:57:41 <minimal> ikke: do you see anything in /var/log/cloud-init.log that points to the original problem regarding SSH authorized_keys?
2024-02-28 19:58:00 <ikke> minimal: I have to redeploy the VM
2024-02-28 19:58:14 <ikke> since I cannot login
2024-02-28 19:59:00 <ikke> But if azure creates: 'Authorized keys from /home/azureuser/.ssh/authorized_keys for user azureuser'
2024-02-28 19:59:34 <ikke> while cloud-init is setup to create 'alpine'
2024-02-28 19:59:44 <ikke> then I guess neither account is setup completely
2024-02-28 20:00:31 <minimal> well i'm guessing that is based on the default user you specified when creating the VM, so if you said "alpine" then it should refer to /hpome/alpine/azureuser/.ssh/authorized_keys
2024-02-28 20:01:26 <minimal> anyway the default account, whatever it is called, has sudo/doas privs so once you can klogin you can become root and look around
2024-02-28 20:02:08 <ikke> yes, that's what I have now
2024-02-29 08:55:40 <ncopa> i have an @amazon.com email address we can contact to ask for them to sponsor the AMI storage
2024-02-29 16:20:07 <tomalok> ncopa it would also be handy to know if they provide some way of syncing images into all regions/partitions, like govcloud and china
2024-02-29 16:54:00 <ncopa> tomalok: who is currently paying for the alpine AMIs?
2024-02-29 17:01:23 <tomalok> ncopa: mcrute is...  i've managed to weed out unused/old images these past few months
2024-02-29 17:38:16 <ncopa> do you have the mail address to mcrute?