2023-04-01 00:51:06 <tomalok> thanks psykose
2023-04-01 16:25:21 <mcrute> tomalok: ack, working on that today
2023-04-01 16:31:38 <psykose> ikke: you still have to update the cloud thing i think
2023-04-01 16:32:54 <minimal> yeah we should be on Cloud 2.0 ;-)
2023-04-01 16:33:30 <minimal> subtitle "This time it's personal!" lol
2023-04-01 16:33:33 <psykose> 4.0 i think
2023-04-01 19:00:32 <ikke> psykose: you have to merge master into production and push that 
2023-04-01 19:18:07 <psykose> aha
2023-04-01 19:19:24 <psykose> thanks
2023-04-01 19:20:05 <ikke> wwwtest tracks master 
2023-04-01 19:21:27 <psykose> :)
2023-04-01 19:21:29 <psykose> not a bad setup
2023-04-01 19:21:44 <psykose> and hey it works
2023-04-01 19:21:47 <psykose> thanks, will remember
2023-04-06 12:25:43 <ncopa> tomalok: do you want review PRs to tiny-cloud or do you think I can just merge them? https://gitlab.alpinelinux.org/alpine/cloud/tiny-cloud/-/merge_requests/37
2023-04-06 13:44:07 <tomalok> ncopa, sorry been crazy around here, i'll take a look this morning
2023-04-06 13:53:10 <ncopa> i think i need to update a commit message. I changed the compression header detection, which was "broken"
2023-04-06 14:03:57 <tomalok> still working on clearing the sleep from my eyes, but at a glance it looks like you started implementing a #cloud-config userdata handler?
2023-04-06 14:04:40 <tomalok> ...but it's tied to nocloud specifically
2023-04-06 14:05:07 <ncopa> yes
2023-04-06 14:05:29 <ncopa> the main goal was to add tests for init-main
2023-04-06 14:06:04 <ncopa> but the way tiny-cloud is implemented made it trivial to add support for #cloud-config ssh-keys
2023-04-06 14:06:23 <ncopa> i have only added tests for nocloud so far
2023-04-06 14:06:42 <ncopa> the plan is to add tests for all cloud providers
2023-04-06 14:07:22 <ncopa> i wonder where we can discuss the work ahead. Maybe I should create issues with specific features I'd like to implement?
2023-04-06 14:07:43 <ncopa> i know you are busy and I don't want disturb more than necessary
2023-04-06 14:07:56 <tomalok> the plan was to detect and handle user-data based on the #whatever specified in it -- if it's done at that level, it becomes usable to all clouds
2023-04-06 14:08:33 <ncopa> but I really really want simple #cloud-config support for the 3.18 release
2023-04-06 14:08:43 <tomalok> again, maybe you have the check for cloud-config somewhere and i missed it.
2023-04-06 14:09:32 <ncopa> what other alternatives are there to #cloud-config?
2023-04-06 14:10:08 <tomalok> i think this can be done for 3.18.  just need to find the right place to split the #! vs #cloud-config vs. raw file
2023-04-06 14:10:19 <ncopa> you mean script
2023-04-06 14:10:35 <ncopa> i think that is already handled by imds?
2023-04-06 14:11:00 <tomalok> i think that imds should focus on the Instance Metadata -- this is userdata processing
2023-04-06 14:12:49 <tomalok> compressed? (decompressed) --> script? (run later) --> cloud-config? (do that) --> plain file
2023-04-06 14:13:31 <ncopa> you are talking about this, right? https://gitlab.alpinelinux.org/alpine/cloud/tiny-cloud/-/merge_requests/37/diffs?commit_id=c350542d4a18102aeb9b2f948199f55920fb9964
2023-04-06 14:13:40 <tomalok> there may be different pieces of "do that" (early, middle, late)
2023-04-06 14:14:58 <tomalok> yeah, nocloud uses AWS format for metadata, so it should be possible to give it a metadata file on ci-data that provides SSH keys for the default user.
2023-04-06 14:15:50 <tomalok> but if you want to go beyond what AWS metadata does, then we need to start hooking in #cloud-config support for it.
2023-04-06 14:16:03 <tomalok> s/it/userdata payload/
2023-04-06 14:16:28 <tomalok> so what cloud-config components do you need implmented for 3.18?
2023-04-06 14:17:13 <ncopa> I was trying to do this: https://cloudinit.readthedocs.io/en/latest/reference/modules.html#authorized-keys
2023-04-06 14:18:49 <tomalok> authorized keys for users that aren't necessarily the default alpine user?
2023-04-06 14:19:41 <ncopa> so, the cloud-init config is a bit confusing there. that is the default user that has the ssh_authorized_keys
2023-04-06 14:20:18 <ncopa> for other users, thare is a 'users:' key
2023-04-06 14:20:37 <ncopa> maybe the default users' ssh-key should go to meta-data and not user-data
2023-04-06 14:20:56 <tomalok> (nod) and for that it'd follow AWS metadata
2023-04-06 14:21:19 <tomalok> (which is i believe what it defaults to)
2023-04-06 14:23:26 <ncopa> yes, you are right. "... and meta-data is a YAML-formatted file representing what you’d find in the EC2 metadata service."
2023-04-06 14:23:57 <tomalok> so if you need authorized_keys for just the default user you already have that part.    ;)  anything else in your 3.18 shopping cart?  ;)
2023-04-06 14:24:36 <ncopa> i also need tiny-cloud to autocreate the default user if its missing
2023-04-06 14:24:48 <ncopa> and auto configure the network if network config is missing
2023-04-06 14:25:05 <ncopa> and auto install and enable sshd
2023-04-06 14:26:17 <tomalok> from the cloud image point of view, at least, those are taken care of at the image level.
2023-04-06 14:26:36 <ncopa> yup. but not from alpine iso
2023-04-06 14:30:22 <tomalok> i'll take a look at what we have in alpine-cloud-images today for that.  i think the default net is simply DHCP on eth0.  does cloud-init handle all these things?
2023-04-06 14:30:55 <tomalok> (why not set up these defaults in the alpine iso?)
2023-04-06 14:37:57 <minimal> tomalok: does cloud-init handle what specifically?
2023-04-06 14:38:16 <ncopa> auto configure network and auto enable sshd etc
2023-04-06 14:38:50 <tomalok> * autocreate default user * autoconfig network if no config * auto install/enable sshd
2023-04-06 14:38:58 <minimal> well the network config is either provided by the metadata server or else (for NoCloud_ via either meta-data or network-config YAML file)
2023-04-06 14:39:08 <tomalok> so this is where this (and more) happens when we build the cloud images -- https://gitlab.alpinelinux.org/alpine/cloud/alpine-cloud-images/-/blob/main/scripts/setup#L177
2023-04-06 14:39:12 <minimal> for ssh that is handled in user-data
2023-04-06 14:40:14 <minimal> https://cloudinit.readthedocs.io/en/latest/reference/modules.html#ssh
2023-04-06 14:40:18 <tomalok> so IF there's no network config, what does it do? IF there's no sshd installed, what does it do?  IF there's no default user installed, what does it do?
2023-04-06 14:40:33 <minimal> there's a fallback to DHCP
2023-04-06 14:40:56 <tomalok> that's pretty much what we set up in the cloud image with a default interfaces file for dhcp on eth0
2023-04-06 14:41:20 <minimal> if there's no sshd installed then I assume it does nothing as openssh being installed is assumed
2023-04-06 14:41:44 <tomalok> when we get to tiny cloud we're assuming that's already available too
2023-04-06 14:41:46 <minimal> as for default user, that is created if the config tells it to do so
2023-04-06 14:42:26 <tomalok> and if not, you just have a box with a root user that doesn't allow logins?
2023-04-06 14:42:33 <minimal> i.e. the stock config creates a default user but if you specify a "user:" section in user-data and DO NOT include "- default" as part of that then another other users will be created but not the default user
2023-04-06 14:43:23 <minimal> cloud-init doesn't disable the root account, the ssh module will, by default, disable "root" logins via SSH
2023-04-06 14:44:03 <minimal> so can specify whether you want a root password in the user-data (whether a specified one or a random one, echoed to the console)
2023-04-06 14:46:38 <ncopa> <@tomalok> (why not set up these defaults in the alpine iso?)
2023-04-06 14:46:45 <ncopa> that is a valid question
2023-04-06 14:46:57 <ncopa> a really good question
2023-04-06 14:47:21 <ncopa> right now, the alpine iso does not set up anything. its very very barebones
2023-04-06 14:47:52 <minimal> tomalok: have a look at https://github.com/dermotbradley/create-alpine-disk-image/blob/main/Virtual/config-examples/dhcp-config/user-data.yaml
2023-04-06 14:48:45 <tomalok> ncopa the next step from barebones is reasonable defaults
2023-04-06 14:48:49 <ncopa> what i want for 3.18 is an unattended installer, that installs system from cidata
2023-04-06 14:49:00 <ncopa> and i thought tiny-cloud was the right place to do that
2023-04-06 14:49:18 <ncopa> but it might not be the proper place
2023-04-06 14:49:51 <tomalok> and it might still be
2023-04-06 14:50:55 <minimal> cidata typically has 2 or 3 YAML files: meta-data, network-config, and user-data
2023-04-06 14:51:02 <tomalok> a list of requirements is a good place to start -- default user / default network / default sshd -- these are all doable.  what's left to be twakable?
2023-04-06 14:51:06 <minimal> network-config can be in either v1 or v2 format
2023-04-06 14:52:06 <ncopa> i think there are very few/small changes in tiny-cloud needed for it to work
2023-04-06 14:52:32 <ncopa> minimal: why is network-config separeate from meta-data and user-data?
2023-04-06 14:52:54 <minimal> user-data never has network configuration
2023-04-06 14:52:58 <tomalok> ncopa: found https://gitlab.alpinelinux.org/alpine/cloud/tiny-cloud/-/issues/35 -- if this is the definitive list, we can work off of that
2023-04-06 14:53:00 <ncopa> what I was kind of hoping for was that user would only need a single file
2023-04-06 14:53:15 <minimal> meta-data CAN include it instead of have a separate network-config file
2023-04-06 14:53:40 <minimal> "that user would only need a single file"? what do you mean by "user"? you mean cidata?
2023-04-06 14:54:06 <tomalok> cidata is a volume, iirc
2023-04-06 14:54:26 <minimal> "cidata" is either a ISO or FAT filesystem with those YAML files
2023-04-06 14:54:28 <tomalok> containing files
2023-04-06 14:54:42 <ncopa> a single file on cidata volume
2023-04-06 14:55:02 <ncopa> if cidata volume is detected on first boot of alpine iso, it would automatically trigger auto-installer
2023-04-06 14:55:23 <ncopa> which means you could just boot the alpine iso and attach a cidata volume
2023-04-06 14:55:32 <minimal> ncopa: nope, both meta-data AND user-data are minimum required file for a cidata filesystem
2023-04-06 14:55:33 <ncopa> and would have it up and running
2023-04-06 14:55:45 <minimal> assuming you are referring to cloud-init cidata
2023-04-06 14:55:48 <ncopa> yes
2023-04-06 14:55:59 <ncopa> i want it to be compatible with cloud-init
2023-04-06 14:56:03 <ncopa> if possible
2023-04-06 14:56:06 <ncopa> and if it makes sense
2023-04-06 14:56:11 <ncopa> but maybe it doesnt
2023-04-06 14:56:23 <minimal> https://cloudinit.readthedocs.io/en/latest/reference/datasources/nocloud.html#file-formats
2023-04-06 14:56:48 <ncopa> oh, right :-/
2023-04-06 14:57:02 <minimal> well "cidata" is a cloud-init invention so unless you want to invent an alternative then you need to abide but there definition
2023-04-06 14:57:19 <minimal> there also can be a (4th) file - vendor-data - though I've never used that
2023-04-06 14:57:51 <minimal> note Openstack have "ConfigDrive" as their alternative to "cidata"
2023-04-06 14:58:45 <ncopa> tomalok: i think this is the only thing needed to auto-create default user: https://tpaste.us/6MMr
2023-04-06 14:59:22 <minimal> from the same cloud-init docs:
2023-04-06 14:59:23 <minimal> "Also, you can inject an /etc/network/interfaces file by providing the content for that file in the network-interfaces field of meta-data."
2023-04-06 14:59:34 <minimal> "Network configuration can also be provided to cloud-init in either Networking config Version 1 or Networking config Version 2 by providing that YAML formatted data in a file named network-config."
2023-04-06 15:00:31 <tomalok> ncopa if you want a simple single file for all your needs, maybe we should define a format ro #tiny-configcontent?
2023-04-06 15:01:08 <minimal> ncopa: so to answer your earlier question about why network-config is a separate file, as seen above if you put network info in the meta-data file then its a "block paste" of a /etc/network/interfaces
2023-04-06 15:01:24 <tomalok> it's time for DayJob™ -- i'll circle back around lunchtime.
2023-04-06 15:01:33 <ncopa> tomalok: thanks!
2023-04-06 15:01:39 <minimal> whereas network-config contains either v1 or v2 format content which is not specific to /etc/network/interfaces
2023-04-06 15:02:43 <ncopa> so if meta-data has no network interfaces file, and it does not exist any network-config, cloud-init will not configure any network, right?
2023-04-06 15:03:00 <minimal> it will fallback to dhcp on the 1st interface
2023-04-06 15:03:09 <ncopa> aha
2023-04-06 15:04:36 <minimal> and here's an example of a simple v2 network-config file: https://github.com/dermotbradley/create-alpine-disk-image/blob/main/Virtual/config-examples/dhcp-config/network-config-v2.yaml
2023-04-06 15:04:58 <ncopa> yeah, i have seen those
2023-04-06 15:05:10 <ncopa> need to work on the MR now
2023-04-06 15:05:16 <minimal> whereas a more complex example would be: https://github.com/dermotbradley/create-alpine-disk-image/blob/main/Virtual/config-examples/static-config/network-config-v2.yaml
2023-04-06 15:05:35 <ncopa> network config will have to come later i think
2023-04-06 15:05:45 <ncopa> i need to get the tests in place first
2023-04-06 15:05:58 <ncopa> otherwise it gets difficult to chagne the existing code
2023-04-06 15:06:01 <minimal> I think you need to decide if you intend to be cloud-init compatible or not
2023-04-06 15:06:52 <minimal> and "nocloud" is potentially confusing if its not compatible with cloud-init NoCloud (yes I know "nocloud" is also a generic term)
2023-04-06 15:07:27 <ncopa> which most use, nocloud or configdrive?
2023-04-06 15:07:56 <minimal> configdrive is only used by OpenStack (well and in theory DigitalOcean)
2023-04-06 15:08:17 <minimal> configdrive is horrible, any time I look at it I get headaches lol
2023-04-06 15:08:25 <ncopa> ok
2023-04-06 15:09:22 <minimal> oh, and ConfigDrive is also used by OVH for their Public Cloud (it's OpenStack based), whereas confusingly OVH's Bare Metal Cloud uses the OpenStack DataSource (Metadata server) lol
2023-04-06 15:11:01 <minimal> most cloud providers don't use either ConfigDrive or NoCloud, the use one of the DataSources that support the relevant matadata server
2023-04-06 15:12:12 <minimal> so AWS => Ec2, Google => Gce, Azure => Azure, DigitalOcean (in theory) => DigitalOcean, Vultr => Vultr, OCI -> Oracle, etc
2023-04-06 15:13:38 <minimal> the reason network info does NOT go in user-data is the conceptual separate of what the provider defines and what their customer ("user") defines - the provider's infrastructure decides what IP addresses, DNS etc a server gets
2023-04-06 15:14:26 <minimal> s/separate/separation/
2023-04-06 15:18:06 <minimal> ncopa: side-note, upstream cloud-init has no support for "doas" config specifications in user-data. However the Alpine cloud-init package does as I implemented it - I did not upstream the doas module as it relies on an Alpine-specific change to the opendoas package that upstream opendoas have never merged
2023-04-06 15:18:22 <ncopa> ok
2023-04-06 15:18:42 <minimal> if you do decide to be NoCloud compatible it would make sense to be compatible which my doas local addition
2023-04-06 20:31:50 <ncopa> tomalok: i have pushed another MR for testing of init-final. After this I think I will try implement the auto creation of default user and sutoconfig of network
2023-04-06 20:32:39 <ncopa> maybe add support for the interfaces file in meta-data
2023-04-07 12:14:57 <clandmeter> tomalok, mcrute: any input regarding the email I send? 
2023-04-07 12:22:19 <tomalok> clandmeter: i fully intend to reply, if only to address the points that you bring up -- scheduling an actual time right now will be difficult in the near term
2023-04-07 12:25:15 <clandmeter> tomalok: its nice to know TZ's and availability time wise and we can work out a date later.
2023-04-07 12:27:50 <tomalok> us/pacific
2023-04-07 12:29:57 <tomalok> right now it's madness at DayJob, so biz hours weekdays would be at best difficult.
2023-04-07 14:09:25 <mcrute> clandmeter: ah yeah sorry, lost track of it... I'm in US/Pacific and 18:00-02:00 CEST on Tuesday is generally workable by me (where 21:00 CEST is optimal), Thursdays tend to be much more busy
2023-04-07 14:10:01 <mcrute> clandmeter: could also make a weekend work if it lines up better with tomalok's schedule but not until mid-May
2023-04-11 13:36:53 <ncopa> im kinda stuck with tiny-cloud, network and CIDATA provider. I think i have the details here https://gitlab.alpinelinux.org/alpine/cloud/tiny-cloud/-/issues/35#note_300697
2023-04-11 13:37:05 <ncopa> its a bit messy dump of my thoughts
2023-04-11 13:37:24 <ncopa> didnt have time to write anythings shorter, sorry
2023-04-11 17:17:58 <vkrishn> once CIDATA inits alpinelinux, does lbu also work, kinda new to this :) ?
2023-04-11 17:57:26 <vkrishn> like where whould 'lbu ci' save data
2023-04-11 18:43:14 <minimal> are you referring to tiny-cloud or cloud-init?
2023-04-11 18:57:02 <vkrishn> i couldn't tell more, is there a wiki-page for both ?
2023-04-11 18:57:12 <vkrishn> i was refering to autoinstall, here https://gitlab.alpinelinux.org/alpine/cloud/tiny-cloud/-/issues/35
2023-04-11 18:58:04 <vkrishn> once such autoinstall in implemented/up and running, how would lbu work ?
2023-04-11 18:58:20 <vkrishn> is implemented^
2023-04-11 18:59:02 <minimal> LBU only tried to either run-from-ram or "data mode" Alpine installs, the Alpine Cloud images are all "Sys mode" (run from disk) images
2023-04-11 18:59:10 <minimal> s/tried/applies/
2023-04-11 19:00:13 <vkrishn> its says , "The idea is to make the alpine iso images look for LABEL=cidata and enable tiny-cloud-nocloud if found"
2023-04-11 19:01:53 <minimal> right, this is referring to the Alpine "normal" ISOs (no the Cloud Images) being able to autoinstall Alpine on a machine. I haven't seen anything that indicates which "mode" such an installation would use
2023-04-11 19:03:02 <vkrishn> looks like new ideas in motion
2023-04-11 19:03:08 <minimal> this autoinstall functionality is currently a work-in-progress, none of the Alpine releases ISOs have it
2023-04-11 19:03:49 <vkrishn> sounds kinda cool
2023-04-11 19:04:15 <minimal> its a derivative of existing cloud-init functionality
2023-04-11 19:32:24 <vkrishn> its like, who whats "live dvd", welcome to "auto live on cloud" ;)
2023-04-12 07:39:46 <ncopa> vkrishn: we need keep lbu compat. They are solving slightly different problems
2023-04-12 07:40:14 <ncopa> my current thinking is that you can use cidata to install the diskless system if you want, and then lbu commit
2023-04-12 08:17:38 <vkrishn> ncopa: i understand, not wanting to add features to lbu
2023-04-12 08:20:19 <vkrishn> i was thinking if a dummy script is made, that takes inputs and creates bare localhost.apkovl.tar.gz, which then can be uploaded to cidata (vol)
2023-04-12 08:21:55 <vkrishn> this can init al, but in a slightly different way
2023-04-12 08:22:11 <vkrishn> just a thought
2023-04-12 08:22:50 <vkrishn> a web based interface is also possible
2023-04-12 08:22:58 <ncopa> yeah, i thought of the same today while thinking about cidata stuff
2023-04-12 08:23:08 <vkrishn> :)
2023-04-12 08:23:08 <ncopa> i saw those cloud-utils script
2023-04-12 08:23:20 <ncopa> and there is a cloud-localds script
2023-04-12 08:23:28 <ncopa> which gernerates a cidata image
2023-04-12 08:23:37 <ncopa> i thought we coudl have a similar for apkovl
2023-04-12 08:23:47 <ncopa> its relatively simple
2023-04-12 08:23:59 <ncopa> mkapkovl or similar
2023-04-12 08:25:09 <vkrishn> will such apkovl differ every vendor+platform ?
2023-04-12 08:25:11 <ncopa> mkapkovl --hostname=myhost --interfaces auto --services sshd --ssh-keys .... 
2023-04-12 08:25:26 <ncopa> no
2023-04-12 08:25:30 <ncopa> but its just an idea
2023-04-12 08:25:30 <vkrishn> cool
2023-04-12 08:25:37 <ncopa> i havent implemented anything yet
2023-04-12 08:25:54 <ncopa> could maybe have a mkapkovl --from-user-data=file
2023-04-12 08:26:05 <ncopa> which is a user-data file
2023-04-12 08:26:27 <vkrishn> or future ready --from-yaml=file
2023-04-12 08:26:27 <ncopa> just ideas though
2023-04-12 08:26:44 <ncopa> yeah that was idea, take a yaml file
2023-04-12 08:29:58 <ncopa> how about a script that can do something like
2023-04-12 08:30:54 <ncopa> apply-cidata --meta-data /path/to/meta-data --user-data /path/to/user-data
2023-04-12 08:31:41 <ncopa> then we can call this from tiny-cloud. the --meta-data is applied from tinly-cloud-eary and the user-data is applied from tiny-cloud-main
2023-04-12 08:32:21 <ncopa> the apply-cidata script could also have an --output-apkovl file.apkovl.tar.gz
2023-04-12 08:32:30 <ncopa> eg it could generate an apkovl from yaml files
2023-04-12 08:35:41 <vkrishn> sounds like both fuctionality , will I be able to use --output-apkovl in non tiny-cloud setup(localy) ?
2023-04-12 08:38:41 <ncopa> that was the idea yes
2023-04-12 08:38:52 <ncopa> basically, create an apkovl from a yaml file
2023-04-12 10:54:33 <ncopa> lima-vm has some cidata support with alpine: https://github.com/lima-vm/alpine-lima
2023-04-12 10:54:38 <ncopa> they are parsing the yaml with awk
2023-04-12 12:48:29 <ncopa> tomalok: do you think that an autoinstall script belongs in tiny-cloud or in alpine-conf project?
2023-04-13 12:25:18 <ncopa> been doing some serious progress: https://gitlab.alpinelinux.org/alpine/cloud/tiny-cloud/-/merge_requests/42
2023-04-13 12:26:53 <psykose> massive
2023-04-13 12:26:55 <psykose> good work :)
2023-04-13 12:27:21 <ncopa> need some refactoring (and rebase) to cleanup
2023-04-13 12:28:00 <ncopa> but I should have enough to test a bootable iso
2023-04-13 12:28:05 <ncopa> just need some glue
2023-04-13 16:29:09 <ncopa> minimal: thank you for clarifying the stuff and for the ephemeral network python code
2023-04-13 16:30:17 <minimal> the python code stuff is not Alpine-specific, I think its just that not many people upstream have used that code recently lol
2023-04-13 16:30:31 <ncopa> yeah
2023-04-13 16:30:54 <ncopa> i think for aws though, it should be enough with a link-local ephemeral address
2023-04-13 16:31:11 <ncopa> i am not too happy with brining up the network under the foot of openrc
2023-04-13 16:31:35 <minimal> ncopa: ah, I forgot to mention, there have been a couple of upstream PRs merged into master recently regarding the NoCloud /proc/cmdline handling that you might want to look at as you're coding stuff for that in tiny-cloud currently
2023-04-13 16:31:37 <ncopa> that is network with full internet connectivity
2023-04-13 16:31:54 <ncopa> link?
2023-04-13 16:32:06 <ncopa> i also think we should clean up the apk_repos stuff upstream
2023-04-13 16:32:29 <ncopa> i kinda like the way I did `apk` in the yaml
2023-04-13 16:32:50 <minimal> https://github.com/canonical/cloud-init/commit/f146fe71733e72b94fad525b8cc9988b1405e760
2023-04-13 16:32:59 <minimal> and possibly https://github.com/canonical/cloud-init/commit/250280ada67995a8449b64027b879d01939d2729
2023-04-13 16:34:15 <ncopa> ah the semicolon thing you mentioned
2023-04-13 16:34:17 <ncopa> ok
2023-04-13 16:34:28 <ncopa> i think nocloud-net is out of scope for 3.18
2023-04-13 16:34:31 <minimal> the next cloud-init is approx 1 month away so this change/additional behaviour will be in that release, so makes sense to handle the "new" way (though the old way will remain in place for backward compatibility for a while)
2023-04-13 16:35:11 <minimal> nocloud-net out of scope? I though you were working on seed url support currently?
2023-04-13 16:35:34 <ncopa> i think local cidata volume is first prio
2023-04-13 16:35:51 <ncopa> we'll see what i get time for
2023-04-13 16:35:58 <ncopa> i had good success today though
2023-04-13 16:36:11 <ncopa> i have a proof-of-concept working now
2023-04-13 16:36:39 <ncopa> it rings up the network, executes runcmd, installs packages
2023-04-13 16:36:44 <ncopa> a few bugs left though
2023-04-13 16:37:05 <minimal> I'm not sure what you mean by "brining up the network under the foot of openrc"
2023-04-13 16:38:13 <ncopa> so, openrc has a virtual service named 'net', the /etc/init.d/networking has a 'provides net' and so has dhcpcd
2023-04-13 16:38:34 <ncopa> it means that you can add a dependency on 'net' in openrc services
2023-04-13 16:38:45 <ncopa> and you can say 'before net'
2023-04-13 16:39:02 <minimal> yes I know what that is, I just don't understand your comments "<ncopa> i think for aws though, it should be enough with a link-local ephemeral address
2023-04-13 16:39:02 <minimal> <ncopa> i am not too happy with brining up the network under the foot of openrc"
2023-04-13 16:39:19 <ncopa> which will guarantee that no network is available when that service is started
2023-04-13 16:39:32 <minimal> and I commented on the "net" dep in my note, it is already present in the cloud-init init.d script
2023-04-13 16:39:41 <ncopa> for example, we have an iptables service that is executed before 'net'
2023-04-13 16:39:49 <minimal> so I'm confused what exactly you're trying to say
2023-04-13 16:40:03 <ncopa> which guarantees that there is no network available before firewall is up
2023-04-13 16:40:39 <minimal> <is confused>
2023-04-13 16:40:47 <ncopa> now, if cloud-init runs a temprorarily dhcp, it means that network may be available (via cloud-init) while openrc thinks there is no network
2023-04-13 16:42:15 <ncopa> what im saying is that im not too happy with internet connectivity before firewall is up
2023-04-13 16:42:29 <ncopa> even if its just temporarily
2023-04-13 16:42:49 <minimal> let's start again
2023-04-13 16:42:55 <ncopa> link-local is different
2023-04-13 16:43:14 <ncopa> because its only local connectivity. no default gateway
2023-04-13 16:43:34 <ikke> default gateway for ipv6 is a link-local address
2023-04-13 16:43:55 <minimal> in cloud-init, for **some** Cloud Providers their DataSource will use specific IP addresses (i.e. 169.x.y.z) to bring up the interface before contacting a metadata server
2023-04-13 16:44:14 <ncopa> ikke: so with ipv6 you could have fullinternet access?
2023-04-13 16:44:26 <ncopa> ikke: i guess not, since source address is link-local
2023-04-13 16:44:33 <ikke> You still need a public ipv6 address
2023-04-13 16:44:37 <ncopa> yeah
2023-04-13 16:44:37 <minimal> for **some** other Cloud Providers their DataSource will use DHCP to get an IP to bring up the interface before contacting a metadata server
2023-04-13 16:45:02 <ncopa> minimal: i understand that. i just dont like it
2023-04-13 16:45:25 <minimal> it depends on particular Cloud Providers whether their metadata server is on specific IPs or not
2023-04-13 16:46:10 <ncopa> yeah. 169.x.y.z is link-local
2023-04-13 16:46:17 <ncopa> so that is acceptable imho
2023-04-13 16:46:18 <minimal> if a Cloud Provider doesn't use specific IPs for the metaserver then there's no alternative to using DHCP
2023-04-13 16:46:43 <minimal> now NoCloud is a different matter
2023-04-13 16:46:46 <ikke> does that dhcp server provide a default gateway, or just connectivity for a single subnet?
2023-04-13 16:47:20 <minimal> ikke: that would be an implementation matter - I've not familiar with how all the various cloud providers do things
2023-04-13 16:47:29 <ncopa> i suppose meta server in same subnet is acceltable
2023-04-13 16:48:06 <ncopa> i guess we can postpone the ephemeral ip problem
2023-04-13 16:48:13 <minimal> that's an implementation-specific matter - a VM can't really know if that's the case or not
2023-04-13 16:48:27 <ncopa> til we add support for some one actually needing it
2023-04-13 16:48:33 <ncopa> yeah
2023-04-13 16:48:54 <ncopa> but first iteration will likely only support cidata volume
2023-04-13 16:49:08 <ncopa> at least for metadata
2023-04-13 16:49:14 <minimal> now for NoCloud with seed use all that cloud-init (or tiny-cloud) has is a base URL to fetch content from - so DHCP appears the only option to me
2023-04-13 16:49:38 <ncopa> so, fetching user-data from network may be supported though
2023-04-13 16:49:41 <ncopa> but not meta-data
2023-04-13 16:49:44 <ikke> ncopa: doesn't setup-alpine setup networking without any firewall as well?
2023-04-13 16:49:54 <ncopa> ikke: it does
2023-04-13 16:50:09 <ncopa> but the point is that openrc is in control
2023-04-13 16:50:39 <ncopa> and you can force services before network is available if you want
2023-04-13 16:51:11 <ncopa> you can express that in openrc dependencies
2023-04-13 16:52:31 <ncopa> i suppose we can seed metadata from dhcp if needed in the future. we just execute a dhcp client but disable default gw
2023-04-13 16:53:44 <ncopa> from what I understood, the current tiny-cloud providers uses: IMDS_ENDPOINT="169.254.169.254"
2023-04-13 16:53:53 <ikke> with ipv6, things could go more automatically though with slaac
2023-04-13 16:53:58 <ncopa> yeah
2023-04-13 16:54:13 <minimal> that's the AWS IPv4 metadata address, they also have a IPv6 one
2023-04-13 16:54:20 <ncopa> yeah
2023-04-13 16:54:26 <ncopa> and its a link-local
2023-04-13 16:54:52 <minimal> AWS made some changes to their IPv6 stuff recently, I posted a link on here about 1 month ago
2023-04-13 16:54:53 <ncopa> so we can use ip4ll (aka zero-conf) for an ephemeral ip
2023-04-13 16:55:33 <minimal> the point is it is provider specific how to setup and talk to their metadata server
2023-04-13 16:55:40 <ncopa> yeah
2023-04-13 16:55:56 <ncopa> but, its chicken-and-the-egg problem
2023-04-13 16:56:03 <minimal> which is why cloud-init has separate DataSources for each provider
2023-04-13 16:56:12 <minimal> why chicken-and-egg?
2023-04-13 16:56:28 <ncopa> if network config is available on network
2023-04-13 16:56:43 <ncopa> eg network config is in metadata
2023-04-13 16:56:53 <ncopa> and to get metadata you need network
2023-04-13 16:57:15 <minimal> that's network config for the running instance, rather than network config to get to the metadata server to get that info
2023-04-13 16:57:59 <minimal> if you look at the various cloud-init DataSources that's part of what they do, they get the network up in the required fashion
2023-04-13 16:59:45 <minimal> and although I haven't used it yet Azure seems to have quite a complicated way of handling access to the metadata
2023-04-13 17:00:38 <ncopa> i have heard
2023-04-13 17:01:07 <minimal> I saw someone complaining that DHCP was taking a long time (10+ minutes?) before giving up
2023-04-13 17:01:32 <minimal> apparently at a particular DC that was overloaded
2023-04-13 17:03:53 <minimal> from a quick glance it seems that cloud-init Google DS uses DHCP in order to bring up the interface to then get to http://metadata.google.internal/
2023-04-13 17:04:07 <minimal> so tiny-cloud must be using DHCP for that already
2023-04-13 17:09:17 <ncopa> * Trying 169.254.169.254... connected
2023-04-13 17:09:34 <ncopa> looks like metadata.google.internal is 169.254.169.254.
2023-04-13 17:10:22 <ncopa> https://cloud.google.com/compute/docs/metadata/querying-metadata search for 169
2023-04-13 17:10:59 <ncopa> it is relatively simple to add support for link-local address
2023-04-13 17:12:32 <minimal> ncopa: re the apk YAML stuff, I wanted a separate option to enable testing repo so as to try and avoid people doing "footgun" by enabled testing for non-Edge Alpine install
2023-04-13 17:12:49 <ncopa> https://datatracker.ietf.org/doc/html/rfc3927
2023-04-13 17:13:12 <ncopa> minimal: yeah that makes sense
2023-04-13 17:13:54 <minimal> yes I'm aware of IPv4 link-local stuff, I've been doing TCP/IP since early 1990s
2023-04-13 17:14:17 <ncopa> :)
2023-04-13 17:14:25 <ncopa> then you know it better than I do
2023-04-13 17:14:47 <ncopa> the only downside appears to be that it takes 10 seconds to get a link-local ipv4 address
2023-04-13 17:14:48 <minimal> yes, I ran an ISP back then :-)
2023-04-13 17:15:29 <ncopa> i enabled zcip in busybox which is an implementation of zero-conf
2023-04-13 17:15:42 <minimal> yeah that's to avoid conflicting with another machine using it (from memory)
2023-04-13 17:15:50 <ncopa> was thinking we could use that in meta-data is on http://169.254.*
2023-04-13 17:15:52 <ikke> similar to ipv6 dad?
2023-04-13 17:16:00 <ncopa> yeah, i think so
2023-04-13 17:16:40 <ncopa> dhcpcd does a link-local fallback if it fails to get a dhcp address
2023-04-13 17:16:46 <minimal> yeah, its in section 2.2 of the rfc
2023-04-13 17:17:01 <ikke> minimal: does it use arp to do thaat?
2023-04-13 17:17:19 <minimal> yupe
2023-04-13 17:17:30 <ncopa> i think ipv6 can get a link-local faster?
2023-04-13 17:17:36 <minimal> look at 2.2.1  of the RFC, it's all spelt out
2023-04-13 17:17:58 <minimal> including references to PROBE_WAIT seconds and PROB_NUM packets
2023-04-13 17:18:34 <minimal> ah, section 9 defines those values
2023-04-13 17:18:40 <ikke> ncopa: "Duplicate Address Detection MUST be performed on all unicast addresses prior to assigning them to an interface"
2023-04-13 17:18:52 <minimal> PROBE_WAIT = 1 sec
2023-04-13 17:18:58 <minimal> PROBE_NUM = 3 packets
2023-04-13 17:19:28 <minimal> PROBE_MIN = 1 second (min delay between packets)
2023-04-13 17:19:41 <ikke> so 3 seconds
2023-04-13 17:19:41 <minimal> PROBE_MAX = 2 seconds
2023-04-13 17:19:59 <ikke> 3-6 seconds
2023-04-13 17:20:16 <minimal> there's also ANNOUNCE_WAIT = 2 seconds ;-)
2023-04-13 17:20:28 <minimal> and a few other settings
2023-04-13 17:20:50 <ncopa> my tests showed 10 seconds with busybox zcip on dummy0 interface
2023-04-13 17:21:08 <minimal> yes, there's also ANNOUNCE_NUM = 2 (packets)
2023-04-13 17:21:24 <minimal> so 10 seconds makes sense
2023-04-13 17:21:41 <ncopa> https://datatracker.ietf.org/doc/html/rfc3927#section-2.3
2023-04-13 17:21:48 <ncopa> i was hoping there was an update by now :)
2023-04-13 17:21:58 <minimal> sending 2 announcement packets (2 seconds apart) after doing probing
2023-04-13 17:22:20 <minimal> yeah I'd optimistically checked as well
2023-04-13 17:23:40 <minimal> I seem to remember that for AWS IPv6 someone recommended disabling DAD as AWS's Nitro hypervisors isolate machines so duplicate addresses cannot happen
2023-04-13 17:23:41 <ncopa> minimal: i'd like to do this in cloud-init's apk repos config: https://tpaste.us/0lxN
2023-04-13 17:23:50 <ncopa> no version there, intentionally
2023-04-13 17:24:08 <minimal> why intentionally?
2023-04-13 17:24:43 <ncopa> it would auto detect version from /etc/os-release so you can reuse the same config for different versions
2023-04-13 17:25:03 <ncopa> also avoids the testing footgun
2023-04-13 17:25:08 <minimal> but you might want the user-data to upgrade to a newer version
2023-04-13 17:25:18 <ncopa> then you set version
2023-04-13 17:25:33 <ncopa> idea is to auto-detect if its unset
2023-04-13 17:25:36 <minimal> right, so its optional in otherwords
2023-04-13 17:25:41 <ncopa> yup
2023-04-13 17:25:44 <psykose> we removed versions and are now a rolling-only distro
2023-04-13 17:25:47 <psykose> thank you for your inquiry
2023-04-13 17:25:57 <minimal> what's the intention of cache: ?
2023-04-13 17:26:16 <ncopa> so you can configure apk cache if you need one
2023-04-13 17:26:45 <ncopa> apk cache will store downloaded packages locally so it does not need to fetch from network on re-install
2023-04-13 17:27:00 <minimal> ok, never used it, yeah that makes sense
2023-04-13 17:27:31 <ncopa> we could also have an apk: { "keys": .... }
2023-04-13 17:27:40 <ncopa> so you can inject apk keys too if needed
2023-04-13 17:28:20 <minimal> from memory I have that for local repo support
2023-04-13 17:28:43 <minimal> it's been a while since I looked at the apk cloud-init module, I'll need to refresh my memory
2023-04-13 17:30:07 <minimal> ah, no, I'm writing the local repo key directly when I create a disk image
2023-04-13 17:30:36 <ncopa> with local base_url, and empty version: https://tpaste.us/a1vM
2023-04-13 17:30:55 <ncopa> resolves to /srv/packages/{main,community}
2023-04-13 17:32:24 <minimal> hmm, wouldn't that be a single "repositories:" with then 2 list items?
2023-04-13 17:32:55 <ncopa> yes. ofc
2023-04-13 17:34:31 <minimal> ok, I'll have a look at that and the current code
2023-04-13 17:35:17 <ncopa> like this: https://tpaste.us/qEYK
2023-04-13 17:35:28 <ncopa> thanks!
2023-04-13 17:37:36 <minimal> what's the significance of the version: "" ?
2023-04-13 17:37:52 <ncopa> so, the idea is that if version is unset, it will autodetect
2023-04-13 17:38:17 <ncopa> but what if you want disable the autodetection and not specify version?
2023-04-13 17:38:41 <minimal> "unset" means no "version:" specified at all? or means version of ""?
2023-04-13 17:38:44 <ncopa> like, if your local packages are in /src/packages/main , and not in /srv/packages/v3.17/main
2023-04-13 17:39:26 <ncopa> unset means no "version:" specified at all
2023-04-13 17:39:40 <minimal> I think that's why I had the separate local repos section as the url would be used as-is
2023-04-13 17:42:21 <ncopa> what if you want auto-insert version to local repo?
2023-04-13 17:46:47 <minimal> yeah, I was talking about how it works currently
2023-04-13 18:13:27 <ncopa> proof-of-concept: https://dev.alpinelinux.org/~ncopa/alpine/alpine-virt-tiny-cloud-x86_64.iso
2023-04-13 18:13:48 <ncopa> boot that iso with cidata (not CIDATA) volume
2023-04-13 18:14:18 <ncopa> starts network, sshd, creates alpine user with the configured ssh key
2023-04-13 18:15:23 <minimal> going offline for a bit but I'll have a play with that later on
2023-04-13 18:29:43 <ncopa> tomalok: when you get a chance: i'd like to know if am on the right track here: https://gitlab.alpinelinux.org/alpine/cloud/tiny-cloud/-/merge_requests/42/
2023-04-13 18:29:56 <ncopa> pretty intrusive changes
2023-04-13 18:48:10 <ncopa> i think it is pretty much in good shape now. tests passes with all shell implementations and I have tested in in qemu.
2023-04-15 18:44:24 <minimal> ncopa: I'm confused, I booted that ISO and there's no tiny-cloud installed in it
2023-04-15 18:52:09 <ncopa> minimal: try boot it with a cidata volume attached
2023-04-15 18:52:26 <minimal> I did
2023-04-15 18:52:47 <ncopa> hum
2023-04-15 18:52:49 <minimal> that wouldn't change whether tiny-cloud package was present or not
2023-04-15 18:52:57 <ncopa> it would
2023-04-15 18:53:24 <ncopa> the initramfs will look for a volume with cidata, and if found it will pull in tiny-cloud
2023-04-15 18:53:27 <minimal> how? the cidata needs to be read/used by something, i.e. tiny-data
2023-04-15 18:53:44 <minimal> ah, you've put stuff in the initramfs
2023-04-15 18:53:54 <ncopa> yup, a simple findfs LABEL=cidata
2023-04-15 18:54:00 <minimal> didn't check there
2023-04-15 18:54:10 <ncopa> if that returns anything, it will pull in the tiny-cloud
2023-04-15 18:54:18 <minimal> anyway I gave it one of my usual cidata ISOs and nothing seemed to happen
2023-04-15 18:54:29 <ncopa> is it label CIDATA?
2023-04-15 18:54:33 <ncopa> or lowercase cidata
2023-04-15 18:54:46 <minimal> whatever the cloud-utils localds tool does, will have to check
2023-04-15 18:55:02 <ncopa> ah, its probably uppercase then
2023-04-15 18:55:12 <ncopa> will change that on monday
2023-04-15 18:55:19 <ncopa> sorry :-/
2023-04-15 18:55:45 <minimal> ok. So needing to pull in tiny-cloud means an internet connection is required
2023-04-15 18:55:54 <minimal> unlike the standard/extended ISOs
2023-04-15 18:55:57 <ncopa> yes
2023-04-15 18:56:03 <ncopa> i started with that
2023-04-15 18:56:31 <ncopa> i used a -virt as work image as its fast to create, small
2023-04-15 18:56:53 <ncopa> i could have added the changes to extended, and then it should work without internet
2023-04-15 18:57:36 <minimal> as its run-from-ram there's no initramfs-init present in the fs to look at either lol
2023-04-15 18:57:48 <ncopa> but i think the first thing im aiming for is a solution for headless. you add you ssh key to cidata volume and boot it
2023-04-15 18:58:19 <ncopa> after you booted it, can you do: blkdid
2023-04-15 18:58:21 <ncopa> blkid
2023-04-15 18:58:29 <ncopa> and see if you find the cidata volume
2023-04-15 18:58:54 <ncopa> anyway, i'll start use the cloud-localds from now on
2023-04-15 18:59:05 <ncopa> have a nice weekend!
2023-04-15 18:59:14 <minimal> you too
2023-04-15 19:00:13 <minimal> actually it is work - seems I had a issue with the order of multiple ISOs on VirtualBox
2023-04-15 19:00:24 <minimal> working
2023-04-16 18:06:52 <tomalok> ncopa: finally got around to skimming this thread...  ;P  i think autoinstall specific to alpine needs to be somewhat isolated from tiny-cloud -- to be able to (in theory) support other distros in the future.  and the same for potentially supporting different init systems other than openrc and other hotplug other than mdev...  i think there's a
2023-04-16 18:06:52 <tomalok> little bit of reorganizing that can be done to better prepare things for divvying cloud/distro/hotplug/init/userdata-handler stuff
2023-04-16 18:09:43 <tomalok> ncopa: for distro-specific operations we'd do something similar to what we do when specifics diverge for various clouds -- we'd load in alpine-specific version of generalized functions...  so if, for example a user needs to be added, and a distro needed to use useradd or adduser or setup-user, that's handled in the generalized function
2023-04-16 18:10:42 <ikke> you mean specialized?
2023-04-16 18:10:47 <tomalok> i also see that there's some additional DMI interpolation that cloud-init is doing with the nocloud seed stuff - we'll probably want to add that eventually too.
2023-04-16 18:11:08 <ikke> ("in the generalized function)
2023-04-16 18:11:18 <tomalok> well, the distro-version of the generalized function
2023-04-16 18:11:42 <tomalok> add_user() --> useradd / adduser / setup-user as appropriate
2023-04-16 18:14:03 <tomalok> would also like to get more "autodetection" of things -- i.e. if the cmdline says "nocloud" or we can see or probe other evidence about init / hotplug / cloud
2023-04-16 18:16:14 <minimal> cloud-init uses things like DMI contents to autodetect the DataSource to use
2023-04-16 18:16:27 <tomalok> minimal: (nod) just saw that
2023-04-16 18:17:05 <tomalok> so if your seed URL is actually a script that builds the desired config, it can use that information to decide what to send
2023-04-16 18:17:07 <minimal> tomalok: yeah have a look at the Ec2 DS as it is actually used by more than just AWS
2023-04-16 18:17:47 <tomalok> there's a lot of things out there that decided to be at least somewhat compatible with what EC2 did
2023-04-16 18:18:27 <minimal> basically the way cloud-init works is that there is a list of enabled DSes and it goes through that in order checking each to see if they should be used
2023-04-16 18:19:27 <minimal> tomalok: many cloud providers are either Ec2 or OpenStack (meta data or ConfigDrive) compatible
2023-04-16 18:23:30 <tomalok> for autodetecting distro, it's a little easier because of /etc/os-release -- but for cloud it'd be an if / elsif / ... / else looking at various signs.
2023-04-16 18:24:31 <tomalok> and if a "cloud provider" is EC2 / OpenStack compatible, then we shouldn't (in theory) need to differentiate further?
2023-04-16 18:24:36 <minimal> which is exactly what cloud-init does
2023-04-16 18:25:23 <minimal> tomalok: being based on Ec2 doesn't mean 100% compatible though, especially as there are distinct versions of Ec2 metadata
2023-04-16 18:25:34 <tomalok> we can determine "nocloud?" easily with the kernel cmdline
2023-04-16 18:25:40 <minimal> i.e. IMDSv1 vs IMDSv2
2023-04-16 18:25:52 <minimal> and the "dated" urls from memory
2023-04-16 18:26:24 <tomalok> the big difference between IMDSv1 and IMDSv2 is the token header.
2023-04-16 18:26:26 <minimal> c-i detects it via "cidata" fs presence
2023-04-16 18:26:38 <tomalok> and that
2023-04-16 18:27:00 <minimal> or if seed base URL has been specified
2023-04-16 18:27:35 <minimal> ConfigDrive works in a similar fashion
2023-04-16 18:31:41 <minimal> in cloud-init the order that various DataSources are check can be defined in /etc/cloud/cloud.cfg or, if not defined, c-i falls back to a built-in list
2023-04-16 18:32:15 <minimal> the order of various DSes in that list is significant/important as its "first match wins"
2023-04-16 18:35:34 <minimal> e.g. I noticed recently that in c-i ConfigDrive comes before DigitalOcean in the list - DO appear to implement both ConfigDrive (ISO with OpenStack defined settings, badly implemented by them) and their own metadata server (DigitalOcean DS) but their metadata service will never be used unless the disk image specifically defines a custom DS list with DigitalOcean before ConfigDrive (or no ConfigDrive at all)
2023-04-17 06:11:14 <ncopa> morning
2023-04-17 06:11:43 <ncopa> tomalok: you want an abstraction layer for distro stuff
2023-04-17 06:14:08 <ncopa> im thinking YAGNI
2023-04-17 14:18:28 <tomalok> ncopa: YAGNI?
2023-04-17 14:19:52 <ikke> https://en.m.wikipedia.org/wiki/You_aren%27t_gonna_need_it
2023-04-17 14:20:51 <tomalok> i read it as yet another generic network interface
2023-04-17 15:58:13 <ncopa> idea is to delay implement abstraction until its needed
2023-04-17 15:59:01 <ncopa> eg delay support multi distro til its actualy needed
2023-04-17 18:42:14 <minimal> I didn't realise that tiny-cloud was intended to be multi-distro
2023-04-17 19:53:18 <tomalok> minimal: there's an invitation for MRs buried in the readme
2023-04-17 19:57:57 <minimal> ah ok, had missed that. There are so many generic cloud-init alternatives around, what's another one eh? ;-)
2023-04-17 19:58:36 <ikke> https://xkcd.com/927/
2023-04-17 19:59:40 <minimal> ikke: I correctly guessed what that would be before viewing it ;-)
2023-04-17 19:59:59 <ikke> :)
2023-04-17 20:00:11 <minimal> also thinking of "Life of Brian"............"bloody splitters!"
2023-04-17 20:01:51 <minimal> not so much competing standards as competing implementations, so no "real" (comprehensive) alternative to cloud-init ever emerges
2023-04-18 14:42:01 <minimal> ncopa: for your test ISO, does it log anywhere what it has done?
2023-04-18 14:47:37 <ncopa> maybe
2023-04-18 14:47:50 <ncopa> i think there might be something /var/log/cloud or somethine
2023-04-18 14:50:13 <minimal> nope, nothing there
2023-04-18 14:50:46 <ncopa> then it might not log enough
2023-04-18 14:50:54 <ncopa> i saw there are some log lines in the soruce code
2023-04-18 14:51:05 <ncopa> maybe in syslog?
2023-04-18 14:51:26 <minimal> ncopa: syslog wouldn't be running in initramfs though
2023-04-18 14:51:45 <ncopa> it only enables tiny-cloud from initramfs
2023-04-18 15:03:43 <minimal> ok, looks like dependency related - no tiny-cloud init.d script declares "need logger"
2023-04-18 15:04:02 <minimal> so guess there is no syslogd running when they do
2023-04-25 12:33:21 <ncopa> minimal: what do you think should be included in logs?
2023-04-25 13:43:32 <ncopa> i think we should have a separate tiny-cloud script, that takes the phase as arg. ie. tiny-cloud early, tiny-cloud main, tiny-cloud final. With that it becomes more distro agnostic
2023-04-25 13:44:02 <ncopa> and the etc/init.d/tiny-cloud* scripts will just be wrappers
2023-04-25 13:46:05 <ncopa> tomalok: i am trying to get this in shape for alpine 3.18 release (a week or two for the first release candidates)
2023-04-25 13:46:10 <ncopa> what should I do to move this forward?
2023-04-25 14:24:50 <ncopa> im also not sure how to do logging properly
2023-04-27 18:55:35 <ncopa> tomalok: is is realistic to get the nocloud support merged for alpine 3.18? the builders are almost ready and im hoping to do rc1 tomorrow or early next week
2023-04-28 14:25:53 <tomalok> ncopa: i'll try to take a look later today, should have some time this weekend as well.
2023-04-28 14:27:16 <tomalok> ncopa / ikke - related - is it possible to get cloud image storage / CDN sorted for 3.18 - so non-AWS images can be made available?
2023-04-28 14:38:42 <ikke> tomalok: I want to discuss that with clandmeter, but that's something I had in mind.
2023-04-28 15:04:57 <ncopa> tomalok: i have factored out openrc scripts and moved the to aports so its more distro agnostic.
2023-04-28 15:05:28 <ncopa> I have not made user creation and other stuff distro agnostic or modular, but I belive that change will be trivial
2023-04-28 15:05:45 <ncopa> i wanted avoid add the extra complexity til when it is actually needed
2023-04-28 15:06:11 <ncopa> *if* we decide to make it distro agnostic we should probably move it out from gitlab.alpinelinux.org too
2023-04-28 15:06:30 <ncopa> but i am ok to keep it as a alpine-only project for now
2023-04-28 15:08:09 <ncopa> what is currently missing is the ephemeral network and ipv6 support
2023-04-28 15:10:44 <ncopa> the absolutely simplest thing to do is to simply `apk add dhcpcd && rc-update add dhcpcd boot && echo "rc_after net" >> /etc/conf.d/tiny-cloud-early` in the aws disk image
2023-04-28 15:11:24 <ncopa> dhcpcd should also handle hotplugged nics
2023-04-28 15:11:51 <ncopa> and handle the default route etc
2023-04-28 16:04:39 <minimal> ncopa: in terms of what is logged, things like which YAML entries are/are not used
2023-04-28 16:05:23 <ncopa> ok
2023-04-28 16:06:07 <ncopa> i dont think we will get unused yaml entries logged anytime soon
2023-04-28 16:06:24 <ncopa> the latest iteration says what it does
2023-04-28 16:07:12 <ncopa> you have an overview of the exat things it logs here: https://gitlab.alpinelinux.org/alpine/cloud/tiny-cloud/-/merge_requests/42/diffs?commit_id=fe02cbacb5d86c6cce6fc49fc5bc2d668c0d409f
2023-04-28 16:07:56 <ncopa> i suppose that better than nothing for now
2023-04-28 16:09:02 <ncopa> the 3.18 builders are done so rc1 will happen on Monday likely
2023-04-28 16:25:13 <minimal> ncopa: /etc/conf.d/tiny-cloud has moved to /etc/tiny-cloud.conf?
2023-04-28 16:31:05 <minimal> this may affect backward compatibility. For example I do have a script that modifies /etc/conf.d/tiny-cloud so I'll now change it to modify the appropriate file depending on Alpine release
2023-04-28 19:01:23 <ncopa> yes and yes
2023-04-28 19:10:53 <ncopa> also, instead of a post-install script generates the config, it is now shipped with the cloud provider subpackage
2023-04-29 02:18:09 <tomalok> ncopa: https://gitlab.alpinelinux.org/alpine/cloud/tiny-cloud/-/merge_requests/41 just has a couple nits, otherwise lgtm
2023-04-29 03:01:56 <tomalok> ncopa: https://gitlab.alpinelinux.org/alpine/cloud/tiny-cloud/-/merge_requests/42 is currently draft -- do you want a review with what's there so far?  (there's quite a bit)
2023-04-29 07:06:45 <ncopa> Yes please. I’d like to know if the direction is good. and if we can ship it “as is” and then fix things that is broken. The question is if format of yaml is ok, if a separate ‘tiny-cloud’ is ok, the move of openrc scripts to aports, the move of config file to /etc/tiny-cloud.conf.
2023-04-29 07:09:13 <ncopa> I’d like to ship it with the rc1 (Monday) and fix whatever is broken in rc2
2023-04-29 07:11:12 <ncopa> or if the draft mr is in an unfixable state,  unrealistic for alpine 3.18 and we need to go back  to drawing board
2023-04-29 07:18:43 <ncopa> I’m hoping for including it in alpine 3.18 and willing to make the needed things happen. But I’m also fine with postponing it for 3.19, or fork it or move parts to a separate project
2023-04-29 07:20:09 <ncopa> we can also move the entire nocloud thing to alpine-conf where the rest of the install scripts are
2023-04-29 07:21:16 <ncopa> i just want to know what needs to be done
2023-04-29 19:38:23 <tomalok> ncopa: i think the overall direction is good -- but i think that i want to take this a bit more incrementally -- and if i read some of the diffs correctly, some of this will break the non-noclouds...  I'm going to work on this today (and probably tomorrow too) and see how far I can get before your Monday.
2023-04-29 21:03:34 <ncopa> Yes, some will break some of the non noclouds. I am willing to fix those as we go. Need to find a good way to test those. Let me know what needs to be done on Monday and all continue where you left off
2023-04-29 21:17:35 <tomalok> if all goes well today/tomorrow, maybe there won't be much left for  you to do ;)
2023-04-29 21:18:39 <tomalok> at the end of this i will probably release tiny-cloud-3.0.0_rc1 or some such and build some edge cloud images to test.
2023-04-30 03:38:46 <tomalok> ncopa: got most of the reorg-related pieces in place (without adding the additional functionality yet) -- will have at that tomorrow.
2023-04-30 14:18:43 <ncopa> Do you want me to rebase my MR?
2023-04-30 14:46:58 <tomalok> ncopa: let me see how far i get today.  is the autoinstall YAML its own thing, or is it valid #cloud-config format?
2023-04-30 16:39:56 <ncopa> Most of it is cloud-config compatible. apk section is not but I think we should fix it in cloud-init 
2023-04-30 16:42:43 <minimal> ncopa: that is on my list to things to look into, I just haven't gotten to it yet
2023-04-30 18:35:29 <tomalok> if it's cloud-config-ish compatible start the yaml file with a #cloud-config line -- instead fo is_userdata_script() we now have userdata_type()