2023-01-12 15:20:24 <tomalok> btw, 3.17.1 cloud images have been published
2023-01-17 13:29:40 <ncopa> hi
2023-01-17 13:29:53 <ncopa> i have been looking at tiny-cloud-nocloud
2023-01-17 13:29:56 <ncopa> it looks really nice
2023-01-17 13:31:42 <ncopa> i was thinking. what if we during initramfs, first boot (not root= as boot option), no apkovl file found, do a `findfs LABEL=cidata`, and if cidata is found, we install tiny-cloud-nocloud provider, and enable the openrc services
2023-01-17 13:44:43 <ncopa> the idea is to use this as base for unattended installs
2023-01-17 18:40:54 <tomalok> ncopa - so tiny-cloud-nocloud would be baked into... all images by default? and then turned on iff cidata's on the kernel commandline?
2023-01-17 18:41:27 <ncopa> thats the idea yes
2023-01-17 18:41:39 <ncopa> but im not 100% its a good idea
2023-01-17 18:42:15 <ncopa> actually, findfs LABEL=cidata would be used
2023-01-17 18:42:40 <ncopa> which means you could attach an iso or disk image with filesystem labeled "cidata"
2023-01-17 18:42:56 <ncopa> and it would enable tiny-cloud-nocloud
2023-01-17 18:43:25 <ncopa> i am thinking of an unattended installer
2023-01-17 18:44:17 <ncopa> other option woudl be to have a custom filesystem label, like LABEL=autoinstall or similar
2023-01-17 18:44:22 <minimal> tomalok: how compatible with cloud-init is the user-data handler?
2023-01-17 18:45:08 <ncopa> i did a boot iso here with it, and it failed due to it could not configure the network
2023-01-17 18:45:53 <tomalok> minimal: user-data will be installed -- but only executed if it's a script...  tiny-cloud does not yet support parsing/taking action from a #cloud-config (iirc) userdata
2023-01-17 18:46:00 <ncopa> i think we could also add a check for tmpfs for the rootfs resize
2023-01-17 18:46:37 <ncopa> so, my idea was to make it parse yaml user data
2023-01-17 18:46:45 <minimal> tomalok: ah....I saw mention of a yaml tool and assumed you'd added some (limited) cloud-config support
2023-01-17 18:46:46 <ncopa> the yx tool is super nice!
2023-01-17 18:46:48 <tomalok> but nocloud provider will install userdata, vendordata, and network config data if it exists on the volume.  the userdata could be a script that pulls other information from the other
2023-01-17 18:47:08 <tomalok> ncopa: thx -- was wondering if anyone noticed yx
2023-01-17 18:47:44 <ncopa> i was thinking of implement it using lua, but this is kinda nicer
2023-01-17 18:47:49 <tomalok> or a "autoinstall" script could already be on the image to do whatever based on the userdata/vendordata/networkconfig files pulled in
2023-01-17 18:48:55 <tomalok> i planned to eventually extend tiny-cloud to support some of the more popular #cloud-config stuff natively (probably via an optional plugin) but have not found the time to do so
2023-01-17 18:49:27 <minimal> tomalok: what's the format of network-config? cloud-init v1 or v2? something else?
2023-01-17 18:50:05 <ncopa> i was thinking of extend tiny-cloud support of configure network, add users (and groups?) with ssh keys add packages, start sshd
2023-01-17 18:50:31 <ncopa> then it would be useful in the base image
2023-01-17 18:51:26 <ncopa> could also look for the "autoinstall" key in config, and pull in an auto-installer
2023-01-17 18:51:29 <ncopa> https://ubuntu.com/server/docs/install/autoinstall
2023-01-17 18:52:23 <tomalok> minimal: it just installs it if it exists, does nothing with it currently
2023-01-17 18:52:23 <ncopa> this is the only change need in initramfs script to get started: https://tpaste.us/4oew
2023-01-17 18:53:23 <ncopa> I was thinking we could implement "network-interfaces" key
2023-01-17 18:53:32 <tomalok> sorry getting pulled back into day job hell ... :(
2023-01-17 18:53:34 <ncopa> https://cloudinit.readthedocs.io/en/22.4.2/topics/datasources/nocloud.html
2023-01-17 18:54:43 <ncopa> but yeah, the basic idea to ship tiny-cloud-nocloud as a part of the iso images
2023-01-17 18:55:28 <minimal> ncopa: yeah the Ubuntu subquity uses/abuses cloud-init for its autoinstall
2023-01-17 18:55:43 <ncopa> the idea was to do something similar
2023-01-17 18:56:26 <ncopa> but im not 100% sure its a good idea. cloud-init is sot of designed to solve other problem
2023-01-17 18:56:26 <minimal> I think that idea is already being done or in process of being done by a commercial Alpine "derivative"
2023-01-17 18:56:46 <ncopa> i think there are like 5 different alpine auto installers out there
2023-01-17 18:57:22 <ncopa> the question is if "cloud-init" config format is a good way to go or not
2023-01-17 18:58:07 <minimal> yeah I'm thinking of Alpaquita Linux which includes cloud-init, seems the have portions of subquity also so I guess they're autoinstalling in a similar fashion to Ubuntu Server
2023-01-17 18:58:31 <ncopa> and if we use it, we will likely need to extend it or make it different from ubuntu and upstream cloud-init
2023-01-17 18:59:20 <minimal> ncopa: for cloud-init user-data there's several supported formats, i.e. one is a simple shell script (which is what tiny-cloud NoCloud is doing), the "main" one is the "#cloud-config" YAML format
2023-01-17 18:59:48 <ncopa> im thinking of (ab)use "#cloud-config" yaml format
2023-01-17 18:59:59 <ncopa> but it cannot be 100% compatible with cloud-init
2023-01-17 19:00:10 <ncopa> and im not sure we want it to be 100% compatible either
2023-01-17 19:00:31 <ncopa> which is why i am not sure looking for LABEL=cidata is a good idea
2023-01-17 19:01:36 <minimal> as an alternative there is Openstacks' ConfigDrive ISO/fs lol
2023-01-17 19:01:50 <minimal> network_data.json.................no!!!!!!!
2023-01-17 19:02:09 <ncopa> well, i want something that is simple, and similar to cloud-init i think
2023-01-17 19:04:08 <minimal> so a subnet of https://cloudinit.readthedocs.io/en/latest/reference/modules.html ?
2023-01-17 19:07:07 <minimal> e.g. write_files, groups, users, set/update hostname, timezone, ssh, set passwords ?
2023-01-17 19:40:20 <tomalok> ncopa: maybe open an issue for tiny-cloud to request implement some minimal level of #cloud-config userdata compatibility -- that would give me a minimal viable product starting point without rabbit holing with features that are not immediately useful
2023-01-17 19:41:28 <tomalok> or if cloud-config format is too complicated, we could define a #tiny-config format to take care what's needed.
2023-01-17 19:41:45 <ncopa> right
2023-01-17 19:42:10 <tomalok> but yx makes this all shell scriptable, in either case.
2023-01-17 19:42:13 <ncopa> not sure if we should use #cloud-config or #tiny-config
2023-01-17 19:42:15 <ncopa> yup
2023-01-17 19:42:24 <tomalok> 1. define requirements
2023-01-17 19:42:29 <tomalok> 2. think about it
2023-01-17 19:42:37 <tomalok> 3. ?
2023-01-17 19:42:39 <tomalok> 4. profilt
2023-01-17 19:42:44 <ncopa> :)
2023-01-17 19:42:53 <minimal> if the format is not compatible (i.e. a subnet) then going with "#tiny-config" makes more sense
2023-01-17 19:43:00 <minimal> s/subnet/subset/
2023-01-17 19:43:22 <ncopa> should we look for LABEL=cidata? or other label?
2023-01-17 19:43:28 <ncopa> i suppose it does not matter
2023-01-17 19:43:34 <tomalok> let's stick to what nocloud knows
2023-01-17 19:43:42 <ncopa> ok
2023-01-17 19:44:40 <tomalok> other cloud modules will also take advantage of #cloud-config/#tiny-config
2023-01-17 19:52:51 <ncopa> tomalok: i wonder what you think about kyua/atf for testing?
2023-01-17 19:53:14 <ncopa> I could try set it up for you, and port your current tests
2023-01-17 19:54:03 <ncopa> oh, tiny-cloud does not have tests
2023-01-17 20:48:22 <mcrute> yeah, sorry, never has had tests. the original version was me just booting EC2 instances to test
2023-01-17 20:48:59 <ncopa> what is your general feeling about kyua/atf?
2023-01-17 20:49:20 <ncopa> completely unknown / don't know is a perfectly fine answer
2023-01-17 20:50:20 <ncopa> I can create a draft MR that does a very simple test so you can get a general feeling about it
2023-01-17 20:51:25 <mcrute> hrm, I’ve not used it so my opinion is uneducated. looks nice from the few examples I’ve looked at
2023-01-17 20:52:10 <mcrute> a draft mr would be appreciated if you don’t mind doing it
2023-01-17 20:52:24 <ncopa> the glue is a bit clunky but when the basic is set up its super nice with atf_check
2023-01-17 20:52:46 <ncopa> atf_check -s exit:5 <cmd>
2023-01-17 20:52:59 <ncopa> will execute <cmd> and expect exit status 5
2023-01-17 20:53:15 <mcrute> although if we need to support yaml config it’s probably time to not have this written in shell scripts
2023-01-17 20:53:56 <ncopa> so, this morning I thought the same, but after having look at yx, i think shell is fine
2023-01-17 20:54:50 <ncopa> we could also rewrite it in lua or rust or something
2023-01-17 20:55:12 <ncopa> i'll create a MR with a sample test using kyua
2023-01-17 20:55:38 <mcrute> my preference would be go or rust if we go that way… something that gives a nice statically linked binary
2023-01-17 20:56:21 <ncopa> i have mixed feelings about those, but yeah, i prefer those over python at least
2023-01-17 20:56:41 <ncopa> lua is nice because it becomes so small. we can implement it all in 1-2 MB
2023-01-17 20:56:45 <mcrute> yeah python’s dependency graph is way way too big IMO
2023-01-17 20:56:52 <ncopa> less that we an do hello world in go
2023-01-17 20:56:58 <psykose> rust is not very good at static binaries
2023-01-17 20:57:22 <ncopa> as i understand rust can generate smaller binaries, but rust lang is very complex compared to things like lua
2023-01-17 20:57:29 <mcrute> haha. we could strip the go binary but lua will always win. I’m just not a big lua fan
2023-01-17 20:58:02 <ncopa> 5-10 years ago i woudl really push for lua, but today? probably not
2023-01-17 20:58:35 <ncopa> i think we can continue with shell, thanks to yx
2023-01-17 20:58:59 <mcrute> yeah yx may solve it. I’ll have to take a look when I get back to a non-touch keyboard
2023-01-17 20:59:26 <mcrute> just trying to parse yaml in pure shell seems like hell
2023-01-17 20:59:37 <ncopa> yeah. thats not doable
2023-01-17 21:00:45 <ncopa> what i had in mind was implement something like yx in lua and do most of it in shell. but yx is even better. no need for lua
2023-01-17 21:02:18 <mcrute> it would be really nice to have tiny-cloud in the base. that would make a lot of stuff I deal with easier
2023-01-17 21:02:56 <mcrute> okay… I’ve got to run again but will take a look at stuff later tonight (PST)
2023-01-17 21:03:04 <ncopa> thanks!
2023-01-17 21:25:18 <ncopa> i've created a draft MR https://gitlab.alpinelinux.org/alpine/cloud/tiny-cloud/-/merge_requests/30 and added a pipeline that tests the scripts with a handful of shells
2023-01-17 21:26:35 <ncopa> the ci currently fails because assemble-interfaces outputs something to stderr - which is ok so i need to fix the test
2023-01-17 21:31:27 <ncopa> anyway, it should give you a feeling of how kyua/atf works
2023-01-24 17:18:31 <ncopa> mcrute, tomalok: what are your thoughts about using kuya for tiny-cloud?
2023-01-24 17:19:25 <ncopa> i have written hundreds of shell script tests for alpine-conf and abuild already
2023-01-24 17:20:22 <ncopa> and i have written a fake wget so we can simulate what happens when downloading things and what happens when wget fails etc
2023-01-24 17:20:41 <ncopa> i am willing to help with writing tests too
2023-01-24 17:45:13 <mcrute> I looked at the MR that ncopa sent and it looks like a nice improvement. I'm +1 on it
2023-01-24 17:45:18 <mcrute> thanks ncopa!
2023-01-24 17:45:44 <mcrute> oh haha, reading fast, I thought that was tomalok talking
2023-01-24 18:43:39 <ncopa> ok. good, i'll continue in that direction then