2023-01-12 15:20:24 btw, 3.17.1 cloud images have been published 2023-01-17 13:29:40 hi 2023-01-17 13:29:53 i have been looking at tiny-cloud-nocloud 2023-01-17 13:29:56 it looks really nice 2023-01-17 13:31:42 i was thinking. what if we during initramfs, first boot (not root= as boot option), no apkovl file found, do a `findfs LABEL=cidata`, and if cidata is found, we install tiny-cloud-nocloud provider, and enable the openrc services 2023-01-17 13:44:43 the idea is to use this as base for unattended installs 2023-01-17 18:40:54 ncopa - so tiny-cloud-nocloud would be baked into... all images by default? and then turned on iff cidata's on the kernel commandline? 2023-01-17 18:41:27 thats the idea yes 2023-01-17 18:41:39 but im not 100% its a good idea 2023-01-17 18:42:15 actually, findfs LABEL=cidata would be used 2023-01-17 18:42:40 which means you could attach an iso or disk image with filesystem labeled "cidata" 2023-01-17 18:42:56 and it would enable tiny-cloud-nocloud 2023-01-17 18:43:25 i am thinking of an unattended installer 2023-01-17 18:44:17 other option woudl be to have a custom filesystem label, like LABEL=autoinstall or similar 2023-01-17 18:44:22 tomalok: how compatible with cloud-init is the user-data handler? 2023-01-17 18:45:08 i did a boot iso here with it, and it failed due to it could not configure the network 2023-01-17 18:45:53 minimal: user-data will be installed -- but only executed if it's a script... tiny-cloud does not yet support parsing/taking action from a #cloud-config (iirc) userdata 2023-01-17 18:46:00 i think we could also add a check for tmpfs for the rootfs resize 2023-01-17 18:46:37 so, my idea was to make it parse yaml user data 2023-01-17 18:46:45 tomalok: ah....I saw mention of a yaml tool and assumed you'd added some (limited) cloud-config support 2023-01-17 18:46:46 the yx tool is super nice! 2023-01-17 18:46:48 but nocloud provider will install userdata, vendordata, and network config data if it exists on the volume. the userdata could be a script that pulls other information from the other 2023-01-17 18:47:08 ncopa: thx -- was wondering if anyone noticed yx 2023-01-17 18:47:44 i was thinking of implement it using lua, but this is kinda nicer 2023-01-17 18:47:49 or a "autoinstall" script could already be on the image to do whatever based on the userdata/vendordata/networkconfig files pulled in 2023-01-17 18:48:55 i planned to eventually extend tiny-cloud to support some of the more popular #cloud-config stuff natively (probably via an optional plugin) but have not found the time to do so 2023-01-17 18:49:27 tomalok: what's the format of network-config? cloud-init v1 or v2? something else? 2023-01-17 18:50:05 i was thinking of extend tiny-cloud support of configure network, add users (and groups?) with ssh keys add packages, start sshd 2023-01-17 18:50:31 then it would be useful in the base image 2023-01-17 18:51:26 could also look for the "autoinstall" key in config, and pull in an auto-installer 2023-01-17 18:51:29 https://ubuntu.com/server/docs/install/autoinstall 2023-01-17 18:52:23 minimal: it just installs it if it exists, does nothing with it currently 2023-01-17 18:52:23 this is the only change need in initramfs script to get started: https://tpaste.us/4oew 2023-01-17 18:53:23 I was thinking we could implement "network-interfaces" key 2023-01-17 18:53:32 sorry getting pulled back into day job hell ... :( 2023-01-17 18:53:34 https://cloudinit.readthedocs.io/en/22.4.2/topics/datasources/nocloud.html 2023-01-17 18:54:43 but yeah, the basic idea to ship tiny-cloud-nocloud as a part of the iso images 2023-01-17 18:55:28 ncopa: yeah the Ubuntu subquity uses/abuses cloud-init for its autoinstall 2023-01-17 18:55:43 the idea was to do something similar 2023-01-17 18:56:26 but im not 100% sure its a good idea. cloud-init is sot of designed to solve other problem 2023-01-17 18:56:26 I think that idea is already being done or in process of being done by a commercial Alpine "derivative" 2023-01-17 18:56:46 i think there are like 5 different alpine auto installers out there 2023-01-17 18:57:22 the question is if "cloud-init" config format is a good way to go or not 2023-01-17 18:58:07 yeah I'm thinking of Alpaquita Linux which includes cloud-init, seems the have portions of subquity also so I guess they're autoinstalling in a similar fashion to Ubuntu Server 2023-01-17 18:58:31 and if we use it, we will likely need to extend it or make it different from ubuntu and upstream cloud-init 2023-01-17 18:59:20 ncopa: for cloud-init user-data there's several supported formats, i.e. one is a simple shell script (which is what tiny-cloud NoCloud is doing), the "main" one is the "#cloud-config" YAML format 2023-01-17 18:59:48 im thinking of (ab)use "#cloud-config" yaml format 2023-01-17 18:59:59 but it cannot be 100% compatible with cloud-init 2023-01-17 19:00:10 and im not sure we want it to be 100% compatible either 2023-01-17 19:00:31 which is why i am not sure looking for LABEL=cidata is a good idea 2023-01-17 19:01:36 as an alternative there is Openstacks' ConfigDrive ISO/fs lol 2023-01-17 19:01:50 network_data.json.................no!!!!!!! 2023-01-17 19:02:09 well, i want something that is simple, and similar to cloud-init i think 2023-01-17 19:04:08 so a subnet of https://cloudinit.readthedocs.io/en/latest/reference/modules.html ? 2023-01-17 19:07:07 e.g. write_files, groups, users, set/update hostname, timezone, ssh, set passwords ? 2023-01-17 19:40:20 ncopa: maybe open an issue for tiny-cloud to request implement some minimal level of #cloud-config userdata compatibility -- that would give me a minimal viable product starting point without rabbit holing with features that are not immediately useful 2023-01-17 19:41:28 or if cloud-config format is too complicated, we could define a #tiny-config format to take care what's needed. 2023-01-17 19:41:45 right 2023-01-17 19:42:10 but yx makes this all shell scriptable, in either case. 2023-01-17 19:42:13 not sure if we should use #cloud-config or #tiny-config 2023-01-17 19:42:15 yup 2023-01-17 19:42:24 1. define requirements 2023-01-17 19:42:29 2. think about it 2023-01-17 19:42:37 3. ? 2023-01-17 19:42:39 4. profilt 2023-01-17 19:42:44 :) 2023-01-17 19:42:53 if the format is not compatible (i.e. a subnet) then going with "#tiny-config" makes more sense 2023-01-17 19:43:00 s/subnet/subset/ 2023-01-17 19:43:22 should we look for LABEL=cidata? or other label? 2023-01-17 19:43:28 i suppose it does not matter 2023-01-17 19:43:34 let's stick to what nocloud knows 2023-01-17 19:43:42 ok 2023-01-17 19:44:40 other cloud modules will also take advantage of #cloud-config/#tiny-config 2023-01-17 19:52:51 tomalok: i wonder what you think about kyua/atf for testing? 2023-01-17 19:53:14 I could try set it up for you, and port your current tests 2023-01-17 19:54:03 oh, tiny-cloud does not have tests 2023-01-17 20:48:22 yeah, sorry, never has had tests. the original version was me just booting EC2 instances to test 2023-01-17 20:48:59 what is your general feeling about kyua/atf? 2023-01-17 20:49:20 completely unknown / don't know is a perfectly fine answer 2023-01-17 20:50:20 I can create a draft MR that does a very simple test so you can get a general feeling about it 2023-01-17 20:51:25 hrm, I’ve not used it so my opinion is uneducated. looks nice from the few examples I’ve looked at 2023-01-17 20:52:10 a draft mr would be appreciated if you don’t mind doing it 2023-01-17 20:52:24 the glue is a bit clunky but when the basic is set up its super nice with atf_check 2023-01-17 20:52:46 atf_check -s exit:5 2023-01-17 20:52:59 will execute and expect exit status 5 2023-01-17 20:53:15 although if we need to support yaml config it’s probably time to not have this written in shell scripts 2023-01-17 20:53:56 so, this morning I thought the same, but after having look at yx, i think shell is fine 2023-01-17 20:54:50 we could also rewrite it in lua or rust or something 2023-01-17 20:55:12 i'll create a MR with a sample test using kyua 2023-01-17 20:55:38 my preference would be go or rust if we go that way… something that gives a nice statically linked binary 2023-01-17 20:56:21 i have mixed feelings about those, but yeah, i prefer those over python at least 2023-01-17 20:56:41 lua is nice because it becomes so small. we can implement it all in 1-2 MB 2023-01-17 20:56:45 yeah python’s dependency graph is way way too big IMO 2023-01-17 20:56:52 less that we an do hello world in go 2023-01-17 20:56:58 rust is not very good at static binaries 2023-01-17 20:57:22 as i understand rust can generate smaller binaries, but rust lang is very complex compared to things like lua 2023-01-17 20:57:29 haha. we could strip the go binary but lua will always win. I’m just not a big lua fan 2023-01-17 20:58:02 5-10 years ago i woudl really push for lua, but today? probably not 2023-01-17 20:58:35 i think we can continue with shell, thanks to yx 2023-01-17 20:58:59 yeah yx may solve it. I’ll have to take a look when I get back to a non-touch keyboard 2023-01-17 20:59:26 just trying to parse yaml in pure shell seems like hell 2023-01-17 20:59:37 yeah. thats not doable 2023-01-17 21:00:45 what i had in mind was implement something like yx in lua and do most of it in shell. but yx is even better. no need for lua 2023-01-17 21:02:18 it would be really nice to have tiny-cloud in the base. that would make a lot of stuff I deal with easier 2023-01-17 21:02:56 okay… I’ve got to run again but will take a look at stuff later tonight (PST) 2023-01-17 21:03:04 thanks! 2023-01-17 21:25:18 i've created a draft MR https://gitlab.alpinelinux.org/alpine/cloud/tiny-cloud/-/merge_requests/30 and added a pipeline that tests the scripts with a handful of shells 2023-01-17 21:26:35 the ci currently fails because assemble-interfaces outputs something to stderr - which is ok so i need to fix the test 2023-01-17 21:31:27 anyway, it should give you a feeling of how kyua/atf works 2023-01-24 17:18:31 mcrute, tomalok: what are your thoughts about using kuya for tiny-cloud? 2023-01-24 17:19:25 i have written hundreds of shell script tests for alpine-conf and abuild already 2023-01-24 17:20:22 and i have written a fake wget so we can simulate what happens when downloading things and what happens when wget fails etc 2023-01-24 17:20:41 i am willing to help with writing tests too 2023-01-24 17:45:13 I looked at the MR that ncopa sent and it looks like a nice improvement. I'm +1 on it 2023-01-24 17:45:18 thanks ncopa! 2023-01-24 17:45:44 oh haha, reading fast, I thought that was tomalok talking 2023-01-24 18:43:39 ok. good, i'll continue in that direction then