2023-01-12 15:20:24 <tomalok> btw, 3.17.1 cloud images have been published
2023-01-17 13:29:40 <ncopa> hi
2023-01-17 13:29:53 <ncopa> i have been looking at tiny-cloud-nocloud
2023-01-17 13:29:56 <ncopa> it looks really nice
2023-01-17 13:31:42 <ncopa> i was thinking. what if we during initramfs, first boot (not root= as boot option), no apkovl file found, do a `findfs LABEL=cidata`, and if cidata is found, we install tiny-cloud-nocloud provider, and enable the openrc services
2023-01-17 13:44:43 <ncopa> the idea is to use this as base for unattended installs
2023-01-17 18:40:54 <tomalok> ncopa - so tiny-cloud-nocloud would be baked into... all images by default? and then turned on iff cidata's on the kernel commandline?
2023-01-17 18:41:27 <ncopa> thats the idea yes
2023-01-17 18:41:39 <ncopa> but im not 100% its a good idea
2023-01-17 18:42:15 <ncopa> actually, findfs LABEL=cidata would be used
2023-01-17 18:42:40 <ncopa> which means you could attach an iso or disk image with filesystem labeled "cidata"
2023-01-17 18:42:56 <ncopa> and it would enable tiny-cloud-nocloud
2023-01-17 18:43:25 <ncopa> i am thinking of an unattended installer
2023-01-17 18:44:17 <ncopa> other option woudl be to have a custom filesystem label, like LABEL=autoinstall or similar
2023-01-17 18:44:22 <minimal> tomalok: how compatible with cloud-init is the user-data handler?
2023-01-17 18:45:08 <ncopa> i did a boot iso here with it, and it failed due to it could not configure the network
2023-01-17 18:45:53 <tomalok> minimal: user-data will be installed -- but only executed if it's a script...  tiny-cloud does not yet support parsing/taking action from a #cloud-config (iirc) userdata
2023-01-17 18:46:00 <ncopa> i think we could also add a check for tmpfs for the rootfs resize
2023-01-17 18:46:37 <ncopa> so, my idea was to make it parse yaml user data
2023-01-17 18:46:45 <minimal> tomalok: ah....I saw mention of a yaml tool and assumed you'd added some (limited) cloud-config support
2023-01-17 18:46:46 <ncopa> the yx tool is super nice!
2023-01-17 18:46:48 <tomalok> but nocloud provider will install userdata, vendordata, and network config data if it exists on the volume.  the userdata could be a script that pulls other information from the other
2023-01-17 18:47:08 <tomalok> ncopa: thx -- was wondering if anyone noticed yx
2023-01-17 18:47:44 <ncopa> i was thinking of implement it using lua, but this is kinda nicer
2023-01-17 18:47:49 <tomalok> or a "autoinstall" script could already be on the image to do whatever based on the userdata/vendordata/networkconfig files pulled in
2023-01-17 18:48:55 <tomalok> i planned to eventually extend tiny-cloud to support some of the more popular #cloud-config stuff natively (probably via an optional plugin) but have not found the time to do so
2023-01-17 18:49:27 <minimal> tomalok: what's the format of network-config? cloud-init v1 or v2? something else?
2023-01-17 18:50:05 <ncopa> i was thinking of extend tiny-cloud support of configure network, add users (and groups?) with ssh keys add packages, start sshd
2023-01-17 18:50:31 <ncopa> then it would be useful in the base image
2023-01-17 18:51:26 <ncopa> could also look for the "autoinstall" key in config, and pull in an auto-installer
2023-01-17 18:51:29 <ncopa> https://ubuntu.com/server/docs/install/autoinstall
2023-01-17 18:52:23 <tomalok> minimal: it just installs it if it exists, does nothing with it currently
2023-01-17 18:52:23 <ncopa> this is the only change need in initramfs script to get started: https://tpaste.us/4oew
2023-01-17 18:53:23 <ncopa> I was thinking we could implement "network-interfaces" key
2023-01-17 18:53:32 <tomalok> sorry getting pulled back into day job hell ... :(
2023-01-17 18:53:34 <ncopa> https://cloudinit.readthedocs.io/en/22.4.2/topics/datasources/nocloud.html
2023-01-17 18:54:43 <ncopa> but yeah, the basic idea to ship tiny-cloud-nocloud as a part of the iso images
2023-01-17 18:55:28 <minimal> ncopa: yeah the Ubuntu subquity uses/abuses cloud-init for its autoinstall
2023-01-17 18:55:43 <ncopa> the idea was to do something similar
2023-01-17 18:56:26 <ncopa> but im not 100% sure its a good idea. cloud-init is sot of designed to solve other problem
2023-01-17 18:56:26 <minimal> I think that idea is already being done or in process of being done by a commercial Alpine "derivative"
2023-01-17 18:56:46 <ncopa> i think there are like 5 different alpine auto installers out there
2023-01-17 18:57:22 <ncopa> the question is if "cloud-init" config format is a good way to go or not
2023-01-17 18:58:07 <minimal> yeah I'm thinking of Alpaquita Linux which includes cloud-init, seems the have portions of subquity also so I guess they're autoinstalling in a similar fashion to Ubuntu Server
2023-01-17 18:58:31 <ncopa> and if we use it, we will likely need to extend it or make it different from ubuntu and upstream cloud-init
2023-01-17 18:59:20 <minimal> ncopa: for cloud-init user-data there's several supported formats, i.e. one is a simple shell script (which is what tiny-cloud NoCloud is doing), the "main" one is the "#cloud-config" YAML format
2023-01-17 18:59:48 <ncopa> im thinking of (ab)use "#cloud-config" yaml format
2023-01-17 18:59:59 <ncopa> but it cannot be 100% compatible with cloud-init
2023-01-17 19:00:10 <ncopa> and im not sure we want it to be 100% compatible either
2023-01-17 19:00:31 <ncopa> which is why i am not sure looking for LABEL=cidata is a good idea
2023-01-17 19:01:36 <minimal> as an alternative there is Openstacks' ConfigDrive ISO/fs lol
2023-01-17 19:01:50 <minimal> network_data.json.................no!!!!!!!
2023-01-17 19:02:09 <ncopa> well, i want something that is simple, and similar to cloud-init i think
2023-01-17 19:04:08 <minimal> so a subnet of https://cloudinit.readthedocs.io/en/latest/reference/modules.html ?
2023-01-17 19:07:07 <minimal> e.g. write_files, groups, users, set/update hostname, timezone, ssh, set passwords ?
2023-01-17 19:40:20 <tomalok> ncopa: maybe open an issue for tiny-cloud to request implement some minimal level of #cloud-config userdata compatibility -- that would give me a minimal viable product starting point without rabbit holing with features that are not immediately useful
2023-01-17 19:41:28 <tomalok> or if cloud-config format is too complicated, we could define a #tiny-config format to take care what's needed.
2023-01-17 19:41:45 <ncopa> right
2023-01-17 19:42:10 <tomalok> but yx makes this all shell scriptable, in either case.
2023-01-17 19:42:13 <ncopa> not sure if we should use #cloud-config or #tiny-config
2023-01-17 19:42:15 <ncopa> yup
2023-01-17 19:42:24 <tomalok> 1. define requirements
2023-01-17 19:42:29 <tomalok> 2. think about it
2023-01-17 19:42:37 <tomalok> 3. ?
2023-01-17 19:42:39 <tomalok> 4. profilt
2023-01-17 19:42:44 <ncopa> :)
2023-01-17 19:42:53 <minimal> if the format is not compatible (i.e. a subnet) then going with "#tiny-config" makes more sense
2023-01-17 19:43:00 <minimal> s/subnet/subset/
2023-01-17 19:43:22 <ncopa> should we look for LABEL=cidata? or other label?
2023-01-17 19:43:28 <ncopa> i suppose it does not matter
2023-01-17 19:43:34 <tomalok> let's stick to what nocloud knows
2023-01-17 19:43:42 <ncopa> ok
2023-01-17 19:44:40 <tomalok> other cloud modules will also take advantage of #cloud-config/#tiny-config
2023-01-17 19:52:51 <ncopa> tomalok: i wonder what you think about kyua/atf for testing?
2023-01-17 19:53:14 <ncopa> I could try set it up for you, and port your current tests
2023-01-17 19:54:03 <ncopa> oh, tiny-cloud does not have tests
2023-01-17 20:48:22 <mcrute> yeah, sorry, never has had tests. the original version was me just booting EC2 instances to test
2023-01-17 20:48:59 <ncopa> what is your general feeling about kyua/atf?
2023-01-17 20:49:20 <ncopa> completely unknown / don't know is a perfectly fine answer
2023-01-17 20:50:20 <ncopa> I can create a draft MR that does a very simple test so you can get a general feeling about it
2023-01-17 20:51:25 <mcrute> hrm, I’ve not used it so my opinion is uneducated. looks nice from the few examples I’ve looked at
2023-01-17 20:52:10 <mcrute> a draft mr would be appreciated if you don’t mind doing it
2023-01-17 20:52:24 <ncopa> the glue is a bit clunky but when the basic is set up its super nice with atf_check
2023-01-17 20:52:46 <ncopa> atf_check -s exit:5 <cmd>
2023-01-17 20:52:59 <ncopa> will execute <cmd> and expect exit status 5
2023-01-17 20:53:15 <mcrute> although if we need to support yaml config it’s probably time to not have this written in shell scripts
2023-01-17 20:53:56 <ncopa> so, this morning I thought the same, but after having look at yx, i think shell is fine
2023-01-17 20:54:50 <ncopa> we could also rewrite it in lua or rust or something
2023-01-17 20:55:12 <ncopa> i'll create a MR with a sample test using kyua
2023-01-17 20:55:38 <mcrute> my preference would be go or rust if we go that way… something that gives a nice statically linked binary
2023-01-17 20:56:21 <ncopa> i have mixed feelings about those, but yeah, i prefer those over python at least
2023-01-17 20:56:41 <ncopa> lua is nice because it becomes so small. we can implement it all in 1-2 MB
2023-01-17 20:56:45 <mcrute> yeah python’s dependency graph is way way too big IMO
2023-01-17 20:56:52 <ncopa> less that we an do hello world in go
2023-01-17 20:56:58 <psykose> rust is not very good at static binaries
2023-01-17 20:57:22 <ncopa> as i understand rust can generate smaller binaries, but rust lang is very complex compared to things like lua
2023-01-17 20:57:29 <mcrute> haha. we could strip the go binary but lua will always win. I’m just not a big lua fan
2023-01-17 20:58:02 <ncopa> 5-10 years ago i woudl really push for lua, but today? probably not
2023-01-17 20:58:35 <ncopa> i think we can continue with shell, thanks to yx
2023-01-17 20:58:59 <mcrute> yeah yx may solve it. I’ll have to take a look when I get back to a non-touch keyboard
2023-01-17 20:59:26 <mcrute> just trying to parse yaml in pure shell seems like hell
2023-01-17 20:59:37 <ncopa> yeah. thats not doable
2023-01-17 21:00:45 <ncopa> what i had in mind was implement something like yx in lua and do most of it in shell. but yx is even better. no need for lua
2023-01-17 21:02:18 <mcrute> it would be really nice to have tiny-cloud in the base. that would make a lot of stuff I deal with easier
2023-01-17 21:02:56 <mcrute> okay… I’ve got to run again but will take a look at stuff later tonight (PST)
2023-01-17 21:03:04 <ncopa> thanks!
2023-01-17 21:25:18 <ncopa> i've created a draft MR https://gitlab.alpinelinux.org/alpine/cloud/tiny-cloud/-/merge_requests/30 and added a pipeline that tests the scripts with a handful of shells
2023-01-17 21:26:35 <ncopa> the ci currently fails because assemble-interfaces outputs something to stderr - which is ok so i need to fix the test
2023-01-17 21:31:27 <ncopa> anyway, it should give you a feeling of how kyua/atf works
2023-01-24 17:18:31 <ncopa> mcrute, tomalok: what are your thoughts about using kuya for tiny-cloud?
2023-01-24 17:19:25 <ncopa> i have written hundreds of shell script tests for alpine-conf and abuild already
2023-01-24 17:20:22 <ncopa> and i have written a fake wget so we can simulate what happens when downloading things and what happens when wget fails etc
2023-01-24 17:20:41 <ncopa> i am willing to help with writing tests too
2023-01-24 17:45:13 <mcrute> I looked at the MR that ncopa sent and it looks like a nice improvement. I'm +1 on it
2023-01-24 17:45:18 <mcrute> thanks ncopa!
2023-01-24 17:45:44 <mcrute> oh haha, reading fast, I thought that was tomalok talking
2023-01-24 18:43:39 <ncopa> ok. good, i'll continue in that direction then
2023-01-29 20:08:55 <minimal> Hmm, Azure marketplace requirement for Linux VM disk images: https://learn.microsoft.com/en-us/legal/marketplace/certification-policies#20033-linux
2023-01-29 20:09:00 <minimal> "OS disk size validation should be between 30GB and 1023GB" lol
2023-01-29 20:09:54 <ikke> :D
2023-01-29 20:10:06 <ikke> Does add to their bottomline
2023-01-29 20:10:16 <ikke> What do you mean, your OS disk is just 2G?
2023-01-29 20:10:36 <minimal> if an Alpine image was created with every single package installed would it be anywhere near 30GB in size? lol
2023-01-29 20:11:10 <minimal> "never mind the quality, feel the size" lol
2023-01-29 20:11:21 <ikke> 34G atm for main + commuinty
2023-01-29 20:11:26 <ikke> community
2023-01-29 20:13:14 <minimal> I dread to think why any OS image would ever need to be 1TB in size
2023-01-30 02:06:47 <tomalok> probably a limitation of their minimum boot disk size -- OCI has a similar minimum size -- that said, a mostly-empty image (as long as it's not raw) is still a l ot smaller than that minimum size.
2023-01-30 02:12:09 <minimal> tomalok: yes VHDs for Azure are sparse, but it means if they are being created using another format first (i.e. RAW) before being converted to VHD then the original image is the full size :-(
2023-01-30 02:14:18 <tomalok> iirc alpine-cloud-images woudl go from qcow2 --> vhd, without the raw.
2023-01-30 02:14:58 <minimal> tomalok: I was talking from my own perspective where I create raw images in generate before converting them to whichever format
2023-01-30 02:15:04 <minimal> s/generate/general/
2023-01-30 02:16:29 <minimal> though I assume that "rule" only applies to their Marketplace rather than private images
2023-01-30 02:24:50 <minimal> looking at an openstack script for Azure that takes a VHD, qemu-img converts it to RAW, resizes it (if needed as Azure requires rounding up to nearest 1MB), then qemu-img converts it again from RAW to VHD lol