Well not the Lisp userland many of us dreamed of but a very interesting project nonetheless.
The handbook[0] is a bit more clearer about the goals:
> Metropolis is a cluster operating system, meaning its goal is to run on a fleet of machines (be it physical or virtual) and pool their resources together into a unified API for operations and developer teams.
> A self-contained operating system: Metropolis is a full software stack, including the Linux kernel, userspace code, Kubernetes distribution and cluster management system. In contrast to traditional cluster administration, there are no puzzles to put together from a dozen vendors. The entire stack is tested as a single deployable unit.
> Eliminates state: Metropolis nodes don't have a traditional read-write filesystem, all of their state is contained on a separate partition with clear per-component ownership of data. All node configuration is managed declaratively on a per-node basis, and all cluster operations are all done by gRPC API.
Seems like they a specific use case in mind which will help keep the scope in check. Definitely looking forward to learn more about he project.
Also golang is a pretty good choice for such a project because it doesn't even have a libc dependency and calls the Kernel directly. So you can indeed have a very pure userland in a garbage collected language.
[0] https://docs.monogon.dev/metropolis-v0.1/handbook/index.html
Hello, I'm one of the contributors to Monogon OS (codename Metropolis).
We didn't expect to go public with our project for a few more months, which is why the end-user documentation is still very bare, sorry! But fixing that is something we're prioritizing over the next few quarters.
But I guess the cat's out of the bag, so I might as well stay around and answer any questions that pop up. We've also quickly set up a public Matrix room, which was long overdue: https://app.element.io/#/room/#monogon-os-community:matrix.o...
This looks really nice.
How do you handle Ingress and network policies? How do you handle multiple clusters on a single physical machine (if I read the docs correctly) for ingress?
Network policies are currently not implemented.
Bring your favourite Ingress controller. We support NodePort services, with plans for a LoadBalancer Service controller given the right network topology (a la MetalLB - either BGP to TOR switches to announce /32 / /128s, or maybe ARP/NDP-based annoucements for simpler deployments).
We don't support colocating multiple Kubernetes clusters on a single machine, or even within a single Monogon OS cluster. If you want multiple Kubernetes clusters, run multiple Monogon OS clusters.
How does this compare to gVisor?
We actually provide gVisor as the default runtime for Kubernetes workloads.
Monogon OS implements the layers underneath Kubernetes and gVisor - it's the operating system which runs on your nodes/machines and which comes together to form a cluster.
Still reading the handbook but this reminds me of Talos Linux[0] - which is also a pure golang k8s focused linux distro.
Thank you for sending this, Talos looks cool! I would not recommend actually running it in production though -- it does not seem possible to set up in a secure way. (unless you have an out-of-band VPN to the machine?)
See this: https://www.talos.dev/v1.6/introduction/getting-started/#mod...
The first time you send the machine config, you have to use the --insecure flag to avoid verifying its TLS cert. More concerning, there seems to be no way for you to authenticate yourself to the new machine. Anyone (most likely an automated scanner) could come in and make it theirs at this point.
Is there a solution for that?
Sure, there are solutions presented in the installation guide [1]. It usually involves using the cloud or virtualization platform's out of band channel, which Talos all supports, to securely provision a config on first boot.
You can also generate a custom installation medium or cloud image that pulls config from your trusted machines if you cannot use out-of-band provisioning.
You can also securely use the insecure maintenance mode when there is a firewall in front of the machine, which prevents access by non-administrator clients to the API ports on IP level.
I'm not a fan of Talos booting into insecure maintenance mode without config w/o prompting for at least a PIN displayed on-screen, but the problem you're describing in no way prevents production use.
You can build a custom iso with a "talos.config" kernel parameter set which instructs Talos to download and apply a configuration on boot.
How can you bootstrap a PKI without having a trusted out of band channel?
You can’t, but other commenters pointed out that the OOB is specialized to each cloud and in another part of the guide.
We run Talos in production at Turnkey, including using it to schedule Nitro Enclaves so our most critical workloads can survive even if the whole cluster is compromised.
As others have mentioned, you do need a bit of DIY boootstrapping for PKI. Hopefully we can make our setup portable enough to open source soon.
Talos appears to rely on existing components to a somewhat greater degree in comparison: https://github.com/siderolabs/pkgs
I had the some thought pop up. I am curious if someone more familiar can do any comparison between the two.
There is also gokrazy[^1], which isn't focused on k8s, but on deploying on a rpi.
[^1]: https://gokrazy.org/
gokrazy can also be used to build little VM images: https://gokrazy.org/userguide/qemu/
This bears passing resemblance to Aurae [0]. Both projects are trying to reimagine the space between the Linux kernel and the cluster using modern languages for improved safety. Beyond that, I can’t confidently compare and contrast the projects at this time.
Perhaps they could merge into systemd, so we can finally be done with it and get to systemd/Linux.
Isn't u-root also basically this?
It looks similar to u-root https://github.com/u-root/u-root, yes, used as part of host firmware. There's a description of u-root in chapter 6 of https://link.springer.com/book/10.1007/978-1-4842-7939-7, too.
I used to be very anti-Go, but I am coming to like it, especially the projects built with it. Nice work!
How can one be anti-Go? It's a solid, fairly resource unintensive language.
Zero abstractability, extremely imperative. It's very good at what it's for, but it's absolutely terrible for anyone who prefers a declarative or even functional style
Is there an advantage to language designed to mix the styles?
Python, Java and C++ do attempt to support all styles to varying degree's. Some of those languages have had varying success. :P
The tradeoff is being able to decide how your project should be organized rather than someone else deciding for you. Sometimes you want that, sometimes you don't. The advantage of mixing paradigms in a single technology is the ability to switch / decide on paradigms without switching technologies.
It’s too simple and refuses to offer any sort of syntactical sugar. Error handling requires lots of boilerplate. No enumerations. Etc etc.
> No enumerations.
Of course it has enumerations. It has had them from the get-go. After all, enums are a workaround to help deal with the shortcomings of a basic type system. You're bound to find them anywhere you find a basic type system, which certainly describes Go.
In fairness, there does seem to be some confusion floating around about the difference between enums and sum types, with some mistakenly calling sum types enums. Indeed, Go does not have sum types. This is what languages with more advanced type systems are bound to provide instead. Enums are pointless when you have a more expressive type system.
> Error handling requires lots of boilerplate.
What patterns are you seeing in other languages when it comes to error handling without the boilerplate?
Certainly there are other languages that provide shortcuts when not handling errors, but that's something else entirely.
Enumeration is traditionally defined as establishing a number of something. iota accomplishes just that. That is what it is there for. Same as every other language that supports enums.
Maybe you’re thinking of the new definition that has recently emerged from confused Rust users that’s just another name for sum types? That’s actually sum types, not enumeration, though.
Letting an error bubble up isn’t handling it. Still interested in seeing what other languages are doing to eliminate the boilerplate when handling errors. For some reason you forgot to speak to it…
Explanation for enumeration: the thing about enumeration is you can ensure that you do an exhaustive handling of all cases in a switch statement. If an enum changes in a future library version you can get a compile error wherever the enum is enumerated in code which helps you handle new cases due to new features in a library. Go can’t do that.
I will agree on Go having bad error handling. That is the one pain point for me. As for complaints about lack of syntactic sugar and anti-expressivity: that's a feature, not a bug.
I dislike Go. It's almost anti-expressive.
It's fine if you want a dull language that your junior devs will find hard to shoot themselves in the foot with. It's like a safer C.
Remove the word "junior" and that's totally my opinion and why I absolutely adore it.
I can write a program while drunk or high and I know that I'll get 99% there, on the other hand python, js and C++ let me write basically whatever and it even let me run it.
Too much typing ‘if err..’ everywhere
Basically Inferno + Limbo revisited.
Or VMS even.
Boneworks-inspired name?
Huh, interesting...oh wait, bazel. closes tab
I'm not familiar with it, can you elaborate why it's controversial?
Bazel is the externalized version of Google's internal build system. I think it has likely been overused by the overzealous in situations where its complexity is unwarranted and so a lot of people dislike it.
However it is absolutely fantastic at what it's intended for, providing a single build system across a large multilingual codebase, and building/running quickly, with extras for running tests, etc.
It also, AFAIR, gives you full reproducibility which is nice.
[flagged]
I for one, celebrate it, regardless of my dislike for Go's type system.
Every step helping taking C out of the picture is a welcome one.
At least it isn't like Microsoft where not matter what, .NET can't displace COM and C++ role on Windows, and isn't even used on new Azure Kubernetes samples.
https://learn.microsoft.com/en-us/samples/azure-samples/aks-...
I wouldn't mind Go if their executables didn't take hundreds of megabytes to do something a C/C++ executable could do in less than 2 megabytes.
There are a bunch of executable size issues languishing on their issue tracker and from what I understand it has to do with how some core features of the language are implemented that causes a combinatoric explosion of entries in a table in the executable. They also made an adjustment to stop compressing that table, which makes sense from a startup time perspective but there are also no levers I can pull to compress the table or disable the feature.
.NET executables have the same issue but I can always dynamically link them against a single runtime.
Funnily enough, .NET's NativeAOT took the opposite route and various tables have dehydrated form in the binary which will be hydrated at startup (with the size wins observed up to 30-35% depending on publish contents)
If you're interested, here's the PR history for the feature:
- https://github.com/dotnet/runtime/pull/77884
- https://github.com/dotnet/runtime/pull/79209
- https://github.com/dotnet/runtime/pull/78545
- https://github.com/dotnet/runtime/pull/79732
- https://github.com/dotnet/runtime/pull/78546
- https://github.com/dotnet/runtime/pull/78688
- https://github.com/dotnet/runtime/pull/78748
And if anything, you can always UPX them except on macOS where it is broken for some reason. There also exists a package that adds it as a build step: https://www.nuget.org/packages/PublishAotCompressed
Yes, this exact reason is why I'm not using UPX either. But it is useful when the conversations eventually devolve into binary size golfing ;) (ironic, I think NativeAOT does really well as of .NET 8 in this department).
I'm curious, what are the dependencies of that said Go program for it to take 100 MiB? Can't imagine a .NET (AOT or trimmed self-contained JIT) binary taking this much unless it is a GUI application.
What makes this a "systems programming project"?
eww, gross
Is this busybox in Go?
What am I looking at?
A sqlc compiler? Why not use sqlite, which is proven beyond doubt?
A POSIX.2 userland in Go would not be unwelcomed in any way.
Am I the only one who thinks it would be fun to try/put this on a Raspberry Pi cluster for fun?
This looks interesting. However I can't find any documentation on how to program for it ? Where is their documentation about how we use this "unified API" which pools resources between nodes ?
It's somewhere in my git stack :).
Until I get to publishing it, the proto/gRPC definitions for node management are a good enough start: https://github.com/monogon-dev/monogon/blob/main/metropolis/...
And the top level API to actually deploy workloads is plain Kubernetes.
IMVHO it's just another demonstration we need code and production together, we need end user programming and most try to deny that, while all tend to, a very small step at a time.
Python popularity, Notebook UIs, WebApps vs widget-based GUIs, this project... are all small examples of the same issue.
Is the end user you refer to a highly technical person? Because 1 in 5 Americans do not have the literacy skills to "complete tasks that require comparing and contrasting information, paraphrasing, or making low-level inferences"[0] so if you're describing end user software as something usable by everyone, expecting users to become programmers is simply an unrealistic expectation. The below average person is simply not smart enough to learn a DSL to get something done, and software should be accessible to everyone, even those who weren't lucky in the IQ lottery.
0: https://nces.ed.gov/pubs2019/2019179/index.asp
Well, "my" end-user is someone who have studied "computer science" a bit at school or to work, since he/she use computers every days, like he/she drive a car, so it's not acceptable he/she master computer usage less at level of mastery inferior to the one he/she master his/her car.
I know very well how illiterate so many people are, but that's not something to be justified but to be corrected. I imaging you do not want a bus driver who can barely make the bus move on the road piloting the bus you are on, why it's normal for you that let's say a tax administration employee is just able to click around while dealing with your taxes?
I've made a small experiment few years ago: with the help of some friend we have introduced for their very first time few kids to a desktop, a NixOS one running Emacs/EXWM. In 2 years at 6-8 years old, they was able to deal with emails, create some nicely formatted documents and doing basic math and so on with it. Two of them was presented a classic modern desktop: they became unable to do practically anything. It's a VERY small experiment of course, but to me it's enough to prove that people can learn if they are pushed toward a certain direction.
It's not that, it's another point: efficiency vs assembly line. Humans are not build to be robots, they are build to evolve.
If we learn a classic desktop paradigm computer system at school, while we learn all the basic and less basic cultural stuff we learn at school to became Citizens, we can profit from this knowledge for life. We can choose to dig deeper or not, but we have something useful for our entire life.
If we learn the modern desktop paradigm we never evolve. We are not Citizens, we are workers in an assembly line and upon any change from the factory owner our acquired knowledge goes to the bin.
I hope to have successfully described the point in my poor English. To give a simple example, I track my bills (well, like many, nothing special), I've crafted a bit of automation (org-mode notes + BeanCount + a bit of py automation), it took two/three days, MUCH more than most users do with modern tools, BUT thereafter anything goes nearly autonomously so I've spent 2/3 days + few seconds looking at my org-agenda regularly vs few minutes to start than keep spending few minutes all the time. In the short term the classic model is not good, in the middle term it's equivalent but demand more intellectual work, in the long term outshine the modern one so much that's like comparing a runner by feet against one on a jet in a speed race. Actually the effort spent in automating my bills notes it's also useful (at least in some parts) to automated other stuff, a bit at a time, an evolutionary step at a time I've built and keep up my PIM, again all the effort put pay back and I've gained valuable knowledge from doing that. The modern approach seems cheaper at first but it's much more expensive in the long run and gives little to no valuable knowledge at all.
The same model apply to any other aspects of our life, one to remain in economics the "ownership model" vs the "rent model", owning a home seems to be much, much more complex than rent one, it demand much more resources, much more computations and projection up front etc BUT it pay back much more thereafter. The rented home is just a regular expense that pile up year after year and at the end you have nothing. The owned home in most cases (essentially all, with insurances) have a final value, normally a big enough one to pay back the capex + opex of the time passed.
Since we are not made to live a single life but to evolve, doing better things a generation after another, passing knowledge and anything we can to newer generation, the modern paradigm is the slave paradigm, he/she produce a new generation that have no benefit from the old one, get nothing, leave nothing in the end. The classic is the human model, where we build families, passing what we have built and the accumulated knowledge, improving a generation after another.
Every piece of software doesn't need to be for everyone. Designing interfaces for the lowest common denominator throws away a lot of ideas that may make some people more productive to benefit people who won't be.