This is cool! I tried it out, running outside my agent with `cloudrouter start .` and got a password request to auth into the server. Opened an issue[1].
It's a cool idea, but personally I don't like the implementation. I usually don't use monolithic tools that cram a lot of different solutions into one thing. For one thing, especially if they're compiled, it's very hard to just modify them to do one extra thing I need without getting into a long development cycle. For two, they are usually inflexible, restricting what I can do. Third, they often aren't very composeable. Fourth, often they aren't easily pluggable/extensible.
I much prefer independent, loosely coupled, highly cohesive, composeable, extensible tools. It's not a very "programmery" solution, but it makes it easier as a user to fix things, extend things, combine things, etc.
The Docker template you have bundles a ton of apps into one container. This is problematic as it creates a big support burden, build burden, and compatibility burden. Docker works better when you make individual containers of a single app, and run them separately, and connect them with tcp, sockets, or volumes. Then the user can swap them out, add new ones, remove unneded ones, etc, and they can use an official upstream project. Docker-in-docker with a custom docker network works pretty well, and the host is still accessible if needed.
As a nit-pick: your auth code has browser-handling logic. This is low cohesion, a sign of problems to come. And in your rsync code:
I was just commenting the other day on here about how nobody checks SSH host keys and how SSH is basically wide-open due to this. Just leaving this here to show people what I mean. (It's not an easy problem to solve, but ignoring security isn't great either)
Re: monolithic tools. I think having template overrides for the user could solve this issue -- although it is a bit tougher to implement. I wanted a monolithic tool because it optimizes for faster startup times and just works but it does sacrifice configurability for the user.
Re: Docker template. I understand the Docker critique. So, the primary use case is an agent uploading its working directory and spinning it up as a dev environment. The agent needs the project files, the dev server, and the browser all in one place. If these are separate containers, the agent has to reason about volume mounts, Docker networking, etc — potentially more confusion, higher likelihood that agents get something wrong. A single environment where cloudrouter start ./my-project just works is what I envisioned.
Re: SSH host keys. SSH never connects to a real host. It's tunneled through TLS WebSocket via ProxyCommand. Also the hostname is fake, we have per-session auth token on the WebSocket layer, and VMs are ephemeral with fresh keys every boot. So, SSH isn't wide-open. We don't expose the SSH port (port 10000); everything goes through our authenticated proxy.
Totally fair point. For me it was just a nice primitive to have -- just one command gives the agent a VM with SSH, file sync, browser, GPU ready to go. Instead of dealing with cloud account setup, security groups, SSH keys, and other shenanigans. For cloudrouter, the dependencies/Docker/VNC/Juypter Lab come pre-baked so you don't need to think about configuring VM environment setups...
Our approach though is more tide-case agnostic and in the direction of brining full-fledged container orchestration converting from development to training and inference
Myself I already pre-provisioned a kubernetes cluster and it just makes new manifests and deploys there. Less dangerous.
Ah, just one step closer to a model with it's own weights file can bootstrap and run itself.
This is cool! I tried it out, running outside my agent with `cloudrouter start .` and got a password request to auth into the server. Opened an issue[1].
[1] https://github.com/manaflow-ai/manaflow/issues/1711
Hey Nick! I figured out the root cause and have pushed a fix. Could you update the package and try again?
It's a cool idea, but personally I don't like the implementation. I usually don't use monolithic tools that cram a lot of different solutions into one thing. For one thing, especially if they're compiled, it's very hard to just modify them to do one extra thing I need without getting into a long development cycle. For two, they are usually inflexible, restricting what I can do. Third, they often aren't very composeable. Fourth, often they aren't easily pluggable/extensible.
I much prefer independent, loosely coupled, highly cohesive, composeable, extensible tools. It's not a very "programmery" solution, but it makes it easier as a user to fix things, extend things, combine things, etc.
The Docker template you have bundles a ton of apps into one container. This is problematic as it creates a big support burden, build burden, and compatibility burden. Docker works better when you make individual containers of a single app, and run them separately, and connect them with tcp, sockets, or volumes. Then the user can swap them out, add new ones, remove unneded ones, etc, and they can use an official upstream project. Docker-in-docker with a custom docker network works pretty well, and the host is still accessible if needed.
As a nit-pick: your auth code has browser-handling logic. This is low cohesion, a sign of problems to come. And in your rsync code:
I was just commenting the other day on here about how nobody checks SSH host keys and how SSH is basically wide-open due to this. Just leaving this here to show people what I mean. (It's not an easy problem to solve, but ignoring security isn't great either)Re: monolithic tools. I think having template overrides for the user could solve this issue -- although it is a bit tougher to implement. I wanted a monolithic tool because it optimizes for faster startup times and just works but it does sacrifice configurability for the user.
Re: Docker template. I understand the Docker critique. So, the primary use case is an agent uploading its working directory and spinning it up as a dev environment. The agent needs the project files, the dev server, and the browser all in one place. If these are separate containers, the agent has to reason about volume mounts, Docker networking, etc — potentially more confusion, higher likelihood that agents get something wrong. A single environment where cloudrouter start ./my-project just works is what I envisioned.
Re: SSH host keys. SSH never connects to a real host. It's tunneled through TLS WebSocket via ProxyCommand. Also the hostname is fake, we have per-session auth token on the WebSocket layer, and VMs are ephemeral with fresh keys every boot. So, SSH isn't wide-open. We don't expose the SSH port (port 10000); everything goes through our authenticated proxy.
What stops just mentioning AWS/Azure/GCP CLI tools to agents?
Totally fair point. For me it was just a nice primitive to have -- just one command gives the agent a VM with SSH, file sync, browser, GPU ready to go. Instead of dealing with cloud account setup, security groups, SSH keys, and other shenanigans. For cloudrouter, the dependencies/Docker/VNC/Juypter Lab come pre-baked so you don't need to think about configuring VM environment setups...
You can but I have to say that there's value in a tool that lets the ai do it using less tokens.
Nothing
Nice, we build something similar at dstack
We recently also added support for agents: https://skills.sh/dstackai/dstack/dstack
Our approach though is more tide-case agnostic and in the direction of brining full-fledged container orchestration converting from development to training and inference
Great demo!
Freaking wow.
Awesome demo!!!
Thanks for such an enjoyable read!