• 1 Post
  • 17 Comments
Joined 2 years ago
cake
Cake day: June 11th, 2023

help-circle
  • This is a really fantastic explanation of the issue!

    It’s more like improv comedy with an extremely adaptable comic than a conversation with a real person.

    One of the things that I’ve noticed is that the training/finetuning that’s done in order to make it give good completions to the “helpful ai conversation scenario” is that it flattens a lot of the capabilities of the underlying language model for really interesting and specific completions. I remember playing around with gpt2 in it’s native text completion mode, and even with that much weaker model, it was able to complete a much larger variety of text styles without sliding into the sameness and slickness of the current chat model fine-tuning.

    A lot of the research that I read on LLMs is using them in the original token completion context, but pretty much the only way people interact with them is through a thick layer of ai chatbot improv. As an example for code, I imagine that one would have more success using an LLM to edit your code if the context that you give it starts out written like it is a review of a pull request for the code, or some other commentary of a form that matches the way that code is reviewed in the training data. But instead of having access to create that context directly, we have to ask for code review through the fogged window of a chat between an AI assistant and a person discussing code. And that form of chat likely isn’t well represented in the training data.





  • I don’t know a lot about tailscale, but I think that’s likely not relevant to what’s possible (but maybe relevant to how to accomplish it).

    It sounds like the main issue here is dns. If you wanted to/were okay with just IP based connections, then you could assign each service to a different port on Bob’s box, and then have nginx point those ports at the relevant services. This should be very easy to do with a raw nginx config. I could write one for you if you wanted. It’s pretty easy if you’re not dealing with https/certificates (in which case this method won’t work anyway).

    Looking quickly on google for npm (which I’ve never used), this might require adding the ports to the docker config and then using that port in npn (Like here). This is likely the simplest solution.

    If you want hostnames/https, then you need some sort of DNS. This is a bit harder. You can take over their router like you suggested. You could use public DNS that points at a private IP (this is the only way I’m suggesting to get public trusted ssl certificates).

    You might be able to use mdns to get local DNS at Bob’s house automatically, which would be very clean. You’d basically register like jellyseer.local and jellyfin.local on Bob’s network from the box and then setup the proxy manager to proxy based on those domains. You might be able to just do avahi-publish -a -R jellyseer.local 192.168.box.ip and then avahi-publish -a -R jellyfin.local 192.168.box.ip. And then any client that supports mdns/avahi will be able to find the service at that host. You can then register those names nginx/npn and I think things should just work

    To answer your questions directly

    1. Yes I think just a box can work
    2. I think locations are going to be a nightmare. don’t use them if you can avoid it. Also plain nginx really isn’t so bad, and you can learn pretty quickly. this is probably fine in npn though
    3. I don’t think you need to dive into iptables or anything like that. Iptables would provide a lower level proxying (level 3 instead of like level 7), which could be useful if you’re running non http services, but isn’t necessary for 99% of web stuff.
    4. I think part of the problem might be working with higher level systems like npn, but a lot of it is just that networking involves so many layers and there are multiple solutions to any problem, all of which require knowing somewhat what’s happening under the surface to understand why they’re failing

    I’d be happy to try and give more specifics if you choose a path similar to one of the above things.