I just spun up Lemmy on my Kubernetes cluster with nginx-unprivileged and ingress-nginx. All is well so far! I’m thinking about posting the Kustomization manifests and continuing to maintain and publish OCI’s per version release of Lemmy.

  • gabe565@lemmy.cook.ggEnglish
    2·
    1 year ago

    Yep I’m still working on a helm chart. Currently, each service is deployed with the bjw-s app-template helm chart, but I’d like to combine it all into a single chart.

    The hardest part was getting ingress-nginx to pass ActivityPub requests to the backend, but we settled on a hack that seems to work well. We had to add the following configuration snippet to the frontend’s ingress annotations:

    nginx.ingress.kubernetes.io/configuration-snippet: |
  if ($http_accept = "application/activity+json") {
    set $proxy_upstream_name "lemmy-lemmy-8536";
  }
  if ($http_accept = "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\"") {
    set $proxy_upstream_name "lemmy-lemmy-8536";
  }
  if ($request_method = POST) {
    set $proxy_upstream_name "lemmy-lemmy-8536";
  }

    The value of the variable is $NAMESPACE-$SERVICE-$PORT.
    I tested this pretty thoroughly and haven’t been able to break it so far, but please let me know if anybody has a better solution!

    • anthr76@lemmy.kutara.ioOPEnglish
      2·
      1 year ago

      Firstly, awesome to hear you’re using bjw-s app-template helm chart. He’s my good friend and former coworker :)

      I’m also doing what @seang96@exploding-heads.com is doing.

      While I don’t consider this completed yet I have posted how I’m doing things so far here

      • gabe565@lemmy.cook.ggEnglish
        3·
        1 year ago

        That’s awesome! I love his Helm chart. It’s the most impressive Helm library I’ve ever seen. I maintain a bunch of charts and I exclusively use his library chart :)

        I just mentioned in a response to @seang96@exploding-heads.com, but I feel like deploying a separate nginx is probably cleaner, I just didn’t want another SPOF that I could break at some point in the future.

    • seang96@exploding-heads.comEnglish
      1·
      1 year ago

      Doing this made / path with Lemmy ui break, but posts and comments were actually updating according to logs. i set it up with a nginx container behind my ingress now and it appears everything works besides my comments and posts not being federated even though I can curl the links for troubleshooting federation without issue. Got any ideas?

      • gabe565@lemmy.cook.ggEnglish
        2·
        1 year ago

        Hmm I’m not sure! That code snippet should only affect routing conditionally. When you added the configuration snippet, did your ingress logs show the requests to / going to the frontend or backend?

        An nginx container behind ingress seems cleaner, I just didn’t want to add another point that I could possibly break lol

        • seang96@exploding-heads.comEnglish
          2·
          1 year ago

          I just found out my posts are finally going through without any changes using the nginx proxy container to nginx ingress method! If you do have a way to do it all directly through nginx I’d love to see how it’s all done, maybe I was missing something outside of the snippet you posted.

          Since it’s currently working I’ll look into spinning a test instance up when I get a chance and play with that ingress annotation.

          • gabe565@lemmy.cook.ggEnglish
            2·
            1 year ago

            Awesome! A separate nginx container is fine, so if it’s working I’d probably leave it. I’ll look through and see if there’s anything I missed in my comment though for brevity.