DSCI is simple yet super flexible pipeline engine to write CI code on regular programming languages, integrates with Forgejo using web hooks. Intended for small teams hosting Forgejo on single VM VPS and willing to create pipelines on regular programming languages

  • thesmokingman@programming.dev
    link
    fedilink
    arrow-up
    1
    ·
    13 hours ago

    Looking at the examples, you’ve just made a brand new GitHub Actions framework. There’s YAML to wrap everything together and a bunch of Python that’s so declarative it might as well be HCL. Do you have an example that’s a bit more than “do what YAML does only in bash?”

    • melezhik@programming.devOP
      link
      fedilink
      arrow-up
      1
      ·
      edit-2
      5 hours ago

      Ok, try to do it on GH actions, share states between tasks/jobs for example:

      tasks/task_one/task.py

      #!/usr/bin/python3
      
      update_state({
        'out1' : 'out1 value',
        'out2' : 'out2 value'
      })
      

      tasks/task_two/task.py

      #!/usr/bin/python3
      
      dict = get_state()
      print(dict["out1"])
      print(dict["out2"])
      

      Or share states between jobs:

      jobs/job1/task.py

      #!/usr/bin/python3
      
      update_state({
        'out1' : 'out1 value',
        'out2' : 'out2 value'
      })
      

      jobs/job2/task.py

      #!/usr/bin/python3
      
      dict = config()
      
      print(dict["_dsci_"]["job1"]["out1"])
      print(dict["_dsci_"]["job1"]["out2"])
      
      

      I can’t imagine how much boilerplate code (if this ever possible ) one needs to write to achieve that on YAML based pipelines (GH Actions/ etc)

      And don’t tell me about jobs artifacts ))

      UPDATE:

      Another good example is to run tasks conditionally, yes using old good if:

      #!/usr/bin/python3
      
      if some_condition(foo, bar): 
          run_task(
             'task1', {
                'foo' : 'foo value',
                'bar' : 'bar value'
             }
          )
      

      The same could be quite awkward in YAML based code

      • thesmokingman@programming.dev
        link
        fedilink
        arrow-up
        1
        ·
        12 hours ago

        This is a boilerplate example. I asked for something more than boilerplate. Give me some reasons why I need an incredibly stateful CI engine.

        • melezhik@programming.devOP
          link
          fedilink
          arrow-up
          2
          ·
          edit-2
          12 hours ago

          it’s just because I think in real world we have a lot of tasks where state is required or extremely beneficial, some examples on top of my head:

          • creation of virtual machines with dynamic IP addresses
          • creation of bug tracking system tickets with unique ticket IDs
          • looking up in databases where fetched records have unique IDs

          etc

          • thesmokingman@programming.dev
            link
            fedilink
            arrow-up
            2
            ·
            10 hours ago

            I wouldn’t do any of that in a CI-only system. If I did, I’d use tools that exist for those jobs that already allow scripting languages.

            1. Why wouldn’t you use IaC tools for this? All the majors have Python already.
            2. What build or deploy task needs to both create and reference a bug ticket?
            3. What build or deploy task needs to do database lookups? Or possibly what task needs those across multiple, independent stages?
            • melezhik@programming.devOP
              link
              fedilink
              arrow-up
              1
              ·
              edit-2
              10 hours ago

              If you use tools gluing them into YAML - you get YAML bloated with time . When you say those tools already having Python - excellent I would like to use those Python libs or SDK directly in my Python code instead of juggling those tools as cli or code blocks inside YAML

              UPDATE: and yeah , re-read again - I guess the most of automation is done today via “CI” pipelines even when those are not meant to be CI only, like you said … anyways the rest I have said stands true for me … don’t bake your code into YAML )