DSCI is simple yet super flexible pipeline engine to write CI code on regular programming languages, integrates with Forgejo using web hooks. Intended for small teams hosting Forgejo on single VM VPS and willing to create pipelines on regular programming languages

  • Sp1983@programming.devOP
    link
    fedilink
    arrow-up
    1
    ·
    edit-2
    22 hours ago

    Ok, try to do it on GH actions, share states between tasks/jobs for example:

    tasks/task_one/task.py

    #!/usr/bin/python3
    
    update_state({
      'out1' : 'out1 value',
      'out2' : 'out2 value'
    })
    

    tasks/task_two/task.py

    #!/usr/bin/python3
    
    dict = get_state()
    print(dict["out1"])
    print(dict["out2"])
    

    Or share states between jobs:

    jobs/job1/task.py

    #!/usr/bin/python3
    
    update_state({
      'out1' : 'out1 value',
      'out2' : 'out2 value'
    })
    

    jobs/job2/task.py

    #!/usr/bin/python3
    
    dict = config()
    
    print(dict["_dsci_"]["job1"]["out1"])
    print(dict["_dsci_"]["job1"]["out2"])
    
    

    I can’t imagine how much boilerplate code (if this ever possible ) one needs to write to achieve that on YAML based pipelines (GH Actions/ etc)

    And don’t tell me about jobs artifacts ))

    UPDATE:

    Another good example is to run tasks conditionally, yes using old good if:

    #!/usr/bin/python3
    
    if some_condition(foo, bar): 
        run_task(
           'task1', {
              'foo' : 'foo value',
              'bar' : 'bar value'
           }
        )
    

    The same could be quite awkward in YAML based code

    • thesmokingman@programming.dev
      link
      fedilink
      arrow-up
      1
      ·
      1 day ago

      This is a boilerplate example. I asked for something more than boilerplate. Give me some reasons why I need an incredibly stateful CI engine.

      • Sp1983@programming.devOP
        link
        fedilink
        arrow-up
        2
        ·
        edit-2
        1 day ago

        it’s just because I think in real world we have a lot of tasks where state is required or extremely beneficial, some examples on top of my head:

        • creation of virtual machines with dynamic IP addresses
        • creation of bug tracking system tickets with unique ticket IDs
        • looking up in databases where fetched records have unique IDs

        etc

        • thesmokingman@programming.dev
          link
          fedilink
          arrow-up
          2
          ·
          1 day ago

          I wouldn’t do any of that in a CI-only system. If I did, I’d use tools that exist for those jobs that already allow scripting languages.

          1. Why wouldn’t you use IaC tools for this? All the majors have Python already.
          2. What build or deploy task needs to both create and reference a bug ticket?
          3. What build or deploy task needs to do database lookups? Or possibly what task needs those across multiple, independent stages?
          • Sp1983@programming.devOP
            link
            fedilink
            arrow-up
            1
            ·
            edit-2
            1 day ago

            If you use tools gluing them into YAML - you get YAML bloated with time . When you say those tools already having Python - excellent I would like to use those Python libs or SDK directly in my Python code instead of juggling those tools as cli or code blocks inside YAML

            UPDATE: and yeah , re-read again - I guess the most of automation is done today via “CI” pipelines even when those are not meant to be CI only, like you said … anyways the rest I have said stands true for me … don’t bake your code into YAML )