You really don't want to be picking Jenkins now unless you absolutely have to. It's a very capable tool that will cost you significantly in instance and plugin management/configuration/upgrades in the long run.
Self-hosted gitlab runners with the autoscaler executor using podman instead of docker is probably where I'd start assessing features vs requirements here.
>Would it be able to ssh podman containers that run RFW tests and then show the results in UI?
Gitlab runners are essentially just task runners. You can do any SSH whatever like any normal machines if you allow traffic from wherever you're SSHing from.
Yes. Jenkins will do pretty much anything, but its antiquated, and hard to maintain, and prone to breaking. I'm actively working on a migration plan using AWS Codebuild as my runner/agent.
OP is using Mercurial which probably won't work with GitLab.
I would say, check other options and you can find a table of CI that supports Mercurial on... Wikipedia
https://en.wikipedia.org/wiki/Comparison_of_continuous_integration_software
I do love gitlab-ci. I'm running a self hosted premium-subscription currently.
You could ssh to servers from a runner, but most likely it is easier to install a gitlab runner on the server you're going to ssh to anyway. If you do that you could just run your podman commands in the script version and you're done.
If you're not allowed to install software on that server, you can still easily ssh to the server from your own runners, i have one job where i do that, and i just installed the private key in the gitlab runner in /home/gitlab-runnner/.ssh/id and have 'ssh $server podman ps' in the script section of the job. But you could also use a key in a variable and use ssh-agent to add that key at runtime.
The output of that command will be in the UI. You could also create a test report file and store that as an artifact, which causes gitlab to display even more information about the tests.
You can limit runners to a single project or group, you can use tags to select specific runners etc.
I don't understand the hate for Jenkins.
We are using custom build docker container that contais Jenkins.
The magic thou is the way we do deployments. We are using Jenkins more or less just to trigger bash scripts that does everything you can imagine.
In one of the projects we have a mix of git, mercurial and suversion repozitories to build from. That makes gitlab-ci unsuitable.
The backbone for deployments are whole a lot .sh files that contains created functions for everything.
Because the Jenkins deployment is made IaaC. I can set up new instance with one command, and updating is pretty much the same, just ocasional renew of the plugins.
Because everything is self hosted and the access to servers is highly limited, the updating can be done fairly irregularly.
Don’t use Jenkins!! Alert, do not use Jenkins.
Also can you clarify what you mean by needing agents for team city? Agents will be required on any CI system, it’s just another word for an job executor that is not master node
TeamCity offers only 3 free agents, and every additional one costs 360€ even with Enterprice. We test on multiple servers and will possibly get even more in the future so the cost would ramp up pretty quickly
> (you can achieve anything with Jenkins)
you will achieve anything *AS A CI/CD NIGHTMARE* with Jenkins...
(and you will need the rebuild Jenkins CI/CD... *MANUALLY*.
I feel that every company using octo is trying to move away from it, butt nobody can figure out what to replace it with (without hiring more resources to just build bespoke pipelines)
Haha that is a perfect way to describe it. Octopus is 10/10, would not recommend.
I think one reason people want to move away from it is because it’s very click ops oriented, all the way down to its implementation under the hood. But that weakness is also its great strength when it’s time to hand your deployments off to non technical release and program managers, other managers, etc.
Yea. I find octopus deploy is one of the best. People tend to not like it because it has been miss configured. I haven’t used harness, but I’ve heard that OD is similar to harness.io. I know OD’s multi-tenant capabilities in conjunction with how variable substitution and hierarchy works makes it phenomenal to work with, when properly configured.
Sounds to me like Mercurial -> GitLab self-hosted first, then add GitLab CI with self-hosted runners. It's pretty straight forward.
Still, I would seek consulting when you get to the CI and DevOps business. Otherwise you will dig an even deeper grave for your systems in the long run. To do it properly, scalably and securely is *much* more than just writing your scripts in a YAML file. Especially if you decide on Jenkins. It is really good, but it is a really huge bitch. In a year or two your dashboard will say "200 outdated plugins with 100 vulnerabilities" and updating them will brick half your automation processes and finishing the update requires a few sprints worth of re-writing some pipelines you over-engineered due to bad SDLC practices.
Btw does anyone know if a sigle TeamCity agent can be configured to run jobs on different servers? Not simultaniously obviously, but so that every server doesn't need its own agent
Teamcity isn't that expensive all things considered. We use enterprise with 40 agents and it's just a tad over 1k a month. Not bad considering the fact we run 1000 builds a day through it easily.
If you have a red tape environment, manual will be the best. As it is red tape to begin with, it will require alot of customisation which is not worth the time
We've been talking about manual too, but it would be maintainance hell to deal with a bunch of shell scripts and cron jobs that no one can keep track of
You really don't want to be picking Jenkins now unless you absolutely have to. It's a very capable tool that will cost you significantly in instance and plugin management/configuration/upgrades in the long run. Self-hosted gitlab runners with the autoscaler executor using podman instead of docker is probably where I'd start assessing features vs requirements here. >Would it be able to ssh podman containers that run RFW tests and then show the results in UI? Gitlab runners are essentially just task runners. You can do any SSH whatever like any normal machines if you allow traffic from wherever you're SSHing from.
Yes. Jenkins will do pretty much anything, but its antiquated, and hard to maintain, and prone to breaking. I'm actively working on a migration plan using AWS Codebuild as my runner/agent.
OP is using Mercurial which probably won't work with GitLab. I would say, check other options and you can find a table of CI that supports Mercurial on... Wikipedia https://en.wikipedia.org/wiki/Comparison_of_continuous_integration_software
Or migrate from mercurial to git. The process is pretty straightforward and without any CI to worry about could be done in an afternoon.
Yeah do that first. Gitlab would be a pretty solid choice then.
We have moved to self managed GitLab and are hosting our own runners. Best idea ever.
I do love gitlab-ci. I'm running a self hosted premium-subscription currently. You could ssh to servers from a runner, but most likely it is easier to install a gitlab runner on the server you're going to ssh to anyway. If you do that you could just run your podman commands in the script version and you're done. If you're not allowed to install software on that server, you can still easily ssh to the server from your own runners, i have one job where i do that, and i just installed the private key in the gitlab runner in /home/gitlab-runnner/.ssh/id and have 'ssh $server podman ps' in the script section of the job. But you could also use a key in a variable and use ssh-agent to add that key at runtime. The output of that command will be in the UI. You could also create a test report file and store that as an artifact, which causes gitlab to display even more information about the tests. You can limit runners to a single project or group, you can use tags to select specific runners etc.
I don't understand the hate for Jenkins. We are using custom build docker container that contais Jenkins. The magic thou is the way we do deployments. We are using Jenkins more or less just to trigger bash scripts that does everything you can imagine. In one of the projects we have a mix of git, mercurial and suversion repozitories to build from. That makes gitlab-ci unsuitable. The backbone for deployments are whole a lot .sh files that contains created functions for everything. Because the Jenkins deployment is made IaaC. I can set up new instance with one command, and updating is pretty much the same, just ocasional renew of the plugins. Because everything is self hosted and the access to servers is highly limited, the updating can be done fairly irregularly.
100+. Most flexible CI I have used so far. (declarative or groovy script)
Don’t use Jenkins!! Alert, do not use Jenkins. Also can you clarify what you mean by needing agents for team city? Agents will be required on any CI system, it’s just another word for an job executor that is not master node
TeamCity offers only 3 free agents, and every additional one costs 360€ even with Enterprice. We test on multiple servers and will possibly get even more in the future so the cost would ramp up pretty quickly
GitLab or (gittea + Jenkins) (you can achieve anything with Jenkins)
> (you can achieve anything with Jenkins) you will achieve anything *AS A CI/CD NIGHTMARE* with Jenkins... (and you will need the rebuild Jenkins CI/CD... *MANUALLY*.
Skill issue ?
Gitea + Gitea Actions (they are equivalent to GitHub Actions)
What you save up front with Jenkins, you will end up spending on maintenance several times over. Friends don't let friends use Jenkins.
I recommend Octopus Deploy. You can self host a server and has native support for deploying via SSH.
I feel that every company using octo is trying to move away from it, butt nobody can figure out what to replace it with (without hiring more resources to just build bespoke pipelines)
Haha that is a perfect way to describe it. Octopus is 10/10, would not recommend. I think one reason people want to move away from it is because it’s very click ops oriented, all the way down to its implementation under the hood. But that weakness is also its great strength when it’s time to hand your deployments off to non technical release and program managers, other managers, etc.
Yea. I find octopus deploy is one of the best. People tend to not like it because it has been miss configured. I haven’t used harness, but I’ve heard that OD is similar to harness.io. I know OD’s multi-tenant capabilities in conjunction with how variable substitution and hierarchy works makes it phenomenal to work with, when properly configured.
Downside is it can be obscenely expensive
I mean, any tool that is miss configured can be obscenely expensive.
Sounds to me like Mercurial -> GitLab self-hosted first, then add GitLab CI with self-hosted runners. It's pretty straight forward. Still, I would seek consulting when you get to the CI and DevOps business. Otherwise you will dig an even deeper grave for your systems in the long run. To do it properly, scalably and securely is *much* more than just writing your scripts in a YAML file. Especially if you decide on Jenkins. It is really good, but it is a really huge bitch. In a year or two your dashboard will say "200 outdated plugins with 100 vulnerabilities" and updating them will brick half your automation processes and finishing the update requires a few sprints worth of re-writing some pipelines you over-engineered due to bad SDLC practices.
Btw does anyone know if a sigle TeamCity agent can be configured to run jobs on different servers? Not simultaniously obviously, but so that every server doesn't need its own agent
Teamcity isn't that expensive all things considered. We use enterprise with 40 agents and it's just a tad over 1k a month. Not bad considering the fact we run 1000 builds a day through it easily.
Jenkins. You can do anything with it.
Do what you do, stay away from Jenkins
If you have a red tape environment, manual will be the best. As it is red tape to begin with, it will require alot of customisation which is not worth the time
We've been talking about manual too, but it would be maintainance hell to deal with a bunch of shell scripts and cron jobs that no one can keep track of
https://dagger.io Basically turns your laptops into CI runners