GitHub Community
-
|
Is it possible to run given steps in parallel? For example, we have a number of dependencies to build before we can build our component, and then test our component. It would be nice if the dependency build steps could first run in parallel, followed by our component build step, and the test step. |
Beta Was this translation helpful? Give feedback.
Replies: 10 comments 10 replies
-
|
Not at this moment. Only jobs can run in parallel, but steps always run sequentially. I don’t know if there are any plans to ever support this. |
Beta Was this translation helpful? Give feedback.
-
|
But jobs do not share workdirs. It would be great if we could simply branch off a job (or a step), and then execute another one starting from there. |
Beta Was this translation helpful? Give feedback.
-
|
ps. visual programming would be a great approach here, in my opinion. |
Beta Was this translation helpful? Give feedback.
-
|
@community is there any update on this? |
Beta Was this translation helpful? Give feedback.
-
|
There's a feature request for it. See @zsolt-dev's comment |
Beta Was this translation helpful? Give feedback.
-
|
Hi, is this still the case that the steps within the job cannot run in parallel. If that is the case, I wonder if there is any related documentation on best structuring the jobs to achieve the same goal (basically, turn the tasks into jobs) |
Beta Was this translation helpful? Give feedback.
-
|
If your steps are shell commands can you run them as a background task using the ‘&’ at the end of the command line. You won’t be told when they finish. I also don’t know if the workflow run will know to wait for the job to finish before it kills the instance. If you REALLY need parallelism, remember that an action allows you to write code. That code could trigger tasks in multiple threads. Seems like a lot of work, but the ability should be there. |
Beta Was this translation helpful? Give feedback.
-
|
The |
Beta Was this translation helpful? Give feedback.
-
|
As @amezick mentioned it is indeed possible via background tasks. And you can wait for them! At our CI, we do something like this:
|
Beta Was this translation helpful? Give feedback.
-
|
Or you can use GNU Parallel! |
Beta Was this translation helpful? Give feedback.
-
|
I am looking forward seeing steps to be runnable in parallel. so that the setup steps can run once (installing dependencies, setting up database etc) but then everything else can be run in parallel |
Beta Was this translation helpful? Give feedback.
-
|
I'm confused why this is still not supported, a year later. This seems like an obvious use case, but all we have to support it is |
Beta Was this translation helpful? Give feedback.
-
|
Same here, I just stumbled upon this feed, and I have a backend & frontend app running in Docker and since the frontend needs the backend to execute e2e tests, I want to install all dependencies once and then run backend & frontend tests in parallel (since they use different databases) |
Beta Was this translation helpful? Give feedback.
-
|
Same here, seems like a missing feature competitors have that would be high on the catch up list to me |
Beta Was this translation helpful? Give feedback.
-
|
I'm having the same issue with actions/upload-artifact not being parallel |
Beta Was this translation helpful? Give feedback.
-
|
I just create a multitask rake task: |
Beta Was this translation helpful? Give feedback.
-
|
can show the detail of it, thanks |
Beta Was this translation helpful? Give feedback.
-
require_relative "config/application"
Rails.application.load_tasks
if Rails.env.development? || Rails.env.test?
namespace :brakeman do
desc "Run Brakeman"
task :run do # rubocop:disable Rails/RakeEnvironment
sh("bin/brakeman --rails6 -A --no-summary --ensure-latest --no-progress")
end
end
namespace :bundle do
desc "Audit Bundler"
task :audit do # rubocop:disable Rails/RakeEnvironment
sh("bin/bundle-audit update")
sh("bin/bundle-audit check")
end
end
desc "Run ERBLint"
task :erblint do # rubocop:disable Rails/RakeEnvironment
sh("bin/erblint --lint-all")
end
desc "Fix ERBLint"
task "erblint:fix" do # rubocop:disable Rails/RakeEnvironment
sh("bin/erblint --lint-all -a")
end
desc "Run ESLint"
task :eslint do # rubocop:disable Rails/RakeEnvironment
sh("yarn lint:js")
end
desc "Fix ESLint problems"
task "eslint:fix" do # rubocop:disable Rails/RakeEnvironment
sh("yarn lint:js --fix")
end
desc "Run ESLint"
task :stylelint do # rubocop:disable Rails/RakeEnvironment
sh("yarn lint:css")
end
desc "Fix ESLint problems"
task "stylelint:fix" do # rubocop:disable Rails/RakeEnvironment
sh("yarn lint:css --fix")
end
require "rubocop/rake_task"
RuboCop::RakeTask.new(:rubocop) do |task|
# task.patterns = changed_files(5)
task.options = %w[-DEP --format fuubar]
end
RuboCop::RakeTask.new("rubocop:fix") do |task|
# task.patterns = changed_files(5)
task.options = %w[-DEa --format fuubar]
end
RuboCop::RakeTask.new("rubocop:fixall") do |task|
# task.patterns = changed_files(5)
task.options = %w[-DEA --format fuubar]
end
# require "bundler-audit/bundler/audit/task"
multitask lint: %i[erblint eslint stylelint rubocop]
multitask "lint:fix" => %w[
erblint:fix
eslint:fix
rubocop:fix
stylelint:fix
]
multitask "lint:fixall" => %w[
erblint:fix
eslint:fix
rubocop:fixall
stylelint:fix
]
multitask ci: %i[
brakeman:run
bundle:audit
erblint
eslint
rubocop
spec
stylelint
]
end |
Beta Was this translation helpful? Give feedback.
-
|
yes to parallel steps! even as simple as true/false parallel steps would be huge time saving! |
Beta Was this translation helpful? Give feedback.
-
|
Please vote for supporting parallel steps here: #14484 |
Beta Was this translation helpful? Give feedback.
Not at this moment. Only jobs can run in parallel, but steps always run sequentially.
I don’t know if there are any plans to ever support this.