taskgraph package¶
Subpackages¶
- taskgraph.actions package
- Submodules
- taskgraph.actions.add_new_jobs module
- taskgraph.actions.add_talos module
- taskgraph.actions.backfill module
- taskgraph.actions.cancel_all module
- taskgraph.actions.mochitest_retrigger module
- taskgraph.actions.purge_caches module
- taskgraph.actions.registry module
- taskgraph.actions.release_promotion module
- taskgraph.actions.retrigger module
- taskgraph.actions.run_missing_tests module
- taskgraph.actions.util module
- Module contents
- taskgraph.cron package
- taskgraph.loader package
- taskgraph.test package
- Submodules
- taskgraph.test.test_create module
- taskgraph.test.test_cron_util module
- taskgraph.test.test_decision module
- taskgraph.test.test_files_changed module
- taskgraph.test.test_generator module
- taskgraph.test.test_graph module
- taskgraph.test.test_morph module
- taskgraph.test.test_optimize module
- taskgraph.test.test_parameters module
- taskgraph.test.test_target_tasks module
- taskgraph.test.test_taskgraph module
- taskgraph.test.test_transforms_base module
- taskgraph.test.test_try_option_syntax module
- taskgraph.test.test_util_attributes module
- taskgraph.test.test_util_docker module
- taskgraph.test.test_util_parameterization module
- taskgraph.test.test_util_python_path module
- taskgraph.test.test_util_runnable_jobs module
- taskgraph.test.test_util_schema module
- taskgraph.test.test_util_templates module
- taskgraph.test.test_util_time module
- taskgraph.test.test_util_treeherder module
- taskgraph.test.test_util_yaml module
- Module contents
- taskgraph.transforms package
- Subpackages
- taskgraph.transforms.job package
- Submodules
- taskgraph.transforms.job.buildbot module
- taskgraph.transforms.job.common module
- taskgraph.transforms.job.debian_package module
- taskgraph.transforms.job.hazard module
- taskgraph.transforms.job.mach module
- taskgraph.transforms.job.mozharness module
- taskgraph.transforms.job.mozharness_test module
- taskgraph.transforms.job.run_task module
- taskgraph.transforms.job.spidermonkey module
- taskgraph.transforms.job.toolchain module
- Module contents
- taskgraph.transforms.job package
- Submodules
- taskgraph.transforms.balrog_submit module
- taskgraph.transforms.base module
- taskgraph.transforms.beetmover module
- taskgraph.transforms.beetmover_cdns module
- taskgraph.transforms.beetmover_checksums module
- taskgraph.transforms.beetmover_l10n module
- taskgraph.transforms.beetmover_repackage module
- taskgraph.transforms.beetmover_repackage_l10n module
- taskgraph.transforms.beetmover_source module
- taskgraph.transforms.bouncer_aliases module
- taskgraph.transforms.bouncer_check module
- taskgraph.transforms.bouncer_submission module
- taskgraph.transforms.build module
- taskgraph.transforms.build_attrs module
- taskgraph.transforms.build_lints module
- taskgraph.transforms.build_signing module
- taskgraph.transforms.checksums_signing module
- taskgraph.transforms.coalesce module
- taskgraph.transforms.diffoscope module
- taskgraph.transforms.docker_image module
- taskgraph.transforms.final_verify module
- taskgraph.transforms.google_play_strings module
- taskgraph.transforms.l10n module
- taskgraph.transforms.marionette_harness module
- taskgraph.transforms.name_sanity module
- taskgraph.transforms.nightly_l10n_signing module
- taskgraph.transforms.partials module
- taskgraph.transforms.partials_signing module
- taskgraph.transforms.partner_repack module
- taskgraph.transforms.push_apk module
- taskgraph.transforms.release_balrog_publishing module
- taskgraph.transforms.release_deps module
- taskgraph.transforms.release_generate_checksums module
- taskgraph.transforms.release_generate_checksums_beetmover module
- taskgraph.transforms.release_generate_checksums_signing module
- taskgraph.transforms.release_mark_as_shipped module
- taskgraph.transforms.release_notifications module
- taskgraph.transforms.release_snap module
- taskgraph.transforms.release_version_bump module
- taskgraph.transforms.repackage module
- taskgraph.transforms.repackage_l10n module
- taskgraph.transforms.repackage_routes module
- taskgraph.transforms.repackage_signing module
- taskgraph.transforms.reverse_chunk_deps module
- taskgraph.transforms.signing module
- taskgraph.transforms.source_test module
- taskgraph.transforms.spidermonkey module
- taskgraph.transforms.task module
- taskgraph.transforms.tests module
- taskgraph.transforms.try_job module
- taskgraph.transforms.update_verify module
- taskgraph.transforms.update_verify_config module
- taskgraph.transforms.upload_generated_sources module
- taskgraph.transforms.upload_symbols module
- taskgraph.transforms.use_toolchains module
- taskgraph.transforms.worker_type module
- Module contents
- Subpackages
- taskgraph.util package
- Submodules
- taskgraph.util.attributes module
- taskgraph.util.bbb_validation module
- taskgraph.util.cached_tasks module
- taskgraph.util.docker module
- taskgraph.util.hash module
- taskgraph.util.parameterization module
- taskgraph.util.partials module
- taskgraph.util.platforms module
- taskgraph.util.python_path module
- taskgraph.util.schema module
- taskgraph.util.scriptworker module
- taskgraph.util.seta module
- taskgraph.util.signed_artifacts module
- taskgraph.util.taskcluster module
- taskgraph.util.templates module
- taskgraph.util.time module
- taskgraph.util.treeherder module
- taskgraph.util.verify module
- taskgraph.util.workertypes module
- taskgraph.util.yaml module
- Module contents
Submodules¶
taskgraph.create module¶
-
taskgraph.create.
create_task
(session, task_id, label, task_def)¶
-
taskgraph.create.
create_tasks
(taskgraph, label_to_taskid, params, decision_task_id=None)¶
taskgraph.decision module¶
-
taskgraph.decision.
full_task_graph_to_runnable_jobs
(full_task_json)¶
-
taskgraph.decision.
get_decision_parameters
(options)¶ Load parameters from the command-line options for ‘taskgraph decision’. This also applies per-project parameters, based on the given project.
-
taskgraph.decision.
taskgraph_decision
(options, parameters=None)¶ Run the decision task. This function implements mach taskgraph decision, and is responsible for
- processing decision task command-line options into parameters
- running task-graph generation exactly the same way the other mach taskgraph commands do
- generating a set of artifacts to memorialize the graph
- calling TaskCluster APIs to create the graph
-
taskgraph.decision.
write_artifact
(filename, data)¶
taskgraph.docker module¶
-
taskgraph.docker.
build_context
(name, outputFile, args=None)¶ Build a context.tar for image with specified name.
-
taskgraph.docker.
build_image
(name, tag, args=None)¶ Build a Docker image of specified name.
Output from image building process will be printed to stdout.
-
taskgraph.docker.
load_image
(url, imageName=None, imageTag=None)¶ Load docker image from URL as imageName:tag, if no imageName or tag is given it will use whatever is inside the zstd compressed tarball.
Returns an object with properties ‘image’, ‘tag’ and ‘layer’.
-
taskgraph.docker.
load_image_by_name
(image_name, tag=None)¶
-
taskgraph.docker.
load_image_by_task_id
(task_id, tag=None)¶
taskgraph.files_changed module¶
Support for optimizing tasks based on the set of files that have changed.
-
taskgraph.files_changed.
check
(params, file_patterns)¶ Determine whether any of the files changed in the indicated push to https://hg.mozilla.org match any of the given file patterns.
-
taskgraph.files_changed.
get_changed_files
(repository, revision)¶ Get the set of files changed in the push headed by the given revision. Responses are cached, so multiple calls with the same arguments are OK.
taskgraph.filter_tasks module¶
-
taskgraph.filter_tasks.
filter_servo
(graph, parameters, graph_config)¶ Filter out tasks for Servo vendoring changesets.
If the change triggering is related to Servo vendoring, impact is minimal because not all platforms use Servo code.
We filter out tests on platforms that don’t run Servo tests because running tests will accomplish little for these changes.
-
taskgraph.filter_tasks.
filter_target_tasks
(graph, parameters, graph_config)¶ Proxy filter to use legacy target tasks code.
This should go away once target_tasks are converted to filters.
-
taskgraph.filter_tasks.
filter_task
(name)¶ Generator to declare a task filter function.
taskgraph.generator module¶
-
class
taskgraph.generator.
Kind
(name, path, config, graph_config)¶ Bases:
object
-
classmethod
load
(root_dir, graph_config, kind_name)¶
-
load_tasks
(parameters, loaded_tasks)¶
-
classmethod
-
exception
taskgraph.generator.
KindNotFound
¶ Bases:
exceptions.Exception
Raised when trying to load kind from a directory without a kind.yml.
-
class
taskgraph.generator.
TaskGraphGenerator
(root_dir, parameters)¶ Bases:
object
The central controller for taskgraph. This handles all phases of graph generation. The task is generated from all of the kinds defined in subdirectories of the generator’s root directory.
Access to the results of this generation, as well as intermediate values at various phases of generation, is available via properties. This encourages the provision of all generation inputs at instance construction time.
-
full_task_graph
¶ The full task graph – the full task set, with edges representing dependencies.
@type: TaskGraph
-
full_task_set
¶ The full task set – all tasks defined by any kind (a graph without edges)
@type: TaskGraph
-
label_to_taskid
¶ A dictionary mapping task label to assigned taskId. This property helps in interpreting optimized_task_graph.
@type: dictionary
-
morphed_task_graph
¶ The optimized task graph, with any subsequent morphs applied. This graph will have the same meaning as the optimized task graph, but be in a form more palatable to TaskCluster.
@type: TaskGraph
-
optimized_task_graph
¶ The set of targetted tasks and all of their dependencies; tasks that have been optimized out are either omitted or replaced with a Task instance containing only a task_id.
@type: TaskGraph
-
target_task_graph
¶ The set of targetted tasks and all of their dependencies
@type: TaskGraph
-
target_task_set
¶ The set of targetted tasks (a graph without edges)
@type: TaskGraph
-
verify_attributes
(all_tasks)¶
-
verify_kinds
(kinds)¶
-
verify_parameters
(parameters)¶
-
verify_run_using
()¶
-
-
taskgraph.generator.
load_graph_config
(root_dir)¶
taskgraph.graph module¶
-
class
taskgraph.graph.
Graph
(nodes, edges)¶ Bases:
object
Generic representation of a directed acyclic graph with labeled edges connecting the nodes. Graph operations are implemented in a functional manner, so the data structure is immutable.
It permits at most one edge of a given name between any set of nodes. The graph is not checked for cycles, and methods may hang or otherwise fail if given a cyclic graph.
The nodes and edges attributes may be accessed in a read-only fashion. The nodes attribute is a set of node names, while edges is a set of (left, right, name) tuples representing an edge named name going from node left to node `right..
-
links_dict
()¶ Return a dictionary mapping each node to a set of the nodes it links to (omitting edge names)
-
named_links_dict
()¶ Return a two-level dictionary mapping each node to a dictionary mapping edge names to labels.
-
reverse_links_dict
()¶ Return a dictionary mapping each node to a set of the nodes linking to it (omitting edge names)
-
transitive_closure
(nodes, reverse=False)¶ Return the transitive closure of <nodes>: the graph containing all specified nodes as well as any nodes reachable from them, and any intervening edges.
If reverse is true, the “reachability” will be reversed and this will return the set of nodes that can reach the specified nodes.
Example
- a ——> b ——> c
`——-> d
transitive_closure([b]).nodes == set([a, b]) transitive_closure([c]).nodes == set([c, b, a]) transitive_closure([c], reverse=True).nodes == set([c]) transitive_closure([b], reverse=True).nodes == set([b, c, d])
-
visit_postorder
()¶ Generate a sequence of nodes in postorder, such that every node is visited after any nodes it links to.
Behavior is undefined (read: it will hang) if the graph contains a cycle.
-
visit_preorder
()¶ Like visit_postorder, but in reverse: evrey node is visited before any nodes it links to.
-
taskgraph.morph module¶
Graph morphs are modifications to task-graphs that take place after the optimization phase.
These graph morphs are largely invisible to developers running ./mach locally, so they should be limited to changes that do not modify the meaning of the graph.
-
taskgraph.morph.
add_index_tasks
(taskgraph, label_to_taskid)¶ The TaskCluster queue only allows 10 routes on a task, but we have tasks with many more routes, for purposes of indexing. This graph morph adds “index tasks” that depend on such tasks and do the index insertions directly, avoiding the limits on task.routes.
-
taskgraph.morph.
add_s3_uploader_task
(taskgraph, label_to_taskid)¶ The S3 uploader task is used by mozharness to upload buildbot artifacts.
-
taskgraph.morph.
amend_taskgraph
(taskgraph, label_to_taskid, to_add)¶ Add the given tasks to the taskgraph, returning a new taskgraph
-
class
taskgraph.morph.
apply_jsone_templates
(try_task_config)¶ Bases:
object
Apply a set of JSON-e templates to each task’s task attribute.
Parameters: templates – A dict with the template name as the key, and extra context to use (in addition to task.to_json()) as the value. -
template_dir
= u'/home/zbraniecki/projects/mozilla-unified/taskcluster/taskgraph/templates'¶
-
-
taskgraph.morph.
derive_misc_task
(task, purpose, image, taskgraph, label_to_taskid)¶ Create the shell of a task that depends on task and on the given docker image.
-
taskgraph.morph.
make_index_task
(parent_task, taskgraph, label_to_taskid)¶
-
taskgraph.morph.
make_s3_uploader_task
(parent_task)¶
-
taskgraph.morph.
morph
(taskgraph, label_to_taskid, parameters)¶ Apply all morphs
-
taskgraph.morph.
update_test_tasks
(taskid, build_taskid, taskgraph)¶ Tests task must download artifacts from uploader task.
taskgraph.optimize module¶
The objective of optimization is to remove as many tasks from the graph as possible, as efficiently as possible, thereby delivering useful results as quickly as possible. For example, ideally if only a test script is modified in a push, then the resulting graph contains only the corresponding test suite task.
See taskcluster/docs/optimization.rst
for more information.
-
class
taskgraph.optimize.
Either
(*substrategies, **kwargs)¶ Bases:
taskgraph.optimize.OptimizationStrategy
Given one or more optimization strategies, remove a task if any of them says to, and replace with a task if any finds a replacement (preferring the earliest). By default, each substrategy gets the same arg, but split_args can return a list of args for each strategy, if desired.
-
should_remove_task
(task, params, arg)¶
-
should_replace_task
(task, params, arg)¶
-
-
class
taskgraph.optimize.
IndexSearch
¶ Bases:
taskgraph.optimize.OptimizationStrategy
-
should_replace_task
(task, params, index_paths)¶ Look for a task with one of the given index paths
-
-
class
taskgraph.optimize.
OnlyIfDependenciesRun
¶ Bases:
taskgraph.optimize.OptimizationStrategy
Run this taks only if its dependencies run.
-
should_replace_task
(task, params, arg)¶
-
-
class
taskgraph.optimize.
OptimizationStrategy
¶ Bases:
object
-
should_remove_task
(task, params, arg)¶ Determine whether to optimize this task by removing it. Returns True to remove.
-
should_replace_task
(task, params, arg)¶ Determine whether to optimize this task by replacing it. Returns a taskId to replace this task, True to replace with nothing, or False to keep the task.
-
-
class
taskgraph.optimize.
SETA
¶ Bases:
taskgraph.optimize.OptimizationStrategy
-
should_remove_task
(task, params, _)¶
-
-
class
taskgraph.optimize.
SkipUnlessChanged
¶ Bases:
taskgraph.optimize.OptimizationStrategy
-
should_remove_task
(task, params, file_patterns)¶
-
-
class
taskgraph.optimize.
SkipUnlessSchedules
¶ Bases:
taskgraph.optimize.OptimizationStrategy
-
scheduled_by_push
= <functools.partial object>¶
-
should_remove_task
(task, params, conditions)¶
-
-
taskgraph.optimize.
get_subgraph
(target_task_graph, removed_tasks, replaced_tasks, label_to_taskid)¶ Return the subgraph of target_task_graph consisting only of non-optimized tasks and edges between them.
To avoid losing track of taskIds for tasks optimized away, this method simultaneously substitutes real taskIds for task labels in the graph, and populates each task definition’s dependencies key with the appropriate taskIds. Task references are resolved in the process.
-
taskgraph.optimize.
optimize_task_graph
(target_task_graph, params, do_not_optimize, existing_tasks=None, strategies=None)¶ Perform task optimization, returning a taskgraph and a map from label to assigned taskId, including replacement tasks.
-
taskgraph.optimize.
remove_tasks
(target_task_graph, params, optimizations, do_not_optimize)¶ Implement the “Removing Tasks” phase, returning a set of task labels of all removed tasks.
-
taskgraph.optimize.
replace_tasks
(target_task_graph, params, optimizations, do_not_optimize, label_to_taskid, removed_tasks, existing_tasks)¶ Implement the “Replacing Tasks” phase, returning a set of task labels of all replaced tasks. The replacement taskIds are added to label_to_taskid as a side-effect.
taskgraph.parameters module¶
-
exception
taskgraph.parameters.
ParameterMismatch
¶ Bases:
exceptions.Exception
Raised when a parameters.yml has extra or missing parameters.
-
class
taskgraph.parameters.
Parameters
(strict=True, **kwargs)¶ Bases:
mozbuild.util.ReadOnlyDict
An immutable dictionary with nicer KeyError messages on failure
-
check
()¶
-
file_url
(path)¶ Determine the VCS URL for viewing a file in the tree, suitable for viewing by a human.
Parameters: path (basestring) – The path, relative to the root of the repository. Return basestring: The URL displaying the given path.
-
is_try
()¶ Determine whether this graph is being built on a try project.
-
-
taskgraph.parameters.
get_contents
(path)¶
-
taskgraph.parameters.
get_head_ref
¶
-
taskgraph.parameters.
load_parameters_file
(filename, strict=True)¶ Load parameters from a path, url, decision task-id or project.
Examples
task-id=fdtgsD5DQUmAQZEaGMvQ4Q project=mozilla-central
taskgraph.target_tasks module¶
-
taskgraph.target_tasks.
filter_beta_release_tasks
(task, parameters, ignore_kinds=None, allow_l10n=False)¶
-
taskgraph.target_tasks.
filter_for_project
(task, parameters)¶ Filter tasks by project. Optionally enable nightlies.
-
taskgraph.target_tasks.
filter_on_nightly
(task, parameters)¶
-
taskgraph.target_tasks.
filter_upload_symbols
(task, parameters)¶
-
taskgraph.target_tasks.
get_method
(method)¶ Get a target_task_method to pass to a TaskGraphGenerator.
-
taskgraph.target_tasks.
standard_filter
(task, parameters)¶
-
taskgraph.target_tasks.
target_tasks_ash
(full_task_graph, parameters, graph_config)¶ Target tasks that only run on the ash branch.
-
taskgraph.target_tasks.
target_tasks_cedar
(full_task_graph, parameters, graph_config)¶ Target tasks that only run on the cedar branch.
-
taskgraph.target_tasks.
target_tasks_default
(full_task_graph, parameters, graph_config)¶ Target the tasks which have indicated they should be run on this project via the run_on_projects attributes.
-
taskgraph.target_tasks.
target_tasks_dmd
(full_task_graph, parameters, graph_config)¶ Target DMD that run nightly on the m-c branch.
-
taskgraph.target_tasks.
target_tasks_file_update
(full_task_graph, parameters, graph_config)¶ Select the set of tasks required to perform nightly in-tree file updates
-
taskgraph.target_tasks.
target_tasks_graphics
(full_task_graph, parameters, graph_config)¶ In addition to doing the filtering by project that the ‘default’ filter does, also remove artifact builds because we have csets on the graphics branch that aren’t on the candidate branches of artifact builds
-
taskgraph.target_tasks.
target_tasks_mozilla_beta
(full_task_graph, parameters, graph_config)¶ Select the set of tasks required for a promotable beta or release build of desktop, plus android CI. The candidates build process involves a pipeline of builds and signing, but does not include beetmover or balrog jobs.
-
taskgraph.target_tasks.
target_tasks_mozilla_release
(full_task_graph, parameters, graph_config)¶ Select the set of tasks required for a promotable beta or release build of desktop, plus android CI. The candidates build process involves a pipeline of builds and signing, but does not include beetmover or balrog jobs.
-
taskgraph.target_tasks.
target_tasks_nightly_desktop
(full_task_graph, parameters, graph_config)¶ Select the set of tasks required for a nightly build of linux, mac, windows.
-
taskgraph.target_tasks.
target_tasks_nightly_fennec
(full_task_graph, parameters, graph_config)¶ Select the set of tasks required for a nightly build of fennec. The nightly build process involves a pipeline of builds, signing, and, eventually, uploading the tasks to balrog.
-
taskgraph.target_tasks.
target_tasks_nightly_linux
(full_task_graph, parameters, graph_config)¶ Select the set of tasks required for a nightly build of linux. The nightly build process involves a pipeline of builds, signing, and, eventually, uploading the tasks to balrog.
-
taskgraph.target_tasks.
target_tasks_nightly_macosx
(full_task_graph, parameters, graph_config)¶ Select the set of tasks required for a nightly build of macosx. The nightly build process involves a pipeline of builds, signing, and, eventually, uploading the tasks to balrog.
-
taskgraph.target_tasks.
target_tasks_nightly_win32
(full_task_graph, parameters, graph_config)¶ Select the set of tasks required for a nightly build of win32 and win64. The nightly build process involves a pipeline of builds, signing, and, eventually, uploading the tasks to balrog.
-
taskgraph.target_tasks.
target_tasks_nightly_win64
(full_task_graph, parameters, graph_config)¶ Select the set of tasks required for a nightly build of win32 and win64. The nightly build process involves a pipeline of builds, signing, and, eventually, uploading the tasks to balrog.
-
taskgraph.target_tasks.
target_tasks_pine
(full_task_graph, parameters, graph_config)¶ Bug 1339179 - no mobile automation needed on pine
-
taskgraph.target_tasks.
target_tasks_promote_devedition
(full_task_graph, parameters, graph_config)¶ Select the superset of tasks required to promote a beta or release build of devedition. This should include all non-android mozilla_{beta,release} tasks, plus l10n, beetmover, balrog, etc.
-
taskgraph.target_tasks.
target_tasks_promote_fennec
(full_task_graph, parameters, graph_config)¶ Select the set of tasks required for a candidates build of fennec. The nightly build process involves a pipeline of builds, signing, and, eventually, uploading the tasks to balrog.
-
taskgraph.target_tasks.
target_tasks_promote_firefox
(full_task_graph, parameters, graph_config)¶ Select the superset of tasks required to promote a beta or release build of firefox. This should include all non-android mozilla_{beta,release} tasks, plus l10n, beetmover, balrog, etc.
-
taskgraph.target_tasks.
target_tasks_push_devedition
(full_task_graph, parameters, graph_config)¶ Select the set of tasks required to push a build of devedition to cdns. Previous build deps will be optimized out via action task.
-
taskgraph.target_tasks.
target_tasks_push_firefox
(full_task_graph, parameters, graph_config)¶ Select the set of tasks required to push a build of firefox to cdns. Previous build deps will be optimized out via action task.
-
taskgraph.target_tasks.
target_tasks_searchfox
(full_task_graph, parameters, graph_config)¶ Select tasks required for indexing Firefox for Searchfox web site each day
-
taskgraph.target_tasks.
target_tasks_ship_devedition
(full_task_graph, parameters, graph_config)¶ Select the set of tasks required to ship devedition. Previous build deps will be optimized out via action task.
-
taskgraph.target_tasks.
target_tasks_ship_fennec
(full_task_graph, parameters, graph_config)¶ Select the set of tasks required to ship fennec. Previous build deps will be optimized out via action task.
-
taskgraph.target_tasks.
target_tasks_ship_firefox
(full_task_graph, parameters, graph_config)¶ Select the set of tasks required to ship firefox. Previous build deps will be optimized out via action task.
-
taskgraph.target_tasks.
target_tasks_try
(full_task_graph, parameters, graph_config)¶
-
taskgraph.target_tasks.
target_tasks_valgrind
(full_task_graph, parameters, graph_config)¶ Target tasks that only run on the cedar branch.
taskgraph.task module¶
-
class
taskgraph.task.
Task
(kind, label, attributes, task, optimization=None, dependencies=None)¶ Bases:
object
Representation of a task in a TaskGraph. Each Task has, at creation:
- kind: the name of the task kind
- label; the label for this task
- attributes: a dictionary of attributes for this task (used for filtering)
- task: the task definition (JSON-able dictionary)
- optimization: optimization to apply to the task (see taskgraph.optimize)
- dependencies: tasks this one depends on, in the form {name: label}, for example {‘build’: ‘build-linux64/opt’, ‘docker-image’: ‘build-docker-image-desktop-test’}
And later, as the task-graph processing proceeds:
- task_id – TaskCluster taskId under which this task will be created
This class is just a convenience wraper for the data type and managing display, comparison, serialization, etc. It has no functionality of its own.
-
classmethod
from_json
(task_dict)¶ Given a data structure as produced by taskgraph.to_json, re-construct the original Task object. This is used to “resume” the task-graph generation process, for example in Action tasks.
-
to_json
()¶
taskgraph.taskgraph module¶
-
class
taskgraph.taskgraph.
TaskGraph
(tasks, graph)¶ Bases:
object
Representation of a task graph.
A task graph is a combination of a Graph and a dictionary of tasks indexed by label. TaskGraph instances should be treated as immutable.
-
for_each_task
(f, *args, **kwargs)¶
-
classmethod
from_json
(tasks_dict)¶ This code is used to generate the a TaskGraph using a dictionary which is representative of the TaskGraph.
-
to_json
()¶ Return a JSON-able object representing the task graph, as documented
-
taskgraph.try_option_syntax module¶
-
class
taskgraph.try_option_syntax.
TryOptionSyntax
(parameters, full_task_graph, graph_config)¶ Bases:
object
-
find_all_attribute_suffixes
(graph, prefix)¶
-
handle_alias
(test, all_tests)¶ Expand a test if its name refers to an alias, returning a list of test dictionaries cloned from the first (to maintain any metadata).
-
parse_build_types
(build_types_arg, full_task_graph)¶
-
parse_jobs
(jobs_arg)¶
-
parse_platforms
(platform_arg, full_task_graph)¶
-
parse_test_chunks
(all_tests, tests)¶ Test flags may include parameters to narrow down the number of chunks in a given push. We don’t model 1 chunk = 1 job in taskcluster so we must check each test flag to see if it is actually specifying a chunk.
-
parse_test_option
(attr_name, test_arg, full_task_graph)¶ Parse a unittest (-u) or talos (-t) option, in the context of a full task graph containing available unittest_try_name or talos_try_name attributes. There are three cases:
- test_arg is == ‘none’ (meaning an empty list)
- test_arg is == ‘all’ (meaning use the list of jobs for that job type)
- test_arg is comma string which needs to be parsed
-
parse_test_opts
(input_str, all_platforms)¶ Parse testspec,testspec,.., where each testspec is a test name optionally followed by a list of test platforms or negated platforms in [].
No brackets indicates that tests should run on all platforms for which builds are available. If testspecs are provided, then each is treated, from left to right, as an instruction to include or (if negated) exclude a set of test platforms. A single spec may expand to multiple test platforms via UNITTEST_PLATFORM_PRETTY_NAMES. If the first test spec is negated, processing begins with the full set of available test platforms; otherwise, processing begins with an empty set of test platforms.
-
task_matches
(task)¶
-
-
taskgraph.try_option_syntax.
alias_contains
(infix)¶
-
taskgraph.try_option_syntax.
alias_matches
(pattern)¶
-
taskgraph.try_option_syntax.
alias_prefix
(prefix)¶
-
taskgraph.try_option_syntax.
escape_whitespace_in_brackets
(input_str)¶ In tests you may restrict them by platform [] inside of the brackets whitespace may occur this is typically invalid shell syntax so we escape it with backslash sequences .
-
taskgraph.try_option_syntax.
parse_message
(message)¶
-
taskgraph.try_option_syntax.
split_try_msg
(message)¶