mozbuild package

Subpackages

Submodules

mozbuild.android_version_code module

mozbuild.android_version_code.android_version_code(buildid, *args, **kwargs)
mozbuild.android_version_code.android_version_code_v0(buildid, cpu_arch=None, min_sdk=0, max_sdk=0)
mozbuild.android_version_code.android_version_code_v1(buildid, cpu_arch=None, min_sdk=0, max_sdk=0)

Generate a v1 android:versionCode.

The important consideration is that version codes be monotonically increasing (per Android package name) for all published builds. The input build IDs are based on timestamps and hence are always monotonically increasing.

The generated v1 version codes look like (in binary):

0111 1000 0010 tttt tttt tttt tttt txpg

The 17 bits labelled ‘t’ represent the number of hours since midnight on September 1, 2015. (2015090100 in YYYYMMMDDHH format.) This yields a little under 15 years worth of hourly build identifiers, since 2**17 / (366 * 24) =~ 14.92.

The bits labelled ‘x’, ‘p’, and ‘g’ are feature flags.

The bit labelled ‘x’ is 1 if the build is for an x86 or ARM64 architecture, and 0 otherwise, which means the build is for a (32-bit) ARM architecture. (Fennec no longer supports ARMv6, so ARM is equivalent to ARMv7.

ARM64 is also known as AArch64; it is logically ARMv8.)

For the same release, x86 and ARM64 builds have higher version codes and take precedence over ARM builds, so that they are preferred over ARM on devices that have ARM emulation.

The bit labelled ‘p’ is a placeholder that is always 0 (for now).

Firefox no longer supports API 14 or earlier.

This version code computation allows for a split on API levels that allowed us to ship builds specifically for Gingerbread (API 9-10); we preserve that functionality for sanity’s sake, and to allow us to reintroduce a split in the future.

At present, the bit labelled ‘g’ is 1 if the build is an ARM build targeting API 16+, which will always be the case.

We throw an explanatory exception when we are within one calendar year of running out of build events. This gives lots of time to update the version scheme. The responsible individual should then bump the range (to allow builds to continue) and use the time remaining to update the version scheme via the reserved high order bits.

N.B.: the reserved 0 bit to the left of the highest order ‘t’ bit can, sometimes, be used to bump the version scheme. In addition, by reducing the granularity of the build identifiers (for example, moving to identifying builds every 2 or 4 hours), the version scheme may be adjusted further still without losing a (valuable) high order bit.

mozbuild.android_version_code.main(argv)

mozbuild.artifacts module

Fetch build artifacts from a Firefox tree.

This provides an (at-the-moment special purpose) interface to download Android artifacts from Mozilla’s Task Cluster.

This module performs the following steps:

  • find a candidate hg parent revision. At one time we used the local pushlog, which required the mozext hg extension. This isn’t feasible with git, and it is only mildly less efficient to not use the pushlog, so we don’t use it even when querying hg.
  • map the candidate parent to candidate Task Cluster tasks and artifact locations. Pushlog entries might not correspond to tasks (yet), and those tasks might not produce the desired class of artifacts.
  • fetch fresh Task Cluster artifacts and purge old artifacts, using a simple Least Recently Used cache.
  • post-process fresh artifacts, to speed future installation. In particular, extract relevant files from Mac OS X DMG files into a friendly archive format so we don’t have to mount DMG files frequently.

The bulk of the complexity is in managing and persisting several caches. If we found a Python LRU cache that pickled cleanly, we could remove a lot of this code! Sadly, I found no such candidate implementations, so we pickle pylru caches manually.

None of the instances (or the underlying caches) are safe for concurrent use. A future need, perhaps.

This module requires certain modules be importable from the ambient Python environment. |mach artifact| ensures these modules are available, but other consumers will need to arrange this themselves.

class mozbuild.artifacts.AndroidArtifactJob(package_re, tests_re, log=None, download_symbols=False, substs=None)

Bases: mozbuild.artifacts.ArtifactJob

package_artifact_patterns = set([u'application.ini', u'**/*.so', u'**/interfaces.xpt', u'platform.ini'])
process_package_artifact(filename, processed_filename)
product = u'mobile'
class mozbuild.artifacts.ArtifactCache(cache_dir, log=None, skip_cache=False)

Bases: object

Fetch Task Cluster artifact URLs and purge least recently used artifacts from disk.

clear_cache()
fetch(url, force=False)
log(*args, **kwargs)
class mozbuild.artifacts.ArtifactJob(package_re, tests_re, log=None, download_symbols=False, substs=None)

Bases: object

find_candidate_artifacts(artifacts)
log(*args, **kwargs)
process_artifact(filename, processed_filename)
process_package_artifact(filename, processed_filename)
process_symbols_archive(filename, processed_filename)
process_tests_artifact(filename, processed_filename)
test_artifact_patterns = set([(u'bin/pk12util', (u'bin', u'bin')), (u'bin/GenerateOCSPResponse', (u'bin', u'bin')), (u'bin/plugins/gmp-*/*/*', (u'bin/plugins', u'bin')), (u'bin/screentopng', (u'bin', u'bin')), (u'bin/geckodriver', (u'bin', u'bin')), (u'bin/components/*.xpt', (u'bin/components', u'bin/components')), (u'bin/certutil', (u'bin', u'bin')), (u'bin/xpcshell', (u'bin', u'bin')), (u'bin/ssltunnel', (u'bin', u'bin')), (u'bin/BadCertServer', (u'bin', u'bin')), (u'bin/OCSPStaplingServer', (u'bin', u'bin')), (u'bin/plugins/*', (u'bin/plugins', u'plugins')), (u'bin/fileid', (u'bin', u'bin')), (u'bin/SymantecSanctionsServer', (u'bin', u'bin'))])
class mozbuild.artifacts.ArtifactPersistLimit(log=None)

Bases: dlmanager.persist_limit.PersistLimit

Handle persistence for artifacts cache

When instantiating a DownloadManager, it starts by filling the PersistLimit instance it’s given with register_dir_content. In practice, this registers all the files already in the cache directory. After a download finishes, the newly downloaded file is registered, and the oldest files registered to the PersistLimit instance are removed depending on the size and file limits it’s configured for. This is all good, but there are a few tweaks we want here: - We have pickle files in the cache directory that we don’t want purged. - Files that were just downloaded in the same session shouldn’t be purged.

(if for some reason we end up downloading more than the default max size,
we don’t want the files to be purged)

To achieve this, this subclass of PersistLimit inhibits the register_file method for pickle files and tracks what files were downloaded in the same session to avoid removing them.

The register_file method may be used to register cache matches too, so that later sessions know they were freshly used.

log(*args, **kwargs)
register_dir_content(directory, pattern=u'*')
register_file(path)
remove_all()
remove_old_files()
class mozbuild.artifacts.Artifacts(tree, substs, defines, job=None, log=None, cache_dir=u'.', hg=None, git=None, skip_cache=False, topsrcdir=None)

Bases: object

Maintain state to efficiently fetch build artifacts from a Firefox tree.

clear_cache()
find_pushhead_artifacts(task_cache, job, tree, pushhead)
install_from(source, distdir)

Install artifacts from a source into the given distdir.

install_from_file(filename, distdir)
install_from_recent(distdir)
install_from_revset(revset, distdir)
install_from_url(url, distdir)
log(*args, **kwargs)
class mozbuild.artifacts.CacheManager(cache_dir, cache_name, cache_size, cache_callback=None, log=None, skip_cache=False)

Bases: object

Maintain an LRU cache. Provide simple persistence, including support for loading and saving the state using a “with” block. Allow clearing the cache and printing the cache for debugging.

Provide simple logging.

clear_cache()
dump_cache()
load_cache()
log(*args, **kwargs)
class mozbuild.artifacts.LinuxArtifactJob(package_re, tests_re, log=None, download_symbols=False, substs=None)

Bases: mozbuild.artifacts.ArtifactJob

package_artifact_patterns = set([u'firefox/firefox-bin', u'firefox/plugin-container', u'firefox/**/interfaces.xpt', u'firefox/application.ini', u'firefox/**/*.so', u'firefox/crashreporter', u'firefox/minidump-analyzer', u'firefox/platform.ini', u'firefox/updater', u'firefox/pingsender', u'firefox/dependentlibs.list', u'firefox/firefox'])
process_package_artifact(filename, processed_filename)
product = u'firefox'
class mozbuild.artifacts.MacArtifactJob(package_re, tests_re, log=None, download_symbols=False, substs=None)

Bases: mozbuild.artifacts.ArtifactJob

process_package_artifact(filename, processed_filename)
product = u'firefox'
class mozbuild.artifacts.PushheadCache(cache_dir, log=None, skip_cache=False)

Bases: mozbuild.artifacts.CacheManager

Helps map tree/revision pairs to parent pushheads according to the pushlog.

parent_pushhead_id(*args, **kwargs)
pushid_range(*args, **kwargs)
class mozbuild.artifacts.TaskCache(cache_dir, log=None, skip_cache=False)

Bases: mozbuild.artifacts.CacheManager

Map candidate pushheads to Task Cluster task IDs and artifact URLs.

artifact_urls(*args, **kwargs)
class mozbuild.artifacts.WinArtifactJob(package_re, tests_re, log=None, download_symbols=False, substs=None)

Bases: mozbuild.artifacts.ArtifactJob

package_artifact_patterns = set([u'firefox/**/interfaces.xpt', u'firefox/**/*.dll', u'firefox/application.ini', u'firefox/platform.ini', u'firefox/dependentlibs.list', u'firefox/*.exe'])
process_package_artifact(filename, processed_filename)
product = u'firefox'
test_artifact_patterns = set([(u'bin/screenshot.exe', (u'bin', u'bin')), (u'bin/GenerateOCSPResponse.exe', (u'bin', u'bin')), (u'bin/BadCertServer.exe', (u'bin', u'bin')), (u'bin/SymantecSanctionsServer.exe', (u'bin', u'bin')), (u'bin/certutil.exe', (u'bin', u'bin')), (u'bin/geckodriver.exe', (u'bin', u'bin')), (u'bin/pk12util.exe', (u'bin', u'bin')), (u'bin/ssltunnel.exe', (u'bin', u'bin')), (u'bin/OCSPStaplingServer.exe', (u'bin', u'bin')), (u'bin/plugins/gmp-*/*/*', (u'bin/plugins', u'bin')), (u'bin/plugins/*', (u'bin/plugins', u'plugins')), (u'bin/components/*', (u'bin/components', u'bin/components')), (u'bin/xpcshell.exe', (u'bin', u'bin')), (u'bin/fileid.exe', (u'bin', u'bin'))])
mozbuild.artifacts.cachedmethod(cachefunc)

Decorator to wrap a class or instance method with a memoizing callable that saves results in a (possibly shared) cache.

mozbuild.artifacts.get_job_details(job, log=None, download_symbols=False, substs=None)

mozbuild.base module

exception mozbuild.base.BadEnvironmentException

Bases: exceptions.Exception

Base class for errors raised when the build environment is not sane.

exception mozbuild.base.BuildEnvironmentNotFoundException

Bases: mozbuild.base.BadEnvironmentException

Raised when we could not find a build environment.

class mozbuild.base.ExecutionSummary(summary_format, **data)

Bases: dict

Helper for execution summaries.

extend(summary_format, **data)
class mozbuild.base.MachCommandBase(context)

Bases: mozbuild.base.MozbuildObject

Base class for mach command providers that wish to be MozbuildObjects.

This provides a level of indirection so MozbuildObject can be refactored without having to change everything that inherits from it.

class mozbuild.base.MachCommandConditions

Bases: object

A series of commonly used condition functions which can be applied to mach commands with providers deriving from MachCommandBase.

static is_android()

Must have an Android build.

static is_artifact_build()

Must be an artifact build.

static is_firefox()

Must have a Firefox build.

static is_git()

Must have a git source checkout.

static is_hg()

Must have a mercurial source checkout.

class mozbuild.base.MozbuildObject(topsrcdir, settings, log_manager, topobjdir=None, mozconfig=<object object>)

Bases: mach.mixin.process.ProcessExecutionMixin

Base class providing basic functionality useful to many modules.

Modules in this package typically require common functionality such as accessing the current config, getting the location of the source directory, running processes, etc. This classes provides that functionality. Other modules can inherit from this class to obtain this functionality easily.

bindir
config_environment

Returns the ConfigEnvironment for the current build configuration.

This property is only available once configure has executed.

If configure’s output is not available, this will raise.

defines
distdir
extra_environment_variables

A specialized version of the memoize decorator that works for class instance properties.

classmethod from_environment(cwd=None, detect_virtualenv_mozinfo=True)

Create a MozbuildObject by detecting the proper one from the env.

This examines environment state like the current working directory and creates a MozbuildObject from the found source directory, mozconfig, etc.

The role of this function is to identify a topsrcdir, topobjdir, and mozconfig file.

If the current working directory is inside a known objdir, we always use the topsrcdir and mozconfig associated with that objdir.

If the current working directory is inside a known srcdir, we use that topsrcdir and look for mozconfigs using the default mechanism, which looks inside environment variables.

If the current Python interpreter is running from a virtualenv inside an objdir, we use that as our objdir.

If we’re not inside a srcdir or objdir, an exception is raised.

detect_virtualenv_mozinfo determines whether we should look for a mozinfo.json file relative to the virtualenv directory. This was added to facilitate testing. Callers likely shouldn’t change the default.

get_binary_path(what=u'app', validate_exists=True, where=u'default')

Obtain the path to a compiled binary for this build configuration.

The what argument is the program or tool being sought after. See the code implementation for supported values.

If validate_exists is True (the default), we will ensure the found path exists before returning, raising an exception if it doesn’t.

If where is ‘staged-package’, we will return the path to the binary in the package staging directory.

If no arguments are specified, we will return the main binary for the configured XUL application.

includedir
is_clobber_needed()
mozbuild_reader(config_mode=u'build', vcs_revision=None, vcs_check_clean=True)

Obtain a BuildReader for evaluating moz.build files.

Given arguments, returns a mozbuild.frontend.reader.BuildReader that can be used to evaluate moz.build files for this repo.

config_mode is either build or empty. If build, self.config_environment is used. This requires a configured build system to work. If empty, an empty config is used. empty is appropriate for file-based traversal mode where Files metadata is read.

If vcs_revision is defined, it specifies a version control revision to use to obtain files content. The default is to use the filesystem. This mode is only supported with Mercurial repositories.

If vcs_revision is not defined and the version control checkout is sparse, this implies vcs_revision='.'.

If vcs_revision is . (denotes the parent of the working directory), we will verify that the working directory is clean unless vcs_check_clean is False. This prevents confusion due to uncommitted file changes not being reflected in the reader.

mozconfig

Returns information about the current mozconfig file.

This a dict as returned by MozconfigLoader.read_mozconfig()

non_global_defines
notify(msg)

Show a desktop notification with the supplied message

On Linux and Mac, this will show a desktop notification with the message, but on Windows we can only flash the screen.

platform

Returns current platform and architecture name

python3

A specialized version of the memoize decorator that works for class instance properties.

repository

A specialized version of the memoize decorator that works for class instance properties.

resolve_config_guess()
resolve_mozconfig_topobjdir(default=None)
statedir
substs
topobjdir
virtualenv_manager
exception mozbuild.base.ObjdirMismatchException(objdir1, objdir2)

Bases: mozbuild.base.BadEnvironmentException

Raised when the current dir is an objdir and doesn’t match the mozconfig.

class mozbuild.base.PathArgument(arg, topsrcdir, topobjdir, cwd=None)

Bases: object

Parse a filesystem path argument and transform it in various ways.

objdir_path()
relpath()

Return a path relative to the topsrcdir or topobjdir.

If the argument is a path to a location in one of the base directories (topsrcdir or topobjdir), then strip off the base directory part and just return the path within the base directory.

srcdir_path()
mozbuild.base.ancestors(path)

Emit the parent directories of a path.

mozbuild.base.samepath(path1, path2)

mozbuild.chunkify module

exception mozbuild.chunkify.ChunkingError

Bases: exceptions.Exception

mozbuild.chunkify.chunkify(things, this_chunk, chunks)
mozbuild.chunkify.split_evenly(n, chunks)

Split an integer into evenly distributed list

>>> split_evenly(7, 3)
[3, 2, 2]

>>> split_evenly(12, 3)
[4, 4, 4]

>>> split_evenly(35, 10)
[4, 4, 4, 4, 4, 3, 3, 3, 3, 3]

>>> split_evenly(1, 2)
Traceback (most recent call last):
    ...
ChunkingError: Number of chunks is greater than number

mozbuild.config_status module

mozbuild.config_status.config_status(topobjdir='.', topsrcdir='.', defines=None, non_global_defines=None, substs=None, source=None, mozconfig=None, args=['doc'])

Main function, providing config.status functionality.

Contrary to config.status, it doesn’t use CONFIG_FILES or CONFIG_HEADERS variables.

Without the -n option, this program acts as config.status and considers the current directory as the top object directory, even when config.status is in a different directory. It will, however, treat the directory containing config.status as the top object directory with the -n option.

The options to this function are passed when creating the ConfigEnvironment. These lists, as well as the actual wrapper script around this function, are meant to be generated by configure. See build/autoconf/config.status.m4.

mozbuild.doctor module

class mozbuild.doctor.Doctor(srcdir, objdir, fix)

Bases: object

check_all()
check_mount_lastaccess(mount)
cpu
fs_lastaccess
getmount(path)
memory
mozillabuild
platform
prompt_bool(prompt, limit=5)

Prompts the user with prompt and requires a boolean value.

report(results)
storage_freespace

mozbuild.dotproperties module

class mozbuild.dotproperties.DotProperties(file=None)

A thin representation of a key=value .properties file.

get(key, default=None)
get_dict(prefix, required_keys=[])

Turns {‘foo.title’:’title’, …} into {‘title’:’title’, …}.

If |required_keys| is present, it must be an iterable of required key names. If a required key is not present, ValueError is thrown.

Returns {} to indicate an empty or missing dict.

get_list(prefix)

Turns {‘list.0’:’foo’, ‘list.1’:’bar’} into [‘foo’, ‘bar’].

Returns [] to indicate an empty or missing list.

update(file)

Updates properties from a file name or file-like object.

Ignores empty lines and comment lines.

mozbuild.faster_daemon module

mozbuild.generated_sources module

mozbuild.generated_sources.get_filename_with_digest(name, contents)

Return the filename that will be used to store the generated file in the S3 bucket, consisting of the SHA-512 digest of contents joined with the relative path name.

mozbuild.generated_sources.get_generated_sources()

Yield tuples of (objdir-rel-path, file) for generated source files in this objdir, where file is either an absolute path to the file or a mozpack.File instance.

mozbuild.generated_sources.get_s3_region_and_bucket()

Return a tuple of (region, bucket) giving the AWS region and S3 bucket to which generated sources should be uploaded.

mozbuild.generated_sources.sha512_digest(data)

Generate the SHA-512 digest of data and return it as a hex string.

mozbuild.gn_processor module

class mozbuild.gn_processor.GnConfigGenBackend(environment)

Bases: mozbuild.backend.base.BuildBackend

consume_finished()
consume_object(obj)
class mozbuild.gn_processor.GnMozbuildWriterBackend(environment)

Bases: mozbuild.backend.base.BuildBackend

consume_finished()
consume_object(obj)
class mozbuild.gn_processor.MozbuildWriter(fh)

Bases: object

finalize()
mb_serialize(v)
terminate_condition()
write(content)
write_attrs(context_attrs)
write_condition(values)
write_ln(line)
write_mozbuild_dict(key, value)
write_mozbuild_list(key, value)
write_mozbuild_value(key, value)
mozbuild.gn_processor.filter_gn_config(gn_result, config, sandbox_vars, input_vars)
mozbuild.gn_processor.find_common_attrs(config_attributes)
mozbuild.gn_processor.find_deps(all_targets, target)
mozbuild.gn_processor.generate_gn_config(config, srcdir, output, non_unified_sources, gn_binary, input_variables, sandbox_variables)
mozbuild.gn_processor.process_gn_config(gn_config, srcdir, config, output, non_unified_sources, sandbox_vars, mozilla_flags)
mozbuild.gn_processor.write_mozbuild(config, srcdir, output, non_unified_sources, gn_config_files, mozilla_flags)

mozbuild.html_build_viewer module

class mozbuild.html_build_viewer.BuildViewerServer(address=u'localhost', port=0)

Bases: object

add_resource_json_file(key, path)

Register a resource JSON file with the server.

The file will be made available under the name/key specified.

add_resource_json_url(key, url)

Register a resource JSON file at a URL.

run()
url
class mozbuild.html_build_viewer.HTTPHandler(request, client_address, server)

Bases: BaseHTTPServer.BaseHTTPRequestHandler

do_GET()
do_POST()
serve_docroot(root, path)

mozbuild.jar module

jarmaker.py provides a python class to package up chrome content by processing jar.mn files.

See the documentation for jar.mn on MDC for further details on the format.

class mozbuild.jar.JarMaker(outputFormat='flat', useJarfileManifest=True, useChromeManifest=False)

Bases: object

JarMaker reads jar.mn files and process those into jar files or flat directories, along with chrome.manifest files.

class OutputHelper_flat(basepath)

Bases: object

Provide getDestModTime and getOutput for a given flat output directory. The helper method ensureDirFor is used by the symlink subclass.

ensureDirFor(name)
getDestModTime(aPath)
getOutput(name)
class OutputHelper_jar(jarfile)

Bases: object

Provide getDestModTime and getOutput for a given jarfile.

getDestModTime(aPath)
getOutput(name)

Bases: mozbuild.jar.OutputHelper_flat

Subclass of OutputHelper_flat that provides a helper for creating a symlink including creating the parent directories.

finalizeJar(jardir, jarbase, jarname, chromebasepath, register, doZip=True)
Helper method to write out the chrome registration entries to
jarfile.manifest or chrome.manifest, or both.

The actual file processing is done in updateManifest.

generateLocaleDirs(relativesrcdir)
getCommandLineParser()

Get a optparse.OptionParser for jarmaker.

This OptionParser has the options for jarmaker as well as the options for the inner PreProcessor.

makeJar(infile, jardir)

makeJar is the main entry point to JarMaker.

It takes the input file, the output directory, the source dirs and the top source dir as argument, and optionally the l10n dirs.

processJarSection(jarinfo, jardir)

Internal method called by makeJar to actually process a section of a jar.mn file.

updateManifest(manifestPath, chromebasepath, register)

updateManifest replaces the % in the chrome registration entries with the given chrome base path, and updates the given manifest file.

mozbuild.mach_commands module

class mozbuild.mach_commands.ArtifactSubCommand(command, subcommand, description=None, parser=None)

Bases: mach.decorators.SubCommand

class mozbuild.mach_commands.Build(context)

Bases: mozbuild.base.MachCommandBase

Interface to build the tree.

build(what=None, disable_extra_make_dependencies=None, jobs=0, directory=None, verbose=False, keep_going=False)

Build the source tree.

With no arguments, this will perform a full build.

Positional arguments define targets to build. These can be make targets or patterns like “<dir>/<target>” to indicate a make target within a directory.

There are a few special targets that can be used to perform a partial build faster than what mach build would perform:

  • binaries - compiles and links all C/C++ sources and produces shared libraries and executables (binaries).
  • faster - builds JavaScript, XUL, CSS, etc files.

“binaries” and “faster” almost fully complement each other. However, there are build actions not captured by either. If things don’t appear to be rebuilding, perform a vanilla mach build to rebuild the world.

build_backend(backend, diff=False, verbose=False, dry_run=False)
configure(options=None, buildstatus_messages=False, line_handler=None)
resource_usage(address=None, port=None, browser=None, url=None)
class mozbuild.mach_commands.Buildsymbols(context)

Bases: mozbuild.base.MachCommandBase

Produce a package of debug symbols suitable for use with Breakpad.

buildsymbols()
class mozbuild.mach_commands.CargoProvider(context)

Bases: mozbuild.base.MachCommandBase

Invoke cargo in useful ways.

cargo()
check(all_crates=None, crates=None)
class mozbuild.mach_commands.ClangCommands(context)

Bases: mozbuild.base.MachCommandBase

clang_complete()
class mozbuild.mach_commands.Clobber(context)

Bases: mozbuild.base.MachCommandBase

CLOBBER_CHOICES = [u'objdir', u'python']
NO_AUTO_LOG = True
clobber(what, full=False)
substs
class mozbuild.mach_commands.Doctor(context)

Bases: mozbuild.base.MachCommandBase

Provide commands for diagnosing common build environment problems

doctor(fix=None)
class mozbuild.mach_commands.GTestCommands(context)

Bases: mozbuild.base.MachCommandBase

gtest(shuffle, jobs, gtest_filter, tbpl_parser, debug, debugger, debugger_args)
prepend_debugger_args(args, debugger, debugger_args)

Given an array with program arguments, prepend arguments to run it under a debugger.

Parameters:
  • args – The executable and arguments used to run the process normally.
  • debugger – The debugger to use, or empty to use the default debugger.
  • debugger_args – Any additional parameters to pass to the debugger.
class mozbuild.mach_commands.Install(context)

Bases: mozbuild.base.MachCommandBase

Install a package.

install(verbose=False)
class mozbuild.mach_commands.Logs(context)

Bases: mozbuild.base.MachCommandBase

Provide commands to read mach logs.

NO_AUTO_LOG = True
show_log(log_file=None)
class mozbuild.mach_commands.MachDebug(context)

Bases: mozbuild.base.MachCommandBase

environment(format, output=None, verbose=False)
class mozbuild.mach_commands.Makefiles(context)

Bases: mozbuild.base.MachCommandBase

empty()
class mozbuild.mach_commands.Package(context)

Bases: mozbuild.base.MachCommandBase

Package the built product for distribution.

package(verbose=False)
class mozbuild.mach_commands.PackageFrontend(context)

Bases: mozbuild.base.MachCommandBase

Fetch and install binary artifacts from Mozilla automation.

artifact()

Download, cache, and install pre-built binary artifacts to build Firefox.

Use |mach build| as normal to freshen your installed binary libraries: artifact builds automatically download, cache, and install binary artifacts from Mozilla automation, replacing whatever may be in your object directory. Use |mach artifact last| to see what binary artifacts were last used.

Never build libxul again!

artifact_clear_cache(tree=None, job=None, verbose=False)
artifact_install(source=None, skip_cache=False, tree=None, job=None, verbose=False)
artifact_toolchain(verbose=False, cache_dir=None, skip_cache=False, from_build=(), tooltool_manifest=None, authentication_file=None, tooltool_url=None, no_unpack=False, retry=None, artifact_manifest=None, files=())

Download, cache and install pre-built toolchains.

class mozbuild.mach_commands.Repackage(context)

Bases: mozbuild.base.MachCommandBase

Repackages artifacts into different formats.

This is generally used after packages are signed by the signing scriptworkers in order to bundle things up into shippable formats, such as a .dmg on OSX or an installer exe on Windows.

repackage()
repackage_dmg(input, output)
repackage_installer(tag, setupexe, package, output, package_name, sfx_stub)
repackage_mar(input, mar, output)
class mozbuild.mach_commands.RunProgram(context)

Bases: mozbuild.base.MachCommandBase

Run the compiled program.

prog_group = u'the compiled program'
run(params, remote, background, noprofile, disable_e10s, enable_crash_reporter, setpref, debug, debugger, debugger_args, dmd, mode, stacks, show_dump_stats)
class mozbuild.mach_commands.RunSettings
config_settings = [(u'runprefs.*', u'string', u'Pass a pref into Firefox when using `mach run`, of the form `foo.bar=value`.\nPrefs will automatically be cast into the appropriate type. Integers can be\nsingle quoted to force them to be strings.')]
class mozbuild.mach_commands.StaticAnalysis(context)

Bases: mozbuild.base.MachCommandBase

Utilities for running C++ static analysis checks and format.

check(source=None, jobs=2, strip=1, verbose=False, checks=u'-*', fix=False, header_filter=u'')
clang_format(show, path, verbose=False)
clear_cache(verbose=False)
install(source=None, skip_cache=False, verbose=False)
print_checks(verbose=False)
static_analysis()
class mozbuild.mach_commands.StaticAnalysisMonitor(srcdir, objdir, total)

Bases: object

current_file
num_files
num_files_processed
on_line(line)
warnings_db
class mozbuild.mach_commands.StaticAnalysisSubCommand(command, subcommand, description=None, parser=None)

Bases: mach.decorators.SubCommand

class mozbuild.mach_commands.StoreDebugParamsAndWarnAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)

Bases: argparse.Action

class mozbuild.mach_commands.Vendor(context)

Bases: mozbuild.base.MachCommandBase

Vendor third-party dependencies into the source repository.

vendor()
vendor_aom(**kwargs)
vendor_rust(**kwargs)
class mozbuild.mach_commands.Warnings(context)

Bases: mozbuild.base.MachCommandBase

Provide commands for inspecting warnings.

database
database_path
join_ensure_dir(dir1, dir2)
list(directory=None, flags=None, report=None)
summary(directory=None, report=None)
class mozbuild.mach_commands.Watch(context)

Bases: mozbuild.base.MachCommandBase

Interface to watch and re-build the tree.

watch(verbose=False)

Watch and re-build the source tree.

class mozbuild.mach_commands.WebRTCGTestCommands(context)

Bases: mozbuild.mach_commands.GTestCommands

gtest(gtest_filter, debug, debugger, debugger_args)

mozbuild.makeutil module

class mozbuild.makeutil.Makefile

Bases: object

Provides an interface for writing simple makefiles

Instances of this class are created, populated with rules, then written.

add_statement(statement)

Add a raw statement in the makefile. Meant to be used for simple variable assignments.

create_rule(targets=[])

Create a new rule in the makefile for the given targets. Returns the corresponding Rule instance.

dump(fh, removal_guard=True)

Dump all the rules to the given file handle. Optionally (and by default), add guard rules for file removals (empty rules for other rules’ dependencies)

class mozbuild.makeutil.Rule(targets=[])

Bases: object

Class handling simple rules in the form: target1 target2 … : dep1 dep2 …

command1 command2 …
add_commands(commands)

Add commands to the rule.

add_dependencies(deps)

Add dependencies to the rule.

add_targets(targets)

Add additional targets to the rule.

commands()

Return an iterator on the rule commands.

dependencies()

Return an iterator on the rule dependencies.

dump(fh)

Dump the rule to the given file handle.

targets()

Return an iterator on the rule targets.

mozbuild.makeutil.read_dep_makefile(fh)

Read the file handler containing a dep makefile (simple makefile only containing dependencies) and returns an iterator of the corresponding Rules it contains. Ignores removal guard rules.

mozbuild.makeutil.write_dep_makefile(fh, target, deps)

Write a Makefile containing only target’s dependencies to the file handle specified.

mozbuild.mozconfig module

exception mozbuild.mozconfig.MozconfigFindException

Bases: exceptions.Exception

Raised when a mozconfig location is not defined properly.

exception mozbuild.mozconfig.MozconfigLoadException(path, message, output=None)

Bases: exceptions.Exception

Raised when a mozconfig could not be loaded properly.

This typically indicates a malformed or misbehaving mozconfig file.

class mozbuild.mozconfig.MozconfigLoader(topsrcdir)

Bases: object

Handles loading and parsing of mozconfig files.

AUTODETECT = <object object>
DEFAULT_TOPSRCDIR_PATHS = (u'.mozconfig', u'mozconfig')
DEPRECATED_HOME_PATHS = (u'.mozconfig', u'.mozconfig.sh', u'.mozmyconfig.sh')
DEPRECATED_TOPSRCDIR_PATHS = (u'mozconfig.sh', u'myconfig.sh')
ENVIRONMENT_VARIABLES = set([u'CFLAGS', u'CC', u'CXXFLAGS', u'CXX', u'LDFLAGS', u'MOZ_OBJDIR'])
IGNORE_SHELL_VARIABLES = set([u'_'])
RE_MAKE_VARIABLE = <_sre.SRE_Pattern object>
find_mozconfig(env={'LC_NUMERIC': 'en_US.UTF-8', 'LESS': '-R', 'GNOME_DESKTOP_SESSION_ID': 'this-is-deprecated', 'NVM_CD_FLAGS': '-q', 'LC_CTYPE': 'en_US.UTF8', 'XDG_CURRENT_DESKTOP': 'GNOME', 'LC_PAPER': 'en_US.UTF-8', 'LOGNAME': 'zbraniecki', 'USER': 'zbraniecki', 'NVM_DIR': '/home/zbraniecki/.nvm', 'PATH': '/home/zbraniecki/projects/mozilla-unified/obj-x86_64-pc-linux-gnu-opt/docs/html/main/_venv/bin:/home/zbraniecki/projects/mozilla-unified/obj-x86_64-pc-linux-gnu-opt/_virtualenv/bin:/home/zbraniecki/.nvm/versions/node/v9.3.0/bin:/home/zbraniecki/.local/bin:/home/zbraniecki/.cargo/bin:/usr/local/bin:/usr/lib/icecream/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl', 'XDG_VTNR': '2', 'HOME': '/home/zbraniecki', 'HG': '/usr/bin/hg', 'LANG': 'en_US.UTF8', 'MANAGERPID': '19882', 'SHELL': '/bin/zsh', 'JOURNAL_STREAM': '9:1697078', 'SESSION_MANAGER': 'local/cintra:@/tmp/.ICE-unix/19900, unix/cintra:/tmp/.ICE-unix/19900', 'LC_MEASUREMENT': 'en_US.UTF-8', 'WAYLAND_DISPLAY': 'wayland-0', 'DBUS_STARTER_ADDRESS': 'unix:path=/run/user/1000/bus, guid=e223a512427e82f7c5d74fed5a985368', 'DISPLAY': ':0', 'NVM_BIN': '/home/zbraniecki/.nvm/versions/node/v9.3.0/bin', 'EDITOR': 'nvim', 'GREP_COLOR': '1;33', 'INVOCATION_ID': 'dd151c654d1c4cc7a462fb51b021703b', 'USERNAME': 'zbraniecki', 'XDG_SESSION_DESKTOP': 'gnome', 'SHLVL': '1', 'XDG_RUNTIME_DIR': '/run/user/1000', 'MAIL': '/var/spool/mail/zbraniecki', 'SSH_AUTH_SOCK': '/run/user/1000/keyring/ssh', 'TERM': 'xterm-256color', 'VTE_VERSION': '5003', 'GDMSESSION': 'gnome', 'XDG_DATA_DIRS': '/home/zbraniecki/.local/share/flatpak/exports/share/:/var/lib/flatpak/exports/share/:/usr/local/share/:/usr/share/', 'DBUS_STARTER_BUS_TYPE': 'session', 'MACH_STDOUT_ISATTY': '1', 'XDG_SESSION_ID': 'c3', 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus, guid=e223a512427e82f7c5d74fed5a985368', 'DESKTOP_SESSION': 'gnome', 'LSCOLORS': 'exfxcxdxbxegedabagacad', 'MOZ_PLUGIN_PATH': '/usr/lib/mozilla/plugins', 'TILIX_ID': '31997304-8f98-44cc-94f1-e05d90485d9b', 'XDG_SESSION_TYPE': 'wayland', 'OLDPWD': '/home/zbraniecki/projects/mozilla-unified', 'GDM_LANG': 'en_US.UTF8', 'LC_MONETARY': 'en_US.UTF-8', 'PWD': '/home/zbraniecki/projects/mozilla-unified', 'COLORTERM': 'truecolor', 'XDG_MENU_PREFIX': 'gnome-', 'LC_TIME': 'en_US.UTF-8', 'LS_COLORS': 'di=34;40:ln=35;40:so=32;40:pi=33;40:ex=31;40:bd=34;46:cd=34;43:su=0;41:sg=0;46:tw=0;42:ow=0;43:', 'PAGER': 'less', 'XDG_SEAT': 'seat0'})

Find the active mozconfig file for the current environment.

This emulates the logic in mozconfig-find.

  1. If ENV[MOZCONFIG] is set, use that
  2. If $TOPSRCDIR/mozconfig or $TOPSRCDIR/.mozconfig exists, use it.
  3. If both exist or if there are legacy locations detected, error out.

The absolute path to the found mozconfig will be returned on success. None will be returned if no mozconfig could be found. A MozconfigFindException will be raised if there is a bad state, including conditions from #3 above.

read_mozconfig(path=None)

Read the contents of a mozconfig into a data structure.

This takes the path to a mozconfig to load. If the given path is AUTODETECT, will try to find a mozconfig from the environment using find_mozconfig().

mozconfig files are shell scripts. So, we can’t just parse them. Instead, we run the shell script in a wrapper which allows us to record state from execution. Thus, the output from a mozconfig is a friendly static data structure.

mozbuild.mozinfo module

mozbuild.mozinfo.build_dict(config, env={'LC_NUMERIC': 'en_US.UTF-8', 'LESS': '-R', 'GNOME_DESKTOP_SESSION_ID': 'this-is-deprecated', 'NVM_CD_FLAGS': '-q', 'LC_CTYPE': 'en_US.UTF8', 'XDG_CURRENT_DESKTOP': 'GNOME', 'LC_PAPER': 'en_US.UTF-8', 'LOGNAME': 'zbraniecki', 'USER': 'zbraniecki', 'NVM_DIR': '/home/zbraniecki/.nvm', 'PATH': '/home/zbraniecki/projects/mozilla-unified/obj-x86_64-pc-linux-gnu-opt/docs/html/main/_venv/bin:/home/zbraniecki/projects/mozilla-unified/obj-x86_64-pc-linux-gnu-opt/_virtualenv/bin:/home/zbraniecki/.nvm/versions/node/v9.3.0/bin:/home/zbraniecki/.local/bin:/home/zbraniecki/.cargo/bin:/usr/local/bin:/usr/lib/icecream/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl', 'XDG_VTNR': '2', 'HOME': '/home/zbraniecki', 'HG': '/usr/bin/hg', 'LANG': 'en_US.UTF8', 'MANAGERPID': '19882', 'SHELL': '/bin/zsh', 'JOURNAL_STREAM': '9:1697078', 'SESSION_MANAGER': 'local/cintra:@/tmp/.ICE-unix/19900, unix/cintra:/tmp/.ICE-unix/19900', 'LC_MEASUREMENT': 'en_US.UTF-8', 'WAYLAND_DISPLAY': 'wayland-0', 'DBUS_STARTER_ADDRESS': 'unix:path=/run/user/1000/bus, guid=e223a512427e82f7c5d74fed5a985368', 'DISPLAY': ':0', 'NVM_BIN': '/home/zbraniecki/.nvm/versions/node/v9.3.0/bin', 'EDITOR': 'nvim', 'GREP_COLOR': '1;33', 'INVOCATION_ID': 'dd151c654d1c4cc7a462fb51b021703b', 'USERNAME': 'zbraniecki', 'XDG_SESSION_DESKTOP': 'gnome', 'SHLVL': '1', 'XDG_RUNTIME_DIR': '/run/user/1000', 'MAIL': '/var/spool/mail/zbraniecki', 'SSH_AUTH_SOCK': '/run/user/1000/keyring/ssh', 'TERM': 'xterm-256color', 'VTE_VERSION': '5003', 'GDMSESSION': 'gnome', 'XDG_DATA_DIRS': '/home/zbraniecki/.local/share/flatpak/exports/share/:/var/lib/flatpak/exports/share/:/usr/local/share/:/usr/share/', 'DBUS_STARTER_BUS_TYPE': 'session', 'MACH_STDOUT_ISATTY': '1', 'XDG_SESSION_ID': 'c3', 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus, guid=e223a512427e82f7c5d74fed5a985368', 'DESKTOP_SESSION': 'gnome', 'LSCOLORS': 'exfxcxdxbxegedabagacad', 'MOZ_PLUGIN_PATH': '/usr/lib/mozilla/plugins', 'TILIX_ID': '31997304-8f98-44cc-94f1-e05d90485d9b', 'XDG_SESSION_TYPE': 'wayland', 'OLDPWD': '/home/zbraniecki/projects/mozilla-unified', 'GDM_LANG': 'en_US.UTF8', 'LC_MONETARY': 'en_US.UTF-8', 'PWD': '/home/zbraniecki/projects/mozilla-unified', 'COLORTERM': 'truecolor', 'XDG_MENU_PREFIX': 'gnome-', 'LC_TIME': 'en_US.UTF-8', 'LS_COLORS': 'di=34;40:ln=35;40:so=32;40:pi=33;40:ex=31;40:bd=34;46:cd=34;43:su=0;41:sg=0;46:tw=0;42:ow=0;43:', 'PAGER': 'less', 'XDG_SEAT': 'seat0'})

Build a dict containing data about the build configuration from the environment.

mozbuild.mozinfo.write_mozinfo(file, config, env={'LC_NUMERIC': 'en_US.UTF-8', 'LESS': '-R', 'GNOME_DESKTOP_SESSION_ID': 'this-is-deprecated', 'NVM_CD_FLAGS': '-q', 'LC_CTYPE': 'en_US.UTF8', 'XDG_CURRENT_DESKTOP': 'GNOME', 'LC_PAPER': 'en_US.UTF-8', 'LOGNAME': 'zbraniecki', 'USER': 'zbraniecki', 'NVM_DIR': '/home/zbraniecki/.nvm', 'PATH': '/home/zbraniecki/projects/mozilla-unified/obj-x86_64-pc-linux-gnu-opt/docs/html/main/_venv/bin:/home/zbraniecki/projects/mozilla-unified/obj-x86_64-pc-linux-gnu-opt/_virtualenv/bin:/home/zbraniecki/.nvm/versions/node/v9.3.0/bin:/home/zbraniecki/.local/bin:/home/zbraniecki/.cargo/bin:/usr/local/bin:/usr/lib/icecream/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl', 'XDG_VTNR': '2', 'HOME': '/home/zbraniecki', 'HG': '/usr/bin/hg', 'LANG': 'en_US.UTF8', 'MANAGERPID': '19882', 'SHELL': '/bin/zsh', 'JOURNAL_STREAM': '9:1697078', 'SESSION_MANAGER': 'local/cintra:@/tmp/.ICE-unix/19900, unix/cintra:/tmp/.ICE-unix/19900', 'LC_MEASUREMENT': 'en_US.UTF-8', 'WAYLAND_DISPLAY': 'wayland-0', 'DBUS_STARTER_ADDRESS': 'unix:path=/run/user/1000/bus, guid=e223a512427e82f7c5d74fed5a985368', 'DISPLAY': ':0', 'NVM_BIN': '/home/zbraniecki/.nvm/versions/node/v9.3.0/bin', 'EDITOR': 'nvim', 'GREP_COLOR': '1;33', 'INVOCATION_ID': 'dd151c654d1c4cc7a462fb51b021703b', 'USERNAME': 'zbraniecki', 'XDG_SESSION_DESKTOP': 'gnome', 'SHLVL': '1', 'XDG_RUNTIME_DIR': '/run/user/1000', 'MAIL': '/var/spool/mail/zbraniecki', 'SSH_AUTH_SOCK': '/run/user/1000/keyring/ssh', 'TERM': 'xterm-256color', 'VTE_VERSION': '5003', 'GDMSESSION': 'gnome', 'XDG_DATA_DIRS': '/home/zbraniecki/.local/share/flatpak/exports/share/:/var/lib/flatpak/exports/share/:/usr/local/share/:/usr/share/', 'DBUS_STARTER_BUS_TYPE': 'session', 'MACH_STDOUT_ISATTY': '1', 'XDG_SESSION_ID': 'c3', 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus, guid=e223a512427e82f7c5d74fed5a985368', 'DESKTOP_SESSION': 'gnome', 'LSCOLORS': 'exfxcxdxbxegedabagacad', 'MOZ_PLUGIN_PATH': '/usr/lib/mozilla/plugins', 'TILIX_ID': '31997304-8f98-44cc-94f1-e05d90485d9b', 'XDG_SESSION_TYPE': 'wayland', 'OLDPWD': '/home/zbraniecki/projects/mozilla-unified', 'GDM_LANG': 'en_US.UTF8', 'LC_MONETARY': 'en_US.UTF-8', 'PWD': '/home/zbraniecki/projects/mozilla-unified', 'COLORTERM': 'truecolor', 'XDG_MENU_PREFIX': 'gnome-', 'LC_TIME': 'en_US.UTF-8', 'LS_COLORS': 'di=34;40:ln=35;40:so=32;40:pi=33;40:ex=31;40:bd=34;46:cd=34;43:su=0;41:sg=0;46:tw=0;42:ow=0;43:', 'PAGER': 'less', 'XDG_SEAT': 'seat0'})

Write JSON data about the configuration specified in config and an environment variable dict to |file|, which may be a filename or file-like object. See build_dict for information about what environment variables are used, and what keys are produced.

mozbuild.preprocessor module

This is a very primitive line based preprocessor, for times when using a C preprocessor isn’t an option.

It currently supports the following grammar for expressions, whitespace is ignored:

expression :
and_cond ( ‘||’ expression ) ? ;
and_cond:
test ( ‘&&’ and_cond ) ? ;
test:
unary ( ( ‘==’ | ‘!=’ ) unary ) ? ;
unary :
‘!’? value ;
value :
[0-9]+ # integer | ‘defined(‘ w+ ‘)’ | w+ # string identifier or value;
class mozbuild.preprocessor.Context

Bases: dict

This class holds variable values by subclassing dict, and while it truthfully reports True and False on

name in context

it returns the variable name itself on

context[“name”]

to reflect the ambiguity between string literals and preprocessor variables.

class mozbuild.preprocessor.Expression(expression_string)
exception ParseError(expression)

Bases: exceptions.StandardError

Error raised when parsing fails. It has two members, offset and content, which give the offset of the error and the offending content.

evaluate(context)

Evaluate the expression with the given context

class mozbuild.preprocessor.Preprocessor(defines=None, marker='#')

Class for preprocessing text files.

exception Error(cpp, MSG, context)

Bases: exceptions.RuntimeError

addDefines(defines)

Adds the specified defines to the preprocessor. defines may be a dictionary object or an iterable of key/value pairs (as tuples or other iterables of length two)

applyFilters(aLine)
clone()

Create a clone of the current processor, including line ending settings, marker, variable definitions, output stream.

computeDependencies(input)

Reads the input stream, and computes the dependencies for that input.

do_define(args)
do_elif(args)
do_elifdef(args)
do_elifndef(args)
do_else(args, ifState=2)
do_endif(args)
do_error(args)
do_expand(args)
do_filter(args)
do_if(args, replace=False)
do_ifdef(args, replace=False)
do_ifndef(args, replace=False)
do_include(args, filters=True)

Preprocess a given file. args can either be a file name, or a file-like object. Files should be opened, and will be closed after processing.

do_includesubst(args)
do_literal(args)
do_undef(args)
do_unfilter(args)
ensure_not_else()
failUnused(file)
filter_attemptSubstitution(aLine)
filter_emptyLines(aLine)
filter_slashslash(aLine)
filter_spaces(aLine)
filter_substitution(aLine, fatal=True)
getCommandLineParser(unescapeDefines=False)
handleCommandLine(args, defaultToStdin=False)

Parse a commandline into this parser. Uses OptionParser internally, no args mean sys.argv[1:].

handleLine(aLine)

Handle a single line of input (internal).

noteLineInfo()
processFile(input, output, depfile=None)

Preprocesses the contents of the input stream and writes the result to the output stream. If depfile is set, the dependencies of output file are written to depfile in Makefile format.

setMarker(aMarker)

Set the marker to be used for processing directives. Used for handling CSS files, with pp.setMarker(‘%’), for example. The given marker may be None, in which case no markers are processed.

setSilenceDirectiveWarnings(value)

Sets whether missing directive warnings are silenced, according to value. The default behavior of the preprocessor is to emit such warnings.

write(aLine)

Internal method for handling output.

mozbuild.preprocessor.preprocess(includes=[<open file '<stdin>', mode 'r'>], defines={}, output=<open file '<stdout>', mode 'w'>, marker='#')

mozbuild.pythonutil module

mozbuild.pythonutil.find_python3_executable(min_version='3.5.0')

Find a Python 3 executable.

Returns a tuple containing the the path to an executable binary and a version tuple. Both tuple entries will be None if a Python executable could not be resolved.

mozbuild.pythonutil.iter_modules_in_path(*paths)
mozbuild.pythonutil.python_executable_version(exe)

Determine the version of a Python executable by invoking it.

May raise subprocess.CalledProcessError or ValueError on failure.

mozbuild.schedules module

Constants for SCHEDULES configuration in moz.build files and for skip-unless-schedules optimizations in task-graph generation.

mozbuild.shellutil module

exception mozbuild.shellutil.MetaCharacterException(char)

Bases: exceptions.Exception

mozbuild.shellutil.split(cline)

Split the given command line string.

mozbuild.shellutil.quote(*strings)

Given one or more strings, returns a quoted string that can be used literally on a shell command line.

>>> quote('a', 'b')
"a b"
>>> quote('a b', 'c')
"'a b' c"

mozbuild.sphinx module

class mozbuild.sphinx.MozbuildSymbols(name, arguments, options, content, lineno, content_offset, block_text, state, state_machine)

Bases: docutils.parsers.rst.Directive

Directive to insert mozbuild sandbox symbol information.

required_arguments = 1
run()
mozbuild.sphinx.format_module(m)
mozbuild.sphinx.function_reference(f, attr, args, doc)
mozbuild.sphinx.setup(app)
mozbuild.sphinx.special_reference(v, func, typ, doc)
mozbuild.sphinx.variable_reference(v, st_type, in_type, doc)

mozbuild.testing module

class mozbuild.testing.SupportFilesConverter

Bases: object

Processes a “support-files” entry from a test object, either from a parsed object from a test manifests or its representation in moz.build and returns the installs to perform for this test object.

Processing the same support files multiple times will not have any further effect, and the structure of the parsed objects from manifests will have a lot of repeated entries, so this class takes care of memoizing.

convert_support_files(test, install_root, manifest_dir, out_dir)
class mozbuild.testing.TestInstallInfo

Bases: object

mozbuild.testing.all_test_flavors()
mozbuild.testing.install_test_files(topsrcdir, topobjdir, tests_root, test_objs)

Installs the requested test files to the objdir. This is invoked by test runners to avoid installing tens of thousands of test files when only a few tests need to be run.

mozbuild.testing.read_manifestparser_manifest(context, manifest_path)
mozbuild.testing.read_reftest_manifest(context, manifest_path)
mozbuild.testing.read_wpt_manifest(context, paths)

mozbuild.util module

class mozbuild.util.DefinesAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)

Bases: argparse.Action

An ArgumentParser action to handle -Dvar[=value] type of arguments.

class mozbuild.util.EmptyValue

Bases: unicode

A dummy type that behaves like an empty string and sequence.

This type exists in order to support mozbuild.frontend.reader.EmptyConfig. It should likely not be used elsewhere.

class mozbuild.util.EnumString(value)

Bases: unicode

A string type that only can have a limited set of values, similarly to an Enum, and can only be compared against that set of values.

The class is meant to be subclassed, where the subclass defines POSSIBLE_VALUES. The subclass method is a helper to create such subclasses.

POSSIBLE_VALUES = ()
static subclass(*possible_values)
exception mozbuild.util.EnumStringComparisonError

Bases: exceptions.Exception

class mozbuild.util.FileAvoidWrite(filename, capture_diff=False, dry_run=False, mode=u'rU')

Bases: _io.BytesIO

File-like object that buffers output and only writes if content changed.

We create an instance from an existing filename. New content is written to it. When we close the file object, if the content in the in-memory buffer differs from what is on disk, then we write out the new content. Otherwise, the original file is untouched.

Instances can optionally capture diffs of file changes. This feature is not enabled by default because it a) doesn’t make sense for binary files b) could add unwanted overhead to calls.

Additionally, there is dry run mode where the file is not actually written out, but reports whether the file was existing and would have been updated still occur, as well as diff capture if requested.

close()

Stop accepting writes, compare file contents, and rewrite if needed.

Returns a tuple of bools indicating what action was performed:

(file existed, file updated)

If capture_diff was specified at construction time and the underlying file was changed, .diff will be populated with the diff of the result.

write(buf)
mozbuild.util.FlagsFactory(flags)

Returns a class which holds optional flags for an item in a list.

The flags are defined in the dict given as argument, where keys are the flag names, and values the type used for the value of that flag.

The resulting class is used by the various <TypeName>WithFlagsFactory functions below.

class mozbuild.util.HierarchicalStringList

Bases: object

A hierarchy of lists of strings.

Each instance of this object contains a list of strings, which can be set or appended to. A sub-level of the hierarchy is also an instance of this class, can be added by appending to an attribute instead.

For example, the moz.build variable EXPORTS is an instance of this class. We can do:

EXPORTS += [‘foo.h’] EXPORTS.mozilla.dom += [‘bar.h’]

In this case, we have 3 instances (EXPORTS, EXPORTS.mozilla, and EXPORTS.mozilla.dom), and the first and last each have one element in their list.

class StringListAdaptor(hsl)

Bases: _abcoll.Sequence

walk()

Walk over all HierarchicalStringLists in the hierarchy.

This is a generator of (path, sequence).

The path is ‘’ for the root level and ‘/’-delimited strings for any descendants. The sequence is a read-only sequence of the strings contained at that level.

class mozbuild.util.ImmutableStrictOrderingOnAppendList(iterable=None, **kwargs)

Bases: mozbuild.util.StrictOrderingOnAppendList

Like StrictOrderingOnAppendList, but not allowing mutations of the value.

append(elt)
extend(iterable)
class mozbuild.util.KeyedDefaultDict(default_factory, *args, **kwargs)

Bases: dict

Like a defaultdict, but the default_factory function takes the key as argument

class mozbuild.util.List(iterable=None, **kwargs)

Bases: mozbuild.util.ListMixin, list

A list specialized for moz.build environments.

We overload the assignment and append operations to require that the appended thing is a list. This avoids bad surprises coming from appending a string to a list, which would just add each letter of the string.

class mozbuild.util.ListMixin(iterable=None, **kwargs)

Bases: object

extend(l)
class mozbuild.util.ListWithAction(iterable=None, **kwargs)

Bases: mozbuild.util.ListMixin, mozbuild.util.ListWithActionMixin, list

A list that accepts a callable to be applied to each item.

A callable (action) may optionally be passed to the constructor to run on each item of input. The result of calling the callable on each item will be stored in place of the original input.

class mozbuild.util.ListWithActionMixin(iterable=None, action=None)

Bases: object

Mixin to create lists with pre-processing. See ListWithAction.

extend(l)
class mozbuild.util.LockFile(lockfile)

Bases: object

LockFile is used by the lock_file method to hold the lock.

This object should not be used directly, but only through the lock_file method below.

exception mozbuild.util.MozbuildDeletionError

Bases: exceptions.Exception

class mozbuild.util.OrderedDefaultDict(default_factory, *args, **kwargs)

Bases: collections.OrderedDict

A combination of OrderedDict and defaultdict.

class mozbuild.util.ReadOnlyDefaultDict(default_factory, *args, **kwargs)

Bases: mozbuild.util.ReadOnlyDict

A read-only dictionary that supports default values on retrieval.

class mozbuild.util.ReadOnlyDict(*args, **kwargs)

Bases: dict

A read-only dictionary.

update(*args, **kwargs)
class mozbuild.util.ReadOnlyKeyedDefaultDict(default_factory, *args, **kwargs)

Bases: mozbuild.util.KeyedDefaultDict, mozbuild.util.ReadOnlyDict

Like KeyedDefaultDict, but read-only.

class mozbuild.util.ReadOnlyNamespace(**kwargs)

Bases: object

A class for objects with immutable attributes set at initialization.

class mozbuild.util.StrictOrderingOnAppendList(iterable=None, **kwargs)

Bases: mozbuild.util.ListMixin, mozbuild.util.StrictOrderingOnAppendListMixin, list

A list specialized for moz.build environments.

We overload the assignment and append operations to require that incoming elements be ordered. This enforces cleaner style in moz.build files.

class mozbuild.util.StrictOrderingOnAppendListMixin(iterable=None, **kwargs)

Bases: object

static ensure_sorted(l)
extend(l)
class mozbuild.util.StrictOrderingOnAppendListWithAction(iterable=None, **kwargs)

Bases: mozbuild.util.StrictOrderingOnAppendListMixin, mozbuild.util.ListMixin, mozbuild.util.ListWithActionMixin, list

An ordered list that accepts a callable to be applied to each item.

A callable (action) passed to the constructor is run on each item of input. The result of running the callable on each item will be stored in place of the original input, but the original item must be used to enforce sortedness. Note that the order of superclasses is therefore significant.

class mozbuild.util.StrictOrderingOnAppendListWithFlags(iterable=None, **kwargs)

Bases: mozbuild.util.StrictOrderingOnAppendList

A list with flags specialized for moz.build environments.

Each subclass has a set of typed flags; this class lets us use isinstance for natural testing.

mozbuild.util.StrictOrderingOnAppendListWithFlagsFactory(flags)

Returns a StrictOrderingOnAppendList-like object, with optional flags on each item.

The flags are defined in the dict given as argument, where keys are the flag names, and values the type used for the value of that flag.

Example

FooList = StrictOrderingOnAppendListWithFlagsFactory({
‘foo’: bool, ‘bar’: unicode

}) foo = FooList([‘a’, ‘b’, ‘c’]) foo[‘a’].foo = True foo[‘b’].bar = ‘bar’

mozbuild.util.TypedList

A list with type coercion.

The given type is what list elements are being coerced to. It may do strict validation, throwing ValueError exceptions.

A base_class type can be given for more specific uses than a List. For example, a Typed StrictOrderingOnAppendList can be created with:

TypedList(unicode, StrictOrderingOnAppendList)
class mozbuild.util.TypedListMixin(iterable=None, **kwargs)

Bases: object

Mixin for a list with type coercion. See TypedList.

append(other)
extend(l)
mozbuild.util.TypedNamedTuple(name, fields)

Factory for named tuple types with strong typing.

Arguments are an iterable of 2-tuples. The first member is the the field name. The second member is a type the field will be validated to be.

Construction of instances varies from collections.namedtuple.

First, if a single tuple argument is given to the constructor, this is treated as the equivalent of passing each tuple value as a separate argument into __init__. e.g.:

t = (1, 2)
TypedTuple(t) == TypedTuple(1, 2)

This behavior is meant for moz.build files, so vanilla tuples are automatically cast to typed tuple instances.

Second, fields in the tuple are validated to be instances of the specified type. This is done via an isinstance() check. To allow multiple types, pass a tuple as the allowed types field.

exception mozbuild.util.UnsortedError(srtd, original)

Bases: exceptions.Exception

mozbuild.util.encode(obj, encoding=u'utf-8')

Recursively encode unicode strings with the given encoding.

mozbuild.util.ensureParentDir(path)

Ensures the directory parent to the given file exists.

mozbuild.util.exec_(object, globals=None, locals=None)

Wrapper around the exec statement to avoid bogus errors like:

SyntaxError: unqualified exec is not allowed in function … it is a nested function.

or

SyntaxError: unqualified exec is not allowed in function … it contains a nested function with free variable

which happen with older versions of python 2.7.

mozbuild.util.expand_variables(s, variables)

Given a string with $(var) variable references, replace those references with the corresponding entries from the given variables dict.

If a variable value is not a string, it is iterated and its items are joined with a whitespace.

mozbuild.util.group_unified_files(files, unified_prefix, unified_suffix, files_per_unified_file)

Return an iterator of (unified_filename, source_filenames) tuples.

We compile most C and C++ files in “unified mode”; instead of compiling a.cpp, b.cpp, and c.cpp separately, we compile a single file that looks approximately like:

#include "a.cpp"
#include "b.cpp"
#include "c.cpp"

This function handles the details of generating names for the unified files, and determining which original source files go in which unified file.

mozbuild.util.hash_file(path, hasher=None)

Hashes a file specified by the path given and returns the hex digest.

mozbuild.util.indented_repr(o, indent=4)

Similar to repr(), but returns an indented representation of the object

One notable difference with repr is that the returned representation assumes from __future__ import unicode_literals.

mozbuild.util.lock_file(lockfile, max_wait=600)

Create and hold a lockfile of the given name, with the given timeout.

To release the lock, delete the returned object.

class mozbuild.util.memoize(func)

Bases: dict

A decorator to memoize the results of function calls depending on its arguments. Both functions and instance methods are handled, although in the instance method case, the results are cache in the instance itself.

method_call(instance, *args)
class mozbuild.util.memoized_property(func)

Bases: object

A specialized version of the memoize decorator that works for class instance properties.

mozbuild.util.mkdir(path, not_indexed=False)

Ensure a directory exists.

If not_indexed is True, an attribute is set that disables content indexing on the directory.

mozbuild.util.pair(iterable)

Given an iterable, returns an iterable pairing its items.

For example,
list(pair([1,2,3,4,5,6]))
returns
[(1,2), (3,4), (5,6)]
mozbuild.util.patch_main()

This is a hack to work around the fact that Windows multiprocessing needs to import the original main module, and assumes that it corresponds to a file ending in .py.

We do this by a sort of two-level function interposing. The first level interposes forking.get_command_line() with our version defined in my_get_command_line(). Our version of get_command_line will replace the command string with the contents of the fork_interpose() function to be used in the subprocess.

The subprocess then gets an interposed imp.find_module(), which we hack up to find the main module name multiprocessing will assume, since we know what this will be based on the main module in the parent. If we’re not looking for our main module, then the original find_module will suffice.

See also: http://bugs.python.org/issue19946 And: https://bugzilla.mozilla.org/show_bug.cgi?id=914563

mozbuild.util.resolve_target_to_make(topobjdir, target)

Resolve target (a target, directory, or file) to a make target.

topobjdir is the object directory; all make targets will be rooted at or below the top-level Makefile in this directory.

Returns a pair (reldir, target) where reldir is a directory relative to topobjdir containing a Makefile and target is a make target (possibly None).

A directory resolves to the nearest directory at or above containing a Makefile, and target None.

A regular (non-Makefile) file resolves to the nearest directory at or above the file containing a Makefile, and an appropriate target.

A Makefile resolves to the nearest parent strictly above the Makefile containing a different Makefile, and an appropriate target.

mozbuild.util.simple_diff(filename, old_lines, new_lines)

Returns the diff between old_lines and new_lines, in unified diff form, as a list of lines.

old_lines and new_lines are lists of non-newline terminated lines to compare. old_lines can be None, indicating a file creation. new_lines can be None, indicating a file deletion.

class mozbuild.util.undefined_default

Bases: object

Represents an undefined argument value that isn’t None.

mozbuild.vendor_aom module

class mozbuild.vendor_aom.VendorAOM(topsrcdir, settings, log_manager, topobjdir=None, mozconfig=<object object>)

Bases: mozbuild.base.MozbuildObject

check_modified_files()

Ensure that there aren’t any uncommitted changes to files in the working copy, since we’re going to change some state on the user.

clean_upstream(target)

Remove files we don’t want to import.

fetch_and_unpack(revision, target)

Fetch and unpack upstream source

generate_sources(target)

Run the library’s native build system to update ours.

Invoke configure for each supported platform to generate appropriate config and header files, then invoke the makefile to obtain a list of source files, writing these out in the appropriate format for our build system to use.

update_mimetype(revision)

Update source tree references to the aom revision.

While the av1 bitstream is unstable, we track the git revision of the reference implementation we’re building, and are careful to build with default feature flags. This lets us answer whether a particular bitstream will be playable by comparing the encode- side hash as part of the mime type.

update_readme(revision, timestamp, target)
upstream_commit(revision)

Convert a revision to a git commit and timestamp.

Ask the upstream repo to convert the requested revision to a git commit id and timestamp, so we can be precise in what we’re vendoring.

upstream_github_commit(revision)

Query the github api for a git commit id and timestamp.

upstream_googlesource_commit(revision)

Query gitiles for a git commit and timestamp.

upstream_snapshot(revision)

Construct a url for a tarball snapshot of the given revision.

upstream_validate(url)

Validate repository urls to make sure we can handle them.

vendor(revision, repo, ignore_modified=False)

mozbuild.vendor_rust module

class mozbuild.vendor_rust.VendorRust(topsrcdir, settings, log_manager, topobjdir=None, mozconfig=<object object>)

Bases: mozbuild.base.MozbuildObject

check_cargo_vendor_version(cargo)

Ensure that cargo-vendor is new enough. cargo-vendor 0.1.13 and newer strips out .cargo-ok, .orig and .rej files, and deals with [patch] replacements in Cargo.toml files which we want.

check_cargo_version(cargo)

Ensure that cargo is new enough. cargo 0.12 added support for source replacement, which is required for vendoring to work.

check_modified_files()

Ensure that there aren’t any uncommitted changes to files in the working copy, since we’re going to change some state on the user. Allow changes to Cargo.{toml,lock} since that’s likely to be a common use case.

check_openssl()

Set environment flags for building with openssl.

MacOS doesn’t include openssl, but the openssl-sys crate used by mach-vendor expects one of the system. It’s common to have one installed in /usr/local/opt/openssl by homebrew, but custom link flags are necessary to build against it.

get_cargo_path()
vendor(ignore_modified=False, build_peers_said_large_imports_were_ok=False)

mozbuild.virtualenv module

class mozbuild.virtualenv.VirtualenvManager(topsrcdir, topobjdir, virtualenv_path, log_handle, manifest_path)

Bases: object

Contains logic for managing virtualenvs for building the tree.

activate()

Activate the virtualenv in this Python context.

If you run a random Python script and wish to “activate” the virtualenv, you can simply instantiate an instance of this class and call .ensure() and .activate() to make the virtualenv active.

activate_path
bin_path
build(python='/usr/bin/python2.7')

Build a virtualenv per tree conventions.

This returns the path of the created virtualenv.

call_setup(directory, arguments)

Calls setup.py in a directory.

create(python='/usr/bin/python2.7')

Create a new, empty virtualenv.

Receives the path to virtualenv’s virtualenv.py script (which will be called out to), the path to create the virtualenv in, and a handle to write output to.

ensure(python='/usr/bin/python2.7')

Ensure the virtualenv is present and up to date.

If the virtualenv is up to date, this does nothing. Otherwise, it creates and populates the virtualenv as necessary.

This should be the main API used from this class as it is the highest-level.

get_exe_info()

Returns the version and file size of the python executable that was in use when this virutalenv was created.

install_pip_package(package)

Install a package via pip.

The supplied package is specified using a pip requirement specifier. e.g. ‘foo’ or ‘foo==1.0’.

If the package is already installed, this is a no-op.

install_pip_requirements(path, require_hashes=True)

Install a pip requirements.txt file.

The supplied path is a text file containing pip requirement specifiers.

If require_hashes is True, each specifier must contain the expected hash of the downloaded package. See: https://pip.pypa.io/en/stable/reference/pip_install/#hash-checking-mode

packages()
populate()

Populate the virtualenv.

The manifest file consists of colon-delimited fields. The first field specifies the action. The remaining fields are arguments to that action. The following actions are supported:

setup.py – Invoke setup.py for a package. Expects the arguments:
  1. relative path directory containing setup.py.
  2. argument(s) to setup.py. e.g. “develop”. Each program argument is delimited by a colon. Arguments with colons are not yet supported.
filename.pth – Adds the path given as argument to filename.pth under
the virtualenv site packages directory.
optional – This denotes the action as optional. The requested action
is attempted. If it fails, we issue a warning and go on. The initial “optional” field is stripped then the remaining line is processed like normal. e.g. “optional:setup.py:python/foo:built_ext:-i”
copy – Copies the given file in the virtualenv site packages
directory.
packages.txt – Denotes that the specified path is a child manifest. It
will be read and processed as if its contents were concatenated into the manifest being read.
objdir – Denotes a relative path in the object directory to add to the
search path. e.g. “objdir:build” will add $topobjdir/build to the search path.

Note that the Python interpreter running this function should be the one from the virtualenv. If it is the system Python or if the environment is not configured properly, packages could be installed into the wrong place. This is how virtualenv’s work.

python_path
up_to_date(python='/usr/bin/python2.7')

Returns whether the virtualenv is present and up to date.

virtualenv_script_path

Path to virtualenv’s own populator script.

write_exe_info(python)

Records the the version of the python executable that was in use when this virutalenv was created. We record this explicitly because on OS X our python path may end up being a different or modified executable.

mozbuild.virtualenv.verify_python_version(log_handle)

Ensure the current version of Python is sufficient.

Module contents