msml.run

This class summaries functions for executing the pipeline.

Functions

build_graph(tasks, exporter, variables)	build the direct acyclic graph from the given arguments.
contains(a, b)
create_conversion_task(slotA, slotB)	creates a task (instance of operator) for the requested conversion.
get_executor(name)
get_known_executors()
get_python_conversion_operator(slotA, slotB)	creates an msml.model.PythonOperator for conversion
initialize_file_literals(first_bucket)
inject_implict_conversion(dag)	Finds type mismatches on edges and injects suitable conversion operators

Classes

AbstractExecutor(msmlfile)
DefaultGraphBuilder(msmlfile, exporter)	Builds the DAG for the given msmlfile and exporter
Executor(msmlfile)	Describes the interface of an Executer.
ExecutorsHelper	static methods needed by some executors
LinearSequenceExecutor(msmlfile)	The LinearSequenceExecuter executes the given MSMLFile in one sequence with no parallelism in topological order.
ParallelExecutor(msmlfile)	The ParallelExecutor makes everything faster, by burning your CPU to a new heat level.
PhaseExecutor(msmlfile)	PhaseExecutor works similar to LinearSequenceExecutor, but provides more control over the phases of pre-, postprocessing, render and execution of the exporter.

Exceptions

class msml.run.Executor(msmlfile)

Bases: object

Describes the interface of an Executer.

An Executer is responsible for calling the operator with the right arguments and parameters in the right order. Additionally it invokes the msml.exporter.Exporter.

init_memory(content)

options

run()

class msml.run.Memory(predefine_variables={})

Bases: object

The memory encapsulate dict helpers.

Parameters:predefine_variables (dict[str, object]) – a dict of predefined values for the internal memory

is_compatible(name)

load_memory_file(filename)

load the given file into the internal dict.

The given filename is executed and the defined variables will be set into the internal memory.

Parameters:filename – a path to a python (executable) file

:type filename:” str

lookup(reference)

lookup a reference

A refernce consists of task and slot name. This method handles:

Str:e.g. “${taskA.slotB}” via msml.model.parse_attribute_value() MSMLVariable: Reference: Parameters:reference (str or msml.model.Reference) – a reference to task and slot

reset()

empties the internal memory

show_content()

pretty print the internal dict to console

update(other)

msml.run.build_graph(tasks, exporter, variables)

build the direct acyclic graph from the given arguments.

Parameters:

• tasks (list[Task]) – a list of msml.model.Task
• exporter (Exporter) – the msml.exporter.Exporter to be weaved int
• variables (list[MSMLVariable]) – MSMLVariable

Returns:

a DAG for the execution

Return type:

msml.model.DiGraph

Warning

The graph building does not validate the dependencies or anything else. You have to do this before or after you used the function. E.g. :py:method:`msml.model.MSMLFile.validate`

msml.run.create_conversion_task(slotA, slotB)

creates a task (instance of operator) for the requested conversion.

Parameters:

• slotA (Reference.Ref) –
• slotB (Reference.Ref) –

Returns:

a task, ready for embedding into the build graph

Return type:

msml.model.Task

msml.run.get_python_conversion_operator(slotA, slotB)

creates an msml.model.PythonOperator for conversion from sort of slotA to sort of slotB

Parameters:

• slotA (msml.model.Reference.Ref) – slot on the outgoing side
• slotB (msml.model.Reference.Ref) – slot on the incoming side

Returns:

an callable conversion operator or none if types incompatible

Return type:

msml.model.PythonOperator

See also

msml.sorts.ConversionNetwork

msml.run.initialize_file_literals(first_bucket)

msml.run.inject_implict_conversion(dag)

Finds type mismatches on edges and injects suitable conversion operators

Warning

This function works and changes the given dag.

Parameters:dag (msml.model.DiGraph) – a directed acyclic graph from msml.run.build_graph() Returns:the modified graph Return type:msml.model.DiGraph

class msml.run.GraphDotWriter(dag)

Bases: object

Export the given dag into the graphviz format.

Parameters:dag (msml.model.DiGraph) – directed acyclic graph

class msml.run.DefaultGraphBuilder(msmlfile, exporter)

Bases: object

Builds the DAG for the given msmlfile and exporter

Args:

msmlfile (MSMLFile) exporter (Exporter)

dag

class msml.run.LinearSequenceExecutor(msmlfile)

Bases: msml.run.AbstractExecutor

The LinearSequenceExecuter executes the given MSMLFile in one sequence with no parallelism in topological order.

run()

starts the execution of the given MSMLFile