Backend codege modules for relay.

The Python interface to the Relay reference interpreter.

class tvm.relay.backend.interpreter.Closure

A closure produced by the interpreter.

class tvm.relay.backend.interpreter.ConstructorValue(constructor, fields, types)
class tvm.relay.backend.interpreter.Executor

An abstract interface for executing Relay programs.

evaluate(expr, binds=None)

Evaluate a Relay expression on the executor.

  • expr (tvm.relay.Expr) – The expression to evaluate.
  • binds (Map[tvm.relay.Var, tvm.relay.Expr]) – Additional binding of free variable.

val – The evaluation result.

Return type:

Union[function, Value]

class tvm.relay.backend.interpreter.Interpreter(mod, ctx, target)

Simple interpreter interface.

  • mod (tvm.relay.Module) – The module to support the execution.
  • ctx (tvm.TVMContext) – The runtime context to run the code on.
  • target (tvm.Target) – The target option to build the function.

Optimize an expr.

Parameters:expr (Expr) – The expression to be optimized.
Returns:opt_expr – The optimized expression.
Return type:Expr
class tvm.relay.backend.interpreter.RefValue(value)
class tvm.relay.backend.interpreter.TensorValue(data)

A Tensor value produced by the interpreter.


Convert a Relay TensorValue into a numpy.ndarray.

class tvm.relay.backend.interpreter.TupleValue(*fields)

A tuple value produced by the interpreter.

class tvm.relay.backend.interpreter.Value

Base class of all values.

Backend code generation engine.

class tvm.relay.backend.compile_engine.CCacheKey(source_func, target)

Key in the CompileEngine.

  • source_func (tvm.relay.Function) – The source function.
  • target (tvm.Target) – The target we want to run the function on.
class tvm.relay.backend.compile_engine.CCacheValue

Value in the CompileEngine, including usage statistics.

class tvm.relay.backend.compile_engine.CachedFunc

Low-level tensor function to back a relay primitive function.

class tvm.relay.backend.compile_engine.CompileEngine

CompileEngine to get lowered code.


clear the existing cached functions


Return a string representation of engine dump.

Returns:dump – The dumped string representation
Return type:str

List items in the cache.

Returns:item_list – The list of items.
Return type:List[Tuple[CCacheKey, CCacheValue]]
jit(source_func, target=None)

JIT a source_func to a tvm.Function.

  • source_func (Union[tvm.relay.Function, CCacheKey]) – The source relay function.
  • target (tvm.Target) – The target platform.

cached_func – The result of lowering.

Return type:


lower(source_func, target=None)

Lower a source_func to a CachedFunc.

  • source_func (Union[tvm.relay.Function, CCacheKey]) – The source relay function.
  • target (tvm.Target) – The target platform.

cached_func – The result of lowering.

Return type:



Get the global compile engine.

Returns:engine – The compile engine.
Return type:tvm.relay.backend.CompileEngine

A compiler from a Relay expression to TVM’s graph runtime.

The compiler is built from a few pieces.

First we define a compiler from a single Relay expression to the graph langauge. We require the expression to be a function. The function’s parameters correspond to the placeholder/inputs and model parameters found in the computation graph representation. The body of the function represents the computation graph.

The compiler’s output is a program in the graph language, which is composed of graph langauge is composed of Node, NodeRef, InputNode, OpNode. This “little language” represents programs in TVM’s graph format.

To connect to the graph runtime, we use a printer that converts our graph format into TVM’s JSON format. The resulting string can be loaded by contrib.graph_runtime or any other TVM runtime compatible systems.

class tvm.relay.backend.graph_runtime_codegen.GraphRuntimeCodegen(mod, target)

The compiler from Relay to the TVM runtime system.


Compile a single function into a graph.

Parameters:func (tvm.relay.Expr) – The function to compile.
  • graph_json (str) – The graph json that can be consumed by runtime.
  • lowered_funcs (List[tvm.LoweredFunc] or Dict[str, List[tvm.LoweredFunc]]) – The lowered functions.
  • params (Dict[str, tvm.nd.NDArray]) – Additional constant parameters.