tvm.contrib.graph_runtime

Minimum graph runtime that executes graph containing TVM PackedFunc.

class tvm.contrib.graph_runtime.GraphModule(module)

Wrapper runtime module.

This is a thin wrapper of the underlying TVM module. you can also directly call set_input, run, and get_output of underlying module functions

Parameters:module (Module) – The interal tvm module that holds the actual graph functions.
module

The interal tvm module that holds the actual graph functions.

Type:Module
debug_get_output(node, out)

Run graph upto node and get the output to out

Parameters:
  • node (int / str) – The node index or name
  • out (NDArray) – The output array container
get_input(index, out=None)

Get index-th input to out

Parameters:
  • index (int) – The input index
  • out (NDArray) – The output array container
get_num_outputs()

Get the number of outputs from the graph

Returns:count – The number of outputs.
Return type:int
get_output(index, out=None)

Get index-th output to out

Parameters:
  • index (int) – The input index
  • out (NDArray) – The output array container
load_params(params_bytes)

Load parameters from serialized byte array of parameter dict.

Parameters:params_bytes (bytearray) – The serialized parameter dict.
run(**input_dict)

Run forward execution of the graph

Parameters:input_dict (dict of str to NDArray) – List of input values to be feed to
set_input(key=None, value=None, **params)

Set inputs to the module via kwargs

Parameters:
  • key (int or str) – The input key
  • value (the input value.) – The input key
  • params (dict of str to NDArray) – Additonal arguments
tvm.contrib.graph_runtime.create(graph_json_str, libmod, ctx)

Create a runtime executor module given a graph and module. :param graph_json_str: The graph to be deployed in json format output by nnvm graph.

The graph can only contain one operator(tvm_op) that points to the name of PackedFunc in the libmod.
Parameters:
  • libmod (tvm.Module) – The module of the corresponding function
  • ctx (TVMContext or list of TVMContext) – The context to deploy the module. It can be local or remote when there is only one TVMContext. Otherwise, the first context in the list will be used as this purpose. All context should be given for heterogeneous execution.
Returns:

graph_module – Runtime graph module that can be used to execute the graph.

Return type:

GraphModule

tvm.contrib.graph_runtime.get_device_ctx(libmod, ctx)

Parse and validate all the device context(s). :param libmod: The module of the corresponding function :type libmod: tvm.Module :param ctx: :type ctx: TVMContext or list of TVMContext

Returns:
  • ctx (list of TVMContext)
  • num_rpc_ctx (Number of rpc contexts)
  • device_type_id (List of device type and device id)