tvm.contrib.graph_runtime

Minimum graph runtime that executes graph containing TVM PackedFunc.

class tvm.contrib.graph_runtime.GraphModule(module)

Wrapper runtime module.

This is a thin wrapper of the underlying TVM module. you can also directly call set_input, run, and get_output of underlying module functions

Parameters

module (Module) – The interal tvm module that holds the actual graph functions.

module

The interal tvm module that holds the actual graph functions.

Type

Module

debug_get_output(node, out)

Run graph upto node and get the output to out

Parameters
  • node (int / str) – The node index or name

  • out (NDArray) – The output array container

get_input(index, out=None)

Get index-th input to out

Parameters
  • index (int) – The input index

  • out (NDArray) – The output array container

get_num_outputs()

Get the number of outputs from the graph

Returns

count – The number of outputs.

Return type

int

get_output(index, out=None)

Get index-th output to out

Parameters
  • index (int) – The output index

  • out (NDArray) – The output array container

load_params(params_bytes)

Load parameters from serialized byte array of parameter dict.

Parameters

params_bytes (bytearray) – The serialized parameter dict.

run(**input_dict)

Run forward execution of the graph

Parameters

input_dict (dict of str to NDArray) – List of input values to be feed to

set_input(key=None, value=None, **params)

Set inputs to the module via kwargs

Parameters
  • key (int or str) – The input key

  • value (the input value.) – The input key

  • params (dict of str to NDArray) – Additonal arguments

share_params(other, params_bytes)

Share parameters from pre-existing GraphRuntime instance.

Parameters
  • other (GraphRuntime) – The parent GraphRuntime from which this instance should share it’s parameters.

  • params_bytes (bytearray) – The serialized parameter dict (used only for the parameter names).

tvm.contrib.graph_runtime.create(graph_json_str, libmod, ctx)

Create a runtime executor module given a graph and module. :param graph_json_str: The graph to be deployed in json format output by nnvm graph.

The graph can only contain one operator(tvm_op) that points to the name of PackedFunc in the libmod.

Parameters
  • libmod (tvm.Module) – The module of the corresponding function

  • ctx (TVMContext or list of TVMContext) – The context to deploy the module. It can be local or remote when there is only one TVMContext. Otherwise, the first context in the list will be used as this purpose. All context should be given for heterogeneous execution.

Returns

graph_module – Runtime graph module that can be used to execute the graph.

Return type

GraphModule

tvm.contrib.graph_runtime.get_device_ctx(libmod, ctx)

Parse and validate all the device context(s). :param libmod: The module of the corresponding function :type libmod: tvm.Module :param ctx: :type ctx: TVMContext or list of TVMContext

Returns

  • ctx (list of TVMContext)

  • num_rpc_ctx (Number of rpc contexts)

  • device_type_id (List of device type and device id)