# Relay Core Tensor Operators¶

This page contains the list of core tensor operator primitives pre-defined in tvm.relay. The core tensor operator primitives cover typical workloads in deep learning. They can represent workloads in front-end frameworks and provide basic building blocks for optimization. Since deep learning is a fast evolving field, it is possible to have operators that are not in here.

Note

This document will directly list the function signature of these operators in the python frontend.

## Overview of Operators¶

Level 1: Basic Operators

This level enables fully connected multi-layer perceptron.

 tvm.relay.log Compute elementwise log of data. tvm.relay.sqrt Compute elementwise sqrt of data. tvm.relay.rsqrt Compute elementwise rsqrt of data. tvm.relay.exp Compute elementwise exp of data. tvm.relay.sigmoid Compute elementwise sigmoid of data. tvm.relay.add Addition with numpy-style broadcasting. tvm.relay.subtract Subtraction with numpy-style broadcasting. tvm.relay.multiply Multiplication with numpy-style broadcasting. tvm.relay.divide Division with numpy-style broadcasting. tvm.relay.mod Mod with numpy-style broadcasting. tvm.relay.tanh Compute element-wise tanh of data. tvm.relay.concatenate Concatenate the input tensors along the given axis. tvm.relay.expand_dims Insert num_newaxis axises at the position given by axis. tvm.relay.nn.softmax Computes softmax. tvm.relay.nn.log_softmax Computes log softmax. tvm.relay.nn.relu Rectified linear unit. tvm.relay.nn.dropout Applies the dropout operation to the input array. tvm.relay.nn.batch_norm Batch normalization layer (Ioffe and Szegedy, 2014). tvm.relay.nn.bias_add add_bias operator.

Level 2: Convolutions

This level enables typical convnet models.

 tvm.relay.nn.conv2d 2D convolution. tvm.relay.nn.conv2d_transpose Two dimensional transposed convolution operator. tvm.relay.nn.dense Dense operator. tvm.relay.nn.max_pool2d 2D maximum pooling operator. tvm.relay.nn.max_pool3d 3D maximum pooling operator. tvm.relay.nn.avg_pool2d 2D average pooling operator. tvm.relay.nn.avg_pool3d 3D average pooling operator. tvm.relay.nn.global_max_pool2d 2D global maximum pooling operator. tvm.relay.nn.global_avg_pool2d 2D global average pooling operator. tvm.relay.nn.upsampling Upsampling. tvm.relay.nn.upsampling3d 3D Upsampling. tvm.relay.nn.batch_flatten BatchFlatten. tvm.relay.nn.pad Padding tvm.relay.nn.lrn This operator takes data as input and does local response normalization. tvm.relay.nn.l2_normalize Perform L2 normalization on the input data tvm.relay.nn.contrib_conv2d_winograd_without_weight_transform 2D convolution with winograd algorithm. tvm.relay.nn.contrib_conv2d_winograd_weight_transform Weight Transformation part for 2D convolution with winograd algorithm.

Level 3: Additional Math And Transform Operators

This level enables additional math and transform operators.

 tvm.relay.nn.leaky_relu This operator takes data as input and does Leaky version of a Rectified Linear Unit. tvm.relay.nn.prelu This operator takes data as input and does Leaky version of a Rectified Linear Unit. tvm.relay.reshape Reshapes the input array. tvm.relay.reshape_like Reshapes the input array by the size of another array. tvm.relay.copy Copy a tensor. tvm.relay.transpose Permutes the dimensions of an array. tvm.relay.squeeze Squeeze axes in the array. tvm.relay.floor Compute element-wise floor of data. tvm.relay.ceil Compute element-wise ceil of data. tvm.relay.sign Compute element-wise absolute of data. tvm.relay.trunc Compute element-wise trunc of data. tvm.relay.clip Clip the elements in a between a_min and a_max. tvm.relay.round Compute element-wise round of data. tvm.relay.abs Compute element-wise absolute of data. tvm.relay.negative Compute element-wise negative of data. tvm.relay.take Take elements from an array along an axis. tvm.relay.zeros Fill array with zeros. tvm.relay.zeros_like Returns an array of zeros, with same type and shape as the input. tvm.relay.ones Fill array with ones. tvm.relay.ones_like Returns an array of ones, with same type and shape as the input. tvm.relay.gather_nd Gather elements or slices from data and store to a tensor whose shape is defined by indices. tvm.relay.full Fill array with scalar value. tvm.relay.full_like Return a scalar value array with the same shape and type as the input array. tvm.relay.cast Cast input tensor to data type. tvm.relay.reinterpret Reinterpret input tensor to data type. tvm.relay.split Split input tensor along axis by sections or indices. tvm.relay.arange Return evenly spaced values within a given interval. tvm.relay.stack Join a sequence of arrays along a new axis. tvm.relay.repeat Repeats elements of an array. tvm.relay.tile Repeats the whole array multiple times. tvm.relay.reverse Reverses the order of elements along given axis while preserving array shape.

Level 4: Broadcast and Reductions

 tvm.relay.right_shift Right shift with numpy-style broadcasting. tvm.relay.left_shift Left shift with numpy-style broadcasting. tvm.relay.equal Broadcasted elementwise test for (lhs == rhs). tvm.relay.not_equal Broadcasted elementwise test for (lhs != rhs). tvm.relay.greater Broadcasted elementwise test for (lhs > rhs). tvm.relay.greater_equal Broadcasted elementwise test for (lhs >= rhs). tvm.relay.less Broadcasted elementwise test for (lhs < rhs). tvm.relay.less_equal Broadcasted elementwise test for (lhs <= rhs). tvm.relay.all Computes the logical AND of boolean array elements over given axes. tvm.relay.any Computes the logical OR of boolean array elements over given axes. tvm.relay.logical_and logical AND with numpy-style broadcasting. tvm.relay.logical_or logical OR with numpy-style broadcasting. tvm.relay.logical_not Compute element-wise logical not of data. tvm.relay.maximum Maximum with numpy-style broadcasting. tvm.relay.minimum Minimum with numpy-style broadcasting. tvm.relay.power Power with numpy-style broadcasting. tvm.relay.where Selecting elements from either x or y depending on the value of the condition. tvm.relay.argmax Returns the indices of the maximum values along an axis. tvm.relay.argmin Returns the indices of the minimum values along an axis. tvm.relay.sum Computes the sum of array elements over given axes. tvm.relay.max Computes the max of array elements over given axes. tvm.relay.min Computes the min of array elements over given axes. tvm.relay.mean Computes the mean of array elements over given axes. tvm.relay.variance Computes the variance of data over given axes. tvm.relay.std Computes the standard deviation of data over given axes. tvm.relay.mean_variance Computes the mean and variance of data over given axes. tvm.relay.mean_std Computes the mean and standard deviation of data over given axes. tvm.relay.prod Computes the products of array elements over given axes. tvm.relay.strided_slice Strided slice of an array. tvm.relay.broadcast_to Return a scalar value array with the same type, broadcast to the provided shape.

Level 5: Vision/Image Operators

 tvm.relay.image.resize Image resize operator. tvm.relay.image.crop_and_resize Crop input images and resize them. tvm.relay.vision.multibox_prior Generate prior(anchor) boxes from data, sizes and ratios. tvm.relay.vision.multibox_transform_loc Location transformation for multibox detection tvm.relay.vision.nms Non-maximum suppression operations. tvm.relay.vision.yolo_reorg Yolo reorg operation used in darknet models.

Level 6: Algorithm Operators

 tvm.relay.argsort Performs sorting along the given axis and returns an array of indicies having same shape as an input array that index data in sorted order. tvm.relay.topk Get the top k elements in an input tensor along the given axis.

Level 10: Temporary Operators

This level support backpropagation of broadcast operators. It is temporary.

 tvm.relay.broadcast_to_like Return a scalar value array with the same shape and type as the input array. tvm.relay.collapse_sum_like Return a scalar value array with the same shape and type as the input array. tvm.relay.slice_like Slice the first input with respect to the second input. tvm.relay.shape_of Get shape of a tensor. tvm.relay.contrib.ndarray_size Get number of elements of input tensor. tvm.relay.layout_transform Transform the layout of a tensor tvm.relay.device_copy Copy data from the source device to the destination device. tvm.relay.annotation.on_device Annotate an expression with a certain device type. tvm.relay.reverse_reshape Reshapes the input array where the special values are inferred from right to left. tvm.relay.sequence_mask Sets all elements outside the expected length of the sequence to a constant value. tvm.relay.nn.batch_matmul Computes batch matrix multiplication of x and y when x and y are data in batch. tvm.relay.contrib.adaptive_max_pool2d 2D adaptive max pooling operator. tvm.relay.contrib.adaptive_avg_pool2d 2D adaptive average pooling operator. tvm.relay.one_hot Returns a one-hot tensor where the locations repsented by indices take value on_value, other locations take value off_value.

Level 11: Dialect Operators

This level supports dialect operators.

 tvm.relay.qnn.op.requantize Requantized operator. tvm.relay.qnn.op.conv2d Quantized 2D convolution.

## Level 1 Definitions¶

tvm.relay.log(data)

Compute elementwise log of data.

Parameters

data (relay.Expr) – The input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.sqrt(data)

Compute elementwise sqrt of data.

Parameters

data (relay.Expr) – The input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.rsqrt(data)

Compute elementwise rsqrt of data.

$1/sqrt(x)$
Parameters

data (relay.Expr) – The input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.exp(data)

Compute elementwise exp of data.

Parameters

data (relay.Expr) – The input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.sigmoid(data)

Compute elementwise sigmoid of data.

Parameters

data (relay.Expr) – The input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.add(lhs, rhs)

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

Examples

x = relay.Var("a") # shape is [2, 3]
y = relay.Var("b") # shape is [2, 1]
z = relay.add(x, y)  # result shape is [2, 3]

tvm.relay.subtract(lhs, rhs)

Subtraction with numpy-style broadcasting.

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.multiply(lhs, rhs)

Multiplication with numpy-style broadcasting.

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.divide(lhs, rhs)

Division with numpy-style broadcasting.

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.floor_divide(lhs, rhs)

Floor division with numpy-style broadcasting.

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.mod(lhs, rhs)

Mod with numpy-style broadcasting.

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.floor_mod(lhs, rhs)

Floor mod with numpy-style broadcasting.

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.tanh(data)

Compute element-wise tanh of data.

Parameters

data (relay.Expr) – The input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.concatenate(data, axis)

Concatenate the input tensors along the given axis.

Parameters
• data (Union(List[relay.Expr], Tuple[relay.Expr])) – A list of tensors.

• axis (int) – The axis along which the tensors are concatenated.

Returns

result – The concatenated tensor.

Return type

relay.Expr

tvm.relay.expand_dims(data, axis, num_newaxis=1)

Insert num_newaxis axises at the position given by axis.

Parameters
• data (relay.Expr) – The input data to the operator.

• axis (int) – The axis at which the input array is expanded. Should lie in range [-data.ndim - 1, data.ndim]. If axis < 0, it is the first axis inserted; If axis >= 0, it is the last axis inserted in Python’s negative indexing.

• num_newaxis (int) – Number of axes to be inserted. Should be >= 0.

Returns

result – The reshaped result.

Return type

relay.Expr

tvm.relay.nn.softmax(data, axis=-1)

Computes softmax.

$\text{softmax}(x)_i = \frac{exp(x_i)}{\sum_j exp(x_j)}$

Note

This operator can be optimized away for inference.

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• axis (int, optional) – The axis to sum over when computing softmax

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.log_softmax(data, axis=-1)

Computes log softmax.

$\text{log_softmax}(x)_i = \log \frac{exp(x_i)}{\sum_j exp(x_j)}$

Note

This operator can be optimized away for inference.

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• axis (int) – The axis to sum over when computing softmax

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.relu(data)

Rectified linear unit.

$out = max(x, 0)$
Parameters

data (tvm.relay.Expr) – The input data

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.dropout(data, rate=0.5)

Applies the dropout operation to the input array.

During training, each element of the input is set to zero with probability p. The whole array is rescaled by 1/(1-p) to keep the expected sum of the input unchanged.

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• rate (float, optional (default=0.5)) – The probability for an element to be reset to 0.

Returns

result – The result of dropout

Return type

tvm.relay.Expr

tvm.relay.nn.batch_norm(data, gamma, beta, moving_mean, moving_var, axis=1, epsilon=1e-05, center=True, scale=True)

Batch normalization layer (Ioffe and Szegedy, 2014). Normalizes the input at each batch, i.e. applies a transformation that maintains the mean activation close to 0 and the activation standard deviation close to 1.

$\begin{split}data\_mean[i] = mean(data[:,i,:,...]) \\ data\_var[i] = var(data[:,i,:,...])\end{split}$

Then compute the normalized output, which has the same shape as input, as following:

$out[:,i,:,...] = \frac{data[:,i,:,...] - data\_mean[i]}{\sqrt{data\_var[i]+\epsilon}} * gamma[i] + beta[i]$

Both mean and var returns a scalar by treating the input as a vector.

Assume the input has size k on axis 1, then both gamma and beta have shape (k,).

Besides the inputs and the outputs, this operator accepts two auxiliary states, moving_mean and moving_var, which are k-length vectors. They are global statistics for the whole dataset, which are updated by:

moving_mean = moving_mean * momentum + data_mean * (1 - momentum) moving_var = moving_var * momentum + data_var * (1 - momentum)

The parameter axis specifies which axis of the input shape denotes the ‘channel’ (separately normalized groups). The default is 1. Specifying -1 sets the channel axis to be the last item in the input shape.

Note

This operator can be optimized away for inference.

Parameters
• data (tvm.relay.Expr) – Input to which batch_norm will be applied.

• gamma (tvm.relay.Expr) – The gamma scale factor.

• beta (tvm.relay.Expr) – The beta offset factor.

• moving_mean (tvm.relay.Expr) – Running mean of input,

• moving_var (tvm.relay.Expr) – Running variance of input.

• axis (int, optional, default=1) – Specify along which shape axis the channel is specified.

• epsilon (double, optional, default=1e-5) – Small float added to variance to avoid dividing by zero.

• center (boolean, optional, default=True) – If True, add offset of beta to normalized tensor, If False, beta is ignored.

• scale (boolean, optional, default=True) – If true, multiply by gamma. If False, gamma is not used. When the next layer is piecewise linear (also e.g. nn.relu), this can be disabled since the scaling will be done by the next layer.

Returns

result – Tuple of normed data (same shape as input), new running mean (k-length vector), and new running variance (k-length vector)

Return type

relay.Tuple([tvm.relay.Expr, tvm.relay.Expr, tvm.relay.Expr])

tvm.relay.nn.bias_add(data, bias, axis=1)

Add 1D bias to the axis of data. This function is a special case of add which allows inference of shape of the bias from data.

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• bias (tvm.relay.Expr) – The bias to be added.

• axis (int, optional) – The axis to add the bias.

Returns

result – The final result.

Return type

tvm.relay.Expr

## Level 2 Definitions¶

tvm.relay.nn.conv2d(data, weight, strides=(1, 1), padding=(0, 0), dilation=(1, 1), groups=1, channels=None, kernel_size=None, data_layout='NCHW', kernel_layout='OIHW', out_layout='', out_dtype='')

2D convolution.

This operator takes the weight as the convolution kernel and convolves it with data to produce an output.

In the default case, where the data_layout is NCHW and kernel_layout is OIHW, conv2d takes in a data Tensor with shape (batch_size, in_channels, height, width), and a weight Tensor with shape (channels, in_channels, kernel_size[0], kernel_size[1]) to produce an output Tensor with the following rule:

$\mbox{out}[b, c, y, x] = \sum_{dy, dx, k} \mbox{data}[b, k, \mbox{strides}[0] * y + dy, \mbox{strides}[1] * x + dx] * \mbox{weight}[c, k, dy, dx]$

Padding and dilation are applied to data and weight respectively before the computation. This operator accepts data layout specification. Semantically, the operator will convert the layout to the canonical layout (NCHW for data and OIHW for weight), perform the computation, then convert to the out_layout.

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• weight (tvm.relay.Expr) – The weight expressions.

• strides (Optional[int, Tuple[int]]) – The strides of convolution.

• padding (Optional[int, Tuple[int]]) – The padding of convolution on both sides of inputs before convolution.

• dilation (Optional[int, Tuple[int]]) – Specifies the dilation rate to be used for dilated convolution.

• groups (Optional[int]) – Number of groups for grouped convolution.

• channels (Optional[int]) – Number of output channels of this convolution.

• kernel_size (Optional[int, Tuple[int]]) – The spatial of the convolution kernel.

• data_layout (Optional[str]) – Layout of the input.

• kernel_layout (Optional[str]) – Layout of the weight.

• out_layout (Optional[str]) – Layout of the output, by default, out_layout is the same as data_layout

• out_dtype (Optional[str]) – Specifies the output data type for mixed precision conv2d.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.conv2d_transpose(data, weight, strides=(1, 1), padding=(0, 0), dilation=(1, 1), groups=1, channels=None, kernel_size=None, data_layout='NCHW', kernel_layout='OIHW', out_layout='', output_padding=(0, 0), out_dtype='')

Two dimensional transposed convolution operator.

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• weight (tvm.relay.Expr) – The weight expressions.

• strides (Tuple[int], optional) – The strides of convolution.

• padding (Tuple[int], optional) – The padding of convolution on both sides of inputs.

• dilation (Tuple[int], optional) – Specifies the dilation rate to be used for dilated convolution.

• channels (int, optional) – Number of output channels of this convolution.

• kernel_size (tuple of int, optional) – The spatial of the convolution kernel.

• groups (int, optional) – Number of groups for grouped convolution.

• data_layout (str, optional) – Layout of the input.

• kernel_layout (str, optional) – Layout of the weight.

• out_layout (Optional[str]) – Layout of the output, by default, out_layout is the same as data_layout

• output_padding (Tuple[int], optional) – Additional zero-padding to be added to one side of the output.

• out_dtype (str, optional) – Specifies the output data type for mixed precision conv2d.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.dense(data, weight, units=None, out_dtype='')

Dense operator. Applies a linear transformation



Y = X * W

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• weight (tvm.relay.Expr) – The weight expressions.

• units (int, optional) – Number of hidden units of the dense transformation.

• out_dtype (str, optional) – Specifies the output data type for mixed precision dense.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.max_pool2d(data, pool_size=(1, 1), strides=(1, 1), padding=(0, 0), layout='NCHW', ceil_mode=False)

2D maximum pooling operator.

This operator takes data as input and does 2D max value calculation with in pool_size sized window by striding defined by stride

In the default case, where the data_layout is NCHW a data Tensor with shape (batch_size, in_channels, height, width), to produce an output Tensor with the following rule:

with data of shape (b, c, h, w) and pool_size (kh, kw)

$\mbox{out}(b, c, y, x) = \max_{m=0, \ldots, kh-1} \max_{n=0, \ldots, kw-1} \mbox{data}(b, c, \mbox{stride}[0] * y + m, \mbox{stride}[1] * x + n)$

Padding is applied to data before the computation. ceil_mode is used to take ceil or floor while computing out shape. This operator accepts data layout specification.

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• strides (tuple of int, optional) – The strides of pooling.

• padding (tuple of int, optional) – The padding for pooling.

• layout (str, optional) – Layout of the input.

• ceil_mode (bool, optional) – To enable or disable ceil while pooling.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.max_pool3d(data, pool_size=(1, 1, 1), strides=(1, 1, 1), padding=(0, 0, 0), layout='NCDHW', ceil_mode=False)

3D maximum pooling operator.

This operator takes data as input and does 3D max value calculation with in pool_size sized window by striding defined by stride.

In the default case, where the data_layout is NCDHW a data Tensor with shape (batch_size, channels, depth, height, width), to produce an output Tensor.

The ceil_mode is used to take ceil or floor while computing out shape. count_include_pad indicates including or excluding padded input values in computation. This operator accepts data layout specification.

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• strides (tuple of int, optional) – The strides of pooling.

• padding (tuple of int, optional) – The padding for pooling.

• layout (str, optional) – Layout of the input.

• ceil_mode (bool, optional) – To enable or disable ceil while pooling.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.avg_pool2d(data, pool_size=(1, 1), strides=(1, 1), padding=(0, 0), layout='NCHW', ceil_mode=False, count_include_pad=False)

2D average pooling operator.

This operator takes data as input and does 2D average value calculation with in pool_size sized window by striding defined by stride

In the default case, where the data_layout is NCHW a data Tensor with shape (batch_size, in_channels, height, width), to produce an output Tensor with the following rule:

with data of shape (b, c, h, w), pool_size (kh, kw)

$\mbox{out}(b, c, y, x) = \frac{1}{kh * kw} \sum_{m=0}^{kh-1} \sum_{n=0}^{kw-1} \mbox{data}(b, c, \mbox{stride}[0] * y + m, \mbox{stride}[1] * x + n)$

Padding is applied to data before the computation. ceil_mode is used to take ceil or floor while computing out shape. count_include_pad indicates including or excluding padded input values in computation. This operator accepts data layout specification.

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• strides (tuple of int, optional) – The strides of pooling.

• padding (tuple of int, optional) – The padding for pooling.

• layout (str, optional) – Layout of the input.

• ceil_mode (bool, optional) – To enable or disable ceil while pooling.

• count_include_pad (bool, optional) – To include padding to compute the average.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.avg_pool3d(data, pool_size=(1, 1, 1), strides=(1, 1, 1), padding=(0, 0, 0), layout='NCDHW', ceil_mode=False, count_include_pad=False)

3D average pooling operator.

This operator takes data as input and does 3D average value calculation with in pool_size sized window by striding defined by stride

In the default case, where the data_layout is NCDHW a data Tensor with shape (batch_size, channels, depth, height, width), to produce an output Tensor.

The ceil_mode is used to take ceil or floor while computing out shape. count_include_pad indicates including or excluding padded input values in computation. This operator accepts data layout specification.

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• strides (tuple of int, optional) – The strides of pooling.

• padding (tuple of int, optional) – The padding for pooling.

• layout (str, optional) – Layout of the input.

• ceil_mode (bool, optional) – To enable or disable ceil while pooling.

• count_include_pad (bool, optional) – To include padding to compute the average.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.global_max_pool2d(data, layout='NCHW')

2D global maximum pooling operator.

This operator takes data as input and does 2D max value calculation across each window represented by WxH.

In the default case, where the data_layout is NCHW a data Tensor with shape (batch_size, in_channels, height, width), to produce an output Tensor with the following rule:

with data of shape (b, c, h, w)

$\mbox{out}(b, c, 1, 1) = \max_{m=0, \ldots, h} \max_{n=0, \ldots, w} \mbox{data}(b, c, m, n)$
Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• layout (str, optional) – Layout of the input.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.global_avg_pool2d(data, layout='NCHW')

2D global average pooling operator.

This operator takes data as input and does 2D average value calculation across each window represented by WxH.

In the default case, where the data_layout is NCHW a data Tensor with shape (batch_size, in_channels, height, width), to produce an output Tensor with the following rule:

with data of shape (b, c, h, w)

$\mbox{out}(b, c, 1, 1) = \frac{1}{h * w} \sum_{m=0}^{h-1} \sum_{n=0}^{w-1} \mbox{data}(b, c, m, n)$
Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• layout (str, optional) – Layout of the input.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.upsampling(data, scale_h=1, scale_w=1, layout='NCHW', method='nearest_neighbor', align_corners=False)

Upsampling.

This operator takes data as input and does 2D scaling to the given scale factor. In the default case, where the data_layout is NCHW with data of shape (n, c, h, w) out will have a shape (n, c, h*scale_h, w*scale_w)

method indicates the algorithm to be used while calculating the out value and method can be one of (“bilinear”, “nearest_neighbor”, “bicubic”)

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• scale_h (tvm.relay.Expr) – The scale factor for height upsampling.

• scale_w (tvm.relay.Expr) – The scale factor for width upsampling.

• layout (str, optional) – Layout of the input.

• method (str, optional) – Scale method to used [nearest_neighbor, bilinear, bicubic].

• align_corners (bool, optional) – Whether to keep corners in proper place.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.upsampling3d(data, scale_d=1, scale_h=1, scale_w=1, layout='NCDHW', method='nearest_neighbor', coordinate_transformation_mode='half_pixel')

3D Upsampling.

This operator takes data as input and does 3D scaling to the given scale factor. In the default case, where the data_layout is NCDHW with data of shape (n, c, d, h, w) out will have a shape (n, c, d*scale_d, h*scale_h, w*scale_w)

method indicates the algorithm to be used while calculating the out value and method can be one of (“trilinear”, “nearest_neighbor”)

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• scale_d (tvm.relay.Expr) – The scale factor for depth upsampling.

• scale_h (tvm.relay.Expr) – The scale factor for height upsampling.

• scale_w (tvm.relay.Expr) – The scale factor for width upsampling.

• layout (str, optional) – Layout of the input.

• method (str, optional) – Scale method to used [nearest_neighbor, trilinear].

• coordinate_transformation_mode (string, optional) – Describes how to transform the coordinate in the resized tensor to the coordinate in the original tensor. Refer to the ONNX Resize operator specification for details. Available options are “half_pixel”, “align_corners” and “asymmetric”.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.batch_flatten(data)

BatchFlatten.

This operator flattens all the dimensions except for the batch dimension. which results a 2D output.

For data with shape (d1, d2, ..., dk) batch_flatten(data) returns reshaped output of shape (d1, d2*...*dk).

Parameters

data (tvm.relay.Expr) – The input data to the operator.

Returns

result – The Flattened result.

Return type

tvm.relay.Expr

tvm.relay.nn.pad(data, pad_width, pad_value=0.0, pad_mode='constant')

This operator takes in a tensor and pads each axis by the specified widths using the specified value.

Parameters
• data (tvm.relay.Expr) – The input data to the operator

• pad_width (tuple of <tuple of <int>>, required) – Number of values padded to the edges of each axis, in the format of ((before_1, after_1), …, (before_N, after_N))

• pad_value (float, optional, default=0.0) – The value used for padding

• pad_mode ('constant', 'edge', 'reflect') – ‘constant’ pads with constant_value pad_value ‘edge’ pads using the edge values of the input array ‘reflect’ pads by reflecting values with respect to the edge

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.lrn(data, size=5, axis=1, bias=2, alpha=1e-05, beta=0.75)

This operator takes data as input and does local response normalization.

Normalize the input in a local region across or within feature maps. Each input value is divided by (data / (bias + (alpha * sum_data ^2 /size))^beta) where n is the size of each local region, and the sum is taken over the region centered at that value (zero padding is added where necessary).

$(data / (bias + (alpha * sum_data ^2 /size))^beta)$
Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• size (int, optional) – The size of the local region to be considered for normalization.

• axis (int, optional) – Input data layout channel axis. Default value is 1 for NCHW format

• bias (float, optional) – The offset parameter to avoid dividing by 0.

• alpha (float, optional) – The scaling parameter.

• beta (float, optional) – The exponent parameter.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.l2_normalize(data, eps, axis=None)

Perform L2 normalization on the input data

$y(i, j) = x(i, j) / sqrt(max(sum(x^2), eps))$
Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• eps (float) – epsilon value

• axis (list of int, optional) – axis over the normalization applied

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.contrib_conv2d_winograd_without_weight_transform(data, weight, tile_size, strides=(1, 1), padding=(0, 0), dilation=(1, 1), groups=1, channels=None, kernel_size=None, data_layout='NCHW', kernel_layout='OIHW', out_layout='', out_dtype='')

2D convolution with winograd algorithm.

The basic parameters are the same as the ones in vanilla conv2d. It assumes the weight is pre-transformed by nn.contrib_conv2d_winograd_weight_transform

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• weight (tvm.relay.Expr) – The weight expressions.

• tile_size (int) – The Tile size of winograd. E.g. 2 for F(2x2, 3x3) and 4 for F(4x4, 3x3)

• strides (tuple of int, optional) – The strides of convolution.

• padding (tuple of int, optional) – The padding of convolution on both sides of inputs before convolution.

• dilation (tuple of int, optional) – Specifies the dilation rate to be used for dilated convolution.

• groups (int, optional) – Number of groups for grouped convolution.

• channels (int, optional) – Number of output channels of this convolution.

• kernel_size (tuple of int, optional) – The spatial of the convolution kernel.

• data_layout (str, optional) – Layout of the input.

• kernel_layout (str, optional) – Layout of the weight.

• out_layout (str, optional) – Layout of the output, by default, out_layout is the same as data_layout

• out_dtype (str, optional) – Specifies the output data type for mixed precision conv2d.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.contrib_conv2d_winograd_weight_transform(weight, tile_size)

Weight Transformation part for 2D convolution with winograd algorithm.

We separate this as a single op to enable pre-compute for inference. Use this together with nn.contrib_conv2d_winograd_without_weight_transform

Parameters
• weight (tvm.relay.Expr) – The weight expressions.

• tile_size (int) – The Tile size of winograd. E.g. 2 for F(2x2, 3x3) and 4 for F(4x4, 3x3)

Returns

result – The computed result.

Return type

tvm.relay.Expr

## Level 3 Definitions¶

tvm.relay.nn.leaky_relu(data, alpha)

This operator takes data as input and does Leaky version of a Rectified Linear Unit.

$y = x > 0 ? x : alpha * x$
Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• alpha (float) – Slope coefficient for the negative half axis.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.nn.prelu(data, alpha, axis=1)

This operator takes data as input and does Leaky version of a Rectified Linear Unit.

$y = x > 0 ? x : alpha * x$
Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• alpha (tvm.relay.Expr) – Slope coefficient for the negative half axis.

• axis (int, optional) – Specify which shape axis the channel is specified.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.reshape(data, newshape)

Reshapes the input array.

Example:

To give user more convenience in without doing manual shape inference, some dimensions of the shape can take special values from the set {0, -1, -2, -3, -4}. The significance of each is explained below:

• 0 copy this dimension from the input to the output shape.

Example:

- data.shape = (2,3,4), newshape = (4,0,2), result.shape = (4,3,2)
- data.shape = (2,3,4), newshape = (2,0,0), result.shape = (2,3,4)

• -1 infers the dimension of the output shape by using the remainder of the input dimensions

keeping the size of the new array same as that of the input array. At most one dimension of shape can be -1.

Example:

- data.shape = (2,3,4), newshape = (6,1,-1), result.shape = (6,1,4)
- data.shape = (2,3,4), newshape = (3,-1,8), result.shape = (3,1,8)
- data.shape = (2,3,4), newshape = (-1,), result.shape = (24,)

• -2 copy all/remainder of the input dimensions to the output shape.

Example:

- data.shape = (2,3,4), newshape = (-2,), result.shape = (2,3,4)
- data.shape = (2,3,4), newshape = (2,-2), result.shape = (2,3,4)
- data.shape = (2,3,4), newshape = (-2,1,1), result.shape = (2,3,4,1,1)

• -3 use the product of two consecutive dimensions of the input shape

as the output dimension.

Example:

- data.shape = (2,3,4), newshape = (-3,4), result.shape = (6,4)
- data.shape = (2,3,4,5), newshape = (-3,-3), result.shape = (6,20)
- data.shape = (2,3,4), newshape = (0,-3), result.shape = (2,12)
- data.shape = (2,3,4), newshape = (-3,-2), result.shape = (6,4)

• -4 split one dimension of the input into two dimensions passed subsequent

to -4 in shape (can contain -1).

Example:

- data.shape = (2,3,4), newshape = (-4,1,2,-2), result.shape = (1,2,3,4)
- data.shape = (2,3,4), newshape = (2,-4,-1,3,-2), result.shape = (2,1,3,4)

Parameters
• data (relay.Expr) – The input data to the operator.

• newshape (Union[int, Tuple[int], List[int]]) – The new shape. Should be compatible with the original shape.

Returns

result – The reshaped result.

Return type

relay.Expr

tvm.relay.reshape_like(data, shape_like)

Reshapes the input array by the size of another array. For an input array with shape (d1, d2, ..., dk), reshape_like operation reshapes the input array into an output array with the same shape as the second input array.

Note

Sizes for both array should be compatible.

Parameters
• data (relay.Expr) – The input data to the operator.

• shape_like (tuple of int) – The new shape. Should be compatible with the original shape.

Returns

ret – The computed result.

Return type

relay.Expr

tvm.relay.copy(data)

Copy a tensor.

Parameters

data (relay.Expr) – The tensor to be copied.

Returns

result – The copied result.

Return type

relay.Expr

tvm.relay.transpose(data, axes=None)

Permutes the dimensions of an array.

Parameters
• data (relay.Expr) – The input data to the operator.

• axes (None or List[int]) – The target axes order, reverse order if not specified.

Returns

result – The transposed result.

Return type

relay.Expr

tvm.relay.squeeze(data, axis=None)

Squeeze axes in the array.

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• axis (None or List[int]) – The set of axes to remove. If axis = None, remove all axis of dimensions 1. If any specified axis has dimension that does not equal 1, it is an error.

Returns

result – The squeezed result.

Return type

tvm.relay.Expr

tvm.relay.floor(data)

Compute element-wise floor of data.

Parameters

data (relay.Expr) – The input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.ceil(data)

Compute element-wise ceil of data.

Parameters

data (relay.Expr) – The input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.sign(data)

Compute element-wise absolute of data.

Parameters

data (relay.Expr) – The input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.trunc(data)

Compute element-wise trunc of data.

Parameters

data (relay.Expr) – The input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.clip(a, a_min, a_max)

Clip the elements in a between a_min and a_max. a_min and a_max are cast to a’s dtype.

Parameters
• a (relay.Expr) – The input tensor.

• a_min (float) – The clip minimum.

• a_max (float) – The clip maximum.

Returns

resulta with elements clipped between a_min and a_max.

Return type

relay.Expr

Examples

tvm.relay.round(data)

Compute element-wise round of data.

Parameters

data (relay.Expr) – The input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.abs(data)

Compute element-wise absolute of data.

Parameters

data (relay.Expr) – The input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.negative(data)

Compute element-wise negative of data.

Parameters

data (relay.Expr) – The input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.take(data, indices, axis=None, mode='clip')

Take elements from an array along an axis.

Parameters
• data (relay.Expr) – The source array.

• indices (rely.Expr) – The indices of the values to extract.

• axis (int, optional) – The axis over which to select values. By default, the flattened input array is used.

• mode (str, optional) – Specifies how out-of-bound indices will behave [clip, wrap, fast]. clip: clip to the range (default). wrap: wrap around the indices. fast: no clip or wrap around (user must make sure indices are in-bound).

Returns

ret – The computed result.

Return type

relay.Expr

tvm.relay.zeros(shape, dtype)

Fill array with zeros.

Parameters
• shape (tuple of int) – The shape of the target.

• dtype (data type) – The data type of the target.

Returns

result – The resulting tensor.

Return type

relay.Expr

tvm.relay.zeros_like(data)

Returns an array of zeros, with same type and shape as the input.

Parameters

data (relay.Expr) – The input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.ones(shape, dtype)

Fill array with ones.

Parameters
• shape (tuple of int) – The shape of the target.

• dtype (data type) – The data type of the target.

Returns

result – The resulting tensor.

Return type

relay.Expr

tvm.relay.ones_like(data)

Returns an array of ones, with same type and shape as the input.

Parameters

data (relay.Expr) – The input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.gather_nd(data, indices)

Gather elements or slices from data and store to a tensor whose shape is defined by indices.

Parameters
• data (relay.Expr) – The input data to the operator.

• indices (relay.Expr) – The shape of output tensor.

Returns

ret – The computed result.

Return type

relay.Expr

Examples

data = [[0, 1], [2, 3]]
indices = [[1, 1, 0], [0, 1, 0]]
relay.gather_nd(data, indices) = [2, 3, 0]

data = [[[1, 2], [3, 4]], [[5, 6], [7, 8]]]
indices = [[0, 1], [1, 0]]
relay.gather_nd(data, indices) = [[3, 4], [5, 6]]

tvm.relay.full(fill_value, shape=(), dtype='')

Fill array with scalar value.

Parameters
• fill_value (relay.Expr) – The value to fill. Must be a scalar.

• shape (tuple of int) – The shape of the target.

• dtype (data type, optional (defaults to data type of the fill value)) – The data type of the target.

Returns

result – The resulting tensor.

Return type

relay.Expr

tvm.relay.full_like(data, fill_value)

Return a scalar value array with the same shape and type as the input array.

Parameters
• data (relay.Expr) – The input tensor.

• fill_value (relay.Expr) – The scalar value to fill.

Returns

result – The resulting tensor.

Return type

relay.Expr

tvm.relay.cast(data, dtype)

Cast input tensor to data type.

Parameters
• data (relay.Expr) – The input data to the operator.

• dtype (str) – The target data type

Returns

result – The casted result.

Return type

relay.Expr

tvm.relay.reinterpret(data, dtype)

Reinterpret input tensor to data type.

Parameters
• data (relay.Expr) – The input data to the operator.

• dtype (str) – The target data type

Returns

result – The reinterpreted result.

Return type

relay.Expr

tvm.relay.split(data, indices_or_sections, axis=0)

Split input tensor along axis by sections or indices.

If indices_or_sections is an integer, the input will be divided equally along given axis. If such a split is not possible, an error is raised.

If indices_or_sections is a tuple of sorted integers, the entries indicate where along axis the array is split.

Parameters
• data (relay.Expr) – The source array.

• indices_or_sections (int or tuple of int) – Indices or sections to split into. Accepts an int or a tuple

• axis (int, optional) – The axis over which to split.

Returns

ret – The computed result.

Return type

relay.Tuple([relay.Expr, relay.Expr])

tvm.relay.arange(start, stop=None, step=None, dtype='float32')

Return evenly spaced values within a given interval.

Note

Similar to numpy.arange, when only one argument is given, it is used as stop instead of start while start takes default value 0.

Warning: Undefined behavior when dtype is incompatible with start/stop/step. It could lead to different results compared to numpy, MXNet, pytorch, etc.

Parameters
• start (tvm.Expr, optional) – Start of interval. The interval includes this value. The default start value is 0.

• stop (tvm.Expr) – Stop of interval. The interval does not include this value.

• step (tvm.Expr, optional) – Spacing between values. The default step size is 1.

• dtype (str, optional) – The target data type.

Returns

result – The resulting tensor.

Return type

relay.Expr

Examples

relay.arange(5) = [0, 1, 2, 3, 4]
relay.arange(1, 5) = [1, 2, 3, 4]
relay.arange(1, 5, 1.5) = [1, 2.5, 4]

tvm.relay.stack(data, axis)

Join a sequence of arrays along a new axis.

Parameters
• data (Union(List[relay.Expr], Tuple(relay.Expr))) – A list of tensors.

• axis (int) – The axis in the result array along which the input arrays are stacked.

Returns

ret – The stacked tensor.

Return type

relay.Expr

tvm.relay.repeat(data, repeats, axis)

Repeats elements of an array. By default, repeat flattens the input array into 1-D and then repeats the elements.

repeatsint

The number of repetitions for each element.

axis: int

The axis along which to repeat values. The negative numbers are interpreted counting from the backward. By default, use the flattened input array, and return a flat output array.

Returns

ret – The computed result.

Return type

relay.Expr

Examples

x = [[1, 2], [3, 4]]
relay.repeat(x, repeats=2) = [1., 1., 2., 2., 3., 3., 4., 4.]

relay.repeat(x, repeats=2, axis=1) = [[1., 1., 2., 2.],
[3., 3., 4., 4.]]

tvm.relay.tile(data, reps)

Repeats the whole array multiple times.

Parameters
• data (relay.Expr) – The input data to the operator.

• reps (tuple of int) – The number of times repeating the tensor data.

• note:: (.) – Each dim size of reps must be a positive integer. If reps has length d, the result will have dimension of max(d, data.ndim); If data.ndim < d, data is promoted to be d-dimensional by prepending new axes. If data.ndim >= d, reps is promoted to a.ndim by pre-pending 1’s to it.

Returns

ret – The computed result.

Return type

relay.Expr

Examples

x = [[1, 2], [3, 4]]
relay.tile(x, reps=(2,3)) = [[1., 2., 1., 2., 1., 2.],
[3., 4., 3., 4., 3., 4.],
[1., 2., 1., 2., 1., 2.],
[3., 4., 3., 4., 3., 4.]]

relay.tile(x, reps=(2,)) = [[1., 2., 1., 2.],
[3., 4., 3., 4.]]

tvm.relay.reverse(data, axis)

Reverses the order of elements along given axis while preserving array shape. By default, repeat flattens the input array into 1-D and then repeats the elements.

Parameters
• data (relay.Expr) – The input data to the operator.

• axis (int) – The axis along which to reverse elements.

Returns

ret – The computed result.

Return type

relay.Expr

Examples

x = [[1., 2.], [3., 4.]]
relay.reverse(x, axis=0) = [[3., 4.], [1., 2.]]

relay.reverse(x, axis=1) = [[2., 1.], [4., 3.]]


## Level 4 Definitions¶

tvm.relay.right_shift(lhs, rhs)

Right shift with numpy-style broadcasting.

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.left_shift(lhs, rhs)

Left shift with numpy-style broadcasting.

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.equal(lhs, rhs)

Broadcasted elementwise test for (lhs == rhs).

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.not_equal(lhs, rhs)

Broadcasted elementwise test for (lhs != rhs).

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.greater(lhs, rhs)

Broadcasted elementwise test for (lhs > rhs).

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.greater_equal(lhs, rhs)

Broadcasted elementwise test for (lhs >= rhs).

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.less(lhs, rhs)

Broadcasted elementwise test for (lhs < rhs).

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.less_equal(lhs, rhs)

Broadcasted elementwise test for (lhs <= rhs).

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.all(data, axis=None, keepdims=False, exclude=False)

Computes the logical AND of boolean array elements over given axes.

Parameters
• data (relay.Expr) – The input boolean tensor

• axis (None or int or tuple of int) – Axis or axes along which a sum is performed. The default, axis=None, will sum all of the elements of the input array. If axis is negative it counts from the last to the first axis.

• keepdims (bool) – If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.

• exclude (bool) – If exclude is true, reduction will be performed on the axes that are NOT in axis instead.

Returns

result – The computed result.

Return type

relay.Expr

Examples

data = relay.Constant(tvm.nd.array([[[ True, True, True],

[ True, True, True], [False, True, False]],

[[ True, False, False],

[ True, True, False], [False, True, True]]]))

relay.all(data, axis=1) # [[False, True, False], # [False, False, False]]

relay.all(data, axis=0) # [[ True, False, False], # [ True, True, False], # [False, True, False]]

tvm.relay.any(data, axis=None, keepdims=False, exclude=False)

Computes the logical OR of boolean array elements over given axes.

Parameters
• data (relay.Expr) – The input boolean tensor

• axis (None or int or tuple of int) – Axis or axes along which a sum is performed. The default, axis=None, will sum all of the elements of the input array. If axis is negative it counts from the last to the first axis.

• keepdims (bool) – If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.

• exclude (bool) – If exclude is true, reduction will be performed on the axes that are NOT in axis instead.

Returns

result – The computed result.

Return type

relay.Expr

Examples

data = relay.Constant(tvm.nd.array([[[ True, True, True],

[ True, True, True], [False, True, False]],

[[ True, False, False],

[ True, True, False], [False, True, True]]]))

relay.any(data, axis=1) # [[True, True, True], # [True, True, True]]

relay.any(data, axis=0) # [[ True, True, True], # [ True, True, True], # [False, True, True]]

tvm.relay.logical_and(lhs, rhs)

logical AND with numpy-style broadcasting.

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.logical_or(lhs, rhs)

logical OR with numpy-style broadcasting.

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.logical_not(data)

Compute element-wise logical not of data.

Parameters

data (relay.Expr) – The input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.maximum(lhs, rhs)

Maximum with numpy-style broadcasting.

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.minimum(lhs, rhs)

Minimum with numpy-style broadcasting.

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.power(lhs, rhs)

Power with numpy-style broadcasting.

Parameters
• lhs (relay.Expr) – The left hand side input data

• rhs (relay.Expr) – The right hand side input data

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.where(condition, x, y)

Selecting elements from either x or y depending on the value of the condition.

Note

The shape of condition, x, and y needs to be the same.

Parameters
• condition (relay.Expr) – The condition array. The n-th element in y is selected when the n-th value in the condition array is zero. Otherwise, the corresponding element from x will be picked.

• x (relay.Expr) – The first array to be selected.

• y (relay.Expr) – The second array to be selected.

Returns

result – The selected array.

Return type

relay.Expr

Examples

x = [[1, 2], [3, 4]]
y = [[5, 6], [7, 8]]
condition = [[0, 1], [-1, 0]]
relay.where(conditon, x, y) = [[5, 2], [3, 8]]

condition = [1, 0]
relay.where(conditon, x, y) = [[1, 2], [7, 8]]

tvm.relay.argmax(data, axis=None, keepdims=False, exclude=False)

Returns the indices of the maximum values along an axis.

Parameters
• data (relay.Expr) – The input data

• axis (None or int or tuple of int) – Axis or axes along which a argmax operation is performed. The default, axis=None, will find the indices of the maximum element of the elements of the input array. If axis is negative it counts from the last to the first axis.

• keepdims (bool) – If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.

• exclude (bool) – If exclude is true, reduction will be performed on the axes that are NOT in axis instead.

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.argmin(data, axis=None, keepdims=False, exclude=False)

Returns the indices of the minimum values along an axis.

Parameters
• data (relay.Expr) – The input data

• axis (None or int or tuple of int) – Axis or axes along which a argmin operation is performed. The default, axis=None, will find the indices of minimum element all of the elements of the input array. If axis is negative it counts from the last to the first axis.

• keepdims (bool) – If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.

• exclude (bool) – If exclude is true, reduction will be performed on the axes that are NOT in axis instead.

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.sum(data, axis=None, keepdims=False, exclude=False)

Computes the sum of array elements over given axes.

Parameters
• data (relay.Expr) – The input data

• axis (None or int or tuple of int) – Axis or axes along which a sum is performed. The default, axis=None, will sum all of the elements of the input array. If axis is negative it counts from the last to the first axis.

• keepdims (bool) – If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.

• exclude (bool) – If exclude is true, reduction will be performed on the axes that are NOT in axis instead.

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.max(data, axis=None, keepdims=False, exclude=False)

Computes the max of array elements over given axes.

Parameters
• data (relay.Expr) – The input data

• axis (None or int or tuple of int) – Axis or axes along which the max operation is performed. The default, axis=None, will find the max element from all of the elements of the input array. If axis is negative it counts from the last to the first axis.

• keepdims (bool) – If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.

• exclude (bool) – If exclude is true, reduction will be performed on the axes that are NOT in axis instead.

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.min(data, axis=None, keepdims=False, exclude=False)

Computes the min of array elements over given axes.

Parameters
• data (relay.Expr) – The input data

• axis (None or int or tuple of int) – Axis or axes along which a minimum operation is performed. The default, axis=None, will find the minimum element from all of the elements of the input array. If axis is negative it counts from the last to the first axis.

• keepdims (bool) – If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.

• exclude (bool) – If exclude is true, reduction will be performed on the axes that are NOT in axis instead.

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.mean(data, axis=None, keepdims=False, exclude=False)

Computes the mean of array elements over given axes.

Parameters
• data (relay.Expr) – The input data

• axis (None or int or tuple of int) – Axis or axes along which a mean operation is performed. The default, axis=None, will compute the mean of all elements in the input array. If axis is negative it counts from the last to the first axis.

• keepdims (bool) – If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.

• exclude (bool) – If exclude is true, reduction will be performed on the axes that are NOT in axis instead.

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.variance(data, axis=None, keepdims=False, exclude=False)

Computes the variance of data over given axes.

Parameters
• data (relay.Expr) – The input data

• axis (None or int or tuple of int) – Axis or axes along which a variance operation is performed. The default, axis=None, will compute the variance of all elements in the input array. If axis is negative it counts from the last to the first axis.

• keepdims (bool) – If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.

• exclude (bool) – If exclude is true, reduction will be performed on the axes that are NOT in axis instead.

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.std(data, axis=None, keepdims=False, exclude=False)

Computes the standard deviation of data over given axes.

Parameters
• data (relay.Expr) – The input data

• axis (None or int or tuple of int) – Axis or axes along which a standard deviation operation is performed. The default, axis=None, will compute the standard deviation of all elements in the input array. If axis is negative it counts from the last to the first axis.

• keepdims (bool) – If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.

• exclude (bool) – If exclude is true, reduction will be performed on the axes that are NOT in axis instead.

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.mean_variance(data, axis=None, keepdims=False, exclude=False)

Computes the mean and variance of data over given axes.

Parameters
• data (relay.Expr) – The input data

• axis (None or int or tuple of int) – Axis or axes along which a mean and variance operation is performed. The default, axis=None, will compute the mean and variance of all elements in the input array. If axis is negative it counts from the last to the first axis.

• keepdims (bool) – If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.

• exclude (bool) – If exclude is true, reduction will be performed on the axes that are NOT in axis instead.

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.mean_std(data, axis=None, keepdims=False, exclude=False)

Computes the mean and standard deviation of data over given axes.

Parameters
• data (relay.Expr) – The input data

• axis (None or int or tuple of int) – Axis or axes along which a mean and standard deviation operation is performed. The default, axis=None, will compute the mean and standard deviation of all elements in the input array. If axis is negative it counts from the last to the first axis.

• keepdims (bool) – If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.

• exclude (bool) – If exclude is true, reduction will be performed on the axes that are NOT in axis instead.

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.prod(data, axis=None, keepdims=False, exclude=False)

Computes the products of array elements over given axes.

Parameters
• data (relay.Expr) – The input data

• axis (None or int or tuple of int) – Axis or axes along which a product is performed. The default, axis=None, will find the indices of minimum element all of the elements of the input array. If axis is negative it counts from the last to the first axis.

• keepdims (bool) – If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.

• exclude (bool) – If exclude is true, reduction will be performed on the axes that are NOT in axis instead.

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.strided_slice(data, begin, end, strides=None)

Strided slice of an array.

Parameters
• data (relay.Expr) – The source array to be sliced.

• begin (list of int) – The indices to begin with in the slicing.

• end (list of int) – Indices indicating end of the slice.

• strides (list of int, optional) – Specifies the stride values, it can be negative in that case, the input tensor will be reversed in that particular axis.

Returns

ret – The computed result.

Return type

relay.Expr

tvm.relay.broadcast_to(data, shape)

Return a scalar value array with the same type, broadcast to the provided shape.

Parameters
• data (relay.Expr) – The input tensor.

• shape (shape) – Provide the shape to broadcast to.

Returns

result – The resulting tensor.

Return type

relay.Expr

## Level 5 Definitions¶

tvm.relay.image.resize(data, size, layout='NCHW', method='bilinear', coordinate_transformation_mode='half_pixel', out_dtype=None)

Image resize operator.

This operator takes data as input and does 2D scaling to the given scale factor. In the default case, where the data_layout is NCHW with data of shape (n, c, h, w) out will have a shape (n, c, size[0], size[1])

method indicates the algorithm to be used while calculating the out value and method can be one of (“bilinear”, “nearest_neighbor”, “bicubic”)

Parameters
• data (relay.Expr) – The input data to the operator.

• size (Tuple of Expr) – The out size to which the image will be resized.

• layout (str, optional) – Layout of the input.

• method (str, optional) – Scale method to used [nearest_neighbor, bilinear, bicubic].

• coordinate_transformation_mode (string, optional) – Describes how to transform the coordinate in the resized tensor to the coordinate in the original tensor. Refer to the ONNX Resize operator specification for details. [half_pixel, align_corners, asymmetric]

• out_dtype (str, optional) – Type to return. If left None returns the same type as input.

Returns

result – The resized result.

Return type

relay.Expr

tvm.relay.image.crop_and_resize(data, boxes, box_indices, crop_size, layout, method='bilinear', extrapolation_value=0, out_dtype=None)

Crop input images and resize them.

method indicates the algorithm to be used while calculating the out value and method can be either “bilinear” or “nearest_neighbor”.

Parameters
• data (relay.Expr) – The input data to the operator.

• boxes (relay.Expr) – A 2-D tensor of shape [num_boxes, 4]. Each row of the tensor specifies the coordinates of a box.

• box_indices (relay.Expr) – A 1-D tensor of shape [num_boxes], box_ind[i] specifies the data that the i-th box refers to.

• crop_size (Tuple of Expr) – The target size to which each box will be resized.

• layout (str, optional) – Layout of the input.

• method (str, optional) – Scale method, it can be either “nearest_neighbor” or “bilinear”.

• extrapolation_value (float, optional) – Value used for extrapolation, when applicable.

• out_dtype (str, optional) – Type to return. If left None returns the same type as input.

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.vision.multibox_prior(data, sizes=(1.0, ), ratios=(1.0, ), steps=(-1.0, -1.0), offsets=(0.5, 0.5), clip=False)

Generate prior(anchor) boxes from data, sizes and ratios.

Parameters
• data (relay.Expr) – The input data tensor.

• sizes (tuple of float, optional) – Tuple of sizes for anchor boxes.

• ratios (tuple of float, optional) – Tuple of ratios for anchor boxes.

• steps (Tuple of float, optional) – Priorbox step across y and x, -1 for auto calculation.

• offsets (tuple of int, optional) – Priorbox center offsets, y and x respectively.

• clip (boolean, optional) – Whether to clip out-of-boundary boxes.

Returns

out – 3-D tensor with shape [1, h_in * w_in * (num_sizes + num_ratios - 1), 4]

Return type

relay.Expr

tvm.relay.vision.multibox_transform_loc(cls_prob, loc_pred, anchor, clip=True, threshold=0.01, variances=(0.1, 0.1, 0.2, 0.2))

Location transformation for multibox detection

Parameters
• cls_prob (tvm.relay.Expr) – Class probabilities.

• loc_pred (tvm.relay.Expr) – Location regression predictions.

• anchor (tvm.relay.Expr) – Prior anchor boxes.

• clip (boolean, optional) – Whether to clip out-of-boundary boxes.

• threshold (double, optional) – Threshold to be a positive prediction.

• variances (Tuple of float, optional) – variances to be decoded from box regression output.

Returns

ret

Return type

tuple of tvm.relay.Expr

tvm.relay.vision.nms()

Non-maximum suppression operations.

tvm.relay.vision.yolo_reorg(data, stride)

Yolo reorg operation used in darknet models. This layer shuffles the input tensor values based on the stride value. Along with the shuffling, it does the shape transform. If ‘(n, c, h, w)’ is the data shape and ‘s’ is stride, output shape is ‘(n, c*s*s, h/s, w/s)’ Example: data(1, 4, 2, 2) = [[[[ 0 1] [ 2 3]]

[[ 4 5] [ 6 7]] [[ 8 9] [10 11]] [[12 13] [14 15]]]]

stride = 2 ret(1, 16, 1, 1) = [[[[ 0]] [[ 2]] [[ 8]] [[10]]

[[ 1]] [[ 3]] [[ 9]] [[11]] [[ 4]] [[ 6]] [[12]] [[14]] [[ 5]] [[ 7]] [[13]] [[15]]]]

Note: stride=1 has no significance for reorg operation.

Parameters
• data (relay.Expr) – The input data tensor.

• stride (int) – The stride value for reorganisation.

Returns

ret – The computed result.

Return type

relay.Expr

## Level 6 Definitions¶

tvm.relay.argsort(data, axis=-1, is_ascend=1, dtype='int32')

Performs sorting along the given axis and returns an array of indicies having same shape as an input array that index data in sorted order.

Parameters
• data (relay.Expr) – The input data tensor.

• valid_count (tvm.Tensor) – The number of valid elements to be sorted.

• axis (int, optional) – Axis long which to sort the input tensor.

• is_ascend (boolean, optional) – Whether to sort in ascending or descending order.

• dtype (string, optional) – The data type of the output indices.

Returns

out – Tensor with same shape as data.

Return type

relay.Expr

tvm.relay.topk(data, k=1, axis=-1, ret_type='both', is_ascend=False, dtype='int32')

Get the top k elements in an input tensor along the given axis.

ret_type specifies the return type, can be one of (“both”, “values”, “indices”).

Parameters
• data (relay.Expr) – The input data tensor.

• k (int, optional) – Number of top elements to select. Return all elements if k < 1.

• axis (int, optional) – Axis long which to sort the input tensor.

• ret_type (str, optional) – The return type [both, values, indices]. “both”: return both top k data and indices. “values”: return top k data only. “indices”: return top k indices only.

• is_ascend (boolean, optional) – Whether to sort in ascending or descending order.

• dtype (string, optional) – The data type of the indices output.

Returns

out – The computed result.

Return type

relay.Expr or List[relay.Expr]

## Level 10 Definitions¶

tvm.relay.broadcast_to_like(data, broadcast_type)

Return a scalar value array with the same shape and type as the input array.

Parameters
• data (relay.Expr) – The input tensor.

• broadcast_type (relay.Expr) – Provide the type to broadcast to.

Returns

result – The resulting tensor.

Return type

relay.Expr

tvm.relay.collapse_sum_like(data, collapse_type)

Return a scalar value array with the same shape and type as the input array.

Parameters
• data (relay.Expr) – The input tensor.

• collapse_type (relay.Expr) – Provide the type to collapse to.

Returns

result – The resulting tensor.

Return type

relay.Expr

tvm.relay.slice_like(data, shape_like, axes=None)

Slice the first input with respect to the second input.

For an input array with shape (d1, d2, ..., dk), slice_like operation slices the the input array corresponding size of second array. By default will slice on all axes.

Parameters
• data (tvm.relay.Expr) – The source array.

• shape_like (tvm.relay.Expr) – The new shape.

• axes (Optional[Tuple[int]]) – List of axes on which input data will be sliced according to the corresponding size of the second input. By default will slice on all axes. Negative axes mean counting in reverse.

Returns

result – The computed result.

Return type

relay.Expr

tvm.relay.shape_of(data, dtype='int32')

Get shape of a tensor.

Parameters
• data (tvm.relay.Expr) – The input tensor.

• dtype (str, optional) – The target data type.

Returns

result – The shape tensor.

Return type

tvm.relay.Expr

tvm.relay.contrib.ndarray_size(data, dtype='int32')

Get number of elements of input tensor.

Parameters
• data (tvm.relay.Expr) – The input tensor.

• dtype (str, optional) – The target data type.

Returns

result – The number of elements of input tensor.

Return type

tvm.relay.Expr

tvm.relay.layout_transform(data, src_layout, dst_layout)

Transform the layout of a tensor

Parameters
• data (relay.Expr) – The source tensor to be transformed

• src_layout (str) – The source layout. (e.g NCHW)

• dst_layout (str) – The destination layout. (e.g. NCHW16c)

Returns

ret – The transformed tensor.

Return type

relay.Expr

tvm.relay.device_copy(data, src_dev, dst_dev)

Copy data from the source device to the destination device. This operator helps data transferring between difference contexts for heterogeneous execution.

Parameters
• data (tvm.relay.Expr) – The tensor to be copied.

• src_dev (Union[TVMContext, str]) – The source device where the data is copied from.

• dst_dev (Union[TVMContext, str]) – The destination device where the data is copied to.

Returns

result – The copied result.

Return type

tvm.relay.Expr

tvm.relay.annotation.on_device(data, device)

Annotate an expression with a certain device type.

Parameters
• data (tvm.relay.Expr) – The expression to be annotated.

• device (Union[TVMContext, str]) – The device type to annotate.

Returns

result – The annotated expression.

Return type

tvm.relay.Expr

tvm.relay.reverse_reshape(data, newshape)

Reshapes the input array where the special values are inferred from right to left.

Example:

The special values have the same semantics as tvm.relay.reshape. The difference is that special values are inferred from right to left. It can be explained in the example below:

- data.shape = (10,5,4), newshape = (-1,0), reshape results in (40,5)
- data.shape = (10,5,4), newshape = (-1,0), reverse_reshape results in (40,5)

Parameters
• data (relay.Expr) – The input data to the operator.

• newshape (Union[int, Tuple[int], List[int]]) – The new shape. Should be compatible with the original shape.

Returns

result – The reshaped result.

Return type

relay.Expr

tvm.relay.sequence_mask(data, valid_length, mask_value=0, axis=0)

Sets all elements outside the expected length of the sequence to a constant value.

This function takes an n-dimensional input array of the form [MAX_LENGTH, batch_size, …] or [batch_size, MAX_LENGTH, …] and returns an array of the same shape.

Parameters
• data (relay.Expr) – The input data.

• valid_length (relay.Expr) – The expected (valid) length of each sequence in the tensor.

• mask_value (float) – The masking value.

• axis (int) – The axis of the length dimension.

Returns

ret – The computed result.

Return type

relay.Expr

Examples

 x = [[[  1.,   2.,   3.], [  4.,   5.,   6.]],
[[  7.,   8.,   9.], [ 10.,  11.,  12.]],
[[ 13.,  14.,   15.], [ 16.,  17.,   18.]]]

relay.sequence_mask(x, valid_length=[1, 1]) =
[[[  1.,   2.,   3.], [  4.,   5.,   6.]],
[[  0.,   0.,   0.], [  0.,   0.,   0.]],
[[  0.,   0.,   0.], [  0.,   0.,   0.]]]

[[[  1.,   2.,   3.], [  4.,   5.,   6.]],
[[  7.,   8.,   9.], [  10.,  11.,  12.]],
[[  0.1,  0.1,  0.1], [  16.,  17.,  18.]]]

tvm.relay.nn.batch_matmul(x, y)

Computes batch matrix multiplication of x and y when x and y are data in batch.

$\mbox{batch_matmul}(x, y)[i, :, :] = \mbox{matmul}(x[i, :, :], y[i, :, :]^T)$
Parameters
• x (tvm.relay.Expr) – The first input.

• y (tvm.relay.Expr) – The second input.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.contrib.adaptive_max_pool2d(data, output_size=None, layout='NCHW')

2D adaptive max pooling operator. This operator is experimental.

This operator takes data as input and does 2D max value calculation across each window represented by WxH.

In the default case, where the data_layout is NCHW a data Tensor with shape (batch_size, in_channels, height, width), to produce an output Tensor with shape (batch_size, in_channels, output_height, output_width).

The pooling kernel and stride sizes are automatically chosen for desired output sizes.

For output_size:

If this argument is not provided, input height and width will be used as output height and width.

If a single integer is provided for output_size, the output size is (N x C x output_size x output_size) for any input (NCHW).

If a tuple of integers (height, width) are provided for output_size, the output size is (N x C x height x width) for any input (NCHW).

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• output_size (tuple of int. optional) – Output height and width.

• layout (str, optional) – Layout of the input.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.contrib.adaptive_avg_pool2d(data, output_size=None, layout='NCHW')

2D adaptive average pooling operator. This operator is experimental.

This operator takes data as input and does 2D average value calculation across each window represented by WxH.

In the default case, where the data_layout is NCHW a data Tensor with shape (batch_size, in_channels, height, width), to produce an output Tensor with shape (batch_size, in_channels, output_height, output_width).

The pooling kernel and stride sizes are automatically chosen for desired output sizes.

For output_size:

If this argument is not provided, input height and width will be used as output height and width.

If a single integer is provided for output_size, the output size is (N x C x output_size x output_size) for any input (NCHW).

If a tuple of integers (height, width) are provided for output_size, the output size is (N x C x height x width) for any input (NCHW).

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• output_size (tuple of int. optional) – Output height and width.

• layout (str, optional) – Layout of the input.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.one_hot(indices, on_value, off_value, depth, axis, dtype)

Returns a one-hot tensor where the locations repsented by indices take value on_value, other locations take value off_value. Final dimension is <indices outer dimensions> x depth x <indices inner dimensions>.

Parameters
• indices (relay.Expr) – Locations to set to on_value.

• on_value (relay.Expr) – Value to fill at indices.

• off_value (relay.Expr) – Value to fill at all other positions besides indices.

• depth (int) – Depth of the one-hot dimension.

• axis (int) – Axis to fill.

• dtype (str) – Data type of the output tensor.

Returns

ret – The one-hot tensor.

Return type

relay.Expr

Examples

indices = [0, 1, 2]

relay.one_hot(indices, 3) =
[[1, 0, 0],
[0, 1, 0],
[0, 0, 1]]


## Level 11 Definitions¶

tvm.relay.qnn.op.requantize(data, input_scale, input_zero_point, output_scale, output_zero_point, axis=-1, rounding='UPWARD', out_dtype='int8')

Requantized operator.

The requantize operator converts one quantized tensor representation to another quantized tensor representation. For the output tensor, we are provided with output scale and zero point. The computation is as follows

Q_output = zp_output + (scale_input)/(scale_output) * (Q_input - zp_input)

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• input_scale (tvm.relay.Expr) – The quantization scale for the input tensor.

• input_zero_point (tvm.relay.Expr) – The zero point of the input tensor.

• output_scale (tvm.relay.Expr) – The quantization scale for the output tensor.

• output_zero_point (tvm.relay.Expr) – The zero point of the output tensor.

• axis (int) – The channel axis for quantization. Default value is -1 which corresponds to the last axis.

• rounding (string, optional) – Defines the rounding direction when the value is midway between two representable values.

• out_dtype (str, optional) – Specifies the output data type.

Returns

result – The computed result.

Return type

tvm.relay.Expr

tvm.relay.qnn.op.conv2d(data, kernel, input_zero_point, kernel_zero_point, input_scale, kernel_scale, strides=(1, 1), padding=(0, 0), dilation=(1, 1), groups=1, channels=None, kernel_size=None, data_layout='NCHW', kernel_layout='OIHW', out_layout='', out_dtype='int32')

Quantized 2D convolution.

This operator convolves quantized data with quantized kernel. The scale of the output quantized tensor is the product of the kernel_scale and input_scale of the input quantized tensors. The zero point of the output quantized tensor is 0. By default, the dtype of output is int32. Please also refer to Requantize operator to understand how to scale back the int32 output to (u)int8.

Parameters
• data (tvm.relay.Expr) – The input data to the operator.

• kernel (tvm.relay.Expr) – The kernel expressions.

• input_zero_point (tvm.relay.Expr) – The zero point of the data distribution.

• kernel_zero_point (tvm.relay.Expr) – The zero point of the quantized_kernel distribution.

• input_scale (tvm.relay.Expr) – The scale for the input tensor. The scale for the input tensor is stored purely for convenience here. See more commentary below.

• kernel_scale (tvm.relay.Expr) – The scale for the weight tensor. The scale for the weight tensor is stored for access to this during relay. This information is not needed in the pass pipeline after qnn.conv2d is lowered to the sequence of steps as in nn.conv2d. See also input_scale in Requantize.

• strides (tuple of int, optional) – The strides of convolution.

• padding (tuple of int, optional) – The padding of convolution on both sides of inputs before convolution.

• dilation (tuple of int, optional) – Specifies the dilation rate to be used for dilated convolution.

• groups (int, optional) – Number of groups for grouped convolution.

• channels (int, optional) – Number of output channels of this convolution.

• kernel_size (tuple of int, optional) – The spatial of the convolution kernel.

• data_layout (str, optional) – Layout of the input.

• kernel_layout (str, optional) – Layout of the kernel.

• out_layout (str, optional) – Layout of the output, by default, out_layout is the same as data_layout

• out_dtype (str, optional) – Specifies the output data type for mixed precision conv2d.

Returns

result – The computed result.

Return type

tvm.relay.Expr