deepctr.layers.sequence module¶
 Author:
 Weichen Shen,wcshen1994@163.com

class
deepctr.layers.sequence.
AttentionSequencePoolingLayer
(att_hidden_units=(80, 40), att_activation='sigmoid', weight_normalization=False, return_score=False, supports_masking=False, **kwargs)[source]¶ The Attentional sequence pooling operation used in DIN.
 Input shape
 A list of three tensor: [query,keys,keys_length]
 query is a 3D tensor with shape:
(batch_size, 1, embedding_size)
 keys is a 3D tensor with shape:
(batch_size, T, embedding_size)
 keys_length is a 2D tensor with shape:
(batch_size, 1)
 Output shape
 3D tensor with shape:
(batch_size, 1, embedding_size)
.
 3D tensor with shape:
 Arguments
 att_hidden_units:list of positive integer, the attention net layer number and units in each layer.
 att_activation: Activation function to use in attention net.
 weight_normalization: bool.Whether normalize the attention score of local activation unit.
 supports_masking:If True,the input need to support masking.
 References
 [Zhou G, Zhu X, Song C, et al. Deep interest network for clickthrough rate prediction[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2018: 10591068.](https://arxiv.org/pdf/1706.06978.pdf)

call
(inputs, mask=None, training=None, **kwargs)[source]¶ This is where the layer’s logic lives.
 Arguments:
 inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
 Returns:
 A tensor or list/tuple of tensors.

compute_mask
(inputs, mask)[source]¶ Computes an output mask tensor.
 Arguments:
 inputs: Tensor or list of tensors. mask: Tensor or list of tensors.
 Returns:
 None or a tensor (or list of tensors,
 one per output tensor of the layer).

compute_output_shape
(input_shape)[source]¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
 Arguments:
 input_shape: Shape tuple (tuple of integers)
 or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
 Returns:
 An input shape tuple.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns:
 Python dictionary.

class
deepctr.layers.sequence.
BiLSTM
(units, layers=2, res_layers=0, dropout_rate=0.2, merge_mode='ave', **kwargs)[source]¶ A multiple layer Bidirectional Residual LSTM Layer.
 Input shape
 3D tensor with shape
(batch_size, timesteps, input_dim)
.
 3D tensor with shape
 Output shape
 3D tensor with shape:
(batch_size, timesteps, units)
.
 3D tensor with shape:
 Arguments
 units: Positive integer, dimensionality of the output space.
 layers:Positive integer, number of LSTM layers to stacked.
 res_layers: Positive integer, number of residual connection to used in last
res_layers
.  dropout_rate: Float between 0 and 1. Fraction of the units to drop for the linear transformation of the inputs.
 merge_mode: merge_mode: Mode by which outputs of the forward and backward RNNs will be combined. One of {
'fw'
,'bw'
,'sum'
,'mul'
,'concat'
,'ave'
,None
}. If None, the outputs will not be combined, they will be returned as a list.

call
(inputs, mask=None, **kwargs)[source]¶ This is where the layer’s logic lives.
 Arguments:
 inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
 Returns:
 A tensor or list/tuple of tensors.

compute_mask
(inputs, mask)[source]¶ Computes an output mask tensor.
 Arguments:
 inputs: Tensor or list of tensors. mask: Tensor or list of tensors.
 Returns:
 None or a tensor (or list of tensors,
 one per output tensor of the layer).

compute_output_shape
(input_shape)[source]¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
 Arguments:
 input_shape: Shape tuple (tuple of integers)
 or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
 Returns:
 An input shape tuple.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns:
 Python dictionary.

class
deepctr.layers.sequence.
BiasEncoding
(sess_max_count, seed=1024, **kwargs)[source]¶ 

call
(inputs, mask=None)[source]¶ Parameters: concated_embeds_value – None * field_size * embedding_size Returns: None*1

compute_mask
(inputs, mask=None)[source]¶ Computes an output mask tensor.
 Arguments:
 inputs: Tensor or list of tensors. mask: Tensor or list of tensors.
 Returns:
 None or a tensor (or list of tensors,
 one per output tensor of the layer).

compute_output_shape
(input_shape)[source]¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
 Arguments:
 input_shape: Shape tuple (tuple of integers)
 or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
 Returns:
 An input shape tuple.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns:
 Python dictionary.


class
deepctr.layers.sequence.
DynamicGRU
(num_units=None, gru_type='GRU', return_sequence=True, **kwargs)[source]¶ 

call
(input_list)[source]¶ Parameters: concated_embeds_value – None * field_size * embedding_size Returns: None*1

compute_output_shape
(input_shape)[source]¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
 Arguments:
 input_shape: Shape tuple (tuple of integers)
 or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
 Returns:
 An input shape tuple.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns:
 Python dictionary.


class
deepctr.layers.sequence.
KMaxPooling
(k=1, axis=1, **kwargs)[source]¶ K Max pooling that selects the k biggest value along the specific axis.
 Input shape
 nD tensor with shape:
(batch_size, ..., input_dim)
.
 nD tensor with shape:
 Output shape
 nD tensor with shape:
(batch_size, ..., output_dim)
.
 nD tensor with shape:
 Arguments
 k: positive integer, number of top elements to look for along the
axis
dimension.  axis: positive integer, the dimension to look for elements.
 k: positive integer, number of top elements to look for along the

call
(inputs)[source]¶ This is where the layer’s logic lives.
 Arguments:
 inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
 Returns:
 A tensor or list/tuple of tensors.

compute_output_shape
(input_shape)[source]¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
 Arguments:
 input_shape: Shape tuple (tuple of integers)
 or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
 Returns:
 An input shape tuple.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns:
 Python dictionary.

class
deepctr.layers.sequence.
SequencePoolingLayer
(mode='mean', supports_masking=False, **kwargs)[source]¶ The SequencePoolingLayer is used to apply pooling operation(sum,mean,max) on variablelength sequence feature/multivalue feature.
 Input shape
 A list of two tensor [seq_value,seq_len]
 seq_value is a 3D tensor with shape:
(batch_size, T, embedding_size)
 seq_len is a 2D tensor with shape :
(batch_size, 1)
,indicate valid length of each sequence.
 Output shape
 3D tensor with shape:
(batch_size, 1, embedding_size)
.
 3D tensor with shape:
 Arguments
 mode:str.Pooling operation to be used,can be sum,mean or max.
 supports_masking:If True,the input need to support masking.

call
(seq_value_len_list, mask=None, **kwargs)[source]¶ This is where the layer’s logic lives.
 Arguments:
 inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
 Returns:
 A tensor or list/tuple of tensors.

compute_mask
(inputs, mask)[source]¶ Computes an output mask tensor.
 Arguments:
 inputs: Tensor or list of tensors. mask: Tensor or list of tensors.
 Returns:
 None or a tensor (or list of tensors,
 one per output tensor of the layer).

compute_output_shape
(input_shape)[source]¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
 Arguments:
 input_shape: Shape tuple (tuple of integers)
 or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
 Returns:
 An input shape tuple.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns:
 Python dictionary.

class
deepctr.layers.sequence.
Transformer
(att_embedding_size=1, head_num=8, dropout_rate=0.0, use_positional_encoding=True, use_res=True, use_feed_forward=True, use_layer_norm=False, blinding=True, seed=1024, supports_masking=False, **kwargs)[source]¶ Simplified version of Transformer proposed in 《Attention is all you need》
 Input shape
 a list of two 3D tensor with shape
(batch_size, timesteps, input_dim)
if supports_masking=True.  a list of two 4 tensors, first two tensors with shape
(batch_size, timesteps, input_dim)
,last two tensors with shape(batch_size, 1)
if supports_masking=False.
 a list of two 3D tensor with shape
 Output shape
 3D tensor with shape:
(batch_size, 1, input_dim)
.
 3D tensor with shape:
 Arguments
 att_embedding_size: int.The embedding size in multihead selfattention network.
 head_num: int.The head number in multihead selfattention network.
 dropout_rate: float between 0 and 1. Fraction of the units to drop.
 use_positional_encoding: bool. Whether or not use positional_encoding
 use_res: bool. Whether or not use standard residual connections before output.
 use_feed_forward: bool. Whether or not use pointwise feed foward network.
 use_layer_norm: bool. Whether or not use Layer Normalization.
 blinding: bool. Whether or not use blinding.
 seed: A Python integer to use as random seed.
 supports_masking:bool. Whether or not support masking.
 References
 [Vaswani, Ashish, et al. “Attention is all you need.” Advances in Neural Information Processing Systems. 2017.](https://papers.nips.cc/paper/7181attentionisallyouneed.pdf)

call
(inputs, mask=None, training=None, **kwargs)[source]¶ This is where the layer’s logic lives.
 Arguments:
 inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
 Returns:
 A tensor or list/tuple of tensors.

compute_mask
(inputs, mask=None)[source]¶ Computes an output mask tensor.
 Arguments:
 inputs: Tensor or list of tensors. mask: Tensor or list of tensors.
 Returns:
 None or a tensor (or list of tensors,
 one per output tensor of the layer).

compute_output_shape
(input_shape)[source]¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
 Arguments:
 input_shape: Shape tuple (tuple of integers)
 or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
 Returns:
 An input shape tuple.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns:
 Python dictionary.

class
deepctr.layers.sequence.
WeightedSequenceLayer
(weight_normalization=True, supports_masking=False, **kwargs)[source]¶ The WeightedSequenceLayer is used to apply weight score on variablelength sequence feature/multivalue feature.
 Input shape
 A list of two tensor [seq_value,seq_len,seq_weight]
 seq_value is a 3D tensor with shape:
(batch_size, T, embedding_size)
 seq_len is a 2D tensor with shape :
(batch_size, 1)
,indicate valid length of each sequence.  seq_weight is a 3D tensor with shape:
(batch_size, T, 1)
 Output shape
 3D tensor with shape:
(batch_size, T, embedding_size)
.
 3D tensor with shape:
 Arguments
 weight_normalization: bool.Whether normalize the weight score before applying to sequence.
 supports_masking:If True,the input need to support masking.

call
(input_list, mask=None, **kwargs)[source]¶ This is where the layer’s logic lives.
 Arguments:
 inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
 Returns:
 A tensor or list/tuple of tensors.

compute_mask
(inputs, mask)[source]¶ Computes an output mask tensor.
 Arguments:
 inputs: Tensor or list of tensors. mask: Tensor or list of tensors.
 Returns:
 None or a tensor (or list of tensors,
 one per output tensor of the layer).

compute_output_shape
(input_shape)[source]¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
 Arguments:
 input_shape: Shape tuple (tuple of integers)
 or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
 Returns:
 An input shape tuple.

get_config
()[source]¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
 Returns:
 Python dictionary.

deepctr.layers.sequence.
positional_encoding
(inputs, pos_embedding_trainable=True, zero_pad=False, scale=True)[source]¶ Sinusoidal Positional_Encoding.
Args:
 inputs: A 2d Tensor with shape of (N, T).
 num_units: Output dimensionality
 zero_pad: Boolean. If True, all the values of the first row (id = 0) should be constant zero
 scale: Boolean. If True, the output will be multiplied by sqrt num_units(check details from paper)
 scope: Optional scope for variable_scope.
 reuse: Boolean, whether to reuse the weights of a previous layer by the same name.
Returns:
 A ‘Tensor’ with one more rank than inputs’s, with the dimensionality should be ‘num_units’