nlp::BERTEmbeddings Class Reference

Inheritance diagram for nlp::BERTEmbeddings:

Collaboration diagram for nlp::BERTEmbeddings:

Public Member Functions
	BERTEmbeddings (BERTConfig bertConf, dl::TransformerConf config)
virtual dl::TensorPtr	forward (const dl::TensorPtr &inputIds, const dl::TensorPtr &inputTokenTypes)

Detailed Description

Definition at line 18 of file bert.hpp.

Constructor & Destructor Documentation

BERTEmbeddings()

nlp::BERTEmbeddings::BERTEmbeddings ( BERTConfig  bertConf, dl::TransformerConf  config  )

inline

Definition at line 26 of file bert.hpp.

                : wordEmbeddings(bertConf.vocabSize, config.dimensions.model),
                  positionalEmbeddings(bertConf.maxPositionEmbeddings, config.dimensions.model),
                  tokenTypeEmbeddings(bertConf.typeVocabSize, config.dimensions.model),
                  layerNorm({config.dimensions.model}) {
            registerSubmodel("word_embeddings", wordEmbeddings);
            registerSubmodel("position_embeddings", positionalEmbeddings);
            registerSubmodel("token_type_embeddings", tokenTypeEmbeddings);
            registerSubmodel("LayerNorm", layerNorm);
        }

Member Function Documentation

forward()

virtual dl::TensorPtr nlp::BERTEmbeddings::forward ( const dl::TensorPtr &  inputIds, const dl::TensorPtr &  inputTokenTypes  )

inlinevirtual

Note that the embeddings returned are references to tensors instead of tensors since they return the references to the learned embeddings.

Todo:

implement

Definition at line 37 of file bert.hpp.

                                                                                                     {
            // auto& inputEmbeds = wordEmbeddings.forward(std::forward<decltype(inputIds)>(inputIds));
            // auto& typeEmbeds = tokenTypeEmbeddings.forward(std::forward<decltype(inputTokenTypes)>(inputTokenTypes));
            // return layerNorm.forward(inputEmbeds + typeEmbeds + posEmbeds);
            throw std::runtime_error("Not yet implemented");
        }

---

The documentation for this class was generated from the following file:

• nlp/transformer/bert.hpp