nlp::BERT Class Reference

[3] More...

#include <bert.hpp>

Inheritance diagram for nlp::BERT:

Collaboration diagram for nlp::BERT:

Public Member Functions
	BERT (BERTConfig config)
virtual dl::TensorPtr	forward (const dl::TensorPtr &input) override

Public Attributes
BERTEmbeddings	embeddings
dl::Transformer	encoder
BERTPooling	pooling

Static Public Attributes
static constexpr dl::TransformerConf	transformerConf

Detailed Description

[3]

Definition at line 64 of file bert.hpp.

Constructor & Destructor Documentation

BERT()

nlp::BERT::BERT ( BERTConfig  config )

inline

Definition at line 76 of file bert.hpp.

                : embeddings(config, transformerConf), encoder(transformerConf), pooling(transformerConf) {
            registerSubmodel("bert.embeddings", embeddings);
            registerSubmodel("bert", encoder);
            registerSubmodel("bert.pooler", pooling);
        }

Member Function Documentation

forward()

virtual dl::TensorPtr nlp::BERT::forward ( const dl::TensorPtr &  input )

inlineoverridevirtual

Todo:

implement

Definition at line 83 of file bert.hpp.

                                                                       {
            throw std::runtime_error("Not yet implemented");
            // return pooling.forward(encoder.forward(embeddings.forward(input)));
        }

Member Data Documentation

embeddings

BERTEmbeddings nlp::BERT::embeddings

Definition at line 71 of file bert.hpp.

encoder

dl::Transformer nlp::BERT::encoder

Definition at line 72 of file bert.hpp.

pooling

BERTPooling nlp::BERT::pooling

Definition at line 73 of file bert.hpp.

transformerConf

constexpr dl::TransformerConf nlp::BERT::transformerConf

staticconstexpr

Initial value:

{
                .dimensions = {.model = 768, .key = 64, .value = 64, .inner = 3072},
                .numEncoders = 12,
                .numAttnHeads = 12
        }

Definition at line 66 of file bert.hpp.

                                                          {
                .dimensions = {.model = 768, .key = 64, .value = 64, .inner = 3072},
                .numEncoders = 12,
                .numAttnHeads = 12
        };

---

The documentation for this class was generated from the following file:

• nlp/transformer/bert.hpp