Embedding技术与实践（详细）.pdfVIP

Embedding技术与实践

语义理解中Embedding意义

importtorch

embeddingtorch.nn.Embedding(num_embeddings4,embedding_dim3)

words[[2,3],[1,2]]

embedembedding(torch.LongTensor(words))

print(embed)

print(embed.size())

tensor([[[0.4015,0.3967,0.8442],

[1.7117,0.2838,0.1853]],

[[-2.2015,0.3107,0.9713],

[0.4015,0.3967,0.8442]]],grad_fnEmbeddingBackward0)

torch.Size([2,2,3])

Word2Vec

最著名的密集表示方法之一是word2vec，由Google于2013年提出的论文：effectiveEstimationof

WordRepresentationsinVectorSpace/abs/1301.3781

代码示例

importtorch

importnumpyasnp

importtorch.nnasnn

importtorch.optimasoptim

importmatplotlib.pyplotasplt

importtorch.utils.dataasData

dtypetorch.FloatTensor

devicetorch.device(cudaiftorch.cuda.is_available()elsecpu)

#模型的相关参数

batch_size8

embedding_size2

#词向量的维度是2

C2

#windowsize

defprepare_data():

#文本预处理

sentences[longgelikedog,longgelikecat,longgelikeanimal,

dogcatanimal,bananaapplecatdoglike,dogfishmilk

like,

dogcatanimallike,longgelikeapple,applelike,longge

likebanana,

applebananalonggemoviebookmusiclike,catdoghate,cat

doglike]

word_sequence.join(sentences).split()

#[longge,like,dog,

longge,like,cat,animal,...]

v