Nlp Highlights

52 - Sequence-to-Sequence Learning as Beam-Search Optimization, with Sam Wiseman

Informações:

Sinopse

EMNLP 2016 paper by Sam Wiseman and Sasha Rush. In this episode we talk with Sam about a paper from a couple of years ago on bringing back some ideas from structured prediction into neural seq2seq models. We talk about the classic problems in structured prediction of exposure bias, label bias, and locally normalized models, how people used to solve these problems, and how we can apply those solutions to modern neural seq2seq architectures using a technique that Sam and Sasha call Beam Search Optimization. (Note: while we said in the episode that BSO with beam size of 2 is equivalent to a token-level hinge loss, that's not quite accurate; it's close, but there are some subtle differences.) https://www.semanticscholar.org/paper/Sequence-to-Sequence-Learning-as-Beam-Search-Optim-Wiseman-Rush/28703eef8fe505e8bd592ced3ce52a597097b031

Experimente 7 dias grátis

Experimente 7 dias grátis

  • Acesso ilimitado a todo o conteúdo da plataforma.
  • Mais de 30 mil títulos, entre audiobooks, ebooks, podcasts, séries, documentários e notícias.
  • Narração dos audiolivros feita por profissionais, entre atores, locutores e até mesmo os próprios autores.
Experimente 7 dias Grátis Promoção válida para novos usuários. Após 7 dias, será cobrado valor integral. Cancele quando quiser.

Compartilhar