<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>BERT — FoundationIndex</title>
    <link>https://foundationindex.ustunozgur.com/s/bert</link>
    <description>Google&apos;s bidirectional Transformer architecture that reads text from both left and right at once—powering search ranking, embeddings, rerankers, and classification rather than chat generation.</description>
    <language>en</language>
    <atom:link href="https://foundationindex.ustunozgur.com/s/bert/feed.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>BERT · Paper · 2024-12-18</title>
      <link>https://foundationindex.ustunozgur.com/s/bert#bert-modernbert-2024</link>
      <guid>https://foundationindex.ustunozgur.com/s/bert#bert-modernbert-2024</guid>
      <pubDate>Wed, 18 Dec 2024 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Warner et al. released ModernBERT, updating the encoder-only Transformer with RoPE, GeGLU layers, alternating local/global attention, unpadding, and an 8,192-token context window trained on 2T tokens for modern embedding and RAG reranking pipelines.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2412.13663&quot;&gt;Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>BERT · Paper · 2019-07-26</title>
      <link>https://foundationindex.ustunozgur.com/s/bert#bert-roberta-2019</link>
      <guid>https://foundationindex.ustunozgur.com/s/bert#bert-roberta-2019</guid>
      <pubDate>Fri, 26 Jul 2019 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Liu et al. at Facebook AI (Meta) published RoBERTa, proving BERT was heavily undertrained and establishing a stronger encoder baseline by dropping Next Sentence Prediction, scaling batch sizes and data, and adopting dynamic masking with byte-level BPE.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/1907.11692&quot;&gt;RoBERTa: A Robustly Optimized BERT Pretraining Approach&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>BERT · Paper · 2018-10-11</title>
      <link>https://foundationindex.ustunozgur.com/s/bert#bert-google-paper-2018</link>
      <guid>https://foundationindex.ustunozgur.com/s/bert#bert-google-paper-2018</guid>
      <pubDate>Thu, 11 Oct 2018 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Devlin et al. at Google AI Language published BERT, showing that pre-training a deep bidirectional Transformer encoder via Masked Language Modeling produced a single backbone that swept 11 NLP benchmarks after lightweight fine-tuning.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/1810.04805&quot;&gt;BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding&lt;/a&gt;&lt;/p&gt;</description>
    </item>
  </channel>
</rss>
