<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Vision Transformer (ViT) — FoundationIndex</title>
    <link>https://foundationindex.ustunozgur.com/s/vision-transformer</link>
    <description>An architecture that applies the Transformer directly to images by slicing a picture into a grid of small square patches and treating each patch like a word in a sentence.</description>
    <language>en</language>
    <atom:link href="https://foundationindex.ustunozgur.com/s/vision-transformer/feed.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>Vision Transformer (ViT) · Paper · 2025-02-20</title>
      <link>https://foundationindex.ustunozgur.com/s/vision-transformer#vit-siglip2-native-aspect-2025</link>
      <guid>https://foundationindex.ustunozgur.com/s/vision-transformer#vit-siglip2-native-aspect-2025</guid>
      <pubDate>Thu, 20 Feb 2025 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Google DeepMind&amp;apos;s SigLIP 2 unified NaViT aspect-preserving patch encoders with sigmoid contrastive, captioning, and self-distillation objectives, providing open-weight ViT backbones tailored for dense OCR, localization, and vision-language models.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2502.14786&quot;&gt;SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>Vision Transformer (ViT) · Paper · 2023-07-12</title>
      <link>https://foundationindex.ustunozgur.com/s/vision-transformer#vit-navit-2023</link>
      <guid>https://foundationindex.ustunozgur.com/s/vision-transformer#vit-navit-2023</guid>
      <pubDate>Wed, 12 Jul 2023 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Dehghani et al. at Google DeepMind published NaViT (Patch n&amp;apos; Pack), packing variable-resolution and native-aspect-ratio patch sequences into single training batches with factorized 2D positional embeddings rather than distorting images into fixed squares.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2307.06304&quot;&gt;Patch n&amp;apos; Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>Vision Transformer (ViT) · Paper · 2021-11-11</title>
      <link>https://foundationindex.ustunozgur.com/s/vision-transformer#vit-mae-2021</link>
      <guid>https://foundationindex.ustunozgur.com/s/vision-transformer#vit-mae-2021</guid>
      <pubDate>Thu, 11 Nov 2021 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Kaiming He et al. at Meta AI introduced Masked Autoencoders (MAE), masking 75% of input image patches and applying the ViT encoder only to the visible 25% before a lightweight decoder reconstructs the missing pixels—cutting self-supervised pretraining compute by over 3x.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2111.06377&quot;&gt;Masked Autoencoders Are Scalable Vision Learners&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>Vision Transformer (ViT) · Paper · 2020-10-22</title>
      <link>https://foundationindex.ustunozgur.com/s/vision-transformer#vit-paper-2020</link>
      <guid>https://foundationindex.ustunozgur.com/s/vision-transformer#vit-paper-2020</guid>
      <pubDate>Thu, 22 Oct 2020 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Alexey Dosovitskiy et al. at Google Research published An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ViT), showing that slicing images into 16x16 patch tokens and feeding them into a standard Transformer outperforms ResNets when pretrained on large image corpora.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2010.11929&quot;&gt;An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale&lt;/a&gt;&lt;/p&gt;</description>
    </item>
  </channel>
</rss>
