Staff research scientist at Google Brain. DeepRank, NoisyStudent, ELECTRA. PhD @StanfordNLP, LuongAttention. Co-founder @vietaiorg and #MeenaBot (now #LaMDA).

United States
Joined December 2008
Pinned Tweet
Glad to share our recent success in applying semi-supervised learning to Google's most fundamental product, Search. Our approach, semi-supervised distillation (SSD), is a simplified version of Noisy Student and an alternative to product distillation. Blog: ai.googleblog.com/2021/07/fr…
3
113
3
583
Thang Luong retweeted
Have you wondered why object detection, unlike classification, has so many sophisticated algorithms? With Pix2Seq (arxiv.org/abs/2109.10852), we simply cast object detection as a language modeling task conditioned on pixels! (with @srbhsxn, Lala Li, @fleet_dj, @geoffreyhinton)
7
41
5
202
Show this thread
Thang Luong retweeted
Searching for code snippets and questions can be frustrating and time-consuming. Until now: you.com will help you solve your coding questions faster and summarize the relevant web for you. Check out our newest #AI feature of code completion and join our beta :)
7
58
4
240
9,602
Thang Luong retweeted
Primer: Searching for Efficient Transformers for Language Modeling arxiv.org/abs/2109.08668 We use evolution to design a new Transformer variant, called Primer. Primer has a better scaling law, and is 3X to 4X faster for training than Transformer for language modeling.
2
121
22
567
Show this thread
Glad to share STraTA (Self-training /w Task Augmentation), a framework to turn any pretrained (or random!) model to a strong few-shot learner. Idea: better use of unlabeled data with an auxiliary task & self-training on broad distribution of pseudo-labels arxiv.org/abs/2109.06270.
Excited to announce our #EMNLP2021 paper that shows how to turn a pre-trained language model or even a randomly initialized model into a strong few-shot learner. Paper: arxiv.org/abs/2109.06270 w/ amazing collaborators: @lmthang, @quocleix, @GradySimon, @MohitIyyer 1/9👇
Show this thread
1
8
0
22
Thang Luong retweeted
Large language models can perform zero-shot learning pretty well if you finetune them on NLP tasks verbalized with instructions. Link: arxiv.org/abs/2109.01652
3
43
4
207
Show this thread
Thang Luong retweeted
Wanted to do speech recognition in Vietnamese? Check out this SOTA self-supervised learning work (13K hours unlabeled Youtube audio, 250 hours labeled speech) by @nguyenvulebinh, VietAI team, also @GoogleDevExpert in ML. Code: github.com/vietai/ASR Demo: huggingface.co/nguyenvulebin…
I want to introduce my work in training the first large-scale Wav2Vec2 model with 13k hours of Vietnamese audio and fine-tuning on 250 hours labeled of the VLSP ASR dataset. Thank @huggingface and @vietaiorg for hosting my model. github.com/vietai/ASR
0
6
0
13
Working with multi models & datasets is complex; there are often wins but also losses. So it's very cool to see the clear success of MuST that extends self-training to the multi-task setting. Congrats to the authors! arxiv.org/abs/2108.11353
How do we combine knowledge from multiple labeled and unlabeled datasets to train a great general model? Multi-Task Self-Training (MuST) trains specialized teachers on labeled data, which then label unlabeled data to train a single general model. arxiv.org/abs/2108.11353
Show this thread
1
5
0
27
Thang Luong retweeted
I am pleased to announce that the camera ready version of my new textbook, "Probabilistic Machine Learning: An Introduction", is finally available from probml.ai. Hardcopies will be available from MIT Press in Feb 2022.
47
811
41
3,951
Show this thread
Sad to learn about the passing of Martin Kay, a pioneer in the field of computational linguistics. Back in 2015, at the age of 80, he was still very eager to learn about Tensorflow and brought his laptop to ask me for help installing Anaconda and Python. en.m.wikipedia.org/wiki/Mart…
5
22
3
201
Thang Luong retweeted
Two books, nine years apart 🥳 I'm so happy. I still can't believe that a US publisher wants to publish my book! Grateful to so many people who made this happen. Designing Machine Learning Systems is scheduled for early 2022. First 3 chapters are here oreilly.com/library/view/des…
50
215
16
2,385
Thang Luong retweeted
More experiments with semantic guidance of a disentangled generative adversarial network for a paper in preparation. Here we start from a GAN-inverted picture of Bertrand Russell, and modify it in various ways using text prompts and a #CLIP-based loss.
7
140
56
627
Show this thread
Thang Luong retweeted
To tackle all the challenges we meet while solving intelligence, we need tools that are as adaptable as possible. Announcing the paper & code for Perceiver IO, an architecture that handles a wide range of data and tasks, all while scaling gracefully: dpmd.ai/perceiver-IO 1/4
24
235
25
854
Show this thread
Cool use of Noisy Student self-training together with other techniques such as Mixup for detecting buildings from Satellite Images. Blog: ai.googleblog.com/2021/07/ma…
I'm very excited to see this work from @GoogleAI's Ghana Research Center, on using satellite imagery to detect 516M buildings across 19.4M km^2 (64% of African continent). See the blog post and the new Open Buildings dataset. ai.googleblog.com/2021/07/ma…
Show this thread
0
1
0
18
Thang Luong retweeted
We’re releasing Triton 1.0, an open-source Python-like programming language for writing efficient GPU code. OpenAI researchers with no GPU programming experience have used Triton to produce kernels that are 2x faster than their PyTorch equivalents. openai.com/blog/triton/
128
1,282
171
6,541
Thang Luong retweeted
*Towards Domain-Agnostic Contrastive Learning* #ICML 2021 by @vikasverma1077 @lmthang Kawaguchi @hieupham789 @quocleix Here's an interesting question: can we do self-supervised learning if we know *nothing* about the domain we are operating on? /n
4
55
3
217
Show this thread
Thang Luong retweeted
#AlphaFold is Deep Learning at its finest. Take a (small!) dataset of aminoacid sequences -> 3D structure. Use some of the best ideas from ML, NLP, CV (transformers, GNNs, BERT, distillation), sprinkle loads of innovation for proteins (Evoformer, MSA representation), and voila.
12
171
8
815
Show this thread
Thang Luong retweeted
This is a day I’ve dreamed of my whole life, this is the reason @DeepMind was founded, to build AI and use it accomplish extraordinary scientific breakthroughs like #AlphaFold 2, to advance science and benefit humanity. I could not be more proud of the incredible team!
Today with @emblebi, we're launching the #AlphaFold Protein Structure Database, which offers the most complete and accurate picture of the human proteome, doubling humanity’s accumulated knowledge of high-accuracy human protein structures - for free: dpmd.ai/alphafolddb 1/
Show this thread
141
1,244
132
6,690
Full author list: Vikas Verma · Thang Luong · Kenji Kawaguchi · Hieu Pham · Quoc Le
0
0
0
1
Show this thread