Few-Shot Learning: Why Your AI Needs to Think Faster

- Home
- Case studies
- Few-Shot Learning: Why Your AI Needs to Think Faster
Problem
Most AI models need large amounts of labelled data for every task they learn. When something new comes along, such as a new product line, a new customer segment, a new category of request or a new user, the usual answer is to collect more data and retrain. That is slow and expensive, and in many situations the data simply does not exist yet.
People are different. Shown a handful of examples, we can usually pick up a new concept and apply it straight away. For businesses that need AI to keep up with change, closing that gap matters: a model that can adapt from a few examples can personalise faster, handle new cases sooner and cost less to keep current.
Also applicable to
- Personalisation: adapting recommendations or interfaces to a new user from very little history.
- Customer support: recognising a new type of request after seeing only a few examples.
- Retail and e-commerce: classifying new products or categories without a large labelled set.
- Document processing: handling new document types or formats quickly.
- Language applications: adapting language models to new domains with limited data.
Solution / Approach
The Elementera team's research on fast adaptation appears in Rapid Adaptation with Conditionally Shifted Neurons, a paper co-authored by Elementera co-founder Soroush Mehri with Tsendsuren Munkhdalai, Xingdi Yuan and Adam Trischler, published at ICML 2018.
The paper works in the framework of meta-learning, or learning to learn, where the aim is to replicate some of the flexibility of human learning in machines. It introduces a mechanism called conditionally shifted neurons:
- Each neuron can shift its activation value by a task-specific amount.
- Those shifts are retrieved from a memory module.
- The memory is populated rapidly from the limited experience available for the current task.
In plain terms, the network does not have to slowly retrain its weights to handle a new task. It looks at the few examples it has been given, stores what is relevant in memory, and then adjusts how its neurons respond, on the fly. The underlying network stays the same; its behaviour adapts to the task in front of it.
Why this matters for a company using AI: adaptation that happens at run time, from a few examples, is what makes AI practical for long-tail and fast-changing problems, where waiting for a big new dataset is not an option.
Impact
According to the paper, models augmented with conditionally shifted neurons achieved state-of-the-art results on meta-learning benchmarks from both the vision and language domains. The published version also adds ablations and a speed comparison with an earlier meta-learning method, MetaNet.
For businesses, this research means:
- Less data per new task: models that can learn from a handful of examples.
- Faster response to change: adaptation happens on the fly instead of through a full retraining cycle.
- Works across data types: the mechanism was shown on both images and language.
- Proven expertise in learning to learn: the Elementera team has published on the methods that make AI more adaptable, and applies that understanding when designing systems that must keep up with new situations.
Technologies
- Meta-learning (learning to learn): training models to adapt quickly to new tasks.
- Memory-augmented neural networks: storing task-specific information for fast retrieval.
- Conditional activation shifts: adjusting neuron behaviour per task without retraining weights.
- Few-shot learning: generalising from very small numbers of examples.
Read the paper: Rapid Adaptation with Conditionally Shifted Neurons (ICML 2018).

Ex-Microsoft. 10+ years of experience in AI/ML, MSc from Université de Montréal (Mila). US patent holder.

AI/ML scientist with 20+ years of research & industry experience. Ex-Google DeepMind, MIT, Harvard, Mila. McGill-Mila PhD.
