excerpt: improving recommendation systems and search in the age of LLMs

Original article: Improving Recommendation Systems & Search in the Age of LLMs

This is a survey of generative recommendation work. Its context is 2023, when industrial teams had only begun exploring generative recommendation.

Recommendation and search have long been influenced by language modeling: Word2vec for embedding-based retrieval, and GRU, Transformer, and BERT for next-item prediction and ranking. The LLM wave is not fundamentally different; it is simply larger in scale and impact. The article reviews industrial progress in model architecture, data generation, training paradigms, and unified frameworks.

LLM- and multimodal-enhanced architectures

Many recommendation models now combine language models with multimodal content to overcome the limits of conventional ID-based approaches. Combining content understanding with behavioral modeling improves cold start and long-tail recommendation.

Semantic IDs (YouTube)

Semantic IDs replace traditional hashed IDs with content-derived features, making preferences for new and long-tail items easier to predict. A VideoBERT-like Transformer first extracts a 2,048-dimensional content embedding; RQ-VAE maps it to a 256-dimensional latent representation, residual-quantizes it, and converts it into discrete semantic IDs. The RQ-VAE is then frozen and used to transform video content before training the large production ranking model.

The paper integrates semantic IDs using either fixed-length N-grams or SentencePiece Model (SPM), which learns variable-length subword-like segments. Directly feeding dense content vectors to the ranking model performed worse than random hashed IDs, likely because large ID embedding tables can memorize more detail. Both semantic-ID variants beat random hashing, especially for cold start; N-grams had a small advantage with small embedding tables, while SPM generalized and scaled better with larger tables.

M3CSR (Kuaishou)

M3CSR uses multimodal visual, textual, and audio embeddings, clusters them with k-means into trainable category IDs, and dynamically adjusts otherwise static content features through user behavior. Its two-tower structure supports precomputation and approximate-nearest-neighbor retrieval. Visual, text, and audio vectors from ResNet, Sentence-BERT, and VGGish are concatenated and clustered into roughly 1,000 groups; a Modal Encoder maps content to a behavior-aligned space.

On the user side, it models both behavior sequences and preferences across modalities. It outperformed multimodal baselines such as VBPR, MMGCN, and LATTICE. Modeling modality-specific preference was important, and multimodal features beat individual modalities across Amazon, TikTok, and Allrecipes. Online A/B tests increased clicks by 3.4%, likes by 3.0%, and follows by 3.1%; cold-start speed and cold-start-video coverage improved by 1.2% and 3.6%.

FLIP, beeFormer, and CALRec

FLIP aligns ID-based models and LLMs through cross-reconstruction between masked tabular IDs and text. It converts user and item IDs to prompted text, jointly pretrains with masked language modeling and masked tabular modeling, adds instance-level contrastive learning, then adaptively fine-tunes TinyBERT and DCNv2. It outperformed ID-only, LLM-only, and naive ID+LLM concatenation; both masking objectives and field-level masking contributed.

beeFormer adds text to user-item interactions and trains a text-only Transformer to capture both semantic and interaction similarity. A sentence Transformer creates item vectors while ELSA models interactions through a low-rank approximation. Recommendation loss backpropagates into the Transformer. It beat several untuned embedding baselines offline, and cross-domain training outperformed single-domain training.

CALRec fine-tunes PaLM-2 XXS for sequential recommendation by converting item attributes to structured text and prompting the model to generate the next item description. It first trains across categories, then in a target category, using generation and contrastive losses. At inference it samples candidates and uses BM25 and scores to deduplicate and select top-k results.

Data generation and analysis with LLMs

LLMs can create missing metadata, synthetic training queries, and scalable evaluation labels. Amazon’s Playlist Search uses an LLM curator to generate playlist descriptions from the first 15 songs, fine-tunes Flan-T5-XL to scale the process, uses an LLM labeler to generate query-playlist examples for a two-tower retriever, and uses an LLM judge for evaluation. The added metadata substantially improved recall while reducing manual work.

Scaling laws, transfer learning, distillation, and LoRA

Scaling Laws found predictable power-law improvement from larger sequence-recommendation models and datasets. Decoder-only Transformers from $9.83\times10^4$ to $0.8\times10^9$ parameters improved steadily on MovieLens-20M and Amazon-2018. Hierarchical adaptive dropout and switching from Adam to SGD stabilized large models. Larger models converged faster, retained gains across epochs, and helped long-tail items and cold-start users.

PrepRec explores zero-shot recommendation through multi-scale item popularity, behavior intervals, and sequence position, without individual item ID embeddings. It needs fewer parameters and transfers across domains. E-CDCTR uses abundant organic food-delivery and local-services interactions to pretrain long-term and short-term embeddings before daily advertising CTR fine-tuning, outperforming several cross-domain baselines.

YouTube’s Bridging the Gap uses auxiliary distillation: hard labels and teacher soft labels remain separate at the prediction layer so a smaller student can learn from a larger multi-objective teacher without inheriting its noise. Google Self-Auxiliary Distillation similarly combines hard labels on high-quality samples with calibrated soft labels for weaker samples. DLLM2Rec distills recommendation knowledge from a Llama2-7B-based teacher into conventional sequence models through importance-weighted ranking and collaborative embedding distillation, improving ranking quality while dramatically reducing inference time.

MLoRA pretrains a common backbone across CTR domains, then applies per-domain low-rank adapters. It improved AUC by about 0.5% on several datasets; online tests reported 1.49% CTR, 3.37% conversion-rate, and 2.71% paid-buyer gains with only 1.76% additional model complexity.

Pinterest’s Taming One-Epoch pretrains ID embeddings with a simple dot-product contrastive objective before downstream fine-tuning, addressing the tendency of deep CTR models with ID embeddings to overfit after one epoch. Netflix Sliding Window Training changes the sampled history window each epoch so a model sees long-term behavior without processing a full long sequence at once; combining it with recent interactions gives the best overall trade-off.

Unified architectures for search and recommendation

Another trend is unifying search and recommendation in one retrieval and ranking framework instead of maintaining a separate model for every scenario.

Bridging Search & Recommendations uses a single generative retrieval model for both tasks. It expands Flan-T5-base’s vocabulary with item IDs, autoregressively predicts item-ID sequences from either user history or query text, and finds multi-task training improves average recall@30 by 16%—33% on podcasts—over single-task versions. It still trails conventional SASRec, BERT4Rec, BM25, and bi-encoder baselines, so it is not yet a complete replacement.

360Brew combines several LinkedIn ID-based ranking models into a 150B model built on Mixtral-8x22B, with continued pretraining, instruction tuning, and supervised fine-tuning. It uses 3–6 months of interaction data from roughly 45 million monthly active users, supports contexts up to 16K, and can match or outperform specialized models across more than 30 scenarios, especially for cold-start users.

UniCoRn is Netflix’s single ranker for search and recommendation. It takes query, source page, user ID, query text, item ID, and task-specific context, filling absent features with null. Joint training achieved roughly 10% recommendation and 7% search improvements while reducing infrastructure overhead.

Unified Embeddings combines Transformer, term, and graph embeddings in a two-tower Etsy model. It fuses product text, graph, title, and attribute vectors with query text, location, and behavior features; uses several negative-sampling strategies; includes quality scores in ANN vectors; and retrieves through a compressed FAISS index under 20 ms p99. It increased overall conversion by 2.63% and organic-search purchases by 5.58%.

Best Buy’s Embedding Long Tail uses filtered interaction data, Llama-13B-generated synthetic queries, and a custom two-tower RoBERTa variant to improve semantic retrieval on long-tail queries, delivering a 3% online conversion lift. YouTube’s User Behavioral Service splits a large sequential user model from the main ranker, asynchronously caching user embeddings to avoid real-time latency; it improved engagement while adding far less cost than integrating the user model directly. Zalando’s Modern Ranking Platform combines lightweight candidate generation, multi-task ranking, shared feature and vector infrastructure, and a controllable policy layer; four A/B tests delivered 15% more interaction and 2.2% more revenue.

Early 2023 attempts to apply LLMs to recommendation and search often underperformed expectations. The practices summarized here show that teams are increasingly finding effective ways to combine LLMs with multimodal and traditional ID-based methods, and to use LLM ideas for data generation and model training. These approaches have produced real industrial gains while reducing manual feature-engineering and labeling effort. LLMs clearly have substantial room to grow in recommendation and search.

September 14, 2025, Suzhou