Marcus: A Hierarchical Transformer for Reaction Condition Prediction

Predicting the right conditions for a chemical reaction has historically been the domain of process chemists with decades of experience. Should this Suzuki coupling use Pd(PPh₃)₄ or Pd(dba)₂/SPhos? Toluene/H₂O at reflux, or 1,4-dioxane at 90 °C? K₂CO₃, K₃PO₄, or Cs₂CO₃? The right answer depends on substrate, scale, and constraints invisible to a SMILES string.

Coley et al. (2018) opened the field with a simple feedforward network on Morgan fingerprints achieving 41% top-1 solvent accuracy. Subsequent work - Maser's Smiles2Conditions (2021), IBM's RXN4Chem (2023), GraphRXN-Conditions (2024) - pushed top-3 accuracy past 80% but treated each condition (solvent, catalyst, base, T) as an independent multi-class classification task. This factorization is wrong: the choice of base depends on the catalyst, and the temperature depends on the solvent's boiling point.

Marcus models the conditions as a structured joint distribution and decodes them autoregressively in a chemically motivated order - first solvent (reflecting compatibility with reagents), then catalyst (conditional on solvent), then base/additive, then temperature, then yield. Each step attends to the reaction graph and to all previously decoded conditions, capturing the cross-condition dependencies that matter in practice.

2.1 Reaction-graph encoder

Each reactant and product is parsed to its molecular graph and atom-mapped via RXNMapper. The encoder operates on the union graph with edge-type embeddings distinguishing intra-molecule bonds, inter-molecule reactant–product correspondence, and edges marking the reaction core (atoms whose bond environment changes). A 12-layer GraphTransformer with rotational position encodings produces a contextualized atom embedding tensor.

2.2 Hierarchical decoder

The decoder produces conditions in a fixed chemically motivated order:

1. Solvent - selected from a 142-class controlled vocabulary (DMSO, THF, MeCN, …) plus a "mixture" head for binary solvent systems
2. Catalyst / pre-catalyst - 384-class vocabulary including ligand-bound complexes (Pd(PPh₃)₄, Pd(dba)₂/SPhos, …)
3. Base / additive - 89 classes
4. Temperature - regression head with Gaussian likelihood
5. Yield - regression head conditioned on all of the above

Each step receives the reaction encoding plus embeddings of all previously decoded conditions via cross-attention. This factorization captures couplings such as "K₂CO₃ doesn't dissolve well in toluene → prefer Cs₂CO₃ when toluene is the solvent."

2.3 Training data

The corpus combines four sources, with deduplication and aggressive filtering for atom-mapping consistency, reasonable yields (5–95%), and exclusion of one-step deprotection trivialities:

• Reaxys-derived corpus (licensed): 3.1M reactions with full conditions and reported yield
• USPTO-MIT-Conditions (open): 1.0M patent reactions parsed by Lowe + manual review
• Open Reaction Database (ORD): 540k reactions from open-source contributions
• Rasyn in-house optimized batch dataset: 87k high-throughput screening rows from partner labs (Aragen, IFM Therapeutics, 1200 Pharma)

We employ a curriculum: the model is pretrained on the noisier literature corpus (Reaxys + USPTO + ORD) for 80% of training, then fine-tuned on the cleaner in-house optimized-batch data. This curriculum is responsible for the final 0.3pp top-3 lift in our ablation.

2.4 Model variants

We evaluate on the USPTO-MIT-Conditions held-out test set (50,287 reactions, no overlap with training) and additionally on a curated internal medicinal-chemistry test set of 4,210 reactions across 12 named reaction classes. All baselines were re-trained on identical training data and evaluated with identical metrics for a fair comparison.

3.1 Temperature regression

Yield prediction is the hardest task in this benchmark - yield is heavily influenced by execution factors (purity of starting material, mixing, temperature control) that are invisible from a reaction SMILES alone. The R² ceiling for any structure-only model is therefore intrinsically bounded; we estimate it at ~0.80 by training an oracle model on the in-house high-throughput screening data with explicit batch metadata.

Marcus-1B achieves R² = 0.72 / MAE = 8.7%, closing more than half of the remaining gap to the oracle ceiling.

Top-3 solvent accuracy on the validation set after progressively adding components to a plain reactant-SMILES → conditions transformer baseline.

Marcus is the conditions backbone for two production features:

• Closed-vocabulary conditions. The 142-class solvent and 384-class catalyst vocabularies cover ~98% of literature reactions but cannot suggest novel solvents or ligands. Open-vocabulary generation is future work.
• Yield ceiling at R² ≈ 0.80. Yield depends on execution quality (mixing, purity, temperature control) that is structurally invisible. Any structure-only model has an information-theoretic ceiling well below R²=1.
• Long-tail named reactions. Reactions with fewer than ~50 training examples (e.g. exotic transition-metal cross-couplings) show 10–15pp lower top-3 accuracy. Targeted active-learning loops with partner labs are addressing this.
• Patent-bias. Reaxys and USPTO over-represent successful published reactions. Marcus may underweight unconventional but viable conditions that are simply absent from the literature.

Marcus models reaction conditions as a structured joint distribution rather than a bag of independent classifications, and decodes them in a chemically motivated order with cross-condition co-attention. Trained on 4.7M reactions and benefiting from a literature → optimized-batch curriculum, Marcus-1B sets new SOTA across solvent, catalyst, temperature, and yield prediction simultaneously. Inside Rasyn it is the difference between a retrosynthesis route and an executable lab protocol.