Contributed Talk and Poster Awards

Abstract:

As generative and foundation models grow more capable, so does the risk that they retain harmful, private, copyrighted, or simply unwanted knowledge. Retraining a model from scratch to remove such content is prohibitively expensive, which has made machine unlearning an increasingly important direction in AI safety. Machine unlearning is a task of erasing specific knowledge from an already-trained model. Yet most existing unlearning methods are blunt instruments: they degrade unrelated capabilities, require large “retain” datasets to preserve performance, or touch far more of the model’s parameters than necessary.

In this talk, I’ll present a more surgical approach to unlearning: one that identifies and modifies only the small subset of a model’s weights actually responsible for a target concept, leaving the rest of the model intact. I’ll walk through how techniques such as singular value decomposition (done in SEMU and BARRIER) can be used to disentangle and localize concept-relevant parameters, why this precision matters for both safety and model utility, and how such methods perform compared to conventional fine-tuning-based unlearning. I’ll close with open challenges in the field: how to verify that a concept has truly been forgotten, how to scale precise unlearning to ever-larger models, and what “forgetting” should even mean for a neural network.

The talk is aimed at researchers and practitioners interested in AI safety, interpretability, and the practical mechanics of controlling what large models know.

Abstract:

Large Language Models (LLMs) are trained to support an increasing number of languages, yet their predefined tokenizers remain a bottleneck for adapting models to lower-resource or distinct-script languages. Existing tokenizer transfer methods typically rely on semantic heuristics to initialize new embeddings, ignoring higher-layer model dynamics and limiting transfer quality. We propose Model-Aware Tokenizer Transfer (MATT), a method that incorporates model internals into the tokenizer transfer process. MATT introduces an Attention Influence Modeling (AIM) objective that distills inter-token communication patterns from a source model into a target model with a new tokenizer, providing an efficient warm-up before standard language modeling. Unlike approaches that focus solely on embedding similarity, MATT leverages attention behavior to guide embedding initialization and adaptation. Experiments across diverse linguistic settings show that MATT recovers a large fraction of the original model’s performance within a few GPU hours, outperforming heuristic baselines. These results demonstrate that incorporating model-level signals offers a practical and effective path toward robust tokenizer transfer in multilingual LLMs.

Abstract:

The growing interest in AI-assisted content creation has motivated game publishers and designers to investigate ML methods that can support game design. Among tabletop games, role playing games (RPGs) remain a particularly popular segment, however, the application of ML to tabletop RPG design is relatively underexplored. One of the most important and time-consuming design tasks is monster creation, which requires estimating encounter difficulty and efficiently adjusting monster characteristics whenever the resulting difficulty does not match the intended gameplay experience or story.

To address this challenge, we developed an ordinal regression model for monster difficulty estimation. Based on this model we are investigating methods for scaling monsters up and down in difficulty and for generating balanced monsters. Our current research focuses on counterfactual explanations (CFs), a branch of explainable AI that rather than answering why a model produced a given prediction, explain what changes to the input would be required to obtain a desired outcome. In the context of monster design, CFs can be used to generate alternative versions of a monster while preserving its core identity and modifying only the characteristics necessary to achieve a target difficulty level.

An additional advantage of CFs is their ability to provide multiple valid alternatives. This naturally supports a human-in-the-loop workflow, allowing designers to select the variant that best matches their creative vision. Such a framework can serve both as a level difficulty adjustment tool and as a foundation for future monster-generation systems.

This domain also provides a unique opportunity to evaluate CFs methods in a realistic setting. While many studies rely on standard tabular ML benchmarks, this dataset contains naturally occurring counterfactual-like examples. Specifically, several monster families include variants representing different difficulty levels of the same underlying creature (e.g., Young, Adult, and Ancient dragons), providing realistic reference points for evaluating generated counterfactuals.

In this work, we present the results of applying counterfactual explanation methods to tabletop RPG monster design and discuss their potential use by professional game designers as well as players interested in creating and balancing their own monsters.

Abstract:

As the volume of time-series data continues to grow, the efficiency of storage is becoming an important consideration in applications that require retaining large collections of signals or long historical records. Simultaneously, time series foundation models (TSFMs) that currently dominate forecasting benchmarks rely on relatively simple tokenization techniques, leaving the efficiency of representation and storage of time series data comparatively underexplored. Motivated by this gap, we introduce an approach that combines a novel method of tokenization via residual hierarchical decomposition and quantization with a modified Transformer decoder, enabling causal processing of signals. Each block of the tokenizer efficiently represents a different frequency range of the temporal signal, as its residual propagates through them sequentially. The decoder-only backbone employs Multi-dimensional Rotational Position Embeddings Interleave (MRoPE-I) and Mixture of Experts (MoE) for effective processing of varying-scale tokens, ending with a quantile head suitable for probabilistic forecasting. The proposed architecture supports variable-length contexts through its length-agnostic tokenizer, while autoregressive decoding allows the prediction horizon to be extended. Our method matches the performance of the state-of-the-art in terms of nMSE across multiple datasets, yet achieves 7.59x compression rate of the original data with around 0.06 MSE reconstruction error after normalization. Our results suggest that through efficient modeling of temporal sequences, we can substantially reduce both the storage and computational complexity, enabling more data storage without negatively impacting downstream tasks, while extending the context and prediction horizons.