Deep Learning for Solar Power Forecasting: A Transformer-Based In-Context Learning Approach

As the role of photovoltaic generation increases in the energy supply chain, forecast errors have larger operational and economic implications. Accordingly, accurate short-term PV forecasts acquire growing importance for the efficient management of power systems and for decision-making in electricity markets. In parallel, deep learning and transformers-based architectures have shown promising performance in forecasting tasks. Large Language Models (LLMs), that rely on a transformer architecture, have also shown benefits from providing as input example textual representations of input-output pairs, this practice is known as in-context learning. This thesis investigates whether in-context learning, performed with transformer encoders, can be used as auxiliary component in day-ahead PV power forecasting when combined with a numerical Long Short Term Memory-based model (LSTM). The empirical analysis uses inverter-level AC power from six ground-mounted PV plants in Italy, together with periodical meteorological inputs and the irradiance signal. The task is framed as a day-ahead forecast problem, leveraging a 168-hour multivariate context, with separate evaluation over full days and daylight hours. We chose Longformer, as encoder, to represent the structured prompt containing past k context exemplars with k ∈ {1, 2, 3}, whose representation is fused with the Long Short Term Memory (LSTM) output. On a held-out test set, text conditioning yields small but systematic improvements over the purely numerical baseline: configurations with one or two exemplars modestly reduce absolute errors, while k = 3 leads to noisy representations that affect the model performance. Overall, few-shot, text-based conditioning emerges as a complementary information channel for PV forecasting, providing modest benefits and suggesting scope for future work on probabilistic forecasts and richer prompt designs.