Search Results for « Temporal Fusion Transformer »

Multi-horizon forecasting often contains a complex mix of inputs – including static (i.e. time-invariant) covariates, known future inputs, and other exogenous time series that are only observed in the past – without any prior information on how they interact with the target. Several deep learning methods have been proposed, but they are typically ‘black-box’ models that do not shed light on how they use the full range of inputs present in practical scenarios. In this paper, we introduce the Temporal Fusion Transformer (TFT) – a novel attention-based architecture that combines high-performance multi-horizon forecasting with interpretable insights into temporal dynamics. To learn temporal relationships at different scales, TFT uses recurrent layers for local processing and interpretable self-attention layers for long-term dependencies. TFT utilizes specialized components to select relevant features and a series of gating layers to suppress unnecessary components, enabling high performance in a wide range of scenarios. On a variety of real-world datasets, we demonstrate significant performance improvements over existing benchmarks, and highlight three practical interpretability use cases of TFT.

The energy generated by a solar photovoltaic (PV) system depends on uncontrollable factors, including weather conditions and solar irradiation, which leads to uncertainty in the power output. Forecast PV power generation is vital to improve grid stability and balance the energy supply and demand. This study aims to predict hourly day-ahead PV power generation by applying Temporal Fusion Transformer (TFT), a new attention-based architecture that incorporates an interpretable explanation of temporal dynamics and high-performance forecasting over multiple horizons. The proposed forecasting model has been trained and tested using data from six different facilities located in Germany and Australia. The results have been compared with other algorithms like Auto Regressive Integrated Moving Average (ARIMA), Long Short-Term Memory (LSTM), Multi-Layer Perceptron (MLP), and Extreme Gradient Boosting (XGBoost), using statistical error indicators. The use of TFT has been shown to be more accurate than the rest of the algorithms to forecast PV generation in the aforementioned facilities.

Abstract Aquaponics offers a soilless farming ecosystem by merging modern hydroponics with aquaculture. The fish food is provided to the aquaculture, and the ammonia generated by the fish is converted to nitrate using specialized bacteria, which is an essential resource for vegetation. Fluctuations in the ammonia levels affect the generated nitrate levels and influence farm yields. The sensor-based autonomous control of aquaponics can offer a highly rewarding solution, which can enable much more efficient ecosystems. Also, manual control of the whole aquaponics operation is prone to human error. Artificial Intelligence-powered Internet of Things solutions can reduce human intervention to a certain extent, realizing more scalable environments to handle the food production problem. In this research, an attention-based Temporal Fusion Transformers deep learning model was proposed and validated to forecast nitrate levels in an aquaponics environment. An aquaponics dataset with temporal features and a high number of input lines has been employed for validation and extensive analysis. Experimental results demonstrate significant improvements of the proposed model over baseline models in terms of MAE, MSE, and Explained Variance metrics considering one-hour sequences. Utilizing the proposed solution can help enhance the automation of aquaponics environments.

Electricity load forecasting plays an important role in the operation of power systems. Inaccurate forecast would reduce the safety of power supply and affect the economic and social activities as well as national defense and security. In addition, the forecast results also support decision-making on electricity generation and market transactions. Traditional methods such as AR, ARIMA, SARIMA have been widely used to forecast short term electricity load. Recently, load forecasting based on artificial and deep neural networks have shown significant accuracy improvement over traditional statistical models. In this research, a novel recurrent neural network named temporal fusion transformer (TFT) is used to forecast short-term electricity load of Hanoi city. The TFT is a newly developed model and it combines the advantages of several other RNN models such as LSTM and the self-attention mechanism. In addition to historical load data, we use temperature and humidity features, and time features such as calendar month, lunar month, days of the week, hours of the day and holidays. The forecast results of TFT are compared with traditional statistical models as well as well-known RNN models. The compared results show that the proposed method is better than other methods in both MAE and MAPE criteria.

Global climate change has intensified the search for renewable energy sources. Solar power is a cost‐effective option for electricity generation. Accurate energy forecasting is crucial for efficient planning. While various techniques have been introduced for energy forecasting, transformer‐based models are effective for capturing long‐range dependencies in data. This study proposes N hours‐ahead solar irradiance forecasting framework based on variational mode decomposition (VMD) for handling meteorological data and a modified temporal fusion transformer (TFT) for forecasting solar irradiance. The proposed model decomposes raw solar irradiance sequences into intrinsic mode functions (IMFs) using VMD and optimizes the TFT using a variable screening network and a gated recurrent unit (GRU)‐based encoder–decoder. Our study specifically targets the 1‐h as well as different forecasting horizons for solar irradiance. The resulting deep learning model offers insights, including the prioritization of solar irradiance subsequences and an analysis of various forecasting window sizes. An empirical study shows that our proposed method has achieved high performance compared to other time series models, such as artificial neural network (ANN), long short‐term memory (LSTM), CNN–LSTM, CNN–LSTM with temporal attention (CNN–LSTM‐t), transformer, and the original TFT model.

Since flight delay hurts passengers, airlines, and airports, its prediction becomes crucial for the decision-making of all stakeholders in the aviation industry and thus has been attempted by various previous research. However, previous delay predictions are often categorical and at a highly aggregated level. To improve that, this study proposes to apply the novel Temporal Fusion Transformer model and predict numerical airport arrival delays at quarter hour level for U.S. top 30 airports. Inputs to our model include airport demand and capacity forecasts, historic airport operation efficiency information, airport wind and visibility conditions, as well as en-route weather and traffic conditions. The results show that our model achieves satisfactory performance measured by small prediction errors on the test set. In addition, the interpretability analysis of the model outputs identifies the important input factors for delay prediction.

Temporal Fusion Transformers (TFT) is a Transformer model for multi-step forecasting tasks. Because TFT models can integrate decoders to import various types of inputs, including static covariates, known future inputs, and other exogenous time series observed only in the past, which are well perform

Abstract Gas turbines play a vital role in various industries. Timely and accurately predicting their degradation is essential for efficient operation and optimal maintenance planning. Diagnostic and prognostic outcomes aid in determining the optimal compressor washing intervals. Diagnostics detects compressor fouling and estimates the trend up to the current time. If the forecast indicates fast progress in the fouling trend, scheduling offline washing during the next inspection event or earlier may be crucial to address the fouling deposit comprehensively. This approach ensures that compressor cleaning is performed based on its actual health status, leading to improved operation and maintenance costs. This paper presents a novel prognostic method for gas turbine degradation forecasting through a time-series analysis. The proposed approach uses the Temporal Fusion Transformer model capable of capturing time-series relationships at different scales. It combines encoder and decoder layers to capture temporal dependencies and temporal-attention layers to capture long-range dependencies across the encoded degradation trends. Temporal attention is a self-attention mechanism that enables the model to consider the importance of each time step degradation in the context of the entire degradation profile of the given health parameter. Performance data from multiple two-spool turbofan engines is employed to train and test the method. The test results show promising forecasting ability of the proposed method multiple flight cycles into the future. By leveraging the insights provided by the method, maintenance events and activities can be scheduled in a proactive manner. Future work is to extend the method to estimate remaining useful life.

Aero-structural shape design and optimization of bridge decks rely on accurately estimating their self-excited aeroelastic forces within the design domain. The inherent nonlinear features of bluff body aerodynamics and the high cost of wind tunnel tests and computational fluid dynamics (CFD) simulations make their emulation as a function of deck shape and reduced velocity challenging. State-of-the-art methods address deck shape tailoring by interpolating discrete values of integrated flutter derivatives (FDs) in the frequency domain. Nevertheless, more sophisticated strategies can improve surrogate accuracy and potentially reduce the required number of samples. We propose a time domain emulation strategy harnessing temporal fusion transformers (TFTs) to predict the self-excited forces time series before their integration into FDs. Emulating aeroelastic forces in the time domain permits the inclusion of time-series amplitudes, frequencies, phases, and other properties in the training process, enabling a more solid learning strategy that is independent of the self-excited forces modeling order and the inherent loss of information during the identification of FDs. TFTs' long- and short-term context awareness, combined with their interpretability and enhanced ability to deal with static and time-dependent covariates, make them an ideal choice for predicting unseen aeroelastic forces time series. The proposed TFT-based metamodel offers a powerful technique for drastically improving the accuracy and versatility of wind-resistant design optimization frameworks.