EUR/USD Forecasting with Text Mining & Deep Learning: A PSO-LSTM Approach

Table of Contents

1. Introduction & Overview

This research presents a novel hybrid framework for forecasting the EUR/USD exchange rate, addressing a critical gap in traditional quantitative models by integrating qualitative textual data. The core innovation lies in combining advanced Natural Language Processing (NLP) techniques—specifically sentiment analysis via RoBERTa-Large and topic modeling with Latent Dirichlet Allocation (LDA)—with a deep learning forecasting engine based on Long Short-Term Memory (LSTM) networks. The model's hyperparameters are further optimized using Particle Swarm Optimization (PSO), creating a robust, data-driven forecasting system termed PSO-LSTM.

The study's primary objective is to demonstrate that incorporating real-time, unstructured textual data from news and financial analyses significantly enhances prediction accuracy over models relying solely on historical price data. By doing so, it captures the market sentiment and thematic drivers that often precede currency movements.

2. Methodology & Framework

The proposed methodology follows a structured pipeline from multi-source data aggregation to final prediction.

2.1 Data Collection & Preprocessing

Quantitative Data: Historical daily EUR/USD exchange rates, including open, high, low, close, and volume, were collected. Technical indicators (e.g., moving averages, RSI) were derived as features.

Qualitative Textual Data: A corpus of financial news articles and market analysis reports related to the Eurozone and US economies was scraped from reputable sources. The text was cleaned, tokenized, and prepared for NLP analysis.

2.2 Text Mining & Feature Engineering

Sentiment Analysis: The pre-trained RoBERTa-Large model was fine-tuned on a financial sentiment dataset to classify each news article's sentiment (positive, negative, neutral) and output a continuous sentiment score. This provides a quantitative measure of market mood.

Topic Modeling: Latent Dirichlet Allocation (LDA) was applied to the corpus to identify latent topics (e.g., "ECB Policy," "US Inflation," "Geopolitical Risk"). The distribution of topics per document and key topic keywords became additional features, capturing the thematic context of news.

The final feature vector for each time step $t$ is a concatenation: $\mathbf{X}_t = [\mathbf{P}_t, S_t, \mathbf{T}_t]$, where $\mathbf{P}_t$ is quantitative/technical features, $S_t$ is the sentiment score, and $\mathbf{T}_t$ is the topic distribution vector.

2.3 PSO-LSTM Model Architecture

The forecasting model is an LSTM network, chosen for its ability to model long-term dependencies in sequential data. The LSTM cell's operation at time $t$ can be summarized by:

$\begin{aligned} \mathbf{f}_t &= \sigma(\mathbf{W}_f \cdot [\mathbf{h}_{t-1}, \mathbf{x}_t] + \mathbf{b}_f) \\ \mathbf{i}_t &= \sigma(\mathbf{W}_i \cdot [\mathbf{h}_{t-1}, \mathbf{x}_t] + \mathbf{b}_i) \\ \tilde{\mathbf{C}}_t &= \tanh(\mathbf{W}_C \cdot [\mathbf{h}_{t-1}, \mathbf{x}_t] + \mathbf{b}_C) \\ \mathbf{C}_t &= \mathbf{f}_t * \mathbf{C}_{t-1} + \mathbf{i}_t * \tilde{\mathbf{C}}_t \\ \mathbf{o}_t &= \sigma(\mathbf{W}_o \cdot [\mathbf{h}_{t-1}, \mathbf{x}_t] + \mathbf{b}_o) \\ \mathbf{h}_t &= \mathbf{o}_t * \tanh(\mathbf{C}_t) \end{aligned}$

Where $\mathbf{x}_t$ is the input feature vector $\mathbf{X}_t$, $\mathbf{h}_t$ is the hidden state, $\mathbf{C}_t$ is the cell state, and $\sigma$ is the sigmoid function.

Particle Swarm Optimization (PSO) was employed to optimize critical LSTM hyperparameters (e.g., number of layers, hidden units, learning rate, dropout rate). PSO searches the hyperparameter space by simulating the social behavior of a bird flock, iteratively improving candidate solutions (particles) based on their own and the swarm's best-known positions. This automates and enhances the tuning process compared to manual or grid search.

3. Experimental Results & Analysis

3.1 Benchmark Model Comparison

The PSO-LSTM model was evaluated against several established benchmarks: Support Vector Machine (SVM), Support Vector Regression (SVR), ARIMA, and GARCH. Performance was measured using standard metrics: Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE).

Chart Description (Imagined): A bar chart titled "Forecasting Performance Comparison (RMSE)" would show the PSO-LSTM bar significantly shorter (lower error) than all benchmark models. A line chart overlaying actual vs. predicted EUR/USD rates would show the PSO-LSTM prediction line closely tracking the actual movement, while other models' lines show greater deviation, especially around volatile periods coinciding with major news events.

Key Finding: The PSO-LSTM model consistently outperformed all benchmark models across all error metrics, demonstrating the superior predictive power of the integrated text-quantitative approach.

3.2 Ablation Study Findings

To isolate the contribution of each data component, ablation studies were conducted:

• Model A: LSTM with only quantitative features (baseline).
• Model B: LSTM with quantitative + sentiment features.
• Model C: LSTM with quantitative + topic features.
• Model D (Full): PSO-LSTM with all features (quantitative + sentiment + topics).

Result: Model D (Full) achieved the lowest error. Both Model B and Model C performed better than the baseline Model A, proving that both sentiment and topic information add value. The performance gain from adding topics was slightly greater than from adding sentiment alone in this study, suggesting thematic context is a powerful signal.

4. Technical Deep Dive

4.1 Mathematical Formulation

The core forecasting problem is formulated as predicting the next period's exchange rate return $y_{t+1}$ given a sequence of past feature vectors: $\hat{y}_{t+1} = f(\mathbf{X}_{t-n:t}; \mathbf{\Theta})$, where $f$ is the PSO-LSTM model parameterized by $\mathbf{\Theta}$, and $\mathbf{X}_{t-n:t}$ is the feature window of length $n$.

The PSO algorithm optimizes hyperparameters $\mathbf{\Phi}$ (a subset of $\mathbf{\Theta}$) by minimizing the forecast error on a validation set. Each particle $i$ has a position $\mathbf{\Phi}_i$ and velocity $\mathbf{V}_i$. Their update equations are:

$\begin{aligned} \mathbf{V}_i^{k+1} &= \omega \mathbf{V}_i^k + c_1 r_1 (\mathbf{P}_{best,i} - \mathbf{\Phi}_i^k) + c_2 r_2 (\mathbf{G}_{best} - \mathbf{\Phi}_i^k) \\ \mathbf{\Phi}_i^{k+1} &= \mathbf{\Phi}_i^k + \mathbf{V}_i^{k+1} \end{aligned}$

where $\omega$ is inertia, $c_1, c_2$ are acceleration coefficients, $r_1, r_2$ are random numbers, $\mathbf{P}_{best,i}$ is the particle's best position, and $\mathbf{G}_{best}$ is the swarm's global best position.

4.2 Analysis Framework Example

Scenario: Forecasting EUR/USD movement for the next trading day.

Step 1 - Data Fetch: System ingests closing price, calculates 10-day SMA, RSI (quantitative). Simultaneously, it fetches the 50 latest news headlines from pre-defined financial APIs.

Step 2 - Text Processing:

• Sentiment Pipeline: Headlines are fed into the fine-tuned RoBERTa-Large model. Output: Average daily sentiment score = -0.65 (moderately negative).
• Topic Pipeline: Headlines are processed by the trained LDA model. Output: Dominant topic = "Monetary Policy" (60% weight), with top keywords: "ECB," "lagarde," "interest rates," "hawkish."

Step 3 - Feature Vector Creation: Concatenate: `[Close_Price=1.0850, SMA_10=1.0820, RSI=45, Sentiment_Score=-0.65, Topic_Weight_MonetaryPolicy=0.60, ...]`.

Step 4 - Prediction: The feature vector is fed into the trained PSO-LSTM model. The model, having learned patterns like "negative sentiment + 'hawkish ECB' topic often precedes Euro strengthening," outputs a predicted return.

Step 5 - Output: Model predicts a +0.3% increase in EUR/USD for the next day.

5. Future Applications & Directions

The framework is highly extensible. Future directions include:

• Real-Time Forecasting: Deploying the model in a streaming architecture for intraday predictions using high-frequency news feeds and tick data.
• Multi-Asset & Cross-Currency Pairs: Applying the same methodology to forecast other major FX pairs (e.g., GBP/USD, USD/JPY) or even cryptocurrency rates, which are notoriously sentiment-driven.
• Integration of Alternative Data: Incorporating signals from social media (e.g., Twitter/X sentiment), central bank speech transcripts analyzed with advanced LLMs, or satellite imagery data for economic activity, following trends seen in hedge fund research.
• Advanced Architecture: Replacing the standard LSTM with more sophisticated variants like Transformer-based models (e.g., Temporal Fusion Transformers) or hybrid CNN-LSTM models to capture both spatial patterns in features and temporal dependencies.
• Explainable AI (XAI): Integrating tools like SHAP or LIME to interpret the model's decisions, identifying which specific news topics or sentiment shifts were most influential for a given prediction, crucial for gaining trust in financial applications.

6. References

1. Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation.
2. Kennedy, J., & Eberhart, R. (1995). Particle swarm optimization. Proceedings of ICNN'95.
3. Liu, Y., et al. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint arXiv:1907.11692.
4. Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet allocation. Journal of machine Learning research.
5. Box, G. E. P., Jenkins, G. M., & Reinsel, G. C. (2008). Time Series Analysis: Forecasting and Control. Wiley.
6. Vaswani, A., et al. (2017). Attention is All You Need. Advances in Neural Information Processing Systems.
7. Investopedia. (2023). Foreign Exchange Market (Forex). Retrieved from investopedia.com.
8. European Central Bank & Federal Reserve Economic Data (FRED) – as representative sources for fundamental data.

7. Analyst's Critical Review